Control-Group Feature Normalization for Multivariate Pattern Analysis of Structural MRI Data using the Support Vector Machine

Abstract

Normalization of feature vector values is a common practice in machine learning. Generally, each feature value is standardized to the unit hypercube or by normalizing to zero mean and unit variance. Classification decisions based on support vector machines (SVMs) or by other methods are sensitive to the specific normalization used on the features. In the context of multivariate pattern analysis using neuroimaging data, standardization effectively up- and down-weights features based on their individual variability. Since the standard approach uses the entire data set to guide the normalization, it utilizes the total variability of these features. This total variation is inevitably dependent on the amount of marginal separation between groups. Thus, such a normalization may attenuate the separability of the data in high dimensional space. In this work we propose an alternate approach that uses an estimate of the control-group standard deviation to normalize features before training. We study our proposed approach in the context of group classification using structural MRI data. We show that control-based normalization leads to better reproducibility of estimated multivariate disease patterns and improves the classifier performance in many cases.

1. Introduction

Machine learning classification algorithms such as the support vector machine (SVM) [1, 2] are often used to map high-dimensional neuroimaging data to a clinical diagnosis or decision. Structural and functional magnetic resonance imaging (MRI) are promising tools for building biomarkers to diagnose, monitor, and treat neurological and psychological illnesses. Mass-univariate methods such as statistical parametric mapping [3, 4, 5] and voxel- based morphometry [6, 7] test for marginal disease effects at each voxel, ignoring complex spatial correlations and multivariate relationships among voxels. As a result, methods have emerged for performing multivariate pattern analysis (MVPA) that leverage the information contained in the covariance structure of the images to discriminate between the groups being studied [8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]. Identifying multivariate structural and functional signatures in the brain that discriminate between groups may lead to a better understanding of disease processes and is therefore of great interest in the field of neuroimaging research.

The SVM is a common choice for estimating multivariate patterns in the brain because it is amenable to high-dimensional, low sample size data. Our focus in this work is on patterns in the brain that reflect structural changes due to disease. However, the methods apply more generally to applications of MVPA using BOLD measurements from fMRI data or measures of connectivity across the brain. The SVM takes as input image-label pairs and returns a decision function that is a weighted sum of the imaging features. The estimated weights reflect the joint contribution of the imaging features to the predicted class label.

Machine learning methods in general, and SVMs in particular, are sensitive to differences in feature scales. For example, a SVM will place more importance on a feature that takes values in the range of [1000, 2000] than a feature that takes values in the interval [1, 2]. This is because the former tends to have a stronger influence on the Euclidean distance between feature vector realizations and therefore drives the SVM optimization. To give all voxels or regions of interest equal importance during classifier training, it is common practice to implement feature-wise standardization in some way, either by normalizing each to have mean zero and unit variance or by scaling to a common domain. For example, [30] scale each feature to be in the interval [0, 1], and [31, 32, 33, 34, 35] normalize to mean zero and unit variance. Such a preprocessing step, while common in practice, tends to be applied without weighing the consequent ramifications in a careful manner. Careful consideration must be given to the choice of feature normalization, as it is directly tied to the relative magnitude of the estimated SVM weights and thus the performance and interpretation of the classifier. While the original idea of feature scaling dates back to the universal approximation theorem from the neural network literature, it has not been explored in detail in the context of neuroimaging and MVPA. This is the object of this manuscript.

The rest of this paper is organized as follows: in Section 2, we provide a brief introduction to MVPA using the SVM, review two popular feature normalization methods, and propose an alternative based on the control-group variability. Using simulations, we compare the performance of different feature normalization techniques in Section 3, followed by an investigation of the effects of feature normalization on an analysis of data from healthy controls and patients with Alzheimer’s disease. We include a discussion in Section 4 and concluding remarks in Section 5.

2. Material and Methods

2.1. Multivariate Pattern Analysis using the SVM

Let ${\left(Y_i,{\mathit{X}}_i^{\top }\right)}^{\top }$, i = 1, …, n, denote n independent and identically distributed observations of the random vector (Y, X^⊤)^⊤, where Y ∈ {−1, 1} denotes the group label, and X ∈ ℝ^p denotes a vectorized image with p voxels. A popular MVPA tool used in the neuroimaging community is the SVM [1, 2]. SVMs are known to work well for high dimension, low sample size data [36]. Such data are common in the neuroimaging-based diagnostic setting. Henceforth, we focus on MVPA using the SVM.

The hard-margin linear SVM solves the constrained optimization problem

$$\begin{array}{l}\arg \underset{\mathit{\upsilon },b}{\min }\frac{1}{2}{\Vert \mathit{\upsilon }\Vert }^2\\ \text{such}\text{that}{Y}_i\left({\mathit{\upsilon }}^{\top }{\mathit{X}}_i+b\right)\ge 1\forall i=1,\dots ,n,\end{array}$$ (1)

where b ∈ ℝ and v ∈ ℝ^p are parameters that describe the classification function. For a given set of training data, let the solution to (1) be denoted by $(\stackrel{\sim }{\mathit{\upsilon }},\stackrel{\sim }{b})$. Then, for a new observation X^new with unknown label Y^new, the classification function $c\left({\mathit{X}}^{\text{new}}\right)=\text{sign}\left({\stackrel{\sim }{\mathit{\upsilon }}}^{\top }{\mathit{X}}^{\text{new}}+\stackrel{\sim }{b}\right)$ returns a predicted group label.

When the data from the two groups are not linearly separable, the soft-margin linear SVM allows some training observations to be either misclassified or fall in the SVM margin through the use of slack variables ξ_i with associated cost parameter C. In this case, the optimization problem becomes

$$\begin{array}{l}\arg \underset{\mathit{\upsilon },b,\mathit{\xi }}{\min }\frac{1}{2}{\Vert \mathit{\upsilon }\Vert }^2+C{\displaystyle \sum _{i=1}^n{\xi }_i}\\ \text{such}\text{that}:\\ Y_i\left({\mathit{\upsilon }}^{\top }{\mathit{X}}_i+b\right)\ge 1-{\xi }_i\forall i=1,\dots ,n,\\ {\xi }_i\ge 0\forall i=1,\dots ,n,\end{array}$$ (2)

where C ∈ ℝ is a tuning parameter that penalizes misclassification, and ξ = (ξ₁, ξ₂, …, ξ_n)^⊤ is the vector of slack variables. For details about solving optimization problems (1) and (2) we refer the reader to [37].

In high-dimensional problems where the number of features is greater than the number of observations, the data are almost always separable by a linear hyperplane [38]. However, when applying MVPA to region of interest (ROI) data such as volumes of subregions in the brain, the data may not be linearly separable. In this case, the choice of C is critical to classifier performance and generalizability. Examples of MVPA using the SVM include classification of multiple sclerosis patients into disease subgroups [39], the study of Alzheimer’s disease [9, 10], and various classification tasks involving patients with depression [40, 41, 42]. This is only a small subset of the relavant literature, which demonstrates the widespread popularity of the approach.

2.2. SVM Feature Normalization for MVPA

The choice of feature normalization affects the estimated weight pattern of a SVM and can lead to vastly different conclusions about the underlying disease process. Two widely implemented approaches are to (i) normalize each feature to have mean zero and unit variance, and (ii) scale each feature to have a common domain such as [0, 1]. Henceforth, we will refer to (i) as standard normalization and (ii) as domain standardization [43].

Let μ_j and σ_j denote the mean and standard deviation of the j^th feature, j = 1, …, p. Denote the corresponding empirical estimates by ${\overline{X}}_j=n^{-1}{\displaystyle {\sum }_{i=1}^n{X}_{i,j}}$ and ${\widehat{\sigma }}_j={\left\{{(n-1)}^{-1}{\displaystyle {\sum }_{i=1}^n{\left(X_{i,j}-{\overline{X}}_j\right)}^2}\right\}}^{1/2}$. Then, subject i’s standard-normalized j^th feature is calculated as

$$X_{i,j}^Z=\frac{X_{i,j}-{\overline{X}}_j}{{\widehat{\sigma }}_j}.$$

Alternatively, subject i’s domain-scaled j^th feature is calculated as

$$X_{i,j}^U=\frac{X_{i,j}-{\min }_i{X}_{i,j}}{{\max }_i{X}_{i,j}-{\min }_i{X}_{i,j}}.$$

One potential drawback of using domain scaling is the instability of the minimum and maximum order statistics, especially in small sample sizes. This may introduce bias in the estimated weight pattern by up- and down-weighting features in an unstable way. In comparison, the standard normalization may seem relatively stable. However, it implicitly depends on the relative sample size of each group and the separability between groups. To see this, let $f_{X_j}$ denote the marginal distribution of X_j, with mean μ_j and variance ${\sigma }_j^2$. Let $f_{X_j|Y=y}$ denote the conditional distribution of X_j given Y = y with mean μ_j,y and variance ${\sigma }_{j,y}^2$. In addition, let γ = pr(Y = 1). Then, μ_j = γμ_j,1 + (1 − γ)μ_j,−1 and

$$\begin{array}{l}{\mathrm{\sigma }}_j^2=E{\left(X_j-{\mu }_j\right)}^2\\ =E{X}_j^2-{\mu }_j^2\\ ={\displaystyle \int x_j^2\{\gamma f_{X_j|Y=1}(x)+(1-\gamma )f_{X_j|Y=-1}(x)\}dx-{\mu }_j^2}\\ =\gamma \left({\sigma }_{j,1}^2+{\mu }_{j,1}^2\right)+\left(1-\gamma \right)\left({\sigma }_{j,-1}^2+{\mu }_{j,-1}^2\right)-{\mu }_j^2.\end{array}$$

After simplification, the previous expression can be written as

$${\sigma }_j^2=\gamma {\sigma }_{j,1}^2+(1-\gamma ){\sigma }_{j,-1}^2+\gamma \left(1-\gamma \right){\left({\mu }_{j,1}-{\mu }_{j,-1}\right)}^2.$$ (3)

The right-hand side of expression (3) shows that the variance of feature j depends on a mixture of the conditional variances of both classes and a term that depends on the squared Euclidean distance between their marginal means. Larger marginal separability of feature j will lead to a larger estimate of the pooled standard deviation used for normalization. Thus, normalizing by the pooled standard deviation can in some cases harshly penalize, or down-weight, features that have good separability, leading to a loss in predictive performance. We demonstrate this using simulated data examples in Section 3.1.

The right-hand side of equation (3) also illuminates how normalization is dependent on the relative within-group sample sizes, which may have adverse effects on classifier performance. Suppose data for MVPA are available from a case-control study where the cases have been oversampled. That is, there is one healthy control for each subject with the disease. Suppose further that the true disease prevalence in the population is rare. Then, the estimate of ${\sigma }_j^2$ will be an equal mixture of the group variances ${\sigma }_{j,1}^2$ and ${\sigma }_{j,-1}^2$, whereas the true ${\sigma }_j^2$ in the population depends more heavily on the control-group variance, ${\sigma }_{j,-1}^2$. Methods for dataset or covariate shift address this issue by weighting individual data points to reflect the distribution of covariates in the population [44, 45]. However, these methods are usually implemented after feature normalization. As a result, the estimated decision rule may be undesirably influenced by the use of a biased estimate of the pooled variance.

As an alternative, we propose normalizing the j^th feature as follows:

$$X_{i,j}^C=\frac{X_{i,j}-{\overline{X}}_j}{{\widehat{\sigma }}_j^C},$$

for all subjects i = 1, …, n, where ${\overline{X}}_j$ is the pooled sample mean of feature j, and ${\widehat{\sigma }}_j^C$ j is the sample standard deviation of the j^th feature calculated using the control-group data only. Note that ${\overline{X}}_j$ and ${\widehat{\sigma }}_j^C$ are computed using only the control-group, but the normalization is applied to subjects from both groups. We refer to this as control normalization. Note that for features that contribute greatly to the separability of the groups, the control-group standard deviation will be smaller than the pooled-group standard deviation. Scaling by this smaller value will implicitly up-weight the most discriminative features in comparison to the standard-normalization. In some studies or applications, there may not be a control group. In this case, a reference group may be chosen based on expert knowledge or the scientific goals of the study. In Section 3 we demonstrate how the choice of feature normalization technique may lead to a tradeoff of classifier properties such as sensitivity and specificity but results in better overall accuracy in many settings.

Another advantage of the control normalization is the resulting interpretability of the feature values. Fixing all other SVM features at a constant value, the estimated weight corresponding to feature j conveys the magnitude and direction of change in the decision function score for a one unit increase in feature j, where the units are in terms of the control-group standard deviation of that feature. In many studies, it is likely that more knowledge exists about the distribution of values in the normal population, as the disease being studied may be highly heterogeneous, rare, or not yet well-understood. Being able to interpret the estimated disease pattern relative to the healthy control distribution may improve the reproducibility and clinical value of the MVPA results.

Figure 1 displays an example of the influence of feature normalization on the estimated SVM weight pattern. We generated n = 100 independent feature vectors with two signal features and 20 noise features each. The signal features X₁ and X₂ are generated from a multivariate normal distribution that differs in its parameters between the two diagnostic groups. The signal features jointly have discriminative power, while the remaining noise features are generated independently from the standard normal distribution. The classes are balanced with n₀ = 50 control observations and n₁ = 50 disease group observations. All features are plotted pre-normalization in the first panel of Figure 1. The correlation between X₁ and X₂ in the control group is ρ₀ = −0.2, and it is ρ₁ = −0.6 in the disease-group. The control-normalized, standard-normalized, and domain-scaled versions are plotted in the second, third, and fourth panels. The estimated SVM decision boundary is projected onto the space of these two features by setting the weights of the noise features to zero. The black line in each panel represents this projected decision boundary.

Figure 1. Influence of feature normalization on the SVM decision boundary. From left to right: original feature scales, control-normalized features, standard-normalized features, domain-scaled features.

We carefully chose the parameters for the toy example in Figure 1 because they represent a worst-case scenario where the choice of feature normalization changed the sign of the estimated weight associated with feature X₁. While changes in magnitude of the estimated SVM weights are expected due to the fact that the SVM is scale-invariant, changes in the direction of the effect of a feature are alarming because this alters the biological interpretation of the feature relative to the overall disease pattern. Using the classifier from the standard normalization, as X₂ increases for any fixed X₁ it is more likely a subject presenting with the pair (X₁, X₂) will be classified as healthy. However, using the control normalization the relationship is reversed so that subjects are more likely to be classified as having the disease. Thus, the multivariate pattern in this example is highly dependent on the choice of normalization. While the difference in results and interpretation may not always be so drastic, this example motivates the need for researchers to adopt a single, interpretable technique for feature normalization when performing MVPA using SVMs. We study the effects of feature normalization for a range of parameter settings in Section 3.1.

Using the same data generating set-up as the toy example in Figure 1 but restricting the data sets to include only the two signal features, we implemented a small experiment to compare the reproducibility of estimated disease patterns using the control normalization, standard normalization, and domain scaling. Specifically, we studied the effects of disease group sample size and normalization method using a two-way analysis of variance (ANOVA). In particular, we were interested in the effect of increasing samples in the disease group on the average slope of the estimated SVM line for each normalization method. In the ANOVA model, we included terms for the main effects of method and sample size as well as an interaction term, with the interaction effect being of highest interest. A significant interaction term implies that the effect of increasing the sample size of the disease group on the average estimated SVM slope differs among the normalization methods. In our simulation, we estimated the average SVM slope for each level of sample size and normalization method using 100 independent iterations. We then repeated this procedure 20 times to provide replications for the ANOVA model. Results are presented in Figure 2 and the p-value for the interaction term was highly significant (< 0.0001). Based on Figure 2 we are able to conclude that the change in the SVM slope using the control normalization is less than the change observed using the other methods when varying the disease group sample size. Conservatively, the change observed from 150 to 450 disease group samples using the control normalization is approximately 50% and 40% less compared to the standard normalization and domain scaling, respectively. This suggests that results from MVPA may be more reproducible across studies with different sample sizes when the control normalization is used to preprocess the features prior to SVM training.

Figure 2. Average estimated SVM slope for the control normalization, domain scaling, and standard normalization when the disease group sample size is varied. The control group sample size is fixed at 300.

3. Results

3.1. Simulations

In this section, we study a range of data-generating models to compare the performance of the control normalization, standard normalization, and domain scaling when using the linear SVM for MVPA. For all simulations, we generate p features, (X₁, X₂, …,X_p)^⊤, the first two of which have varying levels of joint discriminative power. The remaining p − 2 are independent noise features. The first two features are generated as mixtures of multivariate normal distributions. The following steps describe the procedure used to obtain the results in Figures 3–6. For each of M=1,000 iterations, n₀ control subjects are generated as independent draws from the model

$$\left(\begin{array}{c}{X}_1^0\\ X_2^0\end{array}\right)~\text{Normal}\left\{\left(\begin{array}{c}0\\ 0\end{array}\right),\left(\begin{array}{cc}1& {\rho }_0\\ {\rho }_0& 1\end{array}\right)\right\},X_j^0~\text{Normal}(0,1),j=3,\dots ,p.$$ (4)

Non-control group subjects are generated as n₁ independent draws from the model

$$\left(\begin{array}{c}{X}_1^1\\ X_2^1\end{array}\right)~\text{Normal}\left\{\left(\begin{array}{c}-1\\ -1\end{array}\right),\left(\begin{array}{cc}1& {\rho }_1\\ {\rho }_1& 1\end{array}\right)\right\},X_j^1~\text{Normal}(0,1),j=3,\dots ,p,$$ (5)

where $X_j^0$, $X_j^1$, j = 3, …, p, are all mutually independent. Additionally, we generate t₀ = 500 independent control-group samples from model (4) and t₁ = 500 independent samples from model (5) for testing. We then train an SVM using the n₀ + n₁ training samples using the scikit learn library in Python, which internally calls libSVM [46]. When n₀ ≠ n₁, we train a class-weighted SVM that weights the cost parameter by (n₁ + n₀)/n₀ for the control group and by (n₁ + n₀)/n₁ for the disease group. In Figures 3 and 4 the correlations are fixed at ρ₀ = 0, ρ₁ = 0, and we vary n₀, n₁ ∈ {20, 40, …, 200}. In Figures 5 and 6 the sample sizes are fixed at n₀ = 50, n₁ = 50, and we vary ρ₀, ρ₁ ∈ {−0.9, −0.8, …, 0.9}.

Figure 3. Average difference in performance measures between the control normalization and standard normalization for a range of sample sizes. Data generated from models 4 and 5. Results reported are percentages.

Figure 6. Average difference in performance measures between the control normalization and domain scaling for a range of feature correlations. Data generated from models 4 and 5. Results reported are percentages.

Figure 4. Average difference in performance measures between the control normalization and domain scaling for a range of sample sizes. Data generated from models 4 and 5. Results reported are percentages.

Figure 5. Average difference in performance measures between the control normalization and standard normalization for a range of feature correlations. Data generated from models 4 and 5. Results reported are percentages.

We compare the average difference in accuracy, area under the ROC curve (AUC), sensitivity, and specificity of the SVM on the test set. Given the true test labels, accuracy is defined as the percentage of correct classifications using the SVM decision rule learned from the training data. Sensitivity is the percentage of correct positive predictions, and specificity is the percentage of correct negative predictions. The ROC curve is the proportion of true positives as a function of the false positive rate which ranges in [0, 1] as the SVM intercept b is varied across the real line. Larger values of the criteria are desirable and indicate better classifier performance.

Each colored square in the heatmaps represents a self-contained simulation with 1,000 iterations. The color indicates the average difference between a given performance measure between the control-normalized SVM and either the standard-normalized or domain-scaled SVM. Dark blue indicates superior performance of the control normalization. Across the simulations summarized in Figures 3 and 4, average accuracies varied approximately between 60% and 80%, average AUCs varied approximately between 70% and 80%, average sensitivities varied approximately between 55% and 80%, and average specificities varied approximately between 55% and 80%. In Figures 3 and 4, the control normalization performs better on average than the standard normalization and domain scaling for most combinations of within-group sample size. Notable exceptions are when the sample size of the control-group is much smaller than that of the disease-group. The standard normalization appears to improve sensitivity when the disease-group sample size is large but seemingly at the cost of reduced specificity. Overall, the results appear similar when comparing the control normalization to domain scaling.

Next, we present a case where the control normalization demonstrates significant improvement over the alternative feature standardizations. The following procedure was used to obtain the results in Figures 7–8. For each of M=1,000 iterations, n₀ control subjects are generated as independent draws from the model

$$\left(\begin{array}{c}{X}_1^0\\ X_2^0\end{array}\right)~\text{Normal}\left\{\left(\begin{array}{c}2\\ -2\end{array}\right),\left(\begin{array}{cc}1& -0.2\\ -0.2& 1\end{array}\right)\right\},X_j^0~\text{Normal}(0,1),j=3,\dots ,p.$$ (6)

Non-control group subjects are generated as n₁ independent draws from the model

$$\left(\begin{array}{c}{X}_1^1\\ X_2^1\end{array}\right)~\text{Normal}\left\{\left(\begin{array}{c}-1\\ -1\end{array}\right),\left(\begin{array}{cc}5& -0.6\\ -0.6& 5\end{array}\right)\right\},X_j^1~\text{Normal}(0,1),j=3,\dots ,p.$$ (7)

Additionally, we generate t₀ = 500 independent control-group samples from model (6) and t₁ = 500 independent samples from model (7) for testing. We vary the group sample sizes, n₀, n₁ ∈ {20, 40, …, 200}. For this set of model parameters, the control-normalization demonstrates improvement in accuracy, AUC, sensitivity, and specificity for a majority of sample sizes when compared to the standard normalization and domain scaling. In some cases, the gains surpass four percent. While there is a tradeoff between sensitivity and specificity with larger disease-group sample sizes, the overall accuracy and AUC favor the control normalization.

Figure 7. Average difference in performance measures between the control normalization and standard normalization for a range of sample sizes. Data generated from models 6 and 7.

Figure 8. Average difference in performance measures between the control normalization and domain scaling for a range of sample sizes. Data generated from models 6 and 7.

3.2. Case Study

The Alzheimer’s Disease Neuroimaging Initiative (ADNI) (http://www.adni.loni.usc.edu) is a multimillion dollar study funded by a number of public and private resources from the National Institute on Aging (NIA), the National Institute of Biomedical Imaging and Bioengineering (NIBIB), Food and Drug Administration (FDA), the pharmaceutical industry, and non-profit organizations. Aims of the study include developing sensitive and specific image-based biomarkers for early diagnosis of Alzheimer’s disease (AD), as well as monitoring the progression of mild cognitive impairment (MCI) and AD. Understanding and predicting disease trajectories is imperative for the discovery of effective treatments that intervene in the early stages of the disease to prevent irreversible damage to the brain.

The ADNI data are publicly available and as a result have been thoroughly analyzed in the neuroimaging literature [47]. A detailed comparison of SVM classification results using different categories of imaging features is given in [9]. In this section, we compare the performance of different SVM feature normalization techniques using volumes obtained from a multi-atlas segmentation pipeline applied to structural MRIs from the ADNI database [48].

The final dataset used for this analysis consists of labels indicating the presence or absence of AD and the volumes of 137 regions of interest (ROIs) in the brain for each subject. Each region is divided by the subject’s total intracranial volume to adjust for differences in individual brain size. The data consist of 230 healthy controls (CN) and 200 patients diagnosed with AD with ages ranging between 55 and 90. Table 2 displays the number (N) and average age of subjects in each diagnosis by sex group. The overall p-value for the mean difference in age between diagnosis groups was not significant (p = 0.59), nor were the p-values significant when calculated separately by sex (p = 0.24, Female; p = 0.68, Male). AD is associated with atrophy in the brain, and thus the AD group has smaller volumes on average in particular ROIs compared to the CN group.

Table 2.

Demographic summary of the ADNI data. The number (N) and average age of subjects in each group is given.

Diagnosis	Sex	N	Average Age
CN	Female	112	76.15
AD	Female	97	75.05
CN	Male	118	75.83
AD	Male	103	76.20

To give intuition about the differences between the control and standard normalization procedures in the ADNI data, we plot the densities of six features, stratified by group, in Figure 9. Whereas the pooled and control-group estimated variability is nearly identical for features such as the white matter parietal lobe and occipital pole, the control-group variability is less than the pooled variability for more marginally separable features such as the amygdala, hippocampus, inferior lateral ventricle, and parahippocampal gyrus. Thus, we expect a SVM trained after control normalization to place relatively heavier weights on these marginally discriminative features than a SVM trained after standard normalization. Figure 10 displays SVM weight patterns from the three methods. Based on Figure 10, it appears all methods obtain similar estimated disease patterns with a few subtle differences. Table 1 lists the top 10 features in order of the magnitude of their weights. As anticipated, the control normalization places more emphasis on the two amygdala regions because their marginal separability ensures a smaller denominator is used in the control-group normalization step, up-weighting these features compared to the standard normalization.

Figure 9. Density plots of ROI volumes by group.

Figure 10. SVM weight patterns for discriminating between AD and CN subjects by feature standardization method.

Table 1.

Top 10 ranked features by the SVM weights in decreasing absolute value.

Rank	Control Normalization	Standard Normalization	Domain Scaling
1	Left Hippocampus	Left Hippocampus	Left Hippocampus
2	Right Hippocampus	Left Inferior Temporal Gyrus	Right Hippocampus
3	Left Inferior Lateral Ventricle	Left Inferior Lateral Ventricle	Left Inferior Lateral Ventricle
4	Left Inferior Temporal Gyrus	Left Middle Frontal Gyrus	Left Inferior Temporal Gyrus
5	Left Amygdala	Right Hippocampus	Left Middle Frontal Gyrus
6	Right Amygdala	Left Superior Frontal Gyrus	Right Middle Temporal Gyrus
7	Right Middle Temporal Gyrus	Right Middle Temporal Gyrus	Left Superior Temporal Gyrus
8	Left Middle Frontal Gyrus	Left Amygdala	Right Amygdala
9	Right Angular Gyrus	Left Superior Temporal Gyrus	Left Amygdala
10	Right Inferior Lateral Ventricle	Right Calcarine Cortex	Left Middle Temporal Gyrus

We compare the control normalization proposed in Section 2.2 to the standard normalization and domain scaling using 5-fold cross-validated estimates of classifier accuracy, area under the curve (AUC), sensitivity, and specificity. Results are shown in Figure 11. The control normalization and domain scaling outperform the standard normalization across all performance measures, increasing the cross-validated accuracy, sensitivity, and specificity by more than one percent. By a small margin, domain-scaling performs best in terms of prediction for this dataset but at the cost of fitted model interpretability. To quantify the uncertainty in the estimates in Figure 11, we repeated the 5-fold cross-validation proceedure 1000 times using random subsamples of 140 patients and 140 controls. The point estimates are shown with a single standard error on each side. For this particular data set, the performance differences are not statistically significant across the three methods.

Figure 11. 5-fold cross-validation results with measures of uncertainty estimated by sub-sampling the original data.

Finally, to compare the generalizabilitiy of the control normalization to other methods and study robustness of the results of this section to possible confounding by gender, we performed a gender-stratified analysis. Table 2 gives the counts of gender by diagnosis and the average age in these categories. An overall t-test of the mean age across diagnosis groups was not significant, and similarly t-tests of mean age between diagnosis groups calculated separately by gender were not significant. However, it is still possible that disease patterns vary across gender. In addition, the data are sparse for subjects less than 70 years of age. Thus, to study generalizability and robustness of our results to possible age and gender confounding, we split the data by gender and restricted our analysis to subjects 70 and older. Using this subset of the ADNI data, we trained the SVMs for each normalization method on 100 randomly sampled females and applied the model to classify male subjects as AD or CN. We repeated this process 1000 times and report average results in Table 3. We also report average results from the same procedure where we used random subsets of male subjects to train the models and diagnose the female group as AD or CN. On average, we observe slight gains in accuracy and area under the ROC curve when using the control normalization. Gains of one to two percentage points are along the lines of the improvements observed in our simulation study in Section 3.1. While not statistically significant, it is notable that the control normalization outperformed the other methods across all performance measures with the exception of displayed equivalence in sensitivity to the standard normalization.

Table 3.

Gender-stratified analysis of ADNI data.

Training	Method	Accuracy	AUC	Sensitivity	Specificity
	Control Normalization	87%	89%	85%	88%
Females	Domain Scaling	86%	88%	86%	86%
	Standard Normalization	86%	88%	85%	87%
	Control Normalization	86%	89%	78%	92%
Males	Domain Scaling	85%	87%	76%	91%
	Standard Normalization	85%	88%	78%	90%

4. Discussion

Interpretability of estimated disease patterns is a desirable quality for most applications of MVPA to neuroimaging data. Standardizing features by the control-group variability leads to estimated support vector weights whose individual interpretation relies solely on a sample of normal subjects. In some cases, this may increase the reliability of image-based biomarkers for disease classification. Indeed, the use of normative samples to develop cognitive tests is commonplace in the neuropsychological literature, and we refer the reader to [49] and references therein for many examples. Along these lines, we advocate for control-group feature standardization when estimating disease patterns in the brain and developing image-based biomarkers. Additionally, as demonstrated in Section 2.2, the control normalization improves the interpretability of the SVM weight pattern due to the relative stability of the weights across different disease group sample sizes. It is difficult to interpret estimated disease patterns that are affected by sample size, since the underlying “true” SVM weights should not depend on the number of samples in each group.

We showed in Section 2.2 that the standard normalization method depends on the relative sample size of the two groups as well as the marginal separability; in contrast, the control normalization is unaffected by these qualities of the data and hence provides better generalizability across samples. We believe that including a control normalization step in the MVPA preprocessing pipeline is a simple alternative to current practice that promises increased interpretability, generalizability, and performance of the results.

We have focused attention on classification of subjects into groups based on medical diagnosis. In principle, the idea of control-based feature normalization for MVPA naturally extends to classification of events within subjects. For example, MVPA is increasingly applied to fMRI data in order to decode which patterns of activity correspond to a specific cognitive task or mental state [11, 50, 51]. In this setting, one could normalize the fMRI data using the subset of time points where the subject is at rest or performing the control task. Application of these techniques to within-subject timeseries data would have the potential for increasing task decoding accuracy in the face of performance variability that exists both between [52] and within individuals [53, 54], and is known to have a substantial relationship to patterns of activation [55, 56]. Performance variability is a particularly important confound in studies of neuropsychiatric conditions [57, 58] as well as lifespan studies of [59] and aging [60]. Such an application would represent a natural extension of the current work, and the overarching idea remains the same: feature normalization as a data preprocessing step in MVPA should be applied in an intentional way with an understanding of the way in which it might affect the results of the analysis.

5. Conclusion

The roots of feature scaling for preprocessing lie in the neural network literature of the 1990s. The universal approximation theorem, which broadly states that simple neural networks can approximate a rich set of functions, was initially proven for functions defined on a unit hypercube domain using a multilayer perceptron constructed with sigmoidal neurons [61]. It became natural to assume that centering and scaling of data would lead to a faster convergence even though neural networks are theoretically affine invariant [62]. For the most part, the optimization turned out to be more graceful with these scaled inputs, since it slowed down network saturation and avoided the vanishing gradients problem to a certain extent.

However, applying scaling to kernel methods such as SVMs or distance-based methods such as k-means tends to yield completely different results depending on the scaling method used. This is because these methods are not transformation-invariant. In such a case, scaling essentially imposes a form of soft feature selection since it implicitly changes the metric used for computing the kernel matrix. This fact is important in the context of image-based diagnosis using SVMs with region of interest (ROI) data. Scaling implicitly enforces the fact that variation in the amygdala, which is a relatively small structure in terms of volume, is as important as that in the prefrontal lobe, which is much larger in volume. Thus, appropriate scaling of features is an important but under-emphasized issue that we have attempted to call attention to in this manuscript.

It is critical for researchers wishing to interpret the results of MVPA from SVMs to understand how the choice of feature normalization influences the results, as well as how to determine the best method for their scientific question. We have proposed a control-based normalization and demonstrated several advantages of the approach for classifying subjects into groups, for example, by medical diagnosis. Most notably, we have highlighted the possibility of improved classifier performance according to criteria such as accuracy and AUC for a comprehensive set of data generating distributions. The control normalization improves classifier performance by giving higher weight, relative to other standardization techniques, to features with greater marginal separability between groups. Depending on the underlying data generating distribution and relative sample size between groups, different classifiers will experience tradeoffs between sensitivity and specificity. The optimal choice of feature normalization may depend on the unknown data generating distribution as well as certain clinical considerations. As a result, the interpretability of the control normalization is an attractive property that makes it amenable to a vast majority of clinical applications. Considered along with overall increases in accuracy and AUC demonstrated by the control normalization in the simulations and data examples, we advocate for its adoption as standard practice in MVPA using the SVM.

Highlights.

• A control-based feature normalization is proposed for support vector classification.

• Classifier performance is improved over two common feature normalization methods.

• Estimated multivariate patterns are more interpretable using the proposed method.