Table 12.

Methods that address technical problems in classification and prediction

Problem	Approach	Results and conclusions	Reference
Variance in validation strategies can make comparison of the performance of supervised classification algorithms difficult.	Use of an empirical estimator that performs validation on two disjoint sets and comparison with different variance estimators in an AD versus CN classification experiment.	Proposed estimator is constant with sample size and is unbiased with regard to training set size. Recommend against the use of leave one out cross-validation because of its high variance.	[446]
Estimation of the sensitivity of a biomarker to the early diseased stage based on its performance on the fully diseased stage.	Use of empirical likelihood-based (ELB) confidence intervals.	ELB method is more robust than parametric methods and more accurate than nonparametric methods of confidence interval estimation.	[447]
Lack of a method for selecting diagnostic cut points in a multistage disease that is not dependent on correct classification rates.	Developed new measure, maximum absolute determinant for diseases with k stages, which selects cut points using all available classification data.	When applied to ADNI biomarker data, the proposed method more accurately selected cut points for the early diseased stage than existing methods.	[448]
Patterns of atrophy in normal aging can confound multivariate models of atrophy in AD.	Compared two age correction approaches on AD versus CN classification and prediction of MCI to AD progression: (1) using age as a covariate in MRI-derived measures; (2) linear detrending of age-related changes based on CN measures.	Both models improved classification and prediction accuracy. Analysis of incorrectly classified subjects suggested that the influence of cognitive impairment, APOE genotype, and gender is partially masked by age effects.	[449]
Errors in reference test for AD biomarkers (clinical diagnosis or Aβ PET in the absence of a gold standard) cause bias in their diagnostic accuracy.	Uses Bayesian method to determine diagnostic accuracy of AD biomarkers taking imperfectness of reference test into account.	Proposed methodology improved estimates of exact diagnostic values of three CSF biomarkers in the AD versus CN classification.	[450]
Imperfect reference test can lead to bias in accuracy of a combination of diagnostic biomarkers.	Uses Bayesian method to select combination of biomarkers that maximizes the AUC while taking imperfectness of reference test into account.	Proposed methodology improved estimates of AUCs of AD biomarkers over traditional logistic regression model.	[451]
Incomplete/imbalanced data biases estimation of diagnostic accuracy of AD biomarkers.	Novel approach uses augmented weighted estimator for covariate-specific time-dependent receiver operator curves using information from subjects with incomplete data.	Proposed estimator corrected bias and improved efficiency of classification in incomplete data sets over other estimators.	[452]
Incomplete/imbalanced data in multimodal classification.	Extensive and systematic analysis of effectiveness of combinations of sampling techniques (undersampling, oversampling, and a combination), six common feature selection algorithms, and Random Forest and SVM classifiers on AD/MCIc versus AD and MCI versus CN classification problems.	K-Medoids undersampling technique gave best results on imbalanced data sets. Sparse logistic regression with stability selection was best feature selection technique.	[453]
	Novel approach based on collection of feature values into a large incomplete matrix, and subsequent matrix shrinkage and completion using a multitask learning algorithm.	Improved classification accuracy over two recent methods for accounting for missing data (including Incomplete Multi-Source Feature Learning [454]).	[455]
	Novel approach, Multi-Task Linear Programming Discriminant analysis which decomposes the classification problem into different classification tasks, adaptively chooses different feature subsets for each task, then solves them jointly.	Improved classification accuracy of MCIc versus MCInc classification over Incomplete Multi-Source Feature Learning [454].	[456]
	Novel approach (3D-CNN) based on a convolutional neural network that can estimate missing data in an output modality (PET images) using data from an input modality (MR images).	3D-CNN approach used on incomplete data sets achieved similar classification accuracies to using complete data sets in AD versus CN, MCI versus CN, and MCIc versus MCInc tasks and outperformed two commonly used missing data estimation methods.	[457]
Conventional false discovery rate procedures for voxel level multiple testing ignore correlations between neighboring voxels.	Novel approach extends the local significance index procedure with a Markov random field model to consider spatial correlations along neighboring voxels.	When method was applied to ADNI FDG PET data, it outperformed other false discovery rate procedures.	[458]

Abbreviations: AD, Alzheimer’s disease; CN, cognitively normal; ADNI, Alzheimer’s Disease Neuroimaging Initiative; MCI, mild cognitive impairment; MRI, magnetic resonance imaging; CSF, cerebrospinal fluid; AUC, area under the receiver operating characteristic; SVM, support vector machine; MCIc, mild cognitive impairment converters (progressive MCI); MCInc, mild cognitive impairment nonconverters (stable MCI); MR, magnetic resonance; PET, positron emission tomography.