Table 1.
Machine learning papers predicting symptom progression using the Parkinson’s Progression Markers Initiative data set.
| Study | Input data | Outcome measures | Modeling approach | Validation method | Summary | ||||
|---|---|---|---|---|---|---|---|---|---|
| C | D | R | B | I | |||||
| 1. Adams et al. (2021) | ✓ | ✓ | UPDRS III at year 4 | CNN | Ten-fold CV | Trained a CNN to predict motor symptoms at year 4 from baseline raw DAT images and year 0 and 1 UPDRS III. | |||
| 2. Chahine et al. (Chahine et al., 2021) | ✓ | ✓ | Diagnosis of α-synucleinopathy (aSN) | Cox hazard regression | None | Tests DAT SBR at baseline for predicting future diagnosis of PD, Lewy Body Dementia, or Multiple System Atrophy. Reports sensitivity and specificity but no validation. | |||
| 3. Chen et al. (2021) | ✓ | ✓ | ✓ | ✓ | MCI diagnosis in patients with REM disorder | LASSO, Cox hazard regression | Validation set | Predicted time to MCI from baseline genetic, CSF, DAT, and clinical features. | |
| 4. Combs et al. (2021) | ✓ | Cognition (neuropsychological tests) at year 1 | Stepwise regression | Validation set | Prediction of cognitive performance from baseline clinical and cognitive scores in both controls and patients. | ||||
| 5. D’Cruz et al. (2021) | ✓ | ✓ | FoG at year 2 | Vertex-based shape analysis stepwise logistic regression | None (statistical tests of a subset of features in PPMI data set) | Note: Model was fit to internal cohort. Baseline clinical and MRI features used to predict FoG. Variables selected based on significance and stepwise regression. AUC reported but not validated. Statistical tests of MRI features performed on PPMI data. | |||
| 6. Faghri et al. (2018) | ✓ | UPDRS progression sub-type | Nonnegative Matrix Factorization, Gaussian Mixture Models, Random forests | Five-fold CV Test set External validation set | Combined unsupervised and supervised methods to divide patients into progression sub-types and predict sub-type score from baseline clinical measures | ||||
| 7. Gramotnev et al. (2019) | ✓ | ✓ | ✓ | ✓ | Rate of MoCA change | Logistic regression with variable selection based on relative importance | Monte Carlo (after selecting significant features*) | Used baseline clinical, genetic, DAT, and CSF to predict rate of cognitive decline. Variables selected based on performance before validation*. | |
| 8. Gu et al. (2020) | ✓ | ✓ | ✓ | Geriatric Depression Scale at year 2 | XGBoost, Stepwise logistic regression | Ten-fold CV for hyperparameter tuning Validation set | Compared methods for predicting depression severity at year 2 from clinical, CSF, and DAT measures. | ||
| 9. Hayete et al. (2017) | ✓ | ✓ | ✓ | Rate of change in UPDRS II + III and MoCA | Dynamic Bayesian graphical model | None (Statistical tests of a subset of features in PPMI data set) | Note: Model was fit to LAB-PD data. PPMI was used for limited external validation. Predicted motor and cognitive progression mainly at years 5-7 of follow-up. Direct predictive validation was not performed, but a subset of findings was tested statistically in PPMI. | ||
| 10. Jackson et al. (2021) | ✓ | ✓ | Change in UPDRS III at year 1 | Ridge regression | External validation set (PPMI) | Predicted motor decline at 1 year from baseline clinical and DAT factors. Note: trained on placebo arm of clinical trial, tested on PPMI | |||
| 11. Kim et al. (2019) | ✓ | ✓ | ✓ | Freezing of Gait (FoG from UPDRS) at year 4 | Cox hazard regression | None | Predicted future freezing of gait, from baseline clinical measures, CSF, and DAT. Reports AUC but no validation. | ||
| 12. Kim and Jeon (2021) | ✓ | ✓ | FoG (UPDRS) up to year 8 | NA (ROC analysis of NFL) | None | Predicted gait freezing using serum NFL. Reports AUC but no validation. | |||
| 13. Latourelle et al. (2017) | ✓ | ✓ | ✓ | ✓ | Rates of motor and daily living symptoms (combined UPDRS II and III totals, rates estimated from linear mixed-effects models) | Reverse engineering and forward simulation (REFS) model ensemble | Five-fold CV External validation set (LABS-PD) | Large-scale prediction of symptom progression using model ensemble trained on >17,000 SNPs, clinical variables, and DAT and CSF features. <4% variance explained by biological variables. Tested results on external cohort | |
| 14. Ma et al. (2021) | ✓ | ✓ | UPDRS III at multiple years | Multiple ML models (LASSO, ridge, random forests, gradient boosting) with recursive feature elimination | CV within training set for variable selection validation set | Compared performance of multiple ML models in using each year’s clinical and CSF measures to predict subsequent year’s motor scores | |||
| 15. Nguyen et al. (2020) | ✓ | ✓ | ✓ | UPDRS III total and MoCA scores at years 1, 2, 3, and 4 | Deformation-based morphometry, DNN autoencoder | Five-fold CV (after selecting significant regions*) | Predicted motor and cognitive deficits from baseline MRI, CSF, and clinical scores in patients with REM disorder. Variables selected based on performance before validation*. | ||
| 16. Rahmim et al. (2017) | ✓ | ✓ | UPDRS III at year 4 | Random forests | LOO | Year 1 and 2 DAT, MRI, and clinical scores were used to predict UPDRS III total at year 4 | |||
| 17. Ren et al. (2021) | ✓ | Hoehn & Yahr score | Multivariate functional PCA, Cox hazard regression | External validation set (LABS-PD) | Combined unsupervised dimensionality reduction of clinical and cognitive variables with prediction of functional outcomes. | ||||
| 18. Rutten et al. (2017) | ✓ | ✓ | 2-year change in anxiety (STAI) | Linear mixed-effects model with stepwise selection procedure | None | Stepwise selection of features for linear mixed-effects model predicting 2-year change in STAI scores from baseline clinical scores and DAT features. No validation of selected features was performed. | |||
| 19. Salmanpour et al. (2019) | ✓ | MoCA at year 4 | DNNs, LASSO, random forest, ridge, others | Monte Carlo Test set | Tested future cognitive score from a large set of feature selection and prediction algorithms. Genetic algorithm combined with local linear trees performed the best. | ||||
| 20. Schrag et al. (2017) | ✓ | ✓ | ✓ | ✓ | Cognition (MoCA) at 2 years | Logistic regression with variables pre-filtered based on significance | Monte Carlo Ten-fold CV validation set all after selecting significant features* | Predicted cognitive impairment with clinical scores, CSF, APOE status, and DAT. Selected variables before validation*. | |
| 21. Simuni et al. (2016) | ✓ | ✓ | ✓ | Time to initiation of symptomatic treatment | Random survival forest | CV | Predicted time to initiation of symptomatic therapy using random survival forests. No biological variables increased accuracy of prediction above clinical baseline. | ||
| 22. Tang et al. (2019) | ✓ | ✓ | UPDRS III at year 4 | DNN | LOO (after selecting significant features*) | Artificial neural network to predict future motor symptoms from imaging and clinical features. Variables selected based on performance before validation*. | |||
| 23. Tang et al. (2021) | ✓ | ✓ | ✓ | Cognitive decline (MoCA or neuropsychological test scores) | LASSO, Cox hazard Regression | Validation set (t-tests to remove variables that differ between training and validation sets*) | Prediction of cognitive decline using baseline clinical, CSF, and MRI features. Features were filtered after training based on similarity between training and test sets* | ||
| 24. Tsiouris et al. (2017) | ✓ | ✓ | ✓ | ✓ | Rate of change in UPDRS total up to year 2 | RIPPER (Cohen, 1995) | Ten-fold CV | Predicted change in UPDRS scores at 2-and 4-year epochs after selecting from over 600 baseline variables, including genetic, CSF, clinical and imaging features | |
| 25. Tsiouris et al. (2020) | ✓ | ✓ | ✓ | ✓ | Rate of change in UPDRS total up to year 4 | Naïve Bayes, RIPPER | Ten-fold CV | Extended (Tsiouris et al., 2017) to 4 year follow up | |
| 26. Zeighami et al. (2019) | ✓ | ✓ | Change in Global Composite Outcome (Fereshtehnejad et al., 2015) | Voxel-based Morphometry, Independent Component Analysis | Ten-fold CV (after selecting significant voxels) | Tested whether baseline MRI-based atrophy marker could predict change in overall severity. Cross-cohort validation prevented data leakage. | |||
Because of the focus of PPMI on variation in symptom trajectories, we summarize here the 26 machine learning papers from our literature search attempting to predict future symptoms. Input Data: C, clinical/demographic; D, DNA/genotype; R, RNA; B, biomarker/biospecimen; I, imaging; CV, cross-validation; LOO, leave-one-out; UPDRS, Unified Parkinson’s Disease Rating Scale; DNN, deep neural network; CNN, convolutional neural network; MCI, Mild Cognitive Impairment; NFL, Neurofilament light chain; *potential data leakage. In papers containing both null hypothesis tests and measures of predictive accuracy, only variables and methods included in predictive tests are considered here.