Table 1.
Classification results in each outer test fold after using the best setup (=input and classification pipeline) from the internal optimization fold.
| Test fold | Input | Classification pipeline | Balanced accuracy | F1 | Precision | Recall |
|---|---|---|---|---|---|---|
| PD vs. HC | ||||||
| Smartwatch data only | ||||||
| #1 | Acceleration + Rotation | (B) SVM | 69.55% | 87.5% | 85.96% | 89.09% |
| #2 | Acceleration + Rotation | (A) NN | 76.31% | 92.17% | 88.33% | 96.36% |
| #3 | Rotation | (B) CatBoost | 75.4% | 91.23% | 88.14% | 94.55% |
| #4 | Rotation | (B) SVM | 87.9% | 94.55% | 94.55% | 94.55% |
| #5 | Acceleration | (B) SVM | 85.77% | 95.65% | 93.22% | 98.21% |
| Questionnaire only | ||||||
| #1 | NMS | (D) CatBoost | 85.57% | 89.32% | 95.83% | 85.57% |
| #2 | NMS | (D) CatBoost | 88.3% | 92.45% | 96.08% | 88.3% |
| #3 | NMS | (D) CatBoost | 91.02% | 95.41% | 96.3% | 91.02% |
| #4 | NMS | (D) CatBoost | 93.64% | 93.2% | 100.0% | 93.64% |
| #5 | NMS | (D) CatBoost | 90.42% | 92.45% | 98.0% | 90.42% |
| PD vs. DD | ||||||
| Smartwatch data only | ||||||
| #1 | Rotation | (B) SVM | 77.35% | 68.29% | 73.68% | 63.64% |
| #2 | Acceleration + Rotation | (B) SVM | 74.98% | 65.12% | 70.0% | 60.87% |
| #3 | Acceleration | (A) NN | 58.46% | 43.14% | 39.29% | 47.83% |
| #4 | Rotation | (B) SVM | 68.46% | 55.0% | 64.71% | 47.83% |
| #5 | Rotation | (B) SVM | 66.64% | 52.38% | 57.89% | 47.83% |
| Questionnaire only | ||||||
| #1 | NMS | (D) CatBoost | 74.68% | 63.64% | 63.64% | 74.68% |
| #2 | NMS | (D) CatBoost | 60.47% | 42.86% | 47.37% | 60.47% |
| #3 | NMS | (D) CatBoost | 68.62% | 56.0% | 51.85% | 68.62% |
| #4 | NMS | (D) CatBoost | 68.06% | 55.56% | 48.39% | 68.06% |
| #5 | NMS | (D) CatBoost | 67.0% | 53.33% | 54.55% | 67.0% |
Options A, B, and C were applied to smartwatch-data only, option D was applied to questionnaire data only. A: Manually defined features, shallow machine learning, B: Automatic feature extraction via classical signal processing, C: Automatic feature extraction via deep learning (not listed due to underperformance), D: Decision-tree based classifier.