TABLE 5.
Case‐control classification accuracies and ROC AUC measures (on cross‐validation) with white matter integrity features in the cMDD‐STR sample (40 cases and 40 controls)
| Classifier type | Feature selection | Hyperparam optimisation | Outer CV | Inner CV | Feature domain | Classification accuracy (sensitivity/specificity) | ROC AUC |
|---|---|---|---|---|---|---|---|
| PLR | Embedded | Grid search | LOOCV | 10‐fold | FA | 31.25% (35.00/27.50%) | 0.363 |
| MD | 53.75% (55.00/52.50%) | 0.589 | |||||
| Combined | 48.75% (50.00/47.50%) | 0.474 | |||||
| SVM | None | None | LOOCV | ‐ | FA | 48.75% (40.00/57.50%) | 0.484 |
| MD | 57.50% (55.00/60.00%) | 0.536 | |||||
| Combined | 52.50% (50.00/55.00%) | 0.520 | |||||
| Grid search | LOOCV | FA | 50.00% (40.00/60.00%) | 0.505 | |||
| MD | 61.25% (57.50/65.00%) | 0.673 | |||||
| Combined | 53.75% (52.50/55.00%) | 0.559 | |||||
| Statistical filter | None | FA | 40.00% (32.50/47.50%) | 0.345 | |||
| MD | 37.50% (30.00/45.00%) | 0.353 | |||||
| Combined | 30.00% (20.00/40.00%) | 0.283 | |||||
| Grid search | 10‐fold | FA | 52.50% (60.00/45.00%) | 0.476 | |||
| MD | 38.75% (40.00/37.50%) | 0.385 | |||||
| Combined | 38.75% (40.00/37.50%) | 0.328 | |||||
| Sequential elimination | None | FA | 47.50% (40.00/55.00%) | 0.488 | |||
| MD | 53.75% (52.50/55.00%) | 0.534 | |||||
| Combined | 51.25% (55.00/47.50%) | 0.501 | |||||
| DT | None | None | LOOCV | ‐ | FA | 53.75% (50.00/57.50%) | 0.434 |
| MD | 56.25% (55.00/57.50%) | 0.514 | |||||
| Combined | 57.50% (42.50/72.50%) | 0.552 | |||||
| Grid search | LOOCV | FA | 48.75% (65.00/32.50%) | 0.350 | |||
| MD | 47.50% (40.00/55.00%) | 0.372 | |||||
| Combined | 51.25% (47.50/55.00%) | 0.563 | |||||
| Statistical filter | None | FA | 45.00% (35.00/55.00%) | 0.323 | |||
| MD | 43.75% (30.00/57.50%) | 0.331 | |||||
| Combined | 36.25% (30.00/42.50%) | 0.256 | |||||
| Grid search | 10‐fold | FA | 42.50% (47.50/37.50%) | 0.280 | |||
| MD | 40.00% (35.00/45.00%) | 0.204 | |||||
| Combined | 33.75% (35.00/32.50%) | 0.267 | |||||
| Sequential elimination | None | FA | 48.75% (45.00/52.50%) | 0.433 | |||
| MD | 52.50% (52.50/52.50%) | 0.458 | |||||
| Combined | 56.25% (55.00/57.50%) | 0.488 |
Note: Top accuracies for SVM, PLR and DT classifiers are in italics.
Abbreviations: CV, cross‐validation; DT, decision tree; LOOCV, leave‐one‐out cross‐validation; PLR, penalised logistic regression; ROC AUC, receiver operating characteristic area under the curve; SVM, support vector machine.