Table 8.
Performance of task-specific quality classification models in binary setups. The metrics (mean and SD) represent their usual meaning (eg, precision, recall, and F1-scores are evaluated with respect to the good quality class). All values, including AUROCa, are reported on a 0-100 scale. Binary setup borderline→poor indicates that the samples from the borderline class were considered as poor quality during training and evaluation. Likewise, for the borderline→good setup, the borderline class was considered as good quality.
| Task/binary setup | Best model | Accuracy | Precision | Recall | F1-score | AUROC | |||||||
| Finger-tapping | |||||||||||||
|
|
Borderline→poor | Random forest | 81.8 (3.5) | 78.7 (5.6) | 81.8 (5.2) | 80.1 (4.2) | 87.1 (3.4) | ||||||
|
|
Borderline→good | XGBoostb | 83.5 (3.3) | 83.7 (4.2) | 90.7 (3.3) | 87.0 (2.9) | 86.6 (3.6) | ||||||
| Smile | |||||||||||||
|
|
Borderline→poor | SVMc | 77.0 (4.2) | 73.2 (5.6) | 88.2 (4.6) | 80.0 (4.0) | 83.9 (4.0) | ||||||
|
|
Borderline→good | Random forest | 80.1 (4.1) | 88.3 (3.8) | 85.9 (4.0) | 87.0 (3.0) | 77.3 (6.5) | ||||||
| Speech | |||||||||||||
|
|
Borderline→poor | Random forest | 83.9 (3.6) | 90.5 (3.5) | 88.1 (3.6) | 89.2 (2.6) | 84.4 (5.4) | ||||||
|
|
Borderline→good | Random forest | 95.0 (2.2) | 100.0 (0.0) | 94.3 (2.4) | 97.1 (1.3) | 99.5 (0.5) | ||||||
aAUROC: area under the receiver operating characteristic curve.
bXGBoost: extreme gradient boosting.
cSVM: support vector machine.