Table 3. Model performance by age.
This table details the accuracy, sensitivity, specificity, precision, and recall for 8 classifiers (Table 1) and for 4 age ranges found in evaluation of 162 home videos with an average length of 2 minutes. We also provide the IRA, which indicates the frequency with which the model results from all 3 raters’ feature tags agreed on class. The top-performing classifier was LR5, which yielded an accuracy of 88.9%, sensitivity of 94.5%, and specificity of 77.4%. Other notable classifiers were SVM5 and LR10, which yielded 85.4% and 84.8% accuracy, respectively. These 3 best-performing classifiers showed improved classification power within certain age ranges.
| Age group | Statistic | ADTree8 | ADTree7 | SVM5 | LR5 | LR9 | SVM12 | SVM10 | LR10 |
|---|---|---|---|---|---|---|---|---|---|
| Overall (116 ASD, 46 non-ASD, 64.4% male) |
Sensitivity | 100% | 94.5% | 100% | 94.5% | 100% | 100% | 100% | 100% |
| Specificity | 22.4% | 37.3% | 54.9% | 77.4% | 31.4% | 0% | 17.6% | 51.0% | |
| Accuracy | 76.1% | 76.3% | 85.4% | 88.9% | 78.4% | 71.6% | 73.9% | 84.8% | |
| IRA | 76.1% | 67.5% | 68.4% | 71.7% | 75.9% | 71.6% | 79.5% | 70.9% | |
| Precision | 74.3% | 76.3% | 82.3% | 89.7% | 76.0% | 71.6% | 72.4% | 82.0% | |
| UAR | 61.2% | 65.9% | 77.5% | 86.0% | 65.7% | 50% | 58.8% | 75.5% | |
| ≤2 years (17 ASD, 20 non-ASD, 56.8% male) |
Sensitivity | 100% | 100% | 100% | 93.3% | 100% | 100% | 100% | 100% |
| Specificity | 14.2% | 18.2% | 38.1% | 77.3% | 22.7% | 0% | 14.3% | 47.6% | |
| Accuracy | 50% | 51.4% | 62.9% | 83.8% | 54.1% | 40.5% | 50.0% | 96.2% | |
| IRA | 53% | 48.6% | 45.7% | 51.4% | 48.6% | 100% | 66.7% | 100% | |
| Precision | 45.5% | 45.5% | 51.9% | 73.7% | 46.9% | 45.9% | 45.4% | 57.7% | |
| UAR | 57.1% | 59.1% | 69.1% | 85.3% | 61.4% | 50.0% | 57.2% | 73.8% | |
| >2 years and ≤4 years (39 ASD, 17 non-ASD, 66.1% male) |
Sensitivity | 100% | 97.3% | 100% | 91.8% | 100.0% | 100% | 100% | 100% |
| Specificity | 23.6% | 50% | 66.7% | 73.7% | 38.9% | 0% | 22.2% | 58.8% | |
| Accuracy | 76.4% | 50% | 88.9% | 85.7% | 80.4% | 66.7% | 74.5% | 86.8% | |
| IRA | 74.5% | 81.8% | 63.0% | 75.0% | 76.8% | 100% | 85.4% | 77.4% | |
| Precision | 74.5% | 80.0% | 85.7% | 87.2% | 77.6% | 69.6% | 72.5% | 83.7% | |
| UAR | 61.8% | 73.7% | 83.4% | 82.8% | 69.5% | 50.0% | 61.1% | 79.4% | |
| >4 years and ≤6 years (34 ASD, 8 non-ASD, 61.9% male) |
Sensitivity | 100% | 96.8% | 100% | 96.9% | 100.0% | 100% | 100% | 100% |
| Specificity | 40.0% | 60% | 72.7% | 90.9% | 40.0% | 0% | 18.2% | 50.0% | |
| Accuracy | 85.4% | 87.8% | 92.9% | 95.3% | 85.7% | 74.4% | 79.1% | 88.1% | |
| IRA | 85.4% | 78.0% | 78.6% | 79.1% | 85.7% | 93.0% | 76.7% | 71.4% | |
| Precision | 83.8% | 88.2% | 91.2% | 96.9% | 84.2% | 80.9% | 78.0% | 86.5% | |
| UAR | 70.0% | 78.4% | 86.4% | 93.9% | 70.0% | 50.0% | 59.1% | 75.0% | |
| >6 years (26 ASD, 1 non-ASD, 74.1% male) | Sensitivity | 100% | 84.6% | 100% | 96.2% | 100% | 100% | 100% | 100% |
| Specificity | 0% | 0% | 0% | 0% | 0% | 0% | 0% | 0% | |
| Accuracy | 96.2% | 81.5% | 96.2% | 92.6% | 96.2% | 96.2% | 96.2% | 96.3% | |
| IRA | 96.2% | 70.4% | 96.2% | 81.5% | 96.2% | 100% | 100% | 85.2% | |
| Precision | 96.3% | 95.7% | 96.3% | 96.2% | 96.3% | 96.3% | 96.3% | 96.3% | |
| UAR | 50.0% | 42.3% | 50.0% | 48.1% | 50.0% | 50.0% | 50.0% | 50.0% |
Abbreviations: ADTree7, 7-feature alternating decision tree; ADTree8, 8-feature alternating decision tree; ASD, autism spectrum disorder; IRA, interrater agreement; LR5, 5-feature logistic regression classifier; LR9, 9-feature logistic regression classifier; LR10, 10-feature logistic regression classifier; SVM5, 5-feature support vector machine; SVM10, 10-feature support vector machine; SVM12, 12-feature support vector machine; UAR, unweighted average recall.