Table 3.
App performance stratified by sex, race, ethnicity, age, quality score and prediction confidence threshold
| Group | n | NT | Correct | Not correct | AUC (%; 95% CI) | Sensitivity (STD) | Specificity (STD) | PPV (adjusted) | NPV (adjusted) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Autism | ||||||||||
| Sex | Boys | 196 | 158 | 123 | 35 | 89.6 (3.4) | 86.8 (5.3) | 77.8 (3.2) | 48.5 (7.7) | 96.1 (99.6) |
| 38 | 33 | 5 | ||||||||
| Girls | 181 | 170 | 142 | 28 | 89.1 (6.5) | 90.9 (9.1) | 83.5 (2.9) | 26.3 (10.5) | 99.3 (99.8) | |
| 11 | 10 | 1 | ||||||||
| Race | White | 278 | 255 | 211 | 44 | 86.9 (4.9) | 82.6 (7.8) | 82.7 (2.4) | 30.2 (9.2) | 98.1 (99.5) |
| 23 | 19 | 4 | ||||||||
| Black | 39 | 28 | 15 | 13 | 81.2 (8.5) | 90.9 (9.0) | 53.6 (9.5) | 43.5 (4.0) | 93.8 (99.6) | |
| 11 | 10 | 1 | ||||||||
| Other | 60 | 45 | 39 | 6 | 97.6 (2.8) | 93.3 (7.2) | 86.7 (4.6) | 70.0 (12.9) | 97.5 (99.8) | |
| 15 | 14 | 1 | ||||||||
| Ethnicity | Not Hispanic/Latino | 342 | 306 | 245 | 61 | 87.8 (3.8) | 86.1 (5.7) | 80.1 (2.3) | 33.7 (8.4) | 98.0 (99.8) |
| 36 | 31 | 5 | ||||||||
| Hispanic/Latino | 35 | 22 | 20 | 2 | 95.3 (4.3) | 92.3 (7.1) | 90.9 (6.2) | 85.7 (17.7) | 95.2 (99.8) | |
| 13 | 12 | 1 | ||||||||
| Age (months) | 17–18.5 | 164 | 159 | 125 | 34 | 94.5 (7.1) | 1.00 (0.0) | 78.6 (2.8) | 12.8 (9.0) | 1.0 (1.0) |
| 5 | 5 | 0 | ||||||||
| 18.5–24 | 104 | 86 | 72 | 14 | 89.5 (5.1) | 83.3 (9.5) | 83.7 (4.7) | 51.7 (9.8) | 96.0 (99.6) | |
| 18 | 15 | 3 | ||||||||
| 24–36 | 109 | 83 | 68 | 15 | 90.1 (4.2) | 88.5 (6.0) | 81.9 (4.3) | 40.6 (8.8) | 97.8 (99.7) | |
| 26 | 23 | 3 | ||||||||
| Quality score | Higher than 75% | 349 | 310 | 259 | 51 | 89.6 (3.4) | 84.6 (5.0) | 83.5 (2.1) | 39.3 (9.8) | 97.7 (99.6) |
| 39 | 33 | 6 | ||||||||
| Lower than 75% | 28 | 18 | 6 | 12 | 76.1 (10.0) | 1.0 (0.0) | 33.3 (12.3) | 45.5 (3.1) | 1.0 (1.0) | |
| 10 | 10 | 0 | ||||||||
| Prediction confidence threshold | Threshold 5% | 251 | 216 | 201 | 15 | 92.6 (3.1) | 91.4 (4.4) | 93.1 (1.6) | 68.1 (21.9) | 98.5 (99.8) |
| 35 | 32 | 3 | ||||||||
| Threshold 10% | 279 | 243 | 219 | 24 | 92.4 (3.0) | 88.9 (4.9) | 90.1 (2.1) | 57.1 (16.0) | 98.2 (99.7) | |
| 36 | 32 | 4 | ||||||||
| Threshold 15% | 297 | 258 | 228 | 30 | 92.0 (3.0) | 89.7 (5.1) | 88.4 (2.0) | 53.8 (14.1) | 98.3 (99.7) | |
| 39 | 35 | 4 | ||||||||
| Threshold 20% | 311 | 270 | 238 | 32 | 91.6 (3.0) | 87.8 (5.4) | 88.1 (1.7) | 52.9 (13.6) | 97.9 (99.7) | |
| 41 | 36 | 5 | ||||||||
| Diagnostic groups | Autistic versus nonautistic | 475 | 426a | 343 | 83 | 86.4 (3.4) | 81.6 (5.4) | 80.5 (1.8) | 32.5 (8.2) | 97.4 (99.5) |
| 49b | 40 | 9 | ||||||||
| Autistic + DD–LD versus NT | 475 | 328c | 267 | 61 | 71.7 (2.7) | 53.7 (3.9) | 81.4 (2.1) | 56.4 (5.8) | 79.7 (98.8) | |
| 147d | 79 | 68 | ||||||||
| DD–LD versus NT | 426 | 328c | 227 | 101 | 65.1 (3.3) | 55.1 (5.2) | 69.2 (2.6) | 34.8 (3.7) | 83.8 (98.6) | |
| 98e | 54 | 44 | ||||||||
| Autistic versus DD–LD | 426 | 49b | 10 | 39 | 83.3 (3.9) | 80.1 (6.0) | 74.6 (4.3) | 60.9 (6.2) | 88.0 (99.4) | |
| 98e | 73 | 25 |
The operating point (or positivity threshold) corresponds to the one maximizing the Youden index. PPV and NPV values were adjusted for population prevalence. Stratification by diagnosis group refers to neurotypical (NT; first row) and autistic (second row) except for the diagnostic groups category;
aNonautistic group (neurotypical + DD–LD).
bAutistic.
cNeurotypical (NT).
dAutistic + DD–LD.
eDD–LD.
Correct, number of correct diagnosis predictions; not correct, number of incorrect predictions.