Table 3.
Comparative Test Characteristics for Combination of Different DTI/EDI Metrics with Machine-Learning Algorithms
| ROC AUC | Accuracy | Sensitivity | Specificity | PPV | NPV | |
|---|---|---|---|---|---|---|
| Fractional anisotropy | ||||||
| Naive Bayes | 0.66 (0.64–0.67) | 67.9% (67.4%–68.4%) | 18.8% (17.6%–20.1%) | 88.9% (88.4%–89.5%) | 61.8% (59.4%–64.2%) | 69.2% (68.9%–69.6%) |
| Random forest | 0.69 (0.68–0.70) | 69.3% (68.8%–69.8%) | 22.8% (21.4%–24.2%) | 89.3% (88.9%–89.6%) | 75.0% (73.1%–76.8%) | 69.5% (69.1%–69.9%) |
| SVM—linear kernel | 0.56 (0.54–0.58) | 60.3% (59.5%–61.1%) | 48.3% (46.7%–49.9%) | 65.4% (64.3%–66.5%) | 41.1% (39.7%–42.5%) | 73.6% (72.9%–74.3%) |
| SVM—polynomial kernel | 0.55 (0.51–0.57) | 60.0% (59.2%–60.8%) | 48.4% (46.8%–49.9) | 65.0% (63.9%–66.0%) | 41.5% (40.1%–42.8%) | 73.0% (72.3%–73.7%) |
| Neural network | 0.53 (0.51–0.57) | 58.0% (57.2%–58.7) | 33.1% (31.6%–34.5%) | 68.6% (67.7%–69.5%) | 35.2% (33.7%–36.8%) | 67.6% (67.0%–68.1%) |
| Mean diffusivity | ||||||
| Naive Bayes | 0.65 (0.64–0.67) | 66.5% (65.9%–67.0%) | 18.4% (17.2%–19.6%) | 87.0% (86.5%–87.5%) | 60.4% (58.1%–62.8%) | 67.8% (67.4%–68.1%) |
| Random forest | 0.71 (0.70–0.72) | 71.6% (71.1%–72.1%) | 23.5% (22.1%–25.0%) | 92.2% (91.8%–92.6%) | 77.4% (75.5%–79.3%) | 71.8% (71.4%–72.1%) |
| SVM—linear kernel | 0.56 (0.54–0.57) | 59.6% (58.8%–60.4%) | 47.8% (46.2%–49.4%) | 64.7% (63.6%–65.8%) | 40.7% (39.3%–42.0%) | 72.9% (72.2%–73.6%) |
| SVM—polynomial kernel | 0.55 (0.52–0.58) | 58.7% (58.0%–59.5%) | 47.3% (45.8%–48.9%) | 63.6% (62.6%–64.6%) | 40.6% (39.3%–41.9%) | 71.5% (70.8%–72.1%) |
| Neural network | 0.54 (0.52–0.56) | 59.9% (59.1%–60.6%) | 34.1% (32.6%–35.7%) | 70.9% (69.9%–71.8%) | 36.4% (34.8%–38.0%) | 69.8% (69.2%–70.3%) |
| Radial diffusivity | ||||||
| Naive Bayes | 0.66 (0.63–0.69) | 68.6% (68.1%–69.2%) | 19.0% (17.8%–20.3%) | 89.9% (89.3%–90.4%) | 62.4% (60.0%–64.8%) | 70.0% (69.6%–70.3%) |
| Random forest | 0.68 (0.66–0.70) | 70.8% (70.3%–71.3%) | 23.3% (21.9%–24.7%) | 91.2% (90.8%–91.6%) | 76.6% (74.7%–78.5%) | 71.0% (70.6%–71.4%) |
| SVM—linear kernel | 0.56 (0.53–0.58) | 58.4% (57.6%–59.1%) | 46.8% (45.2%–48.4%) | 63.3% (62.3%–64.4%) | 39.8% (38.5%–41.1%) | 71.3% (70.6%–72.0%) |
| SVM—polynomial kernel | 0.57 (0.54–0.60) | 60.6% (59.8%–61.4%) | 48.9% (47.3%–50.5%) | 65.7% (64.6%–66.7%) | 41.9% (40.6%–43.3%) | 73.8% (73.1%–74.5%) |
| Neural network | 0.54 (0.51–0.56) | 59.2% (58.5%–60.0%) | 33.8% (32.3%–35.3%) | 70.1% (69.2%–71.1%) | 36.0% (34.4%–37.6%) | 69.0% (68.5%–69.6%) |
| Edge density | ||||||
| Naive Bayes | 0.71 (0.69–0.72) | 72.2% (71.7%–72.8%) | 20.0% (18.7%–21.4%) | 94.6% (94.0%–95.2%) | 65.7% (63.2%–68.3%) | 73.7% (73.3%–74.0%) |
| Random forest | 0.75 (0.74–0.76) | 75.3% (74.8%–75.9%) | 24.8% (23.3%–26.3%) | 97.0% (96.6%–97.4%) | 81.5% (79.5%–83.5%) | 75.5% (75.1%–75.9%) |
| SVM—linear kernel | 0.59 (0.57–0.62) | 63.4% (62.6%–64.3%) | 50.8% (49.1%–52.6%) | 68.8% (67.7%–70.0%) | 43.3% (41.8%–44.7%) | 77.5% (76.8%–78.2%) |
| SVM—polynomial kernel | 0.60 (0.58–0.63) | 63.8% (63.0%–64.7%) | 51.4% (49.8%–53.1%) | 69.1% (68.0%–70.3%) | 44.1% (42.7%–45.6%) | 77.7% (76.9%–78.4%) |
| Neural network | 0.58 (0.55–0.61) | 63.0% (62.2%–63.8%) | 35.9% (34.3%–37.5%) | 74.6% (73.6%–75.6%) | 38.3% (36.6%–40.0%) | 73.5% (72.9%–74.0%) |
Detailed results for combination of different supervised machine-learning algorithms with DTI and EDI metrics for classification of children with ASD. The results are the average (95% confidence interval) performance for cross-validation of each algorithm among × 1,000 randomly selected validation samples.
AUC, area under the curve; EDI, edge density imaging; NPV, negative predictive value; PPV, positive predictive value; ROC, receiver operating characteristic; SVM, support vector machine.