Table 2.
Fusion model performance.
| COG | COGNC | COGMCI | COGDE | ADD | 4-way | |
|---|---|---|---|---|---|---|
| NACC | ||||||
| Accuracy | 0.804 ± 0.011 [0.790–0.818] | 0.872 ± 0.009 [0.861–0.883] | 0.809 ± 0.011 [0.795–0.823] | 0.926 ± 0.004 [0.921–0.931] | 0.847 ± 0.013 [0.831–0.863] | 0.777 ± 0.011 [0.763–0.791] |
| F-1 | 0.772 ± 0.012 [0.757–0.787] | 0.880 ± 0.009 [0.869–0.891] | 0.592 ± 0.023 [0.563–0.621] | 0.843 ± 0.010 [0.831–0.855] | 0.914 ± 0.008 [0.904–0.924] | 0.601 ± 0.017 [0.580–0.622] |
| Sensitivity | 0.771 ± 0.013 [0.755–0.787] | 0.893 ± 0.020 [0.868–0.918] | 0.569 ± 0.046 [0.512–0.626] | 0.851 ± 0.025 [0.820–0.882] | 0.968 ± 0.019 [0.944–0.992] | 0.602 ± 0.013 [0.586–0.618] |
| Specificity | 0.895 ± 0.006 [0.888–0.902] | 0.850 ± 0.021 [0.824–0.876] | 0.887 ± 0.022 [0.860–0.914] | 0.949 ± 0.007 [0.940–0.958] | 0.189 ± 0.074 [0.097–0.281] | 0.914 ± 0.005 [0.908–0.920] |
| MCC | 0.670 ± 0.018 [0.648–0.692] | 0.744 ± 0.018 [0.722–0.766] | 0.470 ± 0.027 [0.436–0.504] | 0.796 ± 0.012 [0.781–0.811] | 0.249 ± 0.097 [0.129–0.369] | 0.534 ± 0.025 [0.503–0.565] |
| OASIS | ||||||
| Accuracy | 0.754 ± 0.047 [0.696–0.812] | 0.852 ± 0.037 [0.806–0.898] | 0.765 ± 0.050 [0.702–0.827] | 0.890 ± 0.006 [0.883–0.898] | 0.879 ± 0.013 [0.863–0.895] | 0.730 ± 0.045 [0.675–0.786] |
| F-1 | 0.610 ± 0.023 [0.581–0.638] | 0.873 ± 0.037 [0.827–0.919] | 0.156 ± 0.024 [0.126–0.186] | 0.800 ± 0.015 [0.781–0.818] | 0.935 ± 0.008 [0.925–0.945] | 0.468 ± 0.020 [0.443–0.493] |
| Sensitivity | 0.670 ± 0.025 [0.639–0.701] | 0.807 ± 0.066 [0.725–0.888] | 0.526 ± 0.092 [0.412–0.640] | 0.678 ± 0.025 [0.648–0.709] | 0.965 ± 0.023 [0.936–0.993] | 0.514 ± 0.010 [0.501–0.527] |
| Specificity | 0.900 ± 0.013 [0.883–0.916] | 0.932 ± 0.016 [0.912–0.952] | 0.775 ± 0.055 [0.707–0.843] | 0.992 ± 0.003 [0.988–0.995] | 0.127 ± 0.083 [0.024–0.231] | 0.917 ± 0.009 [0.906–0.929] |
| MCC | 0.536 ± 0.029 [0.500–0.573] | 0.716 ± 0.055 [0.648–0.785] | 0.142 ± 0.034 [0.099–0.185] | 0.750 ± 0.014 [0.734–0.767] | 0.128 ± 0.060 [0.053–0.204] | 0.411 ± 0.022 [0.383–0.438] |
Performance of the fusion (CNN + CatBoost) model on the NACC test set and the OASIS cohort is shown. For each model, we reported accuracy, sensitivity, specificity, F-1 score, and Matthew’s Correlation Coefficient (MCC) for all diagnostic tasks. In both datasets, we report classification performance on the overall COG task (i.e., the full classification of NC, MCI, and DE cases) as well as classification performance for constituent subtasks, including the binary classification of NC vs. non-NC (COGNC column), MCI vs. non-MCI (COGMCI column) and DE vs. non-DE (COGDE column). We also report metrics for the ADD task (i.e., the binary classification of AD and nADD following internal designation of DE in the COG task). By inferring the COG task first and the ADD task second, the model generated a 4-way classification of NC, MCI, AD, nADD (4-way column).