Table 2. Cross-validated performance estimates for single-source and multi-source models.
| Model (#) | V-BAR (%) | T-BAR (%) | Sn (%) | Sp (%) | AUC-ROC | CCC | DOPTIMAL / Total |
|---|---|---|---|---|---|---|---|
| Single Source | |||||||
| CRF (1) | 62.0 ± 1.4 | 61.8 ± 7.7 | 65.3 ± 12.7 | 58.3 ± 11.7 | 0.61 ± 0.12 | # | 1 ± 0 / 16 |
| CAM (2) | 77.9 ± 1.4 | 76.1 ± 7.2 | 76.9 ± 9.5 | 75.3 ± 11.2 | 0.83 ± 0.07 | 0.92 ± 0.03 | 15 ± 10 / 170 |
| MRI (3) | 71.4 ± 1.6 | 69.1 ± 8.5 | 68.5 ± 11.8 | 69.6 ± 12.4 | 0.76 ± 0.09 | 0.91 ± 0.03 | 10 ± 5 / 452 |
| PPM (4) | 56.0 ± 2.7 | 53.2 ± 10.0 | 51.2 ± 12.9 | 55.3 ± 14.1 | 0.54 ± 0.11 | 0.10 ± 0.31 | 40 ± 10 / 149 |
| Multi-Source | |||||||
| CONCAT (5) | 79.7 ± 1.4 | 80.0 ± 7.3 | 80.3 ± 10.6 | 79.8 ± 10.9 | 0.86 ± 0.07 | 0.93 ± 0.02 | 10 ± 3 / 787 |
| MKL-Gaussian (6) | 80.3 ± 1.3 | 79.9 ± 6.8 | 83.4 ± 9.9 | 76.4 ± 12.3 | 0.87 ± 0.07 | 0.95 ± 0.01 | 10 ± 3 / 787 |
For each model, several measures of predictive performance are shown (mean ± standard deviation), including balanced accuracy rate on the validation set (V-BAR) and the test set (T-BAR), sensitivity (Sn), specificity (Sp), area under the curve (AUC), and concordance correlation coefficient (CCC). DOPTIMAL is the optimal number of features (shown as median ± median absolute deviation); this parameter was determined via cross-validation (see text). The total number of potential features considered when building each model is shown for reference. Performance estimates for models 7–9 are shown in S1 Table. CRF = Clinical Risk Factors, CAM = Clinical Assessments/Markers, MRI = Magnetic Resonance Imaging, PPM = Plasma Proteomic Markers. Models 1–4: single linear kernel using features only from the given data source (CRF, CAM, MRI, PPM). Model 5 (CONCAT): single linear kernel, concatenating features from all data sources. Model 6 (MKL-Gaussian): 5 Gaussian kernels using features from all data sources. # Robust estimate of CCC could not be obtained for model 1 because only <10 probability sub-intervals could be defined when conducting calibration analysis.