Table 3.
Model accuracy, stability, and calibration data in all study cohorts.
| Model Development Cohort | Internal Validation Cohort 3–6 months | Internal Validation Cohort 12–18 months | STRIDE Cohort 3–6 months | STRIDE Cohort 12–18 months | |
|---|---|---|---|---|---|
| Sample size | 209 | 185† | 89† | 693‡ | 567‡ |
| PSCI subjects, No. (%) | 78 (37.3%) | 35 (18.9%) | 17 (19.1%) | 352 (50.8%) | 286 (50.4%) |
| Stroke to follow-up interval, mean (SD), mo | 3.66 (3.24) | 3.13 (2.38) | 16.35 (4.60) | 5.09 (1.29) | 12.68 (1.37) |
| AUROC (standard error) | 0.82 (0.03) | 0.78 (0.04) | 0.79 (0.06) | 0.75 (0.02) | 0.74 (0.02) |
| 95% confidence interval | 0.76–0.88 | 0.71–0.85 | 0.68–0.90 | 0.71–0.79 | 0.70–0.78 |
| Brier score | 0.17 | 0.13 | 0.13 | 0.20 | 0.21 |
| 10-fold cross validation range | 0.70–0.83 | 0.76–0.83 | 0.78–0.83 | 0.75–0.78 | 0.75–0.77 |
| % PSCI at: | |||||
| Score 0–4, No. (%) | 6 (7.6%) | 7 (7.1%) | 3 (7.0%) | 34 (18.6%) | 37 (23.4%) |
| Score 5–9, No. (%) | 48 (49.0%) | 21 (27.3%) | 10 (25.0%) | 229 (57.5%) | 181 (55.5%) |
| Score 10–14, No. (%) | 24 (75.0%) | 7 (70.0%) | 4 (66.7%) | 89 (79.5%) | 68 (81.9%) |
| At Score ≥ 7: | |||||
| Accuracy | 73.7% | 77.3% | 75.3% | 67.6% | 66.6% |
| Sensitivity | 74.4% | 48.6% | 52.9% | 71.5% | 68.4% |
| Specificity | 73.3% | 84.0% | 80.6% | 63.6% | 64.8% |
| Positive predictive value | 62.4% | 41.5% | 39.1% | 67.0% | 66.4% |
| Negative predictive value | 83.8% | 87.5% | 87.9% | 68.5% | 66.9% |
| Positive likelihood ratio | 2.78 | 3.04 | 2.72 | 1.97 | 1.94 |
| Negative likelihood ratio | 0.35 | 0.61 | 0.58 | 0.45 | 0.49 |
†Data from Singapore prospective cohort; 12–18 month cohorts are subsets of corresponding 3–6 month cohort.
‡Data from STRIDE cohort; 12–18 month cohorts are subsets of corresponding 3–6 month cohort.
Abbreviations: AUROC, area under receiver operating characteristics curve; IQR, interquartile range; PSCI, post-stroke cognitive impairment; STRIDE, Stroke Registry Investigating Cognitive Decline study.