Table 4.
Ten technologies indicated to diagnose ICH. The majority of studies were retrospective, and algorithm performance met or exceeded human performance in binary and multiclass classification. ICH location (e.g., under the calvaria) and anatomical variations (e.g., calcification of the falx) reduced algorithm performance. Human performance generally continues to be the gold standard for evaluating these algorithms. CPH: Cerebral Parenchymal Hemorrhage; EDH: Extradural hemorrhage; ICA: Internal Carotid Artery; IVH: Intraventricular Hemorrhage; MCA: Middle Cerebral Artery; SAH: Subarachnoid Hemorrhage. * Indicates metric was extrapolated from available data.
| Device | Author, Year | Level of Evidence | Dataset Characteristics | Sample Size (Scans) | AUC | PPV | NPV | Accuracy | Sensitivity | Specificity | Other Metrics/Comments |
|---|---|---|---|---|---|---|---|---|---|---|---|
| BriefCase | Ojeda et al., 2019 [63] | Retrospective | Proprietary, Multicenter |
7112 | - | 96% | 98% | 98% | 95% | 99% | BriefCase uses a CNN to analyze non-contrast CTs to detect and triage ICH. |
| Wismüller et al., 2020 [65] | Randomized Clinical Trial | Proprietary, Single Center |
620 | - | - | - | 96% | 95% | 97% | Turn-around times for cases flagged by BriefCase (73 min) were significantly lower than those for non-flagged cases (132 min). | |
| Ginat et al., 2020 [66] | Prospective | Proprietary, Single Center |
2011 | - | 74% | 98% | 93% | 89% | 94% | Accuracy was significantly higher for emergency (96.5%) vs. inpatient (89.4%) cases. False positives had various causes, including: (1) artifacts, (2) thick dura, (3) intra-arterial clot, (4) calcifications, and (5) tumors. | |
| Rao et al., 2021 [69] | Retrospective | Proprietary, Single Center |
5585 | - | - | - | - | - | - | When applied to scans that radiologists reported as negative for ICH, BriefCase found 28 scans with ICH, of which 16 truly did. Subset analysis showed a false positive rate of 32%. | |
| Ginat et al., 2021 [64] | Retrospective | Proprietary, Single Center |
8723 | - | 86% | 96% | - | 88% | 96% | Scan view delay for cases flagged by the software decreased by 37 min for inpatients and 604 min for outpatients. In the ER, time reduction was most prominent during the 9 p.m. to 3 a.m. and 10 a.m. to 12 p.m. periods, and especially during the weekend. | |
| Voter et al., 2021 [67] | Retrospective | Proprietary, Single Center |
3605 | - | 81% | 99% | 96% * | 92% | 98% | Neuroradiologists and the software agreed 97% of the time. Prior neurosurgery decreased model performance. | |
| Kundisch et al., 2021 [68] | Retrospective | Proprietary, Multicenter |
4946 | - | 72% * | 99% * | 97% * | 88% * | 98% * | Software detected 29 additional ICHs (0.59%) in the cohort. False negative rate was 12.4% compared to the radiologist rate of 10.9%. Anatomical variations (e.g., calcifications) were difficult for the algorithm to analyze. | |
| CINA | McLouth et al., 2021 [27] | Retrospective | Proprietary, Multicenter |
814 | - | 80–97% | 92–99% | 96% | 91% | 97% | True positive rates (sensitivity) for ICH subclassification were >90%. ICH < 5 mL had a sensitivity of 72%. |
| Rava et al., 2021 [70] | Retrospective | Proprietary, Single Center |
302 | - | 85% | 98% | 94% | 93% | 93% | 95% of ICH volumes were correctly triaged. 88% of non-ICH cases were correctly classified as ICH negative. | |
| CuraRad-ICH | Ye et al., 2019 [71] | Retrospective | Proprietary, Multicenter |
2836 | 0.8–1.0 | - | - | 75–99% | 61–99% | 82–99% | Algorithm was evaluated for binary classification (ICH vs. no ICH) and multi-type classification (CPH, SAH, EDH, SDH, IVH). |
| Guo et al., 2020 [72] | Retrospective | Proprietary, Multicenter |
1176 | 0.85–0.99 | - | - | 90–98% | 78–97% | 92–100% | Algorithm was evaluated for binary classification (ICH vs. no ICH) and multi-type classification (CPH, SAH, EDH, SDH, IVH). | |
| Rapid ICH | Heit et al., 2021 [74] | Retrospective | Proprietary, Multicenter |
308 | - | 96% | 95% | 95% * | 96% | 95% | |
| HealthICH | Bar et al., 2018 [76] | Retrospective | Proprietary, Multicenter |
1426 | 0.96 | - | - | - | - | - | |
| Accipiolx | |||||||||||
| DeepCT | |||||||||||
| NinesAI | |||||||||||
| QER | |||||||||||
| Viz ICH |