Table 2.
Accuracy of the natural language processing tool CHARTextract to identify Alberta stroke program early CT score (ASPECTS) and collateral vascular status based on diagnostic imaging reports.
| Cohort and stroke-related attributes | Attribute prevalence, n (%) | Sensitivity (%) | Specificity (%) | PPVa (%) | NPVb (%) | Overall accuracy (%) | ||||
| Derivation cohort (n=921) | ||||||||||
|
|
ASPECTS | 98.8 | ||||||||
|
|
|
Not reported | 661 (71.8) | 99.7 | 99.2 | 99.7 | 99.2 |
|
||
|
|
|
<5 | 30 (3.3) | 96.7 | 99.2 | 80.6 | 99.9 |
|
||
|
|
|
≥5 | 230 (25.0) | 96.5 | 99.7 | 99.1 | 98.9 |
|
||
|
|
Collateral status | 98.4 | ||||||||
|
|
|
Not reported | 774 (84.0) | 99.2 | 96.6 | 99.4 | 95.9 |
|
||
|
|
|
Poor | 34 (3.7) | 94.1 | 100 | 100 | 99.8 |
|
||
|
|
|
Intermediate | 19 (2.1) | 78.9 | 100 | 100 | 99.6 |
|
||
|
|
|
Good | 94 (10.2) | 96.8 | 98.8 | 90.1 | 99.6 |
|
||
| Validation cohort (n=399) | ||||||||||
|
|
ASPECTS | 98.5 | ||||||||
|
|
|
Not reported | 275 (68.9) | 99.3 | 96.8 | 98.6 | 98.4 |
|
||
|
|
|
<5 | 10 (2.5) | 70.0 | 100 | 100.0 | 99.2 |
|
||
|
|
|
≥5 | 114 (28.6) | 99.1 | 99.3 | 98.3 | 99.6 |
|
||
|
|
Collateral status | 98.2 | ||||||||
|
|
|
Not reported | 330 (82.7) | 99.7 | 91.3 | 98.2 | 98.4 |
|
||
|
|
|
Poor | 15 (3.8) | 93.3 | 99.7 | 93.3 | 99.7 |
|
||
|
|
|
Intermediate | 7 (1.8) | 71.4 | 100 | 100 | 99.5 |
|
||
|
|
|
Good | 47 (11.8) | 93.6 | 100 | 100 | 99.2 |
|
||
aPPV: positive predictive value.
bNPV: negative predictive value.