Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Dec 1.
Published in final edited form as: Stroke. 2019 Nov 4;50(12):3431–3438. doi: 10.1161/STROKEAHA.119.027076

Fast Automatic Detection Of Large Vessel Occlusions On CT Angiography

Shalini A Amukotuwa 1,2,3, Matus Straka 3, Seena Dehkharghani 4,5, Roland Bammer 3,6,7,§
PMCID: PMC6878187  NIHMSID: NIHMS1540897  PMID: 31679501

Abstract

Background and Purpose:

Accurate and rapid detection of anterior circulation large vessel occlusion (LVO) is of paramount importance in acute stroke patients due to the potentially rapid consumption of at-risk tissue and the limited therapeutic window for endovascular clot retrieval (ECR). Hence, the optimal threshold of a new, fully automated software-based approach for LVO detection was determined and its diagnostic performance evaluated in a large cohort study.

Methods:

For this retrospective study, data were pooled from: two stroke trials, DEFUSE 2 (n=62; 07/08–09/11) and 3 (n=213; 05/17–05/18); a cohort of ECR candidates (n=82; 08/02/14 – 08/30/15) and normals (n=111; 06/06/17 – 01/28/19) from a single quaternary center; and ‘code stroke’-patients (n=501; 01/01/17 – 12/31/18) from a single regional hospital. All CTAs were assessed by the automated algorithm. Consensus reads by two neuroradiologists served as the reference standard. ROC analysis was used to assess diagnostic performance of the algorithm for detection of: 1.) anterior circulation LVOs involving the intracranial internal carotid artery (ICA) or M1 segment middle cerebral artery (M1-MCA); 2.) anterior circulation LVOs and proximal M2 segment middle cerebral artery (M2-MCA) occlusions and; 3.) individual segment occlusions.

Results:

CTAs from 926 patients (median age 70 years IQR: 58–80; 422 females) were analyzed. 395 patients had an anterior circulation occlusion LVO or M2-MCA occlusion (NIHSS 14 [median], IQR: 9–19). Sensitivity and specificity were 97% and 74%, respectively, for LVO detection, and 95% and 79%, respectively, when M2 occlusions were included. On analysis by occlusion site, sensitivities were 90% (M2-MCA), 97% (M1-MCA), and 97% (intracranial ICA) with corresponding AUCs of 0.874 (M2), 0.962 (M1), and 0.997 (intracranial ICA).

Conclusions:

Intracranial anterior circulation LVOs and proximal M2 occlusions can be rapidly and reliably detected by an automated detection tool, which may facilitate intra- and inter-instutional workflows and emergent imaging triage in the care of stroke patients.

Keywords: Stroke, thrombectomy, computed tomography angiography, large vessel occlusion, reperfusion

INTRODUCTION

Large vessel occlusions (LVOs) of the anterior circulation contribute disproportionately to stroke-related dependence and death1. Evidence from recent thrombectomy trials has shown that patients with LVOs have a substantially better clinical outcome after successful endovascular clot retrieval (ECR) than those who were managed medically, provided that they were treated within 24h of the time last seen normal27. These practice-altering studies required having a positive finding of an LVO on vascular imaging and therefore clinical guidelines8 stipulate emergent angiographic imaging (predominantly CTA).

The extended time window in which qualifying patients are eligible for ECR has led to a paradigm shift in stroke care but as a knock-on effect poses substantial logistical challenges for radiology: a 24h-window allows patients to be transferred even from very remote hospitals to ECR centers and still be eligible for treatment. In turn, this mandates that peripheral hospitals can reliably identify LVOs 24/7 with fast report turn-around to expedite treatment decisions. This requirement can pose a challenge for many smaller institutions, given their limited neuroradiology staffing and teleradiology coverage. The detection of an LVO is relatively straightforward for radiologists9, 10. However, the CTA of a patient with an LVO may not always end up at the top of a worklist and it may therefore be overlooked, with considerable medico-legal implications. An automated tool that draws attention to a positive finding would therefore help avoid situations where emergent CTAs are buried in a worklist and positive findings are communicated to the care providers with substantial delay. Even at an ECR hub, the ability to screen CTAs for the presence of LVOs (especially after hours) would help workflow and staffing, and facilitate rapid mobilization of the stroke and interventional neuroradiology teams by alerting them of a positive finding. Aside from optimizing patient care, there are looming financial implications, especially in light of the 2019 Centers for Medicare & Medicaid guideline for payment determination requiring scan interpretation within 45 minutes of ED arrival11.

With these factors in mind, the goal for the new algorithm presented here was to automatically screen patients’ CTAs for the presence of an LVO and alert the reporting radiologist and the stroke team of a positive finding. We hypothesized that an algorithm could be developed that detects intracranial LVOs with almost perfect sensitivity and acceptable false positives. The purpose of this study was to evaluate the diagnostic performance of this algorithm against the reference standard of reads from experienced neuroradiologists in a large cohort study.

Material and Methods

This retrospective study was approved by the IRBs of the participating regional and quaternary hospitals, who waived the requirement for informed consent. The data that support the findings of this study are available from the corresponding author on reasonable request.

a. Patient Selection

A total of 969 patients were included in this retrospective study (Figure. I in the online-only Supplement). The patient population comprised five individual cohorts which constituted a well-represented sample of scanner models from all major CT vendors and typical variants of CTA protocols seen at hospitals: 275 patients pooled from DEFUSE 2 (n=62) (07/08–09/11) and DEFUSE 3 (n=213) (05/17–05/18), two large multi-center stroke trials7, 12; 193 patients came from a single quaternary center of which 82 were patients who had been imaged as potential ECR candidates (08/02/14–08/30/15), and 111 were imaged for non-stroke related indications (06/06/17–01/28/19) with normal anterior circulation. The fifth cohort was a consecutive series of 501 patients who had CTA as part of a ‘code stroke’ work up at a regional hospital that is a primary stroke center (01/01/17–12/31/18). Note that for DEFUSE 2 and 3, only the subset of consented patients who had undergone acute CTA were used.

43 patients (4.4%) were excluded due to: 1. screen failures (n=4, from DEFUSE 2); 2. CTA not being included in the acute CT protocol (n = 7); 3. inadequate data format (thin slice CTA raw data unavailable); and 4. the CTA being deemed by an experienced neuroradiologist (S.A.A) to be technically inadequate therefore of insufficient quality to allow accurate interpretation by a human reader (n = 15 with severe motion in 3, poor/no contrast bolus in 8 and incomplete coverage of the intracranial arteries in 4).

The remaining 926 patients (age 70 [median] IQR: 58–80 years) were analyzed, of which 504 were female (age 69 IQR: 58–78) and 422 male (age 71, IQR: 59–82) (Table 1). 531 of these patients, who were imaged for a diagnostic workup of their cervico-cerebral vasculature, had either no evidence of an anterior circulation vessel occlusion or distal (M3/M4 segments) occlusions only, and for this study were considered controls. Based on CTA expert reads, the remaining 395 patients had an occlusion in the anterior circulation at the following location:

  1. Single site (n=241): cervical ICA (n=15); intracranial ICA (n=16); M1-MCA (n=161); M2-MCA (n=37); and distal MCA (n=12).

  2. Tandem/multiple lesions (n=154): any ICA+M1 (n=124); any ICA+M2 (n=8); M1+M2 (n=5); cervical ICA+intracranial ICA (n=9); and M2+distal MCA (n=8).

Table 1:

Patient Demographics

Male Female Male Female
No. of
Patients
D2 D3 A1 A2 No. of
Males
Age
median: IQR years
No. of
Females
Age
median: IQR years
p- value NIHSS
median: IQR
NIHSS
median: IQR
p- value
All Patients 926 61 210 178 477 504 69: 58–78 422 71: 59–82 0.0793 7: 2–14 7: 2–14 0.1196
No Occlusion 531 1 26 138 369 302 69: 56–77 229 67: 54–80 0.7627 3: 0–7 3: 0–7 0.5402
Anterior Circulation Occlusion 395 60 184 43 108 202 70: 59–79 193 72: 65–83 *0.0026 14: 9–20 14: 10–19 0.9614
Isolated Occlusions 241 44 88 27 82 110 71: 60–81 131 73: 66–82 0.0585 13: 7–19 14: 9–19 0.1443
 Cervical ICA 15 5 0 2 8
 Intracranial ICA 16 0 0 0 16
 M1-MCA 161 33 84 8 36
 M2-MCA 37 6 3 6 22
 M3/M4 MCA 12 0 1 11 0
Tandem Occlusions§ 154 16 96 16 26 92 69: 59–76 62 77: 63–84 0.0192 17: 12–21 14: 12–18 0.2526
 Cervical ICA + M1 21 1 13 1 6
 Intracranial ICA + M1 103 10 72 10 11
 M1-MCA + M2-MCA 5 0 4 1 3
 M2-MCA + M3/M4 MCA 8 0 1 0 4
 Cervical + Intracranial ICA 9 3 3 1 2
 Any ICA + M2-MCA 8 2 3 3 0

Note. – IQR = Interquartile Range. D2 = DEFUSE 2 cohort. D3 = DEFUSE 3 cohort. A1 = ECR cohort + controls from quaternary hospital. A2 = ‘Code Stroke’ cohort from regional tertiary hospital.

§

only combinations of vessel segments are listed where a tandem occlusion was found.

t-test to compare age distributions between male and female

t-test to compare NIHSS distributions between male and female

*

denotes significance (p< 0.05).

£

NIHSS was only available for 847 patients who were imaged as part of their acute stroke diagnostic work-up.

Of those 395 patients with occluded vessels (Figure 1), 15 patients had isolated cervical ICA occlusions and 60 had M2-MCA occlusions without any intracranial LVOs. Of the remaining 320 patients with intracranial LVOs, 16 had isolated intracranial ICA, 161 had isolated M1-MCA occlusions, and 143 had tandem/multiple occlusions: M1+M2 (n=5); cervical ICA+M1 (n=21); intracranial ICA+M1 (n=103); intracranial ICA+M2 (n=5); and intracranial and cervical ICA (n=9).

Figure 1 – Pictorial Description of Automatic LVO Detection Algorithm.

Figure 1 –

After (1) importing raw, thin-slice CTA DICOM images, (2) only slices above C1 are used for further processing and the CT head holder is removed. A (3) CT head template is then co-registered to the patient’s CTA and subsequently the CTA analysis regions (which were previously defined on the CT template) are spatially transformed onto the patient’s CTA scan. Next, all bone is removed (4). Tubular filters are applied (5) to extract vessels. Then, (6) the density (in Hounsfield units) sum of all voxels constituting the large vessels and the density sum of all voxels constituting distal vessels are computed and (7) hemispheric comparisons are made. (8) Areas where the vessel density sum drops below prespecified threshold are highlight as color overlays on maximum intensity projections.

b. Reference standard

For patients enrolled in DEFUSE 2 and 3, the presence and exact location of an occlusive lesion had been previously determined by the investigators, and was verified by a neuroradiologist (S.A.A.) with 8 years post fellowship experience. For the remaining patients, two neuroradiologists (S.A.A and S.D., with 9 years post fellowship experience) evaluated the multimodal CTs including CTA for the presence and site of an occlusive lesion, in consensus, with all clinical and imaging data (including perfusion imaging) available for review. Any disagreements were resolved by review of all available imaging for the patient, including perfusion. These neuroradiologist reads served as the reference standard against which the diagnostic performance of the algorithm was assessed.

c. Algorithm Description

The underlying concept of the LVO detection presented here relies on software that performs elastic registration of three pre-specified anatomic assessment regions (R1, R2, and R3)13 onto and then tubular filtering14 of CTAs to detect reduced opacification of anterior intracranial vessels relative to the contralateral hemisphere (Figure 1). This algorithm was implemented into RAPID 4.9.1 (iSchemaView, Menlo Park, CA) and ran on a conventional computer environment (2x Intel Xeon E5–2680 2.7 GHz CPUs with 8 cores and hyperthreading each, 64 GB RAM, CentOS 7 Linux). The algorithm used in this study has received Conformité Européenne labeling and has been cleared by the US Food and Drug Administration. It was used as provided by the vendor, without any modification or any further pre- or postprocessing. Relative vessel density thresholds for LVO detection can be chosen arbitrarily by the user but for this study the software’s default values were used: <80%–75% (BLUE), <75%–60% (GREEN), <60%–45% (YELLOW), and <45% (RED). For details of the algorithm, including definitions of the R1, R2, and R3 regions, the interested reader is referred to the Supplemental Material.

d. Statistical Analysis

The primary outcome was the diagnostic performance of the algorithm for detecting intracranial LVOs. Sensitivity and specificity for detecting an intracranial LVO was assessed using ROC analysis. Specifically, the algorithm’s ability to detect the presence of CT angiographic signs of an LVO – as indicated by absence or severe reduction of arterial opacification – was assessed for the intracranial ICA and the M1-MCA. The assessment of diagnostic performance was then repeated with proximal M2-MCA occlusions added to the LVO group.

Bootstrap analysis (1,000 repeats) was used to compute 95% confidence intervals for all parameters. The Area-under-the-ROC-curve (AUC) was used in conjunction with the DeLong algorithm for calculating the Standard Error of the AUC15. As the software is primarily intended to be used as a screening tool, a diagnostic sensitivity of ≥95% was made a requirement.

Secondary outcomes were diagnostic performance of the algorithm for detecting LVOs in specific vessel segments and the processing speed. The algorithm’s diagnostic performance was evaluated for detecting occlusions at the following sub-sites: 1. Intracranial ICA (including terminal ICA); 2. M1-MCA; and 3. proximal M2-MCA. For analysis of each occlusion site, occlusions at the other two subsites were excluded.

All statistical testing was performed using MedCalc (MedCalc Version 17.2, MedCalc Software, Ostend, Belgium, 2017). An α level of 0.05 was used to indicate significance for all tests.

Results

Representative examples of automatic lesion detection in four patients with intracranial LVOs are shown in Figure 2. For the 926 cases that were processed, the median turn-around time, i.e. from start of data transmission to receipt of results, was 158 sec (IQR: 140–176 sec) of which the elastic registration was the most time-consuming step (approx. 130 sec).

Figure 2 – Example results for automatic LVO detection.

Figure 2 –

A) 65-yo male with distal ICA occlusion (open arrows) and occlusion of A1-ACA segment (curved arrow) with partial reconstitution of through collaterals (arrow). The area of severe vascular density reduction as determined by the algorithm is shown in red. B) 72-yo female with a left proximal M1-MCA occlusion (open arrows). The area of abnormal density found by the software is highlighted in red. C) 84-yo male with a distal M1 occlusion on the left (open arrows) and the area of abnormal vessel density in red. D) 55-yo male with occlusion of the proximal left superior M2 division (open arrows) and corresponding region picked by the software. The degree of vessel density reduction was less than in the other three patients.

a. Intracranial LVOs

Intracranial LVOs:

320 patients had an intracranial anterior circulation LVO while the remaining 588 did not (Table 2). The automatic algorithm yielded an AUC of 0.941 (95%CI: 0.926–0.957). The sensitivity target of ≥95% was achieved at a <75%–60% (GREEN) threshold, yielding a sensitivity of 96.87% (310/320) (95%CI: 94.3%–98.5%) and specificity of 74.32% (437/588) (95%CI: 70.6%–77.8%) (Figure 3A).

Table 2:

Algorithm’s Diagnostic Performance for Intracranial LVOs

N
(%)
AUC
(95% CI)
Target Sens
(%)
(95%CI in %)
Target Spec
(%)
(95%CI in %)
Threshold
(range)
‘COLOR’
Jmax Sensitivity
(%)
(95%CI in %)
Jmax Specificity
(%)
(95%CI in %)
Jmax
(95%CI)
(threshold range)
‘COLOR’
Intracranial
LVOs
320
(35.24)
0.941
(0.926–0.957)
310/320
(96.87)
94.3–98.5
437/588
(74.32)
70.6–77.8
<75%–60%
GREEN
264/320
(82.50)
77.9–86.5
560/588
(95.24)
93.2–96.8
0.7774
0.7352–0.8160
<45%
RED
No LVO 588
(64.76)
Intracranial
LVOs, incl.
isolated M2s
368
(40.40)
0.947
(0.933–0.962)
351/368
(95.38)
92.7–97.3
431/543
(79.37)
75.7–82.7
<75%–60%
GREEN
334/368
(90.76)
87.3–93.5
469/543
(86.37)
83.2–89.1
0.7713
0.7322–0.8117
<60%–45%
YELLOW
No LVO 543
(59.60)

Note. – ‘Target Sens’ is associated with ‘Threshold’ that yields ≥95% sensitivity at the highest specificity (‘Target Spec’).

Jmax is the maximum Youden index (sensitivity + specifity – 1) across all thresholds. Jmax provides and optimality criterion with equal weighting for sensitivity and specifity and serves as a secondary reference point.

The DeLong algorithm was used to compute the Standard Error of the AUC.

Figure 3 – ROC Analysis.

Figure 3 –

ROC curves for detection of all intracranial LVOs (A) and all intracranial LVOs and proximal M2-MCA segment occlusions (B). Dots on the ROC curve indicate individual threshold levels; the one with the lowest sensitivity and highest specifity is the <45% threshold whereas the highest sensitivity and lowest specificity were at the <80%–75% threshold. The open circle indicates the maximum Youden index. The asterisks indicate the threshold at with the ≥95% sensitivity target was reached with the highest specificity. The significance level in the legend indicates the p-value of the z-statistic derived from the DeLong algorithm.

Intracranial LVOs + M2-MCA occlusions:

368 patients had an anterior circulation LVO or a proximal M2-MCA occlusion, while 543 patients did not. The target sensitivity of ≥95% was met at the <75%–60% (GREEN) threshold, which yielded a sensitivity of (351/368) 95.38% (95%CI: 92.7%–97.3%) and specificity of (431/543) 79.37% (95%CI: 75.7%–82.7%). The overall diagnostic performance – as measured by an AUC of 0.947 (95%CI: 0.933–0.962) – improved slightly by adding M2 segments; this was due primarily to improved specificity with only a minor (1.49%) drop in sensitivity. (Figure 3B). For more details on the ROC analysis, the interested reader is referred to Table I (online-only Supplement).

b. Individual Vessel Segments

For this sub-analysis, the 531 patients who had no LVO or M2-MCA occlusion served as controls.

Intracranial ICA (including ICA terminus):

133 patients had an intracranial ICA occlusion. The algorithm yielded an AUC of 0.977 (95%CI: 0.965–0.989). The ≥95% sensitivity target was achieved with the <60%–45% (YELLOW) threshold that yielded a very high sensitivity of 96.99% (129/133) (95%CI: 92.5%–99.2%) at a specificity of 86.44% (459/531) (95%CI: 83.2%–89.2%) (Figure IIA online-only Supplement) (Table 3).

Table 3:

Algorithm’s Diagnostic Performance for Individual Vessel Segments

No.
occlusions
(%)
No. controls
AUC
(95% CI)
Target Sens
(%)
(95%CI in %)
Target Spec
(%)
(95%CI in %)
Threshold
(range)
‘COLOR’
Jmax Sensitivity
(%)
(95%CI in %)
Jmax Specificity
(%)
(95%CI in %)
Jmax
(95%CI)
(threshold range)
‘COLOR’
Intracranial
ICA
incl. terminal
ICA
133
(20.03)
531
0.977
(0.965–0.989)
129/133
(96.99)
92.5–99.2
459/531
(86.44)
83.2–89.2
<60%–45%
YELLOW
118/133
(88.72)
82.1–93.5
521/531
(98.12)
96.6–99.1
0.8684
0.8174–0.9153
<45%
RED
M1-MCA 290
(35.32)
531
0.962
(0.948–0.976)
281/290
(96.90)
94.2–98.6
423/531
(79.66)
76.0–83.0
<75%–60%
GREEN
243/290
(83.79)
79.0–87.8
521/531
(98.12)
96.6–99.1
0.8191
0.7724–0.8494
<45%
RED
M2-MCA 60
(10.15)
531
0.874
(0.826–0.921)
54/60
(90.00)*
79.5–96.2
398/531
(74.95)
71.0–78.6
<80%–75%
BLUE
52/60
(86.67)
75.4–94.1
423/531
(79.66)
76.0–83.0
0.6633
0.5479–0.7327
<75%–60%
GREEN

Note. – ‘Target Sens’ is associated with ‘Threshold’ that yields ≥95% sensitivity at the highest specificity (‘Target Spec’).

Jmax is the maximum Youden index (sensitivity + specifity − 1) across all thresholds.

The DeLong algorithm was used to compute the Standard Error of the AUC.

*

For M2-MCA segments the ≥95% sensitivity target could not be met.

M1-MCA:

290 patients had an M1-MCA occlusion. The algorithm yielded an AUC of 0.962 (95%CI: 0.948–0.976). Although the diagnostic performance as measured by AUCs was slightly inferior for detection of M1-MCA occlusions compared to intracranial ICA occlusions. The ≥95% sensitivity target was met at the <75%–60% (GREEN) threshold, which yielded a sensitivity of 96.90% (281/290) (95%CI: 94.2–98.6) and specificity of 79.66% (423/531) (95%CI: 76.0%–83.0%) (Figure IIB online-only Supplement).

M2-MCA:

60 patients had a proximal M2-MCA segment occlusion. The automated algorithm performed slightly worse than for detection of the M1-MCA segment occlusions, nevertheless yielding an AUC of 0.874 (95%CI: 0.826–0.921). The ≥95% sensitivity target could not be reached at any threshold but at the <80%–75% (BLUE) threshold a sensitivity and specificity of 90.00% (54/60) (95%CI: 79.5%–96.2%) and 74.95% (398/531) (95%CI: 71.0–78.6), respectively, was achieved (Figure IIC online-only Supplement). For more details on the ROC analysis for individual vessel segments, the interested reader is referred to Table II (online-only Supplement).

c. False Negatives

The number of false negative, where an intracranial LVO was not detected even at the most sensitive threshold (<80%–75%; BLUE), were relatively small (n=14): 1 intracranial ICA, 8 M1-MCA occlusion and 5 proximal M2-MCA occlusions.

For the 8 M1-MCA lesions, there were 3 short-segment or incomplete occlusions with reconstitution of flow immediately distal to the occlusion. Here, trickle flow across an incomplete occlusion or retrograde filling via leptomeningeal collaterals led to normal or increased ipsilateral vessel density. The 5 remaining were mid-to-distal M1-MCA occlusions distal to the R1 region level (Figure 1) with robust leptomeningal collaterals reconstituting the M2-MCA segments, resulting in normal or increased vessel density in the ipsilateral R2 and R3 regions. (Figure III in the online-only Supplement provide examples of these false negative cases). The only intracranial ICA occlusion that was missed was at the skullbase, with normal opacificiation of the supraclinoid ICA (Figure IVa, online-only Supplement)

5 M2-MCA occlusions were missed: one occlusion was located in the upper half of the Sylvian cistern, which was not covered by the R2 template; two were occluded non-dominant proximal superior M2-MCA branches; two were short-segment proximal occlusions of their inferior M2 divisions with reconstitution immediately distal to the occlusion (Figure IVb, online-only Supplement), indicating robust leptomeningeal collaterals. When proximal M2-MCA occlusions were included in the group, in addition to intracanial LVOs, the total number of false negatives decreased from 14 to 9.

d. False Positives

There were 11 false positives for LVO detection at the most specific threshold (RED, where there was a marked inter-hemispheric vessel density difference of <45%). These were attributed to: 1. substantial inter-hemispheric variation in MCA anatomy (n=4) or fetal-origin of the posterior cebebral artery (PCA, n=1); 2. holohemispheric subdural hematoma with mass effect resulting in 17mm midline shift (n=1); 3. an 8mm distal MCA aneurysm; 4. M2-MCA stenosis (n=3); and 5. incomplete recanalization (TICI 2b) after a mechanical thrombectomy attempt 24 hours prior to the CTA (n=1). There were an additional 17 false positives for LVOs detection when M2-MCAs were not included in the LVO group; all 17 were proximal M2-MCA occlusions.

False positives at the <80%–75% (BLUE), <75%–60% (GREEN) and <60%–45% (YELLOW) thresholds were due to: anatomical variation in M1-MCA branching patterns and vessel calibers, fetal origin of the PCAs and other vascular pathology, such as ipsilateral segmental flow reduction in chronic steno-occlusive disease and contralateral increase in blood flow due to reactive hyperaemia (eg due to reperfusion of an infarct or seizures in a patient with a glioblastoma). These examples can be found in the online-only supplement (Figures V and VI).

Discussion

This study evaluated a new algorithm for automated detection of intracranial anterior circulation LVOs and demonstrated that it has excellent diagnostic sensitivity and high specificity. The short processing time (<160 sec) makes its application feasible in the emergent clinical setting.

Previous studies have shown that neuroradiologists can detect LVOs with 89–98% sensitivity and 95–98% specificity9, 16. Automation, which does not achieve this high specificity, cannot replace radiologists; rather, its strength and utility lie in the high sensitivity, which allows expedited diagnosis of LVOs by flagging and prioritizing these scans as requiring urgent radiologist review. A very high sensitivity is a requirement for a screening tool. The algorithm met the targeted sensitivity of ≥95% for the detection of any intracranial LVO. This is comparable to that of experienced neuroradiologists, whose sensitivity for detecting LVOs is high but imperfect, reported to be 90% for detection of ICA occlusions in one study16. Sensitivity is lower for readers with less experience in interpreting cranial CTA, such as general radiologists and trainees, with sensitivity as low as 63% in one study17.

At many centers around the world, trainees are the first to interpret multimodal stroke CTs, which are subsequently formally read by a neuroradiologist. Further, not all hospitals and healthcare services around the world have access to around-the-clock neuroradiology expertise. At some primary stroke centers and community hospitals, such as that from which cohort 5 was drawn, general radiologists interpret multi–modal stroke CTs and there is typically only one on-call radiologist or resident after-hours due to limited resources. Acute stroke scans may be overlooked in this setting when other emergent scans such as trauma are given priority. These factors can contribute to delayed and missed diagnosis of LVOs in the authors’ experience. The fully automated algorithm is likely to be both a valuable diagnostic aide and screening tool in these settings. It can expedite the correct diagnosis by bringing positive findings to the reporting radiologist’s or resident’s attention. It can also facilitate notification of the stroke team and neurointerventionalist, allowing mobilization of the clot retrieval team which in turn would expedite treatment of eligible patients. Another important consideration is that the algorithm provides consistency, in contrast to the surprisingly poor inter-rater agreement between human readers18.

For individual segments, an almost perfect sensitivity was achieved for occlusion of the intracranial ICA or M1-MCA; sensitivity for detection of proximal M2-MCAs was slightly lower. This was attributable to false negatives resulting from short segment (where collaterals reconstituted the M2 segment immediately distal to the occlusion) and incomplete (with antegrade flow) occlusions, where the interhemispheric vessel density reduction was too small for the algorithm to detect. It is thought that robust collaterals confer a longer time-window for treatment19, 20. On post hoc analysis, the algorithm did not miss an LVO in any patient with poor collaterals; these patients are likely to be “fast progressors”, in whom expeditious reperfusion is imperative for tissue salvage19, 20. It is important that radiologists and neurologists are cognizant of the presence and causes of false negatives, and a negative result should not dissuade thorough and careful evaluating the CTA as soon as practicable.

The algorithm’s overall specificity was >74% for intracranial LVO detection and >79% when M2-MCAs were included. For individual segments, specificity was 75% for detection of M2-MCA occlusions, increasing to 80% for M1-MCA and 86% for intracranial ICA occlusions. The justification for including M2-MCA occlusions in the LVO detection algorithm is that they are now a subject of interest as a mainstream target for ECR. Thrombectomy may improve outcomes compared to standard medical management in patients with M2-MCA occlusions21. Detection of M2-MCA occlusions was, however, more challenging due to the greater anatomic variability and smaller caliber of these vessels. Moreover, the version of the software used for this study constrained the region of interest, in which vessel density was determined, to cover only the proximal half of the M2 segments, to the mid-point of the Sylvian cistern; future implementations will expand the region further distally.

The LVO detection tool evaluated in this study had three regions, R1-R3, and four different thresholds for LVO detection, reflecting lesion location and increasing severity of vessel density reduction. The diagnostic sensitivity decreased while specificity increased when the threshold was changed from <80% to <45% vessel density reduction. The (<75%–60%) threshold was found to be optimal; it yielded the desired sensitivity of ≥95% with an acceptable specificity between 70%–80%. For radiologists, this would still substantially decrease the number of scans that require emergent review. If, however, fewer ‘false LVO alerts’ are desired, this can be achieved simply by moving along the ROC curve and trading the target >95% sensitivity for increased specificity.

To our knowledge, this is the first peer-reviewed publication which has introduced an automated LVO detection tool and evaluated its performance in a multicenter study that incorporates a large and diverse cohort of patients. A few conference abstracts have been published recently related to this topic yet unrelated to this algorithm2224. The number and mix of patients enrolled and the results reported in these abstracts vary widely, with a sensitivity for LVO detection of 90%–97% and a broad specificity range of 52%–83%.

A strength of this pooled cohort study is that we included patients who were enrolled in two high-profile multicenter stroke trials7, 12. This provided validation of the algorithm in a preselected cohort of acute ischemic stroke patients with an LVO who were considered thrombectomy candidates. The large number of patients with LVOs allowed robust testing of diagnostic sensitivity. The other large cohort consisted of consecutive patients presenting to a regional hospital with a suspected acute ischemic stroke. Inclusion of this cohort of “all comers” allowed testing of the algorithm on a broad spectrum of stroke mimicks and ensures broad applicability of our findings to the population of patients in whom LVO detection tools will most likely be used. Inclusion of multiple patient cohorts from different sites in this study allowed testing and validation of the algorithm on different makes and models of CT scanners and CTA protocols.

This study has a few pertinent limitations. First, it is a retrospective study. As such, we did not have complete datasets for all patients, particularly with regard to clinical information regarding long-term outcomes and clinical scores. A prospective study that includes at least one comprehensive stroke center (hub) and several peripheral and regional/community hospital (spokes) is required to test whether the tool can be used to streamline intra- and inter-institutional workflows. Limitations related to the algorithm itself include the processing requirement for thin slice CTA raw data and arterial opacification. Thin-slice CTA data is routinely acquired even on older generation multiuslice CT scanners, which may still be in use at some centers. The requirement for contrast opacification of the intracranial arteries also applies to the human reader. There were 15/969 (1.5%) patients in whom arterial opacificiation on CTA was deemed either absent or too poor to allow accurate interpretation by an experienced human reader. 8 of these cases were from cohort 5, a regional hospital. The proportion of technically inadequate studies may be higher at smaller community hospitals where the volume of CTAs performed is smaller and technologist staff are therefore less experienced. All cases where the arterial opacification was deemed to be sufficient to allow human interpretation were successfully processed by the algorithm.

It is noted that the algorithm does not directly detect the clot but rather the resultant loss of vessel opacification, therefore false positives result from chronic occlusions. The purpose of this software is to serve as a triage tool that alerts radiologists to a patient with a potential LVO, and in turn trigger evaluation of the patient’s multimodal CT by the human reader who can then use all available information (not just the CTA) to make a judgement call. Precise localization of the occlusion site and differentiation of chronic occlusions by the algorithm is therefore not critical and was hence not evaluated in this study.

In summary, intracranial LVOs within the anterior circulation – inclusive of proximal M2-MCA occlusions – can be detected effectively and efficiently by an automated computerized screening tool. Future prospective studies may be warranted to determine whether this tool can be used to improve workflow and expedite treatment.

Supplementary Material

Supplemental Material

Acknowledgments

Sources of Funding

The study was funded by grants from the National Insitutes of Health: 1R01EB002711, 1R01NS039325, and 1U10NS086487.

Dr Bammer is a shareholder of iSchemaView, which produces the RAPID CTA software, and HobbitView Inc, an entity unrelated to this work. Dr. Straka is also an iSchemaView shareholder and receives salary support. He has a patent “AIF Selection Algorithm”. Drs Bammer and Straka have a patent pending for “Automated LVO detection on CTA”. Their role in this study was strictly limited to software development and support and statistical tests.

Footnotes

Disclosures

Dr Dehkharghani is a consultant to iSchemaview. Dr Amukotuwa receives no financial support and has no equity in iSchemaview.

References

  • 1.Malhotra K, Gornbein J, Saver JL. Ischemic strokes due to large-vessel occlusions contribute disproportionately to stroke-related dependence and death: A review. Front Neurol 2017;8:651. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Saver JL, Goyal M, Bonafe A, Diener HC, Levy EI, Pereira VM, et al. Stent-retriever thrombectomy after intravenous t-pa vs. T-pa alone in stroke. N Engl J Med 2015;372:2285–2295 [DOI] [PubMed] [Google Scholar]
  • 3.Campbell BC, Mitchell PJ, Kleinig TJ, Dewey HM, Churilov L, Yassi N, et al. Endovascular therapy for ischemic stroke with perfusion-imaging selection. N Engl J Med 2015;372:1009–1018 [DOI] [PubMed] [Google Scholar]
  • 4.Goyal M, Demchuk AM, Menon BK, Eesa M, Rempel JL, Thornton J, et al. Randomized assessment of rapid endovascular treatment of ischemic stroke. N Engl J Med 2015;372:1019–1030 [DOI] [PubMed] [Google Scholar]
  • 5.Berkhemer OA, Fransen PS, Beumer D, van den Berg LA, Lingsma HF, Yoo AJ, et al. A randomized trial of intraarterial treatment for acute ischemic stroke. N Engl J Med 2015;372:11–20 [DOI] [PubMed] [Google Scholar]
  • 6.Nogueira RG, Jadhav AP, Haussen DC, Bonafe A, Budzik RF, Bhuva P, et al. Thrombectomy 6 to 24 hours after stroke with a mismatch between deficit and infarct. N Engl J Med 2018;378:11–21 [DOI] [PubMed] [Google Scholar]
  • 7.Albers GW, Marks MP, Kemp S, Christensen S, Tsai JP, Ortega-Gutierrez S, et al. Thrombectomy for stroke at 6 to 16 hours with selection by perfusion imaging. N Engl J Med 2018;378:708–718 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Powers WJ, Rabinstein AA, Ackerson T, Adeoye OM, Bambakidis NC, Becker K, et al. 2018 guidelines for the early management of patients with acute ischemic stroke: A guideline for healthcare professionals from the american heart association/american stroke association. Stroke. 2018;49:e46–e110 [DOI] [PubMed] [Google Scholar]
  • 9.Becks MJ, Manniesing R, Vister J, Pegge SAH, Steens SCA, van Dijk EJ, et al. Brain ct perfusion improves intracranial vessel occlusion detection on ct angiography. J Neuroradiol 2019;46:124–129 [DOI] [PubMed] [Google Scholar]
  • 10.Srinivasan A, Goyal M, Lum C, Nguyen T, Miller W. Processing and interpretation times of ct angiogram and ct perfusion in stroke. Can J Neurol Sci 2005;32:483–486 [DOI] [PubMed] [Google Scholar]
  • 11.CMS. Hospital outpatient quality reporting program - measures for cy 2019 payment determination. 2019
  • 12.Lansberg MG, Straka M, Kemp S, Mlynash M, Wechsler LR, Jovin TG, et al. Mri profile and response to endovascular reperfusion after stroke (defuse 2): A prospective cohort study. Lancet Neurol 2012;11:860–867 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Klein S, Staring M, Murphy K, Viergever MA. Elastix: A toolbox for intensity based medical image registration. IEEE Transactions on Medical Imaging. 2010;29:196–205 [DOI] [PubMed] [Google Scholar]
  • 14.Frangi AF, Niessen WJ, Vincken KL, Viergever MA. Multiscale vessel enhancement filtering. Medical image computing and computer-assisted intervention - miccai’98. 1998;1496:130–137 [Google Scholar]
  • 15.DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics. 1988;44:837–845 [PubMed] [Google Scholar]
  • 16.Lev MH, Farkas J, Rodriguez VR, Schwamm LH, Hunter GJ, Putman CM, et al. Ct angiography in the rapid triage of patients with hyperacute stroke to intraarterial thrombolysis: Accuracy in the detection of large vessel thrombus. J Comput Assist Tomogr. 2001;25:520–528 [DOI] [PubMed] [Google Scholar]
  • 17.Wagemans BA, van Zwam WH, Nelemans PJ, van Oostenbrugge RJ, Postma AA. 4d-cta improves diagnostic certainty and accuracy in the detection of proximal intracranial anterior circulation occlusion in acute ischemic stroke. PLoS One. 2017;12:e0172356. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Bar M, Kral J, Jonszta T, Marcian V, Kuliha M, Mikulik R. Interrater variability for ct angiography evaluation between neurologists and neuroradiologist in acute stroke patients. Br J Radiol 2017;90:20160670. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Campbell BC, Christensen S, Tress BM, Churilov L, Desmond PM, Parsons MW, et al. Failure of collateral blood flow is associated with infarct growth in ischemic stroke. J Cereb Blood Flow Metab. 2013;33:1168–1172 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Rocha M, Jovin TG. Fast versus slow progressors of infarct growth in large vessel occlusion stroke: Clinical and research implications. Stroke. 2017;48:2621–2627 [DOI] [PubMed] [Google Scholar]
  • 21.Sarraj A, Sangha N, Hussain MS, Wisco D, Vora N, Elijovich L, et al. Endovascular therapy for acute ischemic stroke with occlusion of the middle cerebral artery m2 segment. JAMA Neurol 2016;73:1291–1296 [DOI] [PubMed] [Google Scholar]
  • 22.Barreira CM, Bouslama M, Haussen DC, Grossberg JA, Baxter B, Devlin T, et al. Abstract wp61: Automated large artery occlusion detection in stroke imaging - aladin study. Stroke. 2018;49, Supplement 1 [Google Scholar]
  • 23.Barreira CM, Bouslama M, Lim J, Al-Bayati A, Saleem Y, Devlin T, et al. E-108 aladin study: Automated large artery occlusion detection in stroke imaging study – a multicenter analysis. Journal of NeuroInterventional Surgery. 2018;10:A101–A102 [Google Scholar]
  • 24.Barreira CM, Bouslama M, Lim J, Al-Bayati A, Haussen DC, Grossberg JA, et al. As10–047: Aladin study: Automated large artery occlusion detection in stroke imaging study - a multi-center experience. European Stroke Journal. 2018;3 77 [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Material

RESOURCES