Estimation of Acute Infarct Core and Hypoperfused Region from Baseline Noncontrast Computed Tomography and Computed Tomography Angiography Scans of Patients with Ischemic Stroke

Abstract

BACKGROUND

In acute ischemic stroke, the infarct core and hypoperfused regions are key indicators for assessing and prognosticating patients. They are typically estimated with computed tomography perfusion (CTP). However, because noncontrast CT and CT angiography are more widely available, we trained a neural network to estimate the ischemic lesion from noncontrast CT and CT angiography scans.

METHODS

In this retrospective study, an nnU‐Net model was trained to estimate infarcted and hypoperfused regions from noncontrast CT and CT angiography using reference standards from a commercial CTP software (StrokeViewer). We included data from 859 patients for training and 137 for testing. We used data from the Collaboration for New Treatments of Acute Stroke consortium, including MR CLEAN (Multicenter Randomized Controlled Trial of Endovascular Treatment for Acute Ischemic Stroke in the Netherlands)‐NO‐IV, MR CLEAN‐MED, MR CLEAN‐LATE, and MR CLEAN‐Registry, and a local cohort. In addition to testing our model against StrokeViewer, we also compared our results with 3 other commercial CTP software packages.

RESULTS

Our model achieves a Dice of 0.45 (95% CI, 0.39–0.50) for core and 0.66 (95% CI, 0.62–0.69) for hypoperfused region, underestimating core volume by −9.3 mL (95% CI, −12.5 to −6.1) and hypoperfused region volume by −12.9 mL (95% CI, −21.1 to −4.7) compared with StrokeViewer. When comparing the 4 CTP software packages together, the average of their 2‐by‐2 agreement ranges from a Dice of 0.22 to 0.28 for core, and a Dice of 0.50 to 0.56 for hypoperfused region. This is similar to the average agreement of nnU‐Net with these 4 software packages (average Dice 0.27 for core and 0.56 for hypoperfused). Furthermore, nnU‐Net produces fewer connected components (1.3 for core, 1.6 for hypoperfused) than the average of the 4 CTP software packages (60.8 for core and 110.8 and hypoperfused), indicating more cohesive segmentations.

CONCLUSION

Our model's performance in segmenting infarct core and hypoperfused regions from noncontrast CT and CT angiography is comparable to commercial CTP software packages, with potentially fewer segmentation artifacts. It can therefore be used when CTP is not available.

Although several late window trials have recently simplified the imaging criteria for endovascular thrombectomy ¹ , ² , ³ , ⁴ , the infarct core and hypoperfused region remain important indicators in stroke, especially for the detection of large vessel occlusions. Additionally, the majority of reperfusion therapies in patients with acute ischemic stroke are based on intravenous thrombolysis, and intravenous thrombolysis can be administered after 4.5 hours from the stroke onset only if favorable computed tomography perfusion (CTP) characteristics are present. ⁵

Currently, noncontrast CT (NCCT) and CT angiography (CTA) are not used for infarct core and hypoperfused region estimation, because early ischemic changes on CT scans are subtle and easily missed or overscored. ⁶ , ⁷ Consequently, CTP is still an important part of the ischemic stroke workup. However, CTP has some disadvantages compared with other CT techniques. First, CTP is not as readily available in many stroke centers around the world compared with NCCT and CTA, because of its technical complexity. Second, not all CTP scanners offer full brain coverage, and CTP usually has a lower resolution than NCCT and CTA scans. Third, CTP requires more time to perform and is therefore more prone to movement artifacts. ⁸ Fourth, factors such as patient movement and low cardiac output, which are common in patients with stroke due to restlessness and cardiac comorbidities, can prohibit adequate CTP acquisition. ⁹ Consequently, CTP has a higher failure rate compared with other CT techniques. ¹⁰ Fifth, CTP requires subjecting the patient to a higher dose of radiation and a larger volume of contrast agent compared with CTA. Finally, due to large differences in analyses and tissue classification criteria of the CTP software vendors, there is a high variability in their lesion estimations. ¹¹ , ¹²

A model that can estimate the extent of the infarcted and hypoperfused regions from NCCT and CTA with the same reliability as CTP would be valuable in triage of patients with stroke. This model will be especially valuable in smaller medical centers and medically underserved regions without access to CTP. To the best of our knowledge, we are the first to use both NCCT and CTA to develop a neural network to estimate both the infarct core and hypoperfused regions in the acute phase. We evaluate our automated method against multiple commercially available CTP software packages to determine whether it shows results comparable to CTP.

Nonstandard Abbreviations and Acronyms

CTA

computed tomography angiography

CTP

computed tomography perfusion

DWI

diffusion‐weighted imaging

ISP

IntelliSpace Portal

MR CLEAN

Multicenter Randomized Controlled Trial of Endovascular Treatment for Acute Ischemic Stroke in the Netherlands

NCCT

noncontrast computed tomography

CLINICAL PERSPECTIVE

What Is New?

• This is the first study to develop an artificial intelligence model that can automatically identify both infarct core and hypoperfused regions using only routine noncontrast computed tomography (CT) and CT angiography, eliminating the need for specialized CT perfusion imaging.

• The artificial intelligence model demonstrated segmentation accuracy comparable to 4 different commercial CT perfusion software packages while producing more cohesive results with fewer artifacts, particularly in challenging brain regions.

What Are the Clinical Implications?

• This artificial intelligence approach could enable stroke centers without CT perfusion capabilities to obtain critical tissue characterization for treatment decision‐making, potentially expanding access to evidence‐based stroke care in resource‐limited settings and smaller medical centers worldwide.

METHODS

The data that support the findings of this study are available from the corresponding author upon reasonable request. This study was reported in accordance with the Checklist for Artificial Intelligence in Medical Imaging 2024 ¹³ checklist. We trained a neural network model to automatically identify the infarct core and hypoperfused region using only NCCT and CTA, thereby avoiding dependence on CTP. For training and validating the model, we used reference standard segmentations acquired from StrokeViewer (Nicolab, Amsterdam, The Netherlands), a commercially available, Conformité Européenne‐certified, and Food and Drug Administration‐approved CTP analysis software. Additionally, we further validated our model by comparing it against 3 other commercial CTP software packages.

Experiments

Part I: Different Input Modalities

We studied which input set—(1) NCCT, (2) CTA, or (3) their combination—produced the most accurate segmentations by training 3 separate models, each using one of these input sets. We compared the performance of these models based on their agreement with reference standards from StrokeViewer. The best‐performing model was used for all the subsequent experiments. To assess whether the observed differences in model performance were statistically significant, we performed pairwise comparisons using a paired t‐test when normality was met and a Wilcoxon signed‐rank test otherwise. Normality was assessed using the Shapiro–Wilk test. A P ‐value <0.05 was considered statistically significant. Additionally, we performed 2 more fine‐grained analyses where we stratified the StrokeViewer masks based on (1) volume and (2) onset‐to‐CTP time, and we studied the performance of the best model in each volume or time bracket.

Part II: Comparison With Multiple CTP Software Packages

In the second part of our analysis, we compared the best model with 4 different CTP software packages. Furthermore, we reported the results of pairwise comparisons between these 4 software packages, evaluating their agreement with each other.

Training Details

We used nnU‐Net, ¹⁴ a convolutional neural network model, for all our experiments. All models were trained to segment both the infarct core and hypoperfused region. All models were trained for 1000 epochs on an NVIDIA Tesla V100 GPU with 32 GB memory. We used the 3‐dimensional full resolution setting of nnU‐Net version 2 (Isensee et al., 2020). nnU‐Net uses a 5‐fold cross‐validation training paradigm to train a U‐Net ¹⁵ convolutional neural network model. It also performs automatic preprocessing and postprocessing of the data and selects the best training hyperparameters, making it easily reproducible. All our nnU‐Net models were trained with 5‐fold cross‐validation on the training set. Then, an ensemble of all 5 models is used per the default settings of nnU‐Net to generate results on our separate test sets. To monitor for overfitting, training and validation losses were tracked during training.

Data

The data for this study were collected retrospectively. Our data set included all patients who had baseline NCCT, CTA, and CTP who were included in the following study populations: MR CLEAN (Multicenter Randomized Controlled Trial of Endovascular Treatment for Acute Ischemic Stroke in the Netherlands) Registry, ¹⁶ MR CLEAN‐NO IV, ¹⁷ MR CLEAN‐MED, ¹⁸ and MR CLEAN‐LATE. ¹⁹ Additionally, we included 112 patients with a large vessel occlusion who presented to our comprehensive stroke center and received baseline NCCT, CTA, and CTP but were not included in one of the aforementioned trials. CTP data of all patients were collected and analyzed as part of the CLEOPATRA (Cost‐effectiveness of CT perfusion for Patients with Acute Ischemic Stroke) health care evaluation study. ²⁰ Patients were excluded if (1) CTP analysis with StrokeViewer was unsuccessful due to poor acquisition or severe artifacts (eg, no timely arrival of contrast agent or too few acquisition time points), or (2) NCCT or CTA were missing or their quality was low. Criteria for a low image quality included sharp bone reconstruction kernels, part of the brain missing, and movement or registration artifacts.

Training Set

To ensure a uniform distribution of core volumes between training and validation sets, the data were stratified based on core volume for each cross‐validation fold.

Single‐Vendor Test Set

The single‐vendor test set was used for Part I of our experiments, where StrokeViewer reference standards were used as reference standard labels.

Multivendor Test Set

The multivendor test set was used for the second part of our experiments. This test set consisted of cases that overlapped between our single‐vendor test set and a multivendor data set from Hoving et al. ¹¹ We included only those patients for whom high‐quality baseline NCCT and CTA scans were available. The resulting multivendor test set included patients whose CTP scans were analyzed by 4 different commercially available software packages. The software packages used to analyze this data were (1) StrokeViewer (Version 3.2.11; Nicolab, Amsterdam, The Netherlands), (2) Siemens syngo.via CT Neuro Perfusion (version VB40; Siemens Healthcare, Erlangen, Germany), (3) IntelliSpace Portal (ISP) (version v12.1.5; Philips Medical Systems, Best, The Netherlands), and (4) Vitrea Bayesian CT Brain Perfusion 2D (version 7.14; Vital Images, Minnetonka, Minnesota). In this article, these software packages are referred to as StrokeViewer, syngo.via, ISP, and Vitrea, respectively.

Preprocessing

All analyses were done in the CTP space. The NCCT, CTA, and the reference standard labels from StrokeViewer, Vitrea, and ISP were all registered to the first frame of the CTP scan for each patient. The reference standard labels from syngo.via did not require registration because they were already in the CTP space. Before registration, all the background (outside of the skull) Hounsfield unit values were set to 0. After registration, all the Hounsfield unit values were clipped between [0, 100] for NCCT and [0, 200] for CTA.

Metrics

We used the following metrics to assess our model: Dice, Surface Dice, volumetric difference, absolute volumetric difference, Bland–Altman analysis, and number of connected components.

The Dice coefficient measures the overlap between estimated and reference standard segmentations, ranging from 0 (no overlap) to 1 (full overlap). If both the ground truth and predictions show an empty segmentation, then Dice is set to 1 (indicating complete agreement). Surface Dice ²¹ addresses Dice's bias toward smaller segmentations ²² by evaluating surface alignment within a tolerance distance $\tau$ (we chose $\tau =1$ mm). Volumetric difference captures the discrepancy between estimated and reference volumes, with negative values indicating underestimation. Absolute volumetric difference disregards direction, focusing solely on the magnitude of the error. The Bland–Altman analysis ²³ plots the volumetric difference versus the average of the 2 measurements. The number of connected components reflects segmentation continuity. A high number of connected components in the segmentations of the core or hypoperfused region implies a fragmented segmentation or potential segmentation artifacts.

Dice, volumetric difference, and number of connected components were calculated using the SimpleITK Python package (version 2.3.1). Surface Dice was calculated using the surface‐distance Python package (https://github.com/google‐deepmind/surface‐distance.git, accessed on April 5, 2024). The Bland–Altman analysis was performed using the pingouin Python package (version 0.5.4).

RESULTS

The training set included 859 patients from the MR CLEAN Registry (278), MR CLEAN‐NO IV (129), MR CLEAN‐MED (87), MR CLEAN‐LATE (266), and our local cohort (99). The single‐vendor test set included 137 patients, and the multivendor test set included 45 patients. Of the total patients excluded from this study, 93 were excluded due to unsuccessful CTP analysis, 26 because NCCT or CTA were missing, 8 due to sharp bone reconstruction kernels, 26 because part of the brain was missing, and 3 due to motion or registration artifacts.

Table 1 provides an overview of the training and test set populations. One of the main differences between them is that the onset‐to‐CTP time is twice as high in the training set compared with the test set. Another notable difference is the distribution of occlusion locations, where internal carotid artery occlusions are more common in the test set.

Table 1.

Description of the Patient Populations in the Training and Test Set

	Training set	Test set	Subset of test set
Age–median (y–IQR)	71 (61–80); (n = 855)	70 (61–77.7); (n = 134)	72 (59–77); (n = 43)
Sex (male)–n(%)	455 (53.2%); (n = 855)	72 (53.7%); (n = 134)	19 (44.2%); (n = 43)
Time of onset or last seen well to CTP time (minutes)–median (IQR)	235 (85–601); (n = 794)	115 (61–328); (n = 125)	74 (56–132); (n = 39)
Baseline NIHSS score–median (IQR)	15 (8–19); (n = 833)	15 (9–19); (n = 129)	15 (9–18); (n = 41)
ASPECTS–median (IQR)	9 (7–10); (n = 765)	9 (7–10); (n = 96)	9 (8–10); (n = 17)
Occlusion location–n(%)	(n = 850)	(n = 131)	(n = 40)
ICA	162 (19.0%)	54 (41.2%)	37 (92.5%)
M1	458 (53.9%)	53 (40.4%)	3 (7.5%)
M2	208 (24.5%)	21 (16.0%)	0
Other	5 (0.6%)	1 (0.8%)	0
None	17 (2.0%)	2 (1.5%)	0
Collateral score * –n(%)	(n = 755)	(n = 97)	(n = 15)
Score 0	47 (6.2%)	3 (3.1%)	1 (6.7%)
Score 1	243 (32.2%)	22 (22.7%)	4 (2.7%)
Score 2	313 (41.4%)	50 (51.5%)	5 (33.3%)
Score 3	152 (20.1%)	22 (22.7%)	5 (33.3%)
Core volume StrokeViewer (mL)–median (IQR)	21.2 (2.5–28.0)	24.4 (2.5–31.5)	29.7 (8.3–45.3)
Hypoperfused volume StrokeViewer (mL)–median (IQR)	114.0 (64.1–156.8)	122.6 (69.8–164.2)	120.3 (67.1–164.4)

“Test set” refers to the single vendor test set, and “subset of test set” refers to the multivendor test set, which is a subset of the single vendor test set. In case of missing values, the number of available data points is indicated with n. ASPECTS indicates Alberta Stroke Program Early CT Score; CTP, computed tomography perfusion; IQR, interquartile range; and NIHSS, National Institutes of Health Stroke Scale.

^* Collateral score definitions: Score 0 (absent collaterals), Score 1 (filling = 50% of occluded area), Score 2 (>50% but less than <100%), Score 3 (100% of the occluded area).

Single‐Vendor Test Set Results

Table 2 presents the results of models using NCCT, CTA, or a combination of both as input. The model that combines NCCT and CTA achieved the best performance. Combining data from NCCT and CTA considerably improves the performance for infarct core estimation. However, adding NCCT has only a negligible effect on the performance for the estimation of the hypoperfused region. For core and hypoperfused region assessments, the difference in Surface Dice and Dice between the NCCT+CTA model and the CTA only model is significant (P values <0.01 for both). The volumetric differences were not statistically significant (P value 0.74 and 0.10 for core and hypoperfused region, respectively).

Table 2.

Comparing the Results of Models that Used NCCT, CTA, or Both as Input

	Input(s)	Surface Dice $\uparrow$	Dice $\uparrow$	Volumetric difference (mL)	Absolute volumetric difference (mL) $\downarrow$	Mean number of connected components $\downarrow$
Core	NCCT	0.37 (0.32 to 0.42)	0.31 (0.26 to 0.35)	−11.7 (−16.1 to −7.3)	17.4 (13.6 to 21.2)	1.2 (1.0 to 1.3)
	CTA	0.45 (0.40 to 0.51)	0.39 (0.34 to 0.44)	−9.4 (−12.7 to −6.1)	12.9 (10.0 to 15.9)	1.2 (1.1 to 1.4)
	NCCT+CTA	0.52 (0.46 to 0.57)	0.45 (0.39 to 0.5)	−9.3 (−12.5 to −6.1)	12.2 (9.4 to 15.1)	1.3 (1.1 to 1.4)
Hypoperfused region	NCCT	0.51 (0.48 to 0.54)	0.56 (0.52 to 0.60)	−20.3 (−30.3 to −10.4)	44.3 (37.0 to 51.7)	1.5 (1.4 to 1.7)
	CTA	0.61 (0.58 to 0.65)	0.64 (0.61 to 0.68)	−14.6 (−23.0 to −6.1)	35.9 (29.5 to 42.3)	1.6 (1.4 to 1.7)
	NCCT+CTA	0.63 (0.59 to 0.67)	0.66 (0.62 to 0.69)	−12.9 (−21.1 to −4.7)	34.1 (27.8 to 40.4)	1.6 (1.4 to 1.7)

StrokeViewer infarct core and hypoperfused region masks are used as reference standards. The 95% CIs are reported in parentheses. Up arrows (↑) indicate that a higher value is preferable for the metric. Down arrows (↓) indicate a lower value is preferable. The best scores are highlighted with bold text. CTA indicates computed tomography angiography; and NCCT, noncontrast computed tomography.

The best performing model achieves a Surface Dice of 0.52 (95% CI, 0.46–0.57) for core and 0.63 (95% CI, 0.59–0.67) for hypoperfused region. Compared with the StrokeViewer reference standard segmentations, the best model underestimates the infarct core volume by −9.3 mL (95% CI, −12.5 to −6.1) and the hypoperfused region volume by −12.9 mL (95% CI, −21.1 to −4.7). The mean number of connected components for all models is <2, with the NCCT model having the lowest average number of connected components.

The nnU‐Net core estimations have a similar number of connected components to those of StrokeViewer (StrokeViewer: 0.92 [95% CI, 0.83–1.01], nnU‐Net: 1.3 [95% CI, 1.1–1.4]). However, for the hypoperfused region, the nnU‐Net estimations have significantly fewer connected components compared with StrokeViewer (StrokeViewer: 9.35 [95% CI, 7.63–11.42], nnU‐Net: 1.6 [95% CI, 1.4–1.7]).

Appendix A shows the results of stratifying the segmentations based on volume and onset‐to‐CTP time. The best performing model shows low performance in segmenting cores smaller than 5 mL, achieving a Dice score of 0.16 (95% CI, 0.05–0.27). However, its performance improves considerably for slightly larger cores between 5 and 10 mL, with a Dice score of 0.32 (95% CI, 0.18–0.45). For cores >30 mL, nnU‐Net performs significantly better, achieving a Dice score of 0.49 (95% CI, 0.42–0.55). For the hypoperfused region, the number of connected components in StrokeViewer reference standards increases noticeably with lesion size, ranging from 4.2 for smaller lesions to 13.9 for larger ones. In contrast, nnU‐Net maintains a relatively stable number of connected components, varying from 1.5 to 2.1 across different volume brackets. The performance of nnU‐Net stays relatively consistent across onset‐to‐CTP times ranging from 0 to 25 hours, with the exception of a noticeable improvement in the 4 to6 hour bracket. However, this improvement can be due to the limited sample size in this time bracket, which includes only 9 patients.

The plots of the Bland‐Altman analysis for the hypoperfused region and infarct core are presented in Appendix B. nnU‐Net tends to overestimate smaller cores and underestimate larger core volumes. No trend is seen for the hypoperfused region volumes.

Multivendor Test Set Results

Appendix C shows the results of pairwise comparisons among the 4 CTP software packages, along with comparisons between nnU‐Net and each CTP software. The Surface Dice, Dice, and volumetric differences between nnU‐Net and the 4 CTP software packages are similar to the levels of agreement between the software, packages themselves. As expected, nnU‐Net shows a higher agreement with StrokeViewer than with the other software packages. Detailed pairwise scores between the software packages and nnU‐Net are displayed as confusion matrices in Appendix C.

The Surface Dice scores between nnU‐Net and each CTP software, as well as the agreements among the software packages themselves, are plotted in Figure 1A for the hypoperfused region and Figure 1B for the infarct core. These plots further demonstrate that the levels of agreement between nnU‐Net and each CTP software package are comparable to the agreement observed among the software packages themselves. In hypoperfused region estimation, ISP shows the lowest agreement with all the other software packages.

Figure 1.

Pairwise Surface Dice score comparisons of (A) hypoperfused region and (B) infarct core segmentations among all CTP software packages and our best model (nnU‐Net with NCCT+CTA input). CTA indicates computed, tomography angiography; CTP, computed tomography perfusion; ISP, IntelliSpace Portal; NCCT, noncontrast computed tomography; and SV, StrokeViewer.

Appendix C reports the average number of connected components for each software. In the hypoperfused region segmentations, nnU‐Net has a significantly lower number of connected components (1.6) compared with ISP (157.0), Syngo.via (205.5), Vitrea (71.9), and StrokeViewer (9.3). In infarct core segmentations, only StrokeViewer (1.0) and nnU‐Net (1.3) show a number of connected components close to 1, whereas ISP (108.5), Syngo.via (61.6), and Vitrea (72.2) have much higher values.

To better illustrate the difference between the segmentations of the 4 software packages and nnU‐Net, examples of infarct core and hypoperfused region segmentations are depicted in Figure 2. This figure also highlights how variations in the number of connected components affect the segmentation quality. The masks produced by the CTP software packages tend to be more fragmented and sometimes contain segmentation artifacts, particularly in the cranial and caudal slices, and even in the contralateral hemisphere. In contrast, nnU‐Net's predictions look more cohesive and show fewer artifacts.

Figure 2.

Hypoperfused region and infarct core segmentations for 2 patients, generated by all 4 CTP software packages and our best model (nnU‐Net with NCCT+CTA input). The patient depicted in (A) was chosen randomly from the multivendor test set. For better visibility, a patient with a larger infarct core is selected for (B). CTA indicates computed tomography angiography; CTP, computed tomography perfusion; ISP, IntelliSpace Portal; and NCCT, noncontrast computed tomography.

DISCUSSION

Our model estimated the infarct core and hypoperfused region from NCCT and CTA, achieving an acceptable overlap with reference standards derived from CTP. Our model's agreement with the commercially available software packages was on the same level as the agreement of these 4 different CTP software packages among themselves. Furthermore, our segmentations had the least number of connected components compared with the reference standards, indicating a more cohesive segmentation mask, potentially with fewer artifacts.

Our top‐performing model used both NCCT and CTA as input and achieved a Dice of 0.45 (95% CI, 0.39–0.50) for infarct core segmentation and 0.63 (95% CI, 0.59–0.67) for hypoperfused region segmentation, compared with reference standard labels extracted from StrokeViewer. Given the variability in reference standard labels across previous studies, direct comparison of results was difficult. For example, both Ostmeier et al,. and El‐Hariri et al,. used NCCT as input to nnU‐Net with manually delineated labels (on NCCT), but one reported a Dice of 0.47 and the other a Dice of 0.38, respectively. As an overview, previous studies that used NCCT as input reported Dice scores ranging from 0.36 to 0.45 for infarct core segmentation compared with diffusion‐weighted imaging (DWI) reference standard; ²⁴ , ²⁵ and a Dice ranging from 0.38 to 0.72 compared with expert manual delineations. ²⁶ , ²⁷ , ²⁸ Previous studies that used CTA as input reported a Dice of 0.26 compared with DWI ²⁹ and 0.61 compared with manual delineations on a small test set. ³⁰ Our approach differs from the literature in that we used information from both NCCT and CTA scans to improve the accuracy of our segmentations. Furthermore, our model produced both an infarct core and a hypoperfused region segmentation and could therefore be used to assess tissue at risk.

Our model had a higher agreement with StrokeViewer compared with other CTP software packages, because StrokeViewer labels were used as a reference standard during training. Despite this fact, we observed that our model produced fewer connected components and fewer artifacts in hypoperfused region segmentation compared with StrokeViewer. This indicated that our model could effectively learn the underlying patterns in the data while filtering out some of the noise in the training labels. Visual examination of our results confirmed that our segmentation results were more cohesive and had fewer artifacts, especially in the cranial and caudal slices. Another strength of this work is that our model was trained and tested on a large European data set, including 996 patients, spanning multiple trials and containing scans from different centers and scanners. The diversity in the data enabled us to better validate the generalizability of our model. However, it must be mentioned that the majority of our data are acquired from the MR CLEAN Registry and trials. This reliance may limit our model's applicability to populations that differ from our cohort, such as non‐European patient groups.

After studying the effect of different inputs, we found that incorporating information from both NCCT and CTA was important for infarct core segmentation, whereas adding NCCT contributed little to hypoperfused region segmentation. This observation aligned with our expectations because changes in tissue attenuation can be detected on both modalities, whereas information about perfusion is exclusive to CTA scans. Furthermore, our findings suggested that our model's estimations more closely resemble CTP labels for larger lesions than smaller ones. This was consistent with previous studies that found a higher agreement between CTP software packages for larger lesions. ³¹ In the case of the hypoperfused region segmentations, we observed an increase and a subsequent slight decrease in surface Dice with increasing volume. This decrease could be because the number of connected components increased with lesion size for StrokeViewer, suggesting noisier reference labels in larger lesions. Overall, our model tended to underestimate both the core and hypoperfused regions compared with StrokeViewer, particularly in larger lesions. This underestimation may stem from the omission of artifacts in our segmentations. Another potential reason is that our model only detected regions with hypoperfusion severe enough to manifest on CTA, suggesting that CTP software may have employed a lower threshold for delineating affected regions.

The main limitation of our study was the absence of a gold reference standard for training and testing. The best modality for detecting early ischemic changes is DWI. ³² However, even DWI is not completely accurate in determining the extent of early lesions. Additionally, DWI is expensive and not always readily available in the acute setting. It is also not compatible with certain implants and metallic foreign bodies. ³³ Therefore, it is difficult to generate large databases of patients with stroke with both acute DWI and CT without a significant time delay between the 2 scans. Furthermore, due to the same reasons, the majority of stroke trials that studied the relationship between lesion size and outcome have used CTP to extract imaging biomarkers. ³⁴ Consequently, in this study, we have used CTP to extract the reference standards. However, it is a well‐known issue that there is variability in the results of different CTP software packages. In order to alleviate the effects of this variability, we introduced the multivendor test set, where we compared our model with 4 different software packages. Another limitation of our study was the large difference between the resolution of NCCT and CTA and the resolution of the CTP. To reduce processing time, the output of CTP software packages usually has a lower resolution than NCCT and CTA. Consequently, the resampling and registration operations that were necessary for our method could have introduced artifacts that may have affected our results negatively.

In conclusion, our model successfully segmented the infarct core and hypoperfused region from NCCT and CTA scans. The model's segmentations had a good spatial overlap with StrokeViewer (our reference standard), especially for the hypoperfused region. Furthermore, the performance of our model was comparable to 3 additional commercial CTP software packages, while producing segmentations with fewer artifacts than any of these CTP software packages. Consequently, we believe that our model can be a viable alternative to CTP‐based analyses, especially in cases where access to CTP imaging is limited or where CTP acquisition has failed.

Sources of Funding

This work was part of the Artificial Intelligence for Early Imaging‐Based Patient Selection in Acute Ischemic Stroke (AIRBORNE) project. This project was supported by Top Sector Life Sciences & Health and Nicolab B.V. The CONTRAST consortium acknowledges the support from the Netherlands Cardiovascular Research Initiative, an initiative of the Dutch Heart Foundation (CVON2015‐01: CONTRAST), and from the Brain Foundation Netherlands (HA2015.01.06). The collaboration project is additionally financed by the Ministry of Economic Affairs by means of the PPP Allowance made available by the Top Sector Life Sciences & Health to stimulate public‐private partnerships (LSHM17016). This work was funded in part through unrestricted funding by Stryker, Medtronic, and Cerenovus. The funding sources were not involved in study design, monitoring, data collection, statistical analyses, interpretation of results, or manuscript writing. The CLEOPATRA health care evaluation was funded by Leading the Change (LtC). LtC is financed by Zorgverzekeraars Nederland (ZN) and supports various health care evaluations in the Netherlands as part of the Healthcare Evaluation Netherlands project. LtC was not involved in the study design, monitoring, data collection, statistical analyses, interpretation of results, or manuscript writing, but the progress of the study was continuously monitored by LtC.

Disclosures

Charles Majoie reports a grant from the CVON/ Dutch Heart Foundation, TWIN Foundation, European Commission, Healthcare Evaluation Netherlands, Stryker, and Boehringer Ingelheim (all paid to their institution); and is a (minority interest) shareholder of Nicolab. Henk Marquering is cofounder and shareholder of Nicolab, TrianecT, and inSteps. Wim van Zwam reports that Philips pays reimbursements for presentations and DSMB activities to their institution. Bart Emmer reports grants reports grants from the Dutch Heart Foundation, Dutch Ministry of Economics, and Healthcare Evaluation Netherlands (all paid to their institution).

Supporting information

Appendix A: Stratification of results based on volume and onset‐to‐CTP time

Appendix B: Plots for Bland‐Atlman analysis

Appendix C: Extensive results of the multi‐vendor analysis

Appendix D: Examples of cases with low overlap with the ground truth

Appendix E: Comparison with DWI