Feasibility of a Clinical-Radiomics Model to Predict the Outcomes of Acute Ischemic Stroke

Abstract

Objective

To develop a model incorporating radiomic features and clinical factors to accurately predict acute ischemic stroke (AIS) outcomes.

Materials and Methods

Data from 522 AIS patients (382 male [73.2%]; mean age ± standard deviation, 58.9 ± 11.5 years) were randomly divided into the training (n = 311) and validation cohorts (n = 211). According to the modified Rankin Scale (mRS) at 6 months after hospital discharge, prognosis was dichotomized into good (mRS ≤ 2) and poor (mRS > 2); 1310 radiomics features were extracted from diffusion-weighted imaging and apparent diffusion coefficient maps. The minimum redundancy maximum relevance algorithm and the least absolute shrinkage and selection operator logistic regression method were implemented to select the features and establish a radiomics model. Univariable and multivariable logistic regression analyses were performed to identify the clinical factors and construct a clinical model. Ultimately, a multivariable logistic regression analysis incorporating independent clinical factors and radiomics score was implemented to establish the final combined prediction model using a backward step-down selection procedure, and a clinical-radiomics nomogram was developed. The models were evaluated using calibration, receiver operating characteristic (ROC), and decision curve analyses.

Results

Age, sex, stroke history, diabetes, baseline mRS, baseline National Institutes of Health Stroke Scale score, and radiomics score were independent predictors of AIS outcomes. The area under the ROC curve of the clinical-radiomics model was 0.868 (95% confidence interval, 0.825–0.910) in the training cohort and 0.890 (0.844–0.936) in the validation cohort, which was significantly larger than that of the clinical or radiomics models. The clinical radiomics nomogram was well calibrated (p > 0.05). The decision curve analysis indicated its clinical usefulness.

Conclusion

The clinical-radiomics model outperformed individual clinical or radiomics models and achieved satisfactory performance in predicting AIS outcomes.

INTRODUCTION

Stroke is the second leading cause of death and third leading cause of disability globally [1]. Of those who survive stroke, 40% have moderate functional impairment and 15%–30% suffer from severe permanent disability [2]. The two most effective therapies for the treatment of acute ischemic stroke (AIS) are intravenous alteplase and mechanical thrombectomy [3]. Nevertheless, due to the narrow time window for thrombolytic therapy, only a very small proportion of AIS patients receive intravenous thrombolytic therapy [4], and the majority of AIS patients are limited to similar conventional treatments, including antiplatelet treatment and neuroprotective agents. Therefore, research on the outcomes of AIS patients receiving conventional treatments can benefit more patients. An accurate prediction of patient outcomes would help clinicians understand patients’ conditions at the early stage of onset, have better communication with patients about the risks and expectations of treatment, and make a more individualized treatment plan instead of invariable or similar conventional therapy [5]. It is necessary to improve stroke treatment by the timely identification of patients with AIS who have an increased risk of adverse outcomes.

Previous studies have identified some factors associated with stroke outcome [6,7,8]. The first are clinical factors, such as the National Institute of Health Stroke Scale (NIHSS) score, hypertension, smoking, age, and sex. The second category includes radiological factors, such as the apparent diffusion coefficient (ADC) value and lesion volume on diffusion-weighted imaging (DWI). Among these factors, NIHSS score and DWI lesion volume were reported the most as the former reflects stroke severity, and DWI is the most sensitive technology for the early detection of stroke. Nevertheless, studies on radiological factors have been relatively less explored using radiomics [7,8,9]. The term radiomics, which has gained increasing attention in recent years, refers to the process of transforming medical images into high-dimensional, mineable data via high-throughput extraction of quantitative features, followed by subsequent data analysis to provide decision support [10]. By capturing high-dimensional features, such as texture, advanced shape modeling from target images, and providing additional potential features, radiomics can establish a link between clinical and radiological data [11]. Radiomics has been used as an emerging methodology to study tumor grading and typing, tumor heterogeneity, and survival of glioma patients, with excellent results [12,13]. Recently, radiomics features of MR images have been shown to have potential for the prediction of stroke outcomes [14,15].

The purpose of this study was to develop a prediction model incorporating clinical factors and radiomics features from DWI and ADC maps and to establish a nomogram as a graphic tool for predicting AIS outcomes.

MATERIALS AND METHODS

Patients and Data Acquisition

This retrospective study was approved by the Institutional Review Board (IRB No. TJ-C20111213), and the requirement for written informed consent was waived. A total of 836 patients with AIS from Tongji Hospital between January 2013 and September 2019 were screened for inclusion in the study. The inclusion criteria were as follows: 1) hospitalization within 24 hours from symptom onset and treatment with conventional therapy, 2) acquisition of the NIHSS score at the time of admission, and 3) DWI examination within 3 days of onset. The exclusion criteria were as follows: 1) treatment with intravenous alteplase or mechanical thrombectomy therapy (n = 39), 2) hemorrhagic transformation (n = 61), 3) cerebellar infarction (n = 83), 4) malignancy (n = 41), 5) severe MRI artifacts (n = 22), and 6) lost to follow-up (n = 68). Therefore, 522 patients with the required clinical, radiological, and prognostic data were enrolled in our study. Patients were randomly divided into training (n = 311) and validation (n = 211) cohorts.

Baseline clinical data, including age, sex, stroke history, hypertension, hyperlipidemia, diabetes, smoking, atrial fibrillation, cardiovascular disease, onset-to-MRI time, baseline NIHSS score (NIHSS_baseline), baseline modified Rankin Scale score (mRS_baseline), and Trial of ORG 10172 in Acute Stroke Treatment (TOAST) classification, were derived from the medical records. The AIS locations were classified by a neuroradiologist (7 years of experience) who was blinded to the patients’ clinical information according to the feeding artery of the infarction on DWI.

The mRS score of patients 6 months after hospital discharge was evaluated through a structured telephone interview by a neurologist (5 years of experience) who was blinded to all clinical and image data. The prognosis was dichotomized into good (mRS ≤ 2) and poor (mRS > 2).

The 1.5T/3T MRI scanners (GE Healthcare and Siemens) were used to acquire DWI images in all patients. For the 3T MRI scanner (GE Healthcare), the parameters were as follows: repetition time (TR) = 3000 ms, echo time (TE) = 65.7 ms, slice thickness = 5.0 mm, b value = 0 and 1000 s/mm². For the 3T MRI scanner (Siemens), TR = 6400 ms, TE = 98.0 ms, slice thickness = 5.5 mm, b value = 0, and 1000 s/mm². For the 1.5T MRI scanner (GE Healthcare), TR = 3789 ms, TE = 84.5 ms, slice thickness = 6.0 mm, b value = 0, and 1000 s/mm². All DWI data were imported into the GE ADW 4.6 workstation to calculate ADC maps.

Lesion Segmentation and Radiomics Feature Extraction

The workflow of the radiomics analysis included lesion segmentation, feature extraction, feature selection, and model construction (Fig. 1). To calculate inter-observer agreement of the feature extraction, the lesion was manually segmented along a high signal on DWI slice by slice in an open-source software (ITK SNAP 3.6.0; http://www.itksnap.org) by two neuroradiologists (7 years of experience in neuroimaging) who were blinded to the patients’ clinical information. An interclass correlation coefficient (ICC) higher than 0.75 was considered credible [16]. The lesion region of interest (ROI) of the DWI was directly applied as a template to the corresponding ADC maps. A total of 1310 features were extracted from the ROIs of DWIs and ADC maps using PyRadiomics (3.0.1), according to the instructions of the Image Biomarker Standardization Initiative [17]. These features included first-order, shape-based (including volume), textural, wavelet filtering, Laplacian of Gaussian, square, square root, logarithm, exponential, gradient, and local binary pattern features. A correlation matrix heatmap was used to reveal the patients’ radiomics features and their associations with stroke outcomes.

Fig. 1. Workflow of radiomics analysis.

ADC = apparent diffusion coefficient, DWI = diffusion-weighted imaging, ICC = interclass correlation coefficient, LASSO = least absolute shrinkage and selection operator, mRMR = minimum redundancy maximum relevance, ROC = receiver operating characteristic

Feature Selection and Radiomics Model Building

For feature selection, the relevance and redundancy of the features from the training data were calculated using the Spearman’s correlation coefficient. After eliminating redundant features with a Spearman’s correlation coefficient ≥ 0.8 [18], the minimum redundancy maximum relevance algorithm was applied to select features by considering not only their contributions to prediction but also the dependency among these features [19]. Then, the least absolute shrinkage and selection operator (LASSO) logistic regression, which is a shrinkage method that can conduct active selection from a large and potentially multicollinear set of variables in regression, was applied to select the most effective predictive features using a ten-fold cross-validation [20]. The radiomics score was generated using a linear combination of the final sifted features weighted by the LASSO algorithm, and a radiomics model was then established.

Development of the Clinical and Clinical-Radiomics Prediction Models

Univariable logistic regression analysis was implemented to explore clinical factors for predicting AIS outcomes in the training cohort, and these significant variables were entered into multivariable logistic regression analysis using a backward stepdown selection procedure with a p < 0.05, as the retention criteria to determine the independent clinical predictors of AIS outcomes. The results were presented as odds ratios with 95% confidence intervals (CIs), and a multivariable clinical prediction model was constructed.

Then, a multivariable logistic regression analysis incorporating independent clinical factors and radiomics scores was implemented to establish the final combined clinical-radiomics prediction model using the backward step-down selection procedure. p < 0.05 was considered statistically significant. A clinical-radiomics nomogram was developed in this study.

Statistical Analysis

To check the equality of patient demographic data between cohorts, normally distributed data were analyzed using an independent t test, and non-normally distributed data expressed as median (interquartile range) were examined using the Mann–Whitney U test. Categorical variables were analyzed using the chi-squared test.

The predictive performance of the radiomics, clinical, and clinical-radiomics models was evaluated using receiver operating characteristic (ROC) curves. The area under the ROC curve (AUC) and balanced sensitivity and specificity at the cutoff yielding the largest Youden index value were calculated. The performance of the three models was tested in the training and validation cohorts. The Delong test was used to compare the AUC between the models.

The calibration curve and Hosmer–Lemeshow test were used to assess the calibration performance of the clinical-radiomics nomogram [21]. Decision curve analysis (DCA) was implemented to determine the clinical utility of the clinical-radiomics nomogram by quantifying the net benefits at different threshold probabilities [22].

Statistical analyses were performed using SPSS (version 26.0; IBM Corp.) and R software (version 3.6.3, R Foundation for Statistical Computing). Statistical significance was defined as a two-sided p value < 0.05.

RESULTS

Patient Characteristics

The baseline characteristics of the patients in the training and validation cohorts are shown in Table 1. There were no statistically significant differences between the two cohorts. Patients with poor outcomes accounted for 28.30% (88/311) and 28.44% (60/211) of the training and validation cohorts, respectively.

Table 1. Baseline Characteristics of Patients in the Training and Validation Cohorts.

Characteristic		Training Cohort (n = 311)	Validation Cohort (n = 211)	P
Age, year		58.24 ± 11.6	59.97 ± 11.23	0.092
Sex				0.749
	Male	226 (72.7)	156 (73.9)
	Female	85 (27.3)	55 (26.1)
Stroke history		56 (18.0)	25 (11.9)	0.057
Hypertension		180 (57.9)	120 (56.9)	0.820
Hyperlipemia		44 (14.2)	36 (17.1)	0.364
Diabetes		56 (18.0)	46 (21.8)	0.283
Smoke		150 (48.2)	107 (50.7)	0.578
Atrial fibrillation		13 (4.2)	7 (3.3)	0.614
Cardiovascular		37 (11.9)	29 (13.7)	0.533
Onset-to-MRI time				0.502
	< 24 hours	19 (6.1)	10 (4.7)
	24–72 hours	292 (93.9)	201 (95.3)
Location of AIS				0.537
	Penetrating artery	134 (43.1)	103 (48.8)
	Cor-MCA	89 (28.6)	57 (27.0)
	Cor-ACA	17 (5.5)	11 (5.2)
	Cor-PCA	26 (8.4)	19 (9.0)
	Multi-arteries	45 (14.5)	21 (10.0)
mRSbaseline				0.432
	≤ 2	49 (15.8)	28 (13.3)
	> 2	262 (84.2)	183 (86.7)
NIHSSbaseline		5 (3–7)	4 (2–7)	0.373
TOAST				0.191
	Large-artery atherosclerosis	165 (53.1)	97 (46.0)
	Cardioembolism	32 (10.3)	20 (9.5)
	Small-artery occlusion	56 (18.0)	59 (28.0)
	Other determined etiology	12 (3.9)	4 (1.9)
	Undetermined etiology	46 (14.8)	31 (14.7)
Field strength of scanners				0.568
	1.5T	169 (54.3)	120 (56.9)
	3T	142 (45.7)	91 (43.1)
Outcome				0.972
	Good	223 (71.7)	151 (71.6)
	Poor	88 (28.3)	60 (28.4)

Data are presented as number of patients (%) except for mean ± standard deviation for age and median (interquartile range) for NIHSS_baseline. AIS = acute ischemic stroke, Cor-ACA = cortical branches of anterior cerebral artery, Cor-MCA = cortical branches of middle cerebral artery, Cor-PCA = cortical branches of posterior cerebral artery, mRS_baseline = baseline modified Rankin Scale score, NIHSS_baseline = baseline National Institutes of Health Stroke Scale score, TOAST = Trial of ORG 10172 in acute stroke treatment

Establishment and Performance of the Radiomics Model

Logistic analysis showed that seven radiomic features in DWI and ADC maps were independently associated with AIS outcomes (Supplementary Fig. 1). The exponential gray level non-uniformity (GLN) extracted from DWI and wavelet-low, high, and high-pass filtered image (LHH) cluster prominence from ADC maps weighted the most in the radiomics model. The inter-observer reproducibility of the feature extraction was excellent, with inter-observer ICCs ranging from 0.772 to 0.998 for DWI and 0.760 to 0.992 for ADC. Supplementary Figure 2 shows the DWI, ADC maps, and radiomics features of the two patients with similar clinical factors but different outcomes, highlighting the significance of radiomics in predicting AIS outcomes.

The radiomics model showed an AUC of 0.767 (95% CI 0.709–0.825) with balanced sensitivity and specificity of 0.750 and 0.659, respectively, in the training cohort. In the validation cohort, it yielded an AUC of 0.784 (95% CI 0.712–0.855) with a sensitivity and specificity of 0.750 and 0.695, respectively (Table 2, Fig. 2).

Table 2. Predictive Performance of Three Models in the Training and Validation Cohorts.

Model	Training Cohort (n = 311)			Validation Cohort (n = 211)
	AUC (95% CI)	Sensitivity*	Specificity*	AUC (95% CI)	Sensitivity*	Specificity*
Radiomics model	0.767 (0.709–0.825)	0.750	0.659	0.784 (0.712–0.855)	0.750	0.695
Clinical model	0.823 (0.775–0.871)	0.705	0.789	0.844 (0.788–0.900)	0.717	0.788
Clinical-radiomics model	0.868 (0.825–0.910)	0.739	0.861	0.890 (0.844–0.936)	0.817	0.841

^*Balanced sensitivity and specificity at the cutoff yielding the largest Youden index value. AUC = area under the receiver operating characteristic curve, CI = confidence interval

Fig. 2. ROC curves of the radiomics model, clinical model, and clinical-radiomics model in the training (A) and validation (B) cohorts.

ROC = receiver operating characteristic

Establishment and Performance of the Clinical Model

Multivariable logistic regression analysis showed that age, sex, stroke history, diabetes, mRS_baseline, and NIHSS_baseline were independent clinical predictors of AIS outcomes (p < 0.05) (Supplementary Table 1). The clinical model exhibited an AUC of 0.823 (95% CI 0.775–0.871) with balanced sensitivity and specificity of 0.705 and 0.789, respectively, in the training cohort and an AUC of 0.844 (95% CI 0.788–0.900) with a sensitivity and specificity of 0.717 and 0.788, respectively, in the validation cohort (Table 2, Fig. 2).

Establishment and Performance of the Combined Clinical-Radiomics Model

Independent clinical predictors, including sex, stroke history, diabetes, age, mRSb_aseline, and NIHSS_baseline were combined with the radiomics score by multivariable logistic regression to create a final clinical-radiomics prediction model. It exhibited an AUC of 0.868 (95% CI 0.825–0.910) and balanced sensitivity and specificity of 0.739 and 0.861, respectively, in the training cohort. In the validation cohort, it showed an AUC of 0.890 (95% CI 0.844–0.936) and sensitivity and specificity of 0.817 and 0.841, respectively (Table 2, Fig. 2).

The clinical-radiomics prediction model showed greater predictive performance than the clinical model in the training (vs. an AUC of 0.823, p = 0.004) and validation (vs. an AUC of 0.844, p = 0.004) cohorts. It also showed a greater performance than the radiomics model in both the training (vs. AUC of 0.767, p < 0.001) and validation cohorts (vs. AUC of 0.784, p = 0.002).

A nomogram was constructed based on this model (Fig. 3A). For each factor, we can obtain a point according to the patient’s clinical and radiomics information, and the total point corresponds to the risk of a poor outcome. Calibration curves and Hosmer–Lemeshow test (Fig. 3B) showed that the predicted probabilities of the nomogram were closely aligned with stroke outcome estimates in the training (p = 0.402) and validation cohorts (p = 0.764). DCA demonstrated that if the threshold probability in the clinical decision was greater than 0.06, using the nomogram to predict AIS outcomes provided a greater benefit than the clinical model (Fig. 3C).

Fig. 3. The clinical-radiomics nomogram for predicting acute ischemic stroke outcomes.

A. The developed nomogram based on the clinical-radiomics prediction model to predict the risk of poor stroke outcome. Diabetes: 0, no diabetes; 1, diabetes. Sex: 0, female; 1, male. Stroke history: 0, no stroke history; 1, stroke history; mRS_baseline: 0, ≤ 2; 1, > 2. B. Calibration curves for the nomogram in the training and validation cohorts. The green dashed line represents the ideal prediction and the red dashed line represents the predictive ability of the nomogram. The closer the dashed red line fit to the dashed green line, the greater the prediction accuracy of the nomogram. C. Decision curve analysis for the nomogram. The black line represents the net benefit of assuming no stroke patients have poor outcomes. The purple line is the net benefit of assuming all stroke patients have poor outcome. The orange line, green line, and blue line represent the expected net benefit of predicting stroke outcome using the clinical-radiomics model, clinical model, and radiomics model respectively. mRS_baseline = baseline modified Rankin Scale score, NIHSS_baseline = baseline National Institutes of Health Stroke Scale score

DISCUSSION

In this study, we established and compared the predictive performance of three models for AIS outcomes using a relatively large dataset. The prediction model incorporating clinical factors and radiomics scores achieved satisfactory prediction of AIS outcomes and outperformed the individual clinical and radiomics models in discriminatory ability. Based on the combined clinical-radiomics prediction model, we developed a novel nomogram for clinicians and used DCA to demonstrate its clinical validity.

It has been shown that interventions are not necessarily effective in all stroke patients [23]. There is an increasing demand to formulate individualized treatments for different stroke patients, especially for those who cannot receive thrombolytic therapy. Accurate and prompt risk prediction by the nomogram allows optimal management and treatment strategies for patients with AIS, and patients with poor outcomes could be treated more aggressively and urged to undergo more intensive rehabilitation. Patients with a predicted good outcome will cooperate with treatment with a more positive attitude, and overtreatment can be avoided.

The use of radiomics for stroke prognosis is promising. Qiu et al. [24] used it to anticipate patient recanalization and demonstrated that its performance was better than that of conventional thrombus imaging features, such as length, volume, and permeability. Cui et al. [25] fused radiomics features from six MR modalities with clinical factors to predict stroke outcomes. The time lag between onset and MRI scans was not clear in their study. The final AUC was > 0.8, but the total sample size was only 70, which is far smaller than the sample size of our study. Tang et al. [26] constructed an R score with radiomics features extracted from perfusion maps and DWI and found that the clinical-radiomics nomogram combining radiomics score, clinical information, and treatment options reached an AUC of 0.886 and 0.777 in predicting favorable outcomes at 7 days and 3 months, respectively, after onset. Nevertheless, there was no significant difference between the AUC of the clinical nomogram and clinical-radiomics nomogram, and the absence of long-term clinical assessment in the training dataset reduced the evidence level. Wang et al. [27] developed a clinical-radiomics nomogram including age, NIHSS score at 24 hours post-admission, hemorrhage, and radiomics score (just from DWI) to predict the 3-month outcome of AIS patients; the final AUC was 0.80 and 0.73 in the training and validation cohorts. The nomogram in our study incorporated more clinical factors such as sex, stroke history, and diabetes; additionally, we calculated the corresponding ADC maps based on DWI and extracted more radiomics features. The clinical-radiomics model in our study achieved an AUC of 0.868 and 0.890 in the training and validation cohorts, respectively, demonstrating its excellent predictive value.

Among the 14 radiomics features selected, GLN and wavelet feature cluster prominence were the best predictors; the former measures the similarity of gray-level intensity values in the image; a lower GLN value correlates with a greater similarity, and the latter implies asymmetry about the mean. Both represent the heterogeneity of the infarcts. Therefore, higher values indicate higher signal heterogeneity of the infarcted lesion, the possibility of lesion progression or complications induced by other pathological changes, and a worse outcome.

Regarding clinical factors, multivariable logistic regression analysis showed that age, sex, stroke history, diabetes, NIHSS_baseline and mRS_baseline were independent predictors of AIS outcome. Age, NIHSS_baseline, and mRS_baseline comprised a larger proportion of the nomogram than the other clinical features. The immunity of elderly patients is reduced, and various complications are prone to occur; the latter two reflect the severity of stroke. Several studies have demonstrated that these factors are more stable and effective predictors [28,29]. Diabetes leads to a multiorgan pathology, and hyperglycemia negatively affects fragile cerebral circulation during ischemia. It is associated with death and recurrence after stroke [30]. Stroke history is also related to adverse events in patients with stroke [31]. In our study, the number of women was much lower than that of men, which may be attributed to the protective effect of estrogen, which can decrease cerebrovascular tone and increase cerebral blood flow [32]. Nevertheless, consistent with the result of a previous study [33], women are more likely to get poor outcomes, attributed to an older age of onset, more severe stroke, less social support, a higher incidence of post-stroke depression, and immunosuppression after stroke [34].

Despite these interesting findings, our study had some limitations. First, there may have been a selection bias, as patients who were lost to follow-up were excluded. Second, this was a single-center study, and a prospective validation cohort from other hospitals is needed in future studies. Third, all radiomics features were extracted only from the lesion ROIs of DWIs and ADC maps, and the predictive performance of the nomogram may be better if more MRI sequences are used to extract features. Finally, genomic features were not considered in this study, based on the premise that genetically determined risk of depression has been shown to adversely affect stroke outcomes [35].

In conclusion, the clinical-radiomics model incorporating clinical factors and radiomics scores more accurately predicted the outcomes of AIS patients 6 months after hospital discharge than individual clinical or radiomics models. This could assist clinicians in formulating corresponding treatment plans at the early stage of onset, which may significantly improve the ultimate outcome of stroke patients.

Availability of Data and Material

The datasets generated or analyzed during the study are available from the corresponding author on reasonable request.

Supplement

The Supplement is available with this article at https://doi.org/10.3348/kjr.2022.0160.