Prediction of recurrent ischaemic stroke using radiomics data and machine learning methods in patients with acute ischaemic stroke: protocol for a multicentre, large sample, prospective observational cohort study in China

Abstract

Introduction

Stroke is a leading cause of mortality and disability worldwide. Recurrent strokes result in prolonged hospitalisation and worsened functional outcomes compared with the initial stroke. Thus, it is critical to identify patients who are at high risk of stroke recurrence. This study is positioned to develop and validate a prediction model using radiomics data and machine learning methods to identify the risk of stroke recurrence in patients with acute ischaemic stroke (AIS).

Methods and analysis

A total of 1957 patients with AIS will be needed. Enrolment at participating hospitals will continue until the required sample size is reached, and we will recruit as many participants as possible. Multiple indicators including basic clinical data, image data, laboratory data, CYP2C19 genotype and follow-up data will be assessed at various time points during the registry, including baseline, 24 hours, 7 days, 1 month, 3 months, 6 months, 9 months and 12 months. The primary outcome was stroke recurrence. The secondary outcomes were death events, prognosis scores and adverse events. Imaging images were processed using deep learning algorithms to construct a programme capable of automatically labelling the lesion area and extracting radiomics features. The machine learning algorithms will be applied to integrate cross-scale, multidimensional data for exploratory analysis. Then, an ischaemic stroke recurrence prediction model of the best performance for patients with AIS will be established. Calibration, receiver operating characteristic and decision curve analyses will be evaluated.

Ethics and dissemination

This study has received ethical approval from the Medical Ethics Committee of the Second Affiliated Hospital of Nanchang University (medical research review No.34/2021), and informed consent will be obtained voluntarily. The research findings will be disseminated through publication in journals and presented at conferences.

Trial registration number

ChiCTR2200055209.

Strengths and limitations of this study.

• This study is a multicentre prospective cohort with a relatively large sample size to predict recurrent ischaemic stroke in patients with acute ischaemic stroke (AIS).

• The model incorporates multimodal data such as clinical diagnosis and treatment variables, demographic information, radiomic features, biomarkers, CYP2C19 genotype and follow-up data.

• Advanced radiomics methods facilitate the extraction of numerous radiomic features, thereby improving clinical efficiency and providing new insights for clinicians in identifying the risk of AIS recurrence and determining the optimal timing of intervention.

• Recording of outcomes may have been heterogeneous across centres and, at least in part, relied on structured telephone interviews without neuroimaging follow-up.

Introduction

Stroke is a leading cause of mortality and disability worldwide.¹ The global lifetime risk of stroke was approximately 25%.² The pooled stroke recurrence rate was 7.7% at 3 months, 9.5% at 6 months and 10.4% at 1 year.³ Recurrence of stroke often leads to prolonged hospitalisation and worsened functional outcomes compared with the initial stroke.⁴ Therefore, the identification of high-risk patients for stroke recurrence is of utmost importance.

Currently, various scoring systems have been developed to predict stroke recurrence, each with its own validity to some extent. However, these systems have certain limitations. For instance, certain models such as Stroke Prognosis Instrument-I,⁵ Stroke Prognosis Instrument-II (SPI-II)⁶ and Hankey Score,⁷ suffer from limited patient sample sizes. Other models, like the Dutch transient ischemic attack (TIA) Trial⁸ and Life-Long After Cerebral Ischemia Trial,⁹ have attempted to improve accuracy by including more variables, but they lack validation. Some models, such as the Recurrence Risk Estimator at 90 days (RRE-90)¹⁰ and the Hankey Score, have incorporated imaging parameters to enhance performance, but their complex evaluation procedures may hinder the practical application. Furthermore, certain models, such as Essen Stroke Risk Score,¹¹ SPI-II and RRE-90, exhibit limited applicability to either short-term or long-term stroke recurrences, thereby restricting the overall performance of these models during the entire period of recurrence.

Several studies have investigated the addition of radiomic parameters, such as abnormal diffusion weighted imaging (DWI) features,¹² CT or transcranial Doppler features¹³ and carotid stenosis¹⁴ to the ABCD² (age, blood pressure, clinical weakness, duration, and diabetes) scoring system, resulting in a significant increase in the area under the curve (AUC). However, many of these predictive models lack validation in large cohorts.

Current models for predicting stroke recurrence have several limitations. First, the indicators included in these models are not comprehensive enough. Second, the evaluation of the included imaging indicators is complex and requires professionals to spend a long time. In addition, there is a trade-off between increased accuracy and enhanced ease of use in non-specialist clinical settings, preventing its widespread adoption and clinical value. Therefore, we need an improved prediction tool that can overcome these limitations of current models.

Machine learning, as an important branch of artificial intelligence, has shown great potential in recent progress in medicine, leveraging machine learning techniques to greatly enhance the performance of predictive models. Brain imaging holds crucial importance in both the diagnosis and prognosis of acute ischaemic stroke (AIS). Deep learning, as a subset of machine learning, has emerged as an immensely powerful tool for image processing in recent years, enabling direct learning from raw data and is easier to conduct and has high accuracy compared with hand-crafted approaches.^{15 16} The advent of radiomics has introduced a valuable adjunct to conventional radiological features through quantitative analysis of medical images.¹⁷ In recent years, researchers have harnessed the potential of radiomics in the diagnosis of stroke lesions,^{18 19} prediction of early outcomes^{20 21} and evaluation of long-term prognosis for stroke.^{22 23} Radiomics, a non-invasive method, can identify features that may be difficult to detect visually or with conventional imaging tools, enabling objective assessment of lesions and their heterogeneity.^{24 25} However, limited studies have explored the use of radiomics based on DWI for predicting stroke recurrence. The integration of radiomics into clinical prediction models holds considerable promise for enhancing timely decision-making in patients following a stroke. To achieve this objective, the present study protocol aims to develop and validate an automatic segmentation network model capable of automatically segmenting the images and a machine learning prediction model integrating radiomics features and clinical data to assess the risk of stroke recurrence in patients with AIS. The model will be developed to assess the risk of stroke recurrence in patients with AIS during various post-discharge time intervals, including 1 month, 3 months, 6 months, 9 months and 12 months. The implementation of this model holds the potential to predict the recurrence of AIS and identify individuals at high risk of poor outcomes, thereby improving secondary prevention strategies.

Study objectives

Primary objective

The primary objective of this study is to establish and validate a radiomics-based machine learning prediction model for the assessment of stroke recurrence risk in patients diagnosed with AIS. Developing and validating a programme that can automatically segment the images and extract radiomics features is an important step.

Secondary objective

The secondary objectives are to explore potential risk factors and protective factors that influence the prognosis and safety of patients with AIS.

Methods

Study design

This study is a multicentre, large sample, prospective observational cohort study. The details of the study procedure are shown in figure 1.

Figure 1.

Flow chart of the study design. AIS, acute ischaemic stroke; mRS, modified Rankin Scale.

Study setting

This study is being conducted at 14 comprehensive hospitals in Jiangxi province. Multiple indicators will be assessed at various time points, including baseline, 24 hours, 7 days (if the hospitalisation period is less than 7 days, the discharge date will be considered as the seventh day), 1 month, 3 months, 6 months, 9 months and 12 months during the registry period. The registry is sponsored and conducted by The Second Affiliated Hospital of Nanchang University (China), which is also responsible for data analysis. An independent Data and Safety Monitoring Board (DSMB) oversees the conduct, safety and efficacy of this registry. The study is scheduled to commence in January 2022 and is expected to be completed by December 2024.

Study sample and recruitment

Study participants

The study will recruit participants from neurology departments at the participating hospitals, with AIS volunteers being selected by professional researchers. AIS will be defined as acute neurological impairment caused by focal brain ischaemia, lasting more than 24 hours, or with evidence of acute infarction on brain imaging, irrespective of symptom duration.²⁶

Inclusion criteria

Patients who match all of the criteria listed below will be considered for enrolment in this study:

1. 18–85 years old.

2. Confirmed ischaemic stroke.

3. Acute onset within the period (2 weeks).

4. Voluntary participation and signing informed consent.

Exclusion criteria

Patients who match any of the following criteria will be excluded from participating in this trial:

1. Any malignant tumour diagnosis.

2. Haemorrhagic stroke.

3. Transient ischaemic attack.

4. Traumatic brain injury.

5. Severe cognitive impairment.

6. Participating in any clinical trials involving investigational drugs or medical devices currently.

7. Poor adherence or inability to complete long-term follow-up.

8. With major artefacts on the DWI.

Patient involvement statement

Patients or the public were not involved in the formulation of the research question, the study design or the recruitment or conduct of the study.

Data collection

The clinical data collected in this study encompass the following aspects: basic clinical data, image data, laboratory data, CYP2C19 genotype and follow-up data. Table1 provides a comprehensive overview of the participants’ study schedules.

Table 1.

The study schedule for data collection

Timing	1 day	7 days	1 month (±7 days)	3 months (±7 days)	6 months (±7 days)	9 months (±7 days)	12 months (±7 days)
Informed consent	●
Basic clinical data
Demographics	●
Risk factors	●
Family/personal disease history	●
Medications	●	●	●	●	●	●	●
Physiological tests	●	○
Neurological assessment
TOAST subtypes	●
GCS	●	●
NIHSS	●	●
MRS	●	●	●	●	●	●	●
Image data	●	○
Laboratory data	●	○
CYP2C19 genotype		●
Follow-up data
Stroke recurrence			●	●	●	●	●
Death events			●	●	●	●	●
Adverse events			●	●	●	●	●

●: required, ○: optional.

GCS, Glascow Coma Scale; mRS, modified Rankin Scale; NIHSS, National Institutes of Health Stroke Scale; TOAST, Trial of Org 10 172 in acute stroke treatment.

Basic clinical data

Basic clinical data includes (1) demographics (age, sex, phone number); (2) risk factors (smoking, alcohol, hypertension, diabetes, heart diseases, hyperlipaemia); (3) family/personal disease history; (4) medication (antiplatelet drugs, anticoagulant drugs, hypoglycaemic drugs, antihypertensive drugs); (5) physiological tests (body mass index (BMI), blood pressure, pulse, respiratory rate); (6) neurological assessment (TOAST subtypes, Trial of ORG 10172 in Acute Stroke Treatment; NIHSS, National Institutes of Health Stroke Scale; mRS, modified Rankin Scale; GCS, Glascow Coma Scale).

Image data

Brain imaging includes MRI, CT and Computed tomography angiography (CTA) / Magnetic Resonance Angiography (MRA) / Digital Subtraction Angiography (DSA). All patients underwent an MRI brain imaging within 48 hours of admission. All images were acquired using 3 Tesla MRI scanners.

Imaging reports of the vessels include stenosis location (internal carotid artery, middle cerebral artery, arteriae cerebral artery, posterior cerebral artery, vertebral artery, subclavian artery, common carotid artery, basilar artery, innominate artery), extent of the stenosis and infarction side (left, right or bilateral).

Laboratory data

Laboratory results, including blood routine (white blood cell count, red blood cell count, etc), liver function (total protein, albumin, etc), kidney function (urea, creatinine, etc), muscle enzyme profile (creatine kinase, lactate dehydrogenase, etc), blood lipid profile (total cholesterol, triglycerides, etc), blood clotting function (prothrombin time, prothrombin activity, etc), blood glucose (fasting blood glucose, etc), biomarkers (C-reactive protein, homocysteine, etc).

CYP2C19 genotype test

CYP2C19 genotype was the main determinant of pharmacodynamic response to clopidogrel in healthy volunteers. According to the *2, *3 and *17 genotypes, patients were classified as poor metabolisers, intermediate metabolisers, extensive/normal metabolisers and ultra-metabolisers.^{27 28}

Data management

1. All patient data will be securely stored in a database. The electronic data capture (EDC) was used for data management.

2. Study medical record design: An electronic case report form (eCRF) was constructed according to the study protocol and study medical records.

3. On confirmation and approval by the total investigator, the system administrator creates the project administrator accounts and grants them project administration authority. Project administrators apply to create accounts of investigator, clinical research coordinator (CRC), monitor, inspector and data manager with varying permissions for EDC access.

4. Data entry: CRC is responsible for timely and accurate data entry from the study medical records into the eCRF.

5. Data check: During data entry, the EDC system performs logical checks and generates questions from the system in real time. In addition to queries from the system, the data manager manually checks text data.

6. Source data site check: Monitors log into the EDC at each research site to conduct a 100% review of eCRF data against source data, such as research medical records.

7. Question answering: Queries arising from data review were sent to the investigators. After investigators responded to queries, data managers revised the data on the basis of returned responses.

8. Data lock and export: Once all subjects have completed the trial and all medical records have been entered into the system, the data manager locks the data after review and confirmation by the principal investigator, sponsor, statistical analyst and data manager that the established database is accurate. After the data lock, the data manager imports the data into the designated database and submits it to the statistician for statistical analysis.

9. eCRF archive: After the trial, the eCRF for each subject is generated as a PDF electronic document, and a burned CD is stored.

10. EDC closure: After the trial concludes, the data manager applies for EDC closure, cancels all account access rights on obtaining total investigator’s permission and proceeds to close the EDC after ensuring full data backup.

Primary outcomes

The primary outcome assessed in this study was stroke recurrence within a 1-year period, which included variables such as recurrence occurrence, recurrence frequency, recurrence time and stroke recurrence type. Patients were scheduled for follow-up assessments at 1, 3, 6, 9 and 12 months after their discharge from the hospital. The index event was defined as the first symptomatic AIS during the acute period. Recurrent AIS was defined as a new sudden focal neurological deficit after discharge, which was confirmed on imaging (CT or MRI) examination. The endpoint of this study was the occurrence of recurrent AIS.

Secondary outcome

The secondary outcomes evaluated in this study included all-cause mortality, mRS score (with poor prognosis defined as mRS score ≥3²⁹) and adverse events (including adverse event type, severity, start time, stop time, etc) within 1 year. These outcomes were assessed at 1, 3, 6, 9 and 12 months following discharge from the hospital.

Study procedures

The details of the study procedures are shown in figure 1. Patients who meet the described inclusion and exclusion criteria will sign an informed consent form before being included. During the registry, various indicators will be reviewed for all patients at baseline, 24 hours, 7 days, 1 month, 3 months, 6 months, 9 months and 12 months. This registry was sponsored and administered by the Second Affiliated Hospital of Nanchang University (China), which was also in charge of data analysis. An independent DSMB oversees the registry’s operation, safety and efficacy.

Quality control

The overall framework of this study consists of the admission group, data collection group, follow-up group and quality control group.

1. The admission group comprises neurology clinicians responsible for screening research subjects daily based on the admission criteria. Simultaneously, quality control personnel review the patient’s medical history and imaging reports after enrolment to ensure compliance with the study’s inclusion and exclusion criteria. If a patient is found not to meet the criteria, he or she is promptly eliminated.

2. The data collection group performs data entry at the end of each month, including basic clinical data, image data, laboratory data, CYP2C19 genotype and follow-up data for enrolled patients.

3. The follow-up group conducts regular telephone follow-ups with patients at 1, 3, 6,9 and 12 months after discharge. Additionally, the quality control group randomly selects 30% of the patients, based on the number of follow-up patients at the end of each month, to conduct further telephone follow-ups and verify the accuracy of the follow-up results.

4. The system automatically detects illogical data entries and imposes restrictions.

Statistical analysis

Sample size calculation

We calculated the number of patients needed according to the requirements for sample size calculation of the survival analysis prediction model by Riley et al.³⁰ Based on a previous study,³¹ the AIS recurrence rate within 1 year was 13.08%, the AUC of the model was 0.72. When 20 variables were expected to be included in the model. The ‘pmsampsize’ function of R for Windows (V.4.1.2) was used to calculate the modelling sample size of 1565 cases. Based on previous follow-up experience and given an anticipated dropout rate of 20%, a total of 1957 patients with AIS will be needed. Enrolment at participating centres will continue until the sample size is sufficient.

Clinical data analysis

In this study, statistical analyses were conducted using R software (V.4.2.1), Python software (V.3.0) and IBM SPSS software (V.26.0). Normally distributed continuous variables were presented as mean±SD, while non-normally distributed continuous variables were presented as median with IQR. Student’s t-test was applied to compare normally distributed variables, and the Mann-Whitney U test was used for non-normally distributed variables. Categorical variables were presented as frequencies and percentages, and the χ² test was used to compare these variables between the two groups. Multivariate Cox regression analysis will be used for the time-to-event data to determine the independent clinical predictors of outcomes. For all statistical tests, a p<0.05 was considered statistically significant.

Prediction model development

MRI image acquisition

The DWI-MRIs of all patients were acquired using 3 Tesla MRI scanners using a repetition time of 4090 ms, an echo time of 98.0 ms, a field of view=230 mm×230 mm, a matrix of 192×192, a slice thickness to gap ration of 5 mm/1.5 mm, a b value of 0 and 1000 s/mm².

Infarct lesion segmentation and image processing

Two radiologists, each with 5 years of experience, used the 3D Slicer software (V.4.13.0, https://www.slicer.org) to individually annotate the ischaemic lesions on the DWI images, to validate the consistency of the annotations. The feature extraction and selection process were performed using Python software. The agreement between the annotated volumes was assessed using the intraclass correlation coefficient (ICC). In cases where the ICC was below 0.75, a third imaging specialist was involved to label and verify the annotations. Only the radiomics features with satisfactory reproducibility (ICC>0.75) were chosen for subsequent analysis.

Automatic infarct lesion segmentation model construction

We randomly divided the participants into training and testing groups. Then, we trained an automatic segmentation network on the images using Convolutional Neural Network (CNN) models, including UNet++, FCN and AlexNet. A subset of 500 cases was randomly selected, and the ICC, Pixel Accuracy (PA), Dice Coefficient, etc, were computed to evaluate the concordance between the radiologists’ annotations and the automatic segmentation results obtained from the network, thereby assessing the performance of the automatic segmentation approach. By comparing the performance of different deep learning methods, we identified the optimal model for automatic brain image segmentation. Based on this model, we further segmented all the cases in the data set.

Radiomics feature extraction and selection

Radiomics features were extracted from the annotated DWI images using the PyRadiomics V.3.0.1 software. The radiomics features are listed in table 2. The Pearson intergroup correlation coefficient was used to calculate the correlation and redundancy. Features with correlation coefficients greater than 0.8 were excluded. Subsequently, the optimal radiomics features for the prediction of AIS were screened using the least absolute shrinkage and selection operator (LASSO) with 10-fold cross-validation. Eventually, the radiomics score was calculated for each patient based on the linear combination of weighted selection parameters for the relevant LASSO coefficients of the optimal feature. Other feature selection methods such as recursive feature elimination, principal component analysis and random forest will be used for comparison.

Table 2.

Components and amounts of the radiomics features

Radiomics feature		Amount
Shape features (3D)		18
First-order features		18
Texture features	GLCM	24
	GLRLM	16
	GLSZM	16
	NGTDM	5
	GLDM	14
Wavelet filtering features		744
Laplacian of Gaussian filter features		186

3D, three-dimensional; GLCM, Grey Level Co-occurrence Matrix; GLDM, Grey Level Dependence Matrix; GLRLM, Grey Level Run Length Matrix; GLSZM, Grey Level Size Zone Matrix; NGTDM, Neighbouring Grey Tone Difference Matrix.

Prediction model construction

The flowchart of model construction was shown in figure 2. In this work, the performances of different machine learning methods were compared: support vector machine, logistic regression, eXtreme Gradient Boosting and random forest. The participants were randomly divided into a training cohort and a test cohort. For each algorithm, three prediction models were constructed. Finally, a combined prediction model was built based on both radiomics features and clinical predictors, as well as a clinical prediction model containing only the clinical predictors and a radiomics prediction model containing only the radiomics features were built for comparison. The predictive performance of all three models was compared by calculating the AUC of a receiver operating characteristics curve (ROC), the accuracy, sensitivity, specificity, negative predictive values and positive predictive values. The Delong test was used to compare the differences in ROC curves between the models. The clinical application value of the prediction models was evaluated using a calibration curve and a decision curve analysis. In order to test the generalisation ability of the model, the stratification analysis was performed on the subgroups of sex, age, BMI, the responsible vessel of the infarction and the size of the infarction. For all statistical tests, a p value below 0.05 was considered statistically significant.

Figure 2.

The flowchart of model construction.

Ethics and dissemination

All participating centres have approved the study protocol. Ethical approval for this study was obtained from the Medical Ethics Committee of the Second Affiliated Hospital of Nanchang University (medical research review No.34/2021). This trial has been registered in the Chinese Clinical Trial Registry, with registration completed on 3 January 2022. Patients are required to voluntarily sign an informed permission form stating their willingness to participate in the study. Informed consent will be sought from all study participants. The results will be presented at conferences and published in peer-reviewed publications.

Ethics statements

Patient consent for publication

Not applicable.