A score to predict early risk of recurrence after ischemic stroke

Abstract

Background:

There is currently no instrument to stratify patients presenting with ischemic stroke according to early risk of recurrent stroke. We sought to develop a comprehensive prognostic score to predict 90-day risk of recurrent stroke.

Methods:

We analyzed data on 1,458 consecutive ischemic stroke patients using a Cox regression model with time to recurrent stroke as the response and clinical and imaging features typically available to physician at admission as covariates. The 90-day risk of recurrent stroke was calculated by summing up the number of independent predictors weighted by their corresponding β-coefficients. The resultant score was called recurrence risk estimator at 90 days or RRE-90 score (available at: http://www.nmr.mgh.harvard.edu/RRE-90/).

Results:

Sixty recurrent strokes (54 had baseline imaging) occurred during the follow-up period. The risk adjusted for time to follow-up was 6.0%. Predictors of recurrence included admission etiologic stroke subtype, prior history of TIA/stroke, and topography, age, and distribution of brain infarcts. The RRE-90 score demonstrated adequate calibration and good discrimination (area under the ROC curve [AUC] = 0.70–0.80), which was maintained when applied to a separate cohort of 433 patients (AUC = 0.70–0.76). The model's performance was also maintained for predicting early (14-day) risk of recurrence (AUC = 0.80).

Conclusions:

The RRE-90 is a Web-based, easy-to-use prognostic score that integrates clinical and imaging information available in the acute setting to quantify early risk of recurrent stroke. The RRE-90 demonstrates good predictive performance, suggesting that, if validated externally, it has promise for use in creating individualized patient management algorithms and improving clinical practice in acute stroke care.

GLOSSARY

AUC

= area under the ROC curve;

CI

= confidence interval;

DWI

= diffusion-weighted imaging;

ROC

= receiver operating characteristic;

RRE-90

= recurrence risk estimator at 90 days.

The risk of recurrent stroke is highest during the first 90 days after an index stroke; longitudinal studies indicate that approximately 1 out of every 2 recurrences occurring in the first year occurs within the first 90 days.^1–5 Prevention of such events is critical because early recurrence is associated with severe consequences including longer duration of hospitalization and increased neurologic disability and death.^6,7 Several recent studies consistently indicate that early initiation of available treatments after TIA or minor stroke is imperative to avoid missed opportunities to prevent a recurrent stroke.^8–10 Nevertheless, prioritization occurs in clinical practice because diagnosis and treatment of stroke is often constrained by queues, limited resources, or even by patient reluctance.¹¹ Because the majority of patients will not have a subsequent ischemic event after stroke, a tool that could identify patients at high risk of early recurrent stroke, in whom urgent evaluation and intervention is most justified, would be of tremendous benefit. There is currently no robust prognostic tool for predicting the short-term risk of recurrent stroke. Available prognostic models such as the Stroke Prognosis Instrument II¹² and The Essen Stroke Risk Score¹³ are designed to predict the long-term risk and have not been validated for short-term risk prediction. Our goal was to develop a predictive score based on information typically available to the physician at the time of hospital admission to estimate the 90-day risk of recurrent stroke.

METHODS

Patient population.

The primary derivation cohort was retrospectively identified from consecutive patients with ischemic stroke who were admitted within 72 hours of stroke onset between 2003 and 2006 to a single center. We collected data on previously published clinical1–7,14–18 and imaging features^7,19–23 of index stroke associated with increased risk of recurrent events through medical record search (table 1). MRI analysis was performed by visual assessment of images obtained at the time of index stroke blinded to the patient's recurrence status. Table 2 presents the definitions used for each imaging feature. The image acquisition parameters were summarized in detail elsewhere.²⁴ For each imaging feature, we calculated interexaminer reliability based on assessment of 200 consecutive patients by 2 examiners blinded to each other's assignments.

Table 1 Baseline clinical and imaging characteristics

Table 2 Definitions for imaging features associated with stroke recurrence at 90 days

Two investigators retrospectively acquired 90-day follow-up information through review of data collected within the context of a prospective study that assessed clinical outcome by phone interview²⁵ and inspection of medical records and the social security death index. A separate neurologist adjudicated recurrent events by evaluating all pertinent brain images without the knowledge of clinical and diffusion-weighted imaging (DWI) characteristics of the index stroke.

All patients underwent an assessment of medical history, physical examination, brain CT/CT angiography or MRI/magnetic resonance angiography, EKG, CBC, and blood chemistry. The stroke etiology was classified according to the Causative Classification of Stroke System¹⁸ using information available after initial line of tests listed above.

Standard protocol approvals, registrations, and patient consents.

The study protocol received approval by the local institutional review board.

Generation of the predictive model and statistics.

The study end point was time to last known to be free of recurrent stroke or nonstroke death. We considered that the end point was reached after the first event in patients who had multiple recurrent events within 90 days. We defined recurrent stroke as a clinical incident that is clearly attributable to a new area of brain infarction visualized by imaging as spatially distinct from the index lesion.

We used univariate Cox regression to identify baseline differences in clinical and imaging variables between patients with or without recurrent stroke. We determined the risk of subsequent stroke in relation to each variable by multivariable Cox regression analysis. The regression model included time to recurrent stroke as response and imaging and clinical predictors of recurrence with a univariate p value <0.05 as independent variables. We first generated a clinical-based model (model A) based on pertinent clinical data in the univariate analyses (table 3). We then constructed a second model that included important clinical and imaging information available to the physician at the time of hospital admission. This was called clinical- and imaging-based model or model B (table 3).

Table 3 Independent predictors of 90-day risk of stroke

All covariates except for etiologic stroke subtype were introduced into the regression model as dichotomous variables. The latter was introduced as dummy variables using small artery occlusion as the reference category (table 3). We evaluated each independent variable for adherence to the assumption of proportional hazards and examined the data for collinearity. We assigned scores for each predictor variable generated by rounding the corresponding β-coefficient from the regression model to the nearest integer. We calculated an overall risk score by summing up scores for each independent predictor for a given patient. We quantified the predictive validity of the model by computing the receiver operating characteristic (ROC) curves in patients with 90-day follow-up,²⁶ and compared the area under the ROC curve (AUC) for different models using the Z-test.^26,27 We evaluated the calibration of our models as a measure of agreement between observed risk (estimated risk from the Kaplan-Meier curve) and predicted risks stratified according to risk scores using Hosmer-Lemeshow χ² statistic.

Validation of the predictive model.

We used cross-validation to provide an unbiased internal assessment of the model's accuracy. For this, the whole dataset was randomly partitioned into 2 halves 1,000 times, each time testing one half of the population using integer scores obtained from the coefficients trained by the other half.²⁸ We further validated the model in a separate cohort of consecutive stroke patients admitted to the same institution between 2000 and 2003. We applied the scores generated from the derivation cohort to the validation cohort, computed an ROC curve, and compared its AUC to the original model for significance.

RESULTS

Study population.

The derivation cohort for model A comprised 1,458 consecutive patients. MRI was not available in 201 of the 1,458 patients because of contraindications¹³⁵ or because the treating physician did not consider it clinically necessary.⁶⁶ The imaging-based model (model B) was, therefore, generated in 1,257 patients. Baseline demographics and clinical characteristics of the derivation cohort are summarized in table 1.

Follow-up events and the prognostic model.

Follow-up information was acquired through phone call assessment in 392 and inspection of hospital visit notes in 1,066 patients. Complete 90-day follow-up was available in 806 patients. The remaining either died (213 patients) or had data on follow-up assessment between discharge and 90 days (439 patients). None of the 213 patients who died had recurrent stroke during the follow-up period. In the group with incomplete follow-up, time to follow-up assessment ranged between 3 and 87 days (mean: 14 ± 18 days). Overall, 75% of patient-days in the study were taken into account in the Cox regression model. There was no difference between patients with and without complete follow-up with respect to RRE-90 scores and predictors of early stroke recurrence in table 1 except for isolated cortical infarcts, which were more common in those with complete follow-up (table e-1 on the Neurology® Web site at www.neurology.org). A total of 60 patients developed recurrent ischemic stroke; 4 did not have MRI within the first 72 hours and 2 had recurrent stroke before brain MRI was performed. Analyses that included imaging data were therefore based on 54 recurrence events. The median time from symptom onset to MRI was 11.8 hours (IQR, 5.8–27.1 hours). Thirty patients developed recurrent stroke within the first 14 days. The risk of recurrence adjusted for time to follow-up was 2.6% (95% confidence interval [CI], 1.6%–3.5%) at 14 days and 6.0% (95% CI, 4.5%–7.5%) at 90 days. All univariate clinical and imaging predictors retained significant effect on 90-day risk of recurrent stroke in Cox regression model (table 3). A forward selection model including all baseline variables listed in table 1 identified the same predictors with the Cox regression model.

Both model A and model B fulfilled the proportional hazards assumption and were significant at a p value of <0.001 (log likelihood test). The total predictor degrees of freedom was 5 for 60 outcome events in model A, and 6 for 54 outcome events in model B. The β-coefficients and corresponding integer scores for each independent predictor are presented in table 3. Table 4 shows total point scores and corresponding adjusted risks for models A and B (an automated calculator is available at http://www.nmr.mgh.harvard.edu/RRE-90/). The risk of recurrent stroke at 90 days rose with increasing number of independent predictors in each model (p < 0.001, log-rank test). The calibration χ² statistics revealed a p value for lack of fit of 0.933 for model A and 0.092 for model B, indicating adequate calibration. In the derivation cohort, 8.2% of patients with a score <3 by model A had a score ≥3 according to model B whereas 57% of patients with score ≥3 per model A had a score <3 by model B.

Table 4 Recurrence risk estimator at 90 days (RRE-90) score and risk of recurrent stroke

Evaluation and validation of the predictive model.

ROC analysis was performed in patients with complete 90-day follow-up (n = 728) using the scores generated from the Cox regression model. The AUC was 0.70 (95% CI, 0.63–0.77) for model A and 0.80 (95% CI, 0.73–0.86) for model B (p = 0.048).

The validation dataset comprised 433 consecutive patients and was similar to the derivation cohort except that more patients received anticoagulation in the former (60% vs 41%, p < 0.001). Twenty-six patients developed a recurrent ischemic stroke. The AUC was 0.70 (95% CI, 0.61–0.79) for model A, and 0.76 (95% CI, 0.66–0.87) for model B. There was no difference in AUC between validation and derivation cohorts (p = 0.986 for model A and p = 0.553 for model B). The predicted risks rose as the score increased in the validation dataset (p < 0.001 for linear trend). Internal validation analysis for model B revealed that the mean cross-validated AUC was 0.77 (95% prediction interval, 0.70–0.84), suggesting that the bias coming from predicting on the same dataset used for fitting was approximately 3%.

A time epoch analysis showed that the performance of model B did not significantly change with respect to time from onset to MRI; the AUC was 0.78 (95% CI, 0.71–0.86) for 24 hours and 0.79 (95% CI, 0.72–0.86) for 48 hours. There were 39 patients with a recurrent event within 14 days of the index stroke in both derivation and validation cohorts. The AUC for predicting 14-day risk was 0.80 (95% CI, 0.72–0.87). There were 271 minor (NIH Stroke Scale <4) and 457 major (NIH Stroke Scale ≥4) stroke patients in the derivation dataset. The AUC was 0.83 (95% CI, 0.74–0.93) in minor and 0.78 (95% CI, 0.69–0.86) in major strokes. The AUC for short-term recurrence risk in our dataset was 0.56 (95% CI, 0.49–0.64) for Stroke Prognosis Instrument II and 0.59 (95% CI, 0.53–0.66) for Essen Stroke Risk Score.

DISCUSSION

The current results provide evidence in 5 important domains. First, our findings confirm the published evidence that conventional risk factors for long-term recurrence such as hypertension, diabetes mellitus, and smoking do not confer a risk for stroke over the short term (90 days).^1–7,14 Scores based on predictors of the long-term risk such as the Stroke Prognosis Instrument II and the Essen Stroke Risk Score, therefore, offer only limited predictive value in the short term. Second, as also reported previously, etiologic stroke subtype6,7,16,17,29 and prior history of recent TIA/stroke¹⁷ are significant clinical predictors of 90-day recurrence risk after ischemic stroke. Nevertheless, the accuracy of predictions based on these predictors alone is modest (AUC = 0.70). Third, predictions by clinical variables can be significantly improved when imaging information is taken into account. The resultant score is called recurrence risk estimator at 90 days, or RRE-90. RRE-90 demonstrates good discrimination (AUC = 0.80) and calibration for predicting 90-day risk of recurrent stroke. Fourth, approximately 50% of recurrent strokes occurring in the first 90 days happen within the first 14 days. The RRE-90 provides good discrimination for predicting 14-day risk of recurrence (AUC = 0.80). Finally, the diagnostic performance of the RRE-90 is maintained after cross-validation and when applied to an independent cohort (AUC = 0.76), suggesting that it has promise when advanced to multicenter validation phase.

Various prior studies have also explored the relationship between imaging characteristics and stroke recurrence. However, while doing this, a significant proportion of such studies either failed to provide any imaging confirmation of recurrent stroke or used much less sensitive imaging methods such as computerized tomography.1,2,16,30 The remaining have chosen not to evaluate multiple imaging features in simultaneous context with each other^1,16,17 or assessed a small number of outcome events for the number of independent variables in multivariable analyses.^16,17 The current study undertakes a simultaneous assessment of all recognized imaging and clinical features associated with early recurrence. A major strength of the present study is the use of MRI for confirming the adjudication of recurrent stroke. Conventional definitions based on clinical diagnosis of a temporally distinct event often fail to differentiate a recurrent stroke from an event that is caused by progression or local complications (edema, seizures) of index stroke,³¹ hampering the specificity of diagnoses. Brain imaging allows objective assessment of clinical events as to whether a new event is in fact caused by recurrent infarct, and is routinely used for this in clinical practice. The choice of imaging in the current study was MRI because MRI, in particular DWI, is markedly superior to other imaging techniques in the evaluation of small ischemic lesions as well as in differentiating acute infarcts from chronic lesions.^32,33 Although recent evidence suggests that routine use of MRI in acute stroke is justified,³³ MRI suffers from limited accessibility and applicability. In order to ensure utility in circumstances in which MRI is not readily accessible, the RRE-90 automatically allows risk predictions using only the available clinical data (model A) yet predictions based on clinical data alone are less optimal than those from clinical and imaging-based model (model B).

Although etiologic stroke subtype is a significant predictor for short-term recurrence risk,^6,7,16 this knowledge has thus far not been incorporated explicitly into predictive models. One reason for this is that identification of etiologic stroke subtype requires a comprehensive stroke evaluation and depends on the depth and speed of etiologic stroke investigation, which varies considerably across individual practices.^34,35 It is therefore possible that a recurrent stroke may occur before an accurate subtyping is done. The major premise of the current tool is its ability to allow risk stratification with information available to physician immediately after initial stroke evaluation. Baseline imaging features of infarcts provide the prognostic information that relates to the underlying stroke mechanism. For instance, simultaneous infarcts in multiple circulations often indicate an unstable embolic source that is more proximal than carotid arteries. Infarct topography differentiates isolated cortical infarcts caused by small emboli from an unstable source from isolated subcortical or deep lesions caused by local small artery disease. Multiple acute infarcts often specify factors that simultaneously affect more than one artery such as proximal embolism and vasculitides. Other components of the RRE-90 such as recent history of TIA/stroke and multiple infarcts of different ages provide valuable temporal information that there is a continued risk of recurrent events. Because etiologic subtypes represent a combination of heterogeneous conditions with substantial variation in baseline individual risk, the continuity information plays a key role in discriminating whether or not the underlying stroke mechanism is unstable.^36,37

Strengths of the present study include large sample size, imaging-based objective definition of outcome events, blind assessment of covariates to the outcome events, and Web-based availability of the final predictive model. Limitations include retrospective design, lack of external validation, incomplete follow-up, and single hospital setting where referral bias can potentially occur. Despite a rigorous derivation process, there were patients with missing follow-up information. This, however, unlikely caused a systematic bias toward selection of a particular risk population because most baseline predictors and distribution of RRE-90 scores were similar in cohorts with or without complete follow-up. Nevertheless, external validation is critical for the generalizability of our results. The number of recurrent events per predictor variable in model B was smaller than recommended,³⁸ suggesting that overfitting might have occurred. Likewise, we cannot exclude the possibility of overfitting during the cross-validation procedure because coefficients were generated using only half of the dataset. Small number of recurrent events might have also caused missing of important differences in model performance in the validation dataset. Because the RRE-90 tool was constructed using only baseline clinical and imaging data, it does not take into account the intensity and choice of preventive stroke treatment. Differences in predictive performance of the algorithm may occur in external settings where timing and type of preventive stroke treatments substantially differ. The use of stringent univariate p value threshold for eligibility into multivariable models limited our ability to test all potential predictors. Future studies with larger datasets could address whether incorporation of additional risk factors further improve predictions. Finally, the use of a liberal time window to obtain MRI (72 hours) as opposed to earlier time points may have caused more frequent detection of multiple acute infarcts on baseline MRI as a result of accumulation of ischemic lesions over time. Nevertheless, the time epoch analysis that revealed that the diagnostic performance of the model did not significantly change with respect to time from symptom onset to MRI strongly argues against this.

Risk stratification tools like RRE-90 offer utility in improving stroke care and outcome, because such tools have the potential to boost the development of stroke management algorithms that are based on individual patient characteristics. For instance, admission to specialized stroke centers with necessary infrastructure for prompt etiologic investigation and preventive treatment such as early carotid endarterectomy may offer greater benefit in high-risk patients whereas elective evaluation and management may be justified in low-risk patients in settings where resources are limited. Care at specialized centers can also provide an added benefit from the opportunity to administer treatment timely in the event of a recurrent stroke in high-risk patients. Stratification systems like RRE-90 could also serve as a tool for use in clinical trials testing new preventive strategies. Although 90-day recurrence is not a typical outcome measure in stroke prevention trials, there are several preventive treatments applied at the acute setting (anticoagulation, combination antiplatelets, endovascular procedures) with modest benefit but significant risk or cost that necessitate assessment in the short term for a more targeted approach. The RRE-90 tool can provide an excellent opportunity for this if validated externally. Prospective demonstration that the use of RRE-90 improves current practice and research in acute stroke remains to be determined.

AUTHOR CONTRIBUTIONS

Statistical analysis was conducted by Drs. Vangel and Arsava.

DISCLOSURE

Dr. Ay receives research support from the NIH [R01-NS059710 (PI)]. Dr. Gungor and Dr. Arsava report no disclosures. Dr. Rosand receives research support from the NIH [R01-NS059727 (PI)] and from the American Heart Association. Dr. Vangel reports no disclosures. Dr. Benner serves as a consultant for Siemens Medical Solutions and Bayer Schering Pharma. Dr. Schwamm serves on scientific advisory boards for CoAxia, Inc. and Phreesia; has received honoraria and funding for travel for lectures or educational activities not funded by industry; serves on the editorial board of Neurocritical Care; may accrue revenue on US Patent 6,542,769 (filed 2003): Imaging system for obtaining quantitative perfusion indices; his spouse receives royalties from publishing Obstetric Anesthesia (Cambridge Pocket Clinicians, 2007–2009); serves as a consultant to Medtronic, Inc./CryoCath, Research Triangle Inc., and the Massachusetts Department of Public Health; serves on an external scientific review committee for the Canadian Stroke Network; and receives research support from Forest Laboratories, Inc., the NIH [NINDS P50 NS051343-01 (Co-I), NINDS U01 NS052220 IMS-3 (Site PI), and NCRR 1 UL 1 RR025758-01 (CRC Staff)] and the DHHS/CDC [5 U13 DP001176-02 (PI)]. Dr. Furie has served on scientific advisory boards for Novartis and GE Healthcare and receives research support from the NIH/NINDS [R01-HS011392 (PI) and P50-NS051343 (PI)], the American Heart Association, and the Deane Institute. Dr. Koroshetz is a full-time employee of the NIH and holds stock in NeuroLogica. Dr. Sorensen has served on scientific advisory boards for Olea Medical and Breakaway Imaging; has received funding for travel from Genentech, Inc., Siemens Medical Solutions, Millennium Pharmaceuticals, Inc., AstraZeneca, and for speaking and educational activities not funded by industry; serves as a Section Editor of Stroke and on the editorial boards of The Oncologist and the Journal of Clinical Oncology; may accrue revenue on US Patent 7020578 (issued 2001): Method for evaluating novel, stroke treatments using a tissue risk map, US Patent 6542769 (issued 2000): Imaging system for obtaining quantitative perfusion indices, US Patent 10558343 (issued 2004): Delay-compensated calculation of tissue blood flow, US Patent 11075990 (issued: 2005): High-flow oxygen delivery system and methods of use thereof, and US Patent 11417769 (issued: 2006): Magnetic resonance spatial risk map for tissue outcome prediction; receives royalties from publishing Cerebral MR Perfusion Imaging (Thieme, 2000); has received honoraria from Siemens Medical Solutions, Novartis, GE Healthcare, and for speaking and educational activities not funded by industry; has served as a consultant to Mitsubishi Tanabe Pharma Corporation, AstraZeneca, and Genentech, Inc.; receives research support from Millennium Pharmaceuticals, Inc., Siemens Medical Solutions, AstraZeneca, Genentech Inc., Novartis, Merck Serono, Schering Plough Corp, the NIH [NINDS NS38477 (PI), NCI CA137254 (PI), NINDS NS063925 (PI), and NINDS NS061119 (PI)]; and holds stock and stock options in Epix Pharmaceuticals.

Address correspondence and reprint requests to Dr. Hakan Ay, A.A. Martinos Center for Biomedical Imaging and Stroke Service, Departments of Neurology and Radiology, Massachusetts General Hospital, Harvard Medical School, 149 13th Street, Room 2301, Charlestown, MA 02129 hay@partners.org

Editorial, page 102

Supplemental data at www.neurology.org

e-Pub ahead of print on December 16, 2009, at www.neurology.org.

Disclosure: Author disclosures are provided at the end of the article.

Received February 9, 2009. Accepted in final form August 11, 2009.