Derivation and Application of a Tool to Estimate Benefits From Multiple Therapies That Reduce Recurrent Stroke Risk

Supplemental Digital Content is available in the text.

Background and Purpose—

Lowering blood pressure and cholesterol, antiplatelet/antithrombotic use, and smoking cessation reduce risk of recurrent stroke. However, gaps in risk factor control among stroke survivors warrant development and evaluation of alternative care delivery models that aim to simultaneously improve multiple risk factors. Randomized trials of care delivery models are rarely of sufficient duration or size to be powered for low-frequency outcomes such as observed recurrent stroke. This creates a need for tools to estimate how changes across multiple stroke risk factors reduce risk of recurrent stroke.

Methods—

We reviewed existing evidence of the efficacy of interventions addressing blood pressure reduction, cholesterol lowering, antiplatelet/antithrombotic use, and smoking cessation and extracted relative risks for each intervention. From this, we developed a tool to estimate reductions in recurrent stroke risk, using bootstrapping and simulation methods. We also calculated a modified Global Outcome Score representing the proportion of potential benefit (relative risk reduction) achieved if all 4 individual risk factors were optimally controlled. We applied the tool to estimate stroke risk reduction among 275 participants with complete 12-month follow-up data from a recently published randomized trial of a healthcare delivery model that targeted multiple stroke risk factors.

Results—

The recurrent stroke risk tool was feasible to apply, yielding an estimated reduction in the relative risk of ischemic stroke of 0.36 in both the experimental and usual care trial arms. Global Outcome Score results suggest that participants in both arms likely averted, on average, 45% of recurrent stroke events that could possibly have been prevented through maximal implementation of interventions for all 4 individual risk factors.

Conclusions—

A stroke risk reduction tool facilitates estimation of the combined impact on vascular risk of improvements in multiple stroke risk factors and provides a summary outcome for studies testing alternative care models to prevent recurrent stroke.

Registration—

URL: https://www.clinicaltrials.gov; Unique identifier: NCT00861081.

Evidence-based interventions exist to reduce risk of atherosclerotic vascular disease (ASCVD) events after ischemic stroke. Despite strong evidence supporting therapies that separately target systolic blood pressure (SBP) reduction, cholesterol lowering, decreasing the thrombotic state, and smoking cessation, large implementation gaps persist. For example, among stroke survivors in the United States, nearly half (40%–50%) are not on lipid-lowering therapy; 1 in 5 (22%) have uncontrolled hypertension^1,2; and successful control of multiple risk factors is uncommon.³

A need exists to evaluate care delivery models that aim to implement a comprehensive suite of interventions, targeting multiple risk factors, proven to prevent recurrent stroke. Ideally such studies would assess the impact on hard ASCVD outcomes such as observed rates of stroke, myocardial infarction, and ASCVD death. However, few care delivery model trials are of sufficient size or duration to assess hard ASCVD events; and most studies select changes in individual stroke risk factors, such as SBP, as their primary outcome.^4,5

Trials of care delivery models that target multiple risk reduction therapies require new tools that estimate the impact on recurrent stroke, taking into account changes in multiple risk factors. Although several risk calculators predict recurrent stroke risk in the secondary prevention setting, none are designed to reflect, and thus should not be used to estimate, changes in risk using longitudinal risk factor values for individual patients or for participants in randomized trials (Data Supplement).

This study’s aims are to develop a quantitative tool to estimate the cumulative effect on vascular event risk from addressing multiple stroke risk factors and to apply this tool to existing data from a completed randomized controlled trial (RCT) of a care delivery intervention to reduce recurrent ischemic stroke risk.^4,6

Methods

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Phase I: Development of a Tool to Estimate Reduction in Risk for Recurrent Stroke

The Data Supplement provides a step-by-step guide to the methods used to estimate the combined recurrent stroke or vascular event risk reduction associated with addressing multiple stroke risk factors. First, we identified candidate risk factors for possible inclusion in the tool if they produce measurable changes in risk factors or medication adherence over time and have demonstrated efficacy for reduction of risk of recurrent stroke in high-quality studies. Candidate interventions included treatment of SBP and LDL (low-density lipoprotein) cholesterol, smoking cessation, improved diet and physical activity, and increased use of antiplatelet agents (eg, aspirin, dipyridamole, or clopidogrel) and anticoagulation (in the setting of atrial fibrillation).

Second, we conducted a focused evidence review of high-quality systematic reviews and meta-analyses and extracted respective relative risks (RRs; and 95% CIs) for stroke or vascular events associated with pharmacological or behavioral interventions for each risk factor found eligible in the first step. A supplementary evidence review was conducted to identify studies specific to the setting of cardioembolic stroke or atrial fibrillation.

Finally, we calculated relative reductions in vascular event risk for each individual participant by multiplying, in series, the extracted RRs by documented changes in respective clinical risk factors and medication use for individual participants in a randomized trial described below.

Vascular Risk Outcomes

The primary outcome is the estimated RR reduction (RRR) for recurrent fatal or nonfatal ischemic stroke. The secondary outcome is the estimated relative reduction in ASCVD, inclusive of ischemic stroke, myocardial infarction, or coronary heart disease death. For application to a trial comparing an experimental, chronic-care model–based delivery model to usual care, the predicted effect on these vascular outcomes is presented as the difference in the respective RRRs achieved in the experimental and usual-care arms.

We focus on reduction of stroke risk on the relative rather than the absolute scale for 3 reasons. First, our focus on the RRR is pragmatic: validated equations that estimate absolute risk of recurrent stroke and account for multiple stroke risk factors do not exist; it was beyond the scope of this study to develop one. Second, a focus on the RRR avoids the additional imprecision introduced by absolute risk calculators. Finally, we believe a focus on RR is ethically justified in the setting of secondary prevention, when all participants experience high risk of recurrent stroke.

To present complementary results on an absolute risk scale that may be useful for clinical and policy decision-making, we multiplied the estimated RRRs for experimental and control groups by published estimates of the absolute risk of stroke recurrence at 12 months in the absence of medical interventions (11% [95% CI, 9.0–13.3]).^7,8

We also calculated a modified Global Outcome Score (GO Score)—a measure of the proportion of potential benefit (RRR) possible with ideal implementation of all evidence-based interventions (Data Supplement). The GO Score is based on the approach described by Eddy et al⁹ that intends to represent the extent to which care processes are reducing risk, relative to a target or ideal level of care. In the setting of a healthcare delivery model trial, the aim is to quantify, for each participant, the impact on stroke (or ASCVD) risk since study enrollment, and the modified GO Score represents the proportion of potentially preventable stroke risk achieved with the level of care provided at the end of the trial, given the level of care received at the beginning of the trial by that individual participant. It is calculated as the ratio of the estimated RRR achieved at the end of the trial divided by the estimated ideal RRR when all risk factors are optimally treated for each individual: RRR_achieved/RRR_ideal. The Data Supplement provides an example of calculations for a typical study participant.

Base Case and Alternative Cases

We applied this analytic approach in a base case and 3 additional cases in which assumptions were varied, as a set of sensitivity analyses. Base case assumptions allow estimation of the impact of changes in multiple risk factors on recurrent stroke risk. In the base case, participants taking antiplatelet agents or warfarin were assumed to experience the RRR associated with those therapies (eg, 12% and 61%, respectively), whether or not they reported taking them at baseline. Participants were assumed to reduce their recurrent stroke risk by 21% if they quit smoking. The base case, and all other cases, placed an upper limit on the RRR associated with changes in SBP and LDL cholesterol, based on the maximum change in SBP and LDL reported for the maximum doses of 3 antihypertensive medications,¹⁰ and high potency statins,¹¹ respectively (Data Supplement). All cases further assumed that a lower limit exists below which a further decrease in SBP would not reduce cardiovascular disease risk. This lower limit was set at 110 mm Hg in the base case.

Due to lack of consensus about the impact of risk factors on recurrent stroke risk, we created 3 additional cases that make alternative assumptions about the quantitative relationships between risk factor modifications and recurrent stroke risk. These cases address 3 areas of ongoing debate in the literature: whether to use alternative baseline and minimum SBP thresholds (case 2); whether to constrain the RRRs associated with changes in SBP and LDL to those reasonably attributable to antihypertensive medications and statins (case 3); and how to quantify the benefit of smoking cessation in the first year after stroke (case 4 ignores smoking cessation and also includes the restrictions of cases 2 and 3; Data Supplement).

Among participants whose index stroke was of presumed atherosclerotic origin, all cases assumed the efficacy of pharmacological or behavioral interventions for each risk factor were independent and multiplicative on the relative scale. This approach is supported by evidence that RRRs remain consistent across a range of absolute ASCVD risks and varying treatment backgrounds.^10–13 Among participants with presumed cardioembolic stroke who remain untreated with anticoagulation therapy, we assume treatment of other risk factors provides no benefit; and RR is constant (Data Supplement).

The evidence review failed to identify RCTs to quantify the impact of increased physical activity or improved diet per se on vascular events after stroke. All cases, therefore, assumed that the potential impact of diet and exercise on stroke risk, if any, is mediated through measurable effects on SBP and lipids (LDL cholesterol) and that the RRR due to achieved changes in SBP and LDL estimated from RCTs of medical therapy applies to achieved risk factor changes due to lifestyle modification.

Phase II: Application of the Stroke Risk Reduction Tool Using SUSTAIN Trial Data

Using data from the SUSTAIN trial (Systemic Use of Stroke Averting Interventions), we illustrate how evidence from meta-analyses of clinical trials can be used to model the RRR for recurrent stroke from multiple proven interventions. SUSTAIN was an RCT to improve adherence to guideline-recommended care after hospital discharge following an acute ischemic stroke in a safety net health system.^4,6 The study randomized 404 participants to usual care or to an experimental healthcare delivery model based on the chronic-care model, which was designed to improve multiple recurrent stroke risk factors. The primary outcome of SUSTAIN was change in SBP at 1 year; secondary outcomes included changes in other risk factors, analyzed separately. The comprehensive nature of the SUSTAIN intervention affords an opportunity to demonstrate our approach to modeling the combined impact on stroke risk when changes in multiple risk factors are considered in concert.

Statistical Methods

Our modified probabilistic sensitivity analysis is similar to the approach of cost-effectiveness and other studies that model the impact on health outcomes of >1 efficacious treatment/intervention.^9,14 We account for 2 sources of uncertainty: bootstrapping captures uncertainty in our sample and probabilistic sensitivity (simulation) methods capture uncertainty in the effectiveness of each component risk factor treatment/intervention. Box I in the Data Supplement outlines the steps necessary to calculate the RRR for each bootstrap sample. Briefly, these are to (1) select a bootstrap sample, stratified by experimental healthcare delivery model versus usual-care arms; (2) select simulated model parameters from their prior distributions (truncated log-normal distribution of the 95% CI)¹⁵ independently for each participant (in that bootstrap sample); (3) calculate the mean RR (or absolute RR) in the experimental healthcare delivery model and usual-care arms; (4) calculate the difference (or ratio) of the mean RRR between the experimental and usual-care arms; (5) repeat (steps 1-4) 10 000× to derive the uncertainty range using the bias-corrected percentile method.

Complete-case analyses exclude participants with missing values for individual risk factors or medications. In sensitivity analyses, we used imputed values for missing data, taking a conservative baseline-value-carried-forward approach that assumes no change for LDL and SBP. We also present a sensitivity analysis that permits individuals with atrial fibrillation to benefit exclusively from anticoagulation treatment and another sensitivity analysis that excludes individuals whose index stroke was of presumed cardioembolic origin. Analyses were performed using STATA 13.

SUSTAIN participants provided written informed consent. Institutional Review Boards at University of California Los Angeles and at each of the 4 county hospitals approved SUSTAIN.

Results

Evidence Review and RRs Associated With Pharmacological and Behavioral Interventions for Each Risk Factor

RRRs for stroke associated with observed changes in SBP and LDL, and with antiplatelet medications or warfarin,^16,17 were extracted from meta-analyses of RCTs. Observational studies were used to estimate the RR associated with smoking cessation¹⁸ (Table 1; Data Supplement).

Table 1.

Relative Reductions in CVD and Recurrent Ischemic Stroke Risk for Individual Risk Factors and Data Sources

Application of the Recurrent Stroke Risk Tool to a Trial Comparing an Experimental Healthcare Delivery Model to Usual Care

SUSTAIN Trial Demographics and Changes in Stroke Risk Factors

Demographic characteristics of the experimental healthcare delivery model (n=204) and usual-care (n=200) arms of SUSTAIN participants, as well as observed changes in stroke risk factors among SUSTAIN participants, have been reported previously⁶ and are presented in the Data Supplement. Briefly, with the exception of smoking, participants experienced large improvements in stroke risk factors. However, benefits were similar in experimental and usual-care arms, with the exception of LDL reduction, which was 0.26 mmol/L (10 mg/dL) larger in the experimental arm.

Simulated Impact on Recurrent Stroke Risk

When observed changes in all risk factors and medications were considered in concert for each individual participant and multiplied by the RRs reported in meta-analyses, the estimated achieved RRR for ischemic stroke was 0.36 in both the experimental healthcare delivery model and usual-care arms in the base case (Table 2; Figure 1). Under this case, if all SUSTAIN participants had achieved maximum reductions in SBP and LDL and had fully adhered to antiplatelet and anticoagulation medications, and if all smokers had quit, we estimate that SUSTAIN would have reduced risk of ischemic stroke by nearly four-fifths, on average, in both groups (ideal RRR, 0.80). Thus, both experimental and usual-care arms in SUSTAIN likely achieved slightly less than half the benefit (RRR) that would have been possible, given baseline risk factors and the potential impact of each component risk factor intervention (GO Score, 0.45; Table 2; Figure 2).

Table 2.

Ideal and Achieved RRR of Recurrent Ischemic Stroke in SUSTAIN

Figure 1.

Reduction in relative risk (RR) and absolute risks of ischemic stroke among SUSTAIN (Systemic Use of Stroke Averting Interventions) experimental and usual-care arms across the 4 cases. Absolute RR=RRR×0.11 based on the risk of recurrent stroke of in the first year after stroke (11%). Solid bars in A represent the relative risk reduction (RRR) for recurrent stroke achieved in experimental and usual-care arms at 12 mo follow-up, compared with baseline, for 4 cases (see text for case definitions). Estimates in B represent the difference in RRR (left axis) and absolute (right axis) risk reductions achieved by experimental and usual-care arms. Error bars display uncertainty intervals produced by the probabilistic simulation approach in 10 000 bootstrap samples.

Figure 2.

Modified Global Outcome Score (GO Score) in SUSTAIN (Systemic Use of Stroke Averting Interventions) experimental and usual-care arms across the 4 cases. Solid bars in A represent the GO Score in experimental and usual-care arms, for 4 cases. The GO Score for ischemic stroke risk represents the proportion of potentially preventable stroke risk achieved with the level of care provided at the end of the trial, given the level of care received at the beginning of the trial. It is calculated as the ratio of the estimated relative risk reduction (RRR) achieved at the end of the trial divided by the estimated ideal RRR when all risk factors are optimally treated for each individual (RRR_achieved/RRR_ideal). Estimates in B represent the difference in GO Score between experimental and usual-care arms. Error bars display uncertainty intervals produced by probabilistic simulation in 10 000 bootstrap samples.

Compared with the base case, cases with more conservative assumptions about the RRR associated with changes in blood pressure (case 2) or diet and exercise (case 3) produced estimates of achieved RRR that, as anticipated, were smaller in magnitude (Figure 1). Under the most restrictive case (case 4) that ignored smoking and adopted the conservative assumptions of cases 2 and 3, the estimated RRR was approximately half as large as in the least restrictive case (case 1). Compared with the base case, case 4 produced parameter estimates of smaller magnitude for the maximum RRR possible under ideal implementation of all interventions (ideal RRR, 0.55 and 0.57 in experimental and usual-care arms) and for the proportion of possible benefit achieved (GO Score, 0.33 and 0.28).

All cases estimated similar RRRs in experimental and usual-care arms (Table 2; Figure 1). Results were similar for the secondary outcome of ASCVD risk and for analyses using imputed values for missing data (Data Supplement).

Discussion

We developed a tool to estimate the impact of changes in multiple risk factors on the RR of recurrent ischemic stroke for individuals and populations. Using data from SUSTAIN, we illustrate how the recurrent stroke risk tool can generate meaningful outcomes useful to the evaluation of RCTs designed to address multiple stroke risk factors.

Under base case assumptions, we estimate that the cumulative impact of improved control for multiple risk factors reduced ischemic stroke risk by ≈36%, on average, in both arms of the SUSTAIN trial. As anticipated, more restrictive assumptions reduced the estimated risk reduction in each arm; however, more restrictive scenarios had little influence on comparisons of the risk reduction between arms.

Despite large reductions in recurrent stroke risk, substantial room for improvement remained. A common approach to reporting risk factor changes individually might describe how a year after a stroke, 1 in 3 SUSTAIN subjects failed to lower their SBP below 140 mm Hg⁶; and only 1 in 5 reported taking a high-intensity statin. The modified GO Score illustrates an alternative way to highlight potential improvement that accounts for the combined impact of changes in multiple risk factors. Our results suggest that trial participants in both arms on average averted one-third to one-half of the risk of ischemic stroke they could potentially have prevented through implementation of evidence-based treatments/interventions.

The overall null findings of SUSTAIN, and the large remaining opportunities for improvement in both intervention and usual-care groups, speak to the formidable challenges involved in improving control of recurrent stroke risk factors. We do not intend to imply that it is possible in clinical practice to achieve perfect implementation and adherence to all interventions proven to reduce recurrent stroke risk. Rather, the GO Score provides a complementary means for health systems to monitor and reward achievement of practice- or population-based goals for multiple stroke risk factors, forging a cycle of ever higher achievement.

Our approach to calculating the impact on stroke risk of multiple interventions, by multiplying RRs in series, is similar to that used by cost-effectiveness and other modeling studies^9,14,19,20 and by the Longitudinal ASCVD Risk Assessment Tool developed by Million Hearts. The Million Hearts tool is used to calculate updated ASCVD risk for individual patients in the setting of primary prevention. Our methods differ from those used by Million Hearts in 3 ways. First, whereas the Million Hearts estimates risk of first ASCVD events, our tool estimates reduction in risk for recurrent ischemic stroke. Second, we account for 2 sources of uncertainty in our risk estimates: uncertainty in our data and uncertainty in risk reductions associated with each treatment. Third, whereas Million Hearts emphasizes absolute risk, we focus on RRR. If and when a risk equation is validated to accurately predict recurrent stroke risk on an absolute scale, it could be combined with methods presented here to estimate the impact of multiple interventions on absolute recurrent stroke risk, much as Million Hearts relies on the Pooled Cohort Equations to estimate baseline ASCVD risk.

Tools that estimate the combined effect of changes in multiple risk factors are useful when evaluating care models that produce variation in the magnitude or direction of risk factor changes between study arms. For example, a recurrent stroke risk tool might reduce the likelihood of a type II error in a trial that observes similar SBP changes in experimental and control arms but large differences in changes in other risk factors such as LDL and smoking. Alternatively, the risk tool might highlight important differences between care models when changes in risk factors are too small to demonstrate clinically important differences when considered individually but may produce meaningful differences when considered in concert. Finally, the risk tool may help to gauge whether a modest change in a single risk factor is clinically important, if other risk factors remain unchanged. In SUSTAIN, GO Score results suggest it is unlikely that the observed difference in mean LDL reduction between intervention and control groups produced a meaningful difference in recurrent stroke risk reduction, when changes in other risk factors were not observed.

In clinical settings, the recurrent stroke risk reduction tool could inform decision-making with individual patients who have experienced a stroke. Although we compare mean GO Scores for experimental and control groups in an RCT, the Ideal RRR and GO Score are analogous to the personalized approaches to counterfactual treatment scenarios provided by the Million Hearts tool and could augment discussions about therapeutic priorities and medication adherence for individual patients (Data Supplement).

Limitations

The tool estimates RR of recurrent ischemic stroke and is not applicable in the setting of hemorrhagic stroke.

As for any model, results are influenced by underlying assumptions. Low adherence may diminish the real-word effectiveness of medications, relative to results reported in RCTs. Our tool minimizes overestimation of benefit by using observed biological changes in SBP and LDL cholesterol to calculate stroke risk reductions; and medication adherence was assessed in SUSTAIN by questionnaire. We created a base case and 3 alternative cases with a range of assumptions addressing prominent areas of disagreement in the literature. Our results suggest that the magnitude of estimated RRR in a single study arm is sensitive to model assumptions. However, comparative benefit in experimental versus usual-care groups may be more robust to underlying assumptions. We also conducted 3 sensitivity analyses to illustrate how future studies might assess whether results are robust to alternative assumptions. The first sensitivity analysis used imputed values for variables missing data. The second and third sensitivity analyses addressed uncertainties surrounding the magnitude of risk reductions associated with modifications in ischemic stroke risk factors in the setting of atrial fibrillation. The second sensitivity analysis ignored changes in SBP, LDL, and smoking status among individuals with atrial fibrillation, whose risk of recurrent stroke was assumed to be uniquely influenced by anticoagulation therapy. The third sensitivity analysis excluded individuals with atrial fibrillation (presumed cardioembolic stroke) altogether (Data Supplement). Uncertainty also exists surrounding the magnitude of benefit of specific interventions across ischemic stroke subtypes, and it is possible that our model overestimates the benefit of statins and antihypertensive therapy in select ischemic stroke subtypes. Future models can and should incorporate new evidence of the efficacy of statins and SBP lowering in specific stroke subtypes as it becomes available.

SUSTAIN enrolled subjects shortly after stroke onset, which likely increased baseline SBP values and the magnitude of observed SBP reduction. This likely artificially inflated our estimates of achieved and ideal RRR; however, randomization likely minimized bias in comparisons between groups. SUSTAIN did not record the presence of comorbid conditions known to impact the efficacy of statins or anticoagulation, including heart failure and end-stage renal disease. Stroke subtype was not recorded, and warfarin treatment is used as a proxy for atrial fibrillation. These and other limitations of SUSTAIN highlight opportunities for future studies of care models to modify their design and data collection methods to minimize bias in measurement of blood pressure changes and to appropriately capture key stroke subtypes and comorbidities (Data Supplement).

The absence of clinically important differences between experimental and usual-care arms in SUSTAIN limits our ability to illustrate the potential value of a global risk reduction approach as a complement to individual risk factor outcomes. The choice to use SUSTAIN data was made before results became available; see the Data Supplement for additional limitations related to SUSTAIN.

Conclusions

We demonstrate the feasibility of estimating the impact on the RRs for stroke and ASCVD in the setting of a modest-sized community-based multiple-intervention trial to prevent recurrent stroke and ASCVD. The ability to summarize the impact of complex care models on outcomes of recurrent stroke and ASCVD is relevant to those at risk for stroke, myocardial infarction, and coronary heart disease death who strive to adhere to treatments proven to reduce those risks. Trials of novel healthcare delivery models should complement their reporting of outcomes based on individual risk factor changes and present the estimated RRR for stroke.

Sources of Funding

This study was supported by 5U54NS081764 (National Institute of Neurological Disorders and Stroke), TFR15-00194 (Tides Foundation), and the American Heart Association Pharmaceutical Roundtable and David and Stevie Spina.

Disclosures

Dr Richards is a fund advisor to the Tides Foundation. The other authors report no conflicts.