Predicting major bleeding in patients with noncardioembolic stroke on antiplatelets

Abstract

Objective:

To develop and externally validate a prediction model for major bleeding in patients with a TIA or ischemic stroke on antiplatelet agents.

Methods:

We combined individual patient data from 6 randomized clinical trials (CAPRIE, ESPS-2, MATCH, CHARISMA, ESPRIT, and PRoFESS) investigating antiplatelet therapy after TIA or ischemic stroke. Cox regression analyses stratified by trial were performed to study the association between predictors and major bleeding. A risk prediction model was derived and validated in the PERFORM trial. Performance was assessed with the c statistic and calibration plots.

Results:

Major bleeding occurred in 1,530 of the 43,112 patients during 94,833 person-years of follow-up. The observed 3-year risk of major bleeding was 4.6% (95% confidence interval [CI] 4.4%–4.9%). Predictors were male sex, smoking, type of antiplatelet agents (aspirin-clopidogrel), outcome on modified Rankin Scale ≥3, prior stroke, high blood pressure, lower body mass index, elderly, Asian ethnicity, and diabetes (S₂TOP-BLEED). The S₂TOP-BLEED score had a c statistic of 0.63 (95% CI 0.60–0.64) and showed good calibration in the development data. Major bleeding risk ranged from 2% in patients aged 45–54 years without additional risk factors to more than 10% in patients aged 75–84 years with multiple risk factors. In external validation, the model had a c statistic of 0.61 (95% CI 0.59–0.63) and slightly underestimated major bleeding risk.

Conclusions:

The S₂TOP-BLEED score can be used to estimate 3-year major bleeding risk in patients with a TIA or ischemic stroke who use antiplatelet agents, based on readily available characteristics. The discriminatory performance may be improved by identifying stronger predictors of major bleeding.

Antithrombotic therapy is a cornerstone in secondary stroke prevention, either with oral anticoagulants in patients with a cardioembolic stroke or with antiplatelet agents in patients with a stroke from arterial origin. Antiplatelet therapy successfully reduces the number of serious vascular events by approximately 25%.^1,2 Despite its proven benefit, antiplatelet therapy increases the risk of bleeding. On average, bleeding risks are increased 2-fold in patients on aspirin compared with placebo.³ More potent treatment strategies such as dual antiplatelet therapy increase this risk even further.⁴ Bleeding events appear to be associated with future major vascular events and higher mortality rates.^5,6 Different factors have been proposed that increase bleeding risk, including older age, hypertension, and ethnicity.^6–8 Prediction of bleeding risk based on patient characteristics may help physicians to balance benefits and risks of antiplatelet therapy for individual patients. Also, risk stratification may guide treatment decisions for other preventive strategies, such as gastroprotective agents. A recent systematic review showed that a limited number of prediction models are available for prediction of bleeding in patients on antiplatelet therapy for secondary prevention.⁹ In an external validation study, accurate performance of available models could not be confirmed in patients with a TIA or ischemic stroke.⁹ The purpose of the current study was to develop and externally validate a prediction model to predict the 3-year risk of major bleeding in patients with a TIA or ischemic stroke who use antiplatelet agents.

METHODS

Study population.

The design of the individual patient data (IPD) meta-analysis has been described in detail elsewhere.¹⁰ Briefly, we collected IPD from trials investigating the efficacy of antiplatelet therapy in long-term secondary prevention after a TIA or ischemic stroke. Trials were eligible if they randomized patients with a TIA or ischemic stroke to aspirin, or to antiplatelet drugs that are recommended as first-line treatment in secondary prevention of stroke as an alternative to aspirin or in addition, and had a duration of at least 1 year. Trials had to be published before December 2010 in peer-reviewed journals. Six trials met the inclusion criteria (CAPRIE, ESPS-2, MATCH, CHARISMA, ESPRIT, and PRoFESS^11–16), including 48,023 patients with a TIA or ischemic stroke between 1989 and 2006. Median follow-up ranged from 1.4 to 3.5 years. Details of studies included in the IPD meta-analysis (recruitment period, details of antiplatelet regimens, inclusion criteria, and sample size) are presented in table e-1 at Neurology.org.

Patients with a possible cardioembolic origin of their stroke (those with a history of atrial fibrillation or TOAST classification cardioembolic stroke) were excluded. Also, patients randomized to dipyridamole alone or placebo were excluded, as our interest was in bleeding risk on common antiplatelet regimens after a TIA or ischemic stroke.

We used trial-specific definitions of major bleeding (table e-2). Major bleedings included bleedings that were fatal, intracranial, required hospital admission, or led to significant disability. The outcome was assessed at regular follow-up visits, as specified in the trial protocols (table e-3). Information on candidate predictors was available at the time of outcome assessment.

We performed a literature review to identify candidate predictors of major bleeding. Candidate predictors had to be easily available in clinical practice, and their inclusion was dependent on availability in the trials. We refrained from the inclusion of interactions in our model because we were not aware of clear evidence in the literature on potentially relevant interactions. Potential predictors included patient characteristics (age, sex, Asian ethnicity, body mass index [BMI], smoking, and heavy alcohol use), characteristics of the index event (modified Rankin Scale [mRS] and lacunar stroke subtype), medical history (hypertension, hypercholesterolemia, diabetes mellitus, prior stroke or TIA, cardiovascular disease, and heart failure), and type of the antiplatelet agent. Information on candidate predictors was collected at baseline in each trial. Definitions of candidate predictors are presented in table e-4.

Statistical analysis.

Ten outcome events per candidate predictor is generally accepted as a minimum required sample size to develop multivariable prediction models.^17,18 Given the large number of outcome events available in our study, a sufficiently large number of candidate predictors could be studied.

The proportion of missing data within each trial was low; the percentage of (sporadically) missing values across all candidate predictors and all trials was below 1%. Some candidate predictors were not measured in all trials and were therefore missing systematically (table e-5). Only variables that were available in at least 5 of 6 trials were considered for inclusion in the prediction model. Missing data were imputed multiple times with the MICE package in R, creating 20 imputed sets.

Restricted cubic splines were used to assess whether continuous predictors (age and BMI) could be analyzed as linear terms or needed transformations. A squared term for age was found to be significant; BMI under 30 kg/m² showed a linear association with the outcome. We studied predictors of major bleeding with Cox regression analyses stratified by trial, thereby estimating common predictor effects and separate baseline survival functions per trial. The full model containing all candidate predictors was simplified by performing backward selection based on the Akaike Information Criterion. The proportional hazards assumption was checked by studying log-minus-log plots. The final baseline survival function was estimated based on a Cox regression model in all data pooled, with the linear predictor as the sole variable.

We assessed both discrimination and calibration. Calibration reflects the correspondence between the observed and predicted probabilities of the outcome and was assessed using the Gronnesby and Borgan test and graphically with calibration plots.¹⁹ Discrimination reflects the ability of the model to distinguish between someone with and without the outcome and was evaluated using the concordance statistic (c statistic).²⁰

We performed internal-external cross-validation, a method which allows us to study the consistency and performance of a model across different data sets.²¹ A model was developed in all studies but one, repeating all steps as described previously, and this model was subsequently validated in the remaining study. This process was repeated for all combinations of trials.

We performed bootstrapping to correct for overfitting of the final model. In each bootstrap sample, the entire modeling process was repeated. A shrinkage factor was estimated from the bootstrap procedure, and regression coefficients were shrunk to provide improved predictions for future patients. We translated the regression model into a score chart by dividing all regression coefficients by the smallest coefficient and subsequently rounded them to the nearest integer. The score chart is accompanied by a table displaying estimated 3-year major bleeding risks. Patients were divided into low, medium, and high risks according to their score. We assessed both risk of bleeding and risk of recurrent ischemic events across the risk groups. We subsequently assessed the performance of the developed model for prediction of intracranial hemorrhages, in terms of discrimination and calibration. Methods are described in more detail in the e-Methods section.

We performed 3 sensitivity analyses: one including only patients with an ischemic stroke as index event, one in which we excluded patients who were randomized more than 3 months after their qualifying event, and one in which we excluded patients with missing data.

External validation.

We externally validated the developed prediction model in the PERFORM trial, a randomized clinical trial including 18,417 patients with a recent TIA or ischemic stroke from arterial origin, who were randomized to terutroban or aspirin.²² Seven hundred seventy-three patients experienced a major or life-threatening bleeding event during follow-up (mean follow-up 28.3 months [SD 7.7]). Major or life-threatening bleeding was defined as bleeding that was fatal, symptomatic intracranial, significantly disabling, required hospital admission, transfusion, or surgery. We applied the original regression equation and baseline survival function to the data and calculated 3-year major bleeding probabilities for each patient. Again, discrimination and calibration were assessed by means of the c statistic, calibration plots, and the D'Agostino and Nam test. Results are reported in accordance with the TRIPOD statement.²³ All statistical analyses were performed using R version 3.2.0 and SAS.

Standard protocol approvals, registrations, and patient consents.

The trials were approved by the ethics committee or institutional review board at each participating center, and all patients provided written informed consent.

RESULTS

After exclusion of patients with a possible cardioembolic origin of their stroke (n = 1,829) and patients randomized to placebo or dipyridamole alone (n = 3,082), 43,112 patients remained for the analyses (figure e-1). Major bleeding occurred in 1,530 patients during 94,833 person-years of follow-up. Of these, 155 (10%) were fatal and 273 (18%) were intracranial (nonfatal) (table e-6). The mean observed 1-year risk of major bleeding was 1.9% (95% confidence interval [CI] 1.7–2.0), and the observed 3-year risk was 4.6% (95% CI 4.4–4.9). Table 1 shows the baseline characteristics of patients included in the development population.

Table 1.

Baseline characteristics of 43,112 patients included in 6 trials

The results of the multivariable Cox regression analyses are presented in table 2. The following predictors of major bleeding were identified: male sex, smoking, type of antiplatelet agents, outcome on mRS ≥3, prior stroke, high blood pressure (hypertension), lower BMI, elderly, Asian ethnicity, and diabetes (S₂TOP-BLEED, i.e., male Sex, Smoking, Type of antiplatelet agents, Outcome on mRS, Prior stroke, high Blood pressure, Lower BMI, Elderly, Asian Ethnicity, and Diabetes). History of heart failure was not included in the final model because of varying definitions across trials and conflicting results. We visually inspected log-minus-log plots and detected no deviations from the assumption of proportional hazards. Internal-external cross-validation showed that model performance was comparable across all trials, indicating little heterogeneity in predictor outcome associations and baseline risk (table e-7, table e-8, and figure e-2). We therefore considered it appropriate to use all data, although applying stratified Cox regression analysis to account for the hierarchical nature of the data.

Table 2.

Multivariable hazard ratios for risk of major bleeding from the final model after shrinkage

We combined all identified predictors in 1 model. After shrinkage, the model had a c statistic of 0.63 (95% CI 0.61–0.64). The calibration plot showed good correspondence between predicted and observed risks (figure 1A), and the Gronnesby and Borgan test was not statistically significant (p = 0.74), indicating good overall fit.

Figure 1. Calibration of the S₂TOP-BLEED model in the development data (A) and external validation data (B).

We translated regression coefficients into score charts presented in table 3. The S₂TOP-BLEED score chart can be used in combination with figure 2 to obtain approximate predictions for individual patients. Major bleeding risk ranged from 2% in patients aged 45–54 years without additional risk factors to more than 10% in patients aged 75–84 years with multiple risk factors. A total of 23,678 patients were categorized as low risk (0–10 points; 55%), 16,621 as medium risk (11–15 points; 38.5%), and 2,813 patients as high risk (≥16 points; 6.5%). The observed risk of major bleeding increased across risk groups, as did the risk of recurrent ischemic events (table e-9). The original regression equation and baseline survival function are shown in table e-10. All sensitivity analyses showed largely similar results (table e-11). The S₂TOP-BLEED model showed a c statistic of 0.61 (0.58–0.64) when used to predict intracranial hemorrhage risk. Calibration was adequate after re-estimation of the baseline hazard (figure e-3). Estimated 3-year risks of intracranial hemorrhage based on the S₂TOP-BLEED score are presented in figure e-4.

Table 3.

S₂TOP-BLEED score for major bleeding derived from the multivariable Cox regression model

Figure 2. Estimated 3-year risk of major bleeding (%) in the development data based on the S₂TOP-BLEED score.

Points: number of points on S₂TOP-BLEED score, without the score for age. The predicted probability per age group can be read from the appropriate column. Cells containing less than 5 patients were removed.

External validation.

The baseline characteristics of the validation population are presented in table e-12. Patients in the validation cohort were slightly older (mean age 67 years [SD 8]) and less often had a lacunar stroke. The mean observed 3-year risk of major bleeding was somewhat higher in the validation population (5.5% vs 4.6% in the development population). The prediction model for major bleeding had a c statistic of 0.61 (0.59–0.63) and slightly underestimated major bleeding risk in the validation data, as represented in the calibration plot (figure 1B) and by a significant D'Agostino and Nam test (p < 0.001).

DISCUSSION

We developed the S₂TOP-BLEED score to predict an individualized risk of major bleeding after a TIA or ischemic stroke, based on readily available characteristics. Age was identified as the strongest predictor of major bleeding. Calibration was accurate in the development data, but bleeding risk was slightly underestimated in the external validation data. Discriminatory performance of the model may be improved by identifying stronger predictors of major bleeding.

Previously, 2 models have been developed to predict intracranial hemorrhage in patients with a TIA or ischemic stroke,^24,25 and 1 additional model was developed to predict major bleeding in patients with or at risk of atherothrombosis⁷ (table e-13). Considerable overlap exists between predictors in these models and those identified in our study, including age, hypertension, diabetes, smoking, and antiplatelet agents. Age contained most prognostic information in our model, followed by type of the antiplatelet agent and BMI. The increasing risk of bleeding with higher age seems particularly important, given the rising number of elderly patients with a TIA or ischemic stroke, with around 30% of strokes occurring in patients older than 80 years.²⁶

Clear discrimination between patients with and without a bleeding event based on patient characteristics appears to be difficult, as is shown by the low c statistic of our model and other bleeding models in patients with stroke. Similar results are also seen for major bleeding scores in other domains, such as the HAS-BLED and HEMORR₂HAGES scores for patients with atrial fibrillation.^27,28 In most validation studies, c statistics of these models did not exceed 0.65.^29–31 Prediction of bleeding might be difficult because major bleeding consists of various types of bleedings with different underlying pathophysiologic mechanisms and risk factor profiles. The discriminatory performance of the S₂TOP-BLEED score slightly dropped when applied to predict intracranial hemorrhage, possibly due to differences in risk factors for major and intracranial bleeding, or differences in the strength of the associations between predictors and outcome. Lacunar stroke subtype is reported to be a risk factor for intracerebral hemorrhage,^24,32 but was not identified in the current study, possibly because most major bleedings were gastrointestinal, which may have masked the association. Alternatively, prediction of major bleeding might be difficult because occurrence of bleeding may be a more random process without clear precursors. Discrimination of the current model might be improved by incorporating other, potentially stronger predictors, such as renal failure, history of bleeding (major and minor), ibuprofen or paracetamol use, and results from neuroimaging (e.g., microbleeds).

Although calibration of our model was excellent in the development data, major bleeding risk was slightly underestimated in the external validation. The discrepancy between the observed and predicted risks is likely due to difference in the overall observed risk between the development and validation population (3-year risk 4.6% vs 5.5%), leading to a systematic underestimation. Given the large number of patients included in our development data and consistent performance in the internal-external cross-validation procedure, we chose not to adjust our model.

Although the current model may help to identify patients at high risk of major bleeding events, it does not aim to guide treatment choices for antiplatelet agents, as the risk of bleeding should always be balanced against the risk of recurrent ischemic events. However, the current analyses show considerable overlap in risk factors for bleeding and recurrent ischemic events, as well as increasing risks of ischemic events along with rising bleeding risks. The correlation between the 2 risks suggests that it may be difficult to individualize treatment decisions based on this balance. However, decision analytical studies are needed to assess whether the bleeding risk may outweigh the risk of recurrent ischemic events in a specific subgroup of patients.

An important strength of our study is the large sample size, which enabled us to study a broad range of prognostic factors without the risk of overfitting. Second, quality of the trial data was high, with accurate follow-up and few missing data. Third, patients were included from all continents and had varying ethnic backgrounds, which enhances generalizability of the results. Fourth, although inclusion criteria varied across trials, internal-external cross-validation showed adequate performance in all trials, and sensitivity analyses showed comparable results, which endorses the robustness of the model. Fifth, we externally validated our model in an independent population and found similar discriminatory power and slight underestimation that could be explained by differences in the incidence of major bleeding.

Several limitations of our study need to be addressed. First, we developed our prediction model in a population of trial participants. Because of application of strict inclusion and exclusion criteria, trial participants may not be representative of the entire stroke population. Because patients at highest risk of bleeding have been excluded (e.g., those with a history of bleeding), our model may underestimate bleeding risk. Validation of the current model in observational data would therefore be valuable. Second, the trials included in our IPD meta-analysis are relatively old, and diagnosis and treatment of patients with stroke has improved ever since. The lack of MRI in the older trials will likely have led to some misclassification of stroke subtypes, but may not have had a large influence on the classification of strokes as either cardioembolic or noncardioembolic, which was an important distinction for our study population. Third, we could only study candidate predictors that were measured in the majority of trials. As a result, we were unable to study some potentially relevant predictors, such as renal failure and blood pressure. However, we were able to include hypertension as predictor in our model. Fourth, the definition of major bleeding varied slightly across trials. Unfortunately, we were unable to reclassify major bleeding events according to a standardized definition with the available data. Also, we could not assign weights to different types of bleedings, while the severity and impact of included major bleedings clearly differs. Fifth, some candidate predictors were missing systematically and were imputed with multiple imputations. Although imputation is increasingly recognized as a valid approach for handling of missing data, imputation of systematically missing data is relatively new, and methods for dealing with systematically missing data are topic of further study. Nevertheless, the hazard ratios remained comparable after exclusion of those patients with missing data.

We developed and externally validated a practical score that can generate individualized risk predictions of major bleeding after a TIA or ischemic stroke. Whether this model can guide treatment decisions needs to be investigated in a decision analytical study in which the risk of major bleeding is balanced against the risk of recurrent ischemic events.

GLOSSARY

BMI

body mass index

CI

confidence interval

IPD

individual patient data

mRS

modified Rankin Scale

S₂TOP-BLEED

male Sex, Smoking, Type of antiplatelet agents, Outcome on mRS, Prior stroke, high Blood pressure, Lower BMI, Elderly, Asian Ethnicity, and Diabetes

AUTHOR CONTRIBUTIONS

Nina A. Hilkens: statistical analysis, interpretation of results, and drafting the first version of the manuscript. Ale Algra: concept and design of the study, data collection, statistical analysis, interpretation of results, and revision of the manuscript for important intellectual content. Hans-Christoph Diener: data collection and revision of the manuscript for important intellectual content. Johannes B. Reitsma: statistical analysis, interpretation of results, and revision of the manuscript for important intellectual content. Philip M. Bath, Laszlo Csiba, Werner Hacke, Jaap Kappelle, Peter J. Koudstaal, Didier Leys, Jean-Louis Mas, Ralph L. Sacco, and Pierre Amarenco: data collection and revision of the manuscript for important intellectual content. Leila Sissani: statistical analysis and revision of the manuscript for important intellectual content. Jacoba P. Greving: concept and design of the study, statistical analysis, interpretation of results, and revision of the manuscript for important intellectual content.

STUDY FUNDING

J.P. Greving and N.A. Hilkens are supported by a grant from the Dutch Heart Foundation (grant number 2013T128). J.P. Greving is also supported by a VENI grant from the Netherlands Organization for Health Research and Development (ZonMw), grant number 916.11.129.

DISCLOSURE

N.A. Hilkens reports no disclosures relevant to the manuscript. A. Algra was principle investigator of the ESPRIT trial. H.-C. Diener reports personal fees from Boehringer Ingelheim and Sanofi, during the conduct of the study, and received in the last 36 months honoraria for participation in clinical trials, contribution to advisory boards or oral presentations from Amgen, Allergan, AstraZeneca, Bayer Vital, BMS, Boehringer Ingelheim, CoAxia, Covidien, Daiichi Sankyo, D-Pharm, GlaxoSmithKline, Johnson & Johnson, Lilly, MSD, Medtronic, MindFrame, Neurobiological Technologies, Novartis, Pfizer, Sanofi, Servier, St. Jude, and WebMD Global; financial support for research projects was provided by AstraZeneca, GSK, Boehringer Ingelheim, Lundbeck, Novartis, Janssen-Cilag, Sanofi-Aventis, Syngis, and Talecris; within the past year, H.-C. Diener served as editor of Aktuelle Neurologie, Arzneimitteltherapie, Kopfschmerznews, Stroke News, and the Treatment Guidelines of the German Neurological Society, as coeditor of Cephalalgia and on the editorial board of Lancet Neurology, Stroke, European Neurology, and Cerebrovascular Disorders. J.B. Reitsma reports no disclosures relevant to the manuscript. P.M. Bath was member of the PRoFESS Trial Steering Committee and received honoraria from Boehringer Ingelheim. L. Csiba reports no disclosures relevant to the manuscript. W. Hacke reports receipt of personal fees from Boehringer Ingelheim, Medtronic, Bayer, Daiichi Sankyo, and Neuravi. L.J. Kappelle was member of the steering committee of the ESPRIT trial. P.J. Koudstaal reports no disclosures relevant to the manuscript. D. Leys was a member of the steering committee of MATCH and of ESPRIT. J.-L. Mas reports no disclosures relevant to the manuscript. R.L. Sacco reports personal fees from Boehringer Ingelheim, outside the submitted work. P. Amarenco reports personal fees from AstraZeneca, Pfizer, Bayer, FibroGen, and GSK and received grants from Pfizer, AstraZeneca, Sanofi, BMS, French government, and The Medicine company, outside the submitted work. L. Sissani and J.P. Greving report no disclosures relevant to the manuscript. Go to Neurology.org for full disclosures.