Abstract
Background and Purpose
The ischemic stroke population is heterogeneous. Even in balanced randomized trials, patient heterogeneity biases estimates of the treatment effect toward no effect when dichotomous end points are used. Risk adjustment statistically addresses some of the heterogeneity and can reduce bias in the treatment effect estimate. The purpose of this study was to estimate the treatment effect of tissue plasminogen activator (tPA) in the National Institute of Neurological Disorders and Stroke (NINDS) tPA data set with and without adjustment for baseline differences.
Methods
Using a prespecified predictive model, we calculated unadjusted and risk-adjusted odds ratios (ORs) for favorable outcome for the Barthel Index, National Institutes of Health Stroke Scale, and Glasgow Outcome Scale for the patients in the NINDS tPA stroke trial. To assess the importance of the difference, a new sample size was calculated through the use of the risk-adjusted analysis.
Results
We analyzed 615 subjects. The ORs for the Barthel Index were 1.76 (unadjusted) and 2.04 (adjusted). The National Institutes of Health Stroke Scale and Glasgow Outcome Scale analyses also demonstrated increased ORs after adjustment. The estimated sample size required for the adjusted comparison was 13% smaller than the unadjusted sample.
Conclusions
Risk adjustment in this data set suggests that the true treatment effect was larger than estimated by the unadjusted analysis. Stroke clinical trials should include prospective risk adjustment methodologies.
Keywords: cerebral ischemia; models, statistical; prognosis; risk adjustment; stroke outcome
Despite randomization, chance imbalances can occur in randomized trials, particularly for heterogeneous populations. Many baseline characteristics of stroke patients are related to outcome. Ensuring balance of these combinations of characteristics is difficult. Imbalances that occur can result in a biased estimate of the treatment effect. Although randomization reduces the likelihood of this imbalance, it does not guarantee an equal probability of a given outcome in both groups.
Gail and colleagues1 have demonstrated mathematically that for clinical trials with heterogeneous populations and binary or survival outcomes, randomization alone results in a biased estimate of the treatment effect, even with perfect balance. Heterogeneity in risk factors will bias the univariable estimates toward no effect and can be addressed through covariable adjustment. Steyerberg and colleagues2 demonstrated the importance of covariable adjustment in a group of >30 000 myocardial infarction patients from the Global Utilization of Streptokinase arid TPA for Occluded Arteries (GUSTO 1) data set. Despite the fact that there were no significant imbalances in 17 covariables, risk adjustment demonstrated an increased estimate of the treatment effect, resulting in a 19% reduction in the estimated sample size on recalculation.
The purpose of this study was to estimate the treatment effect of tissue plasminogen activator (tPA) in the National Institute of Neurological Disorders and Stroke (NINDS) tPA stroke trial3 data set with and without adjustment of baseline characteristics related to outcome (risk adjustment). On the basis of the known heterogeneity of the stroke population, we hypothesized that adjustment for predicted stroke outcome would result in an increased estimate of the treatment effect in this trial and demonstrate the value of risk adjustment in clinical stroke trials.
Methods
Study Population
The NINDS tPA stroke trial population has been described in detail.3 Briefly, acute ischemic stroke patients were treated with intravenous tPA or placebo within 3 hours of stroke onset. Children, patients with other complicating conditions, and those with increased risk of bleeding were excluded.
Missing Data
All 624 patients from the NINDS tPA trial were considered for this analysis. Nine subjects were excluded from all analyses because of missing predictor variables, leaving 615 patients for analysis.
Predictive Model
The risk adjustment model used in this analysis has been described elsewhere.4 This model was originally developed in the Randomized Trial of Tirilazad Mesylate for Acute Stroke (RANTTAS)5 data set. The predictors (age, National Institutes of Health Stroke Scale [NIHSS], subtype, history of disability, diabetes, and previous stroke) were used to predict each excellent outcome (Barthel Index [BI],6 NIHSS,7 and Glasgow Outcome Scale [GOS8]), resulting in 3 different models. The previously developed and internally validated models were frozen and forecasted into the tPA data set for this analysis.
Statistical Analysis
The unadjusted treatment effect was estimated by use of univariable analysis with treatment assignment predicting excellent outcome. The adjusted treatment effect was determined using the estimate of prior risk (as determined by the multivariable models) and the treatment assignment. Point estimates and 95% confidence intervals (CIs) for the odds ratio (OR) estimate of the effect of treatment were calculated for both the unadjusted and adjusted analyses. The statistical analysis was completed with SAS version 8.2 (SAS Institute Inc).
Results
The baseline characteristics of the 615 subjects are shown in Table 1. Overall, the tPA-treated group was older and had slightly milder strokes and more lacunar strokes. The predicted probability of excellent outcome was higher for the tpA group than for the placebo group for each of the 3 models (BI, 48%, 47%; NIHSS, 34%, 30%; GOS, 40%, 38%) although none of these differences were significant. This suggests that the tPA group was slightly healthier on the basis of baseline characteristics before the impact of treatment with tPA.
TABLE 1.
Baseline Characteristics
| Baseline Variable | tPA (n = 310) | Placebo (n=305) | P |
|---|---|---|---|
| Mean (SD) age, y | 68 (11.4) | 66 (11.9) | 0.02 |
| Median NIHSS (1,3rd quartiles) | 14 (8.19) | 15 (10,20) | 0.07 |
| Prestroke disability, n (%) | 25 (8) | 30 (10) | 0.95 |
| Lacunar infarct, n (%) | 50 (16) | 30 (10) | 0.03 |
| History of stroke, n (%) | 43 (14) | 37 (12) | 0.53 |
| History of diabetes, n (%) | 68 (22) | 61 (20) | 0.62 |
The Figure demonstrates the distribution of risk as estimated by our BI model in all 615 subjects. The wide distribution of outcomes expected (heterogeneity) in this population is shown.
Figure 1.
Risk distribution of NINDS tPA study patients as predicted by BI model. Each point reflects number of patients within given decile of predicted probability of excellent outcome. First point suggests that ≈50 patients had a 0% to 10% likelihood of excellent outcome as estimated by the model.
The regression analysis results given in Table 2 consistently demonstrate that the estimate of the treatment effect increased (higher OR) with risk adjustment. From these ORs, a new sample size was calculated. Using the unadjusted 615 subjects resulted in the same statistical significance as using only 536 subjects in the adjusted analysis. This is a 13% reduction in the sample size required for the same conclusion.
TABLE 2.
Odds Ratios
| Unadjusted |
Adjusted |
|||
|---|---|---|---|---|
| Model | OR | 95% CI | OR | 95% CI |
| BI | 1.76 | 1.28–2.42 | 2.04 | 1.40–2.97 |
| NIHSS | 2.04 | 1.42–2.93 | 2.07 | 1.38–3.11 |
| GOS | 1.87 | 1.35–2.61 | 2.02 | 1.38–2.95 |
Discussion
These analyses confirm that the difference in baseline characteristics in the NINDS tPA trial do not account for the treatment effect. In fact, risk adjustment not only did not decrease the estimate of the treatment effect but also consistently increased the estimate. These analyses confirm evidence seen in other populations2,9,10 that risk adjustment in a heterogeneous population results in an increased estimate of the treatment effect (farther from null). The analysis of Gail et al1 demonstrates that the risk-adjusted estimates are a better estimate of the truth. These data therefore provide the proof of concept for using prespecified risk adjustment in clinical stroke trials.
The heterogeneity of a population and the imbalance of the groups contribute to the treatment estimate bias.1,2 Risk adjustment simultaneously addressed both the heterogeneity and the imbalance to determine the final adjusted OR.1,2 The adjusted ORs presented here were consistently larger than the unadjusted estimates, demonstrating that in this data set, the unadjusted analysis underestimated the treatment effect. Based on these analyses, stroke clinical trials with binary or survival outcomes should prospectively include risk adjustment methods in the primary analysis to optimize trial efficiency.
Acknowledgments
The NINDS rt-PA Stroke Trial was funded by the NIH-NINDS through contracts to the participating sites. Dr Johnston is supported by the NIH-NINDS (K23NS02168). We gratefully acknowledge the contribution of the NINDS rt-PA Stroke Trial Investigators, without whose efforts this work would not have been possible.
This work was presented in part at the 28th International Stroke Conference of the American Heart Association, Phoenix, AZ, February 15, 2003.
References
- 1.Gail MH, Wieand S, Piantadosi S. Biased estimates of treatment effect in randomized experiments with nonlinear regressions and omitted covariates. Biometrika. 1984;71:431–444. [Google Scholar]
- 2.Steyerberg EW, Bossuyt PMM, Lea KL. Clinical trials in acute myocardial infarction: should we adjust for baseline characteristics? Am Heart J. 2000;139:745–751. doi: 10.1016/s0002-8703(00)90001-2. [DOI] [PubMed] [Google Scholar]
- 3.NINDS rt-PA Stroke Study Group. Tissue plasminogen activator for acute ischemic stroke. N Engl J Med. 1995;333:1581–1587. doi: 10.1056/NEJM199512143332401. [DOI] [PubMed] [Google Scholar]
- 4.Johnston KC, Connors AF, Wagner DP, Knaus WA, Wang XQ, Haley EC. for the Randomized Trial of Tirilazad Mesylate in Acute Stroke (RANTTAS) Investigators. A predictive risk model for outcomes of ischemic stroke. Stroke. 2000;31:448–455. doi: 10.1161/01.str.31.2.448. [DOI] [PubMed] [Google Scholar]
- 5.RANTTAS Investigators. A randomized trial of tirilazad mesylate in patients with acute stroke (RANTTAS) Stroke. 1996;27:1453–1458. doi: 10.1161/01.str.27.9.1453. [DOI] [PubMed] [Google Scholar]
- 6.Mahoney FT, Barthel DW. Functional evaluation: Barthel Index. Md Med J. 1965;14:61–65. [PubMed] [Google Scholar]
- 7.Lyden P, Brott T, Tilley B, Welch KM, Mascha EJ, Levine S, Haley EC, Grotta J, Marler J. Improved reliability of the NIH Stroke Scale using video training: NINDS TPA Stroke Study Group. Stroke. 1994;25:2220–2226. doi: 10.1161/01.str.25.11.2220. [DOI] [PubMed] [Google Scholar]
- 8.Jennett B, Bond M. Assessment of outcome after severe brain damage. Lancet. 1975;1:480–484. doi: 10.1016/s0140-6736(75)92830-5. [DOI] [PubMed] [Google Scholar]
- 9.NINDS tPA Stroke Study Group. Generalized efficacy of t-PA for acute stroke: subgroup analysis of the NINDS t-PA Stroke Trial. Stroke. 1997;28:2119–2125. doi: 10.1161/01.str.28.11.2119. [DOI] [PubMed] [Google Scholar]
- 10.Knaus WA, Harrell FE, Jr, LaBrecque JF, Wagner DP, Pribble JP, Draper EA, Fisher CJ, Jr, Soll L. Use of predicted risk of mortality to evaluate the efficacy of anticytokine therapy in sepsis: the rhlL-Ira Phase III Sepsis Syndrome Study Group. Crit Care Med. 1996;24:46–56. doi: 10.1097/00003246-199601000-00010. [DOI] [PubMed] [Google Scholar]