Abstract
Background
Distribution shifts of the modified Rankin scale (mRs) is used as outcome measure in acute ischemic stroke (AIS) randomized controlled trials (RCT). Distribution across strata of mRs is relevant for sample size calculations and may be affected by eligibility criteria.
Aim
We aimed to assess the distribution of mRs scores across its different strata in AIS according to usual eligibility criteria.
Methods
We computed follow-up mRs strata distribution between an unselected cohort and samples with (a) time from symptom onset < 6 h (b) National Institutes of Health Stroke Scale (NIHSS) scores > 3 and < 25, and (c) both criteria combined. We compared distributions with the Mann-Whitney U Test and calculated sample sizes for each distribution.
Results
We included 5849 AIS patients. The unselected sample had a non-normal distribution with a median of 2. All selection criteria yielded significantly different distributions of mRs (p = 004, 0.02 and 0.02 respectively). This resulted in a significant variation in the calculated sample size when applying different selection criteria, with smaller numbers when RCT selection criteria are used (3616 versus 1553).
Conclusions
The use of usual RCT eligibility criteria result in significant differences in mRs distribution and smaller sample sizes compared to unselected AIS samples.
Keywords: Stroke, Sample size, Clinical trials, Ordinal logistic regression
1. Introduction
The modified Rankin scale (mRs) is an ordinal disability score of 7 categories (0 = no symptoms to 6 = dead) and is currently the preferred method of evaluating outcome in patients with acute ischemic stroke (AIS) after intervention trials, particularly if the effect size is modest and distributed across all strata [1], [2], [3]. Thus, sample size calculations are increasingly determined by the distribution of the scores in the mRs at 90 days after symptom onset in modern AIS randomized clinical trials (RCT) [4], [5], [6].
The data are analyzed using the t-test for difference of means or the Mann-Whitney U test or its variants - the Robust Rank Test and the Ordinal Logistic Regression (OLR) [7], [8]. The latter also known as Whitehead's Odds Ratio derives from the proportional odds model of McCullagh [9], and has been shown to produce the lowest sample sizes [10]. The analysis assumes an equal chance of being in any strata and requires that at least the proportion of subjects in each scale category in one of the groups be specified, leading to a same odds ratio independently of the chosen partitions [11]. Thus the distribution across strata is relevant in deciding which statistical method is most suitable for sample size calculations when analyzing mRs as an ordinal scale [12].
Acute ischemic stroke RCTs eligibility criteria may impact the outcome distribution across strata and have the potential of being significantly different when compared to unselected stroke patients and between different trials. This could affect the sample sizes to an unknown extent.
Our aim was to compare the distribution of follow-up mRs scores across its different strata in patients with acute ischemic stroke (AIS) from an unselected consecutive cohort with 3 groups of patients meeting usual RCTs eligibility criteria. We hypothesized that variations in the distribution of mRs would significantly influence the sample sizes, when using OLR.
2. Methods
In this retrospective cohort study, patients were selected from two prospective hospital stroke registries and one population-based registry (Joinville): two in Brazil (Porto Alegre and Joinville) and one in Chile (Santiago).
For the unselected cohort, we included all adult AIS patients (i.e. ≥ 18 years old) with clinical and imaging diagnosis of AIS, data available on NIHSS at admission, time from symptom onset to admission, mRs at follow up and informed consent given. Patients with Transient Ischemic Attacks and with incomplete data were excluded.
The following eligibility criteria have been used in many acute stroke RCTs and were applied to the cohort to produce 3 samples: (a) time from symptom onset < 6 h (b) National Institutes of Health Stroke Scale (NIHSS) scores > 3 and < 25, and (c) both criteria combined.
The corresponding institutional review board and ethics committee approved the 3 registries were these data were derived from: The RECCA registry in Clínica Alemana, the Joinville Stroke Registry and Hospital das Clínicas, Porto Alegre Stroke Registry.
2.1. Analysis
We computed skewness, kurtosis, median values and used Mann-Whitney U Test to compare mRs strata distribution for each set of criteria compared to the unselected sample. We then applied the method proposed by S. Simon to calculate sample sizes for an acute trial using the distribution of mRs according to the different sets of eligibility criteria [13]. We grouped categories mRs 5 and 6 into one, as is usual in the analysis of AIS RCTs. All sample sizes were calculated with a modest effect size of 2% between categories of mRs in each group, alfa was set at 0.05 and power at 80% and all calculation were 2 tailed.
3. Results
The total unselected cohort consisted of 5849 consecutive patients, the demographic and clinical characteristics of which are described for each individual sample in Table 1.
Table 1.
Demographic and clinical characteristics of sample cohorts.
| Variables | Clínica Alemana, Santiago N = 1078 |
Hospital das Clínicas, Porto Alegre N = 1830 |
Joinville N = 2941 |
|---|---|---|---|
| Mean age (SD) | 70 (16) | 66 (14) | 66 (14) |
| Female (%) | 539 (50) | 934 (51) | 1375 (48) |
| Hypertension (%) | 722 (67) | 1492 (82) | 2170 (74) |
| Diabetes Mellitus (%) | 226 (21) | 510 (28) | 957 (33) |
| Hypercholesterolemia (%) | 259 (24) | 829 (45) | 887 (30) |
| Current smoking (%) | 205 (19) | 373 (20) | 602 (20) |
| Atrial fibrillation (%) | 172 (16) | 338 (19) | 394 (10) |
| Ischemic heart disease (%) | 197 (9) | 418 (23) | 170 (6) |
| Previous stroke/TIA (%) | 259 (24) | 653 (36) | 988 (34) |
| Cardioembolic (%) | 366 (34) | 490 (27) | 844 (29) |
| Aterothrombotic (%) | 140 (13) | 423 (23) | 706 (24) |
| Lacunar (%) | 119 (11) | 377 (21) | 625 (21) |
| Undetermined (%) | 420 (39) | 347 (19) | 597 (20) |
| Other determined (%) | 32 (3) | 193 (10) | 169 (6) |
| NIHSS Median (IQR) | 5 (2–10) | 6 (3–13) | 4 (2–11) |
The selection criteria yielded 3275 (56.0%) patients for NIHSS alone, 2636 (45.1%) for time alone and 1134 (19.4%) for both criteria combined (Table 2).
Table 2.
Distribution of modified Rankin scores at follow up in the total cohort according to usual acute ischaemic stroke trials selection criteria.
| mRs scores | Unselected cohort N (%) |
NIHSS >3 and < 25 N (%) |
Time from onset <6 h N (%) |
Both criteria N (%) |
|---|---|---|---|---|
| 0 | 1211 (20.1) | 391 (11.9) | 531 (20.1) | 132 (11.6) |
| 1 | 1664 (28.4) | 694 (21.2) | 656 (24.9) | 258 (22.6) |
| 2 | 695 (11.8) | 447 (13.6) | 283 (10.7) | 136 (11.9) |
| 3 | 620 (10.6) | 486 (14.8) | 278 (10.5) | 142 (12.5 |
| 4 | 628 (10.7) | 539 (16.5) | 318 (12.1) | 167 (14.7) |
| 5–6 | 1031 (17.6) | 2718 (21.9) | 570 (21.6) | 299 (26.3) |
| Total | 5849 | 3275 | 2636 | 1134 |
The unselected sample had a non-normal distribution of mRs, skewed to the left, with a median of 2 (less disability). When admission NIHSS was used, the mRs distribution was significantly less skewed, with a median of 3 (p = 0.04). When only time from symptom onset was used, the mRs distribution more skewed to lower values but nevertheless different form the unselected sample (p = 0.02). When both criteria were used, the distribution was less skewed with a median mRs of 3 and also significantly different form the whole sample cohort (p = 0.02) (Fig. 1).
Fig. 1.
Percentage distribution of patients modified Rankin scale scores at discharge in the total cohort according to usual acute ischemic stroke trials selection criteria.
A significant variation in the calculated sample size was found when applying different selection criteria to the sample, with need of a smaller recruited number of patients when usual RCT selection criteria are used (Table 3).
Table 3.
Descriptive statistics of samples and resulting sample sizes according to the distribution of modified Rankin scale scores at follow up after applying different selection criteria to the cohort and resulting odd ratios.
| Criteria | Median | Skewness | Kurtosis | Sample size |
|---|---|---|---|---|
| None | 2 | 0.56 | −0.72 | 3616 |
| Time from onset <6 h | 2 | 0.53 | −0.85 | 3364 |
| NIHSS > 3 and <25 | 3 | 0.09 | −0.9 | 1544 |
| Both criteria | 3 | 0.18 | −1.07 | 1553 |
4. Discussion
Our results show sample sizes for ORL are significantly affected by the different distributions of follow-up mRs produced when applying usual eligibility criteria for AIS RCTs, being smaller when the resulting distribution is less skewed. In this sample, the effect was greater when stroke severity, measured with NIHSS, was used as a selection criterion together with time from symptom onset. We also found that the 6-h time window does NOT influence the sample size.
Even though mRs is an ordinal scale, it has been traditionally and still is analyzed as a binary outcome measure with many possible cut-off points [14], [15], [16], [17]. Only recently large stroke trials have used its distribution across all strata as primary end point and thus for sample size calculations [18], [19], [20], [21].
An important advantage of using ordinal or continuous analysis of mRs as primary outcome is the need for smaller sample sizes in effective therapies with small effect size or effects that are distributed across all strata [22]. This advantage may be neglected if the impact of patient selection through eligibility criteria on the distribution of mRs strata is not accounted for. This same analysis could apply to other areas of neurology and medicine that are using ordinal scales as outcome measures. A similar approach to calculate the appropriate sample size, could also be used in case an ordinal scale is dichotomized and the distribution of outcome varies in the subgroups after using different selection criteria.
The main strength of this study is that the populations were large and unselected cohorts of patients with AIS presenting at large academic medical centers that may be eligible for any acute treatment or clinical trial enrolment.
The main weakness is that we assumed an even effect size for all categories in mRs, that is improving 2% for 0–2 and worsening 2% for 3–6. This is not the case in many trials in which there could be in improvement in 0–1 and worsening of 5–6 categories or no change in mortality (mRs = 6).
In conclusion, we found that the application of usual eligibility criteria produced a 50% reduction in the required potential sample sizes for AIS RCT analyzing the mRs scale with OLR. The distribution of follow-up mRs scores are significantly different from the unselected data sample. Sample size calculations should consider this distribution in AIS, if primary outcome is ordinal analysis of the distribution of mRs scores with ordinal logistic regression.
Funding
This work was supported by unconditional grants form from Lundbeck Chile and Clínica Alemana de Santiago. It also received financial support from the Fundacao de Amparo a Pesquisa do Estado de Santa Catarina, University of the Region of Joinville and the Joinville Municipal Health Department. Grant support: Fundacao de Apoio a Pesquisa Cientifica do Estado de Santa Catarina-FAPESC. University of the Region of Joinville Univille. The funding sources had no involvement in study design; collection, analysis and interpretation of data; in the writing of the report; or in the decision to submit the article for publication.
Acknowledgements
We would like to thank our colleagues and all the staff in the participating centers, the patients and their families for making this work possible.
Contributor Information
Verónica V. Olavarría, Email: veroolavarria@yahoo.com.
Alejandro Brunser, Email: abrunser2002@yahoo.com.
Norberto Cabral, Email: norbertocabral@icloud.com.
Sheila Martins, Email: smartins@portoweb.com.br.
Paula Muñoz-Venturelli, Email: pmventurelli@gmail.com.
Gabriel Cavada, Email: gabrielcavada@gmail.com.
Pablo M. Lavados, Email: pablolavados@yahoo.com.
References
- 1.Rankin J. Cerebral vascular accidents in patients over the age of 60. II. Prognosis. Scott. Med. J. 1957 May;2(5):200–215. doi: 10.1177/003693305700200504. [DOI] [PubMed] [Google Scholar]
- 2.Saver J.L. Novel end point analytic techniques and interpreting shifts across the entire range of outcome scales in acute stroke trials. Stroke. 2007 Nov;38(11):3055–3062. doi: 10.1161/STROKEAHA.107.488536. [DOI] [PubMed] [Google Scholar]
- 3.Saver J.L. Optimal end points for acute stroke therapy trials: best ways to measure treatment effects of drugs and devices. Stroke. 2011 Aug;42(8):2356–2362. doi: 10.1161/STROKEAHA.111.619122. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Lees K.R., Bath P.M.W., Schellinger P.D., Kerr D.M., Fulton R., Hacke W. Contemporary outcome measures in acute stroke research: choice of primary outcome measure. Stroke. 2012 Apr;43(4):1163–1170. doi: 10.1161/STROKEAHA.111.641423. [DOI] [PubMed] [Google Scholar]
- 5.Saver J.L., Starkman S., Eckstein M., Stratton S.J., Pratt F.D., Hamilton S. Prehospital use of magnesium sulfate as neuroprotection in acute stroke. N. Engl. J. Med. 2015 Feb 5;372(6):528–536. doi: 10.1056/NEJMoa1408827. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.ENOS Trial Investigators. Bath P.M.W., Woodhouse L., Scutt P., Krishnan K., Wardlaw J.M. Efficacy of nitric oxide, with or without continuing antihypertensive treatment, for management of high blood pressure in acute stroke (ENOS): a partial-factorial randomised controlled trial. Lancet. 2015 Feb 14;385(9968):617–628. doi: 10.1016/S0140-6736(14)61121-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Campbell M.J., Julious S.A., Altman D.G. Estimating sample sizes for binary, ordered categorical, and continuous outcomes in two group comparisons. BMJ. 1995 Oct 28;311(7013):1145–1148. doi: 10.1136/bmj.311.7013.1145. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Bath P.M.W., Gray L.J., Collier T., Pocock S., Carpenter J. Can we improve the statistical analysis of stroke trials? Statistical reanalysis of functional outcomes in stroke trials. Stroke. 2007 Jun;38(6):1911–1915. doi: 10.1161/STROKEAHA.106.474080. [DOI] [PubMed] [Google Scholar]
- 9.McCullagh P. Regression models for ordinal data. J. R. Stat. Soc. Ser. B Methodol. 1980 Jan 1;42(2):109–142. [Google Scholar]
- 10.Optimising Analysis of Stroke Trials Collaboration Calculation of sample size for stroke trials assessing functional outcome: comparison of binary and ordinal approaches. Int. J. Stroke. 2008 May;3(2):78–84. doi: 10.1111/j.1747-4949.2008.00184.x. [DOI] [PubMed] [Google Scholar]
- 11.Whitehead J. Sample size calculations for ordered categorical data. Stat. Med. 1993 Dec 30;12(24):2257–2271. doi: 10.1002/sim.4780122404. [DOI] [PubMed] [Google Scholar]
- 12.Walters S.J. Sample size and power estimation for studies with health related quality of life outcomes: a comparison of four methods using the SF-36. Health Qual. Life Outcomes. 2004 May 25;2:26. doi: 10.1186/1477-7525-2-26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Stats: Sample Size for an Ordinal Outcome. September 22, 2004. http://www.pmean.com/04/OrdinalLogistic.html [Internet]. [cited 2015 Jul 18]. Available from: [Google Scholar]
- 14.Duncan P.W., Jorgensen H.S., Wade D.T. Outcome measures in acute stroke trials: a systematic review and some recommendations to improve practice. Stroke. 2000 Jun;31(6):1429–1438. doi: 10.1161/01.str.31.6.1429. [DOI] [PubMed] [Google Scholar]
- 15.AVERT Trial Collaboration group. Bernhardt J., Langhorne P., Lindley R.I., Thrift A.G., Ellery F. Efficacy and safety of very early mobilisation within 24 h of stroke onset (AVERT): a randomised controlled trial. Lancet. 2015 Jul 4;386(9988):46–55. doi: 10.1016/S0140-6736(15)60690-0. [DOI] [PubMed] [Google Scholar]
- 16.IST-3 collaborative group. Sandercock P., Wardlaw J.M., Lindley R.I., Dennis M., Cohen G. The benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the third international stroke trial [IST-3]): a randomised controlled trial. Lancet. 2012 Jun 23;379(9834):2352–2363. doi: 10.1016/S0140-6736(12)60768-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Anderson C.S., Heeley E., Huang Y., Wang J., Stapf C., Delcourt C., Lindley R., Robinson T., Lavados P., Neal B., Hata J., Arima H., Parsons M., Li Y., Wang J., Heritier S., Li Q., Woodward M., Simes R.J., Davis S.M., Chalmers J. INTERACT2 Investigators, Rapid blood-pressure lowering in patients with acute intracerebral hemorrhage. N. Engl. J. Med. 2013 Jun 20;368(25):2355–2365. doi: 10.1056/NEJMoa1214609. [DOI] [PubMed] [Google Scholar]
- 18.Goyal M., Demchuk A.M., Menon B.K., Eesa M., Rempel J.L., Thornton J. Randomized assessment of rapid endovascular treatment of ischemic stroke. N. Engl. J. Med. 2015 Mar 12;372(11):1019–1030. doi: 10.1056/NEJMoa1414905. [DOI] [PubMed] [Google Scholar]
- 19.Saver J.L., Goyal M., Bonafe A., Diener H.-C., Levy E.I., Pereira V.M. Stent-retriever thrombectomy after intravenous t-PA vs. t-PA alone in stroke. N. Engl. J. Med. 2015 Jun 11;372(24):2285–2295. doi: 10.1056/NEJMoa1415061. [DOI] [PubMed] [Google Scholar]
- 20.Jovin T.G., Chamorro A., Cobo E., de Miquel M.A., Molina C.A., Rovira A. Thrombectomy within 8 hours after symptom onset in ischemic stroke. N. Engl. J. Med. 2015 Jun 11;372(24):2296–2306. doi: 10.1056/NEJMoa1503780. [DOI] [PubMed] [Google Scholar]
- 21.Muñoz-Venturelli P., Arima H., Lavados P., Brunser A., Peng B., Cui L. Head Position in Stroke Trial (HeadPoST) - sitting-up vs lying-flat positioning of patients with acute stroke: study protocol for a cluster randomised controlled trial. Trials. 2015;16:256. doi: 10.1186/s13063-015-0767-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Bath P.M.W., Lees K.R., Schellinger P.D., Altman H., Bland M., Hogg C. Statistical analysis of the primary outcome in acute stroke trials. Stroke. 2012 Apr;43(4):1171–1178. doi: 10.1161/STROKEAHA.111.641456. [DOI] [PubMed] [Google Scholar]