Abstract
Objective
To develop a simple and effective risk score for predicting which stroke patients will have persistent impairment of upper extremity motor function at 90 days.
Design
Post-hoc analysis of clinical trial patients hospitalized with acute ischemic stroke who were followed for 90 days to determine functional outcome.
Setting
Patient were hospitalized at facilities across the United States.
Participants
We created a harmonized cohort of individual patients from the NINDS tPA, ALIAS part 2, IMS-III, DEFUSE 3, and FAST-MAG trials. We split the cohort into balanced derivation and validation samples.
Interventions
Not applicable.
Main Outcome Measures
The primary outcome was persistent arm impairment, defined as an NIHSS arm domain score of 2–4 at 90 days in patients who had a 24-hour NIHSS arm score ≥1. We used LASSO regression to determine the elements of the Persistent UPPer extremity Impairment (PUPPI) index, which we validated as a predictive tool.
Results
We included 1,653 patients (827 derivation and 826 validation), of whom 803 (48.6%) had persistent arm impairment. The PUPPI index gives one point each for age ≥55 years and NIHSS values of worse arm=4, worse leg>2, facial palsy=3, and total NIHSS ≥10. The optimal cutpoint for the PUPPI index was ≥3, at which the area under the curve was >0.75 for the derivation and validation cohorts and when using NIHSS values from either 24 hours or in a subacute/discharge time window. Results were similar across different levels of stroke severity.
Conclusion
The PUPPI index uses readily available information to accurately predict persistent upper extremity motor impairment at 90 days post-stroke. The PUPPI index can be administered in minutes and could be used as inclusion criterion in recovery-related clinical trials or, with additional development, as a prognostic tool for patients, caregivers, and clinicians.
Keywords: 580: Neurologic Disorders, 690: Outcomes Research, 860: Stroke (see Brain Injuries)
Introduction
The goal of accurately predicting behavioral recovery is important for acute ischemic stroke (AIS) patients and caregivers, in particular for upper extremity (UE) motor status, which is highly correlated with post-stroke quality of life.1 Prior research has shown that assessments of UE motor status early post-stroke and biomarkers of corticospinal tract integrity can accurately predict UE motor function at follow-up.2–5 Several risk indices that have been proposed for predicting AIS patients’ UE motor function at follow-up utilize advanced testing modalities, such as neuroimaging or transcranial magnetic stimulation,2,6 or more complex impairment scores such as the Fugl-Meyer Assessment.3,7 While these tools may be accurate, the necessary advanced testing prevents widespread use. In addition, the existing risk indices largely focus specifically on measures of arm motor status and do not incorporate potentially useful information from non-UE motor assessments or global stroke severity. In this context, we sought to develop a simple risk index that could be quickly administered by clinicians at the bedside and would accurately predict persistent UE motor impairment at 90 days after AIS.
Methods
To develop our risk index, we harmonized patients from the deidentified, publicly available datasets for the NINDS tPA, ALIAS part 2, IMS-III, DEFUSE 3, and FAST-MAG trials.8–13 Because of the sensitive nature of the data in this study, requests to access the dataset from qualified researchers trained in human subject confidentiality protocols can be sent to the National Institute of Neurological Disorders and Stroke at https://www.ninds.nih.gov/Current-Research/Research-Funded-NINDS/Clinical-Research/Archived-Clinical-Research-Datasets. IRB approval was not required for the deidentified datasets.
We included all patients from the trials, apart from FAST-MAG, in which we included 1,245 patients with acute stroke and excluded stroke-mimics and patients with intracerebral hemorrhage to maintain consistency with the other trials which excluded such patients. Patients were further excluded if they had no arm impairment at 24 hours from trial enrollment (NIHSS item 5, right and left arm domain=0), died in the first 90 days after stroke onset, or if they lacked NIHSS data at day 90 follow-up. Because each trial had acute interventions, patients were by default within 16 hours of AIS onset at the time of study enrollment, and >95% were within 6 hours of onset. The primary outcome was an NIHSS item 5 right or left arm domain score of 2–4 at 90 days, which we termed persistent arm impairment.
We randomly divided the patients into 2 cohorts with balanced splitting on the trials. The first cohort was used as the derivation sample and the second for validation. The potential variables for our risk index included the patient demographics in Table 1, NIHSS total score, and NIHSS subscores in the following domains: worse arm (item 5), worse leg (item 6), face (item 4), sensory (item 8), language (item 9), vision (item 3), and neglect (item 11).
Table 1.
Baseline demographics of the full cohort and derivation and validation cohorts.
| Full cohort (n=1,653) | Derivation cohort (n=827) | Validation cohort (n=826) | p value | |
|---|---|---|---|---|
| Age | 66.4±12.9 | 66.2±12.6 | 66.7±13.1 | 0.410 |
| Male | 823 (49.8%) | 412 (49.8%) | 411 (49.8%) | 0.980 |
| Other | 102 (6.2%) | 51(6.2%) | 51 (6.2%) | |
| FAST-MAG | 401 (24.2%) | 201 (24.3%) | 200 (24.2%) | |
| Hypertension (n=1,639) | 1,215 (74.1%) | 610 (74.4%) | 605 (73.9%) | 0.810 |
| Hyperlipidemia (n=1,561) | 719 (46.1%) | 354 (44.9%) | 365 (47.2%) | 0.363 |
| Atrial fibrillation (n=1,632) | 396 (24.3%) | 195 (23.8%) | 201 (23.8%) | 0.667 |
| Diabetes mellitus (n=1,646) | 371 (22.5%) | 192 (23.3%) | 179 (21.8%) | 0.459 |
| Cryptogenic | 233 (18.9%) | 114 (18.5%) | 119 (19.4%) | |
| Right-handed (n=1,459) | 1,349 (92.5%) | 678 (92.6%) | 671 (92.3%) | 0.859 |
| Right hemisphere stroke (n=1,443) | 757 (52.5%) | 378 (52.2%) | 379 (52.7%) | 0.710 |
| Intravenous alteplase | 1,077 (65.2%) | 534 (64.6%) | 543 (65.7%) | 0.618 |
| Endovascular therapy | 399 (24.1%) | 200 (24.2%) | 199 (24.1%) | 0.965 |
| Baseline NIHSS (n=1,645) | 15, 10–19 | 15, 10–19 | 15, 10–19 | 0.770 |
| 90-day modified Rankin Scale (n=1,536) | 3, 2–4 | 3, 2–4 | 3, 2–4 | 0.323 |
| Persistent arm impairment | 803 (48.6%) | 406 (49.1%) | 397 (48.1%) | 0.675 |
Binary variables shown as n (%); ordinal variables as median, IQR; and interval variables as mean±SD. Intergroup differences between the derivation and validation cohorts tested with the Chi-squared test for binary variables, Wilcoxon rank sum for ordinal variables, and Student’s t-test for interval variables.
We used restrictive LASSO14 to select the components of our risk index. LASSO covariate selection for regression models is a methodology that is preferred to Stepwise approaches based on its use of a tuning parameter to penalize the number of covariates in the model.15 Restrictive LASSO was employed because of its ability to overcome the potential for multicollinearity and to parse the number of candidate variables to a minimum, which is desirable when developing a streamlined risk score.16 With this approach, the following variables were selected as final components of the index: one point each for age ≥55 years and NIHSS values of worse arm=4, worse leg≥3, facial palsy=3, and total NIHSS score ≥10, for a range of possible scores of 0–5. We calculated the score separately, once using the 24-hour NIHSS values and again using discharge/day 4–10 NIHSS values. In the NINDS tPA trial, the subacute time point of NIHSS was measured between 7–10 days after AIS onset and in FAST-MAG it was at 4 days after onset, whereas in the other trials it was measured at discharge, which was a mean±SD of 5.7±3.8 days after AIS onset. We tested the predictive ability of the different time points at which NIHSS was collected using DeLong’s test, which tests if the area under the receiver operating characteristic curve (AUC) is superior for one model versus another.17
We named this index the Persistent UPPer extremity Impairment (PUPPI) index, with the range of potential scores being 0–5. We calculated the optimal cutpoint for predicting persistent arm impairment based AUC on at all possible cutpoints. For each cutpoint, the AUC, sensitivity, specificity, positive predictive value (PPV), and negative predicative value (NPV) are reported, with exact 95% binomial confidence intervals. As an exploratory analysis, we compared the AUC and PPV of the PUPPI index in patients with a baseline NIHSS of <10, 10–15, and >15. All analyses were performed in Stata 16.1 (StataCorp, College Station, TX).
Results
The derivation of the cohort is shown in Figure 1. Of the 3,548 patients with AIS enrolled in the trials, 1,351 were excluded for having a worse arm score of 0 at 24 hours, 375 for dying prior to 90-day follow-up, and 169 for having incomplete NIHSS data. The remaining harmonized cohort included 1,653 patients, of whom 320, 486, 334, 112, and 401 patients were from the NINDS tPA, ALIAS Part 2, IMS-III, DEFUSE 3, and FAST-MAG trials, respectively. The mean±SD age was 66.4±12.9 years, 49.8% were men, and 65.9% were White. Consistent with the entry criteria in these interventional trials, the severity of AIS was high (median baseline NIHSS=15); 65.2% received intravenous alteplase and 24.1% had endovascular therapy. Patients spanned all five TOAST stroke mechanism categories in the four trials that reported this information (data on stroke mechanism were not available in FAST-MAG; Table 1). The derivation cohort had 827 patients and the validation cohort had 826 patients, with baseline characteristics that were not significantly different (Table 1).
Figure 1.
Derivation of the cohort.
In the full cohort, 803/1,653 (48.6%) had the primary outcome of persistent arm impairment. This proportion did not differ significantly between the derivation and validation cohorts (Table 1). In a logistic regression model fit to persistent arm impairment with all possible values for total NIHSS (0–42), worse arm (1–4), worse leg (0–4), face (0–3), and age (18–90), the AUC was 0.85 using NIHSS score values collected at 24 hours and 0.89 using NIHSS score values collected at discharge/day 4–10. The components of the PUPPI index are shown in Figure 2 and the distribution of the index’s raw scores is in Table 2 along with AUC values that compare each possible cutpoint, highlighting that the best AUC and optimal cutpoint for a positive PUPPI index is ≥3.
Figure 2.
Components of the PUPPI index.
PUPPI index scores range from 0–5. PUPPI scores ≥3 are considered positive for predicting persistent upper extremity motor impairment at 90 days post-stroke.
Table 2.
Raw scores shown for the PUPPI index in the full cohort using age ≥55 and NIHSS values at 24 hours and discharge/day 4–10.
| PUPPI Index | Number of patients | Persistent arm impairment (n, %) | No persistent arm impairment (n, %) | Area under the curve (cut point comparison) |
|---|---|---|---|---|
| Using NIHSS values at 24 hours and age≥55 | ||||
| 0 | 108 | 14 (13.0%) | 94 (87.0%) | - |
| 1 | 495 | 91 (18.4%) | 404 (81.6%) | 0.55 (0 vs 1–5) |
| 2 | 281 | 933 (33.1%) | 188 (66.9%) | 0.73 (0–1 vs. 2–5) |
| 3 | 293 | 202 (68.9%) | 91 (31.1%) | 0.78 (0–2 vs. 3–5) |
| 4 | 446 | 381 (85.4%) | 65 (14.6%) | 0.71 (0–3 vs. 4–5) |
| 5 | 30 | 22 (73.3%) | 8 (26.7%) | 0.51 (0–4 vs. 5) |
| Using NIHSS values at discharge/4–10 days and age≥55 | ||||
| 0 | 146 | 17 (11.6%) | 129 (88.4%) | - |
| 1 | 613 | 101 (16.5%) | 512 (83.5%) | 0.57 (0 vs 1–5) |
| 2 | 209 | 97 (46.4%) | 112 (53.6%) | 0.80 (0–1 vs. 2–5) |
| 3 | 262 | 204 (77.9%) | 58 (22.1%) | 0.81 (0–2 vs. 3–5) |
| 4 | 393 | 358 (91.1%) | 35 (8.9%) | 0.72 (0–3 vs. 4–5) |
| 5 | 30 | 26 (86.7%) | 4 (13.3%) | 0.51 (0–4 vs. 5) |
Using the ≥3 cutpoint for a positive PUPPI index, the AUC in the derivation cohort, using NIHSS scores values collected at 24 hours, was 0.78, with a sensitivity, specificity, and PPV of 75.4%, 80.5%, and 78.9% respectively (Table 3). The ≥3 cutpoint of the PUPPI index performed comparably in the validation cohort with an AUC of 0.78. Using NIHSS score values collected at discharge/day 4–10 yielded even better performance with AUCs of 0.81 and 0.80 in the derivation and validation cohorts, and PPVs of 87.8% and 84.2%, respectively (Table 3). The difference in AUC between 24-hour and discharge/day 4–10 NIHSS was significantly different using DeLong’s test (0.78 vs 0.81, p=0.003). In comparison, the AUC and sensitivity for using worse arm score ≥3 alone as a predictor was poorer: 0.75 and 66.9% using 24-hour NIHSS values, and 0.79 and 69.7% for discharge/day 4–10 NIHSS values. Thus, using worse arm motor impairment alone has a modest ability to predict persistent arm impairment, but at a considerable loss of sensitivity compared to the PUPPI index.
Table 3.
Performance characteristics of the PUPPI index at a cutpoint of 0–2 (negative) vs. 3–5 (positive) in the derivation, validation, and full cohort.
| Cohort | Persistent arm impairment (PAI) (n, %) | AUC for PUPPI index with cutpoint ≥3 (95% CI) | Sensitivity (95% CI) | Specificity (95% CI) | NPV (95% CI) | PPV (95% CI) |
|---|---|---|---|---|---|---|
| Using NIHSS values at 24 hours and age≥55 | ||||||
| Derivation (n=827) | 406/827, 49.1% | 0.78 (0.75–0.81) | 75.4% (70.9–79.5) | 80.5% (76.4–84.2) | 77.2% (73.0–81.1) | 78.9% (74.5–82.8) |
| Validation (n=826) | 397/826, 48.1% | 0.78 (0.75–0.81) | 75.3% (70.8–79.5) | 80.9% (76.8–84.5) | 78.0% (73.8–81.7) | 78.5% (74.0–82.5) |
| Full cohort (n=1,653) | 803/1,653, 48.6% | 0.78 (0.76–0.80) | 75.3% (72.2–78.3) | 80.7% (77.9–83.3) | 77.6% (74.7–80.3) | 78.7% (75.6–81.5) |
| Using NIHSS values at discharge/4–10 days and age≥55 | ||||||
| Derivation (n=827) | 406/827, 49.1% | 0.81 (0.78–0.84) | 71.2% (65.7–76.2) | 90.7% (87.0–93.6) | 77.0% (72.5–81.2) | 87.8% (83.0–91.6) |
| Validation (n=826) | 397/826, 48.1% | 0.80 (0.77–0.83) | 73.2% (67.9–78.1) | 86.8.0% (82.5–90.3) | 77.0% (72.3–81.3) | 84.2% (79.3–88.4) |
| Full cohort (n=1,653) | 803/1,653, 48.6% | 0.81 (0.79–0.83) | 73.2% (70.0–76.3) | 88.7% (78.3–84.5) | 77.0% (73.8–80.0) | 85.9% (82.6–88.8) |
In the exploratory analysis with patients stratified into three groups according to baseline NIHSS total score <10 (n=376), 10–15 (n=502), and >15 (n=767), the AUC/PPV using 24-hour NIHSS values were 0.66/71.9%, 0.75/76.1%, and 0.77/80.2%, respectively, and for the discharge/day 4–10 NIHSS scores, the AUC/PPV was 0.71/77.8%, 0.75/83.4%, and 0.84/87.7%.
Discussion
Accurate prediction of long-term outcomes during the initial hours-to-days post-stroke has high potential value in clinical trials and clinical practice. Here we show that accurate prediction of persistent arm impairment in patients with AIS at 90 days post-stroke is possible using a new tool, the PUPPI index, which can be easily calculated at the bedside using readily available data. The PUPPI index, using a cutpoint of ≥3 to define positivity, performed well using NIHSS score values obtained 24 hours after AIS onset and significantly better using subacute values obtained at discharge or day 4–10 (p=0.003). This finding highlights that post-stroke prediction benefits from greater time of disease expression, although the performance at 24 hours was good enough that the PUPPI index can be performed anytime from 24 hours to discharge.
We were able to derive and validate the PUPPI index using data from five landmark trials and across patients with differing levels of baseline stroke severity. Similar to a small prior study, we found that age and the NIHSS total score and motor domains alone were sufficient to create our predictive model.18 We did not find that the addition of other NIHSS domains, such as language, sensory, neglect, or vision, or the addition of demographics such as sex, medical comorbidities, or TOAST stroke mechanism, improved the ability of the PUPPI index to predict persistent arm impairment. The predictive value of the PUPPI index is also independent of acute interventions such as thrombolytics and thrombectomy.
While there are existing risk indices for predicting AIS patients’ UE motor function at follow-up, they rely on advanced diagnostic testing, such as neuroimaging or transcranial magnetic stimulation,2,6 or more complex impairment scores that are not routinely used in clinical practice, such as the Fugl-Meyer Assessment.3,7 In contrast to prior risk indices, we focused on predicting persistent arm impairment, defined as a worse NIHSS arm domain score of 2–4 at day-90 follow-up, which corresponds to moderate to severe UE motor impairment on the Fugl-Meyer.19 We chose this approach in part because one intended use for the PUPPI index is to identify patients who are optimal candidates for enrollment in clinical trials evaluating restorative interventions to reduce persistent arm impairment. Future research is needed to develop the PUPPI index for other uses, such as prognosticating which patients will recover UE motor function or how intensity of rehabilitation could impact UE motor function.
Study Limitations
Our study has several important limitations. We did not have a separate dataset to externally validate the results of our analysis. The five clinical trials in our cohort did not include detailed information on patient’s discharge destination, so we were not able to account for the potential confounding effects of the type of facility the patient was discharged to and the available resources at the facility.20 Doing so might improve the predictive accuracy of the PUPPI index, although prior research has shown that with the exclusion of outliers in motor recovery there is little variance in the prediction of motor recovery, suggesting that the type of facility and amount of rehabilitation provided with the current approach to standard of care in the U.S. may not be necessary to predict recovery.21 There is also selection bias inherent to using patients enrolled in clinical trials with an intervention, but the rigor of outcome adjudication and the large sample size work to offset this bias. The PUPPI index’s PPV may vary according to characteristics of the population in which it is applied, although it performed well across all three strata of baseline NIHSS score. Finally, we did not predict gradations of UE motor function at follow-up. Future research may benefit from utilizing this approach of combining arm, leg, and face motor impairment in the week after stroke with age and total functional impairment to explore if more detailed and accessible prognostication of patient-specific UE motor function is possible.
Conclusion
The PUPPI index uses readily available information to provide accurate prediction of persistent upper extremity motor impairment at 90 days from AIS onset. The index can be calculated in minutes at the bedside and could be used as an inclusion criterion or stratification variable in rehabilitation research and, with additional development, as a prognostic index for patients, caregivers, and clinicians.
Acknowledgements:
This data was presented as an oral abstract at the International Stroke Conference in 2021, which was a virtual meeting. This article was prepared using datasets obtained from NINDS and does not necessarily reflect the opinions or views of NINDS.
Sources of Funding:
Dr. de Havenon: NIH-NINDS K23NS105924. Dr. Lindgren: Swedish Research Council (2019-01757), Swedish Government, Swedish Heart and Lung Foundation, Region Skåne, Lund University, Skåne University Hospital, Sparbanksstiftelsen Färs och Frosta, Fremasons Lodge of Instruction Eos Lund. Dr. Worrall: Australian-American Fulbright Commission. Dr. Braun: NIH/NICHD (K12HD093427). Dr. Cole: American Heart Association (AHA)-Bayer Discovery Grant (Grant 17IBDG33700328), the AHA Cardiovascular Genome-Phenome Study (Grant-15GPSPG23770000), NIH (Grants: R01-NS114045; R01-NS100178; R01-NS105150), and the US Department of Veterans Affairs.
Disclosures:
Dr. Lindgren receives personal fees from Bayer, Astra Zeneca, BMS Pfizer, and Portola outside this work. Dr. Cramer serves as a consultant for Abbvie, Constant Therapeutics, MicroTransponder, Neurolutions, Regenera, SanBio, Stemedica, Fujifilm Toyama Chemical Co., Biogen, and TRCare. Dr. Worrall is Deputy Editor for Neurology. Dr. de Havenon receives research funding from AMAG and Regeneron. The other authors have nothing to disclose.
Abbreviations
- PUPPI
Persistent UPPer extremity Impairment
- NIHSS
NIH Stroke Scale
- AUC
Area under the receiver operating characteristic curve
- PPV
Positive predictive value
- NPV
Negative predicative value
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.van Lieshout ECC, van de Port IG, Dijkhuizen RM, Visser-Meily JMA. Does upper limb strength play a prominent role in health-related quality of life in stroke patients discharged from inpatient rehabilitation? Top. Stroke Rehabil 2020;27:525–33. [DOI] [PubMed] [Google Scholar]
- 2.Stinear Cathy M, Byblow Winston D, Ackerley Suzanne J, Barber P Alan, Smith Marie-Claire. Predicting Recovery Potential for Individual Stroke Patients Increases Rehabilitation Efficiency. Stroke. 2017;48:1011–9. [DOI] [PubMed] [Google Scholar]
- 3.Boyd LA, Hayward KS, Ward NS, Stinear CM, Rosso C, Fisher RJ, et al. Biomarkers of stroke recovery: Consensus-based core recommendations from the Stroke Recovery and Rehabilitation Roundtable. Int. J. Stroke. 2017;12:480–93. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Nijland Rinske HM, van Wegen Erwin EH, Harmeling-van der Wel Barbara C, Kwakkel Gert. Presence of Finger Extension and Shoulder Abduction Within 72 Hours After Stroke Predicts Functional Recovery. Stroke. 2010;41:745–50. [DOI] [PubMed] [Google Scholar]
- 5.Malmut L, Lin C, Srdanovic N, Kocherginsky M, Harvey RL, Prabhakaran S. Arm Subscore of Motricity Index to Predict Recovery of Upper Limb Dexterity in Patients With Acute Ischemic Stroke. Am. J. Phys. Med. Rehabil 2020;99:300–4. [DOI] [PubMed] [Google Scholar]
- 6.Lin David J, Cloutier Alison M, Erler Kimberly S, Cassidy Jessica M, Snider Samuel B, Ranford Jessica, et al. Corticospinal Tract Injury Estimated From Acute Stroke Imaging Predicts Upper Extremity Motor Recovery After Stroke. Stroke. 2019;50:3569–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.See J, Dodakian L, Chou C, Chan V, McKenzie A, Reinkensmeyer DJ, et al. A standardized approach to the Fugl-Meyer assessment and its implications for clinical trials. Neurorehabil. Neural Repair. 2013;27:732–41. [DOI] [PubMed] [Google Scholar]
- 8.Archived Clinical Research Datasets | National Institute of Neurological Disorders and Stroke [Internet]. [cited 2020 Dec 21];Available from: https://www.ninds.nih.gov/Current-Research/Research-Funded-NINDS/Clinical-Research/Archived-Clinical-Research-Datasets
- 9.NINDS t-PA Stroke Study. Tissue Plasminogen Activator for Acute Ischemic Stroke. N. Engl. J. Med 1995;333:1581–8. [DOI] [PubMed] [Google Scholar]
- 10.Ginsberg MD, Palesch YY, Hill MD, Martin RH, Moy CS, Barsan WG, et al. High-dose albumin treatment for acute ischaemic stroke (ALIAS) part 2: a randomised, double-blind, phase 3, placebo-controlled trial. Lancet Neurol. 2013;12:1049–58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Broderick JP, Palesch YY, Demchuk AM, Yeatts SD, Khatri P, Hill MD, et al. Endovascular Therapy after Intravenous t-PA versus t-PA Alone for Stroke. N. Engl. J. Med 2013;368:893–903. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Albers GW, Marks MP, Kemp S, Christensen S, Tsai JP, Ortega-Gutierrez S, et al. Thrombectomy for Stroke at 6 to 16 Hours with Selection by Perfusion Imaging. N. Engl. J. Med 2018;378:708–18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Saver JL, Starkman S, Eckstein M, Stratton SJ, Pratt FD, Hamilton S, et al. Prehospital Use of Magnesium Sulfate as Neuroprotection in Acute Stroke. N. Engl. J. Med 2015;372:528–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Koch B, Vock DM, Wolfson J. Covariate selection with group lasso and doubly robust estimation of causal effects. Biometrics. 2018;74:8–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Desboulets LDD. A Review on Variable Selection in Regression Analysis. Econometrics. 2018;6:1–27. [Google Scholar]
- 16.Kayanan M, Wijekoon P. Stochastic Restricted LASSO-Type Estimator in the Linear Regression Model. J. Probab. Stat 2020;2020:e7352097. [Google Scholar]
- 17.DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–45. [PubMed] [Google Scholar]
- 18.Kwah LK, Harvey LA, Diong J, Herbert RD. Models containing age and NIHSS predict recovery of ambulation and upper limb function six months after stroke: an observational study. J. Physiother 2013;59:189–97. [DOI] [PubMed] [Google Scholar]
- 19.Hiragami S, Inoue Y, Harada K. Minimal clinically important difference for the Fugl-Meyer assessment of the upper extremity in convalescent stroke patients with moderate to severe hemiparesis. J. Phys. Ther. Sci 2019;31:917–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Deutsch A, Granger CV, Heinemann AW, Fiedler RC, DeJong G, Kane RL, et al. Poststroke Rehabilitation. Stroke. 2006;37:1477–82. [DOI] [PubMed] [Google Scholar]
- 21.Prabhakaran S, Zarahn E, Riley C, Speizer A, Chong JY, Lazar RM, et al. Inter-individual variability in the capacity for motor recovery after ischemic stroke. Neurorehabil. Neural Repair. 2008;22:64–71. [DOI] [PubMed] [Google Scholar]