The Effect of Adjusting for Baseline Stroke Severity in the National Inpatient Sample

Prior research has shown that stroke severity measured on the NIH Stroke Scale (NIHSS) is important for accurate modeling of post-stroke outcome.^1,2 However, NIHSS scores in stroke registries, such as Get With The Guidelines, are not readily available to researchers. In October of 2016, the Centers for Medicare & Medicaid Services released an ICD-10-CM code (R29.7x) for documentation of admission NIHSS, to be used in conjunction with the diagnosis of ischemic stroke (I63.x).³ To determine if an administratively coded NIHSS improved the accuracy of outcome prediction in ischemic stroke patients, we created a cohort from October 2016 to December 2018 of National Inpatient Sample (NIS) non-elective hospital discharges aged ≥18 with a primary discharge diagnosis of ischemic stroke (ICD-10-CM I63.x) and a documented NIHSS. We conformed to the RECORD guidelines and because our study used de-identified data, it was exempt from IRB approval. The data is publicly available at https://www.hcup-us.ahrq.gov.

To evaluate the accuracy of multivariable models fit to the study outcomes of in-hospital death and good outcome (discharge to home or self-care) with and without adjustment for NIHSS, we compared the models’ areas under the receiver operating curves (AUC) using DeLong’s method, which provides a statistical measure of the difference in AUC.⁴ Models were adjusted for age, sex, race/ethnicity, patient urban-rural residence, median household income by ZIP code, expected primary payor, endovascular thrombectomy (EVT), intravenous alteplase (tPA), intubation, APR-DRG mortality risk (model fit to in-hospital death) or APR-DRG severity of illness (model fit to good outcome), Elixhauser Comorbidity Index, hospital Census region, teaching status, and bed size (small/medium/large). As a measure of multicollinearity, we report the models’ mean variance inflation factor (VIF), with a VIF of >5 indicating unacceptable multicollinearity. Statistical analysis was performed using STATA version 16.1 (Stata Corp, College Station, TX).

Of 224,995 discharges with a primary diagnosis of ischemic stroke, we included 76,801 (34.1%) with an available NIHSS (Supplemental Figure 1), of which 49.1% were female, 15.1% received tPA, 7.3% had EVT, and the mean (SD) age was 69.7 (14.0) years. The median (IQR) NIHSS was 4 (2–10), 3.8% had in-hospital death and 35.7% had good outcome. The AUC of the multivariable model fit to in-hospital death without NIHSS was 0.924 (95% CI 0.920–0.929), while with NIHSS it increased to 0.940 (95% CI 0.937–0.943) (p<0.001) (Supplemental Figure 1). The AUC of the multivariable model fit to good outcome increased from 0.778 (95% CI 0.776–0.782) without NIHSS to 0.817 (95% CI 0.814–0.820) with NIHSS (p<0.001) (Supplemental Figure 2). The mean VIF of both models was <3 and the VIF of NIHSS in both models was <3, indicating acceptable multicollinearity.

Several important differences in the point estimates emerged when comparing models before and after adjustment for NIHSS. Without adjustment for NIHSS, EVT was associated with lower odds of good outcome (OR 0.93, 95% CI 0.86–0.99). With NIHSS in the model, a more accurate picture of EVT’s effect on good outcome emerged (OR 2.45, 95% CI 2.24–2.68), consistent with the HERMES collaboration meta-analysis, which reported an odds ratio of 2.49 (95% CI 1.76–3.53).⁵ Without adjustment for NIHSS, women had a higher odds of death than men (OR 1.12, 95% CI 1.02–1.22), but the association did not retain significance after adjustment for NIHSS (OR 0.99, 95% CI 0.90–1.08). Likewise, the point estimate of tPA’s effect on good outcome changed with NIHSS in the model (OR 2.01, 95% CI 1.91–2.12), compared to without NIHSS (OR 1.39, 95% CI 1.31–1.44). Forest plots of the odds ratios for patient sex, EVT, and tPA before and after adjustment for NIHSS are seen in Figure 1 and a forest plot of the full model is shown in Supplemental Figure 3.

Figure 1.

Multivariable models without and with NIH Stroke Scale (NIHSS) showing odds ratios for patient sex, endovascular thrombectomy (EVT), and intravenous alteplase (tPA).

In patients with ischemic stroke, the addition of ICD-coded admission NIHSS to adjusted models fit to in-hospital death or good outcome improves accuracy. Outside-of-disease adjustments, such as the Charlson, Elixhauser or APR-DRG, are important, but a within-disease measure like NIHSS is critical for epidemiologic and health service researchers and quality officers. The results of the VIF testing confirm that NIHSS was not collinear with other severity adjustments, rather it was additive. The main limitations of our analysis include selection bias from focusing on a subgroup of patients with an available NIHSS and the lack of longer-term outcome data after discharge. Nonetheless, we show that associations between post-stroke outcome and patient sex or stroke interventions require adjustment for NIHSS to reveal the true direction and size of effect. This has implications for the interpretation of prior ischemic stroke research that used administrative datasets without NIHSS data.

Nonstandard Abbreviations:

ARP-DRG

All Patients Refined Diagnosis Related Groups

NIS

National Inpatient Sample