Skip to main content
Journal of General Internal Medicine logoLink to Journal of General Internal Medicine
. 2021 Oct 26;37(11):2719–2726. doi: 10.1007/s11606-021-07162-0

Medicare Claim–Based National Institutes of Health Stroke Scale to Predict 30-Day Mortality and Hospital Readmission

Amit Kumar 1,2, Indrakshi Roy 2, Pamela R Bosch 1, Corey R Fehnel 3, Nicholas Garnica 1, Jon Cook 4, Meghan Warren 1, Amol M Karmarkar 5,6,
PMCID: PMC9411458  PMID: 34704206

Abstract

Background

The Centers for Medicare and Medicaid Services (CMS) penalizes hospitals for higher than expected 30-day mortality rates using methods without accounting for condition severity risk adjustment. For patients with stroke, CMS claims did not quantify stroke severity until recently, when the National Institutes of Health Stroke Scale (NIHSS) reporting began.

Objective

Examine the predictive ability of claim-based NIHSS to predict 30-day mortality and 30-day hospital readmission in patients with ischemic stroke.

Design

Retrospective cohort study of Medicare claims data.

Patients

Medicare beneficiaries with ischemic stroke (N=43,241) acute hospitalization between October 2016 and November 2017.

Measurements

All-cause 30-day mortality and 30-day hospital readmission. NIHSS score was derived from ICD-10 codes and stratified into the following: minor to moderate, moderate, moderate to severe, and severe categories.

Results

Among 43,241 patients with ischemic stroke with NIHSS from 2,659 US hospitals, 64.6% had minor to moderate stroke, 14.3% had moderate, 12.7% had moderate to severe, and 8.5% had a severe stroke,10.1% died within 30 days, 12.1% were readmitted within 30 days. The NIHSS exhibited stronger discriminant property (C-statistic 0.83, 95% CI: 0.82–0.84) for 30-day mortality compared to Elixhauser (0.74, 95% CI: 0.73–0.75). A monotonic increase in the adjusted 30-day mortality risk occurred relative to minor to moderate stroke category: hazard ratio [HR]=2.92 (95% CI=2.59–3.29) for moderate stroke, HR=5.49 (95% CI=4.90–6.15) for moderate to severe stroke, and HR=7.82 (95% CI=6.95–8.80) for severe stroke. After accounting for competing risk of mortality, there was a significantly higher readmission risk in the moderate stroke (HR=1.11, 95% CI=1.03–1.20), but significantly lower readmission risk in the severe stroke (HR=0.84, 95% CI=0.74–0.95) categories.

Limitation

Timing of NIHSS reporting during hospitalization is unknown.

Conclusions

Medicare claim–based NIHSS is significantly associated with 30-day mortality in Medicare patients with ischemic stroke and significantly improves discriminant property relative to the Elixhauser comorbidity index.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11606-021-07162-0.

KEY WORDS: stroke severity, Elixhauser, administrative data, CMS, competing risk, survival analysis

INTRODUCTION

Stroke is a leading cause of hospitalization and is associated with high mortality, hospital readmission rates, and long-term disability.1 As part of the Quality Reporting Program (QRP), the Centers for Medicare and Medicaid Services (CMS) has developed stroke-specific risk-standardized 30-day mortality and 30-day readmission rates.2,3 However, current risk adjustment methods using administrative claims data to capture patient-level condition severity remain far from being adequate and precise.4,5 Administrative claims records have inadequate condition-specific severity and no function-related information, limiting predictive accuracy of the current risk-adjustment methods.6,7 This also results in residual confounding, which ultimately compromises the intended objective of risk adjustment and standardization methods to eliminate patient-level case-mix differences for comparing performances on quality measures (QMs) between hospitals.

The National Institutes of Health Stroke Scale (NIHSS) was developed to quantify stroke severity and has the potential to guide clinical decision making, while improving risk standardization methods. The American Heart Association/American Stroke Association have long recommended the use of the NIHSS as a validated risk assessment tool for capturing stroke condition severity.8,9 In October 2016, CMS implemented the International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) codes for NIHSS, allowing hospitals to record and report NIHSS score. CMS plans to incorporate NIHSS scores into their risk-standardized 30-day mortality rate for stroke beginning in fiscal year 2023.10,11 Prior research has also shown that admission NIHSS score is a valid method for predicting in-hospital mortality for patients with stroke.1214

Inclusion of the NIHSS has been shown to improve the predictive ability of risk-adjustment methods for 30-day mortality after stroke.14 However, validation of the NIHSS was not done previously using administrative claims data. Another methodological issue is that these studies have computed 30-day hospital readmission and 30-day mortality rates as mutually exclusive events, without accounting for the occurrence of the other (e.g., mortality precluding hospital readmission) in that time window.1518 Recent work by Saber and Saver has shown the distribution of NIHSS scores in a national sample using Healthcare Cost and Utilization Project (HCUP) data.19 However, this paper did not examine the predictive ability of the NIHSS for patient outcomes such as 30-day readmission and mortality. The objectives of this study were to (1) examine the discriminant performance of Medicare claim–based NIHSS scores for 30-day mortality and 30-day hospital readmission rates and to (2) examine the association between the NIHSS and 30-day hospital readmissions and 30-day mortality rates while controlling for the occurrence of the other (competing) event during that time window.

METHODS

Data

We used 100% patient-level Medicare inpatient claims data which represent administrative claims for all short-term hospitalization of Medicare fee-for-service beneficiaries from October 1, 2016, to December 31, 2017. We extracted information about all diagnostic conditions, surgical procedures, length of stay, and calculated NIHSS. The claims data file was linked with the Master Beneficiary Summary File (MBSF), the Provider of Services file, and the MBSF: Chronic Conditions Segment data to retrieve patient and hospital characteristics. The Master Beneficiary Summary File contains information on beneficiaries’ sociodemographic characteristics and indicators for Medicare enrollment. The chronic condition segment was used to determine the prior history of stroke. The provider of services file was used to retrieve information on hospital-level data such as urban setting and safety-net status of the hospital.

Study Population

The study cohort included Medicare beneficiaries aged 66 and older on fee-for-service(FFS) plans who were admitted to acute or critical access hospitals between October 1, 2016, and November 30, 2017, with an admitting diagnosis of ischemic stroke. Since NIHSS reporting started in October 2016, our cohort includes data from that time. The diagnosis of ischemic stroke was identified using MS-DRG (061, 062, 063) or ICD-10 CM codes (eTable 1). The sample was restricted to patients who had 9 months of continuous Medicare FFS enrollment before and 1 month after the index hospitalization. Patients enrolled in Medicare based on disability or end-stage renal disease (ESRD), who left the hospital against medical advice, were excluded from our sample (Figure 1). Patients (N=1417) admitted from nursing homes, long-term care hospitals, hospices were excluded. Out of this, 43.4% had minor to moderate stroke, 20.2% had a moderate stroke, 20.5% had moderate to severe stroke, and 15.9% had a severe stroke. The final sample comprised 43,241 FFS patients admitted with acute ischemic stroke.

Figure 1.

Figure 1.

The derivation of the cohort

Claim-Based NIHSS

We calculated the NIHSS from ICD-10CM codes associated with acute hospitalization (eTable 2 in the supplement). NIHSS includes 15 items to evaluate the effect of acute cerebral infarction on the levels of consciousness, language, motor strength, ataxia, dysarthria, and sensory loss.20 The NIHSS is administered and documented within 12 h of arrival to an emergency department in patients with ischemic stroke. NIHSS scores range from 0 to 42, with higher values indicating more severe stroke. Our study used the valid four-level NIHSS categories (score 0–7= minor to moderate stroke, 8–13= moderate stroke, 14–21 moderate to severe stroke, and 22–42 = severe stroke), which have demonstrated excellent discriminant properties for 30-day mortality risk.1214 In Figure 2, we have presented the distribution of NIHSS scores among our sample, and for those patients who died and or were readmitted within 30 days.

Figure 2.

Figure 2.

Distribution of National Institutes of Health Stroke Scale (NIHSS) scores in Medicare beneficiaries with ischemic stroke in the USA

Outcomes

The primary outcomes were (1)all-cause 30-day mortality and (2)all-cause 30-day hospital readmission. The date of death was retrieved from the MBSF, with the 30-day period starting from the date of acute hospitalization discharge. We defined all-cause risk-adjusted hospital readmissions (yes/no) as per CMS definition using the 30-day period from the date of index acute hospitalization discharge.

Patient-Level Covariates

We included lengths of stay during acute hospitalization, intensive care unit (ICU) stay (yes/no), receipt of tissue plasminogen activator (tPA), use of a feeding tube, and Elixhauser comorbidity index. The Elixhauser index consists of 30 medical conditions and has shown a high association with acute hospital mortality, length of stay, and hospital charges.21 Comorbidity indices derived from claims data alone do not fully capture prevalence of chronic conditions.22 Therefore, in addition, we determined history of stroke from the chronic conditions segment. We identified tPA and feeding tube use by ICD10-CM procedure codes. We also categorized post-acute discharge destinations after index acute hospitalization into inpatient rehabilitation facilities (IRF), skilled nursing facilities (SNF), home with home healthcare (HH), and home without HH (reference group). Hospital-level variables included urban/rural, safety-net status, and volume of patients with stroke treated annually.

Statistical Analyses

Descriptive statistics were stratified by NIHSS categories for patient demographic, clinical, and hospital characteristics. Chi-square tests were used for categorical variables and t-tests for continuous variables to examine differences by NIHSS categories. Four different logistic regression models were compared (C-statistics) to examine the discriminative property of the NIHSS to predict 30-day mortality and 30-day hospital readmission. The base model included age, gender, race, and Medicare-Medicaid dual eligibility status. The second model included the NIHSS with the base model. The third model included the Elixhauser comorbidity index with the base model. In the fourth model, we added both NIHSS and Elixhauser comorbidity index. Potential C-statistic values range from 0.5, meaning the model does no better than chance, to 1.0, meaning perfect discrimination, with a value higher than 0.70 is considered clinically relevant.23

Past research has computed and estimated hospital readmission risk without accounting for occurrence of a competing event, such aspost-discharge mortality.1518 Competing risks arise where individuals are subject to a number of potential failure events and the occurrence of one event might impede the occurrence of other events. In this context, after acute hospitalization, a patient with stroke might experience hospital readmission, or the patient might die. For readmission, death is a competing event, because hospital readmission can never occur; thus, the risk of readmission is zero. To mitigate this issue, we used competing risk analysis, a specialized form of time-to-event analysis, where the cause-specific hazard function generalizes the hazard function to the competing risks setting, describing the rate of failure from one event (30-day hospital readmission) in the presence of a competing event (30-day mortality) and vice versa. The cumulative incidence function using the Fine and Gray method24 was used to quantify the risk of failure from a particular event type when there are competing risks.

Sensitivity Analysis

We presented the distribution of NIHSS in Medicare beneficiaries by alive and 30-day mortality in Figure 2. Secondly, we performed the competing risk analysis after excluding patients discharged to hospice from acute hospitals. Third, to address the concern that prior stroke may influence the NIHSS score, we performed a subgroup analysis after excluding patients with a previous history of stroke. All statistical tests were 2-sided with a P < .05 indicating statistical significance. All analyses were conducted using SAS, version 9.4 (SAS Institute Inc.).

RESULTS

There were 63,147 Medicare fee-for-service beneficiaries admitted to 2,659 hospitals in the USA for ischemic stroke from October 1, 2016, through November 2017, and reported NIHSS scores. Our final analytical sample includes 43,241 patients with NIHSS after applying exclusion criteria described in Figure 1. Among 43,241 patients, 2,449 (5.6%) patients died during the initial hospital stay and 10.1% died within the 30-day period from the index discharge date. The mean (SD) age of the study population was 80 (8.4) years, 44.8% were male, 82.8% White, 9.1% Black, 3.8% Hispanic, and 16.5% were dual-eligible in Medicare and Medicaid. Patient demographic, clinical, and hospital characteristics categorized by NIHSS severity are presented in Table 1. Most patients (64.6%) were in the minor to moderate stroke category; patients diagnosed in a more severe NIHSS category were more likely to be older, female, and non-White and have had a previous stroke history. Patients in the most severe NIHSS category had the highest proportion of being admitted to an intensive care unit (ICU), having a feeding tube placed, and having a high 30-day mortality rate. Patients in the severe stroke category were also more likely to be treated at safety-net hospitals that treat a high volume of stroke patients. The distribution of 30-day post-discharge mortality and readmission by NIHSS score is presented in Figure 3.

Table 1.

Descriptive Characteristics of Medicare Patients with Ischemic Stroke in the USA by NIHSS Categories

Variables Overall cohort
N=43,241 (100%)
NIHSS categories P-value
0–7
N=27,935 (64.6%)
8–13
N=6,162 (14.3%)
14–21
N=5,483 (12.7%)
22–42
N=3,661 (8.5%)
Age (years) 80 (8.4) 79 (8.1) 80.8 (8.5) 81.9 (8.4) 83.2 (8.4) <.0001
Male (%) 44.8 48.0 41.9 38.4 34.8 <.0001
Race (%)
White 82.8 83.5 81.5 82.4 80.2 <.0001
Black 9.1 8.7 9.9 9.8 9.3
Hispanic 3.8 3.4 4.6 3.9 5.1
Dual status 16.5 13.7 20.8 21.3 22.8 <.0001
Clinical characteristics
Length of stay (mean ±SD) 5.6 (4.1) 4.8 (3.0) 6.5 (5.0) 7.5 (5.0) 7.2 (5.9) <.0001
Elixhauser score 4.3 (1.9) 3.9 (1.9) 4.7 (1.9) 5.1 (1.9) 5.1 (1.9) <.0001
ICU admission 37.4 26.1 48.6 62.9 66.2 <.0001
Feeding tube 7.0 5.5 8.6 9.9 11.3 <.0001
tPA 52.9 57.8 56.6 39.4 29.8 <.0001
Thrombectomy 6.8 1.5 9.3 21.5 21.6 <.0001
History of stroke 31.2 29.7 33.2 32.9 35.9 <.0001
30-day readmission 12.1 11.6 14.9 13.8 9.3 <.0001
30-day mortality 10.1 3.7 11.7 26.3 44.8 <.0001
Post-acute discharge destination, %
Home 27.5 36.4 12.8 7.3 4.2 <.0001
Home with HH 15.5 16.8 12.2 12.2 15.6
SNF 22.3 18.5 29.9 31.0 29.1
IRF 27.0 25.0 36.6 30.7 20.2
Other 7.7 3.3 8.5 18.8 30.9
Hospital characteristics, %
Urban setting 94.1 93.8 94.3 95.4 94.2 <.0001
Types of hospital
Safety-net hospital 24.8 23.2 26.6 28.1 29.5 <.0001
Non-safety-net hospital 74.2 75.9 72.2 70.8 69.2
Critical access hospital 1.0 0.9 1.2 1.1 1.3
Stroke volume
Low (0.6–5.4) 1.2 1.3 1.1 0.9 1.2 <.0001
Medium (5.4–9.4) 38.0 39.5 36.9 34.5 34.0
High (9.4–68.3) 60.8 59.3 62.0 64.6 64.9

Values are presented as “%” for categorical variables and mean ± standard deviation (SD) for continuous variables. Length of stay is based on the number of days (mean ± SD was reported). HH home health, SNF skilled nursing facility, IRF inpatient rehabilitation facilities. Other includes hospice, long-term hospital, home with palliative care, and other federal hospitals. ICU intensive care unit

NIHSS categories: 1= 0–7: minor to moderate stroke, 2= 8–13: moderate stroke, 3= 14–21: moderate to severe stroke, 4= 22–42: severe stroke

Figure 3.

Figure 3.

Distribution of National Institutes of Health Stroke Scale (NIHSS) scores by 30-day readmission and mortality in Medicare beneficiaries with ischemic stroke in the USA

Discriminant Property of Claim-Based NIHSS for Patient Outcomes

The C-statistics for each model predicting patient outcomes are presented in Table 2. For 30-day mortality, the C-statistic from the base model was 0.71 (95% CI: 0.70–0.72), which increased to 0.83 (95% CI: 0.82–0.84) with the addition of the NIHSS. This was a larger increase in the C-statistic compared to the addition of the Elixhauser comorbidity index to the base model (C-statistic = 0.74, 95% CI: 0.73–0.75). There was no significant change in C-statistics for mortality after adding both the Elixhauser and NIHSS to the base model. There was no corresponding improvement in C-statistic with either NIHSS or Elixhauser for the 30-day readmission outcome.

Table 2.

Comparison of C-statistics for Predicting In-hospital Mortality, 30-Day Mortality, and Hospital Readmissions

Models 30-day mortality
C-statistic (95% CI)
30-day readmission
C-statistic (95% CI)
Base model* 0.71 (0.70–0.72) 0.55 (0.54–0.56)
Base model + NIHSS 0.83 (0.82–0.84) 0.59 (0.58–0.60)
Base model + Elixhauser 0.74 (0.73–0.75) 0.59 (0.58–0.60)
Base model + Elixhauser + NIHSS 0.84 (0.83–0.86) 0.61 (0.60–0.62)

*Base model included age, gender, race, and dual eligibility

Competing Risk Analysis

For 30-day post-discharge mortality, using 30-day hospital readmission as a competing event, and minor to moderate stroke as a reference category, there was significantly higher mortality risk for moderate stroke (HR=2.92, 95% CI=2.59–3.29), for moderate to severe stroke (HR=5.49, 95% CI=4.90–6.15), and for severe stroke (HR=7.82, 95% CI=6.95–8.80) reported in Table 3. For 30-day hospital readmission, using 30-day post-discharge mortality as a competing event, and using minor to moderate stroke as a reference category, there was significantly higher readmission risk in the moderate stroke (HR=1.11, 95% CI=1.03–1.20), but significantly lower readmission risk in severe stroke (HR=0.84, 95% CI=0.74–0.95) categories. Figure 4 A and B illustrate the cumulative incidence function that summarizes the competing risk analysis for estimating 30-day mortality and 30-day readmission for the four-stroke severity groups. It indicates a monotonic relationship between NIHSS and mortality, but not with 30-day hospital readmission.

Table 3.

Hazard Ratio Estimates from Competing Risk Analysis and Cause-Specific Hazard Models

Cause-specific hazard
Event=3,422
Censored=37,113
Competing risk
Event of interest=3,422
Competing risk=5,198
Censored=31,915
Hazard ratio
95% CI
Hazard ratio
95% CI
30-day mortality NIHSS (2 vs 1)

3.02

2.69–3.39

2.92

2.59–3.29

NIHSS (3 vs 1)

5.67

5.10–6.30

5.49

4.90–6.15

NIHSS (4 vs1)

8.09

7.26–9.01

7.82

6.95–8.80

Cause-specific hazard

Event=5,198

Censored=35,337

Competing risk

Event of interest=5,198

Competing risk=3,422

Censored=31,915

Hazard ratio

95% CI

Hazard ratio

95% CI

30-day readmission NIHSS (2 vs 1)

1.16

1.07–1.25

1.11

1.03–1.20

NIHSS (3 vs 1)

1.22

1.12–1.34

1.08

0.98–1.18

NIHSS (4 vs 1)

1.08

0.95–1.22

0.84

0.74–0.95

NIHSS categories: 1= 0–7: minor to moderate stroke, 2= 8–13: moderate stroke, 3= 14–21: moderate to severe stroke, 4= 22–42: severe stroke

Patients discharged to hospice were not considered as a competing risk. Patients who died during hospital stay were removed before running these models

In addition to NIHSS categories, models adjusted for the following: age, sex, race, dual status, hospital lengths of stay, ICU stay, receipt of tPA, feeding tube, thrombectomy, stroke history, Elixhauser comorbidity index, discharge destinations, hospital rural/urban, hospital stroke volume, CAH status, and safety-net status

Figure 4.

Figure 4.

A, B Cumulative incidence functions for stroke severity groups for 30-day mortality and 30-day readmission competing risk analysis. NIHSS categories: 1= 0–7: minor to moderate stroke, 2= 8–13: moderate stroke, 3= 14–21: moderate to severe stroke, 4= 22–42: severe stroke

Sensitivity Analyses

When limiting the sample to individuals with prior stroke, we found similar results from competing risk analyses for 30-day mortality (eTable 4 in the Supplementary Appendices). After excluding patients discharged to hospice, there was significantly higher readmission risk only in the moderate stroke (HR=1.16, 95% CI=1.08–1.26) and moderate to severe stroke (HR=1.23, 95% CI=1.12–1.34), as compared to minor to moderate stroke. After excluding patients with prior history of stroke, there was significantly higher readmission risk in the moderate stroke (HR=1.14, 95% CI=1.04–1.26), but not in moderate to severe stroke (HR=1.08, 95% CI=0.97–1.21) and in severe stroke categories (HR=0.87, 95% CI=0.74–1.02) compared to minor to moderate stroke. Additionally, to examine the reasons for 30-day hospital readmission, we reported the top 25 diagnostic conditions (eTable 5 in the Supplement).

DISCUSSION

Our study has three notable findings. First, we found a monotonic relationship and significant association between NIHSS and 30-day mortality after accounting for 30-day readmission as a competing risk. After accounting for 30-day mortality as a competing risk, the risk of 30-day hospital readmission was significantly associated with patients only in NIHSS moderate and moderate to severe stroke categories. Finally, our study also validated the higher predictive ability of ICD-10CM-based NIHSS. Our findings highlight the importance of adding the NIHSS in stroke-specific risk-standardized 30-day mortality rates. As a standardized and validated assessment tool, the NIHSS has been widely utilized in treatment planning in comprehensive stroke centers and clinical trials to monitor treatment effectiveness and quantify patient recovery.25 Therefore, the findings from our study demonstrate the improved discriminatory property of the NIHSS compared to the Elixhauser index and current CMS risk adjustment method.26

Our findings corroborate the previous studies using Get With The Guidelines®-Stroke data and the CMS technical report that demonstrated the inclusion of the NIHSS improved C-statistics to predict 30-day mortality in patients with stroke.27,3 We also found that both NIHSS and Elixhauser were poor predictors of 30-day hospital readmission following discharge from an acute hospital after adjusting for patient-level demographic characteristics. Findings from this and other studies indicate that commonly used risk adjustments by CMS do not accurately predict the risk of hospital readmission.5,22,27 One plausible explanation for the lack of sensitivity of the NIHSS to predict readmission is the fact that the NIHSS was developed to capture neurological impairment during the early acute phase.

Our results have important implications for risk adjustment methods. Despite stroke being the fourth most common cause of mortality, with approximately 15% 30-day mortality in the Medicare population, there was no information about patient-level severity in Medicare and other administrative claims data. More importantly, post-stroke severity has been shown to be a strong predictor of mortality, and also influences timeliness of care delivery such as administration of tPA,28 initiation of intensive physical rehabilitation, and discharge to post-acute care settings.29 Both Charlson and Elixhauser comorbidity indexes have been used extensively in health services studies, but both only use medical diagnosis.21,30 The CMS currently uses the hierarchical condition category (HCC) to develop risk-adjustment scores for individual patients based on an algorithm that includes combinations of diagnoses, patient age, and comorbidities.31 However, past studies have shown that most of these risk adjustment methods could not capture disease severity and demonstrated weak discriminatory property to predict outcomes.22,32,33Condition-specific severity information in Medicare data will maximize predictive accuracy for patient outcomes and minimize hospital penalty risk for treating sicker patients. In the absence of a sensitive risk adjustment, hospitals in the high prevalence regions (“stroke belt”)34,35 and other regions that treat a large number of patients with severe stroke could perform worse on public reporting.

Competing risk analysis allowed us to control for the competing risk of hospital readmission when examining the risk of 30-day mortality, and the competing risk of mortality when examining 30-day hospital readmission. The Fine-Gray model is appropriate for risk modeling about patient prognosis and in its absence, our hazard models created an upward bias of the probability estimates of occurrence of both 30-day mortality and readmission within the NIHSS categories (Table 3), reflecting a caution that has been addressed by previous investigations.36 The use of this analysis has been encouraged for clinical research17,37,38 and competing risk analysis has been applied in patients with hip fractures39 and adults with chronic heart failure.40

Our study has some limitations. First, Medicare claims data do not record the timing of administration of the NIHSS during acute hospitalization. However, CMS strongly recommends hospitals assess and document the first NIHSS within 12 h of arrival at the emergency department. Secondly, since October 1, 2016, hospitals have been reporting ICD-10CM-based NIHSS scores. There were missing scores among low-volume hospitals and non-academic medical hospitals in the earlier months of reporting. However, reporting continues to improve with time. Third, Medicare claims data may have inaccurate coding and potential missing data, although high levels of agreement have been reported in administrative data and medical records.41 Fourth, besides dual enrollment in Medicare and Medicaid, we did adjust for social determinants of health, which can be associated with both readmission and mortality. We have excluded a small number of patients admitted from nursing homes, long-term care hospitals, and hospice. While we did not find differences in demographic characteristics and outcomes, we found that among these groups, the rate of ICU admissions during hospitalization was higher. Our results may not be applicable to patients enrolled in Medicare Advantage plans. Finally, we used a nine-month lookback period to ensure continuous enrollment in FFS than a commonly used 12-month lookback period. Despite these limitations, our study has many strengths. This is the first study to validate the utility of NIHSS in Medicare claims data. Second, we adjusted for both patient and hospital characteristics, minimizing the risk of bias. Third, our study addresses the limitation of previous studies by adjusting post-acute discharge destinations using a combination of claims and post-acute assessment data.

CONCLUSION

To our knowledge, this is the first study to demonstrate the predictive utility of NIHSS using Medicare claims data for 30-day mortality and 30-day hospital readmission for patients with ischemic stroke. This study affirmed a significant association between ICD-10-based NIHSS and 30-day mortality. These findings support the inclusion of the claim-based NIHSS in risk-standardized measures (mortality). Understanding the impact of the NIHSS on patient outcomes has implications for quality reporting programs and improving the transition of care in the era of value-based care for patients with stroke.

Supplementary Information

ESM 1 (20.2KB, docx)

(DOCX 20 kb)

Author Contribution

Dr. Kumar and Karmarkar had full access to all data used in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Kumar, Fehnel, Karmarkar

Acquisition, analysis, or interpretation of data: All authors

Statistical analysis: Roy, Kumar, Karmarkar

Drafting manuscript: All authors

Critical revision of manuscript for important intellectual content: All authors

Administrative, technical, or material support: Bosch, Kumar

Obtaining funding: Karmarkar, Kumar

Funding

This research was supported by National Institutes of Health grants K01-HD086290, K23AG066929, R03HD096372, and U54MD012388. The sponsors had no role in the study design, analysis, interpretation of the data, and in the writing of the manuscript.

Data availability

We had a Data Use Agreement approved by Centers for Medicare & Medicaid Services (DUA RSCH-2019-52868). Interested researchers may replicate the study obtaining the data and supporting information files from the CMS. To reproduce this study requires 100% Medicare inpatient claims data from the SAF files.

Declarations

Ethics Approval

The study was approved by the University Institutional Review Board with a waiver of consent for the use of secondary identifiable data.

Conflict of Interest

Dr. Karmarkar reports grants from the National Center for Medical Rehabilitation Research during the conduct of the study. Dr. Fennel reports grants from the National Institute on Aging during the conduct of the study.

Disclaimer

The National Institutes of Health had no role in the study design, conduct, or analysis; interpretation and reporting of the results; or writing of the manuscript.

Footnotes

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.Stroke Statistics. Accessed December 11, 2020. http://www.strokecenter.org/patients/about-stroke/stroke-statistics/
  • 2.2016 Condition-Specific Measures Updates and Specifications Report Hospital-Level 30-Day Risk-Standardized Readmission Measures. Centers for Medicare & Medicaid Services (CMS); 2016. http://aann.org/uploads/Condition_Specific_Readmission_Measures.pdf. Accessed 16 Jan 2021.
  • 3.Claims-Based and Hybrid Measures of 30-Day Mortality Following Acute Ischemic Stroke Hospitalization Incorporating Risk Adjustment for Stroke Severity. Centers for Medicare & Medicaid Services (CMS); 2015. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/MMS/Downloads/Claims-Based-and-Hybrid-Measures-of-30-Day-Mortality-Following-Acute-Ischemic-Stroke-Hospitalization-Incorporating-Risk-Adjustment-for-Stroke-Severity-Technical-Report-.pdf. Accessed 16 Jan 2021.
  • 4.Kansagara D, Englander H, Salanitro A, et al. Risk Prediction Models for Hospital Readmission: A Systematic Review. JAMA. 2011;306(15):1688. doi: 10.1001/jama.2011.1515. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Kumar A, Karmarkar A, Downer B, et al. Current Risk Adjustment and Comorbidity Index Underperformance in Predicting Post-Acute Utilization and Hospital Readmissions After Joint Replacements: Implications for Comprehensive Care for Joint Replacement Model. Arthritis Care Res. 2017;69(11):1668–1675. doi: 10.1002/acr.23195. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Park S, Basu A. Alternative evaluation metrics for risk adjustment methods. Health Econ. 2018;27(6):984–1010. doi: 10.1002/hec.3657. [DOI] [PubMed] [Google Scholar]
  • 7.Risk Adjustment in Quality Measurement: Supplemental Material to the CMS MMS Blueprint. Published online September 2020. https://www.cms.gov/files/document/blueprint-risk-adjustment.pdf
  • 8.Powers WJ, Rabinstein AA, Ackerson T, et al. Guidelines for the Early Management of Patients With Acute Ischemic Stroke: 2019 Update to the 2018 Guidelines for the Early Management of Acute Ischemic Stroke: A Guideline for Healthcare Professionals From the American Heart Association/American Stroke Association. Stroke. 2019;50(12). 10.1161/STR.0000000000000211 [DOI] [PubMed]
  • 9.Katzan IL, Spertus J, Bettger JP, et al. Risk Adjustment of Ischemic Stroke Outcomes for Comparing Hospital Performance: A Statement for Healthcare Professionals From the American Heart Association/American Stroke Association. Stroke. 2014;45(3):918–944. doi: 10.1161/01.str.0000441948.35804.77. [DOI] [PubMed] [Google Scholar]
  • 10.CMS Stroke Mortality Measure Update. https://www.heart.org/idc/groups/ahaecc-public/@wcm/@gwtg/documents/downloadable/ucm_499392.pdf. Accessed 16 Jan 2021.
  • 11.Mortality Measures Resources. cms.govhttps://qualitynet.cms.gov/inpatient/measures/mortality/resources. Accessed 16 Jan 2021.
  • 12.Adams HP, Davis PH, Leira EC, et al. Baseline NIH Stroke Scale score strongly predicts outcome after stroke: A report of the Trial of Org 10172 in Acute Stroke Treatment (TOAST) Neurology. 1999;53(1):126–126. doi: 10.1212/WNL.53.1.126. [DOI] [PubMed] [Google Scholar]
  • 13.Strowd RE, Wise SM, Umesi UN, et al. Predictors of 30-Day Hospital Readmission Following Ischemic and Hemorrhagic Stroke. Am J Med Qual. 2015;30(5):441–446. doi: 10.1177/1062860614535838. [DOI] [PubMed] [Google Scholar]
  • 14.Fonarow GC, Saver JL, Smith EE, et al. Relationship of National Institutes of Health Stroke Scale to 30-Day Mortality in Medicare Beneficiaries With Acute Ischemic Stroke. J Am Heart Assoc. 2012;1(1). 10.1161/JAHA.111.000034 [DOI] [PMC free article] [PubMed]
  • 15.Gorodeski EZ, Starling RC, Blackstone EH. Are All Readmissions Bad Readmissions? N Engl J Med. 2010;363(3):297–298. doi: 10.1056/NEJMc1001882. [DOI] [PubMed] [Google Scholar]
  • 16.Gupta A, Allen LA, Bhatt DL, et al. Association of the Hospital Readmissions Reduction Program Implementation With Readmission and Mortality Outcomes in Heart Failure. JAMA Cardiol. 2018;3(1):44. doi: 10.1001/jamacardio.2017.4265. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Haneuse S, Lee KH. Semi-Competing Risks Data Analysis: Accounting for Death as a Competing Risk When the Outcome of Interest Is Nonterminal. Circ Cardiovasc Qual Outcomes. 2016;9(3):322–331. doi: 10.1161/CIRCOUTCOMES.115.001841. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Fonarow GC. Unintended Harm Associated With the Hospital Readmissions Reduction Program. JAMA. 2018;320(24):2539. doi: 10.1001/jama.2018.19325. [DOI] [PubMed] [Google Scholar]
  • 19.Saber H, Saver JL. Distributional Validity and Prognostic Power of the National Institutes of Health Stroke Scale in US Administrative Claims Data. JAMA Neurol. 2020;77(5):606. doi: 10.1001/jamaneurol.2019.5061. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Brott T, Adams HP, Olinger CP, et al. Measurements of acute cerebral infarction: a clinical examination scale. Stroke. 1989;20(7):864–870. doi: 10.1161/01.STR.20.7.864. [DOI] [PubMed] [Google Scholar]
  • 21.Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8–27. doi: 10.1097/00005650-199801000-00004. [DOI] [PubMed] [Google Scholar]
  • 22.Kumar A, Karmarkar AM, Graham JE, et al. Comorbidity Indices Versus Function as Potential Predictors of 30-Day Readmission in Older Patients Following Postacute Rehabilitation. J Gerontol A Biol Sci Med Sci. 2017;72(2):223–228. doi: 10.1093/gerona/glw148. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Hosmer DW, Lemeshow S. Applied Logistic Regression. 2. Wiley; 2000. [Google Scholar]
  • 24.Fine JP, Gray RJ. A Proportional Hazards Model for the Subdistribution of a Competing Risk. J Am Stat Assoc. 1999;94(446):496–509. doi: 10.1080/01621459.1999.10474144. [DOI] [Google Scholar]
  • 25.Kasner SE. Clinical interpretation and use of stroke scales. Lancet Neurol. 2006;5(7):603–612. doi: 10.1016/S1474-4422(06)70495-1. [DOI] [PubMed] [Google Scholar]
  • 26.Risk Adjustment: Medicare risk adjustment information. Published online July 31, 2018. https://www.cms.gov/Medicare/Health-Plans/MedicareAdvtgSpecRateStats/Risk-Adjustors
  • 27.Schwartz J, Wang Y, Qin L, et al. Incorporating Stroke Severity Into Hospital Measures of 30-Day Mortality After Ischemic Stroke Hospitalization. Stroke. 2017;48(11):3101–3107. doi: 10.1161/STROKEAHA.117.017960. [DOI] [PubMed] [Google Scholar]
  • 28.Fonarow GC, Smith EE, Saver JL, et al. Timeliness of Tissue-Type Plasminogen Activator Therapy in Acute Ischemic Stroke: Patient Characteristics, Hospital Factors, and Outcomes Associated With Door-to-Needle Times Within 60 Minutes. Circulation. 2011;123(7):750–758. doi: 10.1161/CIRCULATIONAHA.110.974675. [DOI] [PubMed] [Google Scholar]
  • 29.Bland MD, Whitson M, Harris H, et al. Descriptive Data Analysis Examining How Standardized Assessments Are Used to Guide Post–Acute Discharge Recommendations for Rehabilitation Services After Stroke. Phys Ther. 2015;95(5):710–719. doi: 10.2522/ptj.20140347. [DOI] [PubMed] [Google Scholar]
  • 30.Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: Development and validation. J Chronic Dis. 1987;40(5):373–383. doi: 10.1016/0021-9681(87)90171-8. [DOI] [PubMed] [Google Scholar]
  • 31.Pope GC, Kautter J, Ingber MJ, Freeman S, Sekar R, Newhart C. Evaluation of the CMS-HCC Risk Adjustment Model. RTI International; 2011. https://www.cms.gov/Medicare/Health-Plans/MedicareAdvtgSpecRateStats/downloads/evaluation_risk_adj_model_2011.pdf. Accessed 16 Jan 2021.
  • 32.Kumar A, Graham JE, Resnik L, et al. Examining the Association Between Comorbidity Indexes and Functional Status in Hospitalized Medicare Fee-for-Service Beneficiaries. Phys Ther. 2016;96(2):232–240. doi: 10.2522/ptj.20150039. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Kumar A, Graham JE, Resnik L, et al. Comparing Comorbidity Indices to Predict Post–Acute Rehabilitation Outcomes in Older Adults. Am J Phys Med Rehabil. 2016;95(12):889–898. doi: 10.1097/PHM.0000000000000527. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Howard G, Howard VJ. Twenty Years of Progress Toward Understanding the Stroke Belt. Stroke. 2020;51(3):742–750. doi: 10.1161/STROKEAHA.119.024155. [DOI] [PubMed] [Google Scholar]
  • 35.Howard G. Why Do We Have a Stroke Belt in the Southeastern United States? A Review of Unlikely and Uninvestigated Potential Causes. Am J Med Sci. 1999;317(3):160–167. doi: 10.1016/S0002-9629(15)40498-7. [DOI] [PubMed] [Google Scholar]
  • 36.Austin PC, Fine JP. Practical recommendations for reporting Fine-Gray model analyses for competing risk data. Stat Med. 2017;36(27):4391–4400. doi: 10.1002/sim.7501. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Berry SD, Ngo L, Samelson EJ, Kiel DP. Competing Risk of Death: An Important Consideration in Studies of Older Adults: COMPETING RISK OF DEATH IN STUDIES OF OLDER ADULTS. J Am Geriatr Soc. 2010;58(4):783–787. doi: 10.1111/j.1532-5415.2010.02767.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Lau B, Cole SR, Gange SJ. Competing Risk Regression Models for Epidemiologic Data. Am J Epidemiol. 2009;170(2):244–256. doi: 10.1093/aje/kwp107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Lin JC-F, Liang W-M. Mortality, readmission, and reoperation after hip fracture in nonagenarians. BMC Musculoskelet Disord. 2017;18(1):144. doi: 10.1186/s12891-017-1493-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Betihavas V, Frost SA, Newton PJ, et al. An Absolute Risk Prediction Model to Determine Unplanned Cardiovascular Readmissions for Adults with Chronic Heart Failure. Heart Lung Circ. 2015;24(11):1068–1073. doi: 10.1016/j.hlc.2015.04.168. [DOI] [PubMed] [Google Scholar]
  • 41.Rowan CG, Flory J, Gerhard T, et al. Agreement and validity of electronic health record prescribing data relative to pharmacy claims data: A validation study from a US electronic health record database. Pharmacoepidemiol Drug Saf. 2017;26(8):963–972. doi: 10.1002/pds.4234. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

ESM 1 (20.2KB, docx)

(DOCX 20 kb)

Data Availability Statement

We had a Data Use Agreement approved by Centers for Medicare & Medicaid Services (DUA RSCH-2019-52868). Interested researchers may replicate the study obtaining the data and supporting information files from the CMS. To reproduce this study requires 100% Medicare inpatient claims data from the SAF files.


Articles from Journal of General Internal Medicine are provided here courtesy of Society of General Internal Medicine

RESOURCES