Assessment of nursing home reporting of major injury falls for quality measurement on nursing home compare

Abstract

Objective

To assess the accuracy of nursing home self‐report of major injury falls on the Minimum Data Set (MDS).

Data Sources

MDS assessments and Medicare claims, 2011‐2015.

Study Design/Methods

We linked inpatient claims for major injury falls with MDS assessments. The proportion of claims‐identified falls reported for each fall‐related MDS item was calculated. Using multilevel modeling, we assessed patient and nursing home characteristics that may be predictive of poor reporting. We created a claims‐based major injury fall rate for each nursing home and estimated its correlation with Nursing Home Compare (NHC) measures.

Principal Findings

We identified 150,828 major injury falls in claims that occurred during nursing home residency. For the MDS item used by NHC, only 57.5 percent were reported. Reporting was higher for long‐stay (62.9 percent) than short‐stay (47.2 percent), and for white (59.0 percent) than nonwhite residents (46.4 percent). Adjusting for facility‐level race differences, reporting was lower for nonwhite people than white people; holding constant patient race, having larger proportions of nonwhite people in a nursing home was associated with lower reporting. The correlation between fall rates based on claims vs the MDS was 0.22.

Conclusions

The nursing home‐reported data used for the NHC falls measure may be highly inaccurate.

What This Study Adds.

1. Section 1: The federal website Nursing Home Compare reports patient safety measures for nursing homes using data that are self‐reported by nursing homes. The accuracy of these data has long been a concern to academics and policy makers, based on inconsistencies with measures from other sources and small validation exercises.

2. Section 2: We focused on the falls section of the nursing home‐reported data and found only 57.5 percent of major injury falls, identified in claims data, were reported, and that reporting was substantially lower for nonwhite people than white people. The data Nursing Home Compare uses for reporting patient safety related to falls may be highly inaccurate.

1. INTRODUCTION

In 1987, a landmark Institute of Medicine report concluded that the quality of care in many nursing homes was seriously inadequate.1 Since the 1990s, a centerpiece of federal efforts to improve nursing home quality has been a public reporting initiative by the Centers for Medicare and Medicaid Services (CMS) called Nursing Home Compare (NHC).2 Today, NHC reports nursing home performance on several patient safety indicators, among other measures for staffing and inspections. It also creates summaries of these by assigning stars to each home through the Five‐Star Quality Rating System. However, as the underlying quality of resident care data, called the Minimum Data Set (MDS), is self‐reported by nursing homes, it is important to ask: are NHC patient safety measures accurate?

Concerns about the accuracy of the MDS are long‐standing. Investigations over the past decades by the Department of Health and Human Services Office of the Inspector General (HHS OIG),3, 4 the US Government Accountability Office (GAO),5, 6, 7, 8 CMS,9 and the New York Times,10, 11 as well as the limited academic research on the topic,12, 13 have all found discrepancies between the MDS and other sources. Though state inspections have the potential to serve as independent checks of quality of care, these too are partly guided by MDS‐based quality measures.14

We assessed the quality of nursing home reporting of major injury falls by linking MDS assessments with Medicare hospital admission claims at the patient level. We focused on falls for two reasons. First, they are a leading cause of death among older adults and can lead to serious physical and psychological morbidity when not fatal.15 Yet, because they are often preventable, they serve as an important measure of patient safety. Second, relative to other clinical conditions measured by the MDS, we expected falls to be easy to identify and record. To our knowledge, this is the first rigorous national analysis of reporting on any section of the MDS.

2. METHODS

2.1. Datasets

We analyzed January 1, 2011, to September 30, 2015, hospital admission claims of a 100 percent sample of Medicare beneficiaries from the Medicare Provider Analysis and Review (MedPAR) file provided by CMS. We dropped the last three months of 2015 when diagnosis codes in claims switched from International Classification of Diseases, Ninth Revision, Clinical Modification (ICD9‐CM) to the Tenth Revision (ICD10) in order to avoid any transition issues and the use of separate methods for a short time frame. We used enrollment and demographic information from the associated beneficiary summary files (MBSF) to obtain age, sex, race/ethnicity, and whether disability was the current reason for Medicare entitlement. We used the monthly dual‐status codes to flag individuals as being either full duals or restricted/nonduals in the month of their hospital admission.

In the same years as the Medicare claims, we used MDS 3.0 assessments, which collect information on the physical, clinical, and psychological well‐being of each patient.16 Nursing homes must complete assessments at least every 92 days as part of a federal requirement to participate in the Medicare and Medicaid programs. We analyzed fall‐related questions in Section J, Health Conditions of the MDS instrument (Table 1), focusing in particular on J1900C, as the responses to this question for long‐stay residents are used to create an NHC quality measure and are part of the star rating algorithm.

Table 1.

Fall‐related MDS 3.0 items

Section/item description	Item	Question	Possible responses	Binary variable created for reporting status
Fall History on Admission/Entry or Reentry	J1700A	Did the resident have a fall any time in the last month prior to admission/entry or reentry?	Yes No Unable to determine	1 if Yes 0 if No or Unable to determine
	J1700B	Did the resident have a fall any time in the last 2‐6 months prior to admission/entry or reentry?	Yes No Unable to determine	1 if Yes 0 if No or Unable to determine
Any Falls Since Admission/Entry or Reentry or Prior Assessment	J1800	Has the resident had any falls since admission/entry or reentry or the prior assessment, whichever is more recent?	Yes No	1 if Yes 0 if No
Number of Falls Since Admission/Entry or Reentry or Prior Assessment	J1900A	No injury—no evidence of any injury is noted on physical assessment by the nurse or primary care clinician; no complaints of pain or injury by the resident; no change in the resident’s behavior is noted after the fall	One Two or more None	1 if One or Two or more 0 if None
	J1900B	Injury (except major)—skin tears, abrasions, lacerations, superficial bruises, hematomas, and sprains; or any fall‐related injury that causes the resident to complain of pain.	One Two or more None	1 if One or Two or more 0 if None
	J1900Ca	Major injury—bone fractures, joint dislocations, closed head injuries with altered consciousness, subdural hematoma	One Two or more None	1 if One or Two or more 0 if None

^aMDS item J1900C is used by Nursing Home Compare to create a patient safety measure and assign five‐star ratings.

For facility‐level characteristics, we used the Certification and Survey Provider Enhanced Reporting (CASPER) dataset, which are compilations of information collected for the Medicare and Medicaid certification process, LTCfocus data from Brown University for provider identification information, and publicly available data from NHC. We merged each MDS assessment with the most recent record from these datasets prior to the assessment.

2.2. Identification of falls for MDS reporting

Medicare claims data are commonly used to identify fall‐related injuries, including to study the costs of medical care for falls,17, 18 to assess fall events as outcomes,19, 20 and to create measures of potentially avoidable hospitalizations.21 We started with the hospital admission claims and determined a patient had experienced a fall if the admitting or primary diagnosis code fields, or primary external cause code field reported an accidental fall (ICD9‐CM external cause codes E880‐E888, excluding E887), following an algorithm developed by Kim et al for identifying fall‐related injuries in Medicare inpatient claims data.22 In the years of our data, reporting of external cause codes is high, at about 90 percent for all injury cases (Appendix S1). We linked these claims to MDS assessments using beneficiary identifiers and then applied criteria for each MDS fall‐related item to identify an appropriate denominator for reporting (Figure 1).

Figure 1.

Linkage of Medicare admission claims with MDS assessments to create denominators for fall‐related MDS reporting outcomes.

Notes: A, White boxes map out paths to the final denominators used to assess nursing home reporting of fall‐related items on the MDS. Gray boxes identify observations that were not used in analysis. B, J1700A‐J1900C refer to the specific MDS items under study and are described in Table 1. J1900C is the item that is used by CMS for quality reporting on NHC. C, CMS requires discharge assessments and items J1800‐J1900C in particular if a resident is admitted to a hospital. D, NH = nursing home

Based on the MDS assessment instrument user’s manual,16 we interpreted items J1700A‐B as asking about falls that occurred prior to, and J1800‐J1900C as asking about falls that occurred during, nursing home residency. As we describe below, in identifying the claims‐based falls for each MDS item, we applied the reporting rules from the manual to ensure only those falls for which the nursing homes had a clear reporting responsibility were included.

Items J1700A‐B ask about falls in the six months prior to the current nursing home entry. To complete this information, the steps for a nursing home are to ask family and review medical records. To minimize any reporting discrepancies caused by prior nursing home residencies during this look back period, we required patients to have no evidence of nursing home residency during the six months prior to hospital admission for the fall. We dropped the first six months of claims to have this assessment history for each patient. We then kept those cases with a nursing home entry/admission assessment within one month and within two to six months after the hospital admission, respectively, for J1700A and J1700B.

Items J1800‐J1900C ask about falls during the current residency and should be completed on a discharge assessment when a resident is admitted to a hospital. Therefore, we required patients to have a discharge assessment from the nursing home, indicating discharge to a hospital, within one day prior to the hospital admission. Though we only checked the discharge assessment for fall reporting, we also required a readmission assessment from the same nursing home within one day of the hospital discharge to remove cases for which readmission to a different facility could arguably have created confusion about reporting responsibility. Appendix S2 provides further detail, such as the specific MDS and claims variables that were used.

Though J1900C asks about major injury falls, all other fall‐related items ask about any falls. Nonetheless, we restricted all our analyses to falls with a secondary ICD9‐CM diagnosis code for conditions identified in the MDS definition for major injury, namely bone fractures, joint dislocations, closed head injuries with altered consciousness, and subdural hematomas (specific ICD‐9CM diagnosis codes provided in Appendix S3). We refer to the final set of claims‐identified falls for each MDS item as that item’s denominator.

Though the MDS rules may appear complex, the context of the chief MDS fall item of interest, J1900C, is simple: If a person falls during nursing home residency and is admitted to a hospital with major injury, we expect the nursing home to report the fall under this item on a discharge assessment.

2.3. Constructed measures

We defined short‐stay residents as those whose stays would have been covered by Medicare. Medicare covers up to 100 days of postacute nursing home care and requires the Prospective Payment System (PPS) 5‐day assessment to be completed within 8 days of admission.23 For patients who fell prior to nursing home residency, we looked for this assessment within 8 days of the entry/admission assessment. For patients who fell during residency, we looked for this assessment up to 101 days prior to the discharge to hospital. If we did not find the assessment, we determined the resident was long‐stay.

For finer control of variation in injury severity, we created New Injury Severity Scores (NISS) using ICDPIC software.24, 25, 26, 27, 28, 29, 30, 31 The NISS is neither normally distributed nor continuous, so in addition to the numerical score, we created a categorical variable using a breakdown similar to other studies.24, 28, 30 To adjust for health status, we used diagnosis codes on the hospital admission claim to construct combined Charlson/Elixhauser comorbidity scores.32

At the nursing home level, we classified nursing homes into tertiles of size by the total registered resident counts at the time of the CASPER report, with breaks at 65 and 105 residents. We also computed the proportions of residents who are duals, and fall within each race category.

2.4. Outcome measures

MDS item J1900C is used by CMS to create a quality measure as part of its star rating algorithm. Our primary outcome measure was a binary indicator of whether a fall in the J1900C denominator was reported or not. The binary indicators of whether claims‐identified falls were reported in the other MDS items were secondary outcome measures. For J1700A‐J1800, we counted the fall as reported if the nursing home marked “Yes”; for J1900A‐C, we counted the fall as reported if the nursing home marked “One” or “Two or more.”

We calculated a claims‐based rate of major injury falls per 100 residents in each nursing home in 2014, our most recent complete year of data. For each nursing home, we totaled the number of major injury falls during residency in the claims and divided by the total registered resident counts snapshot variable from CASPER.

2.5. Statistical analysis

We computed national reporting rates for each MDS item, separately for short‐ and long‐stay, and white and nonwhite (black, Hispanic, Asian, and other race/ethnicity) nursing home residents. For each item, we divided the total number of claims‐identified falls reported by the number of patients in the denominator, and multiplied by 100.

To assess patient‐ and nursing home‐level characteristics predictive of patient level reporting on J1900C, separately for short‐ and long‐stay patients, we estimated a linear multilevel model with nursing home random effects and year fixed effects. Candidate predictors at the nursing home level included our claims‐based fall rate, the Census region, ownership type, nursing home size, race mix, and proportion dual status, and at the individual level included sex, age (specified via linear splines with cutoff points at quantiles of the age distribution), race, comorbidity score, disability as a reason for current entitlement, dual status, and the NISS as both a categorical and numerical variable. We disaggregated the within‐ and between‐nursing home effects of race and dual status so that their coefficients could be interpreted directly (see Appendix S4 for further detail). Though our main exhibits show linear models to allow interpretability, we included comparison tables with logistic regression models in Appendix S5 that demonstrate the similarity of the results from the two approaches.

Finally, we defined quintiles of the claims‐based fall rates and within each, computed the percent of nursing homes with four‐ or five‐star overall and quality‐domain ratings. We also computed means of these same ratings and the MDS‐based major injury fall measure by quintile. Finally, we estimated the Pearson correlation coefficients between the claims‐based fall rates and these NHC measures.

The appendices provide further methodological detail, sensitivity analyses (Appendices S6 and S7) in which we relax some of our assumptions, and description of the code.

3. RESULTS

We identified 150,828 major injury falls in the hospital claims that occurred during nursing home residency, which we expect to have been reported under J1900C, the MDS item used by NHC (Figure 1). Only 57.5 percent of these were reported (Table 2). Reporting on this item was more complete for long‐stay (62.9 percent) than for short‐stay residents (47.2 percent), and for white (59.0 percent) than for nonwhite residents (46.4 percent). Long‐stay white residents had the highest reporting rate (64.5 percent) and short‐stay nonwhite residents had the lowest reporting rate (37.4 percent). Including additional assessments beyond the discharge assessment did not improve reporting rates (see Appendix S6).

Table 2.

National reporting rates of major injury falls by race and short‐ vs. long‐stay in 2011‐2015

Number of major injury falls in item denominator				Fall item	Percent of major injury falls reported (25th, 75th percentile)d
Short‐stay		Long‐stay			Short‐staya		Long‐stay
White	Nonwhite	White	Nonwhite		White	Nonwhite	White	Nonwhiteb
804 742	85 246	173 032	29 255	J1700A	94.8 (92.3, 100.0)	91.6 (91.7, 100.0)	94.0 (93.3, 100.0)	90.9 (97.6, 100.0)
65 222	10 925	18 385	4 013	J1700B	41.8 (8.3, 66.7)	33.2 (0.0, 66.7)	44.4 (0.0, 100.0)	33.2 (0.0, 100.0)
45 617	6 310	87 043	11 858	J1800	67.8 (50.0, 100.0)	62.6 (0.0, 100.0)	82.8 (71.4, 100.0)	76.1 (60.0, 100.0)
				J1900A	17.4 (0.0, 33.3)	18.9 (0.0, 33.3)	23.5 (0.0, 33.3)	23.1 (0.0, 50.0)
				J1900B	18.0 (0.0, 30.0)	19.2 (0.0, 25.0)	21.0 (0.0, 33.3)	21.8 (0.0, 33.33)
				J1900Cc	48.6 (22.2, 80.0)	37.4 (0.0, 100.0)	64.5 (46.7, 87.5)	51.3 (0.0, 100)

^aPatients who stayed in nursing homes for less than 101 days are classified as short‐stay patients, otherwise long‐stay patients.

^bPatients are categorized as either white or nonwhite, which includes black, Hispanic, Asian, and other race/ethnicity.

^cMDS item J1900C is used by Nursing Home Compare to create a patient safety measure and assign five‐star ratings.

^dThe reporting rates are all statistically significant at an alpha level of 0.05.

On the parent question for J1900C, J1800, which asks about any falls regardless of injury severity, 62.6 and 82.8 percent of major injury falls identified in the claims were reported for short‐stay nonwhite and long‐stay white residents, respectively. Among the falls that were reported under J1800, it is possible some are misclassified under J1900A‐B as having no or only minor injuries. However, due to the design of these particular survey questions, it is difficult to ascertain this.

Reporting completeness on J1700A, which asks about falls in the month prior to the current nursing home residency, was high, ranging from 90.9 percent for long‐stay nonwhite to 94.8 percent for short‐stay white patients. Item J1700B on falls between two to six months prior to the current stay had much lower reporting rates for all groups.

The final models for our main outcome, reporting on J1900C, included all the variables we assessed with linear hypothesis tests (Table 3). Results were generally in the same direction in the short‐ and long‐stay populations, so here we focus on the short‐stay patients. A higher NISS was associated with a higher reporting probability. After adjusting for nursing home‐level differences in racial composition, reporting was lower for Asians by 5.8 percentage points (pp), for black people by 4.2 pp, and for Hispanics by 2.9 pp than for white people. Holding constant patient race, having larger proportions of Asians, black people, or Hispanics (and correspondingly fewer white people) was also associated with lower reporting rates. The between‐ and within‐nursing home coefficients of dual status were positive, indicating that both residing in a home with many duals and being a dual were associated with higher reporting probability (16.9 pp and 5.8 pp, respectively). A 10 pp higher claims‐based fall rate in a nursing home was associated with 0.66 pp lower reporting probability. For‐profit nursing homes were associated with lower reporting probability than all other ownership types, in particular a 4.8 pp lower probability of reporting a fall than a government‐owned nursing home.

Table 3.

Linear multilevel modelsb, c of MDS item J1900Ca reporting outcome in 2011‐2015

	Short‐stay		Long‐stay
Patient‐level characteristics
Female	0.011*		0.018***
Age linear spline
<78	0.003***		0.002***
78‐85	0.005***		0.003**
85‐90	0.000		0.003**
>90	0.003		0.001
Racec
White (Ref)
Asian	−0.058**		−0.041*
Black	−0.042***		−0.037***
Hispanic	−0.029*		−0.015
Other	−0.001		0.006
NISS	0.045***		0.034***
NISS category
1‐15 (Ref)
16‐24	−0.441***		−0.329***
25‐40	−0.934***		−0.728***
40‐75	−1.546***		−1.290***
Disability status	0.009		0.003
Dual status	0.058***		0.054***
Comorbidity score	0.001*		−0.002***

	Short‐stay		Long‐stay
Nursing home‐level characteristics
Claims‐based fall rate	−0.066	−0.041	0.005	−0.025
Between NH dual association	0.169***	0.162***	0.100***	0.086***
Region
South (Ref)
Midwest	0.018**	0.014*	0.031***	0.028***
Northeast	−0.003	−0.010	0.023***	0.020***
West	−0.032***	−0.044***	−0.018*	−0.024***
Ownership type
For‐profit (Ref)
Government	0.048***	0.057***	0.066***	0.070***
Nonprofit	0.010	0.017**	0.031***	0.037***
Other	0.006	0.012	0.023	0.022
Provider size
Large (Ref)
Medium	0.026***	0.028***	0.018***	0.017***
Small	0.036***	0.040***	0.028***	0.028***
Between NH race associationc
Asian	−0.271***	−0.283***	−0.217***	−0.220***
Black	−0.251***	−0.299***	−0.290***	−0.332***
Hispanic	−0.278***	−0.318***	−0.267***	−0.302***
Other	−0.022	−0.065	0.059	0.039
Year
2011 (Ref)
2012	0.016*	0.020**	0.028***	0.030***
2013	0.028***	0.036***	0.028***	0.032***
2014	0.022***	0.026***	0.030***	0.033***
2015	0.021**	0.026***	0.044***	0.048***
Variance explained by random effects	0.014	0.016	0.021	0.022
Variance explained by fixed effects	0.021	0.005	0.014	0.005
Residual variance	0.213	0.229	0.198	0.206
Within NH variance	0.017	0.000	0.008	0.000
Between NH variance	0.019	0.021	0.026	0.028

Abbreviations: NISS, New Injury Severity Score; NH, nursing home.

*P < .05, **P < .01, ***P < .001.

^aMDS item J1900C is used by NHC to create a patient safety measure and assign five‐star ratings.

^bData are modeled at the patient level, and outcome is a binary indicator of whether the patient’s major injury fall was reported.

The patient‐level race measure can be interpreted as follows in the case of black residents: On average, being black rather than white is associated with a 4.2 percentage point lower probability of a major injury fall being reported on J1900C, controlling for nursing home‐level race mix. The nursing home‐level race measure can be interpreted as follows in the case of more black residents: Holding constant patient race, increasing the proportion of black residents from 0 to 1 is associated with a 25.1 percentage point lower probability of a major injury fall being reported on J1900C.

^cLinear multilevel models are shown here for ease of interpretation. Appendix Tables S5 and S6 show multilevel logistic regression models and a comparison table to demonstrate the two approaches produce similar results.

The Pearson correlation between the 2014 claims‐based major injury fall rates and the MDS‐based major injury fall rates reported on NHC was 0.22 (Table 4). Correlations were also poor between the claims‐based fall rates and the NHC quality measure star ratings (−0.05) and overall star ratings (0.05). Across the quintiles of claims‐based fall rates, about half of nursing homes had a four‐ or five‐star overall rating and at least 75 percent had a four‐ or five‐star quality rating.

Table 4.

Correlations between claims‐based fall rates and Nursing Home Compare measures in 2014

Quintiles of claims‐based fall rates, means, 10th, 90th percentiles	Percent of NHs with 4‐ or 5‐star ratings		NH average ratings
	Overall rating	Quality measure rating	Overall rating	Quality measure rating	MDS 3.0 Major injury falls measure (N013.01)
6.0 (4.5, 8.1)	53.3	75.4	3.40	4.03	4.14
3.6 (3.1, 4.2)	51.5	78.2	3.36	4.11	3.55
2.6 (2.2, 2.9)	50.4	80.5	3.35	4.16	3.25
1.8 (1.5, 2.1)	48.8	77.1	3.29	4.08	3.13
1.1 (0.7, 1.4)	47.0	81.9	3.21	4.22	2.65
Correlation coefficients between claims‐based fall rates and measure			0.046	−0.048	0.223

Abbreviation: NH, nursing home.

Claims‐based fall rates are the number of major injury falls identified in Medicare Provider Analysis and Review (MedPAR) per 100 registered residents in each nursing home in the year 2014.

On NHC, the overall rating is based on a nursing home’s ratings for health inspections, quality measures (QMs), and staffing, while the quality rating is based on only the 16 physical and clinical QMs. The NHC MDS 3.0 measure (N013.01) is the percent of long‐stay residents experiencing one or more falls with major injury.

4. DISCUSSION

This is the first national‐level assessment of how nursing homes self‐report major injury falls data, which are used by CMS for quality measurement and public reporting. We found substantial underreporting on the specific MDS item (J1900C) used by NHC. Reporting rates on the MDS of claims‐identified falls by Asian, black, and Hispanic residents were substantially lower than those for white people both within and across nursing homes, consistent with long‐standing concerns about racial disparities in nursing home care.33, 34, 35

For questions about falls during nursing home residency, we conservatively identified a denominator population of major injury falls in Medicare hospital admission claims that nursing homes should be aware of both administratively and clinically. The individuals who experienced these falls were discharged from and returned to the same nursing home within one day of the hospital admission and discharge, respectively, and had discharge assessments from the nursing home. In a sensitivity analysis, allowing falls reported on additional assessments after readmission to the nursing home count had little effect on the reporting rates.

Comparisons between reporting rates of different MDS items suggest some insights. The reporting rate for item J1700A, which asks about falls in the month prior to the current residency, was 94.3 percent —much higher than the rates for J1800 and J1900C, which ask about falls during the current residency. This was contrary to expectations since in the former case nursing homes rely on secondary sources for fall information but in the latter have the resident in their care. One explanation for this could be that administrative processes at admission and discharge lend themselves to differences in reporting accuracy; another could be that nursing homes underreport when they may be considered responsible for a fall.

Second, J1800, which asks about any falls during the current stay, had a higher reporting rate than J1900C, which asks only about major injury falls during the current stay. It may be that nursing homes have trouble with injury severity classification prior to learning the hospital’s diagnosis. However, the patients in our J1900C denominator returned to the same nursing home, nursing homes have 14 calendar days to submit discharge assessments, and detailed policies and procedures are in place for submitting corrections to the MDS. Nonetheless, it may be difficult in practice to follow these rules; alternatively, nursing homes may be downgrading the severity to improve their quality ratings.

To our knowledge, only state and federal offices with oversight responsibilities have compared patient‐level MDS records with other data sources, typically medical records or medical assessments.3, 4, 6, 7, 8, 9 For instance, in a 2014 audit, CMS compared MDS assessments with patients’ medical records in 25 volunteer nursing homes for up to 10 patients per home.9 For falls, 26 percent of reviewed MDS assessments disagreed with the medical record as to whether the patient sustained a fall‐related major injury. These studies, conducted in a handful of sites, based denominators on the MDS rather than a validation source and therefore may have entirely missed cases unreported by the MDS. The HHS OIG and the GAO have repeatedly recommended that CMS check the reliability of MDS data, most recently in September 2018.36

The poor correlation between our claims‐based fall rates and the MDS‐based NHC‐reported fall rates indicates the MDS not only underreports but also may not be informative for comparing nursing homes. Correlations were also poor between the claims‐based measure and the quality‐domain and overall five‐star ratings, though this is less surprising given the five‐star algorithm’s complexity and use of other information. Nonetheless, such inconsistencies between the five‐star ratings and other nursing home‐level quality measures have been documented elsewhere.12, 13, 37 For example, in an analysis of fee‐for‐service Medicare beneficiaries, Neuman et al were unable to estimate consistent associations between readmission or death risk and MDS‐based measures, including those for pain and ulcers.12 Similarly, in a comparison of facility‐reported staffing data with potentially more objective payroll‐based data, Geng et al found discrepancies and evidence of manipulation around surveyor visits.38 Given this context and our results, other MDS‐based measures should also be assessed for accuracy.

This study has limitations. First, our denominators only included Medicare beneficiaries who had falls severe enough to lead to hospital admission and be classified as major injury according to the MDS. For example, patients who received only outpatient service in an emergency department would not be included in our study. This does not affect reporting rates in our sample of serious falls, especially for J1900C, the item used by NHC that focuses on major injury falls, which are more likely to result in hospital admission. However, extrapolating from our estimates to reporting rates for less severe falls is unlikely to produce accurate estimates.

Second, we relied on claims data, which are not medical records, for diagnosis information. If these contain errors, we may have incorrectly identified some cases or missed others, and our comorbidity scores and injury severity scores may also not be accurate. At the same time, these data are used extensively in high‐quality health services research, including to identify fall‐related injuries. Furthermore, it is unlikely that missingness in diagnosis coding for a fall‐related hospital admission claim would be systematically correlated with nursing home reporting of a fall in the MDS, given the two institutions have separate administrative processes and incentives.

Third, it is possible that some claims‐identified falls in our J1800‐J1900C denominators occurred in‐hospital during a visit for an unrelated condition. However, this is unlikely because CMS considers in‐hospital falls to be “never events” and does not reimburse hospitals for the associated costs.39 If “never event” claims are submitted for tracking purposes, they are expected to be submitted as “no‐pay” claims, which are not included in our MedPAR file.

Finally, though we found weak correlations between our claims‐based major injury fall rates and other NHC measures, we were not able to investigate reasons beyond poor MDS reporting.

A few policy implications flow out of our study for measuring and monitoring falls in nursing homes. First, claims‐based measures may be useful supplements or replacements for the MDS‐based patient safety indicator. CMS has already introduced a few claims‐based utilization measures, so the addition may not be overly burdensome. This would be consistent with CMS’s move toward payroll‐based staffing data for NHC, which are likely more reliable than the previous self‐reported source.38 Second, it may be possible to make model‐based corrections of MDS reporting rates based on nursing home characteristics and history. Finally, both claims data and models such as the one we developed may be used for automatic and targeted auditing.

Our study indicates an urgent need to assess the value and limits of patient safety measurement that is based on the MDS. Given the amount of research that has been based on the MDS, it may be important to revisit some of our understanding of nursing home quality of care. For example, given underreporting was worse for underserved populations in our analysis, disparities in these settings may currently be poorly estimated. At the same time, alternative and additional approaches to monitoring and measuring patient safety in nursing homes should be developed.

CONFLICT OF INTEREST

We have no conflicts of interests pertaining to this article.

IRB APPROVALS

The research protocol was approved by the relevant institutional review board at the University of Chicago. All data analyses were conducted by PS and SP.

Supporting information

 

 

Sanghavi P, Pan S, Caudry D. Assessment of nursing home reporting of major injury falls for quality measurement on nursing home compare. Health Serv Res. 2020;55:201–210. 10.1111/1475-6773.13247