Validation of the Minimum Data Set items on falls and injury in two long-stay facilities

Falls are common amongst nursing home (NH) residents with 4–11% of falls resulting in fracture or other serious injury.^1,2. Injurious falls have a higher incidence and mortality in NH residents than falls amongst community dwellers^1,3,4. Therefore, it is important to accurately identify falls in NH residents.

The Minimum Data Set (MDS) was developed as a means to assess the health status of NH residents⁵. In 2010, the MDS v3.0 was deployed in the United States, with considerable changes in how falls and injuries are assessed⁶. The purpose of our study was to validate the MDS v3.0 items on falls and injuries with chart review in two facilities.

Our study was conducted in two long-term care facilities (MA&NC). Both facilities employ software that auto-populates items using the prior MDS assessment, including falls reporting. Eligible residents had at least two valid MDS assessments between 1/2016 and 4/2019. From the first facility we randomly selected 50 residents with an MDS indicator for an injurious fall, 50 with an MDS indicator for a fall without injury, and 50 without fall. Only two major injuries were initially selected, and so we identified an additional 23 residents with an indicator of major injury. From the second facility, we sampled all residents with an injurious fall indicator (n=10), all residents with a fall without injury (n=18), and a random sample of 50 residents without fall.

The MDS v3.0 queries whether a fall has occurred since admission or since the last MDS assessment. If a fall occurred, staff categorize the number of falls with no injury, minor injury, or major injury. A major injury is defined as falls resulting in fracture, dislocation, concussion, or intracranial hemorrhage. Minor injuries are defined as falls resulting in pain, or a skin tear, abrasion, laceration, bruise, hematoma, or sprain.

Blinded, trained abstractors (JM, EH, & AL) conducted the chart review. For each resident, the abstractor reviewed all provider and nursing notes in the electronic medical record between the relevant MDS assessments. Pain was operationalized as lasting >24 hours, required analgesia medications, or required evaluation by a provider. All falls and injuries were recorded in a REDCap database. A second abstractor (CC) reviewed 8 charts with good concordance (Cohen’s Kappa=0.81) Residents were then categorized as having one or more fall, fall with minor injury, and fall with major injury. Fall and injury agreement between the MDS and chart review was assessed using Cohen’s Kappa test. Sensitivity, specificity and positive predictive value (PPV) were calculated.

In total, we included 251 residents. Mean age was 89.4 years (± 8.0), 68.1% were female, and 8.4% were non-white. The average time between MDS assessments was 108 days (±46). Our chart review identified 124 fallers, of whom 20 had one or more major injuries and 85 had one or more minor injuries. The most common major injury was fracture (n=18). The most common minor injuries were pain (n=56) and bruising (n=44).

Table 1 shows agreement between falls and injury as assessed by the MDS and chart review. Kappa agreement was low for all comparisons: 0.44 for falls, 0.30 for major injury, 0.24 for minor injury. The sensitivity, specificity and PPV of the MDS to identify falls was 0.78, 0.66, and 0.69, respectively. For major injuries, the sensitivity, specificity, and PPV was 0.40, 0.93, and 0.33. For minor injuries, the sensitivity, specificity, and PPV was 0.39, 0.84, and 0.55. Results were similar between the two facilities.

Table 1.

Comparison of Falls and Injury ascertained by the MDS (version 3.0) versus chart review

Falls
MDS	Chart Review
		Fall	No Fall	Total
	Fall	96	43	139
	No Fall	27	85	128
	Total	123	112	251
		Sensitivity: 78%	Specificity: 66%
Major Injuries
MDS	Chart review
		Major Injury	No major Injury	Total
	Major injury	8	16	24
	No major injury	12	215	227
	Total	20	231	251
		Sensitivity: 40%	Specificity: 93%
Minor Injuries
MDS	Chart Review
		Minor Injury	No minor Injury	Total
	Minor Injury	33	27	60
	No Minor Injury	52	139	191
	Total	85	166	251
		Sensitivity: 39%	Specificity: 84%

A previous study validating the MDS v2.0 found good specificity (97%) but low sensitivity (53%) to identify falls.⁷To our knowledge, no studies have validated MDS v3.0 fall or injury items. We found the sensitivity and specificity of the MDSv3.0 to identify falls to be quite modest. Falls with major injuries had a greater specificity than falls without injury (0.93 vs 0.66), but sensitivity was quite low (0.40). This suggests that data auto-populated from prior assessments may have been retained rather than changed to the most recent clinical data. Despite the modest validity, the MDS v3.0 question on falls does accurately characterize NH residents at risk for future fracture.⁸ Thus falls documented in the MDS may capture fall risk factors, rather than the actual occurrence of falls.

This study is limited by including only two institutions. However, these facilities are both academically affiliated and have high CMS quality ratings, suggesting that the low accuracy is not limited to low-resource facilities. We oversampled major injuries, and it is possible that in a truly random sample the sensitivity would have been lower.

In summary, facilities and researchers that wish to capture incident falls and fall related injuries may need to consider alternative strategies other than the MDS, such as claims data or facility logs.