Underreporting of Fall Injuries by Older Adults: Implications for Wellness Visit Fall Risk Screening

Abstract

Background

Self-report is the primary method of identifying fall-related injuries (FRIs) among older adults. Fall risk questionnaires now satisfy documentation requirements for risk assessment in the Medicare Annual Wellness Visit—potentially increasing the number of patients screened for FRIs by interview and paving the way for greater public health efforts to prevent and monitor FRIs. However, the accuracy of self-reported FRIs (SFRIs) and whether accuracy varies by patient characteristics are unknown.

Objective

To measure the accuracy and factors affecting the accuracy of SFRIs versus administratively-obtained FRIs (AFRIs).

Design

We utilized 24-month self-report recall data from 2000-2012 Health and Retirement Study data and AFRIs identified using linked inpatient, outpatient, and ambulatory Medicare data. Sensitivity and specificity were assessed, with AFRIs defined using the UCLA/RAND algorithm as the gold standard. Logistic regression models were estimated to identify sociodemographic and health predictors of sensitivity.

Participants

47,215 individuals ages ≥65 years.

Results

Overall sensitivity and specificity were 28% and 92%. Sensitivity was greater for the oldest adults (38%), women (34%), those with more functional limitations (47%), and those with a prior fall (38%). In adjusted results, several patient factors (women, white race, poor functional status, depression, and prior falls) were modestly associated with better sensitivity and specificity. Injury severity (requiring hospital care) most substantively improved SFRI sensitivity (73%).

Conclusion

An overwhelming 72% of patients who received Medicare-reimbursed health care for FRIs failed to report a fall injury when asked. Future efforts to address underreporting in primary care among non-white and healthier older adults are critical to improve preventive efforts. Redesigned questions—for example, that address stigma of attributing injury to falling—may improve sensitivity.

INTRODUCTION

Self-report is the primary method of identifying fall-related injuries (FRIs) among older adults. In 2011, assessment of fall risk became part of health risk assessment within the Medicare Annual Wellness Visit. The American Geriatrics Society (AGS) also recommends screening of prior falls for older adults who visit a clinician (Panel on Prevention of Falls in Older Persons and British Geriatrics 2011). Screening using fall-risk questions can identify new at-risk older individuals who may benefit from fall prevention efforts. Self-reported data from national survey also provides surveillance prevalence estimates.

Using intensive interview methods (e.g., telephone review of patient’s self-recorded falls diaries) and small community samples (Cummings, Nevitt, and Kidd 1988; Ganz, Higashi, and Rubenstein 2005; Hale, Delaney, and Cable 1993; McKenzie, Enraght-Moony, Walker, McClure, and Harrison 2009; Peel 2000), prior studies suggest older adults modestly underreport their falls and FRIs. Accuracy was 56% to 87% for falls (Cummings, Nevitt, and Kidd 1988; Hale, Delaney, and Cable 1993; Mackenzie, Byles, and D’Este 2006; Peel 2000) and 60% for FRIs (Cummings, Nevitt, and Kidd 1988). However, similar studies have not been performed using nationally representative survey data. Additionally, although perceptions of fall risks can differ by gender (Horton 2006) and by race/ethnicity (Bohannon, Hanlon, Landerman, and Gold 1999; Ellis and Trent 2001), little is known about differences in accuracy of fall reporting across patient populations.

To explore these issues, we assessed accuracy of self-reported FRIs using the Health and Retirement Study (HRS), a large, nationally-representative survey of older Americans, with linkages to Medicare claims data. Its large sample enabled assessment of the accuracy of FRIs across varying time intervals lapsed between the injury and interview. The linked Medicare data including outpatient claims allowed for estimates of the full range of injuries. We used a claims-based algorithm (Hoffman, Hays, Shapiro, Wallace, and Ettner 2016, 2016; Kim et al. 2016) as the gold standard to test self-reports of FRIs from the HRS.

METHODS

Data and Study Population

Our final sample included 47,215 adults ages ≥65 from the 2000 to 2012 of the HRS, a biennial, nationally-representative study conducted primarily by telephone interviews. Proxy interviews were conducted for participants unable to participate due to medical or cognitive disability. Post-mortem interviews were conducted with next of kin. Over 80% of participants consented to data linkage with Medicare, which included acute, ambulatory, emergency-room (ER), and nursing home care. Interviews were included if the participant was continuously enrolled in Medicare Parts A and B for the two years prior to the interview.

Outcome Variable

To create our “gold standard,” we utilized a method devised by UCLA/RAND to indicate an administratively-identified FRI, or AFRI (Hoffman, Hays, Shapiro, Wallace, and Ettner 2016, 2016; Kim et al. 2016). An AFRI was identified using ICD-9 diagnostic codes for fractures, head trauma, and joint dislocations, plus external cause-of-injury codes and physician and outpatient procedural codes indicating falls (Hoffman, Hays, Shapiro, Wallace, and Ettner 2016). We classified a patient as having an AFRI in the two years prior to each survey, if an AFRI was observed during (a) the time since their last scheduled survey or (b) the prior two years if missing the prior survey.

Predictor Variable of Interest

We classified an individual as having a self-reported FRI (SFRI) in interview if the respondent reported having a fall with an injury serious enough to require medical treatment. The SFRI was considered to be the “test” in comparison to the “gold standard” represented by the AFRI.

Respondent and Clinical Characteristics

To assess whether respondent characteristics were predictive of reporting accuracy, we assessed: age, gender, and race/ethnicity; general health status, numbers of chronic health conditions (Fauth, Zarit, Malmberg, and Johansson 2007), difficulties performing activities of daily living (ADLs), depressive symptoms (Radloff 1977), and cognitive impairment (Fong et al. 2009; Dal Forno et al. 2006). We measured prior self-reported fall status (no fall versus fall without injury or SFRI in prior survey) and dual Medicare and Medicaid eligibility (KFF 2014). Because the sensitivity is expected to be improved by a more recent or severe event, we examined (a) the days elapsed between the AFRI and survey (0-60, 61-120, 121-360, and 361-720 days) and (b) the severity of the injury. To construct a proxy for injury severity, we adapted UCLA/RAND categories (greatest to least severity): (1) probable inpatient (inpatient care for an injury diagnosis), (2) probable outpatient (injury diagnosis plus fracture splinting, casting, or repair), (3) possible outpatient AFRI (injury diagnosis plus imaging only) and (4) fall-related medical care (fall e-code without injury diagnosis, inpatient or outpatient) (Kim et al. 2016).

Analysis

We assessed the accuracy of respondent SFRIs compared to the AFRI gold standard. The unit of analysis each 2-year interview interval matched with the corresponding lookback period in the administrative data. We assessed (1) sensitivity, the proportion of AFRIs where the patient self-reported an FRI and (2) specificity, the proportion of claims with no AFRIs where the patient did not report an FRI. We first calculated the unadjusted sensitivity and specificity and 95% confidence intervals (CI) overall, then compared differences by respondent and clinical characteristics (Figures 1 and 2) using tests of proportions.

Figure 1.

Sensitivity of Self-Reported Fall-Related Injuries of Older Adults (≥65) by Respondent and Clinical Characteristics (n=7,442)

Note: The figure represents sensitivity (%) and 95% confidence intervals. Data are from the 2000-2012 waves of the Health and Retirement Study. Sensitivity is the proportion of administratively identified fall-related injuries (FRIs) where the patient self-reported an FRI.

Next, we specified multivariable logistic regression models to estimate the odds of (1) a false negative response (i.e., not reporting an SFRI when an AFRI occurred) and (2) a true negative response (i.e., denying SFRI when no AFRI occurred). We adjusted model standard errors using Stata’s cluster command in order to account for repeated observations (Hoffman et al. 2017).

RESULTS

Descriptive Statistics

There were nearly twice as many AFRIs (n=11,080) as SFRIs (n=6,128) (Table 1).

Table 1.

Self-reported and Administratively Identified Fall-related Injuries (FRI) of Older Adults (≥65) by Respondent Characteristics

	Self-reported FRI (%) (n=6,128)	Administratively Identified FRI (%) (n=11,080)
Gender
Male (n=19,488)	9.6	21.7
Female (n=27,727)	15.3	24.7
Age category
65-74 (n=18,956)	9.0	20.5
75-84 (n=17,127)	12.0	26.9
≥85 (n=7,327)	20.2	35.4
Race/Ethnicity
Non-Hispanic White (n=38,211)	13.5	24.4
African-American (n=5,596)	9.0	18.2
Hispanic (n=2,072)	13.3	24.0
Other (n=1,322)	14.0	19.3
Education
Less than high school (n=13,422)	14.3	22.3
High school (n=17,191)	12.4	23.4
Some college (n=8,526)	12.9	24.1
College and above (n=8,074)	12.2	24.9
Health Status
Very good/excellent (n=14,408)	7.6	19.8
Good (n=13,934)	10.7	24.3
Fair/poor (15,024)	17.5	32.1
Number of Chronic Conditions
0 (n=11,151)	13.9	13.4
1 to 2 (n=26,398)	11.4	24.8
3 to 4 (n=9,098)	15.8	31.1
5 to 6 (n=568)	23.1	39.1
Number of ADL limitations
0 (n=32,761)	8.6	21.8
1 to 2 (n=6,703)	18.6	32.3
3 to 5 (n=3,924)	29.6	45.2
Number of depressive symptoms
0 (n=16,512)	7.9	20.5
1 to 2 (n=13,579)	10.3	25.3
3 to 4 (n=5,158)	16.3	29.2
5 to 8 (n=3,888)	19.0	31.1
Non-proxy cognitive impairmenta
Yes (n=5,265)	20.8	34.2
No (n=41,950)	12.0	22.1
Dual eligible status
Yes (n=4,776)	16.7	31.3
No (n=38,247)	11.4	24.7
Prior fallb
Yes (n=13,060)	21.8	28.1
No (n=24,919)	8.5	22.0

^aCognitive impairment measured with proxy respondent

^bPrior fall measured in last survey wave

Note: Data are from the 2000-2012 waves of the Health and Retirement Study. Significant differences (p<0.001) were observed across respondent characteristics for both SFRIs and AFRIs. Approximately 10% of AFRIs occurred 0-2 months and 2-4 months prior, while 34% and 47% of AFRIs occurred 4 months to 1 year and 1-2 years prior to the date of survey administration. Nine percent of AFRIs involved probable inpatient FRIs, 6% were probable outpatient FRIs, and 10% were possible outpatient FRIs; the remainder, 75%, involved fall-related medical treatment.

Unadjusted Accuracy of Self-reported FRIs (Figure 1 and eFigure 1)

Overall, sensitivity and specificity for all respondents were 28% (95% CI: 27%-29%) and 92% (95% CI: 91%-92%), respectively.

In general, sensitivity was higher in sicker patients, for example 25% in patients with zero compared to 36% for 5-6 chronic conditions (Figure 1). Sensitivity for AFRIs that occurred 0-60 days prior to interview was 32% and declined to 25% at 1-2 years prior to interview. Sensitivity was greater for probable inpatient (73%) and probable outpatient (63%) compared to possible outpatient (33%) and fall-related medical care (20%) AFRIs, with respective specificities of 86%, 83%, 76%, and 59%. Specificity was higher and varied less between risk groups than sensitivity (eFigure 1).

Multivariable-Adjusted Results (Table 2 and eTable 1)

Table 2.

Sensitivity Model: Predictors of False Negative Responses of Fall-Related Injuries for Older Adults (≥65) (n=7,916)

	AORa	95% CI		p
Age category (reference: 65-74)
75-84	0.91	0.80	1.04	0.18
≥85	0.76	0.64	0.90	0.001
Male	1.61	1.41	1.84	<0.001
Race/Ethnicity (reference: Non-Hispanic White)
African-American	1.61	1.26	2.04	<0.001
Hispanic	1.23	0.90	1.68	0.20
Other	1.09	0.70	1.68	0.71
Education (reference: < high school)
High school	0.97	0.82	1.15	0.73
Some college	0.93	0.77	1.12	0.43
College and above	0.76	0.62	0.92	0.01
Health Status (reference: Very good/excellent)
Good	0.91	0.78	1.06	0.24
Fair/poor	0.87	0.74	1.03	0.10
Number of Chronic Conditions (reference: 0)
1 to 2	0.85	0.65	0.96	0.02
3 to 4	0.71	0.57	0.87	0.001
5 to 6	0.80	0.52	1.22	0.30
Number of ADL limitations (reference: 0)
1 to 2	0.71	0.61	0.82	<0.001
3 to 5	0.44	0.36	0.54	<0.001
Number of depressive symptoms (reference: 0)
1 to 2	0.96	0.83	1.11	0.60
3 to 4	0.67	0.56	0.81	<0.001
5 to 8	0.75	0.61	0.91	0.004
Cognitive impairmentb	1.08	0.88	1.32	0.47
Dual eligible status	1.16	0.93	1.44	0.18
Prior fallc	0.52	0.46	0.58	<0.001
Severity level (reference: probable inpatient FRI)
Probable outpatient FRI	1.23	0.93	1.63	0.15
Possible outpatient FRI	4.75	3.66	6.16	<0.001
Fall-related medical treatment	10.32	8.33	12.78	<0.001

^aAOR= Adjusted odds ratio

^bCognitive impairment measured with proxy respondent

^cPrior fall measured in last survey wave

Note: Data are from the 2000-2012 waves of the Health and Retirement Study. The analytic sample began with 11,080 individuals with an AFRI; however, for the regression we used complete-case analysis, which resulted in a final sample of 7,916 individuals.

After multivariable adjustment, several factors persisted as more accurate reporting subpopulations. Overall, women, non-Hispanic whites, those with greater functional impairment, depressive symptoms, those with a prior fall, and those with a more severe injury were more accurate reporters of FRIs (Table 2). We did not observe different patterns when examining the odds of false negative self-report when separately examining probable inpatient, probable outpatient, possible outpatient AFRIs, or fall-related medical treatment. Patterns were also similar when accuracy was assessed as the odds of true negative responses (eTable 1).

DISCUSSION

In this national survey of older Americans linked with their inpatient and outpatient Medicare data for health care received for fall injury, self-reporting as the sole method of data collection on a biennial basis greatly underestimated actual health care delivery for fall injury events. We observed an overall sensitivity of only 28%. To our knowledge, this is the first analysis to examine the accuracy of fall self-report and predictors of accuracy using a large, nationally-representative survey.

To the extent that individuals fail to identify prior events, prevention opportunities are missed. For an FRI that occurred one year ago or less, similar to the time period discussed during the Medicare Annual Wellness Visit, less than 1 in 4 FRIs are accurately reported, meaning over 75% of patients may leave the visit without fall prevention activities being initiated. We also found that FRIs are potentially undercounted with decreasing age, among men, non-white patients, and those with better functioning and health. These groups should be recognized in future fall prevention efforts and in survey-based prevalence estimates. In particular, healthier older adults who did not recall or denied having medical care for their FRI are under-recognized. They could benefit from enhanced population-level fall prevention efforts particularly because they may not see themselves as at-risk.

Our estimates of SFRI’s sensitivity are worse than prior work using more intensive interview methods (e.g., weekly diaries), which ranged from 56%-89% (Cummings, Nevitt, and Kidd 1988; Peel 2000; McKenzie, Enraght-Moony, Walker, McClure, and Harrison 2009; Hale, Delaney, and Cable 1993). However, prior studies likely over-estimated sensitivity as they were specifically designed to explore fall recall (Cummings, Nevitt, and Kidd 1988), which likely sensitized patients to recall of falls. They also used shorter recall periods, which (as demonstrated in this research) can improve recall (Schaeffer and Presser 2003; Huttenlocher, Hedges, and Bradburn 1990; Converse and Presser 1986). Finally, they often did not examine less serious injuries, as ours did. When a prior study included outpatient AFRIs, it reported a sensitivity of just 24% (Mackenzie, Byles, and D’Este 2006). We notably found a high proportion of injuries (~75%) in the least severe category (fall-related outpatient care without other evidence of fracture/dislocation/head injury), consistent with another report that most fall injuries are minor such as bruises and sprains (28). Minor injuries were likely included in our least-severe category, which was associated with the poorest accuracy. Certainly, minor injuries are less memorable than injuries requiring casting or surgery. We also speculate, however, in the absence of a procedure, that some patients may not consider medical evaluation as being medical treatment. We suggest that future fall injury screening inquire about “medical attention” to capture the fuller range of FRIs.

While cognitive impairment is expected to worsen accuracy, past studies excluded such individuals (Cummings, Nevitt, and Kidd 1988; Hale, Delaney, and Cable 1993; Ganz, Higashi, and Rubenstein 2005). In the HRS, cognitively-impaired individuals had proxy respondents help them to report falls, which explains why we observed better rather than worse recall in that group in unadjusted sensitivity.

Finally, subtle psychosocial factors may also explain poor accuracy. The topic of falling can be met with embarrassment, fear, or avoidance (Stevens, Noonan, and Rubenstein 2010; Ballinger and Payne 2002; Faes et al. 2010). The word “fall” may carry its own stigma, as the act of falling implies weakness and frailty (Yardley et al. 2008; Stevens, Noonan, and Rubenstein 2010). Perceived stigma may also vary by culture, which could explain our observed differences in accuracy by race/ethnicity (Roe et al. 2008). Second, patients may provide inaccurate information due to lack of insight into the cause of their fall (Cummings, Nevitt, and Kidd 1988; Peel 2000; Ganz, Higashi, and Rubenstein 2005). Patients (and clinicians) often attribute their fall injury to an environmental hazard rather than their own health or behavior (Ballinger and Payne 2002; Yardley, Donovan-Hall, Francis, and Todd 2006). In reality, however, fall in response to tripping results from inability to compensate and prevent the fall from occurring.

To normalize the experience of a fall injury, survey questions could be redesigned to say: “Have you fallen or been injured by falling in the past year, even if the cause was accidental or due to tripping over something in your way?” Surveys might also clearly define falls (Biderman, Cwikel, Fried, and Galinsky 2002) and FRIs so that respondents are prompted to report both minor injuries (i.e., any requiring medical attention) as well as severe FRIs. Further, qualitative research should be performed to better understand cultural sensitivity around fall risk among older non-white adults. Similarly, as the baby boom generation fully moves into old age, this cohort may also require a generational cultural sensitivity when gauging fall risk. Such efforts to understand and address stigma and clarity in fall questions may both improve prevalence estimates and clinical fall risk assessments for Medicare patients. This in turn could assist population-wide efforts to reduce the risks, morbidity, and expenditures associated with fall injuries (Blank et al. 2011).

Limitations

Our gold standard, AFRIs, was limited to coded injuries, which more severely limits outpatient data, where physicians are not required to submit a fall e-code as a diagnosis. If an office evaluation was performed, but no injury was found, then we conservatively counted this as a false positive. Second, we could not consider less-intensive care that some may interpret as medical treatment, for example, telephone-based care. Finally, telephone-based interview methods may not generalize to the clinical setting of the Annual Wellness Visit. Applying these results to the Annual Wellness Visit may require consideration that patients may be more or less forthcoming about their fall injuries on the telephone compared to the clinical visit, but we believe the differences are highly variable from patient to patient. As clinical practices move from using personal physicians to support staff and electronic patient portals to collect information, we believe the telephone as the mode of communication represents little systematic threat to the validity of these results. Finally, we argue that these results bring attention to missed opportunities for fall injury prevention. However, we acknowledge that some of our sample may actually have received preventive care. Unfortunately, it is not possible to capture receipt of preventive care in this dataset.

Conclusion

In summary, using a nationally-representative survey, we found that FRIs are significantly under-reported among older adults. Our data suggest that subgroups of older patients can be targeted for better surveillance of fall injuries. In addition, future efforts to improve accuracy of survey and Medicare Annual Wellness Visit questions for greater accuracy may require overcoming the stigma of attributing injury to falling.

Supplementary Material

supp info

Supplementary File Title: Specificity of Self-Reported Fall-related Injuries.

Impact statement.

We certify that this work is novel research. Up to 70% of older Medicare beneficiaries receiving medical care for injurious fall do not accurately report their fall injury when asked to recall up to 2 years later. Given the new emphasis on collecting fall risk in the Medicare Annual Wellness Visit and the critical importance of asking about falls to prevent future falls in older adults, this information can inform health policy and future fall prevention efforts. To improve future surveillance of fall reporting, this research identifies subgroups of patients with poorer reporting: patients seen only in outpatient settings for their fall-related injury, non-white patients, and patients with no functional disability. In this manuscript, we also discuss how future efforts to improve accuracy of survey and Medicare Annual Wellness Visit questions should address the stigma associated with falling for older adults.