Predicting posthospitalization falls in Brazilian older adults: External validation of the Carpenter instrument

Abstract

Objectives:

This study sought to explore and externally validate the Carpenter instrument’s efficacy in predicting postdischarge fall risk among older adults admitted to the emergency department (ED) for reasons other than falls or related injuries.

Methods:

A prospective cohort study was conducted on 779 patients aged ≥ 65 years from a tertiary hospital in São Paulo, Brazil, who were monitored for up to 6 months post-ED hospitalization. The Carpenter instrument, which evaluates the four risk factors nonhealing foot sores, self-reported depression, inability to self-clip toenails, and prior falls, was utilized to assess fall risk. Follow-up by telephone occurred at 30, 90, and 180 days to identify falls and mortality. Fine–Gray models estimated the predictive power of Carpenter instrument for future falls, considering death as a competing event and sociodemographic factors, frail status, and clinical measures as confounders.

Results:

Among 779 patients, 68 (9%) experienced a fall within 180 days post-ED admission, and 88 (11%) died. The majority were male (54%), with a mean age of 79 years. Upon utilizing the Carpenter score, those with a higher fall risk (≥2 points) displayed more comorbidities, greater frailty, and increased clinical severity at baseline. Regression analyses showed that every additional point on the Carpenter score increased the hazard of falls by 73%. Two primary contributors to its predictive potential were identified: a history of falls in the preceding year and an inability to self-clip toenails. However, the instrument’s discriminative accuracy was suboptimal, with an area under the curve of 0.62.

Conclusions:

While the Carpenter instrument associated with a higher 6-month postadmission fall risk among older adults post-ED visit, its accuracy for individual patient decision making was limited. Given the significant impact of falls on health outcomes and health care costs, refining risk assessment tools remains essential. Future research should focus on enhancing these assessments and devising targeted proactive strategies.

INTRODUCTION

Falls are the primary cause of injuries among older adults,¹ often leading to severe injury, disability, and even death.¹ While numerous studies have emphasized the efficacy of fall prevention interventions within community settings, there remains a discernible gap in targeted strategies posthospital discharge.² This gap is significant, given that nearly 40% of patients experience at least one fall within the 6 months postdischarge, with 54% of these falls culminating in serious injury.^2,3 The Prevention of Falls in the Elderly Trial (PROFET) study further underscored this concern, revealing that older patients managed in emergency services had a pronounced risk of postdischarge falls, with incidences as high as 60% within a year.⁴ In addition, one-third of older adults discharged from the emergency department (ED) after minor trauma (mostly falls) experience functional decline or return visits within 3 months.^5,6 Consequently, pinpointing the primary risk factors for postdischarge falls might be as crucial as managing their repercussions.

In 2009, Carpenter et al.⁷ highlighted an incongruence: while guidelines recommended fall risk screenings for older patients in ED, there was an evident gap in understanding the risk factors inherent to this environment. For the first time, independent 6-month risk factors for falls were described in patients discharged from the ED with any chief complaint other than falls or fall-related injuries.⁷ The research suggested a potential window for primary prevention, targeting patients not conventionally perceived as high-risk but who might exhibit an elevated incidence of falls, injuries, and subsequent hospital revisits.⁷ Corroborating this hypothesis, a 5-year retrospective study analyzing data from 3.8 million older ED patients showed that individuals discharged after a fall often revisited the ED for unrelated complaints.⁸

A systematic review of post-ED fall predictions recognized the Carpenter instrument as the most accurate, defining “high risk” as the presence of two or more risk factors.⁹ This instrument evaluates nonhealing foot sores, self-reported depression, inability to self-clip toenails, and prior falls as independent fall risk factors and assigns a point for each risk factor present, with scores from 0 to 4.⁷ Nonetheless, the instrument’s broader applicability requires further validation, especially since the performance of risk scores often diminishes outside of the original trial settings.^9,10

In summary, while cross-specialty guidelines advocate for fall prevention protocols within emergency care,¹¹ the integration of risk stratification tools for discerning patients most likely to benefit from primary prevention remains underinvestigated.⁷ Notably, no prior research has delved into the postdischarge fall risk factors among older adults admitted to the ED for reasons other than falls or related injuries. Our study aims to bridge this knowledge gap, exploring and externally validating the predictive accuracy of the Carpenter instrument within a sizable cohort of Brazilian older adults monitored for up to 6 months post-ED hospitalization.

METHODS

Design, setting, and sample

We conducted a prospective cohort study involving patients aged ≥ 65 years hospitalized through the ED of a philanthropic tertiary hospital in São Paulo, Brazil, from November 2021 to April 2022. Since 2019, this site has a Level 3 Geriatric Emergency Department Accreditation (GEDA) issued by the American College of Emergency Physicians and is the only one with this accreditation in the southern hemisphere up to this date.¹² Its ED clinical staff is characterized by a high number of geriatricians, encompassing almost one-third of its workforce and is engaged in several policies, procedures, and protocols devised for preventing falls and promoting safe and autonomous mobility for older patients.¹³ Exclusion criteria encompassed patients who were unable to communicate without a proxy, clinically unstable or requiring urgent procedures, inaccessible to research assistants, unwilling to participate, and admitted for fall-related complaints.

The study was approved by the local institutional ethics review board, and all participants or their proxies granted informed consent. Its reporting adheres to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Initiative standards.¹⁴

Study protocol

Within the ED, two trained research assistants performed the baseline evaluations. Their shifts spanned 5 h, covering 40 h weekly on weekdays and 10 h on weekends for 6 months. Eligible patients were enlisted consecutively between the attending physician’s decision for admission and their transfer to an inpatient bed. Data were collected on sociodemographic elements, medical history, comorbidities (as measured by the Charlson Comorbidity Index¹⁵), frail status based on the Clinical Frailty Scale (CFS) scores (0–9) of 5 or higher,¹⁶ acuity illness severity (using NEWS 2 scores¹⁷), and fall-related parameters (via the Carpenter instrument⁷). In-hospital mortality data were sourced from electronic medical records. Postadmission, patients received telephone follow-up at 30, 90, and 180 days to identify fall incidence and mortality.

Measurements and outcomes

The study employed Research Electronic Data Capture (REDCap) resources¹⁸ for every phase of data collection and management. Research assistants equipped with tablets accessed the study’s database for immediate data entry via REDCap during the initial interview and to process the informed consent form. Remote access was available for updating data related to hospital mortality and follow-up interviews. The outcome of our study was the occurrence of falls within 180 days following hospital admission. Importantly, mortality occurring within this period and before any fall was accounted for as a competing event.

Data analysis

Based on their distribution, numeric variables were expressed as means ± SDs or medians with interquartile ranges (IQR). This distribution was ascertained through histograms, skewness, and kurtosis coefficients and the application of the D’Agostinho–Pearson test. The cutoff for nonnormal distribution was p-values below 0.05. Certain variables were also categorized to enhance clinical inter-pretability. Categorical variables were depicted in absolute counts and proportions. For comparisons, we employed Student’s t-test or ANOVA for normally distributed variables and Wilcoxon or Kruskal–Wallis tests for nonnormally distributed ones. Categorical data distributions were contrasted using the chi-square test or Fisher’s exact test when suitable.

We used Fine–Gray subdistribution hazard models to evaluate the risk of falls or death within 180 days postadmission as per the Carpenter instrument. This model analyzed both the continuous score and its categorical divisions (0 points = low risk; 1 point = intermediate risk; ≥2 points = high risk) and each of its constituent subitems. Adjustments in the model accounted for factors such as age, sex, race, education level, marital status, Charlson comorbidity score, frail status, and acuity illness severity. Data representation included the cumulative incidence function curves of falls according to the Carpenter’s risk categories.

We also assessed the Carpenter instrument’s performance in our cohort using receiver operating characteristic analyses and test metrics like the Youden index, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and likelihood ratios. Lastly, we assessed whether the patients’ frail status (CFS <5 vs. ≥5)¹¹ influences the Carpenter accuracy to detect the 6-month risk of falls, considering frailty heightens the odds of falls in older patients.¹⁹ All computations used Stata 17 (StataCorp 2021).

RESULTS

Between November 2021 and April 2022, a total of 1178 hospital admissions occurred during the research assistants’ shifts at the ED. From these, 178 were readmissions of previously included patients, yielding 1000 potentially eligible patients. Subsequently, 170 were excluded for various reasons: inability to communicate without a proxy,¹¹ clinical instability (33), need for urgent procedures (40), refusal to participate (45), and inaccessibility to research assistants (41). This culminated in a cohort of 830 patients. However, 51 of these were admitted due to fall-related complaints and were thus excluded from the current study, resulting in a final sample size of 779 patients (Figure 1).

FIGURE 1.

Study flowchart.

The mean age of our sample was 79 years. The majority were male (54%), identified as White (92%), and had an education median of 15 years (Table 1). Regarding the marital status, 445 (57%) were married or lived with a partner; 103 (13%) were single, divorced, or separated; and 231 (30%) were widowed. The incidence of falls was very similar between single, divorced, or separated patients (8/103 = 8%) compared to those who were widowed (19/231 = 8%).

TABLE 1.

Patient characteristics based on Carpenter instrument categories.

		Carpenter et al. four-item fall risk instrument
	Total N = 779	0 points n = 91	1 point n = 300	≥2 points n = 388	p-value
Sociodemographic factors
Age (years)	79.4 (±9)	74.24 (±6)	78.06 (±8)	81 (±9)	<0.001
Female	359 (46.1)	33 (36.3)	119 (39.7)	207 (53.4)	<0.001
White	716 (91.9)	84 (92.3)	275 (91.7)	357 (92.0)	0.98
Education (years)	15 (11–17)	16 (13–18)	15 (11–17)	14 (11–17)	<0.001
Married/with a partner	445 (57.1)	63 (69.2)	183 (61.0)	199 (51.3)	0.002
Clinical measures
Charlson Comorbidity Index					0.002
0	198 (25.4)	30 (33.0)	87 (29.0)	81 (20.9)
1	183 (23.5)	23 (25.3)	71 (23.7)	89 (22.9)
2	170 (21.8)	20 (22.0)	73 (24.3)	77 (19.8)
3	117 (15.0)	13 (14.3)	34 (11.3)	70 (18.0)
≥4	111 (14.2)	5 (5.5)	35 (11.7)	71 (18.3)
Frail status (CFS ≥ 5)	209 (26.8)	1 (1.1)	50 (16.7)	158 (40.7)	<0.001
NEWS 2	1 (0, 3)	1 (0, 2)	1 (0, 3)	2 (0, 4)	<0.001

Note: Data are reported as mean (±SD), n (%), or median (IQR).

Abbreviations: CFS, Clinical Frailty Scale; IQR, interquartile range; NEWS, National Early Warning Score.

Upon categorizing patients using the Carpenter score, we observed that those with a higher risk of falls (≥2 points) had more comorbidities, exhibited greater frailty, and presented increased clinical severity at baseline compared to those with lower (0 points) or intermediate risks (1 point; Table 1). They were also characterized by older age, higher frequency of female sex, and lower educational level and married status.

Post–ED admission, 68 patients (9%) experienced a fall, while 88 (11%) died within 180 days. Notably, among the patients who died, only five had experienced a fall prior to their death. Therefore, for 83 of the deaths (94%), mortality was considered a competing event. Higher scores on the Carpenter instrument were associated with increased 6-month mortality (score 0 = 2 deaths [2.2%]; score 1 = 28 deaths [9.3%]; score ≥2 = 58 deaths [14.9%]; p = 0.001). Overall, 628 patients were followed for 6 months without experiencing a fall or passing away during this period. The median follow-up time was 180 days with an interquartile range of 180–180 days. A higher Carpenter score was evident among those who fell, primarily driven by a history of falls in the preceding year and an inability to self-clip toenails (Table 2).

TABLE 2.

The incidence of falls during the 6-month follow-up according to the Carpenter instrument items and scores.

	Total N = 779	No falls n = 711	≥1 fall n = 68	p-value
Falls risk factors
Nonhealing foot sores	4 (0.5)	3 (0.4)	1 (1.5)	0.25
Self-reported depression	450 (57.8)	407 (57.2)	43 (63.2)	0.34
Not clipping toenails	486 (62.3)	435 (61.2)	51 (75)	0.025
Previous falls	250 (32.1)	216 (30.4)	34 (50)	<0.001
Carpenter score				0.003
0 point	91 (11.7)	87 (12.2)	4 (5.9)
1 point	300 (38.5)	281 (39.5)	19 (27.9)
2 points	275 (35.5)	250 (35.2)	25 (36.8)
3 points	112 (14.4)	92 (12.9)	20 (29.4)
4 points	1 (0.1)	1 (0.1)	0 (0.0)

Note: Data are reported as n (%).

In our regression analyses, when considered as a discrete variable, each incremental point in the Carpenter score increased the hazard of falls by 73%, whereas scores of ≥2 were associated with a 205% heightened hazard relative to the reference group when the score was categorized (Table 3, Figure 2). In the model exploring the individual subitems, a history of prior falls and the inability to self-clip toenails stood out, increasing the hazard by 112% and 100%, respectively. Nonhealing foot sores were infrequently observed, and self-reported depression was prevalent among both fallers and non-fallers; neither of these factors demonstrated a significant association with an increased hazard of falls. (Table 3).

TABLE 3.

Association between the Carpenter instrument and falls within 6 months of post-ED admission.

	Unadjusted		Adjusteda
	SHR (95% CI)	p-value	SHR (95% CI)	p-value
Carpenter score	1.64 (1.24–2.16)	<0.001	1.73 (1.28–2.35)	<0.001
Carpenter categories
0 point	(Reference)		(Reference)
1 point	1.43 (0.49–4.18)	0.511	1.50 (0.51–4.39)	0.460
2 or more points	2.69 (0.98–7.44)	0.056	3.05 (1.05–8.88)	0.032
Carpenter subitems
Nonhealing foot sores	3.24 (0.42–24.98)	0.260	4.41 (0.49–39.52)	0.185
Self-reported depression	1.24 (0.76–2.02)	0.380	1.34 (0.82–2.20)	0.239
Not clipping toenails	1.83 (1.06–3.14)	0.029	2.00 (1.07–3.77)	0.031
Previous falls	2.22 (1.39–3.53)	0.001	2.12 (1.31–3.44)	0.002

Abbreviations: CFS, Clinical Frailty Scale; SHR, subdistribution hazard ratio.

Note: Estimates were calculated using Fine-Gray models that considered death as a competing risk.

^aAdjusted for age, sex, race, education, marital status, comorbidity score, frail status (CFS ≥ 5), and acute illness severity.

FIGURE 2.

Cumulative incidence of falls by risk category in the Carpenter instrument, assessed via Fine–Gray competing risks. The log-rank test for trend among the three curves achieved a p-value of 0.002.

Within our cohort, the Carpenter score exhibited suboptimal precision in discriminating patients who would fall during the follow-up, denoted by an area under the curve (AUC) of 0.62 (p < 0.001; Figure 3). Using a threshold of ≥2 points, the score maintained a 66% sensitivity and a 52% specificity. Its PPV stood at 12%, while its NPV was 94% (Table 4). Interestingly, Carpenter’s accuracy in detecting the risk of falls did not vary between frail and nonfrail patients (AUC 0.62 and 0.63, respectively).

FIGURE 3.

ROC analysis of the Carpenter instrument’s performance. ROC, receiver operating characteristic.

TABLE 4.

Accuracy of different cutoff points of the Carpenter instrument for predicting 6-month fall.

Cutoff	Sensitivity % (95% CI)	Specificity % (95% CI)	PPV % (95% CI)	NPV % (95% CI)	+LR (95% CI)	−LR (95% CI)
0	100 (94.7–100.0)	0.0 (0.0–0.5)	8.7 (8.7–8.7)	–	1.0 (1.0–1.0)	–
1	94.1 (85.6–98.4)	12.24 (9.9–14.9)	9.3 (8.8–9.9)	95.6 (89.2–98.3)	1.07 (1.0–1.1)	0.48 (0.2–1.3)
2	66.2 (53.7–77.2)	51.8 (48.0–55.5)	11.6 (9.8–13.6)	94.1 (91.9–95.7)	1.37 (1.1–1.7)	0.65 (0.5–0.9)
3	29.4 (19.0–41.7)	86.92 (84.2–89.3)	17.7 (12.4–24.6)	92.8 (91.7–93.8)	2.25 (1.5–3.4)	0.81 (0.7–0.9)
4	0 (0.0–5.3)	99.9 (99.2–100.0)	–	91.3 (91.2–91.3)	–	1.0 (1.0–1.0)

Note: The AUC for the Carpenter instrument was 0.62 (95% CI 0.58–0.66).

Abbreviations: AUC, area under the receiver operating characteristic curve; +LR, positive likelihood ratio; −LR, negative likelihood ratio; NPV, negative predictive value; PPV, positive predictive value.

DISCUSSION

In our prospective cohort study encompassing a large sample of older adults admitted post–ED visits, the Carpenter score, both as discrete and as categorical variables, was associated with a higher 6-month postadmission fall risk. In our sample, two of its four elements (history of previous falls and inability to self-clip toenails) principally bolstered its predictive potential. Yet, its accuracy, as discerned from the AUC analyses, was found lacking. These findings underscore the importance of prior falls and toenail clipping incapacity as fall predictors post–hospital discharge. They also highlight the Carpenter instrument’s utility in pinpointing patients with a notably low fall risk, as it was shown to have a NPV of almost 95% for those with less than 2 points, a more generalizable measure. Conversely, those at elevated risk might benefit from additional evaluations to discern their long-term outcomes.

Our findings differ from prior studies, as we report a model of fall risk prediction adjusted for several other risk factors. The recorded incidence of falls, standing at 9% up to 6 months post–hospital admission, was relatively low. Earlier meta-analyses had estimated the pretest probability of falls 6 months post–ED visit between 14% for the generic geriatric ED population and 31% for those evaluated for falls during the primary ED consultations.⁹ Our institution’s elevated care standards, alongside the established geriatric ED program since 2015, accredited a Level 3 GEDA in both 2019 and 2022, might explain in part this reduced rate.¹³ It is noteworthy that our study had only a single patient with a nonhealing foot sore, high self-reported depression rates across all examined subgroups, and a mortality rate during follow-up surpassing twice that of the original Carpenter instrument derivation cohort, potentially competing for the occurrence of postdischarge falls and contributing to its lower rates in this sample.⁷ These disparities could, in part, be ascribed to varied patient trajectories, especially as our study uniquely targeted post-ED hospitalized patients.⁷ Besides nationality, this might be the reason why our cohort was defined by higher mean age, lower pre-ponderance of female sex and lower rates of independent living than the former one, which included only post-ED discharged patients.⁷

From an economic perspective, the cost implications of post-hospital falls are profound.²⁰ A retrospective cohort study in Hong Kong revealed that the annual health care costs related to post-hospital falls exceeded USD 26.5 million for older adults, with the average cost per faller and fall being USD 11,129 and USD 9596, respectively, considering 2018 exchange rate.²⁰ This immense economic burden and the sharp increase in fall incidence within the first 3 weeks postdischarge²⁰ underscores the pressing need for efficient interventions focused on the immediate period after hospital release. However, a recent multistakeholder scoping review by the Geriatric Emergency Care Applied Research (GEAR) conveyed an absence of consensus on ideal fall screening tools suitable for the ED.²¹ This poses challenges, as both American and European guidelines steadfastly advocate for fall risk screenings as foundational care standards within geriatric EDs.^11,22 Thus, identifying optimal assessment tools remains a critical research priority, and our results suggest that history of falls and inability to self-clip toenails should continue to be explored in future ED fall prevention research. Hopefully, machine learning will be increasingly used to automatically stratify fall risk after ED visits and refer high-risk patients to interventions such as multidisciplinary falls clinic, avoiding added burden to ED staff.²³

LIMITATIONS

Our results should be interpreted considering some limitations. The observational design precludes causality inferences, particularly considering the high rates of previous falls and inability to self-clip toenails in the group without falls during the 6-month follow-up. Moreover, considering the complex nature of falls in older populations, where multiple factors contribute to risk, our analysis was inherently limited by the PRO-AGE study’s sample size and the relatively infrequent occurrence of falls within the 6-month follow-up. This constraint might have affected the precision of our multivariate proportional hazards models’ findings. Future research involving larger cohorts is crucial to more comprehensively understand and elucidate other potential risk factors that could contribute to the performance of the Carpenter instrument.

Also, our analysis centered on a private tertiary hospital’s geriatric ED in São Paulo, potentially circumscribing the broader applicability of our findings, especially when extrapolated to diverse socioeconomic and cultural domains or EDs devoid of dedicated geriatricians. Indeed, sociodemographic variables differed between the groups of patients with 0, 1, and ≥2 points and, as this might have played a part in overall morbidity, accessibility to care and other aspects related to poor environment, we have adjusted our regression analyses taking these variables into consideration.

Finally, as ours is an external validation study, we chose to replicate the cutoff points suggested by the original study⁷ and the best systematic review on this topic to date.⁹ In the former study, the risk of falls increased from 4% among patients without risk factors to 9%, 14%, 24%, and 42% among those with one, two, three, and four risk factors, respectively. The later study concluded that a threshold of two or more abnormalities defines high risk of falls.⁹ All this considered, we felt that it would be premature to review the cutoff based solely on our findings, as this is the first external validation of this score. We hope that future studies will aid in this definition, particularly considering that in our sample the probability of having three or more risk factors was more than twice as big among fallers than among those without this outcome during our 6-month follow-up.

CONCLUSIONS

In our study, the Carpenter instrument and its components demonstrated an association with the 6-month postadmission fall risk among older adults post–ED visit. However, the tool’s accuracy for individual patient decision making in our cohort was limited. Identifying patients at increased risk for falls enables clinicians to implement specific interventions, allocate resources efficiently, and potentially mitigate future health care challenges. Future research should aim to refine these risk assessments, enabling the implementation of targeted proactive strategies to enhance patient care quality and improve long-term outcomes.