Operationalization and Validation of the Stopping Elderly Accidents, Deaths, and Injuries (STEADI) Fall Risk Algorithm in a Nationally Representative Sample

Abstract

Background

Preventing falls and fall-related injuries among older adults is a public health priority. The Stopping Elderly Accidents, Deaths, and Injuries (STEADI) tool was developed to promote fall risk screening and encourage coordination between clinical and community-based fall prevention resources; however, little is known about the tool’s predictive validity or adaptability to survey data.

Methods

Data from five annual rounds (2011–2015) of the National Health and Aging Trends Study (NHATS), a representative cohort of adults age 65 and older in the US. Analytic sample respondents (n=7,392) were categorized at baseline as having low, moderate, or high fall risk according to the STEADI algorithm adapted for use with NHATS data. Logistic mixed-effects regression was used to estimate the association between baseline fall risk and subsequent falls and mortality. Analyses incorporated complex sampling and weighting elements to permit inferences at a national level.

Results

Participants classified as having moderate and high fall risk had 2.62 (95% CI: 2.29, 2.99) and 4.76 (95% CI: 3.51, 6.47) times greater odds of falling during follow-up compared to those with low risk, respectively, controlling for sociodemographic and health related risk factors for falls. High fall risk was also associated with greater likelihood of falling multiple times annually but not with greater risk of mortality.

Conclusion

The adapted STEADI clinical fall risk screening tool is a valid measure for predicting future fall risk using survey cohort data. Further efforts to standardize screening for fall risk and to coordinate between clinical and community-based fall prevention initiatives are warranted.

INTRODUCTION

Approximately one third of adults aged 65 and older in the United States experience a fall each year, frequently resulting in injuries, functional impairments, and mobility restriction [1–4]. Falls are the leading cause of injury-related mortality in older adults [5]. The combined direct and indirect costs of fall-related injuries among older adults exceeded $50 billion in 2010 [5], and these costs are expected to grow substantially by 2040 [2, 6, 7]. Given the high incidence and costs of falls, fall prevention has become a national public health priority [8, 9]. Numerous clinical screening tools have been developed to assess potential fall risk in clinical settings [10–12], yet few studies have evaluated the predictive utility of such tools or adapted them for use with survey cohort data.

The Centers for Disease Control’s Stopping Elderly Accidents, Deaths, and Injuries (STEADI) clinical tool was designed to help health care providers integrate fall risk assessment into routine practice [13, 14]. STEADI comprises a number of resources for clinical providers, including guidelines for implementation, assessment, treatment, and referral. The STEADI toolkit includes an evidence-based algorithm to screen for individual fall risk during clinical visits, with suggestions for appropriate intervention at each risk level [13]. The fall risk algorithm was derived from population-based research on fall risk factors [14] and has been integrated into clinical practice and community-based fall prevention programs [15, 16]. However, few studies have sought to operationalize the STEADI fall risk algorithm within a survey data context or to evaluate the predictive validity of this measure over time in a large sample. Empirical evidence of the STEADI algorithm’s ability to predict future falls is needed to evaluate and compare the effectiveness of fall prevention interventions based on STEADI guidelines, to help inform population estimates of fall risk, and to facilitate coordination between clinical and community-based fall prevention resources.

The current study first aimed to operationalize the STEADI fall risk algorithm using data from an existing nationally-representative cohort of older adults in the US. Second, we aimed to evaluate the predictive validity of the adapted STEADI measure by estimating the association between STEADI fall risk classification and subsequent experience of falls and mortality. We hypothesized that the adapted STEADI measure would strongly and specifically predict fall occurrence, indicating its validity as a measure of fall risk and its potential for application in population-based research on falls and fall prevention.

METHODS

Sample

Data for this study were obtained from the National Health and Aging Trends Study (NHATS), a nationally representative survey of adults aged ≥ 65 funded by the National Institute on Aging (U01AG032947) and administered by the Johns Hopkins Bloomberg School of Public Health. Respondents to the NHATS survey are interviewed annually in-person and asked to report health, social, economic, and other characteristics. Objective measures of physical and cognitive functioning are collected at each annual interview. Sampling weights and stratification units are provided to account for the NHATS complex sampling design and to help generalize results to the broader population of older adults in the US. For the purposes of this analysis, five waves of data (baseline interview and four follow-up waves) were used from participants who entered the cohort during the 2011 baseline wave (n=8,245). Participants who were living in a nursing home or unspecified residential facility at baseline (n=636) and proxy participants with insufficient data on key study variables (n=217) were excluded from analyses, producing an analytic sample of 7,392 participants. As this was a secondary data analysis, this study was deemed exempt from the local institutional review board.

Measures

Outcome Measures

Primary analyses assessed the odds of incident falls across four follow-up waves. At each wave, participants were asked to report whether they had experienced a fall in the past year and, if so, whether they had experienced multiple falls during the past year. The primary analytical outcome was self-report of at least one fall. In secondary analyses, we considered two alternative outcomes: multiple falls and mortality.

STEADI Fall Risk

Fall risk was measured at baseline using an adaptation of the STEADI fall risk algorithm for use with NHATS survey variables (Figure 1).

Figure 1.

Adapted STEADI algorithm for determining fall risk level.

Briefly, the STEADI algorithm, described in detail elsewhere [13], includes a series of hierarchical questions and physical evaluations used to categorize individuals into low, moderate, or high fall risk groups. For the first stage of the adapted measure, participants were asked to report whether they: 1) experienced a fall in the previous year; 2) were worried about falling; 3) felt unsafe walking or standing. Participants who answered ‘no’ to all three questions were categorized as low fall risk. Among participants who answered ‘yes’ to at least one first stage question, the second stage considered results from tandem gait balance tests and chair stand tests to determine additional fall risk. Participants who successfully completed both four-stage balance tests (held each stage for >10 seconds) [17] and chair stand tests (more than five stands in 30 seconds or less) [18] were categorized as having moderate fall risk. Among participants who did not successfully complete objective balance and strength measures, participants who reported one or fewer falls in the preceding year without injury were categorized as having moderate fall risk, whereas participants who reported multiple falls or hip fracture were categorized as having high fall risk.

Covariates

Demographic characteristics

Demographic covariates included baseline age (five-year increments from 65 to 90 and 90+), race/ethnicity (white non-Hispanic, black non-Hispanic, Hispanic, other), sex, education (highest degree completed), and whether the participant lived alone. Health characteristics considered were smoking status (current, former, never), body mass index (kg/m^2) calculated from self-reported height and weight, and self-reported vision or hearing impairment despite use of corrective devices.

Medical conditions

Participants reported whether they had any of ten medical conditions or events in the previous year: heart attack, heart disease, hypertension, arthritis, osteoporosis, diabetes, lung disease, stroke, dementia, or cancer. We considered each condition individually and as a summary total measure (range: 0–10) to assess medical burden.

Functional impairment

Reliance on others for performing daily activities was assessed by self-report of functioning in three domains: household activities (doing laundry, preparing meals, shopping, medication management, banking or paying bills), mobility activities (going outside, moving around inside, and getting out of bed), and self-care activities (eating, bathing, dressing, and toileting) [19]. For each activity, participants were assigned to one of three categories: 1) having no difficulty, 2) having some difficulty, but able to perform activity without assistance, or 3) having some difficulty and requiring assistance. A summary score was created for each domain and for overall functional ability, with higher scores representing greater need for assistance (range: 0–24).

Frailty

As a comparative measure of fall risk, frailty was assessed using five criteria outlined in the phenotypic frailty definition [20, 21] – weakness, slowness, exhaustion, low physical activity, and shrinking. We operationalized frailty according to methods described in previous research using NHATS [19] and detailed in Supplementary Table 1. Paralleling fall risk categories, participants were categorized in three frailty categories: robust (no criteria), pre-frail (1 or 2 criteria), frail (3 or more criteria).

Analysis

Summary statistics of demographic and health-related covariates were generated and compared across STEADI fall risk categories, using chi-square and ANOVA tests for categorical and continuous variables, respectively.

We used logistic regression and mixed-effects longitudinal logistic regression models to estimate the association between STEADI fall risk categorization at baseline and subsequent falls. First, experience of at least one fall over any of the four follow-up waves (1=fell, 0=did not fall) was regressed on STEADI fall risk categories alone. Potential confounding factors were selected for adjusted models based on a 10% or greater difference between crude and adjusted estimates of the association between STEADI and falls [22]. Model-estimated probabilities of falling were compared with actual report of fall(s), to estimate metric properties of the STEADI algorithm (e.g. sensitivity, specificity). Receiver operating characteristics (ROC) curves were plotted and areas under the ROC curves (AUC) were computed to compare discriminant properties of models across the range of potential thresholds between low and high risk, with greater values of AUC indicating better overall discrimination.

Mixed-effects logistic regression models were used to estimate the strength of association between STEADI fall risk categories and falls (fell during previous year=1, did not fall during previous year=0) over four follow-up waves, accounting for intra-individual correlation between fall probabilities across waves. Models included participant-level random intercept terms with fixed-effect terms for model covariates. In secondary analyses, we evaluated the association between STEADI categorization and multiple falls and mortality using logistic mixed-effects models. Statistical analyses were performed using Stata statistical software, version 14.0 (StataCorp, College Station, TX). To permit generalization to the US older adult population, analyses incorporated NHATS sampling weights and complex sampling design elements using Stata survey commands.

RESULTS

According to STEADI fall risk categorization, an estimated 51.7% (95% CI: 50.4%, 53.1%) of US adults aged 65 and older had low fall risk, 38.4% (95% CI: 37.2%, 39.7%) had moderate fall risk, and 9.9% (95% CI: 9.3%, 10.4%) had high fall risk in 2011 (Table 1). Fall risk was significantly greater among older age groups, women, individuals with less educational attainment, and individuals with greater comorbidity. While 38.1% (95% CI: 35.8%, 40.5%) of individuals categorized as low fall risk reported falling in at least one follow-up wave, 66.2% (95% CI: 64.2%, 68.2%) and 81.0% (95% CI: 77.2%, 84.3%) of individuals categorized as moderate and high fall risk, respectively, reported falling in at least one wave. Approximately 67.5% of individuals categorized as having high fall risk reported falling multiple times in at least one follow-up wave.

Table 1.

Characteristics of weighted analytic sample by fall risk category

	Total	Low Risk	Moderate Risk	High Risk
	100%	51.8%	38.4%	9.9%
Characteristics	% or Mean (SE)
Age, yrs
65–69	28.4	33.7	24.9	14.5
70–74	25.2	28.6	22.4	18.1
75–79	19.3	18.2	20.1	18.5
80–84	14.6	11.7	16.6	22.1
85–89	8.6	5.7	10.3	17.1
90+	3.9	2.0	5.0	9.7
Female	56.4	50.0	62.6	65.6
Race and Ethnicity
Non-Hispanic White	81.1	81.1	81.4	80.2
Non-Hispanic Black	8.4	8.6	8.1	8.2
Other	3.6	4.2	3.1	2.4
Hispanic	7.0	6.1	7.5	9.3
Completed college degree	29.3	33.1	27.3	17.6
Lives alone	28.7	25.1	32.5	32.9
Smoker
Never	47.4	46.8	48.1	47.5
Former	44.0	44.4	43.2	45.2
Current	8.6	8.8	8.7	7.3
BMI (kg/m2), mean (SE)	27.6 (0.1)	27.3 (0.1)	28.1 (0.1)	27.5 (0.3)
Disability total, mean (SE)	3.8 (0.1)	2.5 (0.1)	4.4 (0.1)	8.2 (0.2)
Comorbidities, mean (SE)	2.5 (0.1)	2.0 (0.1)	2.8 (0.1)	3.7 (0.1)
Heart Disease	17.5	13.0	19.9	31.6
Hypertension	64.0	59.0	68.5	73.0
Arthritis	53.6	42.2	63.9	73.2
Osteoporosis	21.1	14.9	25.2	37.9
Diabetes	23.9	19.1	26.8	37.8
Lung Disease	15.5	11.4	18.1	27.2
Stroke	10.0	6.0	12.2	22.4
Dementia	4.1	1.4	5.1	14.6
Cancer	26.0	24.1	27.8	29.1
Incident falls
0 falls reported	47.2	61.9	33.6	19.2
1 fall reported	23.4	22.2	27.2	13.5
2+ falls reported	29.5	16.0	39.1	67.3

Note: All estimates incorporate NHATS sampling and weighting elements applied to the analytic sample (n=7,392)

Table 2 displays classification results based on logistic regression of reported falls on STEADI fall risk categories and sociodemographic characteristics. Approximately 65% of the analytic sample was correctly classified, in that a high or low probability of falling predicted subsequent experience or not of a fall. The sensitivity and specificity of the classification model were both 0.65.

Table 2.

Fall risk classification table based on logistic regression^a

Classified	Fell	Did not fall	Total
High riskb	30.3%	18.5%	48.9%
Low risk	16.4%	34.8%	51.1%
Total	46.7%	53.3%	100%
Sensitivity = .65	Specificity = .65	PPV = .62	NPV = .68

Notes:

^aPredicted probabilities from logistic regression of any fall over four follow-up waves on STEADI fall risk category and age, gender, race/ethnicity, and education.

^bPredicted probability of falling ≥ .5

PPV: positive predictive value; NPV: negative predictive value

Figure 2 displays receiver operating characteristics (ROC) curves comparing the discriminant properties of different logistic regression models of falls over follow-up. As indicated by area under the curve (AUC) values, a model including both STEADI fall risk categories and sociodemographic characteristics (AUC=0.668) discriminated between eventual fallers better than a model including sociodemographic characteristics alone (AUC=0.575) (test of differences in AUC: χ²=159.5, p<.001). A model including only STEADI risk categories (AUC=0.641) had similar discriminant properties to the model with both STEADI categories and sociodemographic characteristics.

Figure 2.

Receiver operating characteristic (ROC) curves comparing model prediction of any fall over the follow-up period.

AUC = area under the curve

Results from logistic mixed-effects regression models are displayed in Table 3. STEADI fall risk categories were strong predictors of subsequent fall experience. Individuals categorized as having moderate fall risk at baseline were 2.6 times as likely to experience a fall in the next four years as those with low risk (OR=2.62, 95% CI: 2.29, 2.99), while individuals categorized as having high fall risk were almost five times as likely to experience a fall compared to individuals with low risk (OR=4.76, 95% CI: 3.51, 6.47). STEADI fall risk categorization was a strong predictor of fall occurrence, even when adjusting for other known indicators of fall risk and general health such as frailty, comorbidity, and disability. In contrast, age and gender, strong independent predictors of fall likelihood (results not shown), were not significant predictors when controlling for STEADI categorization.

Table 3.

Mixed effects logistic regression models of falls and mortality

	Any fall	Multiple falls	Mortality
	OR (95% CI)	OR (95% CI)	OR (95% CI)
STEADI fall risk
Low	---	---	---
Moderate	2.62 (2.29, 2.99)	4.05 (3.38, 4.86)	0.95 (0.81, 1.12)
High	4.76 (3.51, 6.47)	13.7 (9.65, 19.4)	1.12 (0.84, 1.50)
Age, yrs
65–69	---	---	---
70–74	1.02 (0.86, 1.21)	0.79 (0.61, 1.02)	1.38 (0.96, 1.99)
75–79	1.08 (0.89, 1.31)	0.84 (0.63, 1.11)	1.85 (1.35, 2.53)
80–84	1.16 (0.98, 1.39)	0.80 (0.60, 1.07)	3.44 (2.39, 4.95)
85–89	1.14 (0.93, 1.41)	0.92 (0.66, 1.27)	4.98 (3.35, 7.41)
90+	1.25 (0.98, 1.60)	0.70 (0.49, 1.00)	9.55 (5.87, 15.5)
Sex
Male	---	---	---
Female	1.07 (0.94, 1.23)	0.68 (0.58, 0.81)	0.71 (0.60, 0.84)
Race/ethnicity
White non-Hispanic	---	---	---
Black non-Hispanic	0.55 (0.48, 0.63)	0.68 (0.57, 0.81)	1.01 (0.81, 1.26)
Other	0.72 (0.53, 0.98)	0.64 (0.38, 1.08)	0.73 (0.36, 1.47)
Hispanic	0.73 (0.49, 1.08)	1.00 (0.72, 1.39)	0.81 (0.60, 1.09)
Frailty
Robust	---	---	---
Prefrail	1.20 (1.05, 1.38)	1.58 (1.27, 1.98)	2.05 (1.63, 2.59)
Frail	1.25 (0.97, 1.60)	1.65 (1.21, 2.26)	3.18 (2.29, 4.42)
Chronic conditions (Total)	1.13 (1.08, 1.18)	1.10 (1.04, 1.16)	1.17 (1.11, 1.24)
Disability (Total)	1.02 (1.00, 1.04)	1.02 (1.00, 1.05)	1.12 (1.08, 1.16)
Time
Year 1	---	---	---
Year 2	1.14 (1.00, 1.30)	1.05 (0.90, 1.23)	1.67 (1.26, 2.20)
Year 3	1.22 (1.07, 1.40)	1.19 (1.02, 1.40)	2.07 (1.36, 3.16)
Year 4	1.16 (0.98, 1.37)	1.14 (0.96, 1.36)	2.01 (1.24, 3.24)

Similarly, STEADI risk categorization was a strong predictor of experiencing multiple falls during follow-up. As shown in Table 3, individuals categorized as having moderate and high fall risk were respectively 4.05 and 13.7 times as likely to report multiple falls compared to low risk individuals (OR_mod=4.05, 95% CI: 3.38, 4.86; OR_high: 13.7, 95% CI: 9.65, 19.4).

STEADI categorization did not significantly predict subsequent mortality during follow-up when adjusting for health-related and demographic characteristics. In contrast, frailty predicted significantly greater risk for mortality, as did a greater number of comorbid medical conditions, greater functional disability, greater age, and male gender.

DISCUSSION

We adapted and operationalized the STEADI fall risk algorithm using data from a large, existing, nationally-representative survey of older adults in the US. Consistent with the primary study hypothesis, individuals categorized as having moderate and high risk of falling, according to the adapted STEADI algorithm, were significantly more likely to experience a fall within four follow-up years and substantially more likely to experience multiple falls compared to low risk counterparts. Furthermore, fall risk categories uniquely predicted fall occurrence but not mortality, suggesting that risk captured by the STEADI algorithm is specific to falls, independent of general health or disability.

Several findings in the present study provide insights to inform STEADI-based interventions and may help expand the use of the STEADI tool to wider use in community samples. First, our results provide empirical evidence that the STEADI risk algorithm accurately measures fall risk among older adults and predicts future fall occurrence in survey data. When future falls were used as a predictive criterion, the STEADI measure demonstrated fair sensitivity and specificity in discriminating low- and high-risk individuals. This finding builds upon the apparent face validity of the algorithm [14] and is consistent with previous studies demonstrating feasibility of the STEADI algorithm as a screening tool in clinical contexts [16, 23, 24]. For example, Casey and colleagues demonstrated the feasibility of integrating the STEADI risk algorithm into electronic health records (EHR) of a primary care practice [23]. Fall risk screening rates increased over follow-up; however, future fall occurrence was not evaluated [23]. Although the accurate discrimination of high and low risk by the STEADI measure might be anticipated given the known risk factors contained in the algorithm, empirical evidence of the tool’s predictive validity helps ensure that interventions based on this risk assessment accurately target high risk individuals.

Second, our results indicate that the STEADI algorithm may be adapted to accurately assess fall risk using existing survey variables not collected in a clinical setting. The process of evaluating and mitigating fall risk often requires significant investment of time and resources from clinicians that may impede screening [7, 23, 25, 26]. Indeed, few older adults who require emergency medical evaluations following a fall receive clinical fall risk assessment or are referred to fall prevention resources [25]. Despite numerous fall prevention strategies and interventions [27–35], lack of coordination between clinical and community-based practice has prevented widespread implementation [25]. The adapted STEADI measure represents a simplified version of the clinically-based STEADI algorithm, containing a limited set of risk indicators frequently available in cohort studies and collected by lay individuals. The relatively modest sensitivity and specificity of the algorithm in predicting falls is thus tempered by simplicity of measurement and adaptability to survey data. Such condensed assessment tools may help overcome the significant costs and resources required for clinical fall risk screening and permit comparisons of fall risk among different populations. Consistency between clinical and community-based screening tools may also provide an important step in developing coordinated fall prevention efforts, connecting vulnerable patients with prevention resources.

Third, study results indicate that STEADI fall risk categories have specificity in predicting fall likelihood, independent of mortality risk and other health-related risk factors. Many fall prevention interventions and fall risk screening tools focus on older adult populations in hospitals, health care settings, or residential care facilities [32–35]. Because individuals in these settings are likely to have a greater degree of medical burden and disability, fall risk characterized in such samples may be conflated with poor health. In the present study, STEADI fall risk categories were strongly associated with fall occurrence and frequency, even when accounting for functional impairment, frailty, and medical comorbidity; however, unlike these other factors, fall risk categories were not significantly associated with mortality. These results suggest that the adapted algorithm captures underlying fall risk independent of general health. Because fall risk is frequently related to factors distinct from medical burden [36–38], the ability to recognize risk independent of general health is an important feature of assessment tools. This feature may be key to identifying remediable or modifiable causes of fall risk and for informing preventive strategies that do not rely on clinical treatment.

Study results should be interpreted considering potential limitations. First, to adapt the STEADI algorithm to existing survey data, some modifications and simplifications were necessary. For instance, NHATS respondents were not asked to complete a Timed Up & Go test, the measure recommended in STEADI guidelines, and thus strength and balance were assessed using optional chair-stand and balance tests. To the extent that modifications did not reflect the original intent of the algorithm, fall risk categories may have been subject to measurement error; however, the strong association between risk categories and subsequent fall occurrence suggests that the adapted algorithm nevertheless contained meaningful information for discriminating fall risk levels. Second, the influence of falls and underlying fall risk on study retention are unknown and may have led to differential loss-to-follow-up. However, secondary analyses indicated that mortality (the most common reason for loss-to-follow-up) was non-differential by baseline fall risk category. Third, we were unable to assess the accuracy of self-reported falls and their putative causes. Although self-report may lead to underestimation of falls [39], self-reported falls have been found to be acceptably accurate compared to clinical assessment [40]. Exclusion of participants with dementia from analyses reduced but did not eliminate potential for inaccurate recall of falls.

Despite potential limitations, to our knowledge, this study is among the first to provide empirical evidence of the predictive validity of STEADI fall risk assessment. Furthermore, the use of nationally-representative NHATS sample data and the longitudinal analytic design accounting for complex sampling elements allows inferences about fall risk to be generalized to older adults in the US.

Commonly limited to clinical settings, the operationalized STEADI fall risk algorithm demonstrated predictive validity for fall risk among a nationally-representative sample of older US adults. Further, the specificity of the adapted STEADI algorithm offered valuable information exclusive to fall risk, independent of traditional indicators of poor physical health. The predictive validity of the adapted STEADI fall risk algorithm indicates its potential for measuring fall risk in community settings and for informing population-based fall prevention initiatives.

KEY MESSAGES.

What is already known on this subject

Falls are a leading cause of injury and injury-related mortality among older adults in the US, yet preventive fall risk screening is lacking. Numerous evidence-based tools have been developed to screen for fall risk during clinical visits, including the Stopping Elderly Accidents, Deaths, and Injuries (STEADI) risk algorithm. STEADI was designed to promote referral to effective community-based fall prevention resources, yet it is unclear whether the STEADI screening algorithm is a valid measure or whether the tool can be adapted for use in a survey-based cohort study.

What this study adds

The STEADI algorithm, when adapted for use in a survey cohort, predicts fall occurrence over time. Survey respondents categorized as having high fall risk were nearly five times more likely to fall over a four-year follow-up period compared to respondents classified as low risk. The adapted STEADI measure is specific to fall risk, as it predicts fall occurrence independent of general physical health and risk of mortality.