Using clinical balance tests to assess fall risk among established unilateral lower limb prosthesis users: cutoff scores and associated validity indices

Abstract

Introduction:

Clinicians are routinely required to make decisions about fall risk among lower limb prosthesis (LLP) user. These decisions can be guided by standardized clinical balance tests, but require population- and test-specific cutoff scores and validity indices to categorize individuals as probable fallers or non-fallers on the basis of test performance. Despite the importance of cutoff scores and validity indices to clinical interpretation of clinical balance test scores, they are rarely reported for LLP users. In their absence, clinicians cannot use results from clinical balance tests to assess the likelihood of a fall by any one patient.

Objective:

Derive cutoff scores, and associated validity indices, for clinical balance tests administered to established unilateral LLP users.

Design:

Cross-sectional study.

Setting:

Outpatient clinic and research laboratory.

Participants:

Established ambulatory unilateral transtibial and transfemoral prosthesis users (n=40).

Intervention:

Not applicable.

Main Outcome Measure(s):

Optimal cutoff scores and related validity indices (i.e., area under the curve, sensitivity, specificity, likelihood ratios) were computed for five balance tests, the Activities-Specific Balance Confidence Scale (ABC), Timed Up and Go (TUG), Four Square Step Test (FSST), Berg Balance Scale (BBS), and Narrowing-Beam Walking Test (NBWT).

Results:

Cutoff scores were identified for the NBWT (≤.43/1.0), TUG (≥8.17 seconds), FSST (≥8.49 seconds), BBS (≤50.5/56), and ABC (≤80.2/100). Validity indices (i.e., area under the curve, sensitivity, specificity, and likelihood ratios) for the NBWT, TUG, and FSST had greater diagnostic accuracy and provided more information about the probability of a fall than those for the BBS or ABC.

Conclusion:

Performance above or below identified cutoff scores for the NBWT, FSST, and TUG provides information about potentially important shifts in the probability of falling among established unilateral LLP users. These results can serve as initial benchmarks to reduce uncertainty surrounding fall risk assessment in established unilateral LLP users, but require future prospective evaluation.

Introduction

Clinicians are routinely required to make important decisions about whether a patient may be at risk for an adverse condition or event. Diagnostic decisions such as these are ideally made using psychometrically sound clinical tests with which the probability of an event or condition occurring or existing can be estimated¹. Establishing population- and test-specific cutoff scores and validity indices is key to applying clinical tests in this manner. Cutoff scores are required to dichotomize continues scales and create benchmarks that categorize individuals as with or without the condition or event on the basis of a “positive” or “negative” test result². Validity indices (e.g., sensitivity, specificity, and likelihood ratios) serve to provide information on the probability of the event or condition. In the absence of cutoff scores and associated validity indices, clinical tests can be administered to evaluate change over time or differences between individuals, but cannot be used to assess the probability with which an adverse event or condition may occur.

Falls remain a frequent event that negatively affects the lives of a substantial portion of lower limb prosthesis (LLP) users^3,4. Over 50% of LLP users report falling at least once a year, with up to 39% reporting multiple falls a year^4–8. Falls among LLP users frequently lead to adverse health outcomes including injury^3,5,7,9,10, financial costs¹¹, reduced mobility^12,13, and diminished quality of life¹⁴. A major barrier to reducing falls among LLP users has been effective screening of those at risk^15,16. Central to this barrier is a scarcity of LLP user-specific cutoff scores and validity indices for contemporary clinical balance instruments such as the Berg Balance Scale (BBS)¹⁷, Timed Up and Go (TUG)¹⁸, Four Square Step Test (FSST)¹⁹, and Activities-specific Balance Confidence (ABC) scale²⁰. A number of studies have assessed the validity and reliability of these instruments among LLP users^{15,16,21–27}, but cutoff scores to infer fall risk have only been established for transtibial prosthesis users with limited prosthetic experience (i.e., 6 months of prosthetic use)²¹. Clinicians therefore, lack the necessary information to assess fall risk among individual patients within the larger LLP user population (i.e., established transtibial and transfemoral prosthesis users).

Two different faller classifications, either ≥1 fall^3,16,28,29 or ≥2 falls^3,15,21 have been used to assess the diagnostic accuracy and validity indices of performance-based clinical balance tests. Validity indices for one performance-based balance test, the Berg Balance Scale (BBS), increased when a classification of ≥ 2 falls was used in a prior study⁸. Adopting a ≥2 falls classification inherently creates a “faller” group that is likely to possess worse balance ability than if it also included individuals with 1 fall (i.e., ≥1 fall classification). As a result, it may be easier to identify fallers versus non-fallers, and possibly make it more difficult to identify people with moderate balance impairments who may be at risk for falls. Despite both faller classifications being used in research and clinical settings, no study of LLP users has systematically evaluated the influence of faller classification on validity indices associated with performance-based clinical balance tests commonly administered to established unilateral LLP users.

The objective of this study was therefore to establish cutoff scores, and associated validity indices, for clinical balance tests administered to established unilateral LLP users. In prior studies it was found that single-task tests like the FSST, TUG, or Narrowing Beam Walking Test (NBWT) demonstrated better discriminant validity than multi-item tests like the BBS, ABC, or the Locomotor Capabilities Index^16,21. Based on this prior research therefore, it was hypothesized that the validity indices (e.g., likelihood ratios) associated with cutoff scores for the FSST, TUG, and the NBWT would exceed those of the BBS or ABC scale. Confirmation of this hypothesis would offer further evidence that clinical tests like the NBWT, FSST, and TUG can be used to assess fall risk among established unilateral transtibial and transfemoral prosthesis users. Further, cutoff scores and related validity indices would provide clinicians with the information needed to appropriately interpret clinical balance test scores to assess fall risk in their patients who use LLPs. Owing to the variety of ways researchers and clinicians can classify individuals as fallers or non-fallers^3,15,16,21, a secondary objective of this study was to determine whether cutoff scores and associated validity indices differed across two common fall classifications (i.e., ≥ 1 fall versus ≥ 2 falls over the past 12 months). It was hypothesized that classifying fallers as ≥ 2 falls would increase validity indices⁸.

Methods

Study Design

A cross-sectional study was performed from July 2016 to May 2017. The STROBE (Strengthening The Reporting of Observational studies in Epidemiology) Statement guideline³⁰ was followed during the collection and reporting of study data. All data were stored and managed using a REDCap database hosted at XXXX³¹. Study protocols were reviewed and approved by a XXXX institutional review board. Study participants provided written informed consent prior to participation.

Participants

Lower limb prosthesis (LLP) users were recruited from local prosthetic clinics. Inclusion criteria included unilateral transtibial or transfemoral amputation due to traumatic, dysvascular, or oncologic causes; 18 years of age or older; one or more years of using a prosthesis (i.e., established users); ability to ambulate at least 10 feet without an upper extremity assistive device (e.g., cane), and use of a comfortable prosthesis (assessed with the Socket Comfort Scale)³². Participants with complications to their contralateral leg (e.g., joint replacement, arthritis, or wounds), amputation of a second limb, or a neurological or cardiovascular condition that limited the ability to complete the study protocol were excluded.

Procedures

Participants completed demographic and prosthetic-related characterization measures, as well as a retrospective falls survey. Wearing their preferred prosthesis-footwear combination, participants were administered five clinical balance tests. Five-minute rest periods were enforced between each balance test. Cutoff scores and validity indices were derived using recommended methods^1,33–35.

Measurements

Participant demographic and characterization measures

Age, height, weight, and sex were collected from study participants via self-report. Medicare Functional Classification Level (MFCL) (i.e., K-level) was determined by a certified prosthetist via interview and physical evaluation, while amputation-related information (i.e., level, etiology, and time since) were obtained via interview with a study investigator. Perceived mobility was assessed with the Prosthetic Limb Users Survey of Mobility (PLUS-M)³⁶. To ascertain the number of falls experienced by each participant over the past 12 months, participants were asked, “In the past year have you had any falls including a slip or a trip in which you inadvertently lost your balance and landed on the ground or lower level?”^37–40 Participants who reported falling in the past year were then asked to recall the number of falls in the past 12 months. To determine whether different faller classifications influence cutoff scores and related validity indices, data was analyzed using two faller classifications: ≥1 fall^3,16,28,29 and ≥2 falls^3,15,21 over the past 12 months.

Clinical balance tests

Five clinical balance instruments, the Activities-Specific Balance Confidence (ABC) Scale²⁰, the Timed Up and Go (TUG)¹⁸, the Four Square Step Test (FSST)¹⁹, the Berg Balance Scale (BBS)¹⁷, and the Narrowing-Beam Walking Test (NBWT)¹⁶ were administered and scored according to the developers’ instructions (Appendix 1). Each balance instrument has demonstrated acceptable levels of validity^{15,16,21–23} and/or reliability (i.e. ICC range .70 – .99)^{15,22,23,25–27,41} among LLP users.

Statistical Analysis

Clinical balance test scores were compared across participant subgroups defined by their reported fall history (i.e., 0 falls, 1 fall, or ≥ 2 falls), and within commonly adopted fall classifications (i.e., 0 falls versus ≥ 1 fall, or 0–1 falls vs. ≥ 2 falls) using parametric (i.e., ANOVA) or non-parametric equivalents (Kruskal-Wallis Test).

Receiver Operating Characteristic (ROC) curves were used to identify cutoff scores^33,42 and related validity indices (i.e., area under the curve, sensitivity, specificity, and likelihood ratios) for each clinical balance test. ROC curves were obtained by plotting the sensitivity of a test against 1-specificity. The area under the curve (AUC) represents the probability of correctly identifying a faller from a randomly selected pair of lower limb prosthesis users, one being a faller and the other a non-faller⁴³. The larger the AUC, the greater the test’s general discriminative ability⁴³. The AUC was selected as a summary measure of diagnostic accuracy⁴⁴. The AUC can assume any value between 0 and 1, with a value of .5 representing chance², and values greater than or equal to .8 recommended as the limit of clinical acceptability⁴⁵. Areas for each test were therefore compared to a threshold of .8 to determine the clinical acceptability of a test. While the AUC provides insight into the overall discriminant ability of a test based on group data, it does not provide clinicians with actionable information when assessing an individual patient. Cutoff scores and validity indices such as likelihood ratios are required to make individual-level decision based on test scores.

Optimal cutoff scores were chosen for each balance test by selecting the point on a test’s ROC curve closest to the (0,1) point⁴⁶. This point was identified by choosing the minimal value of the function (1-sensitivity) ² + (1-specificity)², a method that maximizes sensitivity and specificity^46,47. The test score corresponding to this minimal value was therefore selected as the cutoff score that best discriminates between unilateral LLP users with and without a history of falls². Values for sensitivity (i.e., how often a diagnostic test is positive when a condition is present), and specificity (i.e., how often a diagnostic test is negative in the absence of the condition) at that point were recorded.

One issue with sensitivity and specificity indices is that they have limited clinical utility⁴⁸. Sensitivity, for example, does not describe how often patients with positive tests have the condition of interest (i.e., a fall). Sensitivity indices therefore do not indicate the probability of the event or condition occurring⁴⁹, information that is likely to be of greatest value to clinicians¹. Predictive values alternatively offer probabilities of an event occurring, but are dependent on the prevalence of the event in the study sample and thus rarely generalize beyond the study⁵⁰. Likelihood ratios overcome these aforementioned limitations by quantifying how much the obtained test score increases or decreases the probability of an event occurring^1,35, independent of its prevalence in the sample (i.e., they generalize beyond the study). Likelihood ratios are therefore considered more efficient and clinically useful than sensitivity and specificity values or positive and negative predictive values^34,35,51,52. Unlike the AUC, likelihood ratios provide clinicians with actionable information for individual patients (i.e., the probability a patient will or will not fall based on a positive or negative test outcome). Likelihood ratios were therefore selected as the primary validity index in this study. The likelihood ratio for a positive test (LR+) was computed as sensitivity/1-specificity, while the likelihood ratio for a negative test (LR-) was computed as 1-specificity/sensitivity⁴⁹. Likelihood ratios greater than 5 or less than 0.2 indicate moderate changes in the probability of an event (e.g., fall), while likelihood ratios between 5 and 2, or 0.5 and 0.2 result in small, but potentially important shifts in probability¹. Likelihood ratios greater than 2 or less than 0.2 were considered the minimum for considering a clinical test able to contribute to a fall risk assessment.

A multivariate logistic regression model was developed for each faller classification (i.e., ≥ 1 fall = Model 1, ≥ 2 falls = Model 2) to determine whether a combination of clinical balance test scores and other fall-related variables could enhance validity indices beyond the score from any single test. Univariate logistic regression was first performed with each potential predictor variable (e.g., FSST time, age, sex, etiology, level of amputation, mobility level) as the independent variable, and faller status as the dependent variable. Independent variables found to have an odds ratio of ≥2 or <0.5, and a p-value of <0.05 were retained for further consideration⁵³. To avoid multicollinearity, any of the independent variables retained from the univariate logistic regression that had a strong correlation (i.e., Spearman rho ≥ .75) and a lower odds ratio than the other independent variable with which it was correlated were excluded from the multivariate logistic regression model⁵³. A multivariate logistic regression was then performed with all remaining independent variables included. Validity indices described above were computed from the regression output. Cutoff scores and validity indices were computed for each of the faller classifications (i.e., ≥1 fall and ≥ 2 falls). 95% confidence intervals (CI) were computed to determine the precision of each validity index⁵². All statistical analyses were conducted using SPSS v.25 (Chicago, IL).

Results

The sample included 40 established unilateral lower limb prosthesis (LLP) users (height: 173 ±9.10cm; weight: 78.5 ±14.1kg; SCS: 7.8 ±1.4) (Table 1). All participants were able to perform and compete all tests. Scores for each of the studied tests, stratified by participant fall history and fall classification are presented in Table 2 and Table 3 respectively. Pairwise comparisons revealed that with the exception of the ABC, participants with a history of 2 or more falls had significantly worse test scores than those with a history of either one fall or no falls (p < .027 to .004) (Table 2). There were no statistically significant differences in test scores between participants who reported one and no falls. When a fall classification scheme of 2 or more versus 0–1 falls in the past 12 months was implemented, pairwise comparisons revealed that again with the exception of the ABC, participants reporting 2 or more falls had worse test scores than those reporting 0–1 falls (Table 3). With the exception of the NBWT, these differences in test scores between groups were not observed when a fall classification scheme of 1 or more versus zero falls was implemented (Table 3).

Table 1.

Participant demographics stratified by faller status

Fall Group	Age (yrs) mean (SD)	Sex	Years since Amputationmean (SD)	Amputation Level	Amputation Etiology	PLUS-M mean (SD)	MFCL
No falls	44.9 (14.4)	M (11) F (5)	16.4 (12.8)	TT (14) TF (2)	Dysvascular (1) Non-dysvascular (15)	58.5 (9.05)	K1(0) K2(4) K3(10) K4(2)
1 fall	41.4 (16.3)	M (2) F (5)	10.8 (4.8)	TT (3) TF (4)	Dysvascular (0) Non-dysvascular (7)	56.2 (6.35)	K1(0) K2(1) K3(2) K4(4)
≥ 2 falls	55.4 (11.7)	M (8) F (9)	13.8 (14.8)	TT (8) TF (9)	Dysvascular (6) Non-dysvascular (11)	49.2 (6.71)	K1(2) K2(10) K3(5) K4(0)
Total	48.7 (14.6)	M (21) F (19)	14.3 (12.6)	TT (25) TF (15)	Dysvascular (7) Non-dysvascular (33)	54.1 (8.68)	K1(2) K2(15) K3(17) K4(6)

PLUS-M: Prosthetic Limb User’s Survey of Mobility; TF: Transfemoral; TT: Transtibial; M: Male; F: Female; Other: trauma, cancer, infection; MFCL: Medicare Functional Classification Level

Table 2.

Clinical balance test scores stratified by fall history

Fall Status	NBWT (/1.0) Mean (SD) Range	TUG (sec) Median (IQR) Range	FSST (sec) Median (IQR) Range	BBS (/56) Median (IQR) Range	ABC (/100) Median (IQR) Range
No falls (n=16)	.51 (.13) .27–.76	7.86 (2.83) 5.99–13.0	7.56 (4.87) 5.26–12.8	52.0 (8.50) 40.0–55.0	90.3 (18.9) 63.1–100
1 fall (n=7)	.53 (.16) .25–.74	7.45 (3.04) 5.06–9.18	6.57 (3.08) 4.14–11.8	52.0 (6.00) 50.0–56.0	90.6 (14.1) 53.1–99.2
≥ 2 falls (n=17)	.25 (.16)# .04–.63	10.0 (5.89)* 8.30–19.0	13.8 (6.76)* 6.27–39.3	45.0 (12.5)* 16.0–55.0	74 (26.9) 40.9–96.0
Total (n=40)	.41 (.19) .04–.76	8.75 (3.97) 5.06–19.0	8.83 (5.94) 4.14–39.3	50.0 (8.50) 16.0–56.0	85.0 (23.7) 40.9–100

NBWT: Narrowing Beam Walking Test; TUG: Timed Up and Go; FSST: Four Square Step Test; BBS: Berg Balance Scale;

ABC: Activities-specific Balance Confidence Scale; SD: standard deviation; IQR: interquartile range

^#Test scores significantly different between ≥ 2 falls and 1 fall, as well as ≥ 2 falls and no falls p < .004 (One-way ANOVA)

^*Test scores significantly different between ≥ 2 falls and 1 fall, as well as ≥ 2 falls and no falls p < .027 (Kruskal-Wallis Test of medians)

Table 3.

Clinical balance test scores stratified by two faller classifications

≥ 1 falls (any falls)
Fall Status	NBWT (/1.0) Mean (SD) Range	TUG (sec) Median (IQR) Range	FSST (sec) Median (IQR) Range	BBS (/56) Median (IQR) Range	ABC (/100) Median (IQR) Range
No falls (n=16)	.51 (.13) .27−.76	7.86 (2.83) 5.99−13.0	7.56 (4.87) 5.26−12.8	52.0 (8.50) 40.0−55.0	90.3 (18.9) 63.1−100
≥ 1 falls (n=24)	.33 (.20)# .04−.76	9.39 (5.87) 5.99−13.0	11.4 (8.54) 4.14 −39.3	48.5 (10.8) 16.0−56.0	81.8 (24.9) 40.9 −99.1
≥ 2 falls (multiple falls)
Fall Status	NBWT (/1.0) Mean (SD) Range	TUG (sec) Median (IQR) Range	FSST (sec) Median (IQR) Range	BBS (/56) Median (IQR) Range	ABC (/100) Median (IQR) Range
0–1 falls (n=23)	.51 (.14) .25−.76	7.79 (1.66) 5.06–13.0	7.46 (3.75) 4.14–12.8	52.0 (7.00) 40.0−56.0	90.6 (15.6) 53.1−100
≥ 2 falls (n=17)	.25 (.16)# .04–.63	10.0 (5.89)* 8.30–19.0	13.8 (6.76)* 6.27–39.3	45.0 (12.5)* 16.0−56.0	74 (26.9) 40.9−96.0

NBWT: Narrowing Beam Walking Test; TUG: Timed Up and Go; FSST: Four Square Step Test; BBS: Berg Balance Scale;

ABC: Activities-specific Balance Confidence Scale; SD: standard deviation; IQR: interquartile range

^#Test scores significantly different between No falls and ≥ 1 falls, as well as ≥ 2 falls and 0–1 falls p ≤ .005 (2-sided Independent T-Test)

^*Test scores significantly different between ≥ 2 falls and 0–1 fall p ≤ .001(Mann Whitney Test of medians)

Cutoff scores and validity indices (estimate, 95% CI) for each clinical balance test are presented in Table 4. When fallers were defined as individuals reporting ≥1 fall in the 12 months prior to assessment, the NBWT had the greatest area under the ROC curve (AUC) (.81, .62–.91), as well as the largest specificity (76%, 54–96%) and positive likelihood ratio (3.0, 1.5–6.9). The TUG had the greatest sensitivity (83%, 68–98%), and smallest negative likelihood ratio (.24, .13–.56). The logistic regression model developed for the ≥1 fall classification (i.e., Model 1) included the NBWT, FSST, PLUS-M, and amputation level as predictor variables. Many, but not all, validity indices improved slightly compared to individual clinical balance test scores alone (Table 4). Other candidate predictor variables (e.g., TUG, ABC, MFCL, amputation etiology, age and sex) failed to meet model inclusion criteria (i.e., odds ratio ≥ 2 or less than 0.5, and p < 0.05), during univariate logistic regression.

Table 4.

Validity indices for performance-based clinical balance tests among unilateral lower limb prosthesis users for two faller classifications

≥ 1 falls (any falls)
Test	AUC (95% CI)	Cutoff Score	Sensitivity (95% CI)	Specificity (95% CI)	LR+ (95% CI)	LR− (95% CI)
NBWT (/1.00)	.81 (.62–.91)	≤ 0.43	73% (53%–90%)	76% (54%–96%)	3.0 (1.5–6.9)	.36 (.19–.77)
TUG (sec)	.71 (.54–.88)	≥ 8.17	83% (68%–98%)	68% (46%–92%)	2.6 (1.3–5.6)	.24 (.13–.56)
FSST (sec)	.70 (.53–.86)	≥ 8.49	74% (58%–92%)	68% (46%–92%)	2.4 (1.1–5.2)	.36 (.17–.78)
BBS (/56)	.66 (.47–.83)	≤ 50.5	67% (48%–86%)	62% (39%–86%)	1.8 (.89–3.6)	.53 (.27–1.1)
ABC (/100)	.65 (.47–.82)	≤ 80.2	50% (30%–70%)	74% (54%–96%)	1.9 (.78–5.1)	.67 (.41–1.1)
Model 1	.84 (.71–.96)	N/A	80% (64%−96%)	73% (51%–96%)	3.0 (1.3–7.1)	.27 (.12–.63)
≥ 2 falls (multiple falls)
Test	AUC (95% CI)	Cutoff Score	Sensitivity (95% CI)	Specificity (95% CI)	LR+ (95% CI)	LR− (95% CI)
NBWT (/1.00)	.89 (.78–1.0)	≤ 0.43	88% (73%–100%)	79% (65%–92%)	4.2 (1.7–6.9)	.16 (.042–.60)
TUG (sec)	.88 (.78–.98)	≥ 9.25	82% (64%–100%)	83% (67%–98%)	4.7 (1.9–11.9)	.21 (.075–.61)
FSST (sec)	.88 (.76–.99)	≥ 8.71	94% (83%–100%)	74% (60%–92%)	3.6 (1.8–7.3)	.08 (.012–.54)
BBS (/56)	.81 (.66–.92)	≤ 50.5	88% (73%–100%)	70% (51%–88%)	2.9 (1.5–5.5)	.17 (.045–.64)
ABC (/100)	.71 (.54–.87)	≤ 80.2	65% (42%–87%)	78% (61%–95%)	3.0 (1.3–7.0)	.45 (.23–.89)
Model 2	.90 (.79–1.0)	N/A	76% (56%–97%)	83% (67%–98%)	4.4 (1.7–11.3)	.28 (.12–.68)

ABC: Activities-specific Balance Confidence Scale; AUC: Area Under the Curve; BBS: Berg Balance Scale; CI: Confidence Interval; FSST: Four Square Step test; LR-: negative likelihood ratio; LR+: positive likelihood ratio; NBWT: Narrowing Beam Walking Test; TuG: Timed Up and Go.

Model 1: NBWT; FSST; PLUS-M; amputation level

Model 2: NBWT; TUG; PLUS-M; K-level; amputation etiology

When fallers were defined as individuals reporting ≥2 falls in the 12 months prior to assessment validity indices (i.e., sensitivity, specificity, and likelihood ratios) associated with each of the clinical balance tests generally improved (Table 4). The NBWT again had the largest AUC (0.89, .78–1.0). The FSST, however, exhibited the largest sensitivity (94%, 83–100%) and smallest negative likelihood ratio (.08, .012–.54), while the TUG had the greatest specificity (83%, 67–98%) and positive likelihood ratio (4.7, 1.9–11.9). The logistic regression model developed for the ≥2 falls classification (i.e., Model 2) included several predictor variables consistent with those included in Model 1 (i.e., NBWT, PLUS-M), and others that were unique to Model 2 (i.e., TUG, MFCL, and amputation etiology). Model 2 did not improve validity indices as markedly as Model 1 did (Table 2). Model 2 was not among the top three results for several of the validity indices (i.e., sensitivity and negative likelihood ratio).

Discussion

The objective of this study was to establish cutoff scores and associated validity indices for several clinical balance tests administered to established unilateral lower limb prosthesis (LLP) users. Results supported the hypothesis that validity indices, including likelihood ratios, of the NBWT, FSST, and TUG exceeded those for the BBS or ABC scale. Performance above or below identified cutoff scores for the NBWT, FSST, and TUG appear to provide information about small but potentially important shifts in the probability of falling among unilateral LLP users. The distribution of demographic (e.g., age), amputation (e.g., level, etiology, and time since), and activity characteristics (e.g., PLUS-M, MFCL) of participants in the present study were generally comparable to those reported in large national studies of people with lower limb amputation (n=210 to 1568)^24,54–58, although skewed slightly towards a higher percentage of individuals with non-dysvascular and transtibial amputation. The overall similarity, however, with these samples suggests that the current results can be generalized to the broader population of established unilateral LLP users. Although these results can serve as initial benchmarks to reduce uncertainty surrounding the assessment of fall risk in established unilateral LLP users, clinicians and researchers should also consider the reliability and ease of use of these instruments when deciding whether to adopt them in their clinical or academic practices. Additionally, prospective evaluation of the cutoff scores and their validity indices will be required in future studies.

This study provides the first set of cutoff scores and validity indices for performance-based clinical balance tests based on data collected from established unilateral transtibial and transfemoral prosthesis users. The current results build on the work of Dite et al., (2007) who established cutoff scores and validity indices for several clinical balance tests (e.g., FSST, TUG) using data from a sample of short-term (i.e., 6 months post-discharge) transtibial prosthesis users (Dite, 2007). Notable differences were observed between the cutoff scores in the present study and those reported by Dite et al., (2007). Dite et al., (2007), used a faller classification of ≥ 2 falls and reported cutoffs of 19 and 24 seconds for the TUG and FSST, respectively. Cutoff times of 9.25 and 8.71 seconds were identified for the TUG and FSST in the present study. Times were even lower (i.e., 8.17 and 8.49 seconds, respectively) if the “≥ 1 fall” classification was used. The discrepancies in cutoff times noted between the two studies may be attributed to the study samples. The present study included both transtibial and transfemoral prosthesis users, while Dite et al., (2007) included only transtibial prosthesis users. However, the higher prevalence⁶ and risk of falls⁵⁹ among transfemoral prosthesis users would be expected to increase cutoff times for the FSST and TUG in the present study relative to the prior study, not decrease them. Also, previous studies have not reported statistically significant differences in TUG^16,22 or FSST¹⁶ times between transtibial and transfemoral prosthesis users. Thus, inclusion of transtibial and transfemoral prosthesis users in the present study cannot explain the observed differences in cutoff scores. The observed differences in cutoff times are more likely to be attributed to differences in time since amputation. Dite et al., (2007) studied participants less than a year post amputation (i.e., 6.4 ± 1.5 months after discharge from the rehabilitation unit), while participants in the current study averaged 14.3 years since amputation. This difference implies that distinct cutoff scores may be required to evaluate fall risk at different times post amputation, and that balance ability and fall risk may change markedly after six months of prosthesis use. Longitudinal changes of balance and fall risk may be an important area of future research. The results of these two studies should therefore be considered complimentary rather than conflicting, serving two temporally distinct groups; short-term versus established LLP users.

Existing clinical balance tests appear to provide important information about the probability of falls in established unilateral LLP users. Although none of the likelihood ratios for the clinical balance tests in the present study were sufficient to indicate large and conclusive changes in the probability of a fall event, three of the five clinical balance tests (i.e., NBWT, TUG, FSST) had likelihood ratios that would imply small shifts in the probability of a fall given a positive or negative test (Table 4)⁵⁵. For example, a score equal to or greater than 0.43 on the NBWT would suggest that a LLP user is three times more likely (i.e., LR+ = 3.0) of being a faller than a non-faller. Likelihood ratios for the NBWT, TUG, and FSST were accompanied by 95% confidence intervals that did not overlap with 1.0 (i.e., no change in probability) (Table 4), suggesting that the interpretation of an increase or decrease in the probability of being a faller based on a positive or negative test result can be made with a reasonable level of confidence¹. While the increased likelihood of a fall associated with a positive test on the NBWT, TUG, and FSST relative to the BBS and ABC may be small, it may still be clinically important given the consequences of falls among LLP users^3,5,7,9,11. Although the LR were greater in the NBWT, TUG, and FSST, in many cases their confidence intervals overlapped. However, the application of the indices derived in this study in a larger prospective study is needed to more definitively determine the impact of these differences.

Notably, fall risk assessment models that included multiple tests and factors associated with fall risk (i.e., etiology and level of amputation)^8,59 failed to improve likelihood ratios among LLP users compared to scores on the individual balance tests (Table 4). No additional information would therefore appear to be gained regarding the probability of being a faller among LLP users by combining scores from multiple balance tests with the other fall-related demographic or amputation information considered in this study. This suggests that amputation-related factors, including level or etiology of amputation, may provide less information regarding fall risk among established unilateral LLP users than has been historically considered. Additional research is required to verify this result in a prospective study, and to consider other falls-related demographic and amputation-specific information. As a result, clinicians may be best served by assessing fall risk in established unilateral LLP users by administering, scoring, and interpreting performance on a single balance test that possesses sufficient psychometric rigor, is practical for the given setting, and meets the application needs of the clinician (i.e., discriminate, evaluate, or predict). Examples of such applications may include discriminating between fallers and non-fallers in an observational study, evaluating changes in balance ability pre-post therapy, and predicting fall risk to justify the prescription of prosthetic componentry.

Results of this study also indicate that the classification used to categorize participants as fallers or non-fallers can alter the validity indices of the studied clinical balance tests. Validity indices, such as the AUC, generally improved when fallers were classified as individuals reporting ≥ 2 falls. The observed increase in validity indices based on a more conservative faller classification (i.e., ≥ 2 falls vs. ≥ 1 fall) is consistent with previous research⁸. However, it seems important to note that limiting the classification of fallers to those with a history of multiple falls inherently creates a group of individuals with worse balance ability than a group that includes those who have fallen just once in the prior 12 months (Table 3). Adopting a more conservative ≥ 2 falls classification magnifies differences between fallers and non-fallers, increases the magnitude of validity indices, and improves each test’s ability to identify those at risk for additional falls. It does so however, at the cost of being able to identify those at risk for a fall after experiencing only one fall in the prior 12 months, potentially overlooking those with modest balance impairments. The observed differences also suggest that studies reporting validity indices for clinical balance tests based on different faller classifications may not be directly comparable. For this reason, we recommend that investigators report cutoff scores and associated validity indices for both single and multiple fallers. Similarly, if a test’s validity indices are derived using one classification, test administrators should apply the same classification when applying the test clinically to assess fall risk. Cutoff scores for the FSST and TUG increased when fallers were defined as LLP users reporting multiple falls (i.e., ≥ 2 falls), compared to users reporting one or more falls. This indicates that, when using these tests, administrators should use fall classification-specific cutoff scores (Table 4). In contrast, cutoff scores for the NBWT, BBS, and ABC did not change with how fallers were classified. This indicates that for these tests, administrators can use the same cutoff score irrespective of how they define a faller. Having a single cutoff score may simplify and reduce the burden of scoring and interpreting test performance, a reported restriction to the adoption of balance tests among clinicians^60–64. Whether cutoff scores could be established to differentiate LLP users experiencing no falls, a single, or multiple falls remains to be determined.

Study Limitations

A number of limitations with this study need to be considered when interpreting the results. First, the sample size, while consistent with similar studies^15,21,65, should be increased in future research. A larger sample would facilitate development of cutoff scores to discriminate non-fallers (i.e., zero falls), from single fallers (i.e., 1 fall), and multiple fallers (i.e., ≥ 2 falls), as well as across score intervals for balance tests with a continuous scale (e.g., the NBWT, TUG, and FSST)¹. The study sample also consisted of a larger percentage of traumatic LLP users (i.e., 62.5%) than is reported in the literature (i.e., 17%−60.2%)^24,54–58, potentially limiting the generalization of study results. While this is a challenge in most^{15,23,25,66,67}, but not all^8,21,68–70 research involving LLP users, future efforts to include a larger proportion of dysvascular LLP users is required to increase generalization of study results. The mean age of our sample, 48.7, was slightly younger than that reported in larger, national studies of individuals with lower limb amputation (i.e., mean age from 50 to 55) ^54–56,58, yet the range of ages studied, 24 to 70, is consistent with those prior studies. A follow-up study that focuses on older LLP users (e.g., age ≥ 65) may be warranted to examine balance test cutoff scores in LLP users in that specific subpopulation (i.e., Medicare-eligible individuals).

Falls were assessed retrospectively. This may underestimate fall frequency⁷¹, and lend itself to recall bias^72,73. A prospective study is needed to validate the cutoff scores and validity indices established in this cross-sectional study, and establish a temporal relationship between balance and fall status⁷⁴.

Fall-related injuries were not recorded. Determining whether multiple versus non-multiple fallers are more likely to suffer a fall-related injury^3,10, or if existing clinical tests can discriminate between or predict the probability of falls that result in injuries may be an important consideration in future studies.

Only a limited set of sociodemographic, health, and prosthetic-related factors were recorded and included in the multivariate models. Including other known risk factors for falls among lower limb prosthesis users such as strength⁷⁵, protective stepping⁷⁶, number of medications⁹, and sense of vibration⁷⁷ may improve model performance in future research.

Finally, psychometric properties, including the validity indices reported here, are population specific. The specific indices presented here therefore do not apply to other patient populations.

Conclusion

The primary objective of this study was to establish cutoff scores and associated validity indices for several clinical balance tests that may be administered to established unilateral lower limb prosthesis (LLP) users. Given the limited options available to quantitatively assess fall risk among established unilateral LLP users, the proposed cutoff scores and associated likelihood ratios for the NBWT, TUG, and FSST provide clinicians with tools to reduce the uncertainty associated with estimating the probability of a fall among established unilateral LLP users. More studies establishing and assessing the accuracy of cutoff scores for diagnostic tests like these to predict outcomes among LLP users are urgently needed. Additional research to evaluate the relative reliability, utility, and prospective validity (i.e., testing the cutoff scores and indices proposed here) of these tests is needed to facilitate their widespread adoption in clinical care.

Abbreviations:

ABC

Activities-specific Balance Confidence scale

AUC

area under the curve

BBS

Berg Balance Scale

CI

confidence intervals

FSST

Four Square Step Test

LR

likelihood ratio

MFCL

Medicare Functional Classification Level

NBWT

Narrowing Beam Walking Test

PLUS-M

Prosthetic Limb Users Survey of Mobility

ROC

receiver operating characteristic

STROBE

Strengthening The Reporting of Observational studies in Epidemiology

TUG

Timed Up and Go