Validation of the Narrowing Beam Walking Test in lower limb prosthesis users

Abstract

Objective

To evaluate the content, construct, and discriminant validity of the Narrowing Beam Walking Test (NBWT), a performance-based balance test for lower limb prosthesis users.

Design

Cross-sectional study.

Setting

Research laboratory and prosthetics clinic.

Participants

Forty unilateral transtibial and transfemoral prosthesis users.

Intervention

Not applicable.

Main Outcome Measure(s)

Content validity was examined by quantifying the percentage of participants receiving maximum or minimum scores (i.e. ceiling and floor effects). Convergent construct validity was examined using correlations between participants’ NBWT scores and scores or times on existing clinical balance tests regularly administered to lower limb prosthesis users. Known-groups construct validity was examined by comparing NBWT scores between groups of participants with different fall histories, amputation levels, amputation etiologies, and functional levels. Discriminant validity was evaluated by analyzing the area under each test’s Receiver Operating Characteristic (ROC) curve.

Results

No minimum or maximum scores were recorded on the NBWT. NBWT scores demonstrated strong correlations (|r_s|=.70‒.85) with scores/times on performance-based balance tests (Timed up and Go, Four Square Step Test, and Berg Balance Scale), and a moderate correlation (|r_s|=.49) with the self-report Activities-specific Balance Confidence scale. NBWT performance was significantly lower among participants with a history of falls (p=.003), transfemoral amputation (p=.011), and a lower mobility level (p<.001). The NBWT also had the largest area under the ROC curve (.81), and was the only test to exhibit an area that was statistically significantly greater than 0.50 (i.e. chance).

Conclusion

The results provide strong evidence of content, construct, and discriminant validity for the NBWT as a performance-based test of balance ability. The evidence supports its use to assess balance impairments and fall risk in unilateral transtibial and transfemoral prosthesis users.

Introduction

Falls are one of the most common, costly, and difficult to address health care concerns facing lower limb prosthesis (LLP) users^1,2. Over half of community-living LLP users report having fallen in the past year^3–5, many more than once^4,6. Among LLP users, falls result in activity limitations and participation restrictions^3,7–9, injuries^1,3,10–12, medical attention³, and reduced quality of life¹³. Improving balance and mitigating fall risk are therefore principal goals of rehabilitation^12,14,15 and motivation for prosthetic designs^16,17.

A key challenge for clinicians and researchers is identifying which LLP users are at risk for a fall. Effective screening remains a major barrier because contemporary clinical balance tests are often too easy for LLP users^18–20. Existing tests do not generally pose sufficient physical challenge to expose subtle, but meaningful differences in balance that underlie fall risk¹⁹. As such, they are likely to be limited in their ability to detect fall risk. Further, tests used to assess balance in other clinical populations often exhibit ceiling effects or lack sensitivity in LLP users^19,21,22. For example, the Berg Balance Scale is unable to discriminate between LLP users with and without a history of multiple falls¹⁹, and the Activities-specific Balance Confidence scale cannot distinguish between transtibial and transfemoral LLP users²¹. A more challenging test like the Four Square Step Test²³, which has been shown to discriminate between LLP users with and without a history of falls, has only been validated among new (i.e., not established) LLP users²⁴ (Dite, 2007). There is therefore a need for clinical tests that pose greater challenge to balance control^19,20.

Beam walking is an appealing paradigm for challenging balance control as task difficulty can be easily manipulated (i.e. decreasing beam width increases challenge), and performance can be easily evaluated (i.e. an individual is either on or off the beam). In prior studies, beam walking was shown to measure differences in balance ability between controls and lower limb prosthesis users²⁰, older adults²⁵, and professional ballet dancers²⁰. A standardized Narrowing Beam Walking Test (NBWT) was recently proposed as a way to create a single test condition capable of challenging individuals with a wide range of balance abilities²⁶. The developers proposed a brief administration and scoring procedure that required only five trials to accurately estimate a LLP user’s stable (i.e., typical) performance. While the NBWT appears to be clinically feasible and has the potential to address practical and psychometric limitations of existing clinical balance tests, validation testing is required prior to recommending it to assess balance ability in LLP users.

The objective of this study was therefore to assess the psychometric performance of the NBWT in LLP users. It was hypothesized that performance on the NBWT would exhibit evidence of: 1) content validity (i.e. be free of ceiling and floor effects), 2) convergent construct validity (i.e. be strongly correlated with existing performance-based balance tests and moderately correlated with self-report balance tests), 3) known-groups construct validity (i.e. demonstrate differences between LLP users with clinically-identified differences in balance), and 4) discriminant validity (i.e., be able to discriminate between LLP users with and without a history of falls). Confirmation of these hypotheses will provide evidence that the NBWT can be used to accurately measure balance ability and fall risk in LLP users.

Methods

Study Design

A cross-sectional study was conducted between July 2016 and May 2017 to assess the content validity, convergent and known-groups construct validity, and discriminant validity of the NBWT²⁶ in LLP users. Study protocols were reviewed and approved by a University of Illinois at Chicago institutional review board. All individuals provided written consent prior to participation.

Participants

LLP users were recruited from local prosthetic clinics. Inclusion criteria were 18 years of age or greater; unilateral, transtibial or transfemoral amputation; at least one year of experience using a prosthesis; amputation due to trauma, dysvascular complications, tumor, or infection; use of a comfortable prosthesis to ambulate; and able to walk 10 feet over level terrain without an assistive device. Exclusion criteria included amputation of another limb; contralateral leg complications (e.g., knee replacement, ulcers); an advanced neurologic disorder; or severe pulmonary disease. An a priori power analysis, using pilot NWBT data from four LLP users (i.e., two each with and two without a history of falls), indicated that a minimum of 22 participants would be required to detect significant differences in NBWT scores between fallers and non-fallers. A conservative target of 40 participants was chosen to account for limitations in the small pilot sample and obtain enough data to assess the discriminant validity of the other clinical tests included in the present study.

Procedures

All participants were administered demographic and characterization measures, a retrospective falls survey, four common clinical balance tests, and the NBWT²⁶ by the same rater. Participants wore their desired prosthesis and footwear. A minimum 5-minute rest period was required between tests.

Measurements

Participant demographic and characterization measures

Sociodemographic information including age, height, weight, and sex was collected via self-report. Prosthetic-related information including amputation level, amputation etiology, time since amputation, and Medicare Functional Classification Level (MFCL)²⁷ was collected via interview.

Retrospective falls survey

The number of falls experienced over the previous 12 months was determined via interview. Participants were asked, “In the past year have you had any falls including a slip or trip in which you inadvertently lost your balance and landed on the ground or lower level?”^28–30. Participants were classified as fallers (≥1 falls) or non-fallers (0 falls). A 12-month retrospective period was selected to account for seasonal differences that might influence the likelihood of falls.

Balance Measures

The Activities-Specific Balance Confidence (ABC) Scale is a 16-item self-report measure of balance-related self-efficacy^31,32. It has high internal consistency and good test-retest reliability, convergent construct validity, and known-groups construct validity in LLP users²¹. The ABC was administered via paper survey using the original scale, and scored according to the developers’ instructions³¹.

The Timed Up and Go (TUG) is a performance-based test designed to evaluate participants’ functional mobility. It was developed to assess fall risk in aged adults³³, but has excellent inter- and intra-rater reliability in LLP users^34,35. The TUG was demonstrated once, and then two timed trials were performed. The faster time was used for analysis³⁶.

The Berg Balance Scale (BBS) is a 14-item performance-based test developed to assess static balance and fall risk in adult populations³⁷, but has high inter- and intra-rater reliability and good convergent construct validity in LLP users^19,38. The summary score from all 14 activities was used for analysis.

The Four Square Step Test (FSST) is a performance-based test of dynamic balance²³. It was developed to identify fallers among older adults²³, but has good discriminant validity among new LLP users²⁴. Participants practiced once, and then performed two timed trials. The faster time was used for analysis.

The Narrowing Beam Walking Test (NBWT) is a performance-based test developed to challenge balance control by constraining step width and/or reducing the support surface (Figure 1)²⁶. Consistent with administration procedures²⁶ participants were asked to cross their arms over their trunk and walk along the narrowing beam. The trial was concluded when the participant stepped off the beam, uncrossed their arms, or walked the full length of the beam. The distance walked, divided by the total length of the beam (6.71m) was recorded as the normalized distance. The average normalized distance over trials three to five was used as the NBWT score. This scoring procedure has been shown to provide an accurate estimate of LLP users’ stable, typical performance²⁶.

Figure 1.

Illustration of the Narrowing Beam Walking Test. Participants attempt to walk along four progressively narrower beam segments with their arms crossed over their chest. If participants moved their arms or stepped off the beam that was considered a “balance failure”, and the distance walked to that point is recorded. Height of each segment is 3.8 cm.

Statistical Analysis

Distributions of all balance test scores were evaluated for normality using the Shapiro-Wilk test³⁹.

Content validity was evaluated by examining the number of participants who scored along each test’s range of possible scores. Tests were deemed to exhibit floor or ceiling effects if more than 15% of the participants scored near the minimum or maximum⁴⁰ (i.e., within the first or last bin of their respective histogram).

Convergent construct validity of the NBWT was evaluated by examining the relationship between participants’ NBWT score and scores/times on other balance tests. Spearman correlation coefficients were computed and interpreted based on established thresholds⁴¹. Strong (|r|≥0.7) correlations between NBWT scores and BBS, TUG, or FSST scores/times were considered evidence of convergent validity. Moderate correlations (|r| ≥ 0.3) between NBWT and ABC scores were also considered evidence of convergent validity, given expected differences between performance and self-report measures⁴².

Known-groups construct validity of the NBWT was evaluated by comparing NBWT scores between groups of individuals with expected differences in balance ability. Independent t-tests were used to compare NBWT scores between participants with and without a history of multiple falls, with transtibial and transfemoral amputation, with amputation due to dysvascular and non-dysvascular causes, and classified as K-level 1 or 2 and K-level 3 or 4 ambulators. A Bonferroni correction (α<.0125) was used to account for multiple comparisons. Mann-Whitney U tests were conducted to evaluate the known-groups construct validity of the other clinical balance tests, which are presented as supplementary data. Statistically significant differences between groups were considered evidence of known-groups construct validity.

Discriminant validity was evaluated by analyzing the area under each test’s Receiver Operating Characteristic (ROC) curve⁴³. Areas were first compared to a null value of .5 to assess whether each balance test was better than chance at identifying participants with a history of falls. The significance level for the ROC curve analyses was adjusted with a Bonferroni correction (α<.01) to account for multiple comparisons. Areas were then compared to a threshold of .8, a recommended limit of clinical acceptability⁴⁴. All statistical analyses were performed using SPSS v.24 (Chicago, IL).

Results

Forty LLP users were recruited and participated in the study (Table 1). NBWT scores (mean±95% CI) (0.41±0.13) were normally distributed (W=.964, p=.240). In contrast, BBS scores (median, IQR) (50.0, .334), FSST times (8.83, 5.91 seconds), TUG times (8.75, 3.36 seconds), and ABC scores (85.0, 24.4) were non-normally distributed (W=.713‒.910, p<.008). None of the 40 participants recorded a minimum or maximum score on the NBWT, and only four (10%) recorded a score near the minimum (Figure 2). A large percentage of participants scored near the FSST minimum, or the BBS, and ABC maximums (35%, 63%, and 63% of the sample, respectively).

Table 1.

Participant demographics and characterization measures

	Age (yrs)	Height (cm)	Weight (kg)	Years Since Limb Loss	Sex	Amputation Level	Amputation Etiology	MFCL	Prosthetic Feet and Knees
Mean (SD)	48.7 (14.6)	173 (9.10)	78.5 (14.1)	14.3 (12.6)	M (21) F (19)	TT (25) TF (15)	Trauma (25) Dysvascular (7) Cancer (5) Infection (3)	K1 (2) K2 (15) K3 (17) K4 (6)	ESR (33) SA (4) MA (2) SACH (1)
Range	24–70	155–189	47–107	2–57					MPK (12) NMPK (3)

MFCL: Medicare Functional Classification Level (K-level); M: male; F: female; TT: transtibial; TF: transfemoral; ESR: energy storage and return; SACH: solid ankle cushioned heel; MPK: microprocessor knee; NMPK: non-microprocessor knee

Figure 2.

Distribution of clinical balance test scores. NBWT scores were normally distributed across its range, suggesting that the NBWT is suitable to measure a broad range of unilateral lower limb prosthesis users. Scores on existing clinical balance tests were skewed towards their upper limit and lacked normality, suggesting ceiling effects.

NBWT scores showed moderate-to-strong correlations with scores/times on other balance tests (Figure 3). There were strong negative correlations between NBWT scores and FSST times (r_s=-.80, p<.0001), and between NBWT scores and TUG times (r_s=−.70 p<.0001). Similarly, there was a strong positive correlation between NBWT and BBS scores (r_s=.85, p<.0001), and a moderate positive correlation between NBWT and ABC scores (r_s=.49, p<.001).

Figure 3.

Correlations between participants’ Narrowing Beam Walking Test (NBWT) scores and those on existing clinical balance tests. Correlations (Spearman’s Rho) between participants’ (n = 40) NBWT scores and A) Berg Balance Scale (BBS) scores, B) Four Square Step Test (FSST) times, C) Timed Up and Go (TUG) times, and D) Activities-Specific Balance Confidence (ABC) Scale scores. The best fit line and corresponding 95% CI are presented for each relationship.

NBWT performance differed between participants with expected differences in balance ability (Figure 4). NBWT scores (mean, 95% CI) were significantly lower among participants who were fallers (0.33, 0.25‒0.42) than non-fallers (0.51, 0.43‒0.58), t(38.00)= −3.23, p =.003; participants with transfemoral (0.30, 0.21‒0.40) than transtibial amputation (0.46, 0.39‒0.54), t(30.89)=2.67, p=.011; and participants classified as K1 or K2 ambulators (0.26, 0.18‒0.34) than K3 or K4 ambulators (0.51, 0.45‒0.57), t(33.87)= −5.27, p<.0001. However, NBWT scores for participants with dysvascular amputations (0.29, 0.09‒0.48) were not significantly different than participants with non-dysvascular amputations (0.43, 0.37‒0.50), t(9.05)=1.65, p=.13.

Figure 4.

Known-groups construct validity of the Narrowing Beam Walking Test (NBWT). NBWT scores differed significantly between participants with expected differences in balance ability as measured by A) fall history (faller=24; non-faller=16), B) amputation level (transtibial=25; transfemoral=15), C) amputation etiology (dysvascular=7; non-dysvascular=33), and D) mobility level (K-level 1 and 2=17; K-level 3 and 4=23). Modified boxplots (mean, 95% CI, maximum, minimum) for each group.

NBWT scores discriminated between participants with and without a history of falls (Figure 5). The area under the ROC curve (area, 95%CI) was largest for the NBWT (0.81, 0.62‒0.91), followed by the TUG (0.71, 0.54‒0.88), FSST (0.70, 0.53‒0.86), BBS (0.66, 0.47‒0.83), and ABC (0.65, 0.47‒0.82). The area under the ROC curve was significantly greater than 0.50 for the NBWT (p=.006), but not for any of the other tests (TUG p=.027; FSST p=.038; BBS p=.116; ABC p=.125).

Figure 5.

Receiver Operating Characteristic curves for the A) Narrowing Beam Walking Test (NBWT), B) Timed Up and Go (TUG),C) Four Square Step Test (FSST), D) Berg Balance Scale (BBS), and E) Activities-Specific Balance Confidence (ABC) Scale. The area under the ROC curve, a measure of a test’s diagnostic accuracy, was largest for the NBWT, and significantly greater (p < 0.02) than that of the BBS and ABC.

Discussion

The objective of this study was to assess the content, convergent construct, known-groups construct, and discriminant validity of the NBWT²⁶. Results provide strong evidence of validity, supporting use of the NBWT for assessing balance and fall risk among unilateral transtibial and transfemoral LLP users.

The NBWT exhibited neither ceiling nor floor effects among participants, implying a degree of content validity. Consistent with our hypothesis, all participants were able to start but not complete the NBWT²⁶. None of the participants in this study scored at the NBWT’s maximum or minimum (i.e., 0.0 or 1.0), well below the conservative, 15% threshold used to identify ceiling and floor effects⁴⁰. Therefore, the NBWT addresses a limitation of many existing balance tests; they are too easy or too hard for many LLP users^19,21,22. Absence of ceiling and floor effects in the NBWT suggests the ability to both measure a broad range of clinical patients and quantify changes in balance without reaching the scale limits.

We attribute the absence of ceiling and floor effects to the progressively increasing difficulty of the NBWT’s design (i.e., increasingly narrower walking beams). Tests based on tasks with a fixed level of difficulty often exhibit ceiling and floor effects^25,45–47. Changing difficulty within a single test condition may mitigate ceiling and floor effects present in many contemporary clinical balance tests.

Correlations between balance tests indicate that the NBWT exhibits excellent convergent construct validity, and is suitable for measuring balance in LLP users. As hypothesized, NBWT scores were highly correlated with scores/times on other performance-based balance tests. Observed correlations were similar to those reported between other performance-based balance tests administered to LLP users^19,34. As expected, a moderate correlation between NBWT and ABC scores was observed. Associations between performance-based and self-report measures are generally lower due to variations in how the construct is measured, and the breadth of the concept measured by each type of instrument⁴². The correlation between NBWT and ABC scores was lower or equal to what has been reported between the ABC and other performance-based tests^21,48. The NBWT and ABC may therefore measure different aspects of balance (e.g., demonstrated and perceived balance ability⁵). Characterizing discrepancies between one’s balance, as demonstrated in a performance-based test, and one’s perception of balance, as measured through self-report, may be relevant to understanding the situations leading to falls in LLP users, particularly since the association between demonstrated and perceived balance ability remains consistent in other populations at risk for falls⁴⁹.

NBWT scores differentiated participants who presented differently with respect to balance ability. Lower NBWT scores among LLP users with a history of falls, transfemoral amputation, and lower mobility level were consistent with our hypothesis, and provide evidence of known-groups construct validity for the NBWT. Existing clinical balance tests, like the BBS and ABC do not exhibit similar evidence of known-groups construct validity^19,21. This was confirmed in the present study (Supplementary Table 1). Others, like the FSST and TUG have shown differences with respect to falls history among new transtibial LLP users²⁴. However, new LLP users have greater balance deficits than established LLP users¹⁰. Our results indicate that the FSST and TUG were not able to differentiate between established LLP users with and without a history of falls (Supplementary Table 1), and therefore do not generalize to the broader population of established LLP users. The FSST and the TUG may therefore be better suited to early rehabilitation, while the more challenging NBWT may be a better match for established LLP users.

NBWT scores were not significantly lower among participants with a dysvascular amputation, a factor previously cited to be associated with fall risk^6,14,21. It is likely that the sample of dysvascular LLP users in the present study was too small (n = 7) and heterogeneous with respect to balance ability to detect differences between etiology groups. It may also be that etiology is not as strong an indicator of balance ability as previously suggested, or that the NBWT is unable to detect subtleties in balance due to etiology. Additional testing with dysvascular LLP users will be needed to determine which of these explanations is correct. Despite the similarity in scores based on etiology groups, the NBWT appears to possess the strongest evidence of known-groups construct validity among existing clinical balance tests for ambulatory unilateral LLP users who are at least one year post-amputation.

Lastly, the NBWT discriminated between LLP users with and without a history of falls with greater accuracy than existing clinical balance tests. As hypothesized, the NBWT had the largest area under the ROC curve. Furthermore, only the NBWT had an ROC curve area that was significantly greater than 0.5, and larger than the recommended clinically acceptable threshold of 0.80⁴⁴. Therefore, the NBWT is the only clinical balance test in this study that was capable of discriminating between participants with and without a history of falls with a probability greater than chance.

Research evaluating the discriminant validity of clinical balance tests suited to assessing fall risk in LLP users is limited. Dite et al., reported high sensitivity and specificity for the FSST (92% and 93%, respectively), but only moderate values for the TUG (85% and 74%, respectively)²⁴. However, the prior study included only new (i.e., <6 months experience) transtibial LLP users, whereas participants in the present study included established (i.e., >1 year experience) transtibial and transfemoral LLP users. Additionally, the investigators defined fallers as participants who fell two or more times in the past year, while our definition included one or more falls in the same period. While multiple falls are often used to distinguish “fallers” from “non-fallers”^19,24,50, doing so likely simplifies discrimination and inflates metrics of discriminant validity (e.g., ROC curve area). Additional research is needed to assess whether discriminant validity of clinical balance tests is affected by the operational definition of “faller” prior to reporting cut-off scores.

Study Limitations

The present study only included participants with unilateral LLP users. Additional research is necessary to determine whether this also applies to bilateral LLP users or other populations. The study sample consisted of a larger proportion of traumatic LLP users than what is reported in the literature⁵¹. This may limit generalization of the evidence for validity of the NBWT to individuals with dysvascular amputation.

Falls data were collected via retrospective self-report, and may be affected by recall bias^52,53. While cross-sectional studies are expedient and appropriate for initial validation, they cannot establish a temporal relationship between balance and fall status⁵⁴. Prospective studies are required to gather accurate falls data and evaluate the NBWT’s ability to prospectively predict falls.

The present study was conducted to evaluate the NBWT’s content, construct, and discriminant validity. Research to assess other psychometric properties, including reliability, minimal detectable change, responsiveness, and prospective validity is also needed.

Conclusion

Results of this study provide strong evidence of content, construct, and discriminant validity for the NBWT as a performance-based test of balance ability. The NBWT is a clinically feasible test that can be recommended for evaluating balance in unilateral transtibial and transfemoral LLP users. Because the NBWT can discriminate between LLP users with and without a history of falls with greater accuracy than existing clinical balance tests, it may also serve as a fall risk assessment tool.

Supplementary Material

Supplementary Table 1 - Known-groups construct validity of the Berg Balance Scale (BBS), Four Square Step Test (FSST), Timed Up and Go (TUG), and Activities-specific Balance Confidence (ABC) scale among unilateral lower limb prosthesis users. Body type indicates statistically significant between group differences.

Abbreviations

ABC

Activities-specific Balance Confidence scale

BBS

Berg Balance Scale

FSST

Four Square Step Test

LLP

lower limb prosthesis

MFCL

Medicare Functional Classification Level

NBWT

Narrowing Beam Walking Test

ROC

Receiver Operating Characteristic

SD

standard deviation

TUG

Timed Up and Go