Instrumenting the Balance Error Scoring System for use with patients reporting persistent balance problems after mild traumatic brain injury

Abstract

Objective

To determine whether alterations to the Balance Error Scoring System (BESS), such as modified conditions and/or instrumentation, would improve the ability to correctly classify TBI status in patients with mild TBI with persistent self-reported balance complaints.

Design

A cross-sectional study.

Setting

An outpatient clinic in the Department of Rehabilitation Services at Oregon Health & Sciences University (OHSU).

Subjects

Thirteen subjects (age 16.3 ±2) with a recent history of concussion (mTBI group) and 13 demographically matched control subjects (age 16.7 ±2) (control group).

Intervention

Not applicable.

Main Outcome Measures

Outcome measures included the BESS, Modified BESS (Mod. BESS), Instrumented BESS (Instr. BESS), and Instrumented Modified BESS (Instr. Mod. BESS). All subjects were tested on the non-instrumented BESS and Mod. BESS, scored by visual observation of instability in six and three stance conditions, respectively. Instrumentation of these 2 tests utilized one inertial sensor (APDM-3D), with an accelerometer and gyroscope to quantify bi-directional body sway.

Results

Scores from the BESS and the Mod. BESS tests were similar between groups. However, results from the instrumented measures using the inertial sensor^b were significantly different between groups. The Instr. Mod. BESS had superior diagnostic classification and the largest Area Under the Curve (AUC) when compared to the other balance measures.

Conclusions

A concussion may disrupt the sensory processing required for optimal postural control, measured by sway during quiet stance. These results suggest that the use of portable inertial sensors^b may be useful in the move towards more objective and sensitive measures of balance control post-concussion but more work is needed to increase sensitivity.

As mild traumatic brain injury (mTBI) or concussion frequently goes unreported, the estimated annual US incidence of 1.6 to 3.8 million likely reflects an underestimation.^1,2 Additionally, a recent mTBI increases the risk of sustaining a second mTBI^3–5 and the sequelae of repetitive mTBI may be cumulative.⁶ Thus, premature return-to-play confers serious risk for further brain injury.⁷

A disturbance in balance is a commonly reported symptom post-TBI. The most frequently used clinical scale for post-concussion balance assessment is the Balance Error Scoring Scale (BESS). The BESS measures instability by the examiner’s subjective count of errors in the maintenance of various stances by the patient with his/her eyes closed (feet together, single leg, and tandem stance) and on different surfaces (firm and foam).⁸ Although portable and quick to administer, the BESS suffers from learning ^9,10, practice¹¹, and fatigue¹² effects and may be insensitive to mild impairments.¹³ These factors represent questions about the validity and reliability of the BESS and thus, the decisions emanating from its use.

A recent shortened version of the BESS, the Modified BESS test (Mod. BESS), only includes three stances on firm surface (omitting the three stances on the foam surface). The Mod. BESS was published as a part of the Sport Concussion Assessment Tool 2 (SCAT-2),^14,15 has published norms,^13,16 but may also have problems with sensitivity. A recent study reported no differences in the Mod. BESS between high school students with and without concussion.¹³ The authors suggested that the Mod. BESS may have a ceiling effect and that the presence of foam in the full BESS may be more helpful at classifying those with and without mTBI. However, the full BESS has known psychometric weaknesses, and, like the Mod. BESS, also uses subjective scoring. Objective measures of persisting balance complaints could greatly augment patient safety determinations.

Currently, self-report questionnaires and subjectively scored clinical tests, like the BESS, represent the most frequently utilized method of evaluating and monitoring post-injury complaints. Reliance on these measures for return-to-play and medical management can have grave consequences. A trend towards underreporting mTBI sequelae has been reported in high school and college-aged athletes.^17–19 Many young people experience social pressure to return to their sport before symptoms have fully resolved, contrary to their best interests. The widespread use of self-report measures coupled with the tendency to underreport symptoms has prompted the call for more objective forms of measurement.

The instrumentation of clinical motor tests is increasingly used to achieve objective quantification of movement.^20,21 As balance represents a physically measureable attribute, it lends itself naturally to a technology-based measurement solution. Inertial sensors, the size of a wristwatch, contain accelerometers, gyroscopes and magnetometers, which, when integrated properly, can objectively capture subtle anomalies. Various types of inertial sensors have begun to yield evidence of validity and reliability for balance measurement in people with mild or early Parkinson’s disease,^22,23 Multiple Sclerosis,²⁴ and also in older adults.²⁵ Software²⁶ associated with these sensor readings can automatically calculate a myriad of metrics based upon the features of human movement, making it feasible for pre-programmed, non-expert administration. For example, postural sway during quiet stance can be characterized by its amplitude, frequency and velocity. The National Institutes for Health (NIH) Balance Toolbox²⁷ recently began promoting the use of inertial sensors to assess general balance (non-disease specific) through postural sway. Specifically, NIH recommends a standing balance test that measures anterior-posterior (AP) postural sway during different stance conditions (feet together and tandem on firm and foam surface). However, what is generally referred to as “balance” is comprised of many more elements than postural sway.²⁸ Additionally, postural sway can be measured in more basic dimensions than AP (i.e., medial lateral). Currently, it is not known which sway features and balance conditions are most frequently impaired after mTBI. A recent study showed that the NIH-recommended Standing Balance Test protocol was inferior to the BESS in separating those with and without mTBI²⁹, which, as previously discussed, has demonstrated its own set of weaknesses. ^9–11,30,31 The authors suggest that the Standing Balance Test was not developed directly for concussion but rather as a general balance screening while the BESS was directly developed to assess balance after mTBI. To our knowledge, no studies have yet attempted to instrument the BESS directly using an inertial sensor to improve objective assessment of balance deficits.

The purpose of this study was to determine whether alterations to the BESS, such as modified conditions and/or instrumentation, would improve the ability to correctly classify participants according to TBI status. We hypothesized that: 1. ) Addition of the foam conditions would improve diagnostic accuracy of the BESS over the Mod. BESS and 2.) Instrumentation of the BESS would improve diagnostic accuracy over the non-instrumented BESS.

METHODS

Ethical Review

Oregon Health & Science University (OHSU) Institutional Review Board (IRB) approved this study. All participants enrolled in the study received and signed informed consent forms approved by OHSU IRB. A legal guardian accompanied participants under age 18 and subjects 16 and younger signed an additional assent form. All work was conducted in accordance with the Declaration of Helsinki (1964).

Recruitment

The mTBI participants were recruited from OHSU Sports Medicine Department and the Department of Rehabilitation Services. All mTBI participants were 2–13 months status post mTBI, which had been diagnosed by a physician in the Department of Sports Medicine. All were currently receiving standardized outpatient rehabilitation services for their complaints of continued imbalance and dizziness. Exclusion criteria included: recent orthopedic injuries, other neurological or vestibular disorders unrelated to their brain injury, or outside the age range of 13–19. Healthy age-matched control participants were recruited through advertisements in local community media. The control participants were excluded from participation if there was a history of mTBI, other neurological or vestibular disorders or any recent orthopedic injuries. A total of 30 subjects were screened for this study. Four were unable to participate due to scheduling conflicts.

Participants Characteristics

Thirteen participants with mTBI (age 16.3 ± 1.6; height (cm) 164.3 ± 7; weight (kg) 62.4 ± 12.6; gender M 3, F 10) and 13 demographically-matched healthy controls (age 16.7 ± 2.1; height (cm) 166.7 ± 6; weight (kg) 59.1 ± 9.1; gender M 3, F 10) completed the study protocol (n=26). The mTBI group’s average time post-injury was 5 months ± 3.3, with causes as follows: motor vehicle accident (n=3), soccer (n=4), lacrosse (n=1), wakeboarding (n=1), weightlifting (n=1), horseback riding (n=1), playing on a playground (n=1) and falling out of bed (n=1).

Design and Procedure

This study used a cross-sectional case/control design. Each participant was tested at OHSU’s Rehabilitation Services Department on the 4 primary measures: BESS, Mod. BESS, Instrumented BESS (Inst. BESS) and the Instrumented Modified BESS (Inst. Mod. BESS). The BESS and Mod. BESS were instrumented by adding an inertial sensor^b (Opal by APDM, Inc) at L-5 with an elastic belt (Figure 1). The Opal included two linear accelerometers (medio-lateral (ML) and anterior-posterior (AP)) to detect postural sway displacement at 120 Hz that was wirelessly transmitted to a laptop using Mobility Lab software from APDM.^b,25 Postural sway was automatically quantified both in AP and ML directions using APDM software during each stance condition of these clinical tests by calculating the Root Mean Square (RMS) around the mean position, a metric representing sway dispersion ($\text{RMS}=\sqrt{APac{c}^2+MLac{c}^2}$). RMS has demonstrated excellent reliability in different populations.²³ Most importantly, any physical ‘error’ occurring during standing tasks (i.e. losing balance) would be reflected in a larger RMS value of bi-directional sway.

Figure 1.

Opal sensor placement at L5.

Table 1 details specific test conditions and outcome measurements for the BESS, Mod. BESS, Instr. BESS and Instr. Mod. BESS. The BESS and Mod. BESS were administered per the published instructions⁸ by an experienced and licensed Physical Therapist. Participants were scored subjectively by examiner judgment of errors, which were summed for the total. For example, if the participant opened their eyes or took their hand(s) off their hips, each was counted as 1 error (Total score of 10 errors per condition so 60 points possible for errors). Both Instr. BESS and Instr. Mod. BESS were administered simultaneously to the non-instrumented testing as the data was gathered by the sensors and transmitted wirelessly to the computer. For both instrumented tests, RMS was automatically calculated by the APDM software and was later averaged over all conditions.

Table 1.

Description of the four balance tests conducted in this study; BESS, Mod. BESS, Inst BESS, and Inst. Mod. BESS.

Test		Non-Instrumented	Instrumented
BESS (6 trials)	Body Positions	• Feet together stance • Single leg stance • Tandem stance Eyes closed for all trials	• Feet together stance • Single leg stance • Tandem stance Eyes closed for all trials
	Standing Conditions	• Firm surface • Foam surface 20 seconds for all trials	• Firm surface • Foam surface 30 seconds for all trials
	Measure	Subjective counting of errors in body position Scored (0–10); min=0; max= 60	Sway Root mean square (RMS; m/s2)
Modified BESS (3 trials)	Body Positions	• Feet together stance • Single leg stance • Tandem stance Eyes closed for all trials	• Feet together stance • Single leg stance • Tandem stance Eyes closed for all trials
	Standing Conditions	• Firm surface 20 seconds for all trials	• Firm surface 30 seconds for all trials
	Measure	Subjective counting of errors in body position Scored (0–10); min=0; max= 30	Sway Root mean square (RMS; m/s2)

Statistical Analysis

STATA^c Data Analysis and Statistical Software, Version 11³², was utilized to complete preliminary analyses and hypothesis testing. Unpaired t-tests were computed to verify the successful matching of demographic variables between groups. Independent sample t-tests were also calculated for each version of balance test to determine whether it could detect mean group differences between diagnostic categories. In order to assess our hypotheses about diagnostic accuracy, we computed receiver operating characteristic (ROC) curve area under the curve (AUC) values with the methods developed by Janes, Longton and Pepe³³ in their contributed STATA routines “roccurve” and “comproc”. These routines utilize bootstrap calculations, with 3,000 trials employed here, for the p-values. Each p-value represents the null hypothesis that any difference in the area under the ROC curves occurred due to random sampling alone. To focus on the relevant part of the ROC curves, the AUC is restricted to False Positive Rate < 0.50.

RESULTS

As expected, no significant differences appeared for demographic variables between mTBI and control groups; age (t=0.59; P=0.56); height (t= 0.95; P=0.35); or weight (t=−0.78; P=0.44). Table 2 presents the comparison of balance tests between the mTBI and control group. Scores from the BESS and Mod. BESS were similar between groups. However, results from the instrumented versions of both of these tests were significantly different between groups. However, if adjusted for multiple comparisons for the 4 t tests, only the Instr. Mod BESS remained significant. Figure 2 shows individual scores for each test including a) BESS, b) Mod. BESS c) Instr. BESS d) Instr. Mod. BESS.

Table 2.

Mean ± SD and confidence intervals for the balance measures and comparisons between mTBI group versus age-, height- and weight-matched controls.

Tests	Control		mTBI		P-Value (t-value)
	Mean ± SD	CI	Mean ± SD	CI
BESS (# errors)	13.5 ± 6.35	9.70–17.4	18.5 9.67	12.7–19.4	0.13 (−1.56)
Mod BESS (# error)	2.15 ± 1.77	1.08–3.22	4.53 4.46	1.84–7.24	0.09 (−1.79)
Inst BESS (RMS; m/sec2)	0.37 ± 0.11	0.30–0.43	0.49 0.17	0.38–0.59	0.04 (−2.16)
Inst Mod BESS (RMS; m/sec2)	0.22 ± 0.06	0.18–0.26	0.35 0.14	0.27–0.44	0.01 (−3.03)

Figure 2.

Frequency distributions for the 4 indicated balance tests; a) BESS, b) Mod. BESS, c) Inst. BESS, d) Inst. Mod. BESS. The rows of square symbols indicate mean values. The P-values are for 2-sample t-tests.

In our sample of mTBI patients with persistent complaints of balance impairment or unsteadiness, we defined abnormal balance as 1.5 SD from the control group mean. Using this definition, we found sensitivity and specificity to be the following: BESS (23%, 92%); Mod BESS (31%, 85%); Inst. BESS (38%, 100%) and Instr. Mod. BESS (54%, 100%).

Figure 3 presents the ROC AUC for all of the balance scales. To determine if the addition of foam increased the diagnostic accuracy, we compared the BESS to Mod. BESS and found no difference in AUC (P=0.59). To determine if instrumentation of the BESS and Mod. BESS improved diagnostic accuracy, we compared the instrumented version of the BESS to the non-instrumented and found that the ROC AUC of the Instr. Mod. BESS differed from that of the BESS (P=0.032) and the Mod. BESS (P= 0.035). For completeness, we compared the, Inst. BESS vs. Inst. Mod. BESS and found they did not differ significantly from one another (P=0.09). Table 3 summarizes details of the ROC AUC for the balance measures.

Figure 3.

Receiver Operator Characteristic curves for the True Positive Rate (TPR) and False Positive Rate (FPR) for each balance test.

Table 3.

Summarizes details of the ROC AUC per test.

Test	Area Under Curve	Standard Error	Confidence Interval (95%)
BESS	0.63	0.11	0.41–.085
Mod. BESS	0.64	0.11	0.42–0.86
Inst. BESS	0.70	0.11	0.50–0.91
Mod. Inst. BESS	0.81	0.09	0.64–0.99

DISCUSSION

The primary aim of this study was to determine whether alterations to the BESS, such as modified conditions and/or instrumentation, would improve the ability to correctly classify participants according to TBI status. We found that: 1) The addition of foam in the full version of the BESS, for both non-instrumented and instrumented, did not improve its ability to differentiate between mTBI and control patients. 2) Using the Instr. Mod. BESS resulted in the highest diagnostic accuracy.

The findings from the ROC AUC analysis performed in this study provide some practical considerations. The ROC AUC results suggest that using the Inst. Mod. BESS is better for distinguishing between groups and that the foam stances did not add obvious value in post-injury classification. Our results suggested that adding instrumentation reduced classification errors. For example, in our sample of 13 athletes with persistent self-reported instability, only 3 (23%) would have been classified as having an ‘abnormal’ score on the commonly used BESS leading to premature return-to-play and missed treatment of a deficit receptive to rehabilitation.³⁴ Conversely, by using one inertial sensor on the waist during the test, 31% more (7 of 13) athletes would be documented as abnormal. This difference has very real implications for clinical treatment and return-to-play determinations. However, sensitivity may be still too low to use as a single test for balance control.

Our results did not agree with a recent paper reporting the BESS’ superiority to the accelerometer-based Standing Balance Test.²⁹ One reason for differing results is that our study measured instrumentation of the BESS (designed for mTBI assessment), rather than the Standing Balance Test. Furthermore, our sway metric included both the AP and ML directions while the study by Whitney et al assessed path length only in the AP direction. Instability in the ML direction may be more sensitive to imbalance. ³⁵ Moreover, instability in the AP direction only, particularly in tandem stance conditions where less AP sway occurs naturally, may not reflect imbalance after mTBI.

Postural sway represents one domain of balance, specifically static balance. Testing various stance positions while altering surface and visual inputs challenges ones ability to use sensory information for stability. While postural sway represents the complex sensorimotor control of the nervous system required to maintain equilibrium during stance posture, it may not capture other important domains of balance, including dynamic or cognitive aspects of balance. Insult occurring at any level of efferent neural communication, including corticospinal tract, cerebral peduncles, corona radiate, internal capsule or cerebellar peduncle, could result in suboptimal performance. Furthermore, vestibular nuclei connect to critical areas for balance such as the cerebellum, cranial nerves (3,4 and 6), the thalamus, cortex, and the spinal cord.³⁶ Each of these areas contributes to coordinated balance performance. Thus, each patient may have an individualized balance profile, such that measuring just one domain of balance may be insufficient.

Accurate identification and treatment of balance disorders after mTBI is critical for adolescent athletes. Even subtle balance problems may disrupt academic, psychological, social and physical development and may play a role in further concussions. If measurable deficits cannot be documented, the patient may not receive appropriate rehabilitation services. The mTBI management of adolescents is further complicated by a complex mix of psychosocial factors. Due to the patient’s age and concurrent developmental processes, difficulties with social and emotional development can arise after injury. Sole reliance on subjective complaints can be misleading and under-informative.

Study limitations

A significant limitation to our study is the small sample size. With a larger sample size, the non-instrumented BESS may have detected group differences. A limitation of most balance studies is that a gold standard for abnormality does not currently exist. The most frequently used measure is self-report. As such, we relied upon self-report for this group of patients seeing rehabilitation after mTBI. As previously discussed, there is a documented tendency to underreport in this age group. Thus, we purposely selected young people whose reported symptoms were unresolving and which delayed their return-to-play. As such, these complaints are likely to be valid. Additionally, our sample was fairly heterogeneous regarding time since injury, possibly resulting in the measurement of constituents from fundamentally different groups. Such limitations could limit the generalizability of our findings. Finally, our rater was not blinded to subject diagnosis, representing a risk of investigator bias. However, knowledge of diagnostic classification would be expected to result in worse ratings for cases and better ratings for controls. Thus, investigator bias would have been most likely to result in non-significant differences between non-instrumented and instrumented measures. In general, we found the opposite result, which mitigates our concern over this limitation.

Conclusion

In summary, this study highlights the potential of using of inertial sensors to measure specific domains of balance control in adolescents who have sustained an mTBI. Due to the vast number of children affected by mTBI and to the vulnerable stage of development, it is imperative to identify sensitive and specific measures. Annually, it is expected that at least 173, 285 persons below the age of twenty will be treated for sports and recreation related TBI in emergency departments across the United States.³⁷ With increasing evidence of long-term effects of repeated concussion, it is essential that complete recovery has occurred before returning to full activity. Until now, a sophisticated laboratory was required to perform postural tests of sway and gait analysis. These results suggest that the use of portable inertial sensors^b may be useful in the move towards more objective measures of balance control post-concussion but more work is needed to improve levels of sensitivity.

List of Abbreviations

AP

Anterior-posterior

AUC

Area Under the Curve

APDM Inc.

Ambulatory Parkinson’s Disease Monitoring

BESS

Balance Error Scoring System

Inst. BESS

Instrumented Balance Error Scoring System

Inst. Mod. BESS

Instrumented Modified Balance Error Scoring System

IRB

Institutional Review Board

ML

Medio-lateral

mTBI

mild Traumatic Brain Injury

Mod. BESS

Modified Balance Error Scoring System

OHSU

Oregon Health & Science University

ROC

Receiver Operating Characteristics

RMS

Root Mean Square

SCAT 2

Sports Concussion Assessment Tool 2