Comparison of the Balance Accelerometer Measure and Balance Error Scoring System in Adolescent Concussions in Sports

Abstract

Background

High-technology methods demonstrate that balance problems may persist up to 30 days after a concussion, whereas with low-technology methods such as the Balance Error Scoring System (BESS), performance becomes normal after only 3 days based on previously published studies in collegiate and high school athletes.

Purpose

To compare the National Institutes of Health’s Balance Accelerometer Measure (BAM) with the BESS regarding the ability to detect differences in postural sway between adolescents with sports concussions and age-matched controls.

Study Design

Cohort study (diagnosis); Level of evidence, 2.

Methods

Forty-three patients with concussions and 27 control participants were tested with the standard BAM protocol, while sway was quantified using the normalized path length (mG/s) of pelvic accelerations in the anterior-posterior direction. The BESS was scored by experts using video recordings.

Results

The BAM was not able to discriminate between healthy and concussed adolescents, whereas the BESS, especially the tandem stance conditions, was good at discriminating between healthy and concussed adolescents. A total BESS score of 21 or more errors optimally identified patients in the acute concussion group versus healthy participants at 60% sensitivity and 82% specificity.

Conclusion

The BAM is not as effective as the BESS in identifying abnormal postural control in adolescents with sports concussions. The BESS, a simple and economical method of assessing postural control, was effective in discriminating between young adults with acute concussions and young healthy people, suggesting that the test has value in the assessment of acute concussions.

Traumatic brain injuries (TBIs) pose a serious health problem across the globe. An average of 1.4 million cases every year of TBIs are reported in the United States, but the true incidence of TBIs is unknown.⁷ Sports-related concussions are a major contributor to the number of TBIs. In fact, Langlois et al⁷ estimated that the sports-related TBI incidence rate in the United States ranges from 1.6 to 3.8 million injuries every year. Many people experience persistent disability related to TBIs; although 90% of people with a sports concussion recover within 7 to 10 days, at least 10% of people with a sports concussion experience prolonged symptoms relating to their physical, cognitive, behavioral, and emotional health.²²

A recent international consensus statement from the 3rd International Conference on Concussion recommends that several aspects of a concussion be evaluated, such as dizziness, headache, poor sleep, and emotional problems; the group also reaffirmed that the balance component is a reliable and valid addition to the assessment of athletes with concussions.¹² Furthermore, several studies have suggested that a balance assessment provides highly useful information for estimating prognosis, which is important because predicting the time course and extent of expected recovery represents one of the most challenging aspects of caring for a person with a sports concussion. ^8,19,22 Sheedy et al¹⁹ reported that symptoms from a concussion were worse in patients with dizziness. Others have also confirmed that a complaint of dizziness was a negative prognostic indicator.^8,22

In general, concussions can cause balance problems by adversely affecting either the central nervous system or the inner ear balance mechanism, and both low-and high -technology methods have been shown to measure this impairment in balance function.³ Low-technology methods are usually faster, less expensive, and easier to use, especially in children, but have the disadvantage of being imprecise and susceptible to interrater error.^1,3 High-technology methods typically provide greater sensitivity, precision, and reliability, especially at longer time intervals after injury, but require more expensive, less available equipment and more in-depth analysis.³

The most commonly used low-technology balance assessment tool for those with sports concussions is the Balance Error Scoring System (BESS).¹⁶ The BESS consists of 3 standing positions (double stance, single leg, and tandem) under 2 surface conditions (solid and foam), all performed with eyes closed, and is scored based on the number of errors across trials. The BESS was specifically designed to assess balance in persons with concussions, primarily college-aged patients, and currently is the standard method for assessing balance after concussions for the National Collegiate Athletic Association.¹¹ Guskiewicz and colleagues⁵ have reported that BESS scores worsen for several days after a concussion but resolve after 3 days in high school and collegiate athletes. ^14,15 Additionally, Sabin et al¹⁸ studied 16 high school athletes and found no effect of headaches on the increase in BESS scores after a football game in nonconcussed athletes. The lack of correlation suggests that the BESS does not capture all facets of concussion prognosis.

The most common high-technology balance assessment method is the Sensory Organization Test (SOT) using computerized dynamic posturography.¹³ The SOT quantifies how much a person sways under various sensory-isolating conditions using a force plate. Using the SOT, it has been shown that patients with sports concussions tend to sway more than healthy persons 1 day after injury, and quantifiable balance problems may persist up to 30 days after a concussion contrary to the 3 days reported using the BESS.^5,14,15,20 However, despite the advantages of the SOT, more recent studies^3,4 and recommendations¹² support low technology methods that promote wider applicability.

We chose to evaluate an intermediate technology, the Balance Accelerometer Measure (BAM), which was developed as part of the National Institutes of Health (NIH) Toolbox project, even though it was specifically designed to assess patients with concussions. This NIH sponsored program was an attempt to develop quantifiable tests and measures that could be used across studies to easily test cognitive, emotional, sensory, and motor health and function across the life span (ages 3–85 years) for use in longitudinal studies and clinical trials. Thus, although the BAM was not specifically designed to assess patients with concussions, the BAM is a low-cost balance measure that can be used with persons of any age and can be administered safely with little training.²¹ Because the BAM employs a small wearable sensor that allows balance testing outside the clinic based on an objective sensor signal, it seemed ideal for assessing adolescents with concussions.

The purpose of this research was to evaluate the ability of the BAM and the BESS to assess postural control in distinguishing between high school students at various time points after a concussion as compared with healthy controls. High school students with and without a concussion were recruited to evaluate the BAM and the BESS.

MATERIALS AND METHODS

Patients

Forty-three high school students with a diagnosed concussion (mean age, 15 ±1.2 years) and 27 healthy control participants (mean age, 16 ±1.3 years) were enrolled in this study after informed consent was obtained from their parent or legal guardian and assent from each patient. The study was approved by the Institutional Review Board of the University of Pittsburgh. The patients with concussions were divided into acute and subacute groups based on whether they were tested within 2 weeks of their concussion. This criterion was based on studies by Lau et al^8–10 that defined protracted recovery as greater than 14 days. The demographic characteristics of the 3 groups are presented in Table 1. Note that the proportion of male patients was significantly lower in the healthy group as compared with the concussion groups. The patients with concussions were recruited from the University of Pittsburgh Medical Center Sports Concussion Program. The diagnosis of concussion was based on recent head trauma and presentation to the concussion clinic with post concussive symptoms. Each of the patients with concussions had a sports-related mild TBI. The patients with acute concussions were tested at a mean 8 ±3 days after the concussion. The patients with subacute concussions were tested at a mean 151 ±215 days after the concussion. Healthy control participants were recruited from 3 local high schools and were tested at the high school either during the school day or after school. Patients with concussions were tested at the University of Pittsburgh Center for Sports Medicine.

TABLE 1.

Demographic and Physical Characteristics for Healthy and Concussed Patients^a

Patient Group	n	Age, y	Duration, d	Height, cm	Weight, kg	Male Sex,b n (%)
Healthy	27	15 ± 1.2	NA	167 ± 5.4	68 ± 16	8 (30)
All concussions	43	16 ± 1.3	118 ± 197	170 ± 9.8	69 ± 16	27 (63)
Concussion duration ≤2 wk	10	16 ± 1.5	8 ± 3.2	172 ± 9.3	73 ± 13	9 (90)
Concussion duration >2 wk	33	16 ± 1.2	151 ± 215	170 ± 10	68 ± 17	18 (55)

^aData are shown as mean ± standard deviation, except where indicated. NA, not applicable.

^bP < .01 for sex difference between healthy and concussion groups.

Instrumentation

For the BAM, patients were examined with a dual-axis accelerometer attached anteriorly at the midline on the pelvis using a gait belt. The complete setup and device specifications are described in detail elsewhere.²¹ Briefly, a dual-axis accelerometer (±1.2 g, ADXL213AE, Analog Devices Inc, Norwood, Massachusetts) was used to record mediolateral and anteroposterior accelerations of the pelvis during standing. The device is illustrated in Figure 1. The 16-bit accelerations were transmitted wirelessly via Bluetooth (Roving Networks Inc, Los Gatos, California) transmission at 100 Hz. For the BESS, no instrumentation was required. However, the patients were each evaluated via anterior views on video recordings by a physical therapist at a later time.

Figure 1.

The National Institutes of Health Toolbox device. The sensor (middle) is shown along with its power supply (right) and Bluetooth wireless device (left).

Procedure

The BAM protocol consists of 6 standing balance conditions (Table 2 and Figure 2) that included the following: (1) standing with feet side by side on a firm surface with eyes open, (2) standing with feet side by side on a firm surface with eyes closed, (3) standing with feet side by side on a foam surface with eyes open, (4) standing with feet side by side on a foam surface with eyes closed, (5) tandem stance on a firm surface with eyes open, and (6) tandem stance on a firm surface with eyes closed. Each condition was performed for 45 seconds with the patients’ arms crossed across the chest. The patient was allowed 3 attempts to perform each condition. After completion or the third failed attempt, testing was advanced to the next condition. Patients who were unsuccessful in completing any attempt for a given condition were arbitrarily given the maximum standard score for patients on that condition. Please see Rine et al¹⁷ for further details.

TABLE 2.

The 6 Stance and Surface Conditions Used in This Study

Condition	Eyes	Stance	Surface
1	Open	Together	Solid
2	Closed	Together	Solid
3	Open	Together	Foam
4	Closed	Together	Foam
5	Open	Tandem	Solid
6	Closed	Tandem	Solid

Figure 2.

Balance Accelerometer Measure testing positions. (1, 2) Feet together while standing on a firm surface used for conditions 1 and 2. (3, 4) Feet together while standing on a compliant foam surface used for conditions 3 and 4. (5, 6) Feet in tandem stance while standing on a firm surface used for conditions 5 and 6.

The BESS standard protocol consisted of asking each patient to stand for 20 seconds in the following 6 stance positions with eyes closed and hands on iliac crests: (1) double-leg stance on a firm surface, (2) single-leg stance on a firm surface, (3) tandem stance on a firm surface, (4) double-leg stance on a foam surface, (5) single-leg stance on a foam surface, and (6) tandem stance on a foam surface. An Airex pad (Somersworth, New Hampshire) was used for the foam surface (0.5 m 3 0.4 m 3 0.06 m).

Data Analysis

For the BAM, the normalized path length of the anteroposterior acceleration (NPL AP) was computed using the final 40 seconds of each completed trial after discarding the first 5 seconds of data. The NPL AP was then used to generate sway scores for patients on each condition in which the NPL AP was standardized to the mean for NPL AP sway for 18-to 34 -year-old healthy participants obtained from a previous analysis (9.4 ±2.2 mG/s).²¹

For the BESS, a score was calculated for each condition by adding up the number of errors that occurred, where simultaneous errors were counted as a single error and the maximum total error score for any condition was 10.¹⁶ Errors were any of the following: (1) moving hands off of the iliac crests; (2) opening the eyes; (3) step, stumble, or fall; (4) abduction or flexion of the hip beyond 30_; (5) lifting the forefoot or heel off of the testing surface; and (6) remaining out of the proper testing position for more than 5 seconds.¹⁶ All the scoring was performed by a physical therapist using the video recordings.

Statistical Analysis

Nonparametric statistical analysis was used for all group and condition comparisons because of the nominal count format of BESS scoring and the skewed distribution of standardized BAM scores. Error count scores on each of the 6 BESS conditions and the total BESS score were compared among the healthy group and acute and nonacute concussion groups using the nonparametric Kruskal-Wallis test. The standardized BAM sway scores for each BAM condition and the total BAM score were also compared among the healthy group and acute and nonacute concussion groups using the nonparametric Kruskal-Wallis test. Post hoc individual pairwise group comparisons for conditions identified as having a group effect were described using the Mann-Whitney U test with a type I error rate of .05.

The effect of condition for the BAM and the BESS was assessed using the nonparametric Wilcoxon signed-rank test for dependent samples. Performance was compared between the conditions with eyes open and eyes closed for the BAM and between the conditions with firm surface and foam surface for both tests.

The accuracy of BESS and BAM individual scores for identifying patients in the acute and nonacute concussion groups versus the healthy group was estimated using a receiver operating characteristic (ROC) curve. The plot of sensitivity versus false-positive rates for identification of both concussion groups versus the healthy group was estimated using BESS and BAM sway scores with analysis of the area under the curve (AUC) compared with a null value of 0.50, indicating chance-level accuracy.

RESULTS

Mean standardized BAM scores for participant groups across all sensory/stance conditions and total score are presented in Table 3. There were no significant differences between male and female patients or between concussion groups across all conditions. Using ROC curve analysis for both male and female patients and combined across male and female patients, standardized BAM sway scores on the condition with eyes open/foam surface weakly identified nonacute patients versus healthy participants (AUC, 0.56; 95% confidence interval [CI], 0.52–0.79; P< .05). Patients with acute concussions were not identified on the ROC curve from healthy participants above chance levels of accuracy on any condition or on total standardized BAM sway scores.

TABLE 3.

Balance Accelerometer Measure Standard Scores for Healthy and Concussed Patients^a

Patient Group	EO/Firm	EC/Firm	EO/Foam	EC/Foam	TS/EO	TS/EC	Total
Healthy	1.1 ± 1.2	1.7 ± 1.1	1.6 ± 1.0	6.2 ± 2.7	2.2 ± 1.6	4.9 ± 4.0	17.7 ± 9.5
All concussions	0.9 ± 1.1	1.9 ± 1.4	2.4 ± 1.4	6.7 ± 2.7:	2.3 ± 1.7	7.5 ± 6.9	21.8 ± 11.2
Concussion duration ≤2 wk	1.1 ± 0.9	2.1 ± 1.1	2.1 ± 0.6	7.0 ± 2.6	2.4 ± 1.3	5.7 ± 4.9	20.5 ± 6.5
Concussion duration >2 wk	0.9 ± 1.2	1.8 ± 1.5	2.4 ± 1.6	6.6 ± 2.8:	2.2 ± 1.8	8.1 ± 7.3	22.1 ± 12.3

^aScores standardized to young healthy adults in the EO/firm condition (mean sway, 9.4 ± 2.2 mG/s). Data are shown as mean ± standard deviation for 6 sensory/stance conditions and total scores. EO/Firm, eyes open/firm surface; EC/Firm, eyes closed/firm surface; EO/Foam, eyes open/foam surface; EC/Foam, eyes closed/foam surface; TS/EO, tandem stance/eyes open; TS/EC, tandem stance/eyes closed; Total, sum of condition-specific standard scores.

There was a significant visual condition effect for the BAM for each of the 3 comparisons with eyes open versus eyes closed, that is, eyes open/firm surface versus eyes closed/firm surface, eyes open/foam surface versus eyes closed/foam surface, and tandem stance/eyes open versus tandem stance/eyes closed. For each comparison, sway was significantly (P <.01) larger in the condition with eyes closed for each group except for the condition with tandem stance in the acute concussion group in which the increased sway with eyes closed versus eyes open was not significant. There was also a significant surface condition effect for the BAM for both of the comparisons with firm surface versus foam surface, that is, eyes open/firm surface versus eyes open/foam surface and eyes closed/firm surface versus eyes closed/foam surface. For each comparison, sway was significantly (P <.01) larger in the condition with foam surface.

Mean BESS error count scores for healthy and concussion groups are presented in Table 4 for each condition and for the total BESS score. No differences were found between male and female patients. Significant concussion group differences in BESS scores were found among the 3 groups in the conditions with tandem stance/firm surface (P <.01), double-leg stance/foam surface (P <.02), and tandem stance/foam surface (P <.01). These differences were not significant when the data were analyzed separately for male and female patients probably because of an insufficient size of the participant groups. Post hoc group comparisons showed significantly higher BESS scores (more errors) for the acute concussion group compared with the healthy group on the conditions with tandem stance/firm surface and tandem stance/foam surface (both P <.01). Also, scores for patients in the nonacute concussion group were significantly higher (more errors) than for healthy participants on the conditions with tandem stance/firm surface and tandem stance/foam surface (both P <.02). The BESS scores in the acute concussion group were significantly higher (more errors) than in the nonacute concussion group in the condition with double leg stance/foam surface (P < .04).

TABLE 4.

Balance Error Scoring System Scores for Healthy and Concussed Patients^a

Patient Group	n	DS/Firm	SS/Firm	TS/Firmb	SUB/Firm	DS/Foamc	SS/Foam	TS/Foamb	Total
Healthy	27	0	3.4 ± 2.1	0.5 ± 0.8	3.8 ± 2.4	0.4 ± 0.7	7.9 ± 1.8	3.1 ± 2.2	15.4 ± 5.0
All concussions	43	0.1 ± 0.2	0.32 ± 2.8	1.5 ± 1.7	4.6 ± 3.8	0.4 ± 1	7.5 ± 2.8	5.3 ± 3.0	18.4 ± 8.3
Concussion duration ≤2 wk	10	0.1 ± 0.3	3.7 ± 3.2	2.2 ± 2.2	5.8 ± 4.6	1 ± 1.2	8.7 ± 1.5	6.1 ± 2.9	21.7 ± 8
Concussion duration >2 wk	33	0.03 ± 0.2	3 ± 2.7	1.3 ± 1.6	4.3 ± 3.6	0.3 ± 0.9	7.1 ± 3.1	4.9 ± 3.0	17.4 ± 8.3
Significanced		NS	NS	P < .01	NS	P < .02	NS	P < .01	NS

^aData are shown as mean ± standard deviation for firm and foam surfaces, surface subscores, and total. DS/Firm, double-leg stance/firm surface; SS/Firm, single-leg stance/firm surface; TS/Firm. tandem stance/firm surface; SUB/Firm, subscore for firm surface conditions; DS/Foam, double-leg stance/foam surface; SS/Foam, single-leg stance/foam surface; TS/Foam, tandem stance/foam surface; NS, not significant.

^bSignificant group differences (P < .02) between the healthy group and each of the concussion groups.

^cSignificant group differences (P < .05) between groups with concussion duration ≤2 weeks and those with concussion duration >2 weeks.

^dDifferences among healthy, concussion duration ≤2 weeks, and concussion duration >2 weeks groups (Kruskal-Wallis nonparametric comparison).

Based on the ROC curve analysis, patients in the acute concussion group were accurately identified compared to healthy participants on the conditions with tandem stance/firm surface (AUC, 0.79; 95% CI, 0.61–0.97; P <.01) and tandem stance/foam surface (AUC, 0.80; 95% CI, 0.66–0.95; P<.01) as well as with the foam surface subscore (AUC, 0.77; 95% CI, 0.61–0.94; P<.02) and the total BESS score (AUC, 0.74; 95% CI, 0.53–0.94; P<.04). A total BESS score of 21 or more errors optimally identified patients in the acute concussion group versus healthy participants at 60% sensitivity and 82% specificity. Patients in the nonacute concussion group were accurately identified compared to the healthy participants by BESS scores on the conditions with tandem stance/firm surface (AUC, 0.67; 95% CI, 0.53–0.80; P <.03) and tandem stance/foam surface (AUC, 0.68; 95% CI, 0.55–0.82; P<.02).

There was a significant surface condition effect for the BESS for each of the 3 comparisons with firm surface versus foam surface, that is, double stance/firm surface versus double stance/foam surface, single stance/firm surface versus single stance/foam surface, and tandem stance/firm surface versus tandem stance/foam surface. For each comparison, the number of errors was significantly (P <.01) larger in the condition with foam surface.

DISCUSSION

This study indicated that the BESS may be a useful technique for the assessment of balance in high school–aged patients after a concussion. However, in this study, the BAM did not appear to be useful for assessing balance problems after concussions in this age group.

The BAM protocol used in this study was successfully completed in all patients, which suggests high feasibility. The validity of the BAM as a test of balance is supported by the condition effects, independent of group, wherein sway was larger for the condition with eyes closed versus eyes open and for the condition with foam surface versus firm surface. These condition effects indicate that the BAM was correctly recording postural sway and was sensitive to changes in the sensory modalities available for balance, namely vision and somatosensation.

The BAM was expected to show a difference between healthy control participants and concussed patients because it is computerized with quantitative results that are independent of the evaluator. However, the BAM was not specifically designed for assessing persons with concussions. Rather, the BAM was designed as an easily administered test of balance for a wide age range.¹⁷ In this population of young athletes, the BAM was not sensitive overall to concussions. Possibly, if the BAM postconcussion score could be compared with a baseline BAM score, using a matched-pair design, the BAM might show a difference in sway after concussions.

Currently, the BESS developed by Riemann et al¹⁶ is the tool most commonly used to assess balance after concussions in athletes.¹¹ The BESS was designed to assess athletes after concussions. In this study, the scoring of the BESS was optimized by using videotapes reviewed by an expert. We found the expected condition effect of a foam surface versus a firm surface, with patients experiencing more balance errors on the foam surface. Also, we found that the BESS discriminates healthy persons from concussed patients fairly well.

Our statistical analysis also determined that the 2 tandem stance conditions were the best at separating the healthy participants from the concussed patients. The double stance conditions probably suffered from a floor effect because they were too easy and thus even the concussed persons had few errors, whereas the single-leg stance conditions suffered from a ceiling effect because they were too difficult and thus even the healthy persons had a high error score. Hunt et al,⁶ using a statistical analysis of nonconcussed high school children, suggested that the BESS could be improved by eliminating the double stance conditions. We reanalyzed our data using this suggestion and found no improvement in the ability of the BESS to discriminate healthy persons from concussed patients by eliminating the double stance conditions. Specifically, the AUC for the ROC curve analysis for the BESS without the double stance conditions was 0.72 (95% CI, 0.50–0.94) as compared with an AUC of 0.74 (95% CI, 0.53–0.94) for the standard BESS.

A modified BESS is included in the Sport Concussion Assessment Tool–2 (SCAT-2)¹² in which only the firm surface conditions are included. We reanalyzed our data using the firm surface conditions (Table 4) and determined that the modified BESS that appears in the SCAT-2 did not discriminate any groups in this study.

Our data disagree somewhat with those of Register-Mihalik et al¹⁴ by showing that the BESS can discriminate between healthy and concussed high school–aged persons beyond 3 days. Register-Mihalik et al¹⁴ studied high school and collegiate athletes and found that the BESS returned to baseline 3 days after a concussion. The basis for the disagreement with our findings may be related to the severity of the concussion, the time after concussion when testing was performed, and differences in methodology. Our patients were a referred population to a concussion clinic. Also, Register-Mihalik et al¹⁴ used a preconcussion versus postconcussion comparison, whereas our study was population based. Note that our total BESS scores are higher (more errors) than those reported by Register-Mihalik et al¹⁴ possibly because the BESS was scored in our study by a physical therapist who could rewind the video recording to observe possible errors. The BESS scores in the Register-Mihalik et al¹⁴ study were recorded by athletic trainers from 110 high schools and 14 colleges without video recordings. By allowing a careful review of video recordings in the present study, more balance errors were noticed. The mean number of errors for our concussed sample was 21.8, whereas the mean number of errors for patients 1 day after a concussion was 14.2 in the study by Register-Mihalik et al.¹⁴ Also, the mean BESS score in our study is consistent with a study by Covassin et al² of 150 high school athletes who reported a mean BESS score of 22.2 only 1 day after a concussion.

Our a priori expectation was that the BAM would surpass the BESS in identifying patients with a concussion because the BESS uses observational scoring whereas the BAM is objective and may have less measurement error. Also, scoring the BESS is subjective and relies on clinical expertise and experience. Despite these apparent disadvantages of the BESS over the BAM, the BAM was not as efficacious as the BESS in discriminating between healthy and concussed high school patients. Note that 3 of the conditions of the BAM and the BESS, namely the double stance/firm surface, double stance/foam surface, and tandem stance/firm surface conditions of the BESS, were the same except for arm positions. None of these conditions showed group differences for the BAM, but 2 of them did show changes for the BESS. This suggests that scoring balance errors is preferable to measuring postural sway with an accelerometer for showing the effect of concussions on balance in this age group, assuming no baseline data are available.

The current study has limitations. The concussed patients in this study were referred to a specialty clinic and were thus a more severely injured population compared with typical concussed patients. Also, there was a wide range of times after concussions at which testing was performed. Additional research is required to determine whether the BESS or the BAM can show changes in balance function after concussions on an individual basis. Addressing this issue will require testing athletes both before and after a concussion as well as testing athletes several times after a concussion to observe changes in balance during recovery. These 2 testing techniques could be combined by recording postural sway quantitatively while patients perform the BESS. Another consideration for research is correlating balance testing with neurocognitive function.