Sideline Performance of the Balance Error Scoring System during a Live Sporting Event

Abstract

Objective

The purpose was to examine the influence of a live sporting sideline environment on balance error scoring system (BESS) performance.

Design

Prospective longitudinal cohort study.

Setting

The BESS was performed by all participants at three locations; 1) quiet laboratory, 2) football stadium sidelines, 3) basketball arena sidelines.

Participants

The experimental group had 38 participants (age: 20.1±1.1 years; height: 170.0±7.7 cm; mass: 66.7±9.5 kg) who were female intercollegiate student-athletes (SA). The control group consisted of 38 recreationally active female college students (age: 20.8±1.1 years; height: 162.6±6.0 cm; mass: 63.7±10.6 kg).

Interventions

The two groups performed the tests at the same locations, the SA group during live sporting events and the control group when no event was occurring.

Main Outcome Measures

The dependent variable was the total BESS score. Separate 2 × 3 mixed methods ANOVAs investigated the influence of the environment and practice effect.

Results

There was a significant interaction for group by environment (P=0.004) and the SA group committed more errors at both the football and basketball settings than the control group. The SA group also committed more errors at football (P=0.028) than baseline. The control group demonstrated a likely practice effect with fewer errors during each administration.

Conclusions

BESS score deteriorated when performed on the sidelines of a live sporting event potentially challenging the clinical utility of the BESS. Clinicians need to consider the role of the local environment when performing the BESS test and should perform post-injury tests in the same environment as the baseline test.

Clinical Relevance

When performing balance testing of patients with suspected concussions, clinicians need to consider the environment in which the test is performed and attempt to match the pre-season testing environment.

Introduction

Appropriate concussion management is based upon the accurate and timely recognition of a sports-related concussion.¹ Many concussions present with subtle cognitive, vestibular, and balance deficits, rather than outward signs, which are not easily identifiable by sports medicine clinicians and therefore sensitive objective clinical testing is essential.^2,3 However, current imaging technologies are either insensitive to concussion (e.g., X-Ray, computerized tomography, magnetic resonance imaging (MRI)) or remain experimental in nature (e.g., functional MRI, diffusion tensor imaging).¹ Therefore, current research findings and clinical recommendations call for a multifaceted assessment including measures of postural control, cognitive processing, neuropsychological performance, and self-report symptoms as no individual assessment technique's sensitivity exceeds 70%.^1,4,5 This sensitive assessment battery is required to reduce the risk of subsequent injury and prevent the rare, but often fatal, second impact syndrome.^6,7

Impaired postural control is a cardinal concussion related symptom.⁸ The most commonly utilized postural control assessment technique is the balance error scoring system (BESS).^2,3,9 The 4^th Consensus Statement on Concussion in Sport (4^th CIS) recommends a modified BESS on the firm surface only; however, investigations of athletic trainer's concussion management practice patterns indicate the 6 stances remains the clinical assessment of choice.^1,9 The 6-stance BESS has been validated to static forceplate measures and has high specificity (0.91), but low sensitivity (0.34) for concussion.^10,11 However, the BESS has multiple limitations with two noteworthy concerns being a practice effect with repeat administration and potential environmental influences.^12-16 Indeed, a small but clinically significant improvement, typically 1 – 3 fewer errors, in BESS occurs with repeat administration.^12,15,16 Further, Onate reported significant impairments in single leg foam stance BESS, as well as a non-significant increase of 3.3 total errors for the entire test, in healthy athletes at a baseball practice.¹⁴ This testing paradigm occurred in a baseball dugout during a practice setting; however, it is likely not as strenuous as the live game sideline environment with the increased attentional challenges of the live game environment.

The ability to maintain postural control despite attentional challenges is frequently assessed with dual-task methodologies.¹⁷ While sideline BESS assessment does not include a traditional cognitive challenge (e.g., working memory tasks), non-cognitive challenges including noise and anxiety can elicit similar dual task decrements.^18,19 Specifically, increased environmental noise and anxiety may increase the bodies' forward lean, secondary to anterior center of pressure shift and increased tibialis anterior activity, which may destabilize the individual.^18,20 While the removal of the visual stimuli may mitigate some anxiety based instabilities, the remaining auditory stimuli will still challenge the individual.^19,21 The resulting potential destabilizing effects may result in increased errors in the BESS and should be considered as a confounding variable and potential protocol limitation.

Most clinicians perform baseline BESS testing in a relatively quiet environment (e.g., athletic training room during pre-participation physicals); however, post-injury the BESS is most commonly performed on the sidelines.⁹ As the most frequently utilized balance assessment in the recommended multifaceted concussion assessment, the BESS's potential confounding factors must be elucidated.^3,9 Therefore, the purpose of this study was to investigate the environmental influence, the sidelines of a live collegiate sporting event, on student-athletes' BESS performance. Secondarily, to address potential confounders of both a practice and surface effect, we compared the performance during live events to a control group's performance at the same physical sites when no events were occurring. We hypothesized the live sideline environment would adversely affect BESS performance, potentially mitigating the known BESS practice effect, which would add an additional consideration for clinicians utilizing a sideline BESS evaluation.

Methods

Participants

The experimental participants were 38 National Collegiate Athletic Association (NCAA) division I female student-athletes (SA) who were active members of the intercollegiate women's soccer (N=17), women's volleyball (N=13), and softball (N=8) teams and the control group (control) consisted of 38 recreationally active full-time female college students not currently participating in intercollegiate athletics at the same institution. There were no differences between groups for any demographic variables and no comparisons were made between sports within the SA group. (Table) The exclusion criteria for both groups were self-reported previous formal instruction in the BESS test, current lower extremity injury, chronic ankle instability, concussion within the previous three months, or any balance related disorders which was confirmed prior to each testing session. The SA participants sample consisted of all members of the three intercollegiate athletic teams who meet inclusion/exclusion criteria, were available for all three testing dates, had a videotaped baseline test from their pre-participation physical examination, and agreed to participate in the study. All participants provided written informed consent prior to participating as approved by the institutional review board.

Instrumentation

The BESS was performed on an AIREX (L:20″ × W:16.4″ × H:2.5″) balance pad and was recorded using two video cameras (Cannon HV20, Cannon ZR 500) at 60 Hz. The BESS consists of 6 test position, including 3 stances (Double limb, Single limb, Tandem) on 2 surfaces (Firm and Foam), for 20 seconds each. (Figure 1) The dominant leg was the participant's self-reported preferred kicking leg.¹⁰ The non-dominant limb was the stance limb during the Single stance position and was the back leg during the Tandem stance position. Orthogonally placed cameras were adjusted to maximize the image size while capturing the entire participant.

Figure 1.

The Balance Error Scoring System. The six stances (A: Double leg firm; B: Single leg firm; C: Tandem firm; D: Double leg foam; E: Single leg foam; F: Tandem foam) were performed for 20 seconds each. The BESS errors identified included; 1) the hands coming off of the iliac crest, 2) opening the eyes, 3) step, stumble, or fall, 4) moving the hip into greater than 30° of abduction, 5) lifting the forefoot or heel, and 6) remaining out of the test position longer than 5 seconds.¹⁰

Procedures

All participants performed the BESS on three occasions over the course of two academic years; 2009 – 2011. The SA group performed their baseline test during pre-participation physicals in a quiet controlled environment. (Figure 2A) The next two testing sessions consisted of either a basketball (BB) or football (FB) environment. The order was not randomized as participant availability was limited due to team related travel, practice, or game schedules. The control group performed the first BESS trial in the same quiet laboratory environment and was then matched to a member of the SA group with the subsequent testing order matched accordingly. The control group completed the three test sessions with a minimum of 30 days between test sessions. All participants were instructed not to perform any exercise on the testing day and the SA group was tested on non-practice/game days or prior to practice.

Figure 2.

BESS Environments. The BESS was performed in a quiet laboratory environment (2A), the basketball arena (2B), and the football stadium (2C). The quiet laboratory environment testing occurred in private in the biomechanics laboratory as a station during preparticipation physicals. The basketball arena test site was just off of the court but in close proximity to stands on both sides and behind the participant. The football stadium test site was directly in front of stands with one of the cameras placed in the stairwell seen behind the participant.

The BB testing occurred approximately 5m from the court in an alcove/corner of the facility, an area deemed clinically plausible to conduct a sideline concussion evaluation. (Figure 2B) The two unobstructed cameras were each positioned approximately 7m from the participant. The participants were in close proximity to the stands on three sides and stood facing a camera located under the arena stands. Testing was performed throughout the game and was only stopped during halftime when it was perceived a concussion test would occur away from the game facility (e.g., locker room, athletic training room). For the SA group, the two BB testing dates were in February and December 2010 and were selected as dates on which a large number of potential participants were available for testing. The average attendance for the two BB testing dates was 1,617 which were typical in those seasons. For the control group, testing was performed identically with the exception that no events were occurring in the BB setting and occurred with a rolling testing timeline to allow appropriate matching to the SA participant.

The FB testing session occurred on the sideline of the football field near the 5 yard line, approximately 5m from the sidelines, on a flat area of the grass surface (i.e., off the field's “crown”). (Figure 2C) The distance from the playing field was chosen for the safety of all involved in the study, in an area where two unobstructed cameras could be set-up and capture the participant's performance and not interfere with game day operations. The frontal camera was placed 5m from the participant inside a stairwell and the sagittal camera was placed approximately 10m away along the interior stadium wall. The participants stood facing the stands with their back to the playing surface. A research team member was positioned between the testing site and the playing surface and testing was stopped when a play started from inside the 20 yard line at the testing end of the field or the play appeared to be coming towards the testing area. Several tests were interrupted between stances; however, no tests were stopped during a stance. Similar to BB, testing was conducted throughout the game and only stopped at halftime. The two SA testing dates occurred in September and October of 2010 with typical average attendance (20,252) and with similar temperatures. For the control group, testing was performed identically with the exception that no events were occurring in the FB setting and occurred with a rolling testing timeline to allow appropriate matching to the SA participant.

Data Analysis

The BESS was scored using the revised scoring method such that multiple simultaneous errors were counted as a single error.¹ Total BESS score was calculated by summing the errors from the 6 stances based upon video review by the lead author.¹⁵ Similar to previous BESS studies, the lead author was trained by the senior author and demonstrated satisfactory competency prior to the study.^14,15 The participant's name or group was not disclosed on the video; however, the video's background (live game versus empty facility) made group blinding implausible. A trained member of the research laboratory, not associated with this project, independently reviewed a random 10% of the trials and demonstrated high reliability (Intraclass Correlation Coefficient >0.90) for the video-based scoring.

The SA groups participated in regularly scheduled strength and conditioning sessions throughout their participation. These sessions included muscular strength and power development, cardiovascular conditioning sessions, and neuromuscular balance training. The control participants were instructed to continue their normal physical activity which consisted of traditional aerobic and anaerobic training and intramural sport participation.

Statistical Analysis

A single dependent variable, the overall BESS score, was calculated for each testing session. A 2 (group) × 3 (environment) mixed methods ANOVA was performed to test the primary hypothesis of the environment's effect. A second 2 (group) × 3 (order) mixed methods ANOVA was performed to test the order effect for repeat administration. When significant interactions were present, independent t-tests were conducted as follow-up tests to identify between group differences. Simple contrasts were utilized to compare subsequent test performance to baseline scores. The alpha value was set a-priori set to p=0.05.

Results

There was a significant interaction between group and environment (P=0.014). Between group differences were identified at FB (SA: 12.2±6.4 errors and control: 8.1±3.3 errors; P=0.001; d=0.64) and BB (SA: 10.3±6.3 errors and control: 7.4±3.3 errors; P=0.015; d=0.46), but no baseline differences (P=0.33). (Figure 3) Within the SA group, there was a significant difference between baseline performance and FB (Baseline: 10.6±5.1 errors and FB: 12.2±6.4 errors; P= 0.028; d=0.31), but not between baseline and BB (P=0.647). For the control group, significant differences were identified between baseline and FB (Baseline: 9.4±5.4 errors and FB: 8.1±3.3 errors; P=0.05; d=0.24) and BB (BB: 7.4±3.3 errors, P=0.005; d=0.37).

Figure 3.

BESS Performance by Environment. ^aThere were significant differences between groups at Football (SA: 12.2 ± 6.4 errors and control: 8.1 ± 3.3 errors; P=0.001; d=0.64) and Basketball (SA: 10.3 ± 6.3 errors and control: 7.4 ± 3.3 errors; P=0.015; d=0.46), but not at baseline (P=0.33). ^bThere were significant differences between Baseline and Football for both the SA group (Baseline: 10.6 ± 5.1 errors and Football: 12.2 ± 6.4 errors; P= 0.028; d=0.31) and the control group (Baseline: 9.4 ± 5.4 errors and Football: 8.1 ± 3.3 errors; P=0.05; d=0.24). ^cThere was a significant difference between Baseline and Basketball for the control group only (Baseline: 9.4 ± 5.4 errors and Basketball: 7.4 ± 3.3 errors, P=0.005; d=0.37).

There was a significant interaction between group and testing order (P=0.025). Between group differences were identified at the second (SA: 10.7±5.5 errors and control: 7.8±3.7 errors; P=0.008; d=0.52) and the third (SA: 11.8±7.2 errors and control: 7.7±2.9 errors; P=0.002; d=0.56) testing sessions, but no differences at baseline. (Figure 4) Within the SA group, there were no significant differences between baseline and second (P=0.804) or third (P=0.141) test session. For the control group, there were significant differences between baseline and both second (Baseline: 9.4±5.4 errors and second: 7.8±3.7 errors, P=0.020; d=0.29) and third (7.7±2.9 errors; P= 0.016; d=0.31) test sessions.

Figure 4.

BESS Performance by Order. ^aThere were significant differences between groups at both the second (SA: 10.7 ± 5.5 errors and control: 7.8 ± 3.7 errors; P=0.008; d=0.52) and the third (SA: 11.8 ± 7.2 errors and control: 7.7 ± 2.9 errors; P=0.002; d=0.56) test sessions. ^bFor the control group, there were significant differences between baseline and both second (Baseline: 9.4 ± 5.4 errors and second: 7.8 ± 3.7 errors, P=0.020; d=0.29) and third (7.7 ± 2.9 errors; P= 0.016; d=0.31) test sessions. There were no differences between test sessions for the SA group.

Discussion

The primary purpose of this investigation was to identify the influence of environmental conditions on BESS performance. This study's primary finding was a clinically and statistically significant increase (worsening) in BESS total score for the SA group during live sporting events as opposed to a significant decrease (improvement) in the control group who performed the test at the same locations during quiet times. With repeated BESS administration, an individual's performance should improve as observed with control participants;⁸ however, the SA group performed significantly worse at the FB site both compared to their own baseline performance (+1.4 errors) and compared to the control group at the same site (+4.1 errors). Conversely, the control group improved with each repeat administration, suggesting the physical testing environment itself (e.g., grass surface) was not responsible for the differences noted in SA group. These findings may present a substantial concern for clinicians when performing a sideline postural control assessment following a suspected concussion.

There have been limited investigations of the multifaceted concussion assessment battery during live events and cognitive testing does not appear to be adversely affected.^9,14,22 Specific to the BESS, Onate identified an increase in errors (3.3 errors) when tested in a baseball dugout during practice compared to a quiet baseline environment; however a conservative statistical approach, assessing each stance, prevented the difference from reaching significance.¹⁴ Clinically, this is an important distinction as total BESS score, not stance score, is utilized when making clinical decisions. This study identified significant differences between groups at both FB (4.1 errors) and BB (2.9 errors). Within groups, the SA performed significantly worse at FB compared to baseline (1.7 more errors), whereas the control group performed significantly better than baseline at both FB (1.3 fewer errors) and BB (2.0 fewer errors). Further, the improved performance of the control group at FB supports previous findings that barefoot testing on a grass surface does not adversely affect postural control.²³ The study's results agree that an increase of several errors is normal when performing sideline BESS assessments.¹⁴

Repeat BESS administration is known to improve test performance likely due to a practice effect and baseline values are typically achieved within several days of injury.^12,13,15 Indeed, an “absence of improvement” has been suggested to be a consideration with repeat administration of the BESS and thus it has been recommended primarily as an acute assessment technique which may lack validity in determining recovery.⁸ Consistent with previous findings, the control group committed 1.6 fewer errors between the 1^st and 3^rd test, despite approximately 2 months between test sessions.^13,15 (Figure 4) Conversely, the SA group committed 1.2 more errors between the 1^st and 3^rd test, suggesting the environmental challenges outweighed the expected practice effect improvements. Of further interest, the SA group also performed neuromuscular and balance training programs between test sessions, known to improve BESS performance,²⁴ but still demonstrated reduced performance at the live environments. Finally, testing outdoors on multiple dates incorporates weather changes; however, test conditions were generally similar between sessions. Warmer temperatures are associated with shortening of reflex onset latency which is worthy of consideration as temperatures were typically between 80 – 90° Fahrenheit during testing.²⁵ These design limitations, accepted to increase the clinical utility of the study, should be considered when interpreting this study's results.

Anxiety, a common post-concussion experience, increases postural instability and includes increased contribution of somatosensory activity.^26,27 When visual stimuli are removed, as in the BESS, the anxiety induced postural instability is reduced.²¹ However, auditory stimuli increase postural instability.¹⁹ The participants were subjected to considerable auditory environmental distractions (e.g., spectators, band/cheerleaders, announcements) during this study. In non-pathologically anxious, otherwise healthy adults, anxiety increases the anterior-posterior body sway due to a forward shift of the body's center of mass, secondary to tibialis anterior activation.^18,28 Greater postural sway is significantly positively correlated with increased BESS errors.¹⁰ Finally, increased postural sway is likely exacerbated on the foam surface which reduces the effectiveness of ankle torque required for stability.²⁹ Therefore, it is plausible that the participants performing the BESS on the sidelines of live sporting events would have increased anxiety resulting in increased anterior-posterior postural sway, resulting in increased BESS errors.

This study was designed to maximize clinical relevance and, as such, some measures of control were reduced. The use of force plates, physiological measures, or psychological state anxiety measures may have improved the understanding of the underlying mechanisms; however, this was neither feasible in a live sporting environment nor part of the a-priori research questions, but could be addressed in future studies. The experimental participants were NCAA division I student-athletes, from multiple sports, who should be accustomed to performing athletic activities in stadiums/arenas with typical environmental distractions present albeit at smaller venues and crowds. This study only evaluated performance based on total BESS score regardless of the individual stance scores; however, future studies could further investigate either individual stances or of the modified BESS, firm stances only, recommended by the 4^th CIS.¹ Further, this study only assessed healthy participants and therefore the environmental influence on acutely concussed individuals is unknown. Finally, due to logistical challenges of scheduling student-athletes testing at live sporting events and the lag time from baseline testing, the SA group had a longer delay between test sessions than the control group (mean 177.0 ± 92.2 days and 57.7 ± 36.4 days, respectively for the 3 test dates). The a-priori decision to have at least a 30 day window between test sessions for the control group occurred to maximize participant retention and was supported by previous research findings which suggested there were no differences between BESS performance after 30 days.¹⁶

Pre-participation baseline concussion testing, including the BESS, is typically performed in a testing environment which limits distractions; however, assessment of suspected concussions occurs on the sidelines where considerable distractions exist.^3,9 The main finding of this study was a clinically and statistically significant increase (worsening) in BESS score during a live sideline assessment. Clinicians should be aware of this additional BESS limitation when performing SCAT-3 post-injury assessments and consider matching baseline and post-injury testing environments. While the practicality remains a challenge, previous research has indicated that fatigue impairs BESS performance for 13 – 18 minutes;³⁰ therefore, clinicians could use this time to remove the individual from the playing environment and perform the BESS in a quiet distraction free area. However, this recommendation would presuppose that multiple health care providers were available to allow one individual to leave the immediate event area and conduct the testing without sacrificing appropriate urgent or emergent health care needs. Finally, the BESS is only one component of a multifaceted concussion SCAT-3 and research should continue to investigate the determinants of testing paradigm.

Table 1. Participant Demographics.

There were no differences between groups for age (p=0.068), height (p=0.057) or weight (p=0.209) based on independent sample t-test.

	Age (years)	Height (cm)	Weight (kg)
Student-Athletes (SA)	20.1 ± 1.1	170.0 ± 7.7	66.7 ± 9.5
Control Participants	20.8 ± 1.1	162.6 ± 6.0	63.7 ± 10.6