Clinical and Device-based Metrics of Gait and Balance in Diagnosing Youth Concussion

Abstract

Purpose:

Evaluate the discriminatory ability of two clinical measures and one device-based measure of gait and balance for concussed youth.

Methods:

We enrolled 81 cases and 90 controls age 14-18 years old from August 2017-June 2018. Controls were recruited from a suburban high school, and cases were recruited from the concussion program of an academic pediatric tertiary care center. Tests included two clinical measures: (1) Complex tandem gait, scored as sway/errors walking forward and backward eyes open (EO) and closed (EC); (2) Modified balance error scoring system (mBESS), scored as total number of errors on three standing tasks; and one device-based measure: (3) modified clinical test of sensory interaction and balance (mCTSIB) using the Biodex Biosway Balance System, scored as a sway index. Sensitivity, specificity, ideal cut-point, and area under the receiver operating characteristic curve (AUC) were calculated for all test components.

Results:

Ideal cut-point for total number of sway/errors for tandem gait=5, sensitivity 41%, specificity 90%. Ideal cut-point for total mBESS errors=4, sensitivity 55%, specificity 75%. Ideal cut-point for mCTSIB=1.37, sensitivity 37%, specificity 88%. Among each test, some individual components outperformed overall composites, in particular tandem gait (specificity forward eyes open = 99%, sensitivity backward eyes closed = 81%). Among the 40 cases and 65 controls with all 3 assessments, AUC (95% CI) for tandem gait=0.63 (0.52,0.75), mBESS=0.70 (0.60,0.81), and mCTSIB=0.54 (0.42,0.66).

Conclusions:

A device-based measure of balance did not produce better discriminatory ability than two clinical assessments. Complex tandem gait has the additional benefit of being an easy-to-perform and graded test with highly sensitive and specific individual components.

INTRODUCTION

Sport- and recreation-related concussions are common injuries sustained by pediatric and adolescent patients (1), often presenting with balance and gait deficits (2-4). Various clinical methods for assessing balance and postural stability after concussion exist, including the Modified Balance Error Scoring System (mBESS), as part of the Sport Concussion Assessment Tool–5^th edition (SCAT-5) (5,6,7) and tandem gait testing (8,9). There are also increasingly available device-based biomechanical measures, including devices with sensory integration and balance protocols (8,10).

Previous studies have evaluated each of these clinical and device-based measures in assessing balance in concussed and non-concussed athletes, including the mBESS (11,12), timed tandem gait testing (8), and biomechanical force plate devices (13). Previous studies have also shown complex tandem gait testing, measured as errors or sway during tandem gait performance in several conditions (walking forward and backward with eyes open and closed), to be highly predictive of prolonged concussion symptoms in pediatric patients, making this testing promising in distinguishing concussed from non-concussed youth, including non-athletes (14,15). Indeed, a recent meta-analysis called for further investigation of complex gait testing (16). Finally, two recent studies have been performed, one demonstrating improved discriminatory ability when adding an inertial-based sensor to the mBESS (17), and another showing that the timed tandem gait was superior to mBESS in discriminating injured youth with concussion from healthy youth (18). The subjects of all of these studies were collegiate athletes, who have different balance abilities compared with children due to both developmental factors and athletic ability (3). No studies have compared performance on complex tandem gait testing to standard clinical testing via the mBESS or a device-based measure of balance via a force plate device among concussed and non-concussed youth.

The primary objective of this study was to describe the ability of complex tandem gait testing, the mBESS, and a modified clinical test of sensory interaction and balance (mCTSIB) via a biomechanical force plate device to discriminate concussed from non-concussed youth. The secondary objective was to determine ideal cut-points on these tests to maximize sensitivity and specificity.

METHODS

Study Design and Patient Population

Subjects (both cases and controls) were recruited as part of a large prospective observational study and completed balance testing from August 2017 through July 2018. Controls were recruited from a private suburban high school. At the time of testing, controls were beginning participation in their respective soccer, basketball, and lacrosse seasons. Cases were recruited as a convenience sample from the same private suburban high school, as well as from the concussion program of a tertiary care academic pediatric center, a referral program that evaluates patients with acute injury. Cases received a diagnosis of concussion in accordance with the 5^th Consensus Statement on Concussion in Sport (19); none of the assessments utilized in this study were used to diagnose a subject with a concussion. All cases presented with a sport- or recreation-related injury, with assessments completed within 28 days of injury. Exclusion criteria for both cases and controls included an injury of the lower extremities (defined as an injury with use of an active support device, such as a brace, walking boot, splint, cast, or crutches; or an injury visibly affecting gait), being within one month of clearance from a previous concussion. Any subject who sustained a concussion during the study period was included in the case cohort only. Informed consent from a parent or guardian and child assent was obtained for those under 18 years of age prior to enrollment, and informed consent from the study subject was obtained directly for those 18 years of age or older. The study was approved by the Institutional Review Board of the Children’s Hospital of Philadelphia.

Data Collection

Demographic characteristics, including date of birth, sex, and race/ethnicity were collected via self-report by controls at enrollment and abstracted from the electronic health record for cases. Participants completed a survey upon enrollment that included a question about typical number of hours of weekly exercise (for cases, prior to injury). All assessments were conducted by trained study coordinators or sports medicine specialist, all of whom received standardized training from the same sports medicine specialist, including multiple observed practice sessions. Assessments on controls were performed exclusively in a dedicated space at the suburban high school, and assessments on cases were performed either in the high school or in the sports medicine clinics of the tertiary care academic center. Testing was performed as part of a larger battery of clinical and objective measures; therefore, not all patients completed all balance measures due to constraints on staff, time, and device availability (Table 1).

Table 1.

Demographic characteristics and frequency of completed balance assessments among concussed (cases) and non-concussed (controls) subjects

	Cases (N=78)	Controls (N=88)	P-value
Mean age in years (range)	16.1 (14.0-18.6)	15.9 (14.0-18.5)	0.30
Male, N (%)	38 (49%)	44 (50%)	0.87
Race/ethnicity, N (%)			0.82
Non-Hispanic white	61 (78%)	69 (78%)
Non-Hispanic black	8 (10%)	8 (9%)
Other / unknown	9 (12%)	11 (13%)
Weekly hours of exercise, N (%)			0.61
<5 hours	7 (9%)	3 (3%)
5-10 hours	30 (38%)	36 (41%)
11-15 hours	23 (30%)	29 (33%)
>15 hours	15 (19%)	18 (21%)
Unknown	3 (4%)	2 (2%)
Mean days from injury to assessment (SD)	11.4 (7.2)	n/a
Completed balance assessments, N (%)
Complex tandem gait	78 (100%)	68 (77%)	<0.001
mBESS	55 (71%)	85 (97%)	<0.001
mCTSIB	54 (69%)	88 (100%)	<0.001
All three balance tests	40 (51%)	65 (74%)	0.003

mBESS = Modified Balance Error Scoring System; mCTSIB: modified clinical test of sensory interaction and balance; SD = standard deviation

Testing Metrics

Complex Tandem Gait

The complex tandem gait testing was performed as part of a visio-vestibular examination, and involved evaluating a subject walking in tandem under four conditions (five steps each): forward with eyes open, forward with eyes closed, backward with eyes open, and backward with eyes closed (20). For each condition, sway (yes/no) and the number of steps off line (errors) were recorded. Sway was defined as any steps off the tandem line, raising of arms for stability, or any truncal movement off a vertical line extending from the crown of the subject’s head to the midline between his or her feet. Each condition was treated as a dichotomous condition (any error/sway being abnormal, no errors or sway being normal). A composite score was calculated by summing the total number of steps off line (up to 5 per condition) and the presence of sway per condition, for a maximum score of 24.

Modified Balance Error Scoring System (mBESS)

The mBESS involved evaluating the number of observed errors for a subject standing with eyes closed with both feet together, feet in tandem (non-dominant in the back), and on one foot (non-dominant). Each stance was completed over a 20-second trial, with errors defined as lifting the hands off of the iliac crest; opening the eyes; any step, stumble or fall; moving the leg into greater than 30-degrees abduction; lifting the forefoot or heel; and remaining out of test position for more than five seconds (21). Errors for each stance were recorded (up to a maximum of 10), as well the total number of errors (up to a maximum of 30).

Modified Clinical Test of Sensory Interaction and Balance (mCTSIB)

The mCTSIB was performed on the Biosway Portable Balance System (Model 950-460, Biodex Medical Systems) to provide an objective assessment of balance with variation of visual, vestibular, and somatosensory inputs. It was comprised of four conditions, which involved a double leg stance with eyes open and eyes closed on both a firm and dynamic (foam) surface. For each of the four conditions, a sway index, defined as the standard deviation of the average center of gravity position measured radially from center, was measured via the force plate sensor, with a range of 0-4, with a higher number correlating with poorer performance (22). In addition to the sway indices for the four conditions, a composite sway index, the mean of the four conditions, was calculated by the Biosway system.

Statistical Analysis

Bivariate distributions of demographic characteristics and completion of testing metrics among cases and controls were compared using F-tests and Wilcoxon rank-sum tests for continuous variables and chi-square statistics for categorical variables. For each dichotomous testing metric (the four individual conditions of complex tandem gait), the sensitivity and specificity of each individual component was calculated for all subjects completing the test. Receiver operating characteristic (ROC) curves were then obtained for continuous testing metric components, including the continuous composite scores of each assessment. The Youden index, the point on the ROC curve furthest vertically from the chance line (positive diagonal) that maximizes sensitivity and specificity for a continuous test, was used to identify the “optimal” cut-point of each testing metric component among any patient completing the individual test, and sensitivity and specificity at this cut-point was reported (23). The area under the receiver operating characteristic curve (AUC) was then calculated for the composite scores of each test among only subjects who completed all three tests, and compared pair-wise using a non-parametric generalized U-statistic (24,25). Two sensitivity analyses were performed. Given the varied nature of time since injury in our sample, we evaluated the discriminatory nature of all three tests in cases assessed less than or equal to 7 days of injury and those assessed greater than 7 days from injury. Additionally, we evaluated the discriminatory ability of partial versions of tandem gait and mBESS, including only the components found be most useful (walking forward eyes open and backward eyes closed for tandem gait, and single leg stance and tandem stance for mBESS). Analyses were conducted using SAS 9.4 (SAS Institute Inc., Cary, NC, USA).

RESULTS

A total of 78 cases and 88 controls were included in our analytic sample. Of the 78 cases, 11 (14%) were evaluated at the high school and 67 (86%) in the academic pediatric center concussion program. Cases and controls were comparable across sociodemographic characteristics, including sex, age, race/ethnicity, and usual weekly hours of exercise (Table 1). Cases were seen on average 11.4 days (standard deviation [SD] 7.2 days) following injury. The proportion of patients completing each test is shown in Table 1, with 95% of patients completing at least two of the three tests.

In evaluating all patients completing complex tandem gait, cases had more sway/errors than controls for each of the four conditions (Table 2). The median (IQR) number of total sway/errors, on a scale of 0-24, was 3 (1, 6) among cases compared with 2 (0, 3.5) among controls (p=0.007; Table 2). The ideal cut-point to maximize sensitivity and specificity for the total number of sway/errors was 5, which yielded a sensitivity of 41% and specificity of 90%. The component with the greatest sensitivity was backward eyes closed (81%), while the component with the greatest specificity was forward eyes open (99%).

Table 2.

Characteristics of balance assessments among concussed (cases) and non-concussed (controls) subjects

	Cases	Controls	P-value	Ideal Cut- Pointa	Sensitivity	Specificity
Complex tandem gait	N=78	N=68
Forward eyes open, any sway or error, N (%)	19 (24)	1 (1)	<0.001		24%	99%
Backward eyes open, any sway or error, N (%)	31 (40)	11 (16)	0.002		40%	84%
Forward eyes closed, any sway or error, N (%)	49 (63)	32 (47)	0.06		63%	53%
Backward eyes closed, any sway or error, N (%)	63 (81)	41 (60)	0.006		81%	40%
Total gait errors and sway, median (IQR)	3 (1, 6)	2 (0, 3.5)	0.007	5	41%	90%
Modified balance error scoring system (mBESS)	N=55	N=85
Double leg stance, median errors (IQR)	0 (0, 0)	0 (0, 0)	0.03	1	5%	100%
Single leg stance, median errors (IQR)	3 (1, 5)	2 (0, 3)	0.007	4	42%	82%
Tandem stance, median errors (IQR)	1 (0, 2)	0 (0, 1)	<0.001	1	60%	65%
Total mBESS errors, median (IQR)	4 (2, 7)	2 (1, 3)	0.001	4	55%	75%
Modified Clinical Test of Sensory Interaction and Balance Test (mCTSIB)	N=54	N=88
Firm surface, eyes open, mean sway index (SD)	0.82 (0.54)	0.65 (0.31)	0.02	0.58	70%	53%
Firm surface, eyes closed, mean sway index (SD)	1.15 (0.89)	0.88 (0.48)	0.02	1.00	44%	78%
Foam surface, eyes open, mean sway index (SD)	1.14 (0.50)	0.97 (0.40)	0.03	1.02	52%	72%
Foam surface, eyes closed, mean sway index (SD)	2.19 (0.80)	1.98 (0.44)	0.05	2.34	35%	80%
Composite mCTSIB index, mean sway index (SD)	1.32 (0.57)	1.12 (0.32)	0.008	1.37	37%	88%

IQR = interquartile range; SD = standard deviation

^aIdeal cut-point presented for continuous measures only. Sensitivity and specificity listed for continuous measures at ideal cut-point.

Balance measures using the mBESS also differed between cases and controls (Table 2). The distribution of the number of errors for single leg stance and tandem stance, as well as the total number of errors, was higher among cases than controls (Table 2). The ideal cut-point for total mBESS errors among all patients completing the test was 4, which yielded a sensitivity of 55% and specificity of 75%. For the double leg stance, only 3 of the 55 cases had any errors (compared with 0 among controls), yielding a sensitivity of 5%.

The mean sway index for all four conditions of the mCTSIB on the biomechanical force plate device (eyes open and closed on a firm and dynamic surface) was higher in cases than controls (Table 2). The composite sway index was also higher among cases (1.32, SD: 0.57) than controls (1.12, SD: 0.32; p-value=0.008). The ideal cut-point for the composite sway index among all subjects completing the mCTSIB testing was 1.37, which yielded a sensitivity of 37% and a specificity of 88%.

Receiver operating characteristic curves for total errors of complex tandem gait, total errors of mBESS, and the composite score of mCTSIB for only patients completing all three tests (40 cases and 65 controls) are shown in Figure 1. The AUC for complex tandem gait was 0.63 (95% CI 0.52, 0.75), for mBESS was 0.70 (95% CI 0.60, 0.81) and for the mCTSIB was 0.54 (95% CI 0.42, 0.66; Table 3). In comparing the AUC’s, there was no statistical difference between complex tandem gait and either mBESS (p-value=0.32) or mCTSIB (p-value=0.21), however there was a statistical difference between mBESS and mCTSIB (p-value=0.02). In evaluating our two sensitivity analyses, removing double leg stance from the mBESS did not change the overall AUC (p-value=1.00), while creating a composite of complex tandem gait using only forward eyes open and backward eyes closed conditions (partial tandem gait) produced a similar AUC as the complete test (p-value=0.28, Table 3). In comparing cases assessed ≤7 days from injury (15 cases completing all 3 tests) to those assessed >7 days after injury (25 cases completing all 3 tests), we noted similar AUC’s for each of our measures of balance (0.61 in those ≤7 days vs. 0.65 in those >7 days for complex tandem gait; 0.69 vs. 0.71 for mBESS; and 0.52 vs. 0.56 for mCTSIB).

Figure 1.

Receiver operating characteristic (ROC) curves among patients completing all three balance tests.

Table 3.

Comparison among patients completing all balance measures

	Cases (N=40)	Controls (N=65)	AUC (95% CI)	Test for ROC curve difference (1)a	Test for ROC curve difference (2)b
Complex tandem gait, median total sway/errors (IQR)	3 (1,6)	2 (0,4)	0.63 (0.52, 0.75)	(ref)	0.32
Partial tandem gait, median total sway/errors (IQR)c	2 (1,4)	1 (0,2)	0.66 (0.55, 0.77)	0.28	n/a
mBESS, median total errors (IQR)	4.5 (1,7)	1 (0,3)	0.70 (0.60, 0.81)	0.32	(ref)
Partial mBESS, median total errors (IQR)d	4.5 (1,7)	1 (0,3)	0.70 (0.60, 0.81)	n/a	1.00
mCTSIB, mean sway index (SD)	1.29 (0.60)	1.10 (0.23)	0.54 (0.42, 0.66)	0.21	0.02

AUC = area under the receiver operating characteristic curve; CI = confidence interval; IQR = interquartile range; mBESS = Modified Balance Error Scoring System; mCTSIB: modified clinical test of sensory interaction and balance; ROC = receiver operating characteristic

^aTest for ROC curve difference (1) utilizes composite complex tandem gait as a reference

^bTest for ROC curve difference (2) utilizes composite mBESS as reference

^cPartial tandem gait composite includes only two conditions (walking forward eyes open and backward eyes closed)

^dPartial mBESS involves only two conditions (single leg stance and tandem stance)

DISCUSSION

This study evaluated the ability of three different balance measures to discriminate between concussed and non-concussed youth, specifically comparing two clinical measures with a device-based assessment of balance via a biomechanical force plate device (mCTSIB). Overall, we found complex tandem gait and mBESS to have similar discriminatory ability (AUC of 0.63 and 0.70, respectively), with mCTSIB performing more poorly than mBESS (AUC of 0.54).

As technology advances, a number of studies have evaluated the use of device-based measures that provide fully objective assessments to distinguish concussed from non-concussed subjects. Paniccia et al (10) and Fabri et al (26) assessed postural stability using a force plate device in healthy children age 5-18 and found sway indices approximating the mCTSIB values measured in controls in our study. Paniccia et al found that subjects with a history of concussion had increased sway on more demanding tasks and that postural stability was worse in the subgroup of subjects who reported concussion-like symptoms (10). Imhoff et al evaluated the effects of active rehabilitation on a postural stability using a biomechanical force plate device in concussed children age 10-17 and found values post-intervention that approximated our postural sway indices in non-concussed children (27). Baracks et al found an inertial based sensor (different than our device-based measure) distinguished concussed from non-concussed college athletes with a high specificity (71%−98% using varying cut-points) but low sensitivity (33%−54%), consistent with our measures at the ideal cut-point for the mCTSIB (sensitivity 37%, specificity 88%) (13).

Our study found the clinical measures performed similar to or slightly better than the device-based measure. Similar to our study, King et al attempted to compare the discriminatory ability of a standard clinical test (mBESS) with a device-based test (an inertial sensor while performing the mBESS) and found, contrary to our results, the inertial sensor led to an increased AUC (from 0.61 to 0.75) (17). These data were captured among college athletes who were seen more acutely than our patients (generally within two days), likely with a different baseline level of postural stability compared with the children evaluated in our study (3). Given their current cost and potential limitations for implementation outside of specialty clinical centers (including sidelines, training rooms, and primary care offices), our data suggest limited added value of the device-based assessment beyond clinical measures at this time.

We found that our two clinical assessments, the mBESS and the complex tandem gait, produced similar discriminatory ability when assessed as a composite score. Previous studies by Buckley et al (11) and Downey et al (12) have shown the mBESS to have broad sensitivity ranges (20%−71%) in collegiate athletes, however the AUC is not reported in either study for comparison to our measures. Oldham et al, in comparing the timed tandem gait to the mBESS in college athletes, found an AUC for the mBESS lower than that obtained in our study (Oldham: 0.535; current study: 0.70) (18). There may be several limitations to the mBESS itself, including a high number of false positives, as well as significant variance (5). In addition, we found in our study that one of the three tests used in the mBESS, the double leg stance, provided little useful information, with an extremely low sensitivity (5%), due to nearly all of the concussed patients performing the test without error. This finding suggests that this element may not essential to the mBESS and could potentially be removed to minimize patient testing time, though doing so does not increase the overall discriminatory ability (the AUC of the mBESS with the double leg stance removed was identical to the AUC with it included).

The tandem gait testing evaluated in this study differs from the tandem gait testing that is performed in the SCAT-5 (20,21). Rather than using timed testing, which has been shown previously to have a high false positive rate (7), we implemented complex tandem gait testing in this study. The need for complex gait testing was highlighted in a recent systematic review, which found inconsistencies in the findings of studies evaluating single gait tasks (such as timed testing) and emphasized the need to evaluate dual- and complex-gait tasks (16). While we did not directly compare complex to timed tandem gait in this study, the multifaceted nature of complex tandem gait makes it an appealing alternative to the timed version. Our data shows that complex tandem gait yielded a highly sensitive component (having the patient walk backward with eyes closed while evaluating for sway or errors), as well as a highly specific component (having the patient walk forward with eyes open while evaluating for sway or errors). Interestingly, when taken as a composite of all four tests, the Youden Index calculates an ideal cut-point (5) higher than the median number of errors for cases (3), suggesting a more clinically relevant measure may be utilizing the individual components, or, as shown by a similar AUC to the overall test, only evaluating forward eyes open and backward eyes closed as a composite measure. Given variability in baseline gait and balance abilities of children (as evidenced by the abnormalities on vestibular and oculomotor testing in non-concussed children) (28,29), the nature of a graded, complex gait task should assist in distinguishing balance deficits throughout the normal developmental spectrum. Other complex gait testing, such as dual-task gait, which integrates a cognitive task simultaneously with gait assessment, was not evaluated in this study, but has shown similar discriminatory promise as an additional method to distinguish concussed from non-concussed patients (30), especially in the subacute stage of recovery and in detecting persistent deficits (4,31,32). Finally, but perhaps most importantly, previous studies conducted these assessments in specialized sports medicine clinics. As the vast majority of concussions are initially evaluated outside of this specialized realm (33), integrating testing that can be performed by general practitioners without specialized equipment is of paramount importance to assessment of the concussed pediatric patient. We have previously demonstrated that our complex tandem gait has been readily adopted by both primary care providers and emergency medicine providers at our institution (34,35).

There are several limitations to our study. Although controls were junior varsity or varsity athletes from a single suburban high school, all enrolled students were required to participate in at least one sport every school year. Therefore, the non-concussed cohort represents a wide range of athletic skill level, which in fact may be more generalizable than studies of more athletic populations. Some cases were enrolled from the private suburban high school; however, most (86%) were enrolled from a concussion program at a tertiary referral center. The patients seen in this program may be more severely injured than general concussed patients seeking care with their primary care doctor. As cases were assessed an average of 11 days from injury, our study is not able to determine the ability of these tests to distinguish concussed patients immediately following injury, though our sensitivity analysis demonstrated similar results in the group of cases assessed within 7 days of their initial injury. While recruited from different centers, the sociodemographic characteristics of the subjects from the high school and the referral center were overall very similar. Although the clinical assessments used in this analysis included subjective reporting of errors and/or sway, we attempted to minimize this bias with standardized training of the study coordinators by a sports medicine specialist. Though we did not obtain inter-rater reliability for our two clinical assessments among the examiners in this study, previously, among non-sports medicine providers, we have shown high inter-rater reliability for the complex tandem gait (kappa >0.60 showing substantial agreement) (36). In addition, multiple prior studies have evaluated inter-rater reliability of the Balance Error Scoring System (BESS) with varied results (37,38), though a systematic review by Bell and colleagues concluded that the BESS has moderate to good reliability (39). One study evaluating the mBESS by Hunt and colleagues showed moderate to good reliability (intraclass correlation coefficient between 0.50 and 0.75) (40).

Overall, this study provides guidance on the clinical utility of three balance assessments, and provides supporting evidence that while device-based measures of balance are appealing, a clinical assessment can provide a similar, if not more effective, means of distinguishing concussed from non-concussed adolescents. In addition, when compared with the mBESS, our complex tandem gait shows similar overall discriminatory ability, with the added value of an individual component with high sensitivity to exclude injury, suggesting the need for a challenging component of balance testing, such as dual- and complex-gait tasks, to allow providers effectively rule out an injury. Future studies should further examine and compare these measures in a broader population (e.g., non-athletes, younger patients), as well as evaluate their discriminatory ability in the more acute timeframe following injury.