The Convergent Validity of the SWAY Balance Application to Assess Postural Stability in Military Cadets Recovering from Concussion

Abstract

Background

Concussions are often accompanied by balance disturbances. Clinically accurate evaluation systems are often expensive, large, and inaccessible to most clinicians. The Sway Balance Mobile Application (SWAY) is an accessible method to quantify balance changes.

Purpose

To determine the known groups and convergent validity of the SWAY to assess balance after a concussion.

Study Design

Case-Control Study.

Methods

Twenty participants with acute concussion and twenty controls were recruited. At initial, one-week, and final return to activity (RTA) evaluations, all participants completed the Sports Concussion Assessment Tool (SCAT-5), and balance control measured by SWAY mBESS and NeuroCom Balance Master Sensory Organization Test (SOT). Mixed model ANOVAs were used to detect differences in SWAY mBESS and NeuroCom SOT scores with time (initial, one-week, final RTA) as the within-subjects factor and group (concussed, healthy) as the between-subjects factor. Spearman’s Rho correlations explored the associations between NeuroCom SOT scores, SWAY scores, SCAT-5 symptom scores, and time in days to final RTA.

Results

The sampled population was predominantly male and age (20 ± 1), and BMI differences were insignificant between groups. The SWAY did not detect differences between healthy and concussed participants and did not detect change over time [F(2,40) = .114, p = 0.89; F(2,40)= .276, p =0.60]. When assessing the relationship between the SWAY and the SOT, no correlation was found at any time point (r = -0.317 to -0.062, p > 0.05). Time to RTA demonstrated a moderate correlation with both SCAT-5 symptom severity score (r = .693, p < 0.01) and SCAT-5 total symptom score (r = .611, p < 0.01) at the one-week follow-up.

Conclusion

The SWAY mBESS does not appear to be a valid balance assessment for the concussed patient. The SWAY mBESS in patients with concussion failed to demonstrate convergent validity and did not demonstrate an ability to validate known groups. When assessing the time to final RTA, the one-week post-initial assessment SCAT-5 symptom severity and total scores may help determine the length of recovery in this population.

Level of Evidence

Level 3

INTRODUCTION

Concussions cause a range of symptoms and can be challenging to diagnose.^1–3 Over 450,000 military personnel have sustained some type of traumatic brain injury (TBI) since 2000, most of which were mild traumatic brain injuries (mTBI), also known as concussions.^4–7 However, this is likely an underestimate as over 50% of military concussions go unreported, similar to civilian sport-related concussions.^4–6 Concussions in collegiate athletes may be more common than in the general population, and their incidence may be even higher within a military cadet population due to the demands and risks taken.^8–10 Understanding these individuals’ symptoms better may help manage the concussed individual with the appropriate return to activity.¹¹

Decreased balance is one impairment that is often identified in patients after a concussion.^12–16 This is especially true for an acute traumatic event that needs to be diagnosed in military return-to-duty or athletic return-to-play scenarios.¹⁵ Unfortunately, the evaluation of concussion symptoms in most cases is rudimentary and consists of using non-instrumented tools to assess static balance.^16,17 The Balance Error Scoring System (BESS) is one of the most commonly used tests to evaluate impaired balance within the sports population.¹⁸ This test requires the patient to maintain their balance in different test positions while the clinician tallies technical “errors” for the duration of the test.¹⁸ Clinician-assessed BESS testing has demonstrated moderate to good reliability in the assessment of static balance; however, clinicians may lack detection of subtle changes in postural sway that do not result in visible errors scored in the BESS or may simply miss errors because of the multitude of movements occurring simultaneously.^19–22 Objective and instrumented assessments of static balance uses force plates or reflective markers.^17,23–27 One of these quantitative methods is the NeuroCom sensory organization test (SOT).²⁷ However, this method and other similar devices are neither practical for an on-field assessment nor cost-effective for most clinicians and first-line providers to implement.^17,28,29

Static balance may also be assessed in a portable and affordable manner through the use of mobile technology. The Sway Balance Mobile Application (SWAY) can be accessed on most smartphones or tablets. The SWAY Modified Balance Error Scoring System (mBESS) may be more clinically feasible, can be completed without significant training, and may be used in austere environments. In healthy participants, the SWAY mBESS demonstrated good test-retest reliability.³⁰ Many youth and college athletic programs use the SWAY for concussion baseline testing.³¹ However, no research has examined healthy young individuals after experiencing a recent concussion.

The primary purpose of this study was to determine the known groups and convergent validity of the SWAY to assess balance after a concussion. An additional objective was to determine the relationship between reported concussion symptoms and time to return to full activity.

METHODS

Participants

Twenty participants that suffered a recent concussion and a group of 20 matched healthy, non-concussed controls were recruited prospectively through convenience sampling within a military physical fitness center and physical therapy clinic. The study was approved by the Regional Health Command-Atlantic Institutional Review Board, and all participants provided written informed consent prior to participation.

Inclusion/Exclusion Criteria

Participants in the concussed group were cadets within three days of a concussion diagnosed by a medical provider. Participants in the healthy group were cadets without any recent concussion or lower extremity injury that would affect their balance. Participants who reported a history of lower extremity surgery involving the foot or ankle, concussion within the prior six months, or any disorders known to affect balance (Parkinson’s, BPPV, etc.) were excluded from both groups.

Study Design

This study was a case-control design separated into three aims. The first aim was to assess the ability of the SWAY application to detect differences in static balance between participants with a concussion as compared to a group of healthy controls (known groups validity). The second aim was to determine the relationship between the SWAY application and other established clinical measures, such as the NeuroCom SOT (convergent validity). The third aim explored relationships between assessed outcomes (SCAT-5, SWAY mBESS, NeuroCom SOT) and the time to final RTA.

Procedures

All participants completed an initial assessment including self-reported symptoms using the SCAT-5 and static balance using the SWAY and Neurocom SOT. The average duration of the assessment was approximately 30-45 minutes. Participants in the concussion group and control group completed the same assessments again at a one-week follow-up and final RTA follow-up. The final reassessment (RTA) was completed at the time the patient was cleared by their medical provider to return to full activity. Control participants attended a return to activity time point evaluation equivalent to their concussion-matched participant.

Outcome Measures

Static Balance: SWAYModified Balance Error Scoring System (mBESS). The SWAY mBESS protocol consisted of five test positions. Each position was performed for ten seconds with the participant’s eyes closed. The positions in order are feet together, tandem left foot forward, tandem right foot forward, single leg right, and single leg left, all with the participant holding the mobile device to their chest.^30,32 (Figure 1) A proprietary algorithm is used by the app to calculate a SWAY balance score. The score is derived from information collected within the mobile device’s inertial sensors. The SWAY mBESS scores can range from 0 to 100, with greater scores indicating better balance.

Figure 1. Demonstration of the Sway Modified Balance Error Scoring System Test Position: Single Leg Left Stance.

Static Balance: NeuroCom Sensory Organization Test. The participant is presented with six conditions of varying sensory input, including eyes open, eyes closed, sway surround, and sway support. This test is used to evaluate the participant’s use of somatosensory, visual, and vestibular input to maintain their balance. The NeuroCom SOT Balance Master is equipped with two 9- x 18in (23- x 46-cm) force plates.³³ The visual surroundings and the support surface rotate in the sagittal plane.³³ The primary outcome of the SOT is the equilibrium score, which ranges from 0-100. An equilibrium score is calculated based on how effectively the participant can maintain their theoretical limits of stability(established as a total of 12.5 degrees in the anterior-posterior direction).³³ Greater postural stability is indicated by decreased postural sway in the anterior-posterior direction and results in a higher equilibrium score.³³ The participant receives an equilibrium score of 0 for a trial if they fall or receive a negative value (sway more than the theoretical limit of 12.5 degrees).³³

Subjective Symptoms:The Sports Concussion Assessment Tool (SCAT-5). The SCAT-5 is a multi-item questionnaire used in the sports and athletic setting to assist in evaluating cognitive, sleep, affective, and physical symptoms.³⁴ The SCAT-5 is a responsive instrument that distinguishes normal baseline levels of neurocognitive function from a concussive injured athlete.^35,36 There is also evidence to suggest the SCAT-5 may be used as a mental health screening tool after a baseline concussion screen.³⁴ The SCAT-5 consists of an on-field and off-field assessment. The on-field assessment has four steps, some of which include evaluation of red flags, observable signs, memory assessment (Maddocks questions), and examination that includes a Glasgow Coma screen. The off-field screen consists of a six-step assessment that includes a subjective assessment of 22 symptoms to a final decision matrix after performing the multi-step process.

Statistical Analysis

Descriptive statistics for age, height, body mass, prior history of concussion, number of concussions, and time from concussive event to time of evaluation were analyzed with means and standard deviations calculated. T-tests and chi-square tests were used to compare differences between groups as appropriate with continuous and categorical data. Mixed model ANOVAs were used to detect differences in SWAY mBESS and NeuroCom SOT scores with time (initial, one-week, final RTA) as the within subjects factor and group (concussed, healthy) as the between subjects factor. Paired and independent t-tests, with Bonferroni corrections for multiple comparisons, were used for post hoc testing. Cohen’s d effect sizes were calculated, with 0.3 indicating a small, 0.5 indicating a medium, and 0.8 indicating a large effect.³⁷ Normality and skewness were assessed, and Spearman’s Rho correlations were used to explore the associations between NeuroCom SOT scores, SWAY mBESS scores, SCAT-5 severity / total symptom scores, and time to final RTA. Correlation coefficients were interpreted as low-fair (r = 0.25 - 0.49), moderate-good (r = 0.50 - 0.74), and strong (r ≥ 0.75).³⁷ The significance level for all analyses was set at α = .05, and all tests were two-tailed. All statistical analyses were completed using SPSS (version 28; IBM Corp, Armonk, NY, USA).

RESULTS

Forty cadets consented to participate in this study, 20 with a recent concussion and 20 healthy matched controls. The groups were equivalent at baseline, except that the concussion group reported a significantly greater frequency of prior history of concussions (Table 1). One participant in the concussion group did not complete their final data collection, and these data were carried forward with the last values recorded by the clinic. The same participant was cleared and returned to activity by an outside provider; that date was used as the final RTA date. Age, BMI, height, weight, and race distribution were not significantly different p>0.05 between groups, indicating successful matching. (Table 1)

Table 1. Participant demographics.

		Total	Group
		(N=40)	Post-concussion (mean ± SD)	Control (mean ± SD)	P- value PC / Control
Sex,	Male Female	32 8	16 4	16 4	na na
Age, mean (mean ± SD)		20 ± 1	20 ± 2	20 ± 1	p=0.60
Weight, lbs (mean ± SD)		174.60 ± 29.04	172.75 ± 31.71	176.45 ± 26.80	p=0.69
Height, in (mean ± SD)		69.67 ± 3.53	69.70 ± 3.61	69.65 ± 3.53	p=0.97
BMI, Kg/cm2		25.19 ± 3.10	24.88 ± 3.48	25.50 ± 2.74	p=0.53
Time from the concussive event to the evaluation, days (mean ± SD)
	Initial evaluation		1.8 ± 1.1	na
	One week		8.15 ± 1.3	na
	Final follow-up (RTA)		38.30 ± 43.3	na
# of prior concussions			*1.4 ± 1	*0.4 ± .75	p<0.001
% of prior history of concussion		24 (60%)	18 *(90%)	6 *(30%)	p<0.001

PC, Post-concussion; SD, standard deviation; lbs, pounds; in, inches; BMI, body mass index; cm², centimeters squared; Kg, kilograms; RTA, return to activity

*statistically significant differences noted between group demographics

Known Groups Validity

A mixed-model ANOVA examined the effect of a recent history of concussion on balance scores at three different times: initial, one-week follow-up, and final RTA. For the SWAY mBESS there were no significant main effects for time [F(2,40) = .121, p = 0.87)] or group [F(2,40)= .296, p =0.59)]. There was also no significant group-by-time interaction [F(2,40)= 1.284, p =.28] (Table 2, Figure 2).

Table 2. Group Means for SWAY & NeuroCom SOT.

	Group
	Post-concussion (mean ± SD)	Control (mean ± SD)
SWAY       Initial evaluation       One-week       Final (RTA)	83.49 ± 14.03 80.73 ± 13.61 80.41 ± 17.17	78.25 ± 14.73 79.46 ± 13.50 80.15 ± 13.13
NeuroCom SOT       Initial evaluation       One-week       Final (RTA)	70.90 ± 9.86* 77.25 ± 11.67* 81.85 ± 6.95	78.05 ± 5.88* 84.50 ± 4.94* 85.25 ± 4.01

SD, standard deviation; SWAY, Sway Balance Mobile Application; RTA, Return to Activity

SOT, Sensory Organization Test; RTA, Return to Activity

*statistically significant differences noted between group scores

Figure 2. Group Balance Mean Scores for SWAY and NeuroCom SOT.

SOT, NeuroCom Sensory Organization Test; RTA, Return to Activity

^* statistically significant difference noted between concussed and control group

^* statistically significant difference noted between initial and one-week follow-up

There was a significant main effect of time [F(2,40) = 34.59, p < 0.01] and group [F(2,40)= 8.25, p < 0.01] for the NeuroCom SOT. However, there was no group-by-time interaction [F(2,40)= 1.915, p =0.16]. In both groups, post-hoc testing revealed that scores increased significantly from the initial evaluation to one-week follow-up (p<0.01) but did not change significantly from the one-week follow-up to the final RTA (p=0.08). (Table 2, Figure 2) The control group scored greater at all time points except at the final RTA (Initial evaluation p<0.01, one-week follow-up p=.02, final RTA p=0.07).

Convergent Validity

No significant correlations were found between the SWAY mBESS and the NeuroCom SOT at any time point in the concussed group (r = -0.317 to -0.062, p > 0.05) and the control group (r = 0.275 to 0.387, p > 0.05) .

Relationships with RTA

Time to full return to activity was positively correlated with both SCAT-5 symptom severity score (r = .693, p < 0.001) and SCAT-5 total symptom score (r = .611, p = 0.004) at the one-week follow-up. Time to full return to activity was not related to SCAT-5 symptom severity score (initial- r =.239, RTA- r =.114) or SCAT-5 total symptom score (initial- r =.196, RTA- r =.132) (Table 3).

Table 3. Correlations table between time to RTA and SCAT-5 Total Symptom/Symptom Severity Score.

	Time to RTA
SCAT-5 total symptoms Initial Evaluation	.196
SCAT-5 total symptoms One-week evaluation	*.611
SCAT-5 total symptoms Final RTA	.132
SCAT-5 symptom severity Initial Evaluation	.239
SCAT-5 symptom severity One-week evaluation	*.693
SCAT-5 symptom severity Final RTA	.114

SCAT, Sports Concussion Assessment Tool;

RTA, Return to Activity

*= Statistically significant correlation noted

DISCUSSION

This study aimed to determine the SWAY’s ability to identify differences in static balance between participants with a recent concussion and matched healthy controls (known groups validity). Additional objectives were to determine the relationship between other commonly used clinical measures (SCAT-5, NeuroCom SOT) and the SWAY (convergent validity). The SWAY mBESS detected no differences in static balance in participants after a recent concussion when compared to healthy controls. SWAY mBESS scores also had no significant correlation with static balance clinical assessment tools such as the NeuroCom SOT, suggesting it may not be a valid assessment to interpret balance disturbances within a concussed population. Total symptoms and symptom severity at one-week (SCAT-5) were associated with time to final return to activity.

Previous authors within the literature have suggested that the SWAY application may be able to detect balance deficits in patients with diagnoses known to cause balance problems. In Parkinson’s patients, postural sway differences were identified using accelerometers similar to the SWAY.^38,39 Alkathiry and colleagues observed that accelerometers were a precise method to measure postural sway among adolescents with concussions.⁴⁰ Conversely, in this study, the SWAY could not distinguish between patients with concussion and healthy controls. These conflicting observations may be due to different accelerometer placements that do not account for differences in balance strategies; the accelerometer was placed on the low back at the level of the sacrum in the Alkathiry study versus the sternum for this study. Some participants in this study may have used an alternate postural control strategy or accelerometer stabilization method that made the sensor unable to detect changes but it could have possibly been detected with a placement similar to Alkathiry’s study.⁴⁰

Other authors have found that increased sway and potentially the SWAY application itself can discriminate between injured patients with neurological and musculoskeletal conditions and healthy controls.^38,39,41,42 When assessing Parkinson’s disease progression, Mancini et al. observed a progressive increase in the acceleration excursions for the Parkinson Diseased participant.^38,39 The Sway application score is correlated with increased postural sway in older participants (aged 50-71).⁴² The results of this study do not align with the findings of Mancini and other authors, which may be due to the participant’s ability to employ different balance strategies as suggested above. The SWAY application may not help discriminate balance deficits in patients with concussions.

Multiple studies have attempted to validate the SWAY compared to other valid and responsive balance tests.^43–48 One comparison was made between the BESS, BioDEX and SWAY balance, in which no significant correlation was found.⁴⁶ Another study that compared the SWAY and the modified BESS observed a strong negative correlation indicating ability to determine balance deficits in healthy older adults.⁴⁸ The SWAY application has also been compared to the Neurocom VSR sport in which a moderate inverse correlation was reported, but testing was also performed in a different manner than what is normally performed with the modified BESS test.⁴⁷ However, the SWAY has not been previously compared to a clinical gold standard balance assessment like the NeuroCom SOT and within the otherwise healthy and young concussed population. In contrast to the SWAY mBESS, this study did find that the NeuroCom SOT could accurately discriminate between patients with concussions and healthy controls. These findings supplement the available research supporting the NeuroCom SOT as a valid and reliable tool that assesses static balance and postural sway.^19,49,50 As other authors have noted, this research observed a learning effect using the NeuroCom SOT, suggesting that the healthy and concussed participants in this study improved similarly on this balance assessment over time as those in existing literature.^49,51,52 This learning effect is not unique to the NeuroCom SOT, as other authors have reported improvements on the BESS are observed with repeated balance testing up to 60 days after initial testing.^53,54 Overall, these findings align with previously published research and support the Neurocom’s continued value in discriminating between a healthy and a balance-challenged population.

Prior authors have suggested that the SWAY application may offer a valuable means of providing objective evaluations on the sidelines or in emergency departments.⁵⁵ Prior authors have also reported moderate to strong correlations between the traditional mBESS and the SWAY mBESS in healthy participants.^55,56 In this study of patients with a concussion, SWAY mBESS scores were not significantly correlated with the NeuroCom SOT scores. This aligns with one other published instrumented assessment balance study that found little to no relationship between the SWAY mBESS scores, BESS scores, and Biodex balance systems in a similar healthy college age population.⁴⁶ Contrary to the results found within this study, Mackensie et al. observed that the NeuroCom VSR sport demonstrated a moderate inverse relationship with the SWAY balance assessment in older adults.⁴⁷ However, these participants performed a special assessment using the Modified Clinical Test of Sensory Interaction and Balance (mCTSIB). Due to the conflicting evidence concerning the convergent validity of the SWAY mBESS and other measures, caution should be taken when using the SWAY mBESS in a post-concussed population.

In this study, total number of concussion symptoms and greater symptom severity scores on the SCAT-5 at one-week post-concussion were associated with longer recovery times. This finding is in alignment with Aderman et al., reporting that those who had elevated SCAT-3 $(\ge 2)$ total symptoms at the initiation of return to activity had 22% longer recovery times.¹¹ These results suggest a longer recovery for patients with concussion who have increased SCAT-5 total symptom and symptom severity scores at one-week. This may have some relationship to the implementation timeline of return to activity programming, and further research into matching the optimal exercise intensity to one-week SCAT-5 symptoms may be warranted.

Limitations

This study has several limitations. The population assessed was a relatively young, healthy, and active population, which may not generalize to other population groups. Contrary to the recommendations put forth by Bret et al., the current study did not include an initial familiarization test to perform two to three baselines prior to recording results because repeated testing is not ethical in a recently concussed population.⁵⁷ Additionally, the concussed population within this study may not have had severe enough balance problems at the initial evaluation. The impact of natural psychosocial factors on desired performance may differ between concussed and healthy participants and should be considered. This is because performance may be influenced, as the results could determine if the participant is ready to return to activity. A further subset evaluation of those patients with concussions who have self-reported balance deficits may be needed to identify if the assessment lacks this specification.⁴⁰ Future research on the SWAY application should focus on other assessment components, such as reaction time and memory, exploration of accelerometer placement, and attempting to identify an optimal return to full activity timeline should be emphasized.

CONCLUSION

These findings do not support the use of the SWAY mBESS for assessing static balance control as part of the acute assessment of and during the recovery from a concussion. The SWAY mBESS did not discriminate between healthy controls and patients with a concussion and was not correlated with a validated measure of balance in patients with a concussion. This may be due to the SWAY’s inability to detect balance strategies due to the proximal placement of the accelerometer. One-week follow-up assessment SCAT-5 total symptom and symptom severity scores may help determine the length of RTA in this population. More research is needed to determine the best clinical measure of balance in patients with a concussion and the optimal exercise intensity based on symptom severity at RTA.

DISCLAIMER

The opinions or assertions contained herein are the private views of the authors and are not to be construed as official or reflecting the views of the United States Army or Department of Defense.

Conflicts of Interest and Source of Funding

The authors declare no conflicts of interest. This work was funded by the Telemedicine and Advanced Technology Research Center (TATRC) at the U.S. Army Medical Research and Development Command through the Advanced Medical Technology Initiative (AMTI). The views, opinions and/or findings contained in this research/presentation/publication are those of the author(s)/company and do not necessarily reflect the views of the Department of Defense and should not be construed as an official DoD/Army position, policy or decision unless so designated by other documentation. No official endorsement should be made. Reference herein to any specific commercial products, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the U.S. Government.

ETHICS APPROVAL

Study was approved by the Naval Medical Center Portsmouth Institutional Review Board (RHC-A-20-051).