TEST‐RETEST CONSISTENCY OF A POSTURAL SWAY ASSESSMENT PROTOCOL FOR ADOLESCENT ATHLETES MEASURED WITH A FORCE PLATE

Abstract

Purpose/Background:

Postural control assessments can provide a powerful means of detecting concussion‐related neurophysiological abnormalities and are considered an important part of the concussion management processes. Studies with college athletes indicate that postural sway analyzed using complexity metrics may provide a sensitive and novel way to detect post‐concussion postural control impairments. The purpose of this study was to determine if a postural sway assessment protocol (PSAP) measured using a force plate system can serve as a reliable assessment tool for adolescent athletes.

Methods:

The short‐term and long‐term test‐retest reliability of the PSAP was examined in a group of adolescent female athletes under eyes open and eyes closed conditions. Detrended fluctuation analysis was used to evaluate the complexity of the times series data (i.e., degree of self‐similarity across time scales). Conventional measures of standard deviation and total path length (distance traveled by the center‐of‐pressure) were also assessed.

Results:

The complexity and conventional measures generally demonstrated good reliability coefficients for short‐term and long‐term test‐retest reliability with both eyes open and eyes closed conditions. Intra‐class Correlation Coefficient (ICC) values ranged from .38‐.90 The highest ICC values corresponded with the short‐term reliability for the eyes open condition, while the lower ICC values corresponded with the long‐term reliability for the eyes closed condition.

Conclusions:

The results of this study indicate that the PSAP demonstrated good short‐term and long‐term test‐retest reliability. In addition, no evidence of learning effects was elicited through this study. Future studies should further explore the validity and feasibility of the use of this protocol for different age groups, different types of athletes, and longitudinal evaluations of post‐concussion impairments.

Clinical Relevance:

This study provides preliminary support for the utility of a postural sway assessment protocol measured using a force plate for use with adolescent athletes.

Level of Evidence:

Level III

INTRODUCTION

In the context of sports, one injury known to cause impairments in postural control is sport‐related concussion.¹ Good postural control is necessary for purposeful movement and is associated with overall musculoskeletal and neurological health.^2,3 Concussions can disrupt the nervous system's ability to process and integrate sensory information which leads to difficulty with postural control.^1,4 Therefore, assessment of postural control is considered an important part of concussion management processes and is listed as one of the fundamental evaluation principles for the latest consensus statement regarding concussion.⁴

Emerging evidence indicates that postural control assessments can provide an important means of de‐tecting concussion‐related neurophysiological abnor‐malities.¹,^5‐7 The Balance Error Scoring System (BESS) is a popular post‐concussion postural control assessment tool for both clinical and research purposes.^4,8 The BESS consists of three different stances (double leg stance, single leg stance, tandem stance) performed on two different surfaces (stable and foam) for 20 seconds each while the subjects' eyes are closed. A trained rater observes the athlete and counts errors (e.g., falling, lifting foot, changing stance, opening eyes) as they occur. The sum of the number of errors recorded across all trials provides a total BESS score. The BESS is able to distinguish between healthy and concussed college athletes through the 3‐5 days following a concussion injury.^4,16 According to as systematic review published by Bell and colleagues in 2011, inter‐rater and intra‐rater coefficient reliability values have ranged from 0.60‐0.92 and 0.57‐0.85 respectively.⁸ The BESS has also been validated against the Sensory Organization Test in a sample of college athletes with significant correlations between the two tests observed for 5 of the 6 stances (r = 0.31‐0.79, p < 0.01).^8,9

Nonetheless, multiple studies have identified a number of the limitations associated with using the BESS for post‐concussion postural control evaluations. For example, serial administrations of the BESS can elicit learning effects.¹⁰ Performance on the BESS can also be affected by age/maturational factors.^11,12 For example, test‐retest reliability has been demonstrated to be worse for younger children as compared to older children.¹¹ Additionally, hip and core strength and endurance can potentially serve as confounding variables for BESS scores.¹³ As such, the BESS may be limited in terms of further understanding the role that postural control impairments play with specific post‐concussion symptoms and risk for prolonged recovery, particularly for younger athletes.^14,15

Several authors have suggested that a more sensitive and powerful way to detect concussion‐related postural control impairments may necessitate the use of force plate assessments, and in particular, complexity metrics applied to center‐of‐pressure (COP) trajectories.^5,6,16 Cavanaugh and colleagues found that the postural control deficits detected by complexity metrics may persist past the shorter period of time in which impairments are typically able to be detected by other postural control measures.^5,6,16 However, the full utility of force place assessments in post‐concussion assessment remains unclear due to an insufficient number of studies published in this area.^4,17

One way to further understand the value of these assessments is to evaluate if they are able to overcome some of the reliability and reliability‐related validity concerns associated with the BESS and other measures. Therefore, the purpose of this study was to determine if a postural sway assessment protocol (PSAP) measured using a force plate and designed specifically to capture post‐concussion impairments in adolescents can potentially serve as a reliable option for assessment of postural control in adolescent athletes. The authors hypothesized that the protocol would demonstrate good short‐term test‐retest reliability (STR) and good long‐term test‐retest reliability (LTR) for postural control evaluations of young athletes with minimal observable learning effects during serial administrations.

METHODS

Adolescent females (n =19) ages 14‐18 years involved in their high school's soccer program were prospectively recruited for this study during their offseason. All participants were screened in order to ensure subjects included for analysis did not have a previous history of concussion, other neurological conditions that might affect postural control, or musculoskeletal injuries of the back or lower extremities. The methods for this study were approved by the investigators' Institutional Review Board. Informed written consent was obtained from all participants and their parents prior to subjects' participation in the study. All rights of subjects were protected.

Estimates of reliability can vary depending on the type of reliability being analyzed.¹⁸ The testing procedures for this study were designed to provide an assessment of the test‐retest reliability for the protocol for two different conditions (eyes open and eyes closed) when the test is administered repeatedly within a short amount of time (STR) and at a later period of time (LTR). To evaluate STR, participants for this part of the study were randomly assigned to either the eyes opened condition group or the eyes closed condition group. Subjects performed three trials of quiet standing on the force plate with a self‐selected period of rest between each trial (approximately 1‐2 minutes). The first trial followed the condition of the group to which the subject was randomly assigned. The second trial consisted of the opposite condition, while the third trial was completed using the condition of the group to which the subject was randomly assigned. For example, a subject that was selected for the eyes open condition completed the first and third trials with their eyes open and the second trial with their eyes closed. To evaluate LTR, a subset of the original 19 girls returned to the laboratory again 120 days later to complete one additional trial with eyes open and one additional trial with eyes closed. Each participant's first trial for each condition on Day 1 was compared to their respective trials on Day 120.

The postural control trials consisted of center‐of‐pressure (COP) data collected on an AccuSway+ force plate (AMTI, Boston, Massachusetts) and Balance Clinic software (AMTI, Boston, Massachusetts) with a sampling rate of 100 Hz. Subjects were given instructions to stand as naturally as possible on the force plate with their barefeet together and arms relaxed at their sides (Figure 1). Subjects were encouraged to stand quietly without trying to prevent or create any unnatural sway patterns. Each trial lasted two minutes following the recommendations of the protocol proposed by Gao and colleagues.¹⁹

Figure 1.

Stance for force plate protocol.

Custom MATLAB (The MathWorks, Inc.) code (created by members of one of the author's laboratories) was used to compute the conventional measures of total path length (TPL), standard deviation in the anterior‐posterior direction (StdAP) and standard deviation in the medial‐lateral direction (StdML) as well as the complexity metric known as detrended fluctuation analysis (DFA) for the anterior‐posterior and medial‐lateral directions (described as DFA‐AP and DFA‐ML respectively). TPL quantifies the total distance traveled by the COP over the entire 2‐minute trial. StdAP and StdML quantify the spread of data points around the mean position in each direction. DFA is a non‐linear time series analysis technique that quantifies the extent to which variations in the data are “fractal” (self‐similar and correlated across time scales) and reveals the long‐range correlations in a time series via a metric called the Hurst exponent.^19‐21 This technique has been explained in detail elsewhere.^{19,20,22‐24} Intra‐class correlation coefficients were then calculated to determine the degree of association between participants' scores on each of the COP metrics for trial 1 and trial 2 for both STR and LTR using a two‐way mixed effects model and type consistency with SPSS (19.0 for Windows). Level of significance was set at p < 0.01. This yielded test‐retest reliability estimates for TPL, StdAP, StdML, DFA‐AP, and DFA‐ML for each condition (eyes open vs eyes closed) and each type of reliability (STR vs LTR).

RESULTS

The sample for STR consisted of a total of 19 participants (10 for the eyes open condition and 9 for the eyes closed condition). The sample for LTR consisted of 13 of the 19 original participants (7 from the eyes open condition group, 6 from the eyes closed condition group) who were available to come back for a retest 120 days later. Six of the original participants were unable to return to the laboratory on the specified day due to vacation or transportation conflicts. Details regarding subject demographics are reported in Table 1.

Table 1.

Demographic features of adolescent females in each trial subset

Trial	Age (years; mean ± SD)	Height (cm; mean ± SD)	Weight (kg; mean ± SD)
Eyes open STR (n=10)	15.5+1.27	164+0.06	59.2+10.1
Eyes closed STR (n=9)	15.3+1.11	161+0.07	59.1+6.09
Eyes open/Eyes closed LTR (n=13)	15.6+1.04	163+0.04	59.4+8.57

STR refers to short‐term reliability. LTR refers to long‐term reliability

Descriptive statistics for each COP metric for each trial and condition are reported in Table 2. With regard to STR, no statistical differences were found between results for trial 1 and trial 2 for either the eyes opened or eyes closed conditions (p < 0.01). The ICC values for the initial trials and the subsequent corresponding trials ranged from 0.77‐0.90 for the eyes open condition and from 0.69‐0.84 for the eyes closed condition. All ICC measured values were statistically significant per the ICC, and the 95% Confidence Intervals (reported in Table 3).

Table 2.

Descriptive statistics for each center‐of‐pressure metric and each trial

Trial	Metric	Trial 1 Results (mean ± SD)	Trial 2 Results (mean ± SD)
Eyes open STR (n=10)	TPL	127.49 ± 16.31	129.94 ± 20.10
	StdAP	0.19 ± 0.06	0.20 ± 0.06
	StdML	0.29 ± 0.13	0.26 ± 0.11
	DFA‐AP	1.42 ± 0.09	1.41 ± 0.09
	DFA‐ML	1.47 ± 0.11	1.44 ± 0.08
Eyes closed STR (n=9)	TPL	151.15 ± 21.43	156.17 ± 21.76
	StdAP	0.33 ± 0.11	0.34 ± 0.10
	StdML	0.32 ± 0.12	0.31 ± 0.10
	DFA‐AP	1.40 ± 0.06	1.39 ± 0.05
	DFA‐ML	1.39 ± 0.07	1.38 ± 0.09
Eyes open LTR (n=13)	TPL	151.15 ± 21.43	156.17 ± 21.76
	StdAP	0.33 ± 0.12	0.34 ± 0.10
	StdML	0.32 ± 0.12	0.31 ± 0.10
	DFA‐AP	1.40 ± 0.06	1.39 ± 0.05
	DFA‐ML	1.39 ± 0.07	1.38 ± 0.09
Eyes closed LTR (n=13)	TPL	146.47 ± 18.45	148.08 ± 23.80
	StdAP	0.32 ± 0.08	0.29 ± 0.09
	StdML	0.33 ± 0.11	0.32 ± 0.14
	DFA‐AP	1.41 ± 0.0	1.38 ± 0.05
	DFA‐ML	5 1.42 ± 0.06	1.43 ± 0.07

STR refers to short‐term reliability. LTR refers to long‐term reliability. TPL refers to total path length or the total distance traveled by the center‐of‐pressure. StdAP and StdML refer to the standard deviation of the variability about the mean location in the anterior‐posterior and medial‐lateral directions respectively. DFA‐AP and DFA‐ML refer to the degree of self‐similarity across varying windows of time for the anterior‐posterior and medial‐lateral time series respectively.

Table 3.

Center‐of‐pressure ICC values and 95% Confidence Intervals for test‐retest reliability for each center‐of‐pressure metric

Trial	ICC, 95% Confidence Interval
Eyes open STR (n=10)	TPL (0.90, 0.64‐0.97)*
	StdAP (0.83, 0.45‐0.95)*
	StdML (0.89, 0.62‐0.97)*
	DFA‐AP (0.80, 0.39‐0.95)*
	DFA‐ML (0.77, 0.31‐0.94)*
Eyes closed STR (n=9)	TPL(0.83, 0.41‐0.96)*
	StdAP (0.87, 0.53‐0.97)*
	StdML (0.81, 0.37‐0.95)*
	DFA‐AP (0.79, 0.32‐0.95)*
	DFA‐ML (0.69, 0.10‐0.92)*
Eyes open LTR (n=13)	TPL (0.84, 0.55‐0.95)*
	StdAP (0.74, 0.35‐0.91)*
	StdML (0.73, 0.33‐0.91)*
	DFA‐AP (0.67, 0.21‐0.89)*
	DFA‐ML (0.74, 0.35‐0.91)*
Eyes closed LTR (n=13)	TPL (0.67, 0.21‐0.89)*
	StdAP (0.79, 0.44‐0.93)*
	StdML (0.38, ‐0.2‐0.76)
	DFA‐AP (0.50, ‐0.09‐0.81)
	DFA‐ML (0.51,‐ 0.03‐0.82)

TPL refers to total path length or the total distance traveled by the center‐of‐pressure. StdAP and StdML refer to the standard deviation of the variability about the mean location in the anterior‐posterior and medial‐lateral directions respectively. DFA‐AP and DFA‐ML refer to the degree of self‐similarity across varying windows of time for the anterior‐posterior and medial‐lateral time series respectively.

^*Statistical significance, based on 95% Confidence Interval

Likewise, no statistical differences were found between the Day 1 trials and Day 120 trials for either the eyes opened or eyes closed conditions. (p > 0.01). The ICC values for these trials ranged from 0.68‐0.91 for the eyes opened condition and 0.66‐0.88 for the eyes closed condition. All ICC values were found to be statistically significant for the eyes opened condition. However, the ICCs for the eyes closed condition were only statistically significant for the TPL and StdAP measures.

DISCUSSION

When postural control is impaired, athletes may be at a substantially Level increased risk for a subsequent head injury or an injury to another part of the body.^25,26 Therefore, it is essential that clinicians have reliable and valid clinical tools at their disposal to help detect when post‐concussion postural control impairments are present in patients. With no statistically significant differences between STR trials, the results of this study indicate that minimal learning effects occurred with repeated administrations of the PSAP. In addition, in contrast to the BESS, the force plate measures in this study generally demonstrated higher reliability coefficients for test‐retest reliability with adolescent athletes.⁸ Notably, study samples for test‐retest evaluations of the BESS do not directly match the age of the participants for this study. However, a sample of slightly younger subjects yielded a test‐retest reliability coefficients of 0.56 for younger children (ages 9 to 11 years) and 0.68 for older children (ages 12‐14 years).¹¹ Likewise, the reliability coefficient for total BESS errors for a sample of slightly older subjects (mean age 20.42 years, standard deviation of 2.08 years) was 0.64. Collectively, these data support the hypotheses that the PSAP would demonstrate good short‐term and long‐term test‐retest reliability for postural control evaluations of adolescent athletes with minimal learning effects arising during serial administrations.

Results from previous studies have indicated that this type of force plate assessment may improve the sensitivity of post‐concussion postural control assessments.⁵ As further supported by this study, the use of force plate protocols for post‐concussion postural control assessments may yield additional benefits. For example, the use of force plates mitigates the subjectivity and rater reliability issues associated with an observer‐rated test such as the BESS. Furthermore, one of the reasons that the BESS and other observer‐rated tests may produce less reliable results and can be susceptible to additional confounding variables is that balance challenges that entail higher levels of neuromuscular control and strength are necessary in order to distinguish between healthy and impaired states.⁸ Force plate assessment protocols such as the one described here may be able to detect subtle impairments using much simpler postural control tests.²⁷ Therefore, performance on these tests may be less likely to be influenced by the confounding variables of age and hip strength/endurance that are known to affect performance on the BESS. Moreover, simpler postural positions may be less likely to exacerbate post‐concussion symptoms while the test is being administered.

Although the costs and practicality of having a force plate on a sideline or in a school facility may raise some concerns, the culture surrounding management of youth sport concussions may be somewhat different from that of older athletes. Many states have now passed legislation that requires youth athletes suspected of sustaining a concussion to be cleared by a health or medical professional before they are allowed to return to activities.²⁸ As many youth sports do not have medical personnel available on the sidelines, treatment is often sought in pediatrician offices, sports medicine clinics, or emergency departments. Therefore, the purchase and use of force plates as an assessment tool may become more feasible with the ability to offset costs through medical billing efforts spread across a larger population of youth athletes. In addition, the technology associated with these measures is gradually becoming more accessible with increased availability of cheaper, commercial options. For example, Clark et al., demonstrated good measurement properties between the Nintendo Wii Balance Board and laboratory force plates.²⁹

There are a number of novel aspects to this study. First, to the authors' knowledge, no study has investigated the test‐retest reliability of this type of force plate protocol in adolescent athletes. In particular, the times series/complexity metrics for post‐concussion postural control have only been tested so far in collegiate athletes. Therefore, this study is a first‐step toward establishing a line of research that evaluates post‐concussion postural sway force plate assessments with adolescent athletes. Future studies may benefit from further exploration of the relationships between scores on these postural control assessments and post‐concussion symptoms and recovery processes.

Nonetheless, this study does have several limitations. First, only healthy, female adolescents served as participants in this study. Likewise, there was a relatively small sample size. Therefore, the generalizability of these results to males, younger aged children, older athletes, and injured individuals may not be appropriate. Secondly, the LTR ICC's for Std‐ML, DFA‐AP, and DFA‐ML only ranged from 0.38‐0.51 for the eyes closed condition with ICCs that were not statistically significant. The reason for this is unclear. However, it may be that the lack of visual contributions and the narrow base‐of‐support was too challenging and ultimately compromised the reliability of the test for these measures. Therefore, the eyes open condition may be a more reliable and thus more optimal method of assessment. Finally, at this time, the validity of this force plate protocol has only been evaluated for college athletes. For example, Cavanaugh and colleagues have demonstrated that complexity metrics applied to postural control assessments can detect differences in postural control following a concussion beyond the typical 3‐5 window suggested for evaluation using other measures such as the BESS and Sensory Organization Test.⁶ Sosnoff et al, extended Cavanaugh's work and provided evidence that complexity metrics applied to postural sway data in a variety of conditions may provide a way to capture deficits that persist months to even years following a concussion.³⁰ Future studies should investigate the sensitivity and specificity of postural sway complexity measures with respect to post‐concussion postural control impairments with direct comparisons to the BESS in children and adolescents younger individuals.

CONCLUSION

This study demonstrated that postural sway assessments measured using a force plate may provide a reliable alternative to non‐instrumented assessments of post‐concussion postural control such as the BESS. Future studies examining the use of force plates comparing healthy children and adolescents to those with a recent concussion are warranted.