REPRODUCIBILITY OF THE MODIFIED STAR EXCURSION BALANCE TEST COMPOSITE AND SPECIFIC REACH DIRECTION SCORES

ABSTRACT

Background

The mSEBT is a screening tool used to evaluate dynamic balance. Most research investigating measurement properties focused on intrarater reliability and was done in small samples. To know whether the mSEBT is useful to discriminate dynamic balance between persons and to evaluate changes in dynamic balance, more research into intra- and interrater reliability and smallest detectable change (synonymous with minimal detectable change) is needed.

Purpose

To estimate intra- and interrater reliability and smallest detectable change of the mSEBT in adults at risk for ankle sprain.

Study Design

Cross-sectional, test-retest design

Methods

Fifty-five healthy young adults participating in sports at risk for ankle sprain participated (mean ± SD age, 24.0 ± 2.9 years). Each participant performed three test sessions within one hour and was rated by two physical therapists (session 1, rater 1; session 2, rater 2; session 3, rater 1). Participants and raters were blinded for previous measurements. Normalized composite and reach direction scores for the right and left leg were collected. Analysis of variance was used to calculate intraclass correlation coefficient values for intra- and interrater reliability. Smallest detectable change values were calculated based on the standard error of measurement.

Results

Intra- and interrater reliability for both legs was good to excellent (intraclass correlation coefficient ranging from 0.87 to 0.94). The intrarater smallest detectable change for the composite score of the right leg was 7.2% and for the left 6.2%. The interrater smallest detectable change for the composite score of the right leg was 6.9% and for the left 5.0%.

Conclusion

The mSEBT is a reliable measurement instrument to discriminate dynamic balance between persons. Most smallest detectable change values of the mSEBT appear to be large. More research is needed to investigate if the mSEBT is usable for evaluative purposes.

Level of Evidence

Level 2

BACKGROUND

Ankle sprains are common in young adults, participating in competitive, court and team sports such as soccer, volleyball, tennis, hockey and running.^{1, 2} Patients with ankle sprains suffer from pain, limitations in activities and participation restrictions, resulting in high costs for individual sport participants and society.^1,3–5 Considering the impact of ankle sprains, injury prevention is important in sports. Dynamic balance deficits, defined as limitations in the ability to hold a stable base of support while making purposeful movements like reaching,⁶ are related to an increased risk for ankle sprain.^7–11 For the development of effective prevention programs, a screening tool to detect and evaluate persons with dynamic balance deficits is needed.

The modified Star Excursion Balance Test (mSEBT) is a screening tool, widely used by physical therapists to detect dynamic balance deficits, and to evaluate dynamic balance improvement in individuals after following a preventive training program.^7,12–15 The mSEBT measures dynamic balance, while a person maintains balance on a single leg and simultaneously reaches as far as possible with the other leg along the reach line of three directions (anterior, posteromedial and posterolateral). Originally composite scores (sum scores of the three directions) of the mSEBT were used as an index for dynamic balance. Recently, there is emerging evidence that specific reach directions of the mSEBT correlate with some specific lower extremity impairments.^7,16–18 Therefore, scores of separate reach directions as well as composite score are used as indices of dynamic balance.

To determine whether the mSEBT is a reliable measurement instrument to evaluate individual dynamic balance performance, insight into measurement properties of the test is necessary. For diagnostic purposes reliability (the degree until which the mSEBT can differentiate balance performances between persons) is an important parameter. For evaluative purposes, information about measurement error and the minimal amount of change above measurement error (smallest detectable change synonymous with minimal detectable change) is needed to know whether changes in performance should be attributed to measurement error or to true changes in performance.¹⁹ In the current study reliability, measurement error and smallest detectable change (SDC) are referred to under the umbrella term reproducibility. For practitioners and researchers it is important to gain insight in reproducibility of the mSEBT under several conditions. Namely, when the mSEBT is used by different raters (interrater), and by the same raters (intrarater) on different occasions. Intra- and interrater reliability of the mSEBT were investigated in different studies and seem to be good to excellent.^6,20–24 However, commonly identified flaws in these studies were small sample sizes, absence of a clear measurement protocol or missing information regarding measurement error and SDC values. Information about intra- and interrater reproducibility, based on high quality studies is needed to confirm intra- and interrater reliability of the mSEBT and to investigate whether the mSEBT is useful to evaluate changes in performance in persons at risk for ankle sprain over time. Therefore the aim of the current study is to estimate the intra- and interrater reproducibility in healthy young adults participating in sports at risk for ankle sprain.

METHODS

Participant characteristics

A convenience sample of volunteers was recruited in the Netherlands at Utrecht School of Medicine, University Medical Center Utrecht, Utrecht University; and at University of Applied Sciences Utrecht; and from researchers’ personal networks. Students and workers from Utrecht School of Medicine and University of Applied Sciences Utrecht were informed about this research and asked to participate via e-mail, and their digital learning environment. Furthermore, friends and colleagues of the researchers were informed about the research and asked to participate. Included were healthy persons between 18 and 30 years old who participated at least once a week in a selection of sports with increased risk for ankle sprain. Based on the number of ankle injuries treated in emergency departments, the following sports were defined as sports with increased risk for ankle sprain: running, soccer, volleyball, futsal, hockey, badminton, tennis, martial arts, handball, squash, gymnastics, korfball, basketball.²⁵ Excluded were persons who reported lower extremity injury within the past twelve months, cerebral concussion within the previous three months, history of prior ankle surgery, and history of neurological disorders affecting balance.

Participant characteristics such as age, gender, dominant leg (preferred kicking leg), history of ankle sprain, participation in sports with increased risk for ankle sprain and sport frequency were collected. The study was approved by the University Medical Center Utrecht Medical Ethics Committee, METC-protocol number 13-156/C. Informed written consent was obtained from all subjects prior to participation in the study and the rights of the subjects were protected throughout the study.

Rater characteristics

Four physical therapists were randomly chosen from a group of 10 physical therapists that participated in the master program Physical Therapy Science at University Medical Centre Utrecht and were selected by convenience sampling. The chosen physical therapists were trained during two standardization sessions, each session took two hours. During the standardization sessions a specifically developed measurement protocol was discussed and practiced. This protocol also included verbal instructions (test instructions and encouragements) given to the participant.

Measurement procedure and protocol mSEBT

A cross-sectional, test-retest design (rater 1, rater 2, rater 1) was used to examine the intra- and interrater reproducibility of the mSEBT (Figure 1). The four raters were rotated in two couples of two raters with one test couple simultaneously measuring two participants (Figure 1). Each participant performed three test sessions by two raters (session 1, rater 1; session 2, rater 2; session 3, rater 1) within one hour. To ensure blinding of raters for previous measurement, forms with the recorded data were collected after each test session (by a research assistant). In order to prevent missing items all data were checked by a research assistant before participants left the test location.

Figure 1.

Study design used to investigate reproducibility of the mSEBT.

Before testing was started, participants viewed an instructional video, which demonstrated the test and the testing procedure, followed by four practice trials in each of the three reach directions on each leg.²³ Participants performed the mSEBT standing at the center of a grid laid on the floor with three reach lines in the form of an Y. The three reach lines were labeled in relation to the stance leg as anterior (A), posteromedial (PM) and posterolateral (PL) directions, with two angles of 135 ° (between the A and PM line and between the A and PL line) and one angle of 90 ° (between the PM and PL line) (Figure 2). The lines were constructed with standard tape measures and transparent tape on the floor. For the anterior reach direction participants were standing with the most distal part of the big toe at the cross of the Y at the beginning of the anterior tape measure. For the posterior reach directions participants were standing with the most posterior part of the heel at the cross of the Y at the beginning of the posterior tape measures,⁶ were barefooted and asked to place both hands on the hips. Participants were instructed to reach as far as possible along each of the three reach lines, make a light touch on the line with the most distal part of the big toe and return the reaching leg back to the center while maintaining a single-leg stance with the other leg. Participants performed three trials in each direction on each leg. The test started with the right leg as stance leg followed by the left leg in successively the A, PM and PL reach directions. Ten seconds of rest were provided between the different trials of one reach direction. A trial was discarded and repeated if participants (1) took weight on the reaching foot; (2) failed to bring back the reaching foot to the starting position without losing control; (3) failed to keep both hands on hips; (4) failed to keep the stance foot at the same place; or (5) failed to keep the forefoot or heel of the stance foot on the floor.^6,23,26 A maximum of six attempts per reach direction was allowed to obtain three valid scores. The three valid scores were averaged and used to calculate normalized mSEBT scores. The rater recorded scores of each trial in each reach direction in centimeters.

Figure 2.

Participant performing the mSEBT in the three reach directions that are labeled in reference to right stance leg.

Calculating normalized mSEBT scores

To normalize reach distances the participant's leg length was measured from the anterior superior iliac spine to the distal tip of the medial malleolus with the participant lying supine.^26,27 Right and left leg length were measured two times and averaged. A standard tape measure was used to quantify the distance in centimeters.

For each reach direction, the mean out of the scores of three trials was calculated and normalized. Normalization was accomplished by dividing the mean reach distance by the participants (stance) leg length and then multiplying by 100%.²⁷ In order to calculate a composite score, the mean of the three normalized reach direction scores was calculated.

Data analysis

According to the recommendations of the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) group and a sample size calculation as described by Giraudeau and Mary, at least 50 participants should be included in the study.^28,29 The COSMIN initiative aims to improve the selection of health measurement instruments. As part of this initiative, the COSMIN group developed a critical appraisal tool (a checklist) containing standards for evaluating the methodological quality of studies on the measurement properties of health measurement instruments (http://www.cosmin.nl).

Descriptive statistics including frequencies, percentages, means, and standard deviations were used to describe the research population. Measurement properties (intra- and interrater reliability, measurement error and SDC values) were estimated according to the recommended methods of the COSMIN group.³⁰

Reliability

Reliability was assessed with the intraclass correlation coefficient (ICC). Analysis of variance was used to calculate ICC_consistency (model 3.1) for intrarater reliability, and ICC_agreement (model 2.1) for interrater reliability.^19,31,32 Interpretation of ICC scores was as following: Poor reliability ICC < 0.40, Fair reliability ICC 0.40 - 0.70, Good reliability ICC 0.70 – 0.90 and Excellent reliability ICC > 0.90.³³

Measurement error

Measurement error (intra- and interrater agreement) was investigated calculating Bland and Altman 95% limits of agreement (LoA). LoA illustrates the range and magnitude of the differences in measurements between or within raters.³⁴

Smallest detectable change

SDC refers to the smallest amount of change, which falls outside the measurement error of the measurement instrument.¹⁹ To calculate SDC values, first intra- and interrater measurement error were expressed in the units of measurement error: standard error of measurement agreement (SEM_agreement). Analysis of variance was used to calculate SEM_agreement.^19,31 SDC was based on SEM and calculated using the following formula: SDC = 1.96*2*SEM.¹⁹

All statistical analyses were performed using SPSS version 20 (SPSS Inc., Chicago, IL). All data entry was double checked by another researcher. In case of missing data, no further actions were taken and available data were analyzed when missing data remains below 5%.

RESULTS

Participant characteristics

Sixty-six adults between the ages of 18 to 30 years signed up to participate. Nine did not met the inclusion criteria and two did not show up. Therefore, 55 participants (62% female) were included in this study, with a mean age of 24.0 (SD 2.9) years; a mean leg length of 90.9 (SD 6.4) cm, and quarter of participants had a history of ankle sprain. Participants performed sports with an increased risk for ankle sprain on average 2.3 (SD 1.2) times a week. See Table 1 for participant characteristics.

Table 1.

Participant characteristics (N = 55).

Characteristics
Men / Female, n/n (%/%)	21/34 (38/62)
Age (y), mean (SD)	24.0 (2.9)
Right/left leg length (cm), mean (SD)	90.9 (6.3)/ 90.9 (6.4)
Dominant leg right / dominant leg left, n/n (%/%)	51/4 (92.7/7.3)
History of ankle sprain, n (%)	14 (25.5)
Weekly frequency participation in risk sports, mean (SD)	2.3 (1.2)
Participation in risk sports, n (%)
Running	36 (65)
Soccer	10 (18)
Volleyball	6 (11)
Futsal	5 (9)
Hockey	3 (5)
Badminton	3 (5)
Tennis	3 (5)
Martial arts	2 (4)
Handball	2 (4)
Squash	2 (4)
Gymnastics	1 (2)
Korfball	1 (2)

Intrarater and interrater reliability and agreement parameters were assessed on 55 pairs of observations by four physical therapists (3 males, working experience ranged from six months to 11 years). Missing data was below 2%. Therefore, no further actions were taken and available data were analyzed.

Reliability

Intrarater reliability for both the right and the left leg was good to excellent (ICC's for the right leg ranged from 0.87 (PM and PL) to 0.91 (Composite and A) and for the left leg from 0.89 (A) to 0.93 (Composite)).

Interrater reliability for the right leg was good to excellent (ICC values ranged from 0.87 (PM and PL) to 0.92 (A)). For the left leg excellent interrater reliability was found for all separate direction scores as well as the composite score (ICC values ranged from 0.92 (PL) to 0.94 (Composite)) (See Table 2).

Table 2.

Intra- and interrater reliabilty and agreement for composite and separate reach direction scores of the mSEBT (N = 55).

		Intrarater			Interrater
Direction	Mean (SD)	ICC*	SEM	SDC	ICC*	SEM	SDC
		Consistency	Agreement	Agreement	Agreement	Agreement	Agreement
Right leg
Composite score	72.3 (2.9)	0.91	2.6	7.2	0.91	2.5	6.9
Anterior	65.3 (5.4)	0.91	1.7	4.7	0.92	1.6	4.4
Posteromedial	78.0 (9.6)	0.87	3.9	10.7	0.87	3.7	10.3
Posterolateral	73.6 (11.7)	0.87	4.6	12.7	0.87	4.4	12.3
Left leg
Composite score	73.0 (7.3)	0.93	2.2	6.2	0.94	1.8	5.0
Anterior	66.0 (5.4)	0.89	0.7	1.8	0.93	1.6	4.4
Posteromedial	80.0 (8.8)	0.90	3.0	8.4	0.93	2.5	6.9
Posterolateral	73.1 (10.8)	0.91	3.9	10.8	0.92	3.2	9.0

^*All values except ICC are normalized excursion reach distance (reach distance / leg length x 100%).

Measurement error

Intrarater LoA for the composite score of the right leg demonstrated a mean difference in normalized scores within raters of -1.4%. For the left leg the mean difference in composite score within raters was -1.6% (See Table 3).

Table 3.

Mean differences in normalized reach direction scores and 95% limits of agreement within and between raters.

		Intrarater			Interrater
Direction	Mean (SD)	Mean difference within raters	Upper bound of 95% LoA	Lower bound of 95% LoA	Mean difference between raters	Upper bound of 95% LoA	Lower bound of 95% LoA
Right leg
Composite score	72.3 (2.9)	−1.4	5.4	−8.1	−0.3	6.6	−7.2
Anterior	65.3 (5.4)	−0.2	4.5	−5.0	0.3	4.7	−4.2
Posteromedial	78.0 (9.6)	−2.0	8.0	−12.0	−0.4	9.9	−10.8
Posterolateral	73.6 (11.7)	−2.1	10.0	−14.2	−5.0	8.7	−18.7
Left leg
Composite score	73.0 (7.3)	−1.6	3.8	−7.0	−2.8	4.0	−5.6
Anterior	66.0 (5.4)	−0.2	5.0	−5.4	−0.2	4.4	−4.5
Posteromedial	80.0 (8.8)	−1.6	6.3	−9.5	−1.6	6.4	−7.4
Posterolateral	73.1 (10.8)	−2.8	6.5	−12.2	−2.8	6.4	−10.1

LoA = limits of agreement

Interrater LoA for the right leg demonstrated a mean difference between raters for the composite scores of -0.3%. For the left leg the mean difference in composite scores between raters was -2.8% (See Table 3).

Smallest detectable change

Intrarater SDC values for the right leg ranged from 4.7% (A) to 12.7% (PL) and the value for the composite score of the right leg was 7.2%. Intrarater SDC values for the left leg ranged from 1.8% (A) to 10.8% (PL) and the SDC value for the composite score of the left leg was 6.2%.

Interrater SDC values for the right leg ranged from 4.4% (A) to 12.3% (PL) and the SDC value for the composite score of the right leg was 6.9%. Interrater SDC values for the left leg ranged from 4.4% (A) to 9.0% (PL) and the interrater SDC value for the composite score of the left leg was 5.0%. See Table 2 for all SDC values for the mSEBT.

DISCUSSION

In the current study, intra- and interrater reproducibility of the mSEBT were investigated in a sufficiently large sample of 55 healthy young adults participating in sports at risk for ankle sprain. The results show that the mSEBT has good to excellent intra- and interrater reliability despite the large variation in work experience (from 6 months to 11 years) between the four trained physical therapists who administered the test. Therefore, the current study results suggest that anyone can administer the test if trained properly. SDC values for the composite scores appear to be quite large. Changes in normalized composite scores of at least 6.9% for the right leg and 5.0% for the left leg are needed to observe a true change in performance of dynamic balance.

Earlier studies investigating reliability of the mSEBT gave indications that the mSEBT is a reliable measurement instrument.^6,20-24 However, these studies did not fully accomplish current methodological recommendations for clinimetric research as described in the COSMIN standard.²⁹ Other authors used small sample sizes or a poor description of the used measurement procedures. The current study performed according to the COSMIN standard, sufficiently powered, and used a transparent measurement protocol according to the recommendations of Gribble and Plisky,⁶ confirmed that the mSEBT is a reliable measurement instrument. Recapitulating, the results of the current study support previous findings, that the mSEBT is a reliable measurement instrument, which can be used in practice by one or more raters.

Questions arise whether the found SDC values are acceptable and the mSEBT is an appropriate tool to evaluate dynamic balance performance and risk for ankle sprain in individuals. SDC values for composite reach directions seem to be large and therefore small changes on the mSEBT might occur due to measurement error instead of actual improvement in dynamic balance. Currently, there is no research studying the required minimal improvement in dynamic balance to decrease risk for ankle sprain. To draw conclusions whether the mSEBT is usable tool for evaluative purposes, further research is needed to investigate the required minimal clinically important difference.

It should be noted that the reach distances obtained in the current study are relatively low compared to normative data from healthy subjects in earlier research. In other studies the PM reach direction scores are close to 90% or 100% from leg length,^16,22,35 where the scores in the current study and the study of McCann³⁶ are around 80% of leg length. The differences in reach distance scores compared with earlier results could be explained by several reasons. First, the population in our study is not completely comparable to the populations in the previous studies. In the current study, a large proportion of female participants, of recreational activity level, and mostly active in running participated, which may have resulted in relatively low reach distance scores. Second, differences in calculations and protocols used in the studies may have resulted in lower reach distance scores compared to previous studies. In the current study the average score was used for calculation, while in some other studies the highest score out of three trials was used.¹⁶ The protocol used in the current study was a very strict protocol (e.g. both hands had to stay on the hips, forefoot and heel had to keep contact with the floor and start position of the great toe and heel at the beginning of the tape) where other studies handled a less stringent protocol.^16,22,35 Moreover, the current authors also looked into whether fatigue could have affected the observed reach distances. None of the participants reported fatigue symptoms during the test sessions and post hoc analyses on the composite scores of the left and right leg showed no significant differences in composite scores between the different sessions. This indicates that the mSEBT performance was stable across the test sessions and there were no signs of a deterioration of performance due to fatigue within the hour testing.

In the current study, injury risk stratification based on the cut-off points found by Plisky et al. and Butler et al. does not seem useful.^16,17 Plisky found that in high school female basketball players athletes with a composite reach distance less than 94.0% leg length were 6.5 times more likely to have a lower extremity injury including ankle sprains.¹⁶ Butler reported that male college football players who scored below 89.6% leg length were 3.5 times more likely to get injured.¹⁷ However, risk stratification based on these cut-off points seems not plausible because almost all participants in the current study performed below the cut-off points and it is not likely that all of them had an elevated risk for lower extremity injuries. As earlier mentioned an important explanation for the lower test scores in the current study are the differences between the study populations. As suggested by Butler et al. there is need for developing population-specific cut-off points to screen athletes for injury risk.¹⁷

A remarkable finding of the current study is that SDC values for the left leg are systematically lower than SDC values for the right leg. A possible explanation for the difference in SDC values could be the effect of cross-education. Earlier research has shown that unilateral balance training was effective in improving neuromuscular reactions to perturbations during single-leg stance for the trained but also for the untrained leg.³⁷ In the current study each test was started with the right leg as the stance leg and therefore a cross-education effect due to training of the right stance leg may have caused a more stable performance of the leg. Furthermore, it could be hypothesized that because of right feet dominance in the study population (right leg dominance in 51 out of 55 participants) the left leg is mostly the stance leg during balance tasks and therefore is more trained in dynamic balance. This could have led to a more stable performance in maintaining balance for the left leg, resulting in lower SDC values. Although, previous studies do not show differences in mean scores for dynamic balance between the dominant and non-dominant leg,^38–40 no research has investigated differences in measurement error between the dominant and non-dominant leg. In summary, it is unclear whether the differences in SDC values between the left and right leg found in the current study should be attributed to the test sequence used in the measurement protocol or to differences between the dominant and non-dominant leg.

It should be noted that in the current measurement protocol the leg length test was part of the evaluation of the mSEBT. The leg length test is a potential contributor to measurement error. In our study protocol the raters measured the participant's leg length in each of the three test sessions. Since the mSEBT is used in practice to evaluate change in dynamic balance an investigator is not interested in change in leg length. Therefore, measuring leg length at baseline and using these values across the successive tests would be more efficient and decreases the measurement error between mSEBT measurements. Nevertheless, post hoc analyses on measurement error of the leg length measurements showed only a minor contribution to measurement errors. In the current sample inter-rater (average) leg length SEM agreement values were respectively 0.90cm and 0.93cm for the right and the left leg and for both legs ICC values for inter-rater reliability were higher than 0.99. Looking at intra-rater (average) leg length, SEM consistency values were respectively 0.46cm and 0.43cm for the right and the left leg and for both legs inter-rater reliability ICC values were 0.98.

When interpreting the data it has to be taken into account that the SDC values investigated in the current study are based on a sample of healthy young adults participating in sports with increased risk for ankle sprain. SDC values are population specific and may be different in a population suffering from ankle or other lower extremity injuries.^41,42 Therefore, the authors recommend that future research should study the measurement properties of the mSEBT using the same measurement protocol but in different populations at risk for lower extremity injuries and in populations suffering from different lower extremity injuries. Another consideration is that since each test was started with the right leg, the possible influence of a cross education effect could not have been ruled out. The authors therefore recommend that future research should focus on the influence of possible cross education effects, and on side to side differences in measurement error and SDC values, when using a controlled test order.

CONCLUSION

The mSEBT is a reliable measurement instrument used to discriminate dynamic balance between healthy young adults, participating in sports with increased risk for ankle sprain. For evaluative purposes changes of the composite score of at least 6.9% and 5.0% for the right stance leg and the left stance leg respectively are needed to feel confident that real improvement has occurred in an individual's dynamic balance. More research is needed to investigate if the mSEBT is usable for evaluative purposes of individual dynamic balance, and whether differences in SDC values are due to leg dominance or cross education effect.