Reliability and validity of the sideways step test and its correlation with motor function after stroke

Abstract

[Purpose] This study investigated the intra-rater, inter-rater and test-retest reliability of the sideways step test (SST), its correlation with other indicators of stroke-specific impairment, and the cut-off count best discriminating subjects with stroke from their healthy counterparts. [Subjects and Methods] Forty-three subjects with chronic stroke and 41 healthy subjects older than 50 years participated in this study. The SST was administered along with the Fugl-Meyer motor assessment for the lower extremities (FMA-LE), the five-times sit to stand (5TSTS) test, the Berg Balance Scale (BBS), the movement velocity (MVL) by the limits of stability (LOS) test, the ten-metre walk (10mW) test, the timed “Up and Go” (TUG) test and the Activities-specific Balance Confidence (ABC) scale. [Results] The SST showed good to excellent intra-rater, inter-rater and test-retest reliability. The SST counts correlated with 5TSTS times, 10mW times, TUG times, and the FMA-LE and BBS scores. SST counts of 11 for the paretic leg and 14 for the non-paretic leg were found to distinguish the healthy adults from subjects with stroke. [Conclusion] The sideways step test is a reliable clinical test, which correlates with the functional strength, gait speed, and functional balance of people with chronic stroke.

INTRODUCTION

Stroke is one of the leading causes of functional impairment, resulting in permanent disability in 15 to 30% of its survivors¹⁾. For those who regain mobility, more than 80% have permanent impairment of walking²⁾. Walking is a complex motor task³⁾. Dynamic standing balance, strength, motor control and precise movement of the center of pressure (COP) are all required for achieving a normal walking gait. During walking, the COP shifts forward after heel strike and laterally to the stance leg during the stance phase. The COP then shifts forward and laterally to the opposite leg for the subsequent heel strike³⁾. Thus, the ability to maintain balance while undertaking a forward and lateral destabilizing force during self-generated limb movement is a prerequisite for walking and functional mobility^4,5,6,7).

Deficits in dynamic standing balance during external perturbation, as well as self-initiated limb movement, have been reported in previous studies of stroke survivors^{8, 9)}. The step test was developed to assess the dynamic standing balance of subjects with stroke¹⁰⁾. The step test was devised by Hill et al.¹⁰⁾, and it measures the number of times an individual is able to place one foot on and off a low step placed at the front within 15 seconds. The step test assesses the ability to stabilize the body over the stance leg while undertaking a forward destabilizing movement. However, there is a lack of reliable and valid outcome measures for assessing the ability to maintain dynamic standing balance while undertaking a lateral destabilizing movement.

The sideways step test (SST) is a modification of the step test¹⁰⁾. The difference between the SST and the ST is the direction of stepping. In the ST the subject is required to step up onto a step in the forward direction^{9, 10)}, whereas in the SST the subject steps up sideways. Thus, the SST challenges lateral stability and the results better reflect the ability of the stance leg to maintain dynamic standing balance while undertaking a lateral destabilizing force¹¹⁾. It also requires anticipatory postural adjustment and adaptive reactions, revealing the subject’s ability to maintain dynamic balance.

The aims of this study were: to determine the intra-rater, inter-rater and test-retest reliability together with the minimal detectable change (MDC) of the SST with subjects with stroke; to examine the relationship between SST results and stroke-specific impairment; and to determine the sensitivity of the SST and a cut-off score differentiating between subjects with stroke and age-matched healthy subjects.

SUBJECTS AND METHODS

Forty-three subjects with stroke aged over 55 were recruited from local self-help groups for patients with stroke via poster advertisement, as the incidence of stroke approximately doubles after the age of 55¹²⁾. Subjects with stroke were included if they: were older than 55 years; were at least 9 months post-stroke; were medically stable; were able to walk more than 10 meters with or without an assistive device; and had an Abbreviated Mental Test¹³⁾ score of 7 or above. Subjects were excluded if they had another neurological diagnosis in addition to stroke, had co-morbid disabilities, or had other musculoskeletal problems that might have affected mobility and the assessment protocol.

Forty-one healthy older adults aged over 55, comparable with those of the stroke survivors, were recruited from local community centers through poster advertisements. The healthy older adults were recruited in order to determine the cut-off scores of STT counts between healthy older adults and subjects with stroke. Healthy older adults were excluded if they had any condition which might have affected their mobility or the assessment protocol, such as uncontrolled diabetes mellitus.

All of the subjects gave their written informed consent before the assessment. Ethical approval for the study was obtained from the ethics committee of The Hong Kong Polytechnic University and the study was conducted following the guidelines of the Declaration of Helsinki.

The subjects with stroke were assessed in 2 separate sessions (session 1 and session 2) 3 to 10 days apart. The SST was conducted in both sessions and each time was assessed simultaneously by the same 2 independent raters (rater A and B) who were trained in stroke rehabilitation. The procedures are illustrated in Fig. 1.

Fig. 1.

Schematic diagram of the data collection and reliability estimation procedures of the sideways step test

To determine correlations between the SST results and the results of other stroke-specific impairment tests, tests were conducted in one of the two sessions by one of the raters, A or B. The tests included the Fugl-Meyer motor assessment for the lower extremities (FMA-LE)¹⁴⁾, the five-times sit to stand (5TSTS) test¹⁵⁾, assessment using the Berg Balance Scale (BBS)¹⁶⁾, the movement velocity (MVL) by limits of stability (LOS) test¹⁷⁾, the ten-meter walk (10mW) test¹⁸⁾ at a comfortable speed, the timed ”Up and Go” (TUG) test¹⁹⁾, and rating using the Activity-specific Balance Confidence (ABC) scale²⁰⁾. The order of testing was randomized by drawing lots. Two minutes rest was allowed between each assessment to avoid fatigue.

For the healthy subjects, the SST was administered once by either rater A or B in the same assessment period as the subjects with stroke. The data collected were used for the calculation of cut-off scores. All the tests were carried out in a university laboratory.

Descriptions of the tests.

Sideways Step Test (SST): The SST starts with the subject standing unsupported with the feet shoulder width apart, with a 10 cm high wooden step (35 cm wide and 65 cm deep) next to the foot to be tested. The subject is asked to repeatedly put their tested foot on the step and then back to the floor as quickly as possible in 15 seconds. The number of steps was counted by both rater A and rater B working independently. Only completed steps, with the whole foot stepping on the wooden step and then back on the floor, were counted. Walking aids are not allowed in this test. The test was stopped if the subject lost his or her balance or needed assistance with balance. After two practice trials, three timed trials were then conducted with each foot. A 30-second rest was allowed between trials to minimize the effects of fatigue. The test began with the non-paretic leg, and was followed by the paretic leg. Among the healthy subjects the dominant leg was tested first.

Fugl-Meyer Motor Assessment for the Lower Extremities (FMA-LE): The FMA-LE tests movement synergy, reflexes and coordination¹⁴⁾. The assessment consists of 17 items scored 0 to 2, and has a maximum score of 34. It has excellent inter-rater reliability for subjects with chronic stroke (ICC=0.93)^{21, 22)}.

Five-times Sit To Stand Test (5TSTS): The 5TSTS test assesses the functional muscle strength of the lower limbs¹⁵⁾. It has excellent intra-rater, inter-rater and test-retest reliability (ICC=0.97 to 0.99) for subjects with chronic stroke²³⁾. The subjects were asked to stand up and then sit down as quickly as possible 5 times from an armless chair (seat height, 45 cm) with their arms folded on the chest. After 2 practice trials, the total time needed was recorded 3 times, and the average time needed was used in the analysis.

Berg Balance Scale (BBS): The BBS assesses static and dynamic balance. It consists of 14 items rated using a 5-point scale, with a maximum score of 56¹⁶⁾. Excellent inter-rater and test-retest reliability (ICC=0.98 to 0.99) have been shown for subjects with acute stroke²⁴⁾.

Movement Velocity (MVL) of Limits of Stability Test (LOS): Movement velocity (MVL) is the average speed of the centre of pressure (COP) displacement in degrees per second when the subject tries to shift their COP toward a given target¹⁷⁾. It is measured by dynamic posturography, and in this study a SMART Balance Master^a system was used. The system comprises a computer screen which provides the subject with a visual representation of their COP’s movement, a pair of force platforms to monitor the COP, and an overhead body harness for safety. Previous studies of subjects with stroke have shown that the LOS test has good test-retest reliability (ICC=0.78–0.91) for patients with stroke^{25, 26)}.

In this study each subject was asked to wear the harness and stand on the platform facing the screen. They were then instructed to chase the target on the computer screen by transferring their body weight to move the cursor forward, right, backward and left, in that order. Only the right and left shift data were analyzed.

Ten-meter Walk Test (10mW): The 10mW test measures the time taken to cover 10 m at a self-selected walking speed on a 14-metre track, allowing 2 meters at each end for acceleration and deceleration. It has excellent reliability (ICC=0.94) for subjects with chronic stroke¹⁸⁾. The test was performed in triplicate and the average was used in the analysis.

Timed “Up and Go” Test (TUG): The TUG test was used to assess the subjects’ functional mobility¹⁹⁾. It has excellent test-retest reliability (ICC=0.95–0.96) for subjects with chronic stroke^{18, 27)}, and excellent inter-rater and intra-rater reliabilities (ICC=0.99) for frail elderly persons¹⁹⁾. The subjects were instructed to stand up from a chair, walk for 3 meters, turn around, walk back to the chair and sit down. After 1 practice trial, three trials were carried out and the average was used in the analysis.

Activities-specific Balance Confidence Scale: The ABC scale is a self-administered questionnaire which examines perceived confidence in maintaining balance while performing 16 ambulatory activities common in daily life using a 0 to 100% scale²⁰⁾. The Cantonese version of the ABC scale used in this study has been demonstrated to have very good inter-rater and test-retest reliabilities (ICC=0.850–0.990) when administered to frail elderly persons²⁸⁾.

Data analysis was performed with the aid of version 20 of the SPSS software package (IBM^©SPSS^©Statistics, IBM Corporation). The significance level was chosen as 0.05 for all the analyses unless otherwise specified. Descriptive statistics were compiled to evaluate the subjects’ demographic characteristics. The ICCs model 3 (ICC_3,1 and ICC_3,2) was used to calculate the degree of intra-rater and inter-rater reliability, respectively, as either raters or subjects were considered as random effects. The ICC model 2 (ICC_2,1) was used to calculate the degree of test-retest reliability as both raters and subjects are considered as random effect with single rating²⁹⁾. The results of the test-retest reliability and standard deviation were then used to calculate the standard error of measurement (SEM) and minimal detectable change (MDC) using the following formulas³⁰⁾ with the estimation based on the 95% confidence interval:

Where S_x is the standard deviation of the SST counts and r_xx is the reliability coefficient.

The correlations between the SST counts and the results of the stroke specific impairment tests were established using Pearson correlation coefficients when the data were normally distributed, or Spearman correlation coefficients if they were not. After Bonferroni’s adjustment, correlations were considered statistically significant when the p-value was greater than 0.0125 (0.05/4), as 4 outcome measures (FMA-LE, 5TSTS Test, BBS and LOS) were considered as the primary outcomes.

Receiver operating characteristic (ROC) curves were plotted, and the balance between sensitivity (true positives) and specificity (false positives) was examined²⁵⁾. The area under the ROC curve (the AUC) quantified the accuracy of using the SST as an assessment tool to distinguish healthy older adults from subjects with stroke based on their performance using a null hypothesis that the AUC equals 0.5²⁹⁾. The best cut-off score was sought using the Youden index for the trade-off between sensitivity and specificity³¹⁾.

RESULTS

The forty-three subjects with stroke (31 males, 12 females) had a mean age of 60.4 (SD=5.5) and the mean of the 41 healthy subjects (5 males, 36 females) was 61.6 (SD = 5.2). The demographic data and the outcome measures are presented in Tables 1–3.

Table 1. Mean values of the characteristics of the stroke patients and healthy elderly.

Parameters	Mean values
	Healthy (n=41)	Stroke (n=43)
Age (yrs)	61.6 ±5.2	60.4±5.5
Gender (M/F)	5 / 36	31 / 12*
Paretic side (R/L)	NA	23/20
Non-dominant side (R/L)	3/38	NA
Height (cm)	156.0±7.4	160.7±6.6*
Weight (kg)	59.0±9.1	65.9±8.6*
Body Mass Index (kg/m2)	24.3±3.5	25.5±2.6

Values are mean ± SD. R: right; L: left. *Indicates a difference significant at the 5% level of confidence between healthy elderly and subjects with stroke.

Table 3. Mean values of the outcome measures of the stroke patients.

Outcome measures	Paretic	Non-paretic
Sideways Step Test (counts)	7.4±2.5	8.7±2.4
Fugl-Meyer Motor Assessment − LE	23.6±4.0
Five-times Sit to Stand Test (sec)	15.5±4.6
Berg Balance Scale	52.7±2.9
Movement velocity (deg/sec)	65.5±21.4	77.3±12.3
Ten-meter Walk Test (sec)	12.9±4.2
Timed “Up and Go” Test (sec)	17.7±5.1
Activities-specific Balance Confidence Scale	74.8±14.3

Values are mean ± SD.

For the subjects with stroke, the mean SST counts of the non-paretic and paretic lower limbs as stepping legs were 8.7 (SD=2.4) and 7.4 (SD=2.5), respectively. For the healthy subjects, the mean SST counts of the dominant and non-dominant lower limbs as stepping legs were 16.3 (SD=3.7) and 15.9 (SD=3.8), respectively (Table 2).

Table 2. Mean values of the Sideways Step Test (SST) of the stroke patients in subjects with stroke and healthy elderly.

	Mean values
	Stroke (n=43)	Healthy (n=41)
Paretic / non-dominant side (counts)	7.4±2.5	15.9±3.8*
Non-paretic / dominant side (counts)	8.7±2.4	16.3±3.7*
p values	≤0.0001*	0.077

Values are mean ± SD. *Indicates a difference significant at the 5% level of confidence between the healthy elderly and stroke patients.

The SST counts of both lower limbs of the subjects with stroke were significantly fewer (p≤0.0001) than those of the healthy subjects. A significant difference in average SST count (p≤0.0001) was found between the paretic and non-paretic lower limbs as stepping legs of the subjects with stroke, but no such difference was found between the non-dominant and dominant lower limbs of the healthy subjects.

The data reveal that the SST has good to excellent reliability for subjects with chronic stroke (Table 4). The inter-rater reliability (ICC=0.99–1.00) was excellent, and the intra-rater reliability (ICC=0.89–0.92) and test-retest reliability (ICC=0.82–0.93) were both good. The MDC values were 1.9 and 2.7 for the paretic and non-paretic lower limbs as stepping legs, respectively.

Table 4. Reliability of the Sideways Step Test for subjects with stroke.

Reliability	ICC	Rater	Session	Mean values (counts)		ICC (95% CI)
				Paretic side	Non-paretic side	Paretic side	Non-paretic side
Intra-rater	ICC (3,1)	A	1	7.3±2.5	8.6±2.4	0.92 (0.88–0.96)	0.89 (0.82–0.93)
			2	7.5±2.5	9.7±2.6	0.89 (0.82–0.93)	0.92 (0.87–0.95)
		B	1	8.0± 2.4	8.9±2.4	0.91 (0.85–0.94)	0.89 (0.83–0.94)
			2	8.2±2.7	10.0±2.6	0.90 (0.84–0.94)	0.92 (0.87–0.95)
Inter-rater	ICC (3,2)	A-B	1	N/A	N/A	0.99 (0.99–1.00)	0.99 (0.98–0.99)
			2	N/A	N/A	0.99 (0.99–1.0)	0.99 (0.98–0.99)
Test-retest	ICC (2,1)	A	1–2	N/A	N/A	0.92 (0.79–0.96)	0.84 (0.54–0.93)
		B	1–2	N/A	N/A	0.93 (0.80–0.97)	0.82 (0.51–0.92)

Values are mean ± SD, or as otherwise indicated. ICC: intraclass correlation coefficient; CI: confidence interval

After applying Bonferroni’s adjustment, the SST counts were found to be negatively correlated with 5TSTS and TUG completion times (r=−0.54 to −0.72, p≤0.0001) of both lower limbs. Furthermore, the average SST counts of the paretic lower limbs demonstrated significant and moderate correlation with BBS ratings (r=0.51, p≤0.0001) and 10mW test times (r=−0.64, p≤0.0001). A fair, but still significant association was found between SST count of the non-paretic lower limb and BBS score and MVL toward the paretic side (r=0.45–0.48, p=0.003), and 10mW time (r=−0.45, p≤0.01). The average SST counts of both sides correlated with the FMA-LL results (r=0.41–0.44, p≤0.0001). The correlation results are summarized in Table 5.

Table 5. Spearman’s correlation of the Sideways Step Test counts with other outcome measures of stroke patients.

Outcome measures	SST of paretic side	SST of non-paretic side
	Spearman’s rho	Spearman’s rho
Fugl-Meyer Motor Assessment − LE	0.44*	0.41*
Five-times Sit to Stand Test (sec)	−0.62*	−0.58*
Berg Balance Scale	0.51*	0.45*
Movement Velocity (deg/sec)	0.28	0.48*
Paretic side	0.17	0.15
Non-paretic side
Ten-meter Walk time (sec)	−0.64*	−0.45*
Timed “Up and Go” time (sec)	−0.72*	−0.54*
Activities-specific Balance Confidence Scale	0.14	0.08

P: paretic side; NP: non-paretic side. *Indicates a significant difference after Bonferroni adjustment at (a p value ≤ 0.05/4; p≤0.01).

An average SST count of 11 of the paretic or non-dominant leg as the stepping leg (sensitivity = 90.7% ; specificity = 90.2%), and of 14 (sensitivity = 100% ; specificity = 78%) of the non-paretic or dominant leg as the stepping leg was found to best distinguish the two groups of subjects. The area under the receiver operating characteristic curve was 0.951 for the paretic and non-dominant lower limbs as stepping legs, and 0.943 for the non-paretic and dominant lower limbs as stepping legs.

DISCUSSION

This is the first published study to investigate the reliability and discriminatory power of the sideways step test, and the correlations between its counts and stroke-specific impairment assessments of subjects with stroke.

The subjects with stroke had an average step count on the paretic side (7.4±2.5) that was significantly lower (p≤0.0001) than that on the non-paretic (8.7±2.4) side, but no such difference was found in the healthy subjects (16.3±3.7 and 15.9±3.8) (Table 2). This finding is consistent with those of previous studies^{9, 10)}. Stepping sideways reflects the ability to maintain dynamic standing balance while undertaking a lateral destabilizing movement, which needs a certain level of strength, coordination and standing balance. As previous studies have demonstrated that weakness, spasticity, and coordination and balance deficits exist in the lower limbs after stroke^{2, 32, 33)}, SST performance of the paretic leg is expected to be poorer than that of the non-paretic side.

Comparison of the average SST counts of the two subject groups showed that the SST count was significantly (p≤0.0001) lower for both the paretic and non-paretic lower limbs of the subjects with stroke. This difference is consistent with previous findings^{9, 10, 27)} and is presumably the result of the decreased muscle strength of both lower limbs³⁴⁾. It might also be due to decreased speed of muscle activation and increased time to reach peak torque. Ipsilateral weakness following stroke appears multifactorial³⁵⁾. In fact, motor control of one limb is controlled by both brain hemispheres via the cortical spinal tract, so a lesion in one hemisphere may influence the limbs on both sides differently³⁵⁾. In addition, the impairment on the paretic side definitely hinders stepping sideways compared to the non-paretic side, as the paretic lower limb has to bear weight and maintain balance during the single-leg stance phase of each step.

Consistent with the results of the conventional step test, our results demonstrate that the SST has excellent inter-rater reliability, and generally good intra-rater and test-retest reliabilities for subjects with chronic stroke. The good to excellent reliability must result at least in part from the well-defined assessment protocol, standardized procedure and clear instructions. Sufficient rest between trials and between sessions is also be important. The MDC values obtained in our study were 1.9 and 2.7 for the paretic and non-paretic lower limbs, respectively. This indicates that changes in SST counts greater than these values is unlikely to be due to random variation in measurement³⁰⁾, but rather changes in the performance of the subjects²⁹⁾. This should be particularly useful for the estimation of subjects’ progress during rehabilitation.

FMA-LE Scores: The significant correlations demonstrated by FMA-LE scores and SST scores of the paretic and non-paretic lower limbs could be explained by the fact that FMA is a comprehensive quantitative measure of motor impairment following stroke²¹⁾, while the SST is a measure of functional balance requiring good lower limb motor function. Indeed, several items of the FMA-LE (tests IIa, IIb and VI) involve components of sideways stepping, so the correlation is not surprising.

5TSTS Times: Stepping sideways and upward needs functional muscle strength and balance ability as it involves single-leg stance and lifting of the lower leg. The 5TSTS test is a quantitative measure of functional strength of the lower limbs which has previously been shown to have a strong negative correlation with the BBS score³⁶⁾. The significant negative correlation observed between the SST counts and the 5TSTS test times (r=−0.583 to −0.620, p≤0.0001) is, therefore, not unexpected.

BBS Scores: SST counts significantly correlated with BBS scores, as the BBS assesses dynamic standing balance and it includes items which are similar to the SST protocol. For example, items 12–14 involve stepping, tandem stands and single-leg stands. In addition, BBS scores are known to correlate with the TUG test times^{19, 27, 37)}, possibly explaining the significant correlation between SST counts and BBS scores.

MVL: The major difference between the LOS test and the SST is the speed and base of support during weight shift. Subject performing the LOS test shift their COP in a double-leg stance position at a self-selected speed, while in the SST, they have to perform weight shift in the single-leg stance at the fastest possible speed. The different positions and speed requirement might explain the fair correlation observed.

Gait Velocity: The SST count of the paretic lower limb significantly correlated with gait velocity. A previous study demonstrated a significant correlation between gait velocity, balance performance and FMA-LE results³⁸⁾. This may explain the significant correlation of SST with gait velocity.

TUG times: The significant correlation between SST counts and TUG times (r=−0.544 to −0.715, p≤0.0001) can be explained by the fact that the TUG test is a functional balance assessment tool involving shifts of the COP in different directions while standing up, walking and turning. In addition, the TUG times of subjects with chronic stroke are known to be strongly associated with the amount of lateral displacement of the pelvis and asymmetry in single limb stance during walking³⁹⁾, which are also some of the impairments affecting sideways stepping.

ABC Scores: ABC scores do not correlate well with stair walking ability or walking performance^{40, 41)}. ABC scores are self-rated, subjective assessments of balance ability when performing ambulatory tasks, and subjects’ perception of balance may not reflect their actual skill or performance⁴²⁾.

This is the first published study to determine cut-off SST counts reliably distinguishing subjects with stroke from healthy subjects. A cut-off count of 11 steps for the paretic or non-dominant lower limb has a sensitivity of 90.7% and yields an AUC of 94.3%. The cut-off count of 14 steps for the non-paretic or dominant side has a sensitivity of 100% and an AUC of 95.1%. The discriminatory power is very satisfactory.

This study had several limitations. First, the SST protocol focused entirely on the number of steps successfully completed in 15 seconds; therefore, the quality of the action was not taken into account, and some compensatory strategies might have been overlooked. Second, the results only apply to subjects fulfilling the same inclusion criteria; they should not be too readily generalized to other populations with different demographic characteristics. Third, as no walking aid was allowed during the testing, only those subjects at a certain functional level could complete the test, and the results should not be applied to subjects with lower mobility.

The cut-off scores need to be interpreted with caution because of the unequal distribution of males and females in the two subject groups. Gender differences in muscle strength⁴³⁾ and the performance of functional tasks⁴⁴⁾ have been reported, and this might have affected the results. The sample size was small, and it might not have been of sufficient size to detect significant correlations with the stroke-specific impairment assessments. This study had 4 primary outcome measures, which resulted in a higher threshold for significance, and the need for more subjects to detect true correlations⁴⁵⁾.

No causal relationships could be established because this was a cross-sectional study and only functional strength of the lower limbs was measured. Future studies should look at hip and knee strength, spasticity and proprioception to investigate additional correlations with SST performance. The influence of step height should also be examined to optimize and standardize the SST. In addition, the investigation of the predictors of SST is worthy of further study as they could identify the key component affecting sideways weight-shift performance and guide stroke rehabilitation.