A prospective study to establish the minimal clinically important difference of the Mini-BESTest in individuals with stroke

Abstract

Objective:

To determine the minimal clinically important difference of the Mini-BESTest in individuals’ post-stroke.

Design:

Prospective cohort study.

Setting:

Outpatient stroke rehabilitation.

Subjects:

Fifty outpatients with stroke with a mean (SD) age of 60.8 (9.4).

Intervention:

Outpatients with stroke were assessed with the Mini-BESTest before and after a course of conventional rehabilitation. Rehabilitation sessions occurred one to two times/week for one hour and treatment duration was 1.3–42 weeks (mean (SD) = 17.4(10.6)).

Main measures:

We used a combination of anchor- and distribution-based approaches including a global rating of change in balance scale completed by physiotherapists and patients, the minimal detectable change with 95% confidence, and the optimal cut-point from receiver operating characteristic curves.

Results:

The average (SD) Mini-BESTest score at admission was 18.2 (6.5) and 22.4 (5.2) at discharge (effect size: 0.7) (P = 0.001). Mean change scores on the Mini-BESTest for patient and physiotherapist ratings of small change were 4.2 and 4.3 points, and 4.7 and 5.3 points for substantial change, respectively. The minimal detectable change with 95% confidence for the Mini-BESTest was 3.2 points. The minimally clinical importance difference was determined to be 4 points for detecting small changes and 5 points for detecting substantial changes.

Conclusions:

A change of 4–5 points on the Mini-BEST is required to be perceptible to clinicians and patients, and beyond measurement error. These values can be used to interpret changes in balance in stroke rehabilitation research and practice.

Introduction

Improving mobility and reducing the risk of falls are key rehabilitation goals after stroke. A recent expert panel recommended the Mini-Balance Evaluation Systems test (Mini-BESTest) ¹ as part of a minimum data set for measuring balance in adults. ² However, a key barrier to uptake of the Mini-BESTest is the limited information regarding its psychometric properties across clinical populations. In particular, the minimal clinically important difference, which reflects the amount of improvement that is perceived as important to patients and care providers, was highlighted by clinicians as an important facilitator for outcome measure uptake. ³ To date, Mini-BESTest research in stroke has been limited to reporting of reliability, validity, and ability to discriminate by fall status. ⁴ There is a need to identify the amount of change on the Mini-BESTest that can be understood as both real and important to patients and clinicians. The aim of this study was to determine the minimal clinically importance difference of the Mini-BESTest in individuals post stroke receiving outpatient rehabilitation.

Methods

This study is part of a single-center, prospective, longitudinal study in which physiotherapists and researchers have partnered in an Integrated Knowledge Translation paradigm, with the purpose of generating new data to improve the utility of the Mini-BESTest in people with stroke. Potential participants were screened by their treating physiotherapist and enrolled as appropriate. Eligibility criteria included: residents of Manitoba, Canada, with a valid provincial health care number who were referred for stroke; stroke patients referred for standing balance assessment; treating physiotherapist observation of instability in transfers or walking; history of falls in the past two years or self-reported problem with balance; lack of other neurological conditions that would significantly affect balance; ability to stand for 10 seconds without physical assistance; a minimum of two weeks of outpatient physiotherapy services anticipated; ability to communicate in and understand English; and ability to follow test instructions and provide informed consent. In line with guidelines for establishing the minimal clinically important difference of outcome measures, a sample size of 50 was targeted. ⁵

The rehabilitation program was tailored to individual patient needs based on an initial assessment and consistent with Canadian stroke best practice recommendations. ⁶ The program typically included repetition and practice of lower limb strength training, multiple components of balance with an emphasis on static and dynamic stability and anticipatory control in sitting and standing, progressive functional tasks including walking, and stair climbing, as appropriate. The need for gait aides and lower extremity orthotics was evaluated and devices were modified as necessary. There was a strong emphasis on patients completing home exercises on a daily basis. Parts of each session were used to progress to the home exercises. As part of patient education, fall prevention strategies were also reviewed. Physiotherapy sessions were one hour long and occurred one to two times a week.

On admission and discharge, treating physiotherapists administered the Mini-BESTest, a 14-item test of dynamic balance. The test is divided into four subcomponents (anticipatory postural adjustments, postural responses, sensory orientation, and dynamic gait) and scored on a three-level ordinal scale with a total score out of 28 points (higher scores indicate better balance). ¹ At discharge, both the patients and physiotherapists completed a global rating of change scale to rate the amount of change perceived in the patients’ balance on a 7-point scale: much better (7); a little better/meaningful (6); a little better/not meaningful (5); about the same (4); a little worse/not meaningful (3); a little worse/meaningful (2), and much worse (1). A 7-point global rating of change scale has shown adequate responsiveness, face validity, and clinical relevance with patient satisfaction.^7–9

Prior to the study, a comprehensive and established training program based on health professional behavior change theory ¹⁰ was administered to treating physiotherapists (N = 5) to increase knowledge, skills, and confidence for administering and scoring the Mini-BESTest, and a pilot inter-rater reliability trial including five patients with stroke was conducted. The Mini-BESTest was administered by the treating physiotherapist and the procedures were video recorded. All physiotherapists scored each patient’s performance from the video on two separate occasions – approximately seven days apart. Intraclass correlation coefficients of 0.95 and 0.93 were obtained for the total score of the Mini-BESTest on sessions 1 and 2, respectively.

A research assistant extracted data from the clinical record on admission date, age, diagnosis, stroke severity, co-morbidities, and fall history. The research assistant also administered the Chedoke McMaster Stroke Assessment, ¹¹ Timed Up and Go Test, ¹² and Activities-specific Balance Confidence scale, ¹³ to describe the sample population. The University of Manitoba Health Research Ethics Board approved this study (ethics reference number: HS19725), and all patients provided written consent. This study was funded by the Health Sciences Centre Foundation, Winnipeg (recipient KM Sibley) and supported in part by the Canada Research Chairs Program (recipients MK Beauchamp and KM Sibley).

Statistical analysis

Descriptive statistics were computed using measures of central tendency and dispersion. Box plot examination showed no outliers in the data. A paired t-test was used to determine if there was a difference in the mean change score between intake and discharge. Cohen’s effect size was also determined. ¹⁴ Ceiling and floor effects were examined for the Mini-BESTest total score and considered significant if ⩾15% of participants were at the upper or lower limits of the test score.

The minimal clinically important difference ¹⁵ was based on a combination of distribution and anchor-based approaches. For the anchor-based approach, we determined the mean absolute Mini-BESTest change scores for each answer on the global rating of change scale. For the distribution based-approach, we first determined the standard error of measurement calculated as $\mathrm{Sb}\times \sqrt{1-\mathrm{r}}$ , where Sb is the SD of our sample at intake, and r is the test-retest reliability coefficient. ³ Following, the minimal detectable change with 95% confidence, which refers to the smallest amount of change that falls outside of measurement error, was calculated as $1.96\times \sqrt{2}\times \mathrm{SEM}$ . Minimal clinically important difference thresholds were determined by triangulating the estimates from the mean change score on the global rating of change scale with the standard error of measurement for small changes, and with the minimal detectable change for substantial changes. We have used this approach successfully in our other studies.^16,17

The receiver operating characteristic curve was also used to determine the minimal clinically important difference using the Mini-BESTest as the diagnostic test for discriminating between improved and unchanged patients based on the global rating of change scale. The sample was divided into groups based on the global rating of change scores: patients with any improvement (global rating of change scale: a little better, much better) vs. those who were unchanged (global rating of change: about the same, a little better but not meaningful change). The data point closest to the upper left corner of the curve, the point that optimizes sensitivity and specificity, was chosen as the optimal threshold for detection of a change, with the area under the curve of the receiver operating characteristic reflecting the measure’s accuracy. An area under the curve of 0.7 or greater was considered acceptable accuracy by convention. ⁵

Weighted kappa was used to determine the inter-rater agreement between patient and physiotherapist global rating of change scores. To interpret the kappa statistics, the following criteria were used: no agreement = <0.0, poor agreement = 0.0–0.20, fair agreement = 0.21–0.40, good agreement = 0.41–0.60, very good agreement = 0.61–0.80, and excellent agreement = 0.81–1.00. ¹⁸ Spearman’s rho and Kendall’s tau-b were used for the correlations between anchor and physiotherapist global rating of change scores. Interpretation of the correlations were as follows: negligible = 0.0–0.10, weak = 0.11–0.39, moderate = 0.40–0.69, strong = 0.70–0.89, and very strong = 0.90–1.00. ¹⁹

Results

A flowchart of participant recruitment is provided in Figure 1. Fifty patients with stroke (mean (SD) age 60.8 (9.4) years, men = 34/50) participated in the study (Table 1). Treatment duration varied from 1.3 to 42 weeks (mean (SD) = 17.4 (10.6)) and on average patients completed 91% of the treatment proposed.

Figure 1.

Participant recruitment flowchart.

Table 1.

Sociodemographic and clinical characteristics of the participants (n = 50).

Characteristics
Age (years), mean (SD)	60.8 (9.4)
Men, n	34
Type of stroke, n
Hemorrhagic	11
Ischemic	10
Unknown	29
Right side affected, n	34
Time between stroke and admission (days), mean (SD)	124.4 (106.5)
Min/max	16/459
Treatment duration (days), mean (SD)	121.8 (74.3)
Min/max	9/294
Number of physiotherapy sessions offered, mean (SD)	17.3 (13.2)
Min/max	4/69
Number of physiotherapy sessions attended, mean (SD)	15.5 (11.8)
Min/max	4/58
CMSA
Foot, mean	5.2
Median	5
Leg, mean	5.6
Median	6
Comorbidities, n
Cardiac	43
Diabetes	15
Mobility and balance tests at intake, mean (SD)
Mini-BESTest total score	18.2 (6.5)
TUG time (seconds)	14.2 (8.7)
TUG dual-task time (seconds)	17.9 (10.7)
ABC scale score (%)	61.6 (21.2)
Mean change in the Mini-BESTest
Intake	18.3 (0.9)
Discharge	22.4 (0.7)

SD: standard deviation; CMSA: Chedoke-McMaster stroke assessment: 7-point scale impairment inventory, from most impairment (one flaccid paralysis) to no impairment (seven normal movement); TUG: timed up and go; Mini-BESTest: mini balance evaluation systems test; ABC: activities-specific balance confidence scale.

The average (SD) Mini-BESTest score at admission was 18.2 (6.5) and 22.4 (5.2) at discharge (effect size: 0.7) and this difference was statistically significant (P = 0.001). The Mini-BESTest showed no floor or ceiling effects. At admission, 2 (2.9%) participants scored 5, and 1 (1.4%) participant scored 27. At discharge, 1 (1.4%) participant scored 7 and 5 (7.2%) scored 27.

The interrater reliability of the global rating of change scale ratings between patients and physiotherapists showed no agreement, weighted kappa = 0.06 (probability value = 0.54,), 95% CI (-.131, .256). The correlation between global rating of change score and change in the Mini-BESTest was 0.33 (P = 0.020) for therapists, and 0.09 (P = 0.519) for patients.

The minimal clinically important difference for small (4 points) and substantial change (5 points) was similar for both patient and physiotherapist perception and aligned well with the distribution-based minimal detectable change (Table 2).

Table 2.

Meaningful change estimates for the Mini-BESTest based on patient and physiotherapist ratings using the global rating scale and distribution-based methods (n = 50).

Mini-BESTest component	Global rating of change scale collapsed, mean (SD)			SEM	MDC95	MCID small change*	MCID substantial change*
	About the same	A little better	Much better
Anticipatory patient	0.7 (0.8)	0.4 (0.7)	0.9 (0.9)	0.4	1.0	1	1
Therapist	0.4 (0.7)	0.8 (0.9)	0.7 (0.8)			1	1
Reactive patient	0.7 (1.2)	1.1 (1.6)	1.1 (1.2)	0.7	1.8	1	2
Therapist	0.1 (0.3)	1.2 (1.2)	1.2 (1.6)			1	2
Sensory patient	0.3 (0.7)	0.6 (1.0)	0.6 (0.7)	0.4	1.0	1	1
Therapist	0.1 (0.9)	0.6 (0.8)	0.7 (0.7)			1	1
Dynamic patient	2.1 (1.3)	1.4 (1.9)	2.0 (2.3)	1.0	2.6	2	3
Therapist	0.7 (1.7)	1.5 (1.4)	2.6 (2.3)			2	3
Total score patient	3.8 (2.3)	3.5 (3.8)	4.7 (3.9)	1.1	3.2	4	5
Therapist	1.3 (2.8)	4.2 (2.6)	5.2 (4.1)			4	5

SEM: minimal clinically important difference; MDC: minimal detectable change; MCID: minimal clinically important difference.

Global Rating Change scale collapsed: about the same (a little better/not meaningful and about the same); a little better/meaningful and much better.

^*The minimal clinically important difference value for small change was selected as the larger of either the anchor-based estimate for small change (about the same and a little better) or the distribution-based standard error of measurement, rounded to the nearest whole number. The minimal clinically important difference value for substantial change was selected as the larger of either the anchor-based estimate for substantial change (much better) or the distribution-based MDC₉₅, rounded to the nearest whole number.

A total of 23 (46%) patients perceived an improvement, of those, 6 (mean change = 4) patients improved based on the minimal clinically important difference for small change, and 17 (mean change = 8.1) for substantial change. The area under the curve for the Mini-BESTest based on the patients’ perception of change was not significantly better than chance. The receiver operating characteristic curve analysis for change in the Mini-BESTest total score according to physiotherapist perception showed adequate discrimination between patients that improved and those with small or no change (area under the curve = 0.774, 95% CI = 0.59–0.95; probability value = 0.011) with an optimum cut-off value of ⩾1 point (95% sensitivity; 56% specificity) (Figure 2). Table 3 shows a comparison of the number of patients that improved based on the MCID for small and substantial changes and based on the receiver operating characteristic curve.

Figure 2.

Receiver operating characteristic curve for discrimination between improved and unchanged participants on the Mini-BESTest according to physiotherapist perception.

Table 3.

Total number of patients classified as improved based on the minimal clinically important difference (MCID) value obtained from different methods (n = 50).

Methods	Cut-off value	Number of patients that improved
MCID substantial change	⩾5	17
MCID small change	4	6
*ROC curve	⩾1	44

^*Receiver operating characteristic curve, sensitivity 95.1% and specificity 55.6%.

Discussion

This is the first study to provide estimates of clinically meaningful change for the Mini-BESTest in people with stroke. We determined that a change of 4–5 points on the Mini-BESTest reflects the minimal clinically important difference for people with stroke attending outpatient rehabilitation. This information is critical for increasing the interpretability of the Mini-BESTest in stroke rehabilitation research and practice.

A challenge in identifying the optimal minimal clinically important difference value is deciding how much change is needed to be considered “important,” and for whom. Although there was poor agreement between physiotherapists and patients in their perception of change, it is reassuring that our minimal clinically important difference estimates for perceived important change are similar for both patients and physiotherapists. In terms of how much change is needed to constitute a minimal clinically important difference, we have provided estimates for both small change and substantial change to account for different applications and contexts. For example, if forming the basis of a power calculation for a clinical trial, the smallest possible difference threshold is preferable, whereas in clinical practice when evaluating change in individual patients, a more considerable change threshold can be selected. Thus, we have recommended a minimal clinically important difference of 4 points for detecting small changes on the Mini-BESTest in individuals with stroke, and 5 points for substantial change.

Based on the receiver operating characteristic curve, a change of 1-point on the Mini-BESTest was able to identify patients rated as substantially improved by physiotherapists, with acceptable accuracy but with relatively low specificity (55.6%). It is important to note that this change of 1-point in the Mini-BESTest total score is lower than the minimal clinically important difference estimates obtained based on the patient and physiotherapist-rated anchors, as well as below our distribution-based estimate for real change beyond measurement error. Therefore, we would caution against using this value obtained from the receiver operating characteristic curve as a minimal clinical important difference and suggest the values in Table 2 instead. Indeed, the fact that the standard error of measurement aligns with mean change in Mini-BESTest for patient- and physiotherapist-reported “small change,” and that the minimal detectable change aligns well with perceived substantial change, increases our confidence in these estimates. We have also provided the corresponding estimates for minimal clinically important difference values for each of the Mini-BESTest sub-scores (range of 1–3 points depending on the sub-scale), as clinicians may wish to target an individual sub-system of balance in their clinical practice.

In this study, a correlation of 0.33 was obtained for physiotherapist-reported global rating of change and change in Mini-BESTest but was negligible for the patient-reported anchor. This makes sense given the lack of agreement between physiotherapists and patients with respect to perceived change, and resulted in only 46% (n = 23) of the patients perceiving improvement. A review of studies in patients with balance disorders reveals minimal clinically important difference values in the range of 10%–17% of the total score, consistent with the results of the present study.^20–22 A small change of 4 points represents a variation of about 14% in the Mini-BESTest total score, and a substantial change of 5 points, represents a change of 17%. This corresponds to an effect size of 0.7 (standard deviations), which is considered to be clinically relevant. ²³ In general, our data suggest that patients with stroke may be weighting different factors or values more strongly in their rating of balance change after therapy, or that they may have had difficulty reflecting on their change over time. The physiotherapists may have also been biased toward a positive change rating as they are actively invested in improving patient function. This would be an important area for future research to ensure that the goals of patients with stroke are being met in rehabilitation.

Our data has some limitations. Because patients were enrolled in a rehabilitation program, most patients improved and we are only able to provide minimal clinically important difference values for improvement; estimates for decline may differ. For our distribution-based estimates of standard error of measurement and minimal detectable change, we used test-retest reliability coefficients from a previous study in stroke. ⁴ The minimal clinically important difference values reported in this study may not be generalizable to other stroke populations with different mobility levels, for example, among inpatients with stroke.

In summary, our study found that for individuals with stroke attending outpatient rehabilitation, a change of 4–5 points on the Mini-BESTest total score is required to be beyond measurement error and perceptible to both patients and clinicians. These values can be used to interpret the results of rehabilitative interventions designed to improve balance and fall risk in individuals with stroke.

Clinical messages.

• For people undergoing post-stroke rehabilitation, a minimal clinically important difference of 4 points for detecting small and 5 points for substantial changes is recommended on the Mini-BESTest.

• Minimal clinically important difference of 1–3 points on the Mini-BESTest sub-scores might help clinicians to assess change in individual balance subsystems.