Chronic Stroke Outcome Measures for Motor Function Intervention Trials: Expert Panel Recommendations

Abstract

Background

About half of stroke survivors experience severe and significant long-term disability. The purpose of this paper is to review the state of the science and to make recommendations for measuring patient-centric outcomes in interventions for motor improvement in the chronic stroke phase.

Methods and Results

A nine-member expert panel reviewed evidence to identify measures of upper and lower extremity function used to-date as outcomes in trials with patients who experienced a stroke 6 or more months prior to assessment. Outcome measures were screened using StrokEDGE consensus panel recommendations, and evaluated for availability of a published minimal clinically important difference (MCID). Measures meeting these criteria were further evaluated with regard to their level of measurement, psychometric properties, and ability of MCID to capture gains associated with improved function and clinical relevance to patients, to arrive at recommendations. A systematic literature review yielded 115 clinical trials of upper and lower extremity function in chronic stroke that used a total of 34 outcome measures. Seven of these had published MCIDs and were recommended or highly recommended by StrokEDGE. Those are the Fugl-Meyer Upper Extremity and Lower Extremity scales, Wolf Motor Function Test, Action Research Arm Test, Ten Meter and Six Minute Walk Tests, and the Stroke Impact Scale. All had evidence for the psychometric performance, although the strength of evidence for validity varied, especially in chronic stroke populations Fugl-Meyer Upper and Lower Extremity scales showing the strongest evidence for validity.

Conclusions

The panel recommends that the Fugl-Meyer Upper and Lower Extremity scales be used as primary outcomes in intervention trials targeting motor function in chronic stroke populations. The other six measures are recommended as secondary outcomes.

In spite of great advances in acute treatment, many stroke survivors experience debilitating sequelae.^1–4 About 800,000 people in the United States have a stroke each year and approximately two-thirds of these require rehabilitation. While evidence-based rehabilitation programs are able to improve functional abilities, the steepest slope of motor recovery occurs within the first 6 months.⁵ Residual and often disabling long-term deficits are common ^{6, 7} and frequently due to impaired motor function.^8–10

Novel interventions to improve motor function, such as stem cell therapy and brain-computer interfaces, are being developed to reduce disability for stroke survivors in the chronic phase of recovery – six or more months after stroke onset. As with all proposed therapies, regulatory and clinical decision-makers must weigh their benefit against their risks. To do so requires evidence-based measures of motor function specific to the chronic stroke phase. Although numerous measures for post-stroke motor function exist, it is not clear which most accurately measure meaningful change in the chronic phase.

The purpose of this paper, therefore, is to review the state of the science of outcome measures for motor function in the chronic stroke phase and to make recommendations as to which measures should be used as outcomes for interventions aimed to reduce chronic motor deficits.

METHODS

Expert panel selection

Based on nominations from national societies (American Heart Association, American Academy of Neurology, American Association of Neurological Surgeons, American Occupational Therapy Association, and the American Physical Therapy Association) and leading experts in the field, the research team (MOE, TP, SC, and SM) selected nine individuals for an expert panel representing a diversity of practice settings, educational background, geographic regions, and research and clinical foci.

Procedures

Panel members met twice in person and twice by teleconference to guide the research team in four steps. First, the panel reviewed and provided input on the search strategy for the research team to identify publications of clinical trials of chronic stroke motor function (see Appendix 1 for description of search strategy). Second, the upper and lower extremity measures of motor function identified in the literature were screened to include only those that were either recommended or highly recommended by the StrokEDGE consensus group for use in chronic stroke rehabilitation. The details of StrokeEDGE are reported in more detail elsewhere, ¹¹ but briefly, the Neurology Section of the American Physical Therapy Association completed a rigorous evaluation of outcome measure psychometric properties and clinical utility for patients with stroke to provide recommendations on the appropriateness of use (highly recommended, recommended, unable to recommend at this time, not recommended) by practice setting and patient acuity (sub-acute, acute, chronic). ¹¹ Third, the measures remaining were evaluated for the availability of a minimal clinically important difference (MCID) in the rehabilitation measures database.¹² The database, developed under a grant from the Department of Education, was designed to help clinicians and researchers identify optimal measures for assessing patient outcomes in all phases of rehabilitation. The panel further excluded measures without a MCID because this piece of information is critical for evaluating intervention benefit. Lastly, the panel developed recommendations for primary and secondary measures of upper and lower extremity function based on analysis of each against three criteria:

1. Level of measurement following the World Health Organization’s International Classification of Functioning, Disability and Health System (ICF¹³) framework which characterizes measures as reflecting status in the domains of body function, activity, and participation. Measures of domains closer to patient experience (i.e., activity or participation domain) were favored over measures of body function;

2. Strength of psychometric properties including reliability (inter-rater reliability, internal consistency reliability, test-retest reliability), responsiveness (the measure’s ability to reflect change from pre-treatment to post-treatment) and validity (face validity, content validity, concurrent validity, predictive validity and construct validity); and

3. Quality of MCID in that measures, which had their MCIDs specifically established for chronic stroke and derived using patient-based anchor methods, were favored over those with MCIDs only established for acute stroke or those based on distribution-based methods (i.e., based on a statistical measure of spread) or expert judgment.

RESULTS

Identifying candidate measures

We identified 115 clinical trials of interventions to improve motor function in chronic stroke patients; 55 of those looked at lower extremity and 60 at upper extremity function (see Appendix 1 for list of references). Those trials yielded 34 unique measures, 14 of which were used in trials of upper extremity, eight of lower extremity and 12 of upper and lower extremity function. Table 1 indicates the frequency of use, the targeted area, the level of recommendation for use as a chronic stroke rehabilitation outcome according to the StrokEDGE panel for each measure; and for those recommended or highly recommended, the existence of an MCID. Notably, despite its frequent use as an assessment of functional mobility in the acute phase of recovery (onset to 6-months post-stroke),¹⁴ the modified Rankin Scale (mRS¹⁵), was not used as an outcome in any of the chronic stroke clinical trials. Furthermore, the mRS has no MCID defined for chronic stroke and only limited information exists for its responsiveness and prognostic value for recovery in chronic stroke.¹⁶

Table 1.

Properties of measures identified in 115 chronic stroke rehabilitation clinical trials reviewed

Measure Name	Frequency count (%)	Target Extremity	StrokEDGE Recommendation*	MCID
Fugl-Meyer upper extremity	40 (34.8)	Upper	4	Yes
Motor Activity Log	28 (24.3)	Upper	4	No
6 Minute Walk Test	27 (23.5)	Lower	4	Yes
Timed Up and Go	27 (23.5)	Both	4	No
10 Meter Walk Test	26 (22.6)	Both	4	Yes
Modified Ashworth Scale	26 (22.6)	Both	3	No
Berg Balance Scale	23 (20.0)	Lower	4	No
Wolf Motor Function Test	22 (19.1)	Upper	3	Yes
Stroke Impact Scale	15 (13.0)	Both	4	Yes
Box and Block Test	12 (10.4)	Upper	3	No
Fugl-Meyer lower extremity	12 (10.4)	Lower	4	Yes
Action Research Arm Test	10 (8.7)	Upper	3	Yes
Functional Independence Measure	7 (6.1)	Both	2	No
ABILHAND	6 (5.2)	Upper	n/a	No
Functional Ambulation Category	6 (5.2)	Both	2	No
Functional Reach	6 (5.2)	Upper	4	No
Emory Functional Ambulation Profile	5 (4.3)	Both	n/a	No
Activities-Specific Balance Confidence Scale	4 (3.5)	Lower	3	No
Barthel Index	4 (3.5)	Both	3	No
Dynamic Gait Index	4 (3.5)	Lower	4	No
Jebsen Hand Function Test	4 (3.5)	Upper	n/a	No
Motricity Index	4 (3.5)	Both	2	No
Rivermead Mobility	4 (3.5)	Lower	3	No
SF-36/12	4 (3.5)	Both	3	No
VO2 Max	4 (3.5)	Lower	3	No
Active Range of Motion	3 (2.6)	Upper	n/a	No
Arm Motor Ability Test	3 (2.6)	Upper	3	No
Motor Assessment Scale	3 (2.6)	Both	n/a	No
Nine-Hole Peg Test	3 (2.6)	Upper	3	No
Canadian Occupational Performance Measure	2 (1.7)	Upper	2	No
Stroke Specific Quality of Life Scale	2 (1.7)	Both	2	No
Chedoke Arm and Hand Activity Index	1 (0.9)	Upper	1	No
Goal Attainment Scale	1 (0.9)	Upper	2	No
Five Times Sit to Stand	1 (0.9)	Lower	2	No
Medical Research Council Scale	1 (0.9)	Lower	n/a	No

^*NOTE: StrokEDGE Recommendation: 4=highly recommended, 3=recommended, 2=unable to recommend at this time, 1=not recommended, n/a=not considered.

The panel evaluated seven measures in step 4 (indicated in bold in Table 1) from which to develop recommendations for measuring upper and lower extremity motor function in the chronic stroke phase. Those were the Fugl-Meyer Upper Extremity subscale (FM-UE), the Fugl-Meyer Lower Extremity subscale (FM-LE), the Wolf Motor Function Test (WMFT), the Action Research Arm Test (ARAT), the Ten Meter Walk Test (10MWT), the Six Minute Walk Test (6MWT), and the Stroke Impact Scale (SIS). Six are performance-based measures (i.e., they reflect clinician or therapist ratings of actions performed by the patient) and one measure, the Stroke Impact Scale (SIS), is based on self-report (i.e., patient or proxy answers questions about his/her ability to perform specific activities). These measures are described in more detail below and Table 2 summarizes the panel’s evaluation of the three criteria in step 4.

Table 2.

Properties of short-listed measures

Scale	ICF Domain	MCID1	Psychometric Properties
			Reliability2	Responsiveness3	Validity4
FM UE	Body function	++	+++	+++	+++
FM LE	Body function	+	+++	+++	++
WMFT	Activity	+	+++	++	+++
ARAT	Activity	+	+++	+++	++
10 meter walk	Activity	---	+++	---	++
6 minute walk	Activity	---	++	---	+
SIS subscales	Activity, performance	++	++	+	++

¹++ =established for chronic stroke using patient-based anchor, + = established for chronic stroke using arbitrary anchor or distribution-based methods, --- = not established for chronic stroke.

²+++ = reliability coefficients are consistently at .85 or above, ++ = reliability coefficients are between .70 and .85.

³+++ = large effect size, ++ = moderate effect size, + = small effect size, --- = not established for chronic stroke.

⁴+++ = substantial validity evidence, ++ = moderate validity evidence, + = minimal validity evidence.

1. Fugl-Meyer upper extremity

The FM-UE¹⁷ assesses movement of the biceps, triceps, shoulder, elbow, forearm, hand, wrist and finger with performance of 33 tasks. Clinicians rate patient performance on each task for quality of movement on a scale from 0 (no active motion) to 2 (motion appears to be normal). The maximum score on the FM-UE subscale is 66 points (possible range 0 to 66).

The FM-UE measures performance at the body function domain. The FM-UE has excellent psychometric properties for assessment of motor function in chronic stroke, including high inter-rater reliability,¹⁸ and excellent test-retest reliability¹⁹ and responsiveness.^{19, 20} There is also substantial evidence for the validity of this measure among chronic stroke patients. For example, Hsieh et al²⁰ report correlations of the FM-UE with ARAT and WMFT scores ranging from 0.71 – 0.76 at pre-treatment and 0.51 – 0.74 at post-treatment, reflecting high construct validity. Pre-treatment scores on the FM-UE have also been shown to be significantly predictive of post-treatment scores on the Functional Independence Measure (FIM) (rho=0.42 with both FIM-Total and FIM-Motor²⁰). It has an established MCID for chronic stroke that was derived based on therapists’ evaluation of patients’ global rating of change (GROC) after an intervention.²¹

2. Fugl-Meyer Lower Extremity

The FM-LE assesses movement of the Achilles, patellar, hip, knee, and ankle with performance of 16 tasks that are rated on a scale from 0 (no active motion) to 2 (motion appears to be normal). The maximum score on the FM-LE subscale is 34 points (possible range 0 to 34).

The FM-LE is a body function domain measure. The FM-LE has high internal consistency and test-retest reliability, as well as excellent responsiveness.¹⁹ Further, there is support in the literature for face and content validity^{22, 23} of the FM-LE in the chronic stroke population. Evidence for other types of validity for this measure is lacking. Although Gladstone²³ suggests an MCID based on 10% of the total scale, the FM-LE does not have an empirically established MCID for use with chronic stroke patients.

3. Wolf Motor Function Test

The WMFT²⁴ is a measure of upper extremity motor ability that is quantified through 15 timed movement tasks and 2 strength-based tasks. The performance time of every WMFT item is measured to derive a time subscale (WOLF-Time) and a 6-point Functional Ability Scale (WOLF-FAS) rates the quality of movement for each of the tasks with values ranging from 0 (no attempt made to use the more affected upper extremity) to 5 (movement appears to be normal).

The WMFT reflects functioning at the activity domain. Like the FM-UE, both scales of the WMFT have excellent psychometric properties for assessment in chronic stroke including high inter-rater reliability, excellent test-retest reliability,²⁵ and evidence for responsiveness.²⁶ There is substantial evidence for the validity of this measure among chronic stroke patients. For example, as noted above, correlations of the WMFT scales with the FM-UE and ARAT are significant at pretreatment and post-treatment, reflecting moderate construct validity.²⁶ Hsieh et al also report strong predictive validity for the WMFT-TIME (rho of 0.47 and 0.43 with FIM-Total and Motor, respectively) but predictive validity coefficients for the WMFT-FAS were not significant (rho of 0.17 and 0.19 with FIM-Total and Motor, respectively).²⁰ The WMFT has an established anchor-based MCID for chronic stroke that was derived using a change on the FM-UE ranging from 6–10 as the anchor.²⁷

4. Arm Research Action Test

The ARAT,²⁸ an evaluative measure of upper extremity motor ability consisting of 19 movement tasks divided into 4 sub-tests (grasp, grip, pinch, and gross arm movement), assesses a patient’s ability to handle objects differing in size, weight and shape.²⁹ The ARAT contains a 3-point functional ability scale that rates the quality of movement for each of the tasks and has values ranging from 0 (not able to perform any part of the test) to 3 (movement appears to be normal). The maximum score on the ARAT is 57 points (possible range 0 to 57).

The ARAT reflects measurement at the activity domain level. The ARAT has excellent inter-rater³⁰ and test-retest reliability³¹ for assessment in chronic stroke, and strong evidence of responsiveness.²⁶ Like the FM-UE and WMFT scales, there is evidence for the validity of the ARAT in chronic stroke patients, as these three measures have been shown to correlate highly with one another at both pre- and post-treatment assessments.²⁶ However, like the WMFT-FAS, the predictive validity coefficients for the ARAT as reported by Hsieh were not significant (rho of 0.22 and 0.26 with FIM-Total and Motor, respectively).²⁰ The ARAT has an established anchor-based MCID for chronic stroke that was derived using a 10% change on the total scale as the anchor.³⁰

5. Ten Meter Walk Test (gait velocity)

The Ten Meter Walk Test is an evaluative measure of walking speed which requires a 20-meter, indoor, flat straight hallway. The first and last 5 meters are used to accelerate and decelerate while only the middle 10 meters are recorded. The patient is instructed to walk at a self-selected speed, using whatever walking aids might be needed, such as a walker or cane. The velocity is calculated as distance divided by time.

The 10MWT is an activity level domain measure. Although there is evidence for excellent internal consistency and test-retest reliability of the 10MWT among chronic stroke patients,³² the evidence for responsiveness is scarce for this population. Validity evidence is also limited, but can be inferred from its association with community ambulation. For example, a compilation of many studies has established walking speed thresholds appropriate for assessing and monitoring functional status in the home and community and overall health in a wide range of populations including stroke patients ³³. However, there is no established MCID for the 10MWT in chronic stroke populations, specifically.

6. Six Minute Walk Test (gait endurance)

The Six-Minute Walk Test (6MWT³⁴) quantifies the distance a patient can walk at self-selected walking speed on a flat, hard surface in a period of 6 minutes.

The 6MWT is an activity level domain measure. Despite its frequent use in chronic stroke trials, there is very little psychometric evidence for the 6MWT in chronic stroke populations. With the exception of excellent test-retest reliability noted by Flansbjer 2005,³² all of the available evidence is based on acute stroke populations. Further, much of the evidence for the psychometric performance of this measure, most notably that pertaining to responsiveness and MCID, pertains only to acute stroke patients.³⁵ However, the test has clear face validity among chronic stroke patients. The 6MWT does not have an established MCID for chronic stroke.

7. Stroke Impact Scale

The Stroke Impact Scale³⁶ is a multidimensional self-reported measure of stroke outcomes. The SIS version 3.0 consists of 59 items, each of which is rated on a 5-point Likert scale ranging from 1 (unable to complete that item) to 5 (no difficulty experienced at all) and is divided into 8 subtests or domains, four of which are closely related to upper and lower extremity mobility (hand function, ADL/IADL, mobility, strength). There is also a 16-item subtest from the SIS (the SIS-16³⁷) that measures physical functioning and focuses primarily on lower extremity function. However, there is no documentation of its performance in patients with chronic stroke, thus we do not consider it further here.

The SIS represents assessment across the activity and participation domains. The four SIS subscales have adequate internal consistency and test-retest reliability. However, three of the four subscales have low responsiveness.³⁸ Aside from the validity evidence for the proxy version of the SIS scales in chronic stroke populations,³⁹ validity evidence for the subscales is limited; baseline measures of hand function are significantly correlated (r=0.51), with FM UE scores at post treatment in chronic stroke patients providing some evidence for the predictive validity of this subscale.³⁸ MCIDs have been established for the chronic stroke population for each of the four subscales related to motor function.⁴⁰

DISCUSSION

Recommendations of the multidisciplinary expert panel were aimed to identify which existing measures of motor status are best suited to capture the effects of emerging interventions targeting residual motor deficits in the chronic phase after stroke. Overall, no existing measure was recommended or highly recommended by StrokeEDGE, had a published MCID for chronic stroke, and met all three evaluation criteria of measuring outcomes at the activity or participation level, having solid psychometric evidence and having a MCID that was established based on patient rating. In weighing the strengths and weaknesses of all measures, this panel recommends using the FM-UE and FM-LE scales as primary outcomes. The panel further recommends using either the WMFT or ARAT for upper and 10MWT or 6MWT for lower extremity functioning as secondary outcomes. As noted above, the mRS, which is commonly used to assess changes in functional ability in the acute phase after stroke, is not regarded as suitable for the chronic phase, as it lacks an MCID, as well as sufficient information on its responsiveness and prognostic value for recovery.¹⁶

The recommendation of the FM-UE and FM-LE scales is based on three considerations. First, their psychometric properties are strong and well-documented. In particular, the excellent validity evidence, which shows associations of FM-UE and FM-LE scores with activity measure scores, outweighs our concerns over the fact that they measure outcomes at the body function level, and facilitates their meaningful interpretation by clinicians and patients;²³ concerns that these FM scales are body function level assessments are further mitigated by their very close correlation with measures of activity limitation.²⁰ Second, they are by far the most frequently used measures in relevant clinical trials, which has resulted in a rich body of empirical evidence for their utility and meaningfulness, as well as familiarity to the stroke rehabilitation field, which also strengthens ability to interpret and compare findings across studies. Third, a patient-centric MCID for chronic stroke has been established for the FM-UE.

While conceptually attractive because of their inherent patient-centeredness, none of the existing measures that are based on self-report can be recommended at this point. The panel’s main concerns were the limited evidence for validity in chronic stroke populations and the lower reliability compared to other short-listed measures. Among the ramifications of this lower measurement precision is its negative impact on the responsiveness of the subscales. Additionally, the self-report assessment format can be particularly problematic in situations where treatment assignment is not blinded to the patient, as the patient perspective on his or her own functioning can be biased in this case. In addition, frequent concomitants to motor deficits after stroke, such as aphasia and neglect, could significantly impact the utility of self-report measures when applied to broad populations of patients with stroke.

The panel’s review of the evidence has highlighted several directions for future research. In light of the increased recognition of the value of patient-centered outcomes for clinical and regulatory decision-making, future studies are needed to more fully develop the psychometric evidence for the SIS, especially with respect to its validity in reflecting true improvement in functioning. Further, if self-reported measures such as the SIS are to be appropriately used in clinical trials with chronic stroke patients, it will be important to use study designs that minimize the potential for bias that may be associated with a lack of blinding.

Although a body function score such as the Fugl-Meyer will work well where the goal is restoration of motor function such as stem-cell therapy or treatments targeted at brain recovery and reorganization, in the chronic phase many studies target disability or activity, which call for assessment of participation outcomes. Thus, studies are also needed to provide more evidence for the performance-based measures. In particular, the establishment of patient-based MCIDs would greatly enhance the value of the WMFT and ARAT measures. Future research is also needed to evaluate and document all aspects of the psychometric performance of the 10MWT and 6MWT in chronic stroke populations. The use of these measures as secondary outcomes in future studies will facilitate the accumulation of such evidence.

One limitation of this work may be that the panelists for the StrokEdge and this consensus committee were all American. It is possible that country and continent differences in usage of outcome measures may make acceptability of the panel’s recommendations outside of the U.S. a challenge. However, among the 115 clinical trials we evaluated, nearly 70% (n=80) were based on research conducted outside of the U.S. Thus despite the exclusive US nationality of the panelists, the evidence they considered was internationally representative, and it is reasonable to assume that these results would likely be duplicated with a more international panel.

At the present state of science, the FM-UE and FM-LE scales represent viable measures for efficacy trials of interventions aimed to improve motor function in the chronic phase after stroke.