Outcome measures in post-stroke arm rehabilitation trials: do existing measures capture outcomes that are important to stroke survivors, carers, and clinicians?

Abstract

Objective:

We sought to (1) identify the outcome measures currently used across stroke arm rehabilitation randomized trials, (2) identify and compare outcomes important to stroke survivors, carers and clinicians and (3) describe where existing research outcome measures capture outcomes that matter the most to stroke survivors, carers and clinicians and where there may be discrepancies.

Methods:

First, we systematically identified and extracted data on outcome measures used in trials within a Cochrane overview of arm rehabilitation interventions. Second, we conducted 16 focus groups with stroke survivors, carers and clinicians using nominal group technique, supplemented with eight semi-structured interviews, to identify these stakeholders’ most important outcomes following post-stroke arm impairment. Finally, we described the constructs of each outcome measure and indicated where stakeholders’ important outcomes were captured by each measure.

Results:

We extracted 144 outcome measures from 243 post-stroke arm rehabilitation trials. The Fugl-Meyer Assessment Upper Extremity section (used in 79/243 trials; 33%), Action Research Arm Test (56/243; 23%), and modified Ashworth Scale (53/243; 22%) were most frequently used. Stroke survivors (n = 43), carers (n = 10) and clinicians (n = 58) identified 66 unique, important outcomes related to arm impairment following stroke. Between one and three outcomes considered important by the stakeholders were captured by the three most commonly used assessments in research.

Conclusion:

Post-stroke arm rehabilitation research would benefit from a reduction in the number of outcome measures currently used, and better alignment between what is measured and what is important to stroke survivors, carers and clinicians.

Introduction

Up to 77% of stroke survivors experience upper limb (arm) impairment,¹ which affects function² and reduces health-related quality of life.³ Rehabilitation strategies, including those for the arm after stroke, should be based on research evidence. However, only moderate-quality evidence supports the use of interventions to rehabilitate the arm in current clinical practice.⁴ There is a demand from stroke survivors, carers, clinicians and researchers for research into interventions to improve arm function after stroke.^5,6

Efficacy of interventions should be demonstrated using measures that accurately and consistently capture change following treatment.⁷ Researchers currently use a wide range of measures to assess the efficacy of arm interventions after stroke within randomized controlled trials; recent work has identified at least 48 arm-related measures,⁸ indicating heterogeneity in what is in current use, as well as the wide range of possible targets for arm interventions including specific impairments, spasticity, pain or task-specific function. The measures in current use are highly varied in their focus and methods, impacting on researchers’ ability to compare and aggregate data from different studies to examine overall efficacy. Consensus on appropriate measures would enhance our ability to detect efficacy of interventions through pooled analysis.⁹ It has been acknowledged that selection of measures for use in trials should capture domains of importance to patients, carers and clinicians, consider the psychometric properties of measures, and feasibility for use in clinical and research settings.^10,11

There is a need for consensus on measure use in post-stroke arm rehabilitation trials.⁸ The Core Outcome Measures in Effectiveness Trials (COMET) initiative^10,12 provides guidance on development of consensus recommendations, highlighting the importance of targeting outcomes that are important and relevant to patients and clinicians.

Considering the views of stroke survivors, clinicians and researchers, the National Institute for Neurological Disorders and Stroke-Common Data Element¹³ recommends items for inclusion as part of standardized data collection across all stroke trials, and the International Consortium for Health Outcomes Measurement recommends measures for standardized data collection in stroke clinical practice.¹⁴ Other recommendations exist for general stroke outcomes and reflect physicians’ opinions on important outcomes according to the World Health Organization International Classification of Functioning, Disability and Health framework.¹⁵

The Stroke Recovery and Rehabilitation Round Table, consisting of researchers and clinicians, has generated consensus recommendations for core data collection across sensorimotor stroke rehabilitation trials, including a recommendation to use the Action Research Arm Test for measurement of arm activity limitation across trials.¹⁶ In addition, work has been completed to describe the psychometric properties of 53 available arm measures¹⁷ in order to inform selection. However, due to the wide-ranging impact of stroke on people’s lives,¹⁸ arm-specific measures are unlikely to capture all important outcomes.

To date, there is no clear consensus recommendation for the selection of measures in post-stroke arm rehabilitation randomized trials. Furthermore, there is a lack of information about which outcomes are most meaningful to stroke survivors, carers and clinicians. With a view to inform recommendations for selecting measures in future trials, we sought to investigate (1) existing measures used in post-stroke arm rehabilitation research studies, (2) outcomes important to stroke survivors, their carers and practising clinicians and (3) where important outcomes are captured by existing measures.

Methods

We conducted a mixed-methods study involving three distinct components. First, by means of a systematic exploration, we identified measures used in current post-stroke arm rehabilitation trials. Second, we identified outcomes that were important to stroke survivors and carers, and clinicians using focus groups and interviews. Finally, using systematic mapping, we described how well the currently used measures captured the outcomes of importance to stroke survivors, carers and clinicians.

Outcome measures used in published randomized controlled trials

Search strategy and selection criteria

We identified all systematic reviews contained within a Cochrane overview of systematic reviews for arm rehabilitation after stroke.⁴ Systematic reviews met the inclusion criteria for our exploration if they were published, and examined arm rehabilitation in adults with stroke. The overview also identified reviews that were in progress in February 2014; we considered these ‘in progress’ reviews and included those that met our criteria and had been published by June 2015.

From the eligible systematic reviews, we identified all included trials. We included randomized controlled trials aimed at improving functional recovery or reducing arm impairment in adults with a clinical diagnosis of stroke. One author (J.D.M.) applied these criteria to titles, abstracts and, where necessary, full texts. We searched for relevant reviews and trials in all languages and arranged costless translation where feasible.

Data extraction

Due to the wide-ranging impact of arm dysfunction on life following stroke,^2,3,18 data were extracted on all measures used in post-stroke arm rehabilitation trials, rather than only measures related to the arm. One author (J.D.M.) extracted data on assessment tool characteristics including their name, purpose and any modifications. A second author (M.A.) independently extracted data from a random sample of trials (n = 30) for comparison; the authors met to discuss and resolve any differences, involving a third author if required, although this was not necessary.

We defined a measure as a reproducible ‘scale, scoring system, questionnaire or other tool used for measuring an outcome’ (p.214).¹⁹ Two independent authors applied this definition to each assessment tool, then agreed on the final list of included outcome measures through discussion. A third author was available if agreement could not be reached; however, this was not required. We defined an outcome as ‘a measurable variable within an outcome domain. The outcome can be measured at a variety of time points, which must be clearly stated by authors of clinical trials’ (p.214).¹⁹

Data analysis

We tabulated the frequency of use of each measure across all identified trials. We reported as separate the instances where the same measure was used to capture different outcomes (e.g. visual analogue scale could be used to assess pain as well as anxiety).

Identification of important outcomes according to stroke survivors, carers and clinicians

Ethical approval was granted by North West – Greater Manchester West Research Ethics Committee (15/NW/0939).

Patient and public involvement

An advisory group comprising three stroke survivors with arm impairment (two of whom had aphasia), and two carers, informed and piloted our methods and participant materials.

Design

We undertook focus groups utilizing Nominal Group Technique. Nominal Group Technique is a consensus method that allows participants to reflect on, record and express their views in a structured and equitable way,²⁰ ensuring equal participation regardless of impairments.²¹ We supplemented the focus group data with semi-structured interviews with stroke survivors and carers who were unable to attend the focus groups to ensure that we captured the views of a wide sample of participants. See Figure 1 for an overview of these methods.

Figure 1.

Overview of methods used for the identification of important outcomes of stroke survivors, carers and clinicians.

Participants

Eligible participants were adult stroke survivors with arm impairment, who were able to give informed consent and felt able contribute in a focus group (for focus group participants), carers of stroke survivors with arm impairment, who were able to give informed consent, and clinicians with experience in working with stroke survivors with arm impairment. Carers did not have to attend the session with the person that they cared for. Participants were excluded if they were unable to give informed consent.

Research sites and participants were purposively sampled to represent urban and rural populations, a range of stroke disabilities, ages, times since stroke and clinical experience.

We calculated anticipated recruitment based on previous literature²² indicating that data saturation could be reached with 10–16 Nominal Group Technique focus groups and 5–8 interviews. Therefore, we aimed to recruit 136 participants to 16 focus groups (N = 8 stroke survivor and carer focus groups; N = 8 clinicians focus groups) with eight participants per group (N = 64 stroke survivors and carers; N = 64 clinicians) and eight interviews (N = 8 stroke survivors and/or carers).

Case ascertainment, recruitment and consent

All participants were identified face to face or via letter, by local collaborators at eight Scottish health board sites. Potential participants for both the focus group and interviews expressed interest and were then screened and consented for inclusion by one author (J.D.M.), a physiotherapist with 10 years’ clinical experience with stroke survivors.

Data collection

We conducted Nominal Group Technique focus groups with stroke survivors and carers, and separately with clinicians. One author (J.D.M.) conducted all focus groups with the assistance of two facilitators who also acted as scribes or provide assistance for stroke survivors if required. Focus groups were held at local sites and lasted between 1.5 and 2 hours.

Semi-structured interviews took place with stroke survivors, and where appropriate their carers, on stroke rehabilitation wards or in their homes for those unable to join groups. For the interview group, one author (J.D.M.) conducted all interviews following a prespecified schedule to help to identify levels of importance for the outcomes identified by stroke survivors and carers. Each interview lasted between 20 and 80 minutes. All sessions were audio recorded and interviews transcribed verbatim.

Stroke survivors were asked: ‘How does your arm affect your life after your stroke and what matters most to you about this?’ Carers were asked: ‘After their stroke, what matters most about your family member or friend’s arm that affects your role as a carer?’ Clinicians were asked: ‘When working with a stroke patient’s arm, which outcomes are most important to you?’ Within each focus group, participants generated, shared and recorded their statements. These statements were then discussed and clarified. The top five statements were individually and privately ranked in order of importance for each participant according to the process set out by Van Breda.²³ Summed scores were then generated to identify each group’s consensus on important statements.²³

Data analysis

Data were analysed according to methods set out by Van Breda.²³ One author (J.D.M.) conducted qualitative content analysis on the statements identified in the focus groups,²⁴ coding ‘meaning units’ defined as ‘statements that relate to the same central meaning’ (p.106).²⁴ A second author (M.A., F.V.W. or A.P.) independently coded meaning units for each statement for comparison and verification. Meaning units were then grouped into categories, and categories were classed as outcomes. The wording of the outcomes was chosen to select a word or phrase most closely related to that used by the participants in statements to demonstrate the nuance of the different outcomes. Statements with multiple ‘meaning units’ and codes were assigned to all applicable categories and thus outcomes. Outcome categories were arranged under themes and subthemes. Once the qualitative outcome categories were agreed, outcome categories were ranked considering (1) the number of statements in the outcome category, (2) the number of top five statements in the outcome category and (3) the average sum of statements in the outcome category to produce the most popular outcome categories.²³ In order to do this, the following steps were undertaken:

1. The average score for each statement was calculated, according to the methods outlined by Van Breda.²³

2. The generated average sum of scores was used to identify the statements with the highest average sum of scores to reveal the top five statements for each focus group.

3. Statements were considered under their respective outcome category to provide a sum of average scores for each outcome category.²³

We managed interview data using NVivo. Interview transcripts were similarly coded using qualitative content analysis. The outcomes identified from stroke survivor and carer focus groups and interviews were then triangulated; this involved cross-checking, comparing and contrasting outcomes to identify areas of similarity.²⁵

How well do existing measures capture outcomes of importance to stroke survivors, carers and clinicians?

Having identified the measures currently used in arm rehabilitation trials for stroke and the outcomes that were most important to stroke survivors, carers and clinicians, we compared and contrasted the data to identify whether important outcomes were captured by the existing measures and where there were any discrepancies. To do this, we described the individual constructs contained within each measure that we identified from trials (e.g. Barthel Index comprises toileting, feeding, dressing, grooming and mobility). We mapped constituent constructs within each trial measure to the outcomes that were identified as important by the stroke survivors, carers and clinicians, indicating where an outcome was captured, either wholly or in part, by a measure. One author (J.D.M.) completed all mapping. Three authors (A.P., F.V.W. and M.A.) each second reviewed an agreed upon proportion of the measures. Agreement between authors was >80% on this proportion of measures, and any discrepancies were discussed to reach consensus. As per the protocol, agreement of >80% on the mapping meant that one author (J.D.M.) completed the remaining mapping.

Results

Measures used in published randomized controlled trials

From 54 systematic reviews included in a Cochrane overview, 43 met our inclusion criteria (Figure 2). These systematic reviews contained 736 randomized controlled trials, of which 243 met our inclusion criteria (Figure 3). From these 243 trials, data were extracted on 188 assessment tools, of which 144 met our prespecified definition of a measure (Figure 4). Half (72/144) of the measures used in existing randomized controlled trials were used only once (Supplemental Table 1 contains all extracted measures). The 10 most frequently used measures are detailed in Table 1.

Figure 2.

Inclusion of systematic reviews identified from overview.⁴

Figure 3.

Inclusion of studies identified within systematic reviews.

Figure 4.

Inclusion of measures from list of assessment tools extracted from studies.

Table 1.

Ten most frequently used measures in stroke arm rehabilitation trials.

Rank	Research measure	Number of uses across 243 trials (%)	Number of times original measure was modified* (%)
First	Fugl-Meyer Assessment Upper Extremity section	79 (33)	12/79 (15)
Second	Action Research Arm Test	56 (23)	1/56 (2)
Third	Modified Ashworth Scale	53 (22)	0/53
Fourth	Motor Activity Log	43(18)	7/43 (16)
Fifth	Functional Independence Measure	35 (14)	11/35 (31)
Sixth	Goniometer to assess range of movement	32 (13)	4/32 (13)
Seventh	Wolf Motor Function test	29 (12)	5/29 (17)
Eighth=	Barthel Index	28 (11.5)	1/28 (4)
Eighth=	Dynamometry to assess strength	28 (11.5)	0/28
Tenth	Visual Analogue Scale for pain	24 (10)	5/24 (21)

Note: = denotes measure ranked as having equal frequency of use, as another measure *modifications are defined as researcher amending original measure by, for example, excluding or changing items in the measure. Formal published modified measures (e.g. modified Ashworth Scale) are described separately.

Identification of outcomes of importance to stroke survivors, carers and clinicians

We gained consent from 111 participants. There were no drop-outs during data collection. Participants included 43 stroke survivor participants (16 women and 19 men in focus group; 3 women and 5 men in interview) 10 carer participants (5 women and 3 men in focus group; 1 women and 1 man in interview). See Table 2 for stroke survivor and carer demographics.

Table 2.

Stroke survivor and carer demographics for focus groups and interviews.

All focus group and interview participants demographics	Stroke survivors (n = 43)	Carers (n = 10)
Women (%)	19 (44.2)	6 (60)
Median age (range)	60 years [IQR: 55, 67] (39–89 years)	63.5 years [IQR: 57.8, 71.5] (34–78 years)
Median modified Rankin Scale (range)	2 [IQR: 2, 3] (1–5)	n/a
Median time post stroke (range)	1.3 years, IQR [0.6, 3.0] (4 days to 11 years)	n/a
Live alone (%)	11 (25.6)	n/a
Self reported that speech was affected by stroke (%)	21 (48.8)	n/a
Self reported that mood was affected by stroke (%)	30 (69.8)	n/a

IQR: interquartile range; n/a: not applicable.

There were 58 clinician participants (median years’ experience in stroke rehabilitation 13.5, interquartile range (7.0, 19.8)). The clinicians’ group included 23 physiotherapists, 15 occupational therapists, 7 nurses, 5 support workers, 4 consultants, 1 Allied Health Professional consultant, 1 speech and language therapist, 1 clinical neuropsychologist and 1 orthotist.

We had planned to recruit 136 participants, but did not meet this target. Nevertheless, we achieved data saturation after 14 focus groups and 6 interviews; saturation was defined as attainment of no more new outcomes identified with each subsequent focus group or interview. All collected data were used in the analysis.

Stroke survivor and carer focus groups identified 43 important outcomes, and clinicians identified 32 important outcomes; of these, 9 outcomes were shared between both groups. Therefore, 66 unique, important outcomes relating to living with arm impairment after stroke were identified (see Supplemental Table 2 for all outcomes and illustrative participant quotes and Supplemental Tables 3(a) and (b) for themes and subthemes for stroke survivor and carer, and clinician focus group outcomes).

Stroke survivors identified the outcomes of ‘Independence, freedom and autonomy’, ‘Difficulty (with routine tasks)’ and ‘Everyday tasks’ as their three most important outcomes (Figure 5).

Figure 5.

The 10 most important outcomes for stroke survivors, carers and clinicians and the rank of the outcomes.

Note: = denotes outcome ranked equally important as another outcome. Outcomes ranked 1 were the most important.

For the outcome ‘Independence, freedom & autonomy’ stroke survivors spoke about the impact of arm dysfunction on their independence, of wanting the freedom to do what they wanted to do, when they wanted to do it. This was demonstrated with the quote:

My independence has been stripped away from me because I can’t drive. (Focus group, stroke survivor)

For stroke survivors, here independence is not necessarily related to doing something unaided or independently (which was identified in other outcomes), as may be described by clinicians but instead about the loss of freedom and independence. As well as being distinct from level of independence, ‘Independence, freedom and autonomy’ was separate from ‘Reliance on others’, which focused more on the dependence on another person being more of an issue than the loss of independence:

I have to rely on my partner to do everything. (Focus group, carer)

‘Difficulty (with routine tasks)’ was illustrated by quotes such as,

Every ordinary action you once did without thinking about was impossible at first and is now very difficult and slow. (Focus group, stroke survivor)

Here, the focus was on general difficulties resulting from stroke rather than the task itself. ‘Everyday tasks’ evidenced the wide-ranging impact of upper limb problems on general everyday tasks rather than specific activities of daily living, as described by one carer,

I have to do more tasks for my partner. e.g. opening bottle tops, and help her find other ways, but we do it together. (Focus group, carer)

Stroke survivors and carers did describe the importance of specific daily tasks in separate statements (e.g. personal care and meal time); these were ranked lower and are reported separately (Supplemental Table 2).

Clinicians identified ‘Everyday tasks’ and ‘Pain’ equally as their most important outcomes (Figure 5). Similar to the stroke survivors and carers, clinicians more frequently spoke about the broad impact of upper limb function on daily life in ‘Everyday tasks’ with one clinician reporting that it was important to ‘Increase functional use of upper limb in everyday tasks’ rather than focussing on the specific tasks. Clinicians described the importance of ‘No pain or manageable pain’. Shared priorities across both groups included ‘Everyday tasks’, ‘Pain’ and ‘Muscle tone & spasticity’ (Figure 6). Clinicians mentioned specific outcomes that were identified by stroke survivors, including driving, working and hobbies. However, they instead described these outcomes as part of the overall picture in a wider context such as everyday tasks, life and leisure or overall function.

Figure 6.

The nine outcomes identified by both stroke survivors and carers, and clinicians, and the rank of importance of these shared outcomes for each group.

Note: = denotes the outcome ranked equally important as another outcome. Ranking started at 1 for the most important outcome down to 43 (lower importance) for stroke survivors and carers, and down to 32 (lower importance) for clinicians.

The triangulation of interview and focus group outcomes and statements revealed similarity in 35/43 focus group outcomes supported by statements in the interview data. Interviews identified four categories that were discussed but not ranked in focus group data and thus did not form outcomes: ‘Attitudes of others’, ‘Need’, ‘In bed’ and ‘Prognosis’ and one category that was not discussed at all in focus groups: ‘Anticipating difficulties at home’.

How well do existing measures capture outcomes of importance to stroke survivors, carers and clinicians?

We mapped the 144 measures used in clinical trials to the 66 most important unique outcomes identified by stroke survivors, carers and clinicians. Supplemental Table 4 describes which of the 10 most important outcomes from stroke survivors, carers and clinicians (generated from the stroke survivor and carer, and clinician focus group statements) were captured by the 10 most frequently used measures in trials.

One outcome (Everyday tasks) was ranked in the top 10 by both stroke survivors and carers, and clinicians. Thus combining the top 10 outcomes from both stroke survivors and carers, and clinicians generated 19 unique, most important outcomes.

The stroke survivors and carers’ most important outcome of ‘Independence, freedom and autonomy’ was not captured by any of the 10 most frequently used measures. The clinicians’ most important outcome of ‘Everyday tasks’, which included activities of daily living, was not captured by any of the three most frequently used measures.

Only three measures (Assessment of Quality of Life Scale, Stroke Impact Scale version 2.0 and Stroke Impact Scale version 3.0) captured stroke survivors and carers’ most important outcome of ‘Independence, freedom & autonomy’, but these were not commonly used in clinical trials being used in 1/243, 12/243 and 6/243 trials, respectively. The three most frequently used measures in trials (Fugl-Meyer Assessment Upper Extremity section, Action Research Arm Test and modified Ashworth Scale) only captured between one and three important outcomes in the top 10 of both stroke survivor and carers, and clinicians (‘Pain’, ‘Difficulty’, ‘Tone and spasticity’, and ‘Arm care, positioning, and hygiene’). Of the 10 most frequently used outcome measures, the Barthel Index captured the highest number of important outcomes in each of the stroke survivors and carers, and clinicians top 10 lists, and was ranked eighth equal in frequency of use in randomized trials.

Discussion

We sought to identify measures currently used across stroke arm rehabilitation randomized trials and found 144 measures in current use across 243 trials. We also aimed to identify and compare outcomes important to stroke survivors and carers, and clinicians and describe where existing research measures capture outcomes that matter most to stroke survivors, carers and clinicians and where there may be discrepancies. Our results indicate that, despite some overlap, stroke survivors and carers tend to prioritize different outcomes when compared with clinicians. Despite a wide range of measures in current use across randomized trials, those most frequently used do not adequately capture the outcomes that are considered most important by stroke survivors, carers and clinicians.

Consistent with previous work,⁸ we observed heterogeneity in research measure use: the most frequently used research measure (Fugl-Meyer Assessment Upper Extremity section) was used in only one-third of trials and 50% of measures were used only once. Even the single construct of pain was assessed using 21 different measures. Furthermore, 21/188 (11%) of identified assessment tools were poorly described and therefore not reproducible, and even well-described measures were modified, for example, researchers changed the Fugl-Meyer Assessment Upper Extremity section on 12/79 (15%) occasions. Heterogeneity and inconsistencies in arm outcome assessment limit the ability to pool data in a meaningful way and synthesize evidence of the efficacy of post-stroke arm interventions to inform guidelines and clinical practice. Therefore, this uncoordinated measurement of outcomes undermines the evidence base for clinical practice. Targeted selection from a narrower collection of measures for use across stroke arm rehabilitation randomized trials would facilitate the generation of high-quality data for efficacy analyses. This selection should capture outcomes that are relevant to stroke survivors, carers and clinicians.^10,11

We identified stroke survivors and carers, and clinicians’ outcomes of importance related to living with arm dysfunction after stroke. Important outcomes generally differed between stroke survivors and carers on one hand, and clinicians on the other hand. Our results show that stroke survivors identified specific things that matter to them, and clinicians generally identified outcomes in a wider context. This is not surprising, given that the questions posed to participants were designed to identify their own priorities and we anticipated heterogeneity in outcomes. Furthermore, we expected a difference in the wording and terminology used between stroke survivors and carers, and clinicians plus reported outcomes to reflect nuance of the wording. Perhaps surprisingly, a lot of the outcomes identified by stroke survivors who were in-patients on rehabilitation wards in their acute/subacute phase (<six months post stroke) were similar to those identified by those who were in the home and in the chronic phase (⩾six months post stroke). This means that the outcomes identified in focus groups would in part reflect those that matter to stroke survivors across the post-stroke phases. However, researchers and clinicians may wish to consider the outcomes identified in the interviews by those at home or in the ward to explore outcomes that may be more relevant to different groups of stroke survivors. For example, ‘Prognosis’ and ‘Anticipating difficulties at home’ may be considered more important for in-patients.

We found few shared outcomes between stroke survivors and carers, and clinicians and considerable variations in ranking of any shared outcomes (e.g. pain was ranked #1 for clinicians and #12 for stroke survivors and carers). This disparity could have implications for person-centred care,²⁶ since stroke survivors and carers, and clinicians may be approaching post-stroke arm rehabilitation with differing expectations and goals. Nevertheless, clinicians acknowledged the need to prioritize person-centred outcomes (ranked 5/32) and goal attainment (ranked 6/32). The difference in outcomes and rankings could be an indication of the different approaches to rehabilitation; stroke survivors and carers may focus on their long-term life goal, while clinicians may focus more on short and intermediate goals or different outcomes to enable long-term goal attainment. Clinicians consider a range of outcomes relevant to post-stroke arm rehabilitation such as ‘Pain’ ‘Everyday tasks’ and ‘Tone and spasticity’ that were also frequently captured in trials. However, clinicians also identified outcomes such as ‘Acceptance’, ‘Expectation’, ‘Coping and self-management’ and ‘Knowledge and understanding’; these highlight the complexity of arm intervention delivery in clinical practice. Yet, our findings show that these outcomes are not frequently captured in research. Research outcome measures that addressed clinician priorities, such as person-centred outcomes and goal attainment, were not frequently used in randomized trials. For example, Goal Attainment Scaling and the Canadian Occupational Performance Measure were used in 4/243 and 1/243 trials, respectively.

In future randomized trials, there is scope to capture a greater number of important outcomes by carefully selecting complementary primary, secondary and tertiary outcome measures from a smaller pool of available measures. As well as capturing important outcomes, the selected research outcome measures must consider the aims of the trial, potentially including whether the intervention aims to target restitution of pre-stroke behaviour or compensation with a new behaviour. It is clear that work is needed to improve capture of the outcomes that are important to stroke survivors, carers and clinicians, while ensuring that researchers select from a smaller pool of outcome measures to enhance comparability.

We build on previous work^8,12–17 to provide a comprehensive picture of outcome measure use across arm rehabilitation randomized trials in stroke. We did not restrict our study to only those outcome measures that captured arm impairment, but included the full range of measures used in arm rehabilitation trials following stroke. Similarly, we asked stroke survivors and carers to prioritize what matters most to them in terms of life with arm problems due to stroke, rather than the aspects of arm recovery that were most important to them. Outcomes from stroke survivors and carers, and clinicians extended beyond the arm as a body structure and identified how arm impairment relates to other areas of the lived experience, including problems with frustration, quality of life, and independence. This is consistent with recent work exploring experiences of upper limb dysfunction following stroke.¹⁸ Our study provides a wider picture of the overall impact of arm impairment on life after stroke and describes the extent to which important outcomes are captured across current trials. Furthermore, our study provides evidence on the importance of looking beyond utilizing only arm-specific measures in upper limb rehabilitation trials in stroke.

Our study has some limitations. We identified randomized trials based on a Cochrane overview rather than undertaking a new systematic review. However, the rigour of the original Cochrane overview⁴ permitted the systematic and comprehensive identification of trials in this area; we also supplemented these data by contacting the authors of reviews that were on-going at the time of publication to identify recent reviews and randomized trials. Furthermore, we generated data on important outcomes based on participants sampled from Scotland only. So it is possible that different group compositions may have produced different outcomes and ranking. However, we sought to mitigate potential bias by purposively sampling to obtain a range of stroke survivor participants with different levels of impairment and chronicity, clinicians with different experience and expertise, as well as urban and rural locations across the country.

We took steps to ensure trustworthiness in the collection, analysis and reporting of the qualitative data. This included: use of a reflexive journal to critically reflect on the researchers’ position, interactions, emerging themes and outcomes and any points of note that may affect the understanding of the data; de-briefs following focus groups; member checking of analysis of focus group statements to avoid bias and enhance dependability and credibility; triangulation of data using multiple sources (focus group and interview); and detailed description of methods and reporting of outcomes and statements to ensure transparency and dependability.

The results of our study have implications for clinical practice and future research. Clinicians and researchers may wish to consider the outcomes of importance identified in this study when working with stroke survivors with arm dysfunction and their carers.

Clinical messages.

• The outcomes that were important to stroke survivors and carers differed from those identified by clinicians. Stroke survivors and carers identified specific things that matter to them, and clinicians generally identified outcomes in a wider context.

• Current post-stroke arm rehabilitation trials do not fully capture the outcomes that are important to stroke survivors, carers and clinicians.