Abstract
Purpose:
To evaluate the dissemination and implementation impacts of a rehabilitation intervention.
Methods:
Systematic evaluation of data sources including academic publishing metrics, publications, and surveys was used to describe the dissemination and implementation impact of the graded repetitive arm supplementary program (GRASP). Three categories in the Payback Framework were evaluated: knowledge production and dissemination, benefits to future research and research use, and real-world uptake and implementation.
Results:
In the Knowledge production and dissemination category, seven publications, authored by the GRASP research team, were associated with the GRASP, and there were approximately 17,000 download counts of GRASP manuals from the website from 120 countries. In the Benefits to future research and research use category, 15 studies and 8 registered clinical trials, authored by researchers outside of the GRASP team, have used GRASP as an intervention. In the real-world uptake and implementation category, GRASP has informed recommendations in 2 clinical guidelines and 20 review papers, and had high implementation uptake (e.g., 35% [53/154] of UK therapists surveyed had used GRASP; 95% [649/681] who downloaded GRASP had used it). More than 75% of those who had used GRASP identified that GRASP provides more intensity in upper extremity rehabilitation, is evidence-based and easy to implement, and the equipment and manual are easy to obtain.
Conclusion:
The Payback Framework is useful to evaluate the dissemination and implementation impacts of a rehabilitation intervention. GRASP has been implemented extensively in clinical practice and community in a relatively short time since it has been developed.
Key Words: information dissemination, implementation science, upper extremity, rehabilitation, stroke
Translational research moves scientific discoveries from bench research to clinical studies and clinical studies to practice settings and communities.1 The impact of translational research on health care service delivery and subsequently on public health has been slow. A UK study estimated that it takes 10 years or more to implement evidence-based health interventions from phase 3 clinical trials to real-world practice settings.2
Research impact can be captured from a range of aspects such as academic research metrics, translation into clinical practice and health policy, and influence on the broader economic benefits.3 While studies have assessed the overall impact of funded health research studies on various aspects (e.g., health systems, policymaking, society, and economic), these studies were mostly conducted by governments, organizations, and higher education institutes with the aim of allocating funds and resources.4 Individual researchers have been utilizing traditional bibliometrics such as the number of citations or H-index to assess and monitor academic output and research-related impact of their academic publications. With the ultimate goal of improving health care and public health, researchers in health research must facilitate knowledge uptake and expand research benefit beyond academia to stakeholders. Therefore, it is necessary for researchers to systematically measure the impact of health research on health care delivery in real-world settings and policy making. Understanding the impact of health research will enable researchers to identify the current status of knowledge uptake and develop future steps for knowledge translation and dissemination to narrow the long research-to-practice gap.
Stroke is a leading cause of disability worldwide, with over 13 million new cases each year.5 One of the most common consequences following stroke is upper limb paresis. Approximately 60% of individuals post-stroke display upper extremity (UE) paresis post-stroke.6 Impairment in the upper extremity causes difficulties in performing daily activities, negatively affecting quality of life.7 A variety of interventions have been developed and supported by research evidence to improve UE outcomes, such as constraint-induced movement therapy, mirror therapy, virtual reality, mental practice, and repetitive task practice.8 In the case of stroke rehabilitation, despite advances in research, a wide gap remains between research and real-world rehabilitation practice.9 Previous studies have identified several barriers that hinder the uptake of research findings into clinical practice, such as complexity of interventions, lack of knowledge or skill, and lack of resources to implement.10,11 In fact, only 2.5% of published stroke rehabilitation research evaluates the implementation of evidence-based interventions into practice,12 and the impact of these evidence-based interventions in the real world on broader aspects (e.g., health care delivery, policy making, etc.) remains largely unknown, possibly due to lack of evaluation or implementation.
Graded Repetitive Arm Supplementary Program (GRASP)
One example of an evidence-based rehabilitation intervention is the Graded Repetitive Arm Supplementary Program (GRASP) (http://neurorehab.med.ubc.ca/grasp/) developed by Canadian researchers to improve paretic UE function for people post-stroke based on intensive, repetitive, and task-specific practice. The hospital GRASP published in 2009 was supported by a larger multi-site randomized controlled trial in inpatient stroke rehabilitation settings, where GRASP was used as a self-administered treatment protocol in addition to regular therapy monitored by the therapists at each site using face-to-face delivery for 4 weeks.13,14 This study showed that the GRASP group had a greater post-treatment improvement on UE function than the control group receiving education protocol (effect size d = 0.45). Other versions were subsequently developed, including a Home GRASP for individuals discharged home,15 a group GRASP,16 and a virtual GRASP (done in a group online) with post-treatment effect sizes ranging from 0.57 to 0.84 for UE motor impairment and function.17 These GRASP have been shown to be effective in improving motor recovery (i.e., the Chedoke Arm and Hand Inventory and the Fugl-Meyer Assessment for upper extremity) and increasing the use of affected UE during daily activities.15
The purpose of this study was to evaluate the dissemination and implementation impact of a specific evidence-based intervention, GRASP, expanding beyond traditional research metrics. A number of approaches have been developed to evaluate research impact4, and the Payback Framework18 is one of the most commonly used methodological frameworks.4 While the Payback Framework has been used to evaluate entire health care systems or organizations, it has not been utilized to assess the impact of specific interventions. This paper is the first to use a methodology framework to systematically assess the impact of an intervention in any population on knowledge production, the research system, and policy service development. Collating and standardizing the reporting of the extent of implementation of an intervention outside of a research context can assist in understanding the reach and significance in real-world settings.
Methods
Conceptual framework
We adopted the Payback framework from its original five categories to produce a three-category conceptual model focused on the evaluation of the impacts of an intervention on the dissemination and implementation in real-world settings (Table 1). The first two categories were similar to the original Payback framework: (1) knowledge production and dissemination, and (2) benefits to future research and research use. The last category was (3) real-world uptake and implementation which included a subheading on cost-effectiveness and allocation of services adapted from the original Health sector benefits category from the Payback framework. Two new subheadings were added to this third category. One new subheading addressed “informing recommendation for clinical guidelines and review papers” as changing guidelines demonstrates widespread acceptance of an intervention in clinical care and review papers typically reflect the state of knowledge or current practice and provide the evidence for a particular point of view in a field. The other new subheading addressed “Informed development and uptake of services” which produced evidence of the intervention being utilized in current care or the reach of the intervention into current practice.
Table 1.
Summary of the Adapted Research Paybacks
Adapted payback category | Adapted payback subcategory | Evidence |
---|---|---|
Knowledge production and dissemination | Peer-reviewed publications | • A total of 9 papers published in peer-reviewed journals. • A total of 541 citations on Goggle Scholar and 300 citations on the Web of Science. See bibliometric report in Table 2. |
Dissemination | • Approximately 17,000 download counts of the GRASP manuals from the website from 120 countries (see details in Table 3) • A total of 30 presentations at conferences and workshops |
|
Benefits to future research and research use | Generating future research by other researchers | • A total of 15 publications that have used GRASP as an intervention. • Eight registered trials have used the GRASP as an intervention. |
Capacity building | • GRASP website, and manuals (Home and Hospital versions) have been created. • The GRASP manuals have been translated to eight languages by clinicians who would like to use the GRASP for their local sites. |
|
Real-world uptake and implementation | Informing recommendations in clinical guidelines and review papers | • The GRASP has been suggested as a supplementary training by Canadian Stroke Best Practice and Health Quality Ontario. • A total of 20 review papers cited the original GRASP paper. |
Informed development and uptake of services | • 22% of UK rehabilitation therapists had experience of using the GRASP (Connell, 2014). • 35% of UK rehabilitation therapists have used the GRASP in treating individuals with stroke (Stockley et al., 2019). • The Stroke Recovery Association of British Columbia implemented GRASPs (in-person and virtual) in the community. • The GRASP has been recommended as a home exercise programme by two national non-profit organizations: The March of Dimes Canada and the Heart and Stroke Foundation of Canada. • 95% of the survey respondents (N = 649) who downloaded the manual previously have used the GRASP (see details in Table 4). |
|
Improved allocation of services or cost-effectiveness of services | • Service typically provided by hospital functions were delivered via community programmes (via Stroke Recovery Association of British Columbia) • Cost-effectiveness not established |
GRASP = Graded Repetitive Arm Supplementary Program.
Data sources
Knowledge production and dissemination
Bibliometric analysis was conducted to understand the Knowledge Production subcategory in the adapted Payback framework. The number of peer-reviewed journal articles, the impact factor of a journal that the given article was published in, number of citations, number of access/download counts, and the Altmetric Attention Score were collected for activities involving GRASP. The Altmetric Attention Score and number of access/download counts were collected on the journal website. As variations in databases used to assess citation impact were found among different databases,18 the total number of times that a given publication has been cited by other publications was collected from two different databases: Google Scholar and Web of Science Core Collection from the date of publication to January 11, 2023. People who access the program manuals from the website must fill out a short website download form which collects information on the demographics of the person and their intentions in using GRASP. To understand the Dissemination subcategory, data from the website download form was collected at the time when the individuals downloaded the manuals from the GRASP website from May 2015 to May 2019.
Benefits to future research and research use
To examine the Generating future research by other researchers subcategory, research by other researchers that utilized the GRASP as an intervention in the study were identified from a search on Web of Science and clinical trial registry databases. Articles (N = 100) that cited the original GRASP study13 on the Web of Science were reviewed to determine if the study used the GRASP as an intervention. Searches on ClinicalTrials.gov, Australian New Zealand Clinical Trials Registry (ANZCTR), International Standard Randomised Controlled Trial Number (ISRCTN) registry, and European Union (EU) Clinical Trials Register using the search term “Graded repetitive arm supplementary program” in the field of other terms were used to identify any registered clinical trials that used the GRASP as an intervention (retrieved on May 1, 2020).
Real-world uptake and implementation
Information in the review papers that included the original GRASP paper13 and clinical guidelines were extracted to examine how the GRASP has Informed recommendations in clinical guidelines and review papers. To understand how the GRASP has informed the Development of rehabilitation services in clinical practice and community uptake, findings from two published survey studies, one interview study,20–22 and two implementation studies16–17 were extracted. In addition, a survey was conducted to investigate the knowledge uptake by the users of the GRASP website. Individuals who had downloaded the manuals previously (N = 9707) were sent an email containing the survey link and informed consent process, a brief description of the purpose of the research, and an invitation to complete the survey in July 2019. A follow-up reminder was sent via email to the individuals who had not completed the survey 2 months later. The survey was established by a panel of therapists and researchers with extensive experience in survey design and the GRASP. Prior to disseminating the survey, one pilot test was carried out with clinicians (n = 3), and minor changes were made based on the feedback received. There were two versions of the survey: one for those who had used GRASP before and one for those who had not used GRASP before. The survey is provided in the Appendix. Only the findings from those who had used GRASP before were presented in this paper. The time to complete the survey was 5 minutes. The survey study protocol was approved by the University of British Columbia Behavioural Research Ethics Board, study number H19-01731.
Results
Table 1 provides a summary of the dissemination and implementation impact of the GRASP.
Knowledge production and dissemination
Peer-reviewed publications
Nine publications13–17,21–24 were associated with the GRASP, including the first clinical trial results with the full bibliometric report in Table 2.
Table 2.
Bibliometric Report
Publications; first author (year) | Journal (impact factor*) | Number of citations | Access/download counts† | Altmetric attention score‡ | |
---|---|---|---|---|---|
Web of science | Google scholar | ||||
Harris (2009) | Stroke (10.170) | 143 | 260 | Access: not available Download: 34,441 |
2 |
Harris (2010) | Physical Therapy (3.140) | 45 | 81 | Access: 3,960 Download: 763 |
Not available |
Connell (2014) | Implementation Science (8.000) | 38 | 59 | Access: 8,404 Download: Not available |
5 – good attention score compared to outputs of the same age (75th percentile) |
Connell (2014) | Physical Therapy (3.140) | 21 | 39 | Access: 3,816 Download: 1,004 |
1 |
Connell (2014) | Archives of Physical Medicine and Rehabilitation (4.060) | 21 | 38 | Not available | Not available |
Connell (2016) | Physical Therapy (3.140) | 10 | 16 | Access: 4,605 Download: 937 |
1 |
Simpson (2017) | Disability and Rehabilitation (2.439) | 17 | 37 | Access: 4,015 Download: Not available |
15 – In the top 25% of all research outputs scored by altmetric |
Yang (2021) community | Physical Therapy (3.140) | 3 | 5 | Access: 1,350 Download: 369 |
4 – Above-average attention score compared to outputs of the same age (62nd percentile) |
Yang (2021) virtual | Physical Therapy (3.140) | 2 | 6 | Access: 3,651 Download: 990 |
3 – Above-average attention score compared to outputs of the same age (55th percentile) |
2021 Journal impact factor.
Access/download counts and the altmetric attention score were collected on January 11, 2023, on the journal website.
The altmetric attention score is a weighted count of all of the online attention including mentions in public policy documents and references in Wikipedia, the mainstream news, social networks, blogs and more.
Dissemination
To facilitate the uptake of the GRASP by knowledge users, the research team created the GRASP website in 2011, where related resources such as manuals and videos have been uploaded and can be accessed at no cost. People who access the program manuals from the website must fill out a short website download form which collects information on the demographics of the person and their intentions in using GRASP. As shown in Table 3, as of June 2019, there had been approximately 17,000 download counts of GRASP manuals from the website from 120 countries in total, with about half of download counts from North America and a third from Europe. The program reached a wide range of knowledge users on the Internet, including rehabilitation therapists and assistants (80%), students (13%), individuals with stroke and their families (5%), doctors (<1%), and organizations (<1%). As most of the people who downloaded the manuals from the website were rehabilitation therapists and assistants, the intended setting where the GRASP would be used was mostly related to their work settings (e.g., hospital, community, and home care). The intended format was mainly individual sessions between therapist and patient. A total of 30 presentations have been done at local, national, and international conferences and workshops.
Table 3.
Respondents Characteristics, Intended Settings, and Intended Format of the GRASP from People who downloaded the GRASP Manuals from the GRASP Website
% | N | |
---|---|---|
Continents (n = 17,529) | ||
North America | 44.87 | 7866 |
Europe | 29.01 | 5085 |
Oceania | 19.16 | 3359 |
Asia | 5.57 | 977 |
Africa | 0.84 | 148 |
South America | 0.54 | 94 |
Profession and/or background (n = 17,535) | ||
Occupational therapist | 50.58 | 8870 |
Physiotherapist | 24.33 | 4267 |
Student | 13.43 | 2355 |
Rehab assistant | 4.41 | 773 |
Individual with stroke | 2.44 | 427 |
Caregiver (family member, friend) | 2.22 | 390 |
Doctor | 0.79 | 139 |
Organization | 0.67 | 117 |
Other | 1.12 | 197 |
Setting where the GRASP is intended to be used (n = 17,581) | ||
Hospital | 55.80 | 9811 |
Community | 22.01 | 3869 |
Educational Institution | 9.46 | 1664 |
Independently | 4.48 | 787 |
Home care | 4.06 | 713 |
Other | 4.19 | 737 |
Intended format of the GRASP (N = 17,544) | ||
Individual sessions b/w therapist and patient | 76.84 | 13480 |
Independently | 11.68 | 2050 |
Group setting | 8.23 | 1443 |
Other | 4.89 | 858 |
GRASP = Graded Repetitive Arm Supplementary Program.
Benefits to future research and research use
Generating future research by other researchers
The GRASP has been used by other researchers, as shown by 15 publications25–39 and 8 registered ongoing clinical trials. Three studies25–27 used the GRASP as an experimental intervention. Of these three studies, one was a study protocol in a one-group quasi-experimental design,25 a second found that GRASP is superior than constraint-induced movement therapy in improving UE function,26 and a third found GRASP to improve UE function in a one-group pre-post design (effect size d = 1.61).27 Six studies28–33 and four registered clinical trials (ACTRN12616000029493, ACTRN12618000443291, ACTRN12615000665538, and ACTRN12619000596101) combined the GRASP with noninvasive brain stimulation, telehealth, or knowledge translation interventions to quantify the additional benefits these interventions or the combined intervention.
The GRASP has also been used as a standard therapy to serve as an active comparator in six publications34–39 and the four registered ongoing clinical trials (NCT02688413, NCT01721668, NCT02136433, and ACTRN12619001557123). Of these six publications, three publications were study protocols.36,37,39 Two studies found GRASP to be similarly satisfying and effective to a video game intervention on UE function.34,35 The third study using less dosage (30 hours of training over a 10-week period) than the Home GRASP protocol (1 hour/day, 6 days a week for 8 weeks) found GRASP to be similarly effective to music-supported therapy on UE impairment.38
Capacity building
GRASP website and manuals have been generated to help to build capacity across the research system. The GRASP manuals have been translated to eight languages by clinicians who intended to use the GRASP for their local sites. These resources are available on the website (https://neurorehab.med.ubc.ca/grasp/) at no cost.
Real-world uptake and implementation
Informing recommendations in clinical guidelines and review papers
The GRASP has been recommended by one national clinical practice guideline, Canadian Stroke Best Practices and one provincial guideline, Health Quality Ontario, as a supplementary training programme to increase the treatment intensity during hospitalization and at home.40,41 The original GRASP paper has been included in 20 review papers (13 of these were systematic reviews) where the GRASP is described as a self-directed rehabilitation programme after stroke,42–48 repetitive task practice,49–51 task-specific training,52 resistance training programme,53 unsupervised practice,54 mixed approach using mixed elements (e.g., strengthening, task exercises, and stretching),46 bilateral training,55 caregiver-mediated exercises,56 high-intensity exercise therapy,57 rehabilitation programmes available on the Internet,58 intervention with a published protocol,59 or personalized out of therapy protocol.60
Informed development of services – clinical practice and community uptake
Two survey studies have demonstrated an increase in using the GRASP by rehabilitation therapists in the UK since 2014. The first survey study in 201421 showed that 22% of UK neuro physiotherapists and occupational therapists had the experience of using the GRASP and 41% of therapists knew of the GRASP but had never used it. A recent study done by the same research group demonstrated that 35% (53/154) of neuro physiotherapists and occupational therapists use the GRASP in treating individuals with stroke with mild and moderate impairments.20 An interview study conducted to understand the implementation of the GRASP in Canada found that the GRASP has been adapted to some degree when used in clinical practice with less dose and wider coverage in clients’ motor recovery.22 One non-profit organization, the Stroke Recovery Association of British Columbia, an affiliate of the March of Dimes Canada, has been continuously delivering the GRASP in the community since 2019.16–17 The GRASP has been suggested by national non-profit organizations, the March of Dimes Canada, and the Heart and Stroke Foundation of Canada, as a home exercise programme for community-dwelling individuals with stroke.
In total, 681 people responded to the survey that was sent to people who have downloaded the programme manual from the website, giving an approximate response rate of 7%. Of these respondents, 95% of them reported that they had used GRASP (649). The survey results are presented in Table 4. From those who had used GRASP, the highest number of users were from North America (53%), the second highest was from Europe (27%), and followed by Oceania, Asia, South America, and Africa. GRASP has been used mostly for clinical practice (87%) during individual therapy sessions (72%) and as an adjunct outside of therapy sessions (60%). More than 75% of the GRASP users agreed that the GRASP provides more intensity in upper extremity rehabilitation, the GRASP is evidence-based, the programme is easy to implement, and the equipment and manual are easy to obtain. Although GRASP has been taken up widely in clinical practice and in the community, we found that it is not always delivered in the way in which it was shown to be effective. This is shown by the survey results (Table 4D) that approximately one-quarter of the respondents did not ensure that their clients logged the practice time and did not check their clients’ weekly logs.
Table 4.
Results from a Survey Sent Out to People who Downloaded the GRASP Manuals
A. Responses (%) categorized by continents to where the GRASP was intended to use (N = 638) | ||
---|---|---|
% | No. of respondents | |
North America | 52.98 | 338 |
Europe | 27.43 | 175 |
Oceania | 14.11 | 90 |
Asia | 4.23 | 27 |
South America | 0.78 | 5 |
Africa | 0.47 | 3 |
B. Responses (%) to the purpose and setting of the GRASP from people who have used the GRASP. | ||
---|---|---|
What purpose have you used this program for? | ||
Clinical practice | 86.83 | 554 |
Exercise program for yourself/ friend/ family member who had a stroke in the past | 12.07 | 77 |
Teaching | 9.72 | 62 |
Research | 2.82 | 18 |
In what setting do you use this program? | ||
I am a therapist using the program with clients on a one-to-one basis within their sessions | 71.94 | 459 |
I am a therapist asking my clients to do the program independently outside of their sessions (as an adjunct) | 60.19 | 384 |
I am a therapist using the program with a group of clients within their sessions | 14.26 | 91 |
I am a person with stroke doing my exercises without supervision | 4.23 | 27 |
I am a person with stroke doing my exercises with supervision | 1.57 | 10 |
Other | 11.13 | 71 |
How did you find out about GRASP? | ||
Colleagues | 54.86 | 350 |
Internet | 25.24 | 161 |
Education session | 18.97 | 121 |
Health care provider | 18.97 | 121 |
Research paper | 15.83 | 101 |
Conference presentation | 10.03 | 64 |
Clients | 1.72 | 11 |
Support group | 0.94 | 6 |
Friends/family | 0.78 | 5 |
Other | 9.09 | 58 |
C. Responses (%) to the “Intervention characteristics” statements from therapists who have used the GRASP | |||
---|---|---|---|
Please identify to which extent you agree or disagree with the following statements regarding the reasons why you selected to use GRASP | |||
Agree | Disagree | Neutral | |
Manuals are available online at no cost | 96.37% | 3.63% | 0.27% |
Easy for a patient to do /Easily implemented into current practice | 90.21% | 0.63% | 9.16% |
Detailed protocol written in the manual | 88.47% | 0.32% | 11.22% |
Provide a patient with more intensity in their arm and hand rehabilitation | 87.52% | 1.58% | 10.90% |
Equipment is easy to obtain | 83.57% | 2.05% | 14.38% |
Supported by research studies | 76.78% | 1.58% | 21.64% |
A senior therapist or supervisor supported the use of GRASP | 55.61% | 15.48% | 28.91% |
It was recommended by a health care provider | 46.45% | 19.12% | 34.44% |
The only program I could find that focuses on hand/arm impairment | 32.07% | 25.91% | 42.02% |
D. Fidelity. Responses (%) to “When delivering the GRASP, which of the followings do you do (choose all that apply)” (N = 547) | ||
---|---|---|
% | N | |
Coach the patient and/or family on how to do the GRASP exercises | 95.61 | 523 |
Involve family and/or caregivers with GRASP if available | 84.46 | 462 |
Equip the patient with the GRASP book | 84.46 | 462 |
Encourage and set targets for stroke-affected hand use in everyday home activities | 73.3 | 401 |
Work through barriers to doing daily GRASP exercises | 63.44 | 347 |
Progress the patient weekly so exercises are always challenging | 58.14 | 318 |
Equip the patient with the GRASP equipment | 55.39 | 303 |
Ensure the patient logs the GRASP practice time | 28.34 | 155 |
Check the patient's weekly logs | 25.78 | 141 |
Other | 9.69 | 53 |
GRASP = graded repetitive arm supplementary program.
Improved allocation of services or cost-effectiveness of services
The delivery of GRASP via community programmes has improved the allocation of rehabilitation resources, which are typically provided in hospitals and private clinics.16–17 However, the cost-effectiveness of services has not been assessed.
While GRASP has been used widely across the world, we provide one illustrative example of how its dissemination crossed from high to low-income economy countries. The GRASP has been implemented in an outpatient rehabilitation programme by a Canadian OT volunteering in a low- and middle-income country (a small village in southern India) where there are limited rehabilitation resources. The main modifications instituted to contextualize the GRASP for this outpatient programme were translating the manual to the local language (Tamil), finding alternative equipment for items that were not easily accessible, and running the programme in a group setting.
Discussion
This is the first study to systematically evaluate the research impacts of an evidence-based intervention (i.e., the GRASP) beyond traditional research metrics using the adapted Payback framework. Using a structured framework and multiple data sources, we demonstrated that the GRASP has international reach and has impacted on research, guidelines, clinical and community practice, and ultimately individuals with stroke.
We found the adapted Payback Framework useful to describe the dissemination and implementation impact of a rehabilitation intervention in a standardized format for communication. Given how real-world influences of research on society have been given increasing importance and recognition, it is crucial to measure how research achieves impact outside academia. As it is not clear how to systematically capture and report the research influence beyond academic metrics, we specifically chose to focus on the reach and uptake in services or practice to provide useful information about dissemination and implementation beyond the research setting. From multiple data sources, we found that in a relatively short time frame (11 years) since the original GRASP paper was published in 2009,13 the impact of the GRASP has reached beyond the academic field to change rehabilitation service in clinical practice and in community. While data to fill out the Payback Framework came from a number of sources, we recommend a few critical sources that assisted in understanding implementation: a website from which the number of downloads and basic demographics of clinicians or patients downloading the details about the intervention can be collected; survey of those who download the detailed intervention to follow-up as to whether they implemented the intervention; and surveys of different groups of clinicians in the field as to their knowledge or implementation of the intervention. Going forward, we recommend that rehabilitation researchers utilize the adapted Payback Framework and multiple data sources outlined in this study to evaluate the impact of their research works and plan for the next step to disseminate and translate the research evidence.
The high adoption and implementation of GRASP in the academic field and beyond the academic field (clinical practice and community) may result from the intervention being highly standardized and evidence based. It has been shown that clinical replicability of rehabilitation interventions in randomized controlled trials is inadequate partly due to insufficient description of interventions.61 Although the usual care has been commonly used as a comparator to other interventions or as a baseline intervention to which new technologies (e.g., noninvasive brain stimulation, robotics, etc.) are added, it often differs within and between studies. Common reasons are multiple treatments, different orders of therapies, and personal behaviours that affect these factors, which may influence the study results.62 It is likely that researchers have chosen GRASP as a viable and replicable comparator or to combine with other experimental treatment (e.g., noninvasive brain stimulation or telehealth) as it is designed based on motor learning principles,58 and it is a standardized programme with progressions.35,38 The recommendation of the GRASP cited in clinical practice guidelines (i.e., Canadian Stroke Best Practices)41 may be another key factor for high adoption within Canada. Evidence from previous studies published in internationally recognized journals in rehabilitation13,15 showing the beneficial effects of the GRASP on UE function after stroke likely contributed to the adoption of GRASP across the world, especially in North America and Europe. This is consistent with the survey results showing that more than 75% of respondents agreed that GRASP is supported by research studies.
Another reason for the high adoption and implementation of GRASP is that it can be adapted and tailored easily to meet local needs while being effective. Our previous clinical trials13,15 and implementation studies16–17 demonstrated the effectiveness and adaptability of the programme in different settings and formats (i.e., face-to-face delivery in hospital setting, home exercise programme monitored by phone, group classes in community centres, or virtual delivery). The GRASP instructor’s manual and exercise manual (https://neurorehab.med.ubc.ca/grasp/), which provide information on how to implement the GRASP and written and pictorial guidance for each exercise, are also available for the users to facilitate the uptake of the programme. These are coincident with the survey results showing that the vast majority of respondents agreed that GRASP is easy to implement into current practice, and that reasons for using GRASP were reported to be the detailed protocol in the manual and that the manuals are available online at no cost.
Compared to other evidence-based rehabilitation interventions, GRASP is low-cost and does not require expensive equipment as with robotic training or a large amount of clinical time to deliver the intervention as with constraint-induced movement therapy. While robotic training enables a high-intensity training regime and has shown beneficial effects on UE function in individuals with stroke,63 robotic training equipment involves a significant capital expenditure for rehabilitation providers (e.g., cost up to several hundred thousand dollars per device), which may impede the uptake and use of such evidence-based rehabilitation intervention. In contrast, the equipment for GRASP can be easily obtained in one’s home (e.g., cups, paper clips, coins, etc.) or purchased from a store or online (e.g., grippers, weights) at a small cost. In places where resources are limited (e.g., remote and rural areas, low- and middle-income countries) or for individuals doing exercises by themselves, GRASP is more feasible. While large amounts of clinical time were not required, the intervention does depend on motivation and willingness of patients (and potentially caregivers) to be self-directed with their exercise. The finding that some therapists are not ensuring accountability of their patient’s practice could reduce the effectiveness of the intervention.
Multimodal dissemination strategies were used to distribute evidence-based information, including improving the reach of evidence, enhancing the ability to apply evidence, and the combination of both.64 Academic knowledge production and dissemination, such as peer-reviewed publications and presentations at local, national, and international conferences, was done to increase the reach of information to researchers and clinicians. Hosting free resources on the website (e.g., manuals, videos, etc.) and holding knowledge translation workshops were also done to distribute the evidence widely to all stakeholders (e.g., individuals with stroke, their caregivers, researchers, and clinicians) and allow the appropriate use of evidence. To enhance the ability to deliver the GRASP, we provided additional resources, such as supporting materials that explain the implementation of the programme in specific settings and a fidelity checklist in the instructor manuals.
This study has potential limitations. It is important to note that the data for real-world dissemination and uptake is likely under-reported despite using robust methods and the adapted Payback framework to collect data and including a wide range of outcomes to demonstrate the impact of GRASP. Although the sampling error is reduced when the sample size is more than 150,65 the low response rates of the surveys should still be acknowledged. The response rates were low in the survey of people who downloaded the programme manual (N = 681) and in two previous survey studies by Connell and colleagues21 (N = 274) and Stockley and colleagues20 (N = 156), which were not unexpected for these types of survey. Another limitation is that one item in the survey did not have an “other” option. Approximately 60% have an “other” option. The respondents could only choose the options that were provided in the survey.
Conclusion
This is the first study using the adapted Payback framework to assess the impact of an evidence-based intervention (i.e., the GRASP). We demonstrated high dissemination and uptake of the GRASP beyond the academic field in clinical practice and community internationally. Moreover, we identified attributes of the research that enhanced the real-world uptake and maximized the impacts for clinical practice and community. The findings of this study could help determine how to assess the impact of evidence-based rehabilitation intervention in the future. Most importantly, the insights that arise from this study could inform researchers on how to facilitate the dissemination and uptake of the research evidence.
Key messages
Due to a lack of evaluation of the implementation of evidence-based intervention into practice, the research impact of these studies in the real world, such as health care delivery or policy making, remains largely unknown. Using the adapted Payback framework, this study is the first to systematically examine the research impacts of an evidence-based rehabilitation intervention (i.e., the GRASP) beyond traditional research metrics. High dissemination and uptake of the GRASP in clinical practice and community were shown. Attributes of the GRASP that may facilitate the uptake were also discussed.
Data Availability Statement
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Appendix
Questionnaire for GRASP program users
This 5-minute survey will ask you about your experiences using the GRASP. Your confidentiality will be respected. You will be given the option of providing your email address if you would like to be entered into the draw for a gift certificate, however, this information will be removed from your survey answers prior to analysis. If you are currently running the GRASP at your site, you will be given the option of providing the name and location of your site. No information that may directly or indirectly disclose your identity will be released or published. You can also choose to decline to provide your contact information.
1. Have you used GRASP or are you currently using GRASP?
Yes, go to question 2
No, go to question 10
2. What purpose have you used this program for (choose all that apply):
Clinical practice
Research
Teaching
Exercise program for yourself/ friend/ family member who had a stroke in the past
Other: Please specify ________
3. In what setting do you use this program (choose all that apply):
I am a person with stroke doing my exercises with supervision
I am a person with stroke doing my exercises without supervision
I am a therapist using the program with clients on a one-to-one basis within their sessions
I am a therapist using the program with a group of clients within their sessions
I am a therapist asking my clients to do the program independently outside of their sessions (as an adjunct)
Other: Please specify ________
4. In which city and country have you used GRASP?
City________
Country ________
5. How did you find out about GRASP (choose all that apply)?
Research paper
Conference presentation
Education session
Healthcare provider
Colleagues
Your clients
Support group
Friends/ family
The Internet
Other, please specify ________
6. Please identify to what extent you agree or disagree with the following statement regarding the reasons why you selected to use GRASP (agree, neutral, disagree):
Provide a patient with more intensity in their arm and hand rehabilitation
Supported by research studies
Easy for a patient to do/ Easily implemented into current practice
Equipment is easy to obtain
Manuals are available online at no cost
Detailed protocol written in the manual
The only program I could find that focuses on hand/arm impairment
It was recommended by a health care provider
A senior therapist or supervisor supported the use of GRASP
7. When delivering the GRASP, which of the following do you do (choose all that apply):
Equip the patient with the GRASP book
Equip the patient with the GRASP equipment
Coach the patient and/or family on how to do the GRASP exercises
Involve family and/or caregiver with GRASP if available
Ensure the patient logs the GRASP practice time
Progress the patient weekly so exercises are always challenging
Check the patient's weekly logs
Work through barriers to doing daily GRASP exercises
Encourage and set targets for stroke-affected hand use in everyday home activities
Other, please specify ________
8. What worked well with GRASP?
Answer: |
9. What did not work well with GRASP?
Answer: |
10. If GRASP is currently being run at your site, we would like to add your site to the list of “sites that use GRASP” on our website. Please provide the name and location of your site if you would like us to add your site to the list.
Name of your hospital/clinic ________
Location of your site (City, Province/State, Country) ________
11. Why did you not use the GRASP program, or have stopped using the GRASP program? (Choose all that apply)
Change of staff position
Other priority
Not enough time
Not treating patients who need upper extremity rehabilitation
Hard to incorporate into practice
Hard to find the appropriate equipment
Exercises are too difficult
Exercises are too easy
Exercises are too boring
Other, please specify ________
Janice Eng, an expert physiotherapy researcher from the University of British Columbia, is being recognized during the 75th Anniversary of Physiotherapy Canada for her many contributions to advancing research in Canada and internationally.
References
- 1.Rubio DM, Schoenbaum EE, Lee LS, et al. Defining translational research: implications for training. Acad Med. 2010;85(3):470–5. 10.1097/ACM.0b013e3181ccd618. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Hanney SR, Castle-Clarke S, Grant J, et al. How long does biomedical research take? Studying the time taken between biomedical and health research and its translation into products, policy, and practice. Health Res Policy Syst. 2015;13(1):1–18. 10.1186/1478-4505-13-1/TABLES/3. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Yang CL, Waterson S, Eng JJ. Implementation and evaluation of the virtual graded repetitive arm supplementary program (GRASP) for individuals with stroke during the COVID-19 pandemic and beyond. Phys Ther. 2021;101(6):1–9. 10.1093/PTJ/PZAB083. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Cruz Rivera S, Kyte DG, Aiyegbusi OL, Keeley TJ, Calvert MJ. Assessing the impact of healthcare research: a systematic review of methodological frameworks. PLoS Med. 2017;14(8):e1002370. 10.1371/journal.pmed.1002370. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Lindsay MP, Norrving B, Sacco RL, et al. World Stroke Organization (WSO): global stroke fact sheet 2019. Int J Stroke. 2019;14(8):806–17. 10.1177/1747493019881353. Medline: [DOI] [PubMed] [Google Scholar]
- 6.Simpson LA, Hayward KS, McPeake M, Field TS, Eng JJ. Challenges of estimating accurate prevalence of arm weakness early after stroke. Neurorehabil Neural Repair. 2021;35(10):871–9. 10.1177/15459683211028240. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Kwakkel G, Kollen BJ. Predicting improvement in the upper paretic limb after stroke: a longitudinal prospective study. Restor Neurol Neurosci. 2007;25(5-6):453–60. Medline: [PubMed] [Google Scholar]
- 8.Pollock A, Farmer SE, Brady MC, et al. Interventions for improving upper limb function after stroke. Cochrane Database Syst Rev. 2014;2014(11):CD010820. 10.1002/14651858.CD010820.pub2. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Walker MF, Fisher RJ, Korner-Bitensky N, McCluskey A, Carey LM. From what we know to what we do: translating stroke rehabilitation research into practice. Int J Stroke. 2013;8(1):11–7. 10.1111/j.1747-4949.2012.00974.x. Medline: [DOI] [PubMed] [Google Scholar]
- 10.Kayes NM, Martin RA, Bright FA, Kersten P, Pollock A. Optimizing the real-world impact of rehabilitation reviews: increasing the relevance and usability of systematic reviews in rehabilitation. Eur J Phys Rehabil Med. 2019;55(3):331–41. 10.23736/S1973-9087.19.05793-9. Medline: [DOI] [PubMed] [Google Scholar]
- 11.Juckett LA, Wengerd LR, Faieta J, Griffin CE. Evidence-based practice implementation in stroke rehabilitation: a scoping review of barriers and facilitators. Am J Occup Ther. 2019;74(1):7401205050p1. 10.5014/ajot.2020.035485. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Lynch EA, Chesworth BM, Connell LA. Implementation—the missing link in the research translation pipeline: is it any wonder no one ever implements evidence-based practice? Neurorehabil Neural Repair. 2018;32(9):751–61. 10.1177/1545968318777844. Medline: [DOI] [PubMed] [Google Scholar]
- 13.Harris JE, Eng JJ, Miller WC, Dawson AS. A self-administered graded repetitive arm supplementary program (grasp) improves arm function during inpatient stroke rehabilitation. Stroke. 2009;40(6):2123–8. 10.1161/STROKEAHA.108.544585. Medline: [DOI] [PubMed] [Google Scholar]
- 14.Harris JE, Eng JJ, Miller WC, Dawson AS. The role of caregiver involvement in upper-limb treatment in individuals with subacute stroke. Phys Ther. 2010;90(9):1302–10. 10.2522/PTJ.20090349. Medline: [DOI] [PubMed] [Google Scholar]
- 15.Simpson LA, Eng JJ, Chan M. H-GRASP: the feasibility of an upper limb home exercise program monitored by phone for individuals post stroke. Disabil Rehabil. 2017;39(9):874. 10.3109/09638288.2016.1162853. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Yang CL, Bird ML, Eng JJ. Implementation and evaluation of the graded repetitive arm supplementary program (GRASP) for people with stroke in a real world community setting: case report. Phys Ther. 2021;101(3):1–8. 10.1093/PTJ/PZAB008. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Yang CL, Waterson S, Eng JJ. Implementation and Evaluation of the Virtual Graded Repetitive Arm Supplementary Program (GRASP) for Individuals With Stroke During the COVID-19 Pandemic and Beyond. Phys Ther. 2021;101(6):pzab083. 10.1093/ptj/pzab008. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Donovan C, Hanney S. The ‘Payback Framework’ explained. Res Eval. 2011;20(3):181–3. [Google Scholar]
- 19.Plana NM, Massie JP, Bekisz JM, Spore S, Diaz-Siso JR, Flores RL. Variations in databases used to assess academic output and citation impact. N Engl J Med. 2017;376(25):2489–91. 10.1056/NEJMc1616626. Medline: [DOI] [PubMed] [Google Scholar]
- 20.Stockley R, Peel R, Jarvis K, Connell L. Current therapy for the upper limb after stroke: a cross-sectional survey of UK therapists. BMJ Open. 2019;9(9):e030262. 10.1136/bmjopen-2019-030262. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Connell LA, McMahon NE, Watkins CL, Eng JJ. Therapists’ use of the graded repetitive arm supplementary program (GRASP) intervention: a practice implementation survey study. Phys Ther. 2014;94(5):632–43. 10.2522/ptj.20130240. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Connell LA, McMahon NE, Harris JE, Watkins CL, Eng JJ. A formative evaluation of the implementation of an upper limb stroke rehabilitation intervention in clinical practice: a qualitative interview study. Implement Sci. 2014;9(1):90. 10.1186/s13012-014-0090-3. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Connell LA, McMahon NE, Simpson LA, Watkins CL, Eng JJ. Investigating measures of intensity during a structured upper limb exercise programme in stroke rehabilitation: an exploratory study. Arch Phys Med Rehabil. 2014;95(12):2410–9. 10.1016/J.APMR.2014.05.025. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Connell LA, McMahon NE, Tyson SF, Watkins CL, Eng JJ. Case series of a knowledge translation intervention to increase upper limb exercise in stroke rehabilitation. Phys Ther. 2016;96(12):1930–37. 10.2522/ptj.20150694. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.McDonnell MN, Koblar S, Ward NS, Rothwell JC, Hordacre B, Ridding MC. An investigation of cortical neuroplasticity following stroke in adults: Is there evidence for a critical window for rehabilitation? BMC Neurol. 2015;15(1):109. 10.1186/s12883-015-0356-7. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Dowarah BP, Bora JH. To study the effects of CIMT (Constraint-Induced Movement Therapy) Vs. GRASP (Graded Repetitive Arm Supplementary Program) training in improvement of arm and hand function in chronic stroke patients: a comparative study. Res Rev J Neurosci. 2019;8(3):24–9. 10.37591/RRJON.V8I3.591. [DOI] [Google Scholar]
- 27.Murdolo Y, Brown T, Fielding L, Elliott S, Castles E. Stroke survivors’ experiences of using the Graded Repetitive Arm Supplementary Program (GRASP) in an Australian acute hospital setting: a mixed-methods pilot study. Aust Occup Ther J. 2017;64(4):305–13. 10.1111/1440-1630.12363. Medline: [DOI] [PubMed] [Google Scholar]
- 28.Allman C, Amadi U, Winkler AM, et al. Ipsilesional anodal tDCS enhances the functional benefits of rehabilitation in patients after stroke. Sci Transl Med. 2016;8(330):330re1. 10.1126/scitranslmed.aad5651. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Arnao V, Riolo M, Carduccio F, et al. Effects of transcranial random noise stimulation combined with Graded Repetitive Arm Supplementary Program (GRASP) on motor rehabilitation of the upper limb in sub-acute ischemic stroke patients: a randomized pilot study. J Neural Transm. 2019;126(12):1701–6. 10.1007/s00702-019-02087-9. Medline: [DOI] [PubMed] [Google Scholar]
- 30.Burridge JH, Lee ACW, Turk R, et al. Telehealth, Wearable Sensors, and the internet: will they improve stroke outcomes through increased intensity of therapy, motivation, and adherence to rehabilitation programs? J Neurol Phys Ther. 2017;41(suppl 3):S32–8. 10.1097/NPT.0000000000000183. Medline: [DOI] [PubMed] [Google Scholar]
- 31.Levy T, Killington M, Lannin N, Crotty M. Viability of using a computer tablet to monitor an upper limb home exercise program in stroke. Physiother Theory Pract. 2021;37(2):331–41. 10.1080/09593985.2019.1625092. Medline: [DOI] [PubMed] [Google Scholar]
- 32.Lin S, Mann J, Mansfield A, Wang RH, Harris JE, Taati B. Investigating the feasibility and acceptability of real-time visual feedback in reducing compensatory motions during self-administered stroke rehabilitation exercises: a pilot study with chronic stroke survivors. J Rehabil Assist Technol Eng. 2019;6(6):205566831983163. 10.1177/2055668319831631. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Welsby E, Ridding M, Hillier S, Hordacre B. Connectivity as a predictor of responsiveness to transcranial direct current stimulation in people with stroke: protocol for a double-blind randomized controlled trial. JMIR Res Protoc. 2018;7(10):e10848. 10.2196/10848. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Yacoby A, Zeilig G, Weingarden H, Weiss R, Rand D. Feasibility of, adherence to, and satisfaction with video game versus traditional self-training of the upper extremity in people with chronic stroke: a pilot randomized controlled trial. Am J Occup Ther. 2019;73(1):7301205080p1–14. 10.5014/ajot.2019.026799. Medline: [DOI] [PubMed] [Google Scholar]
- 35.Rand D, Weingarden H, Weiss R, et al. Self-training to improve UE function at the chronic stage post-stroke: a pilot randomized controlled trial. Disabil Rehabil. 2017;39(15):1541–48. 10.1080/09638288.2016.1239766. Medline: [DOI] [PubMed] [Google Scholar]
- 36.Rand D, Zeilig G, Kizony R. Rehab-let: touchscreen tablet for self-training impaired dexterity post stroke: study protocol for a pilot randomized controlled trial. Trials. 2015;16(1):277. 10.1186/s13063-015-0796-9. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Kairy D, Veras M, Archambault P, et al. Maximizing post-stroke upper limb rehabilitation using a novel telerehabilitation interactive virtual reality system in the patient's home: study protocol of a randomized clinical trial. Contemp Clin Trials. 2016;47:49–53. 10.1016/j.cct.2015.12.006. Medline: [DOI] [PubMed] [Google Scholar]
- 38.Fujioka T, Dawson DR, Wright R, et al. The effects of music-supported therapy on motor, cognitive, and psychosocial functions in chronic stroke. Ann N Y Acad Sci. 2018;1423(1):264–74. 10.1111/nyas.13706. Medline: [DOI] [PubMed] [Google Scholar]
- 39.Allegue DR, Kairy D, Higgins J, et al. Optimization of upper extremity rehabilitation by combining telerehabilitation with an exergame in people with chronic stroke: protocol for a mixed methods study. JMIR Res Protoc. 2021;9(5):e14629. 10.2196/14629. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Health Quality Ontario; Ministry of Health and Long-Term Care. Quality-based procedures: clinical handbook for stroke (acute and postacute). Toronto: Health Quality Ontario; 2016. December. p. 132 Available from: http://www.hqontario.ca/evidence/evidence-process/episodes-of-care#community-stroke. [Google Scholar]
- 41.Teasell R, Salbach NM, Foley N, et al. Canadian stroke best practice recommendations: rehabilitation, recovery, and community participation following stroke. Part one: rehabilitation and recovery following stroke; 6th Edition Update 2019. Int J Stroke. 2020;15(7):763–88. 10.1177/1747493019897843. Medline: [DOI] [PubMed] [Google Scholar]
- 42.Morais HCC, Gonzaga NC, de Souza Aquino P, de Araujo TL. Strategies for self-management support by patients with stroke: integrative review. Rev Esc Enferm USP. 2015;49(1):136–43. Portuguese. 10.1590/S0080-623420150000100018. Medline: [DOI] [PubMed] [Google Scholar]
- 43.Turner-Stokes L, Ashford S, Esquenazi A, et al. A comprehensive person-centered approach to adult spastic paresis: a consensus-based framework. Eur J Phys Rehabil Med. 2018;54(4):605–17. 10.23736/S1973-9087.17.04808-0. Medline: [DOI] [PubMed] [Google Scholar]
- 44.Da-Silva RH, Moore SA, Price CI. Self-directed therapy programmes for arm rehabilitation after stroke: a systematic review. Clin Rehabil. 2018;32(8):1022–36. 10.1177/0269215518775170. Medline: [DOI] [PubMed] [Google Scholar]
- 45.Hayward KS, Brauer SG. Dose of arm activity training during acute and subacute rehabilitation post stroke: a systematic review of the literature. Clin Rehabil. 2015;29(12):1234–43. 10.1177/0269215514565395. Medline: [DOI] [PubMed] [Google Scholar]
- 46.Lin IH, Tsai HT, Wang CY, Hsu CY, Liou TH, Lin YN. Effectiveness and superiority of rehabilitative treatments in enhancing motor recovery within 6 months poststroke: a systemic review. Arch Phys Med Rehabil. 2019;100(2):366–78. 10.1016/j.apmr.2018.09.123. Medline: [DOI] [PubMed] [Google Scholar]
- 47.Scrivener K, Jones T, Schurr K, Graham PL, Dean CM. After-hours or weekend rehabilitation improves outcomes and increases physical activity but does not affect length of stay: a systematic review. J Physiother. 2015;61(2):61–7. 10.1016/j.jphys.2015.02.017. Medline: [DOI] [PubMed] [Google Scholar]
- 48.Rabadi MH. Review of the randomized clinical stroke rehabilitation trials in 2009. Med Sci Monit. 2011;17(2):RA25–43. 10.12659/msm.881382. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Nilsen DM, Gillen G, Geller D, Hreha K, Osei E, Saleem GT. Effectiveness of interventions to improve occupational performance of people with motor impairments after stroke: an evidence-based review. Am J Occup Ther. 2015;69(1):6901180030p1-9. 10.5014/ajot.2015.011965. Medline: [DOI] [PubMed] [Google Scholar]
- 50.Noorkõiv M, Rodgers H, Price CI. Accelerometer measurement of upper extremity movement after stroke: a systematic review of clinical studies. J Neuroeng Rehabil. 2014;11(1):144. 10.1186/1743-0003-11-144. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.de Sousa DG, Harvey LA, Dorsch S, Glinsky JV. Interventions involving repetitive practice improve strength after stroke: a systematic review. J Physiother. 2018;64(4):210–21. 10.1016/j.jphys.2018.08.004. Medline: [DOI] [PubMed] [Google Scholar]
- 52.Wattchow KA, McDonnell MN, Hillier SL. Rehabilitation interventions for upper limb function in the first four weeks following stroke: a systematic review and meta-analysis of the evidence. Arch Phys Med Rehabil. 2018;99(2):367–82. 10.1016/j.apmr.2017.06.014. Medline: [DOI] [PubMed] [Google Scholar]
- 53.Logan LR. Rehabilitation techniques to maximize spasticity management. Top Stroke Rehabil. 2011;18(3):203–11. 10.1310/tsr1803-203. Medline: [DOI] [PubMed] [Google Scholar]
- 54.Stewart C, McCluskey A, Ada L, Kuys S. Structure and feasibility of extra practice during stroke rehabilitation: a systematic scoping review. Aust Occup Ther J. 2017;64(3):204–17. 10.1111/1440-1630.12351. Medline: [DOI] [PubMed] [Google Scholar]
- 55.Russell W, Pritchard-Wiart L, Manns PJ. Clinician perspectives on cross-education in stroke rehabilitation. Disabil Rehabil. 2018;40(22):2644–9. 10.1080/09638288.2017.1356382. Medline: [DOI] [PubMed] [Google Scholar]
- 56.Vloothuis JDM, Mulder M, Veerbeek JM, et al. Caregiver-mediated exercises for improving outcomes after stroke. Cochrane Database Syst Rev. 2016;12(12):CD011058. 10.1002/14651858.CD011058.pub2. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Veerbeek JM, van Wegen E, van Peppen R, et al. What is the evidence for physical therapy poststroke? A systematic review and meta-analysis. PLoS One. 2014;9(2):e87987. 10.1371/journal.pone.0087987. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Van Vliet P, Pomeroy VM, Wolf SL, Kwakkel G. Time to empower people with stroke. J Neurol Phys Ther. 2015;39(3):139–41. 10.1097/NPT.0000000000000089. Medline: [DOI] [PubMed] [Google Scholar]
- 59.Lohse KR, Pathania A, Wegman R, Boyd LA, Lang CE. On the reporting of experimental and control therapies in stroke rehabilitation trials: a systematic review. Arch Phys Med Rehabil. 2018;99(7):1424–32. 10.1016/j.apmr.2017.12.024. Medline: [DOI] [PubMed] [Google Scholar]
- 60.McDonald MW, Hayward KS, Rosbergen ICM, Jeffers MS, Corbett D. Is environmental enrichment ready for clinical application in human post-stroke rehabilitation? Front Behav Neurosci. 2018;12:135. 10.3389/fnbeh.2018.00135. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Negrini S, Arienti C, Pollet J, et al. Clinical replicability of rehabilitation interventions in randomized controlled trials reported in main journals is inadequate. J Clin Epidemiol. 2019;114:108–17. 10.1016/j.jclinepi.2019.06.008. Medline: [DOI] [PubMed] [Google Scholar]
- 62.Negrini S, Arienti C, Kiekens C. Usual care: the big but unmanaged problem of rehabilitation evidence. Lancet. 2020;395(10221):337. 10.1016/S0140-6736(19)32553-X. Medline: [DOI] [PubMed] [Google Scholar]
- 63.Dixit S, Tedla JS. Effectiveness of robotics in improving upper extremity functions among people with neurological dysfunction: a systematic review. Int J Neurosci. 2019;129(4):369–83. 10.1080/00207454.2018.1536051. Medline: [DOI] [PubMed] [Google Scholar]
- 64.McCormack L, Sheridan S, Lewis M, et al. Communication and dissemination strategies to facilitate the use of health-related evidence. Evid Rep Technol Assess (Full Rep). 2013;(213):1–520. 10.23970/ahrqepcerta213. Medline: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Fowler FJ. Survey research methods. 2nd ed. New York: SAGE Publications Inc.; 1993. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.