Using remote learning to teach clinicians manual wheelchair skills: a cohort study with pre- vs post-training comparisons

Abstract

Purpose:

To test the hypothesis that remote learning to teach clinicians manual wheelchair skills is efficacious.

Materials and Methods:

A convenience sample of therapists (physical and occupational) and students were enrolled in pairs in a cohort study with pre- versus post-training comparisons. The intervention was a hybrid of self-study and hands-on practice paired with remote feedback for ten intermediate and advanced manual wheelchair skills. Participants practiced with self-selected frequency and duration, uploading a session log and video(s) to an online platform. A remote trainer provided asynchronous feedback prior to the next practice session. Capacity and confidence in completing the ten skills were evaluated using the Wheelchair Skills Test Questionnaire (WST-Q). Knowledge of wheelchair skills training and motor learning was assessed using a 62-item Knowledge Test. Secondary outcome measures included skill achievement, as confirmed by submitted video recordings, and participant feedback about the training.

Results:

Across 41participants, scores were higher at follow-up compared to baseline for WST-Q capacity (73.9±19.1 vs 16.8±15.6, p<0.001), WST-Q confidence (80.1±12.2 vs 47.6±18.2, p=0.003) and knowledge (70.8±7.5 vs 67.0±5.4, p=0.004).

Conclusions:

Remote learning can increase wheelchair skills capacity and confidence as well as knowledge about such training and assessment. This model should be further investigated as a delivery method for training rehabilitation professionals.

Clinical Trial Registration Number: NCT01807728

Introduction

Wheelchair skills not only optimize independence of the user and increase participation, they also decrease caregiver burden and reduce the likelihood of placement in a long-term-care facility.^1–4 Unfortunately, many wheelchair users still lack some of the skills they need. One reason for this deficit could be a lack of access to skilled trainers. In one study, only 8% of physical and occupational therapists reported training their clients in advanced wheelchair skills and cited lack of resources and uncertainty on how to implement wheelchair skills training into practice as major barriers to providing such training.⁵

The Wheelchair Skills Program (WSP) has been shown to be effective in training wheelchair users and caregivers with the standard set of skills necessary to traverse environments for everyday life.^6–12 This has been documented in 16 randomized controlled trials (RCTs) to date and two meta-analyses.^6,13,14 To date, the WSP has been delivered to health-care professionals through in-person training with a train-the-trainer approach, but this method has limitations.^7,15,16 Clinicians are only trained during the course itself and then return to practice without additional feedback in clinical settings. Additionally, this type of training is inaccessible for many clinicians due to the need for time away from patient care for training as well as the need to pay for the clinician to travel to the course or for the trainer to travel to the clinic to provide the course locally.

Given these limitations, other approaches to make training more accessible could significantly increase the number of clinicians available to provide skills training.¹⁷ One method to address the inaccessibility of trainer training is use of the Internet. Online learning can increase the accessibility of educational tools, is effective as a learning method, and has been proven to be effective in other areas of healthcare education.^18–21 However, to our knowledge, there have been no previous studies evaluating the utility of online education for motor learning, as distinct from meeting cognitive objectives.

The goal of this study was to evaluate the effectiveness of remote learning for clinicians using a cohort study with pre- and post-comparisons. We hypothesized that, similar to previous studies of in-person wheelchair skills training for clinicians, participating clinicians would improve in their capacity and confidence in performing wheelchair skills as well as their knowledge of wheelchair skills assessment and training principles.

Materials and Methods

Study design

This was a cohort study with pre- vs post-training comparisons.

Setting

Participants were enrolled between April 2017 and March 2018 across four Spinal Cord Injury Model Systems Centres: University of Pittsburgh Model Center on Spinal Cord Injury, Northern New Jersey Spinal Cord Injury System, South Florida Spinal Cord Injury Model System, and Midwest Regional Spinal Cord Injury Care System. The University of Pittsburgh Institutional Review Board approved this work and participants provided informed consent. The remote trainer was a content expert and an experienced trainer-of-trainers with over 40 years of experience in wheelchair training and provision.

Inclusion/exclusion criteria

Participants were enrolled in pairs to ensure a spotter for safety when practicing skills; the pairs were told that they would complete the study intervention together. Participants were eligible to participate if they were able-bodied with no restrictions regarding activities of daily living, between 18 and 75 years of age, without injury that would preclude wheelchair use, and completed baseline assessment. Participants were permitted to have previous wheelchair experience.

Demographic data

At baseline participants completed a demographic questionnaire to record age (years), sex (male/female), race (Caucasian, African-American, Hispanic or Asian American), clinical discipline (physical therapy, occupational therapy, therapy assistant, other), whether they were still students (yes/no), whether they were certified Assistive Technology Professionals (yes/no) and their reasons for enrolling in the study (open-ended response). Demographic questionnaires were completed online through the Qualitrics Online Survey System.

Sample size

We used a power analysis based on the published study completed with occupational therapy students¹⁶ with a mean ± SD total percentage pre-training Wheelchair Skills Test (WST) score of 64.8%±9.0 and post-training score of 81.0%±5.2. This suggested that we would need a sample size of 23 (6 per site) for a 95% power to detect such a difference between pre/post. Due to withdrawals before the intervention at one site (figure 2), we over-enrolled participants to ensure adequate statistical power for primary and secondary analyses.

Figure 2:

Flow chart of participation in study activities.

Intervention

The training was a hybrid of self-study followed by practice paired with remote feedback. It utilized the assessment and training protocols of the Wheelchair Skills Program (WSP),²² which are supported by a body of research evidence showing effectiveness in teaching clinicians and wheelchair users.^13,23 The training targeted 10 intermediate and advanced skills (table 3) in the WSP, selected based on the most frequent patient-selected goals in a study of individuals with spinal cord injury⁹, that built on the transient caster pop to master obstacles (“gets over gap”, “gets over threshold”, “ascends low curb”, “descends low curb”, “ascends high curb”, “descends high curb”) and wheelies (“performs stationary wheelie”, “turns in place in wheelie position”, “descends high curb in wheelie position” and “descends steep incline in wheelie position”).

Table 3:

Participant Post-Training Success Based on Video Review Across Skills Ordered by Achievement (n = 41)

Skill	n (%) Participants Successful*
Gets over threshold (2cm)	40 (98%)
Gets over gap (15cm across)	38 (93%)
Ascends low curb (5cm)	35 (85%)
Descends low curb (5cm)	33 (80%)
Performs stationary wheelie (30 seconds)	33 (80%)
Turns in place in wheelie position (180°)	26 (63%)
Descends high curb (15cm)	22 (54%)
Ascends high curb (15cm)	20 (49%)
Performs wheelie down high curb (15cm)	19 (46%)
Performs wheelie down steep incline (10°)	18 (44%)

^*n (%) of participants with Wheelchair Skills Tests (WST) scores of 2

A flow diagram of the intervention can be seen in figure 1. File exchange between the learner and the remote trainer took place on University of Pittsburgh Box with a private folder created for each participant. On this platform, participants were provided with an electronic copy of the WSP Manual Version 4.3²² and instructed to review the introduction to wheelchair skills training (Sections 5.1–5.22), spotter safety (Sections 6.1–6.16), and Sections 7.27–7.35 on each of the 10 skills. Instructional video-recordings extracted from a full-day in-person WSP training-of-trainers as part of a previously reported study⁹ were also provided for each of these skills.

Figure 1:

Study design for baseline assessment, material review, training feedback loop, and follow-up assessment.

Participants were instructed to practice the skills in pairs, with self-selected frequency and duration of sessions. Each site made manual wheelchairs available for participants and suggestions regarding the settings in which practice might take place. At the end of each session, participants were asked to complete a training log either by recording responses through video or by logging responses in a Word document. Participant logs included the participant identification, date and location of training, minutes spent practicing, minutes spent spotting, skills practiced, successes with skills, difficulties encountered, and the name of video-recording(s) submitted to study staff.

The remote trainer reviewed the video-recordings, sometimes using slow-motion, and written feedback was provided asynchronously, with a target latency of no more than 2 days. The feedback was provided on a form developed for the purpose and included overall comments followed by feedback for each skill on the setting, video quality, WST capacity score, spotter issues, and learner issues.

Outcome Measures

All questionnaires were completed online through the Qualtrics Online Survey System and are described in table 1. For the Wheelchair Skills Test Questionnaire (WST-Q) we calculated the total percentage score for only the 10 items targeted by the study intervention. Participants were asked to upload either video-recording(s) representative of their attempts (in the case they did not master the skill during the session) or video-recording(s) showing skill mastery. All participants used personal smartphones for video recording. Skill achievement was defined by a WST capacity scores of 2 [“pass”] by study staff based on the submitted video-recordings. At the end of the training, participants completed a feedback survey that mirrored previous in-person wheelchair skills²⁴ and maintenance²⁵ training of clinicians.

Table 1:

Primary and Secondary Outcome Measures

Primary Outcome Measures
Outcome Measure		# Items	Domain	Responses	Scoring
Wheelchair Skills Test Questionnaire 4.3 (WST-Q) 22,30	Capacity	10	“Can you complete this skill?”	“Yes” (2), “Yes with difficulty” (1), “No” (0).	0–100%
	Confidence	10	“How confident are you?”	“Fully” (2), “Somewhat” (1) and “Not at all” (0).	0–100%
Knowledge Test 15		62	Overall knowledge of wheelchair skill assessment and training (e.g. “Augmented feedback can hinder learning in some situations”, “As the fading process leads to less and less frequent feedback, the trainer should focus on only the most recent attempt rather than summarizing a series of attempts”)	True/False	0–100%
Secondary Outcome Measures
Outcome Measure				Responses
Feedback Survey	Did you find the workshop:		Useful?	Strongly disagree, somewhat disagree, neutral, somewhat agree, strongly agree
			Relevant?
			Easily Tolerated?
			Understandable?
			Enjoyable?
	Was the duration of the session:			Too long, just right, too short
	Would you encourage others to participate in this training program?			Definitely yes, probably yes, not sure, probably not, definitely not
	Please describe any part of the course		You found difficult to understand	Open ended
			That will be useful in your own life
	Please describe any content we should		Add or emphasize in the course
			Reduce or remove from the course
	Please describe any other improvements you would make to the course
Outcome Measure				Scoring
Wheelchair Skills Test (WST) 22,31				“Pass” (2), “Pass with difficulty” (1), “Fail” (0)

Data Analysis

Analysis was completed using IBM SPSS Statistics version 24 with an alpha level of 0.05. Baseline and follow-up WST-Q confidence and knowledge scores were compared using paired t-tests. WST-Q capacity was not normally distributed and was assessed using the Wilcoxon Sign Rank test. Stepwise linear regression was used to determine if the following independent variables were predictors for post-pre-training change in WST-Q capacity and confidence scores: total practice time, discipline, baseline capacity, and baseline confidence. Discipline was dichotomized to licensed professional (0) or student (1). Because the number of sessions and time spent practicing and spotting were highly correlated (R=0.888, p<0.001), practice time was selected as the most clinically relevant predictor to include. Descriptive analysis was used to determine the percent of participants who achieved skill success based on video review and to analyze participant feedback on surveys.

To investigate potential attrition bias, between-group comparisons were made based on those who completed follow-up and those who did not. Sex, race, age, clinical discipline, motivation for participation in the study, years of experience and baseline WST-Q scores were compared using Chi square or independent t-tests with the exception of subtotal WST-Q capacity (Mann Whitney U test). Additionally, a sensitivity analysis for the primary outcome measures (change in WST-Q capacity, WST-Q confidence and knowledge) was completed using both last observation carried forward and multiple imputation using fully conditional specification methods.²⁶

Results

Participants

We enrolled a convenience sample of thirty-six pairs (n=72 participants), with 41 participants completing all study activities. Participant enrollment and completion of study activities are detailed in Figure 2. While not an inclusion criterion, all participants were clinicians. Participants enrolled primarily as pairs in the same discipline: 11 pairs of physical therapists, 7 pairs of physical therapy students, 1 pair of occupational therapists, 1 pair of a physical therapist with a physical therapy assistant, and 3 pairs of occupational therapists with physical therapists.

Participant demographics and baseline wheelchair skills proficiency are presented in table 2; we found no significant differences between participants who did/did not complete the intervention and/or follow-up data collection. No participants were Assistive Technology Professionals.

Table 2:

Participant demographics and baseline proficiency in wheelchair skills

Participant Characteristics	Participants with Complete Data (n=41)	Participants Who Did Not Complete the Intervention and/or Missing Follow-up Data (n=9)	Between Groups Comparison
n (%)			Significance (p-value)
Female	37 (90.2%)	8 (88.9%)	0.960
Caucasian	32 (78.0%)	8 (88.9%)	0.619
Clinical Discipline			0.440
Licensed Clinician (PT, OT)	31 (75.6%)	7 (77.8%)
Physical Therapy Student	10 (24.4%)	2 (22.2%)
Motivation for Participation			0.913
Interest	38 (92.7%)	8 (88.9%)
Job requirement	3 (7.3%)	1 (11.1%)
Site			0.051
Site 1	10 (24.4%)	4 (44.4%)
Site 2	3 (7.3%)	3 (33.1%)
Site 3	13 (31.7%)	1 (11.1%)
Site 4	15 (36.6%)	1 (11.1%)
Mean (SD)			Significance (p-value)
Age	32.2 (6.7)	31.2 (6.7)	0.122
Years of Experience	5.7 (6.6)	5.7 (6.6)	0.129
Baseline Wheelchair Skills Test Questionnaire (WST-Q)
Capacity	16.9 (15.5)	9.4 (9.2)	0.140
Confidence	47.7 (18.8)	37.8 (12.5)	0.133
Knowledge	67.0 (5.4)	64.5 (5.1)	0.586

Sessions

Participants completed the hands-on portion of the intervention in a mean ± SD of 3.1 (1.7) sessions. The mean (SD) time spent practicing and spotting in a session was 27.6 (17.2) and 26.1 (16.0) minutes, respectively. Session durations ranged from 15–120 minutes. The mean (SD) total time for practice and spotting across all sessions was 91.2 (41.3) and 86.3 (35.3) minutes, respectively. Total time for practice and spotting combined ranged from 40–330 minutes.

Wheelchair Skills Test Questionnaire

Participants demonstrated statistically significant improvements compared to baseline in WST-Q capacity (73.9±19.1 vs 16.8±15.6, p<0.001), and confidence (80.1±12.2 vs 47.6±18.2, p=0.003) following the intervention). The sensitivity analyses demonstrated that the results for the full sample were similar in magnitude, direction and extent of statistical significance to those for the 41 participants who completed all components of the study (Supplemental Material).

Almost two-thirds (61.8%) of the variance in change in WST-Q capacity was explained by the multiple regression equation (F[3,32] = 17.268, p<0.001), with significant predictors being lower baseline capacity (β= −0.728, p<0.001), being a student versus a professional (β= 19.193, p<0.001), and more time spent practicing (β= 0.182, p=0.004). About three-quarters (81.7%) of the variance in change in WST-Q confidence was explained by the multiple regression equation (F[3,32] = 46.279, p<0.001) with significant predictors of lower baseline confidence (β= −0.586, p<0.001), being a student (β= 13.009, p=0.001), and more time spent practicing (β= 0.120, p=0.002).

Knowledge Test

Participant scores on the Knowledge Test improved significantly post-training (70.8±7.5 vs 67.0±5.4, p=0.004).

Wheelchair Skills Test

A summary of skill achievement, presented as the percentage of participants who were successful based on remote WST scoring, can be found in table 3.

Participant Feedback

Participants considered that the training was useful (n=38, 93%), relevant (n=39, 95%), easily tolerated (n=35, 85%), and that it was understandable (n=36, 88%). The duration of training was considered “just right” by 36 (80%) whereas 1 (2%) felt it was “too short” and 4 (10%) too long. Thirty-three (80%) participants would encourage others to participate in this type of training. Participants who reported dissatisfaction with the training stated they were under the impression the study would take less time to complete and cited challenges coordinating time with their partners to practice. Additional details on content of the learner feedback and perceived challenges can be found in the Supplemental Appendix.

Trainer Feedback

Participants received feedback from the remote trainer after a median [IQR] delay of 1 [1,4] days. Details on the content of the remote trainer’s feedback, learner issues, and challenges perceived by the remote trainer can be found in the Supplemental Appendix.

Discussion

We met our study objectives, corroborating the hypothesis that remote learning resulted in significant improvements in wheelchair skills capacity and confidence to complete a set of intermediate and advanced skills.

Practical training was completed across an average of three sessions, each lasting approximately one hour (inclusive of both practice and spotting time). Distributed training of this type better supports motor learning as compared to one-day massed practice (which is sometimes necessitated for in-person training of trainers).²² The significant improvements seen with an average total of 83.1 minutes of practice suggests that it is possible to learn these skills through a minimal time commitment in the context of competing clinical loads or coursework.

The magnitude of improvements in WST-Q scores are comparable to previous studies using ‘boot-camp’ style WSP training for clinicians.^7,16,27 On the basis of the regression analysis, greater WST-Q improvements were predicted by lower baseline scores, status as a student, and more time spent practicing. This suggests that those with less skill and greater time available to practice may benefit most from this type of intervention.

The breadth of participants’ backgrounds across therapy disciplines and inclusion of students indicates a wide interest in learning wheelchair skills. Although many of the participants were experienced therapists, baseline total WST-Q scores suggest that these participants had not received adequate education related to wheelchair skills training prior to the study.

Although dosage was not prescribed in this study, future studies or clinical implementation of remote learning can be informed by the time participants spent on practice in this study. The finding that improvements are related to the duration of training is similar to that of the motor skills literature, in general,²² and to the study by Worobey et al.,⁹ in particular, who reported that learners who spent more time practicing showed greater improvements in both capacity and confidence. Of note, the maximum time spent on practice across participants was less than 3 hours and the mean was only about half of that.

Compared to the WST-Q, Knowledge scores were higher at baseline, which may reflect general knowledge of motor learning principles among therapists and students. However, Knowledge scores did improve to a slight but statistically significant extent, indicating that learners increased their knowledge training and assessment. Content for this domain was primarily covered through self-review of the WSP Manual. Training videos were positively reviewed as compared to reading the WSP Manual, and a comparatively smaller improvement in the Knowledge outcome measure may speak to learner preferences on how material is delivered.

Post-training skill achievement was less than 50% for the advanced-wheelie and high-curb skills. Some participants did not feel comfortable trying these skills. Success was determined based on the researchers’ review of videos, so if the participant did not upload a video of the skill or if they uploaded a video that did not meet the requirements of skills as outlined by the WSP, they were not deemed successful.

The feedback from participants was generally positive. Some participants provided feedback that they would have preferred to be prescribed training dosage with a set schedule and timeline for practice sessions. This was not provided so as to provide flexibility for participants to practice on their own schedules and to learn at their own paces. Autonomy in practice conditions (e.g. timing of practice and feedback) has been found to be more effective than a more structured regime.²²

Some participants reported that limited access to equipment and obstacles was a barrier. Although challenges corresponding to those addressed by wheelchair skills training are readily available in the natural and built environments near most rehabilitation facilities, having these in one place makes practice easier. Better guidelines for participants and study site leads would probably have been beneficial. Participants’ preference for video-based instructions is useful feedback for the future.

Study Limitations

There were a number of study limitations, some of which have already been noted. Participants were enrolled in pairs to ensure safety through spotting. When one member of the dyad was unable to continue, this forced both members to stop participation. Some dyads were not located at the same clinical site and/or had conflicting schedules, leading participants to withdraw from the study. Time constraint was a common reason for withdrawing prior to the study intervention, which speaks to challenges with completing additional training in the context of current productivity demands on clinicians. Despite the drop-outs, we still had adequate statistical power to test our hypotheses because we enrolled more participants than the power analysis suggested were necessary. Also, the sensitivity analyses showed that, despite these challenges, results were similar when the cohort was analyzed as a whole versus just those who completed the study. Nevertheless, those who did not complete the study may have been less positive in feedback about the intervention.

Regarding study design, the total dosage of the training was low (average of <3 hours) and increased training time might have improved results. Further, based on individual skill success rates, training was not completed to “mastery”. Finally, we did not evaluate for the impact of the training intervention on spotting capacity/confidence or the learner’s ability to subsequently train others, which could be evaluated in future studies.²⁸

Future Studies

Similar to previous work^7,16,27, this study determined the impact of training on a clinician’s ability and confidence to perform wheelchair skills. Future studies should explore not just a trainer’s own skills following this type of training, but also their ability to teach others how to perform the skills. Complementary materials could be incorporated that provide a focus on a train-the-trainer approach including how to deliver the training to wheelchair users. This was a cohort study and future work should consider use of a comparison group, potentially through a randomized wait-list control design. It could also be valuable to determine the relative contributions of different components of the intervention: self-study, dyad practice, and asynchronous feedback. Although the design of this study utilized dyads of learners, in implementation there may be benefits to completing the intervention in larger cohorts. As such, a pool of spotters would be available and there would less impact of a single individual being unable to continue with the intervention. Additionally, future studies could investigate remote training in a direct-to-user model that may help to provide training in areas where a trained clinician is not available or instances, such as the current global pandemic, in which face-to-face training may not be possible or preferred.

Despite the study limitations and the need for further study, our findings have important implications for the future training of healthcare professionals regarding wheelchair skills (and potentially other motor skills). The implications include learners in less-resourced settings where there is limited access to skilled trainers. Even in well-resourced settings, the COVID-19 pandemic of 2020 has highlighted the desirability of being able to learn without in-person contact between learners and trainers.²⁹

Conclusions

Self-study and practice in pairs complemented by remote asynchronous feedback is effective in increasing capacity and confidence to complete a set of ten intermediate and advanced manual wheelchair skills. This model should be further investigated as a delivery method for improving knowledge translation of wheelchair skills training.