Development and evaluation of an efficient training program to facilitate the adoption of a novel neurorehabilitation device

Abstract

Many rehabilitation devices are not adopted by therapists in practice. One major barrier is therapists’ limited time and resources to get training. The objective of this study was to develop/evaluate an efficient training program for a novel rehabilitation device. The program was developed based on structured interviews with seven therapists for training preference and composed of asynchronous and in-person trainings following efficient teaching methods. The training program was evaluated for six occupational therapy doctoral students and six licensed therapists in neurorehabilitation practice. Training effectiveness was evaluated in a simulated treatment session in which 3 trainees shifted their roles among therapist applying the device, client, and peer assessor. In results, 11 of the 12 trainees passed the assessment of using the device in simulated treatment sessions. One trainee did not pass because s/he did not plug in the device to charge at the end. The in-person training fit within 1-h lunch break. All trainees perceived that they could effectively use the device in their practice and both asynchronous and in-person training easily fit into their schedule. This project serves as an example for development of an efficient and effective training program for a novel rehabilitation device to facilitate clinical adoption.

Introduction

Advances in sensorimotor neurorehabilitation research have resulted in many new devices that could improve patient outcomes.^1–6 However, even with demonstrated clinical efficacy in clinical trials, new devices are hardly adopted by therapists in neurorehabilitation practice.^7,8 Multiple barriers to adoption of new rehabilitation devices in clinical practice have been identified.^9,10 For example, for robotic devices, main barriers for adoption include cost and clinical feasibility.¹¹ Yet, there are other barriers to implementation of many low-cost, low-profile rehabilitation devices.^6,12–16

One such barrier is therapists’ access to training.^17,18 Trainings are traditionally in person with non-reimbursable hours, leaving therapists the burden of attending trainings during their free time and using their own money to cover registration and travel costs.¹⁹ While many trainings and courses have been converted to virtual formats since the COVID-19 pandemic, they can last several hours or days during therapists’ free time. In addition, a complete virtual format does not allow hands-on, experiential training that many adult learners need.²⁰ These barriers limit therapists’ access to valuable trainings that could drive improved patient outcomes.

Our long-term goal is to facilitate therapists’ access to new rehabilitation device trainings. The purpose of this project was to develop and evaluate an efficient yet effective training program for a novel rehabilitation device, TheraBracelet. TheraBracelet is a wearable device that looks like a wristwatch.²¹ TheraBracelet delivers imperceptible random-frequency vibratory stimulation to the paretic wrist to increase cortical excitation and neural communication^22–24 and subsequently improve upper extremity sensorimotor control^25–30 and motor recovery in stroke survivors.^31–34 This device was chosen because it has not yet been introduced to the clinical market or clinicians, and we have intimate knowledge of the device.

Methods

The overall study design was as follows. First, we explored local therapists’ training preference by a structured interview. Second, we developed a training program using established teaching methods while satisfying the local therapists’ preference. Third, we administered the training program to local therapists and evaluated effectiveness and efficiency of the training. This study was deemed a quality improvement project by the local Institutional Review Board.

Participants

An email invitation was used to recruit participants. For the interview, seven local licensed occupational or physical therapists participated. They had an average of 15 years (standard deviation, SD = 10) of experience in outpatient (71%), acute (43%), and research (100%) settings. For the training program evaluation, 12 trainees were recruited. This sample size is sufficient to estimate a training pass rate of 95% for a margin of error of 12.5% with 95% confidence. All trainees were either interested or currently work in neurorehabilitation. Six of the trainees were students in their third, second, and first year of the Occupational Therapy Doctorate program (n = 3, 2, and 1, respectively). The other six trainees were licensed occupational and physical therapists (n = 3 each) with an average of 12.5 years (SD = 6.7) of experience. They had experience working in outpatient rehabilitation (83%), acute care (50%), and research (33%), with 67% of them having worked in more than one setting. We formed two student groups and two therapist groups. Each group included three trainees due to the COVID-19 pandemic to decrease exposure. The student groups underwent the training program and evaluation first, followed by the therapist groups.

Interviews on therapists’ training preference

Therapists were interviewed for the training preference via online video conference due to COVID-19 restrictions. Each therapist was interviewed individually to enable expression of their true thoughts, unaffected by group dynamics that may ensue in group interviews.³⁵ One person led the interview with the therapist, while the other person took notes. Using structured interview,³⁶ therapists were asked to choose their preferred training duration, time of the day, in-person versus virtual video conferencing training format, and training group size. Therapists were also asked to list barriers and facilitators for effective training, based on their reflection of previous trainings they attended.

Pedagogical framework and training program design

We developed a training program using established teaching methods while satisfying the local therapists’ training preference. The training program framework is shown in Figure 1. Learning objectives were laid out following Bloom’s Taxonomy.³⁷ To reduce in-person training durations, we used blended learning³⁸ in which some contents are delivered asynchronously via online, as opposed to all in-person learning. Blended learning has been used for training many health professions including medical students and surgeons and has been found to be an effective means of didactic education based on significantly improved knowledge and skills upon completion of the training.^39,40 We also used flipped classroom⁴¹ in which basic contents are studied independently and asynchronously by students, followed by applied hands-on learning during in-person class times. This is in contrast with the traditional instruction that starts with a presentation of the basic knowledge during class time, followed by applied hands-on activity to be completed asynchronously as assignments by students. Flipped classroom has been shown to be more effective than traditional instruction, with higher performance on assessments by undergraduate⁴² and health professions students.⁴¹ Flipped classroom has also been shown to be perceived favorably by students because they could pause, rewind, and review the video lectures which facilitates individualized learning.⁴²

Figure 1.

Training program framework following the Bloom’s Taxonomy. Four learning objectives are ordered from the lower to higher order thinking skills. Each learning objective is matched with corresponding learning activities and evaluations. Activities that can be completed by trainees independently are done asynchronously, while others are in person.

Training program evaluation

Training effectiveness was evaluated in the peer assessment during the simulated treatment session. The peer assessment evaluated trainees’ presentation of the expected outcomes to patients, proper device setup, and treatment design with the device, using pre-defined rubric (Table 1). Trainees were required to score at least “Clinically Acceptable” on all categories to pass the training. We also timed the in-person training duration to check if it indeed fit within the 1-h lunch break.

Table 1.

Peer assessment criteria and results. The total number of trainees that met each criterion is shown (with the number of student/therapist breakdown).

	N (student/therapist)
Device introduction
Excellent	9 (5/4)
1) Explains potential outcomes of using the device
2) Explains how the device works
3) Describes activities the patient will complete while using the device
4) Explains why the device is appropriate for this patient
Clinically Acceptable: Completes 1–3 of the above	3 (1/2)
Not Met: Does not introduce the device to patient	0 (0/0)
Device donning
Excellent	12 (6/6)
1) Checks patient’s skin prior to donning the device to patient’s wrist
2) Appropriately dons the device to patient’s wrist
Clinically Acceptable: Appropriately dons the device to patient’s wrist so that it is not too tight or too loose	0 (0/0)
Not Met: The device is too tight or too loose on patient’s wrist	0 (0/0)
Device set-up
Excellent: Sets up the device swiftly (2–3 min)	12 (6/6)
1) successfully logs into the software
2) Creates patient profile with first and last initials
3) successfully calibrates the device
Clinically Acceptable: Takes too long to complete above tasks (>3 min)	0 (0/0)
Not Met: Did not complete all the above tasks	0 (0/0)
Simulated Treatment Session
Excellent: Therapist constructs activities using the device on the affected hand >75% of session	12 (6/6)
Clinically Acceptable: 50–74% of session	0 (0/0)
Not Met: <50% of session	0 (0/0)
Device doffing
Excellent	11 (6/5)
1) doffs the device and plugs in both the device and the smartphone running the device software
2) Checks skin post session
Clinically Acceptable: Doffs the device and plugs in both the device and smartphone	0 (0/0)
Not Met: Doffs the device but does not plug in both the device & smartphone	1 (0/1)

In addition, perceived preparedness to use the device and the training program’s fit to their schedule were assessed using 5-point Likert scale (Table 2). For the asynchronous training, trainees rated those via REDCap^TM emailed along with the videos, prior to in-person training. For in-person training, trainees rated immediately after the in-person training on paper. They were asked to provide clarification as needed.

Table 2.

Trainees’ perceived preparedness to use the device and the training program’s fit with their schedule as shown in Likert scale. The total number of responders (with the number of doctoral student/therapist breakdown) is shown for each Likert scale.

	Strongly disagree	Disagree	Neutral	Agree	Strongly agree
Preparedness
I feel prepared to independently use the device after the asynchronous training	0 (0/0)	0 (0/0)	0 (0/0)	9 (6/3)	3 (0/3)
Both asynchronous and in-person training prepared me for using the device in practice	0 (0/0)	0 (0/0)	2 (1/1)	1 (0/1)	9 (5/4)
Fit with work schedule
The asynchronous training easily fit into my schedule	0 (0/0)	0 (0/0)	0 (0/0)	4 (4/0)	8 (2/6)
The in-person training easily fit into my schedule	0 (0/0)	0 (0/0)	0 (0/0)	1 (0/1)	11 (6/5)

Results

Therapists’ training preference

Four out of seven therapists preferred the training duration to be 60 min or less. The other three therapists suggested 2–3 hours of training. Six therapists preferred training on weekdays, while one stated any day of the week. Six therapists preferred trainings at lunchtime instead of before or after work. Five therapists preferred in-person training to video conferencing whereas two therapists were neutral. Six therapists preferred small group training (3–15 people), while one therapist preferred one-on-one, and none preferred large group training. Facilitators of effective training included 1) interaction between participants and presenter, 2) application to practice, 3) knowledgeable speakers, and 4) handouts. Barriers to effective training were 1) too long lecture and 2) more of a sales pitch than education on the device. These responses informed the format of the developed training program, including the in-person training during 1-h lunch break in a small group and ample opportunities for interaction with the presenters for questions and clarifications along with handouts.

Training program design

The training was designed to start with asynchronous online didactic lecture videos, followed by an in-person, hands-on practice. We created three videos of 3–6 min each, because student engagement is highest when information is chunked into small components.⁴³ The three videos described (1) the scientific mechanisms behind the device and the device hardware and software components, (2) patients that are appropriate for using the device and therapeutic outcomes patients may expect from using the device, and (3) device setup procedure and potential treatment activities. The videos were emailed to trainees 1 week prior to the scheduled in-person training, along with a reminder email 2 days prior.

The in-person training was designed to last less than 1 hour to fit as an in-service during therapists’ lunch break. At the beginning, didactic information was reviewed for spaced retrieval.⁴⁴ Spaced retrieval refers to retrieving the information for an activity or short test a few days after the information was learned and has been shown to be an effective way to improve long-term retention.⁴⁴ To ensure trainee attentiveness, trainees were informed of impending quizzes right after.⁴⁵ Anonymous quizzes were developed using Poll Everywhere^® to assess trainees’ didactic knowledge. First, using two case studies, trainees were asked to determine whether the cases are appropriate to use the device. Second, trainees were asked to write possible outcomes that patients may obtain from using the device. Incorrect answers or questions were immediately explained. The review and quizzes were designed to last for approximately 10–15 min.

Then followed simulated treatment sessions using the device for experiential learning.^20,46 Three-person trainee groups alternated being the therapist, patient, and peer assessor. A peer assessor graded the therapist based on pre-defined grading criteria (Table 1) and was asked to discuss what the therapist overlooked, if any, during debrief that followed immediately after each simulated session. This rotation combined with peer assessment allows each trainee to have hands-on practice using the device, evaluate another trainee, and reflect on each other’s performance, thereby enhancing learning.⁴⁷

Training program evaluation

Eleven of the 12 trainees passed the peer assessment by receiving at least “Clinically Acceptable” on all assessment categories (Table 1). Eight trainees received “Excellent” in all categories. One therapist trainee received “Not Met” for device doffing because s/he did not plug in the device for charging at the end of the simulated treatment. This trainee took the role of a therapist in the first simulated treatment session, before having a chance to observe others’ performance and grading. This error was well noted by the group during debrief and was not repeated. The trainee also ensured grading this part as a peer assessor for the other trainee. For the quizzes, 10 trainees correctly identified patient appropriateness for two case studies. Two student trainees incorrectly identified one of the cases and they were immediately explained. All student and therapist trainees correctly expressed outcomes from using TheraBracelet in the quiz. The mean duration of the in-person training was 39 min (SD = 10) with 23 min (SD = 8) for the simulated treatment sessions. The in-person training duration was less than 1 hour for all training groups.

All trainees agreed or strongly agreed that they felt prepared to independently use the device after the asynchronous training (Table 2). After the in-person training, 10 out of 12 trainees agreed or strongly agreed that both asynchronous and in-person training prepared them for using the device. The other two participants responded neutral and wrote that the asynchronous training alone was sufficient to use the device. All trainees agreed or strongly agreed that both the asynchronous and in-person trainings easily fit into their schedule.

Discussion

Efficient yet effective training that fits into therapists’ work schedule and creates preparedness to use the learned device is needed to facilitate therapists’ participation and learning to use novel rehabilitation devices and their adoption in rehabilitation practice. This study developed an efficient training program for a novel rehabilitation device based on therapists’ preferences and established pedagogical framework. This study demonstrated that the new rehabilitation device training meets the therapists’ preference for efficiency, by utilizing asynchronous training and 1-h lunch break for hands-on training. Efficiency did not sacrifice effectiveness, as most trainees demonstrated good performance during simulated treatment sessions, along with perceived preparedness to use the device in practice, and the program was designed based on the knowledge of established teaching methods. This work serves as a model to enhance therapists’ participation in training to increase device introduction in clinics.

Efficiency of learning how to use the device for practice was achieved using blended learning,³⁸ flipped classroom,⁴² and peer assessment.⁴⁷ By delivering the lecture content via asynchronous videos, we reduced the in-person training time. While basic knowledge may be obtained through independent, asynchronous learning, higher-level learning³⁷ in health professions may require experiential learning to achieve the mastery in applying new skills with practice.²⁰ Experiential learning of hands-on practice with simulated patients allows for application to clinical scenarios and reflection, which can solidify therapist preparedness to use a new device.^20,46 A randomized controlled study showed that online learning followed by hands-on lab practice resulted in the best learning outcome compared to only online or only hands-on lab for occupational therapy students learning how to evaluate community accessibility.⁴⁸ In addition, the social learning theory suggests that peer assessment in which a student assesses another student is beneficial for both the student assessor and the student being assessed as socialization and communication between students facilitate deeper learning.⁴⁷

The impact is that this work serves as a model for developing efficient yet effective training that fits into therapists’ work schedule and prepares them to apply novel rehabilitation devices to increase therapists’ device adoption in practice. Attending trainings and continuing education courses can be taxing on therapists. These trainings are typically not reimbursable, and impede therapists’ free time, negatively affecting their work life balance. Trainees in this study possibly experienced the training program efficiency as the program design was based on understanding of therapists’ preferences, making the program accessible, and the knowledge of established teaching methods and pedagogical framework. Efficient training is expected to facilitate therapists’ participation to learn new rehabilitation methods, while reducing encumbrance to their quality of life.^9,18 Better access to training for therapists could enhance translation of new rehabilitation methods from research to practice,⁷ thereby improving patient outcomes with enhanced evidence-based practice.

The in-person training was preferred by the therapist and was focused on experiential learning to allow trainees to learn from experience in applying new skills and subsequent interpretation and reflection.^20,46 Surprisingly, two out of 12 trainees reported that asynchronous training videos may be sufficient without in-person training. The possibility of making the in-person training optional may be considered for simple rehabilitation devices such as TheraBracelet, while complex systems may require in-person training and multiple training sessions to ensure mastery of skills.^39,40

Limitations

Generalizability is limited to the specific device. The training program was developed and evaluated for initial knowledge acquisition only, and not for retention or refresher training for long-term use. External validity could be improved with therapists outside of our local area working in outpatient neurorehabilitation clinics.

ORCID iD

Na J Seo https://orcid.org/0000-0001-6446-5905