Abstract
Background:
No recognized standards exist for teaching patients and their caregivers ventricular assist device (VAD) self-care skills. We compared the effectiveness of a VAD simulation-based mastery learning (SBML) self-care training curriculum with usual VAD self-care training.
Methods and Results:
VAD patients and their caregivers were randomized to SBML or usual training during their implant hospitalization. The SBML group completed a pretest on 3 VAD self-care skills (controller, power source, and dressing change), then viewed videos and participated in deliberate practice on a simulator. SBML participants took a posttest and were required to meet or exceed a minimum passing standard (MPS) for each of the skills. The usual training group completed the existing institutional VAD self-care teaching protocol. Before hospital discharge, the SBML and usual training groups took the same 3 VAD self-care skills tests. We compared demographic and clinical information, self-confidence, total participant training time and skills performance between groups. Forty participants completed the study in each group. There were no differences in demographic and clinical information, self-confidence or training time between groups. More participants in the SBML group met the MPS compared to the usual training group for controller [37/40 (93%) vs. 25/40 (63%); p=0.001], power source [36/40 (90%) vs. 9/40 (23%), p<0.001], and dressing change skills [19/20 (95%) vs. 0/20, p<.001].
Conclusions:
SBML provided superior VAD self-care skills learning outcomes compared to usual training. This study has important implications for patients due to the morbidity and mortality associated with improper VAD self-care.
Clinical Trial Registration:
Since 2006, over 25,000 ventricular assist devices (VADs) have been implanted in patients with advanced heart failure and registered in the Interagency Registry for Mechanically Assisted Circulatory Support (INTERMACS).1 Although clinical outcomes continue to improve, with 2-year survival rates of newer generation devices approaching 83%,2 potentially preventable complications that negatively impact survival and health-related quality of life (HRQOL) remain.1,3 After implantation of a VAD, patients and their caregivers need to acquire a significant amount of knowledge and skills related to VAD self-care in a very short time. They must master important skills including controller, power source, and sterile driveline exit site dressing changes, while simultaneously adapting to life with a VAD, understanding new medication and dosing schedules, and accepting responsibility for avoiding adverse events including driveline infections and pump failure.4 The training patients and caregivers receive to perform these crucial skills occurs after device implantation but before hospital discharge and may be suboptimal due to the often emergent nature of the procedure and availability of patients, caregivers, and training staff.
It is critical that VAD patients and caregivers have the knowledge and skills to accurately and effectively perform necessary self-care skills. In 2017, the Food and Drug Administration (FDA) recalled the HeartMate II™ after 26 patients died from the inability to change the controller.5 This recall was attributed to patients lacking proper training on controller changes “because it can present a significant challenge to patients that are … untrained. For these patients, a slow or improper driveline changeover places them at risk of serious injury or death.”5 The device was reinstated after a human factors change was made to make inserting the driveline into the controller easier, yet patient and caregiver education was not addressed. This is related to the fact that there are no universal standards delineating the content or process for teaching patients and their caregivers crucial VAD self-care skills. Yarboro et al. demonstrated that a multidisciplinary approach including rigorous education of patients and caregivers significantly reduced driveline site infections.6 Thus, a rigorous VAD self-care training curriculum has both the potential to reduce risks such as driveline infections or death from improper controller changes and the opportunity to set a standard for training.
Simulation-based mastery learning (SBML) has the potential to provide rigorous VAD self-care training and set a universal standard. SBML is an intense form of competency-based learning performed on a mannequin in which all learners are required to meet or exceed a minimum passing standard (MPS) on a knowledge-based examination and/or skills checklist before completion of training. This model prevents uneven skill acquisition and improves patient safety.7–14 Learners who are unable to meet the MPS undergo further study of knowledge components or deliberate skills practice until they can be retested and meet the standard. In mastery learning, time varies but learning outcomes are uniform. SBML has been successfully used to improve skills of nurses,10 medical students,15 residents,7 fellows,8 and attending physicians16 and provides superior educational outcomes compared to non-SBML simulation training.17 Although simulation training is widespread in medical education, little is known about use of simulation-based education for patients and caregivers18–21 and mastery learning has never been used for these groups.
We recently performed interviews of VAD patients, caregivers, VAD coordinators (nurses and advanced practice providers), and physicians on perceptions of VAD training, identifying aspects perceived to be difficult and easy for patients and caregivers; barriers and facilitators to learning and training; VAD coordinators’ experience with training processes; and perceived critical aspects of VAD self-care training.22 Based on these interviews and known evidence and best practices, we created a VAD SBML self-care curriculum for patients and caregivers to fill potential gaps in training and address VAD self-care needs.22 The purpose of the current study was to compare the performance of VAD self-care knowledge, skills, and self-confidence between participants educated with SBML vs. usual VAD training protocols. We hypothesized that VAD patients and/or their caregivers would demonstrate better VAD self-care knowledge, skills on controller, power source and dressing changes, and self-confidence after the SBML intervention compared to patients and caregivers undergoing usual VAD training.
METHODS
Trial Design / Setting
We performed a single center randomized-controlled pilot trial of a VAD self-care SBML curriculum for patients and caregivers at Northwestern Memorial Hospital (NMH) from June 2017 to February 2019. NMH is a tertiary care academic medical center in Chicago with 894 beds, where more than 50 VADs are implanted in patients annually. Our primary endpoint was a comparison of knowledge and skills performance between patients and caregivers participating in SBML compared to those participating in usual (control) VAD self-care training. Our secondary endpoints included associations between sociodemographic and clinical information, self-confidence, and training time and knowledge and skills performance between groups. The Northwestern University Institutional Review Board approved this study. Anonymized data will be made publicly available upon completion of final analyses of 1-year clinical outcomes of this trial.
Sample
Patients with advanced heart failure refractory to guideline-directed medical therapy who were scheduled to receive an left, right, or biventricular assist device (HeartWare™, HeartMate II™, and HeartMate 3™ devices) and their caregivers were eligible to participate in the study. Patients identified one person as their primary caregiver who was then eligible to participate in the trial. Inclusion criteria were ≥18 years of age and ability to read and speak English. Patients without designated caregivers were excluded from the study. Data from participants were censored at time of patient death or heart transplant.
Patients and caregivers were enrolled by RSH, KBS, JSK or GPN and provided informed consent before VAD implant surgery whenever possible. However, if the patient was too ill to provide consent before surgery, consent was obtained after surgery. After providing written informed consent, patient and caregiver pairs were randomized (1:1 ratio by ERC) to the SBML intervention or usual training by a computerized “coin flip.” VAD teaching was provided during the patients’ implant hospital stay. Patients and caregivers were not compensated to participate in the study because VAD training was a necessary part of their hospital care. Patient and caregiver training occurred in an empty conference room or at the patient’s bedside.22
SBML Intervention
The SBML intervention, described in detail previously, was taught by 3 VAD coordinators (RSH, KBS, GPN).22 In brief, patients and caregivers first underwent a baseline (pretest) assessment on 3 VAD self-care skills using a simulator (Figure 1) and previously published skills checklists.22 This assessment included controller, power source, and driveline dressing changes (caregiver only). Subsequently, patients and caregivers watched videos created by the investigators (also provided to participants at discharge) on controller and dressing changes. The dressing change video included images of abnormal driveline exit skin sites. Patients and caregivers also were provided with a manufacturer’s video about life with a VAD. Written materials regarding medications and VAD parameter information were reviewed with patients and caregivers by the VAD coordinators, with opportunities for questions and clarification if needed. Visual reminder cards explaining and documenting needed actions for common VAD alarms were used during training and were provided to patients and caregivers for reference at discharge. Subsequently, the patient and caregiver participated in deliberate practice of controller, power source, and dressing changes (caregiver only) on the simulator (minimum of 3 times for each procedure). Finally, participants underwent a posttest of skills on controller, power source, and dressing changes (caregiver only) on the simulator using the same skills checklist from baseline testing. All participants were required to meet or exceed a previously established MPS on the relevant checklists for each skills examination.22 Those who were unable to meet or exceed the MPS underwent further deliberate skills practice on the simulator until they were able to achieve the MPS upon retesting. Participants were not shown the skills checklists. Instead they received specific feedback that they could incorporate into further practice sessions as necessary.
Figure 1.
Image of a VAD Coordinator Supervising a Caregiver Participating in Deliberate Practice of a Driveline Exit Site Dressing Change on the Simulator. Photo by Kim H Le Mezo.
All SBML training and testing sessions were standardized and performed by the 3 VAD coordinators who were trained on how to the perform the SBML intervention and calibrated on checklist scoring.22 Random audits were performed by 2 investigators (JHB, JEW) on 20% of training/testing sessions to ensure fidelity of the intervention.
Usual Training
The usual training group was taught by 2 VAD coordinators (LES, AMJ) using an institutional approved VAD self-care training protocol already in use before this study began. Patients and caregivers in this group also were provided with a manufacturer’s video about life with a VAD. VAD coordinators trained these patients and caregivers on controller, power source, and driveline dressing changes using a non-implanted VAD. Dressing change training was performed on a table or the patient. VAD coordinators also verbally reviewed critical alarms and medications (but reminder cards were not provided). Training sessions occurred 2 or more times for both the patient and caregiver depending upon availability. Before patient discharge, the VAD coordinator observed the skills of patients and caregivers. Training was deemed complete when a) the patient and caregivers were confident they could perform the skills and b) the VAD coordinator believed the patient and caregiver were proficient.
SBML and Usual Training Groups’ Assessments at Hospital Discharge
At the time of hospital discharge and following training, both groups completed knowledge and skills examinations. The knowledge exam was previously developed as an open book/source written examination.22 Written exam responses were reviewed in person with a VAD coordinator, based on group assignment (RSH, KBS, GPN or LES, AMJ). Patients and caregivers in the SBML group were required to meet or exceed an MPS on the written examination and remediated the exam as required to meet this standard.
Subsequently, groups participated in a discharge test of VAD self-care skills including controller, power source, and driveline dressing changes (caregiver only) using the skills checklist. The VAD coordinators who were trained and calibrated to use the checklists (RSH, KBS, GPN) performed all discharge skills testing. Controller changes were performed on the simulator, while power source and dressing changes were performed on the patient when possible (using the simulator as a back-up). All participants, regardless of group assignment, were corrected if they made errors that could cause patient harm (e.g., a caregiver that violated sterile technique during a dressing change would be required to go through the cleaning or gloving procedure again). Participants in the usual training group who were unable to meet or exceed the MPS on the discharge tests were given further training but were not retested, whereas those in the SBML group were required to retest until meeting the MPS. Figure 2 summarizes training for SBML and usual trained participants.
Figure 2.
Flow diagram Showing the Education Occurring in the Simulation-based Mastery Learning and the Usual Training Groups of the Study.
Measurement
Sociodemographic characteristics were collected from participant self-report questionnaires including data on patient and caregiver age, sex, race/ethnicity, marital status, number of children, maximum education level achieved, employment status, and medical insurance type (patient only). Medical records data collected included date of implant, VAD type (HeartWare™ or HeartMate II™ and HeartMate 3™ devices), implant strategy (bridge to transplant or destination therapy), ventricle(s) supported, reason for implant, INTERMACS profile, New York Heart Association (NYHA) class, glomerular filtration rate, and presence or absence of diabetes or lung disease.
Separate written examinations and skills checklists were previously created for the HeartWare™ and HeartMate™ devices.22 The written examinations were adaptations of examinations created by manufacturers and included multiple choice, fill in the blank, and matching questions on VAD self-care topics including controller functions, alarms, and medications. Checklists were created for each of the 3 skills: controller, power source, and driveline dressing changes. The checklist scoring was dichotomous and graded as 1= done correctly or 0 = not done/done incorrectly. Details on the creation of both the written examinations and skills checklists are published and can be found elsewhere.22 The MPSs were previously established using the Mastery Angoff technique.22
The established MPSs allowed patients and caregivers to miss no more than 1 of 35 items on the written examination, 0 of 12 items on the controller change skills checklist, 0 of 9 items on the power source change checklist, and 1 of 32 items on the dressing change skills checklist for the HeartWare™ device. Participants could miss no more than 1 of 54 items on the written examination, 0 of 12 items on the controller change skills checklist, 0 of 12 items on the power source change checklist, and 2 of 32 items on the dressing change skills checklist for the HeartMate™ devices.22 For the dressing change checklist, if sterile technique was compromised or there was apparent damaging pull to the driveline, SBML training participants automatically needed to repeat training and testing, regardless of the final checklist score. A random sample of 20 percent of all pre- and posttests were also scored in real time by a second VAD coordinator (either RSH, KBS, or GPN) who was blinded to pre- or posttest status. A random sample of 20 percent of discharge tests were co-graded in real time by 1 of 2 authors (JHB, JEW) who were blinded to group assignment (SBML or usual training).
Before discharge testing, patients and caregivers reported self-confidence for 5 VAD self-care skills including controller, power source, and dressing changes, as well as troubleshooting emergency VAD malfunction, and recognizing signs and symptoms that require immediate contact with the VAD team. The self-confidence questionnaire was adapted from previously created self-confidence questionnaires for clinical skills and was scored from 0 = very low confidence to 100 = very high confidence.8,10,16,23–25 All training time including testing, reviewing information, practice, and watching videos was logged by VAD coordinators.
Statistical Analyses
We report standardized differences for sociodemographic and clinical characteristics between the SBML and usual training groups. We compared self-confidence and total time spent training between the SBML and usual training groups using the independent t-test or Wilcoxon Rank Sum test. We compared pre- to posttests scores of patients and their caregivers in the SBML trained group using the Wilcoxon Signed Rank test. We used the Wilcoxon Rank Sum test to compare discharge test score differences between the SBML trained and usual training group for both patients and caregivers. We used the Chi-Square test (or Fisher’s Exact test if the expected cell size was less than 5) to compare the percent of patients and caregivers meeting the MPS between the two groups. We calculated inter-observer checklist agreement on the pre-, post-, and discharge tests using the Kappa coefficient.
We used Spearman’s Rho coefficient and Wilcoxon Rank Sum or independent t-tests to evaluate associations between sociodemographic and clinical information, self-confidence, and training time with discharge test skills performance. We did not perform multiple regression analyses to evaluate the effect of sociodemographic and clinical variables, self-confidence, and time differences between groups on skills performance because all univariate associations between these variables and skills performance were non-significant (p>0.10). Statistical analyses were performed using IBM SPSS version 25 (Chicago, IL).
Sample Size Calculation
Our prior work shows a difference between SBML and usual training skills performance scores of approximately 33% among practicing clinicians. SBML skills checklist performance averages 94.9% (SD=6.6) items correct, while usual trained performance averages 61.5% (SD=24.5) for procedures including paracentesis, central line insertion and maintenance, cardiac auscultation, lumbar puncture, and thoracentesis.7,8,10–12,15,16,23,24,26 We anticipated our usual training group to perform similarly to traditionally trained clinicians in prior studies. We needed a sample size of 8 subjects per group to attain 90% power to detect this 33% difference between groups [SBML 95% (SD=7) vs. usual trained 61% (SD=24)], using an independent two-sided t-test with an alpha of 0.05. Therefore, we were over powered for the primary endpoint in this study.
RESULTS
Sixty patients and caregiver pairs were eligible for the study; 40 pairs completed the study through discharge testing and were included in the final analysis (Figure 3). Table 1 displays sociodemographic and clinical information for those 40 patients and caregivers. The majority of patients were implanted with the HeartWare™ device (n=37); all patients implanted with the HeartMate 3™ devices (n=3) were randomized to the intervention. Patients with HeartMate II™ devices were not enrolled in our study.
Figure 3.
Consort Flow Diagram of Progress Through the Phases of Study.
Table 1.
Demographics of Simulation-based Mastery Learning (SBML) trained and Usual Trained Patients and Caregivers Who Participated in the Randomized-Controlled Trial.
| Characteristic | Patients n=40 | Caregivers n=40 | ||||
|---|---|---|---|---|---|---|
| SBML trained n=20 | Usual trained n=20 | Standardized difference | SBML trainedn=20 | Usual trained n=20 | Standardized difference | |
| Age, years, mean (SD) | 54.3 (12.4) | 56.0 (13.3) | .132 | 48.7 (14.9) | 56.0 (14.2) | .504 |
| Sex, no. (%) | .112 | .120 | ||||
| Female | 6 (30) | 5 (25) | 15 (75) | 16 (80) | ||
| Male | 14 (70) | 15 (75) | 5 (25) | 4 (20) | ||
| Race, no. (%) | .074 | .036 | ||||
| African American/Black | 6 (30) | 6 (30) | 6 (30) | 7 (35) | ||
| American Indian/Alaskan Native | 1 (5) | 0 | 1 (5) | 0 | ||
| Asian | 1 (5) | 1 (5) | 1 (5) | 1 (5) | ||
| Caucasian/White | 12 (60) | 13 (65) | 12 (60) | 12 (60) | ||
| Ethnicity, no. (%) | .471 | .324 | ||||
| Hispanic or Latino/a | 2 (10) | 0 | 1 (5) | 0 | ||
| Non-Hispanic or Latino/a | 18 (90) | 20 (100) | 19 (95) | 20 (100) | ||
| Marital status, no. (%) | .261 | .065 | ||||
| Married/Partner | 15 (75) | 13 (65) | 16 (80) | 14 (70) | ||
| Separated/Divorced | 3 (15) | 4 (20) | 0 | 3 (15) | ||
| Single | 2 (10) | 2 (10) | 4 (20) | 3 (15) | ||
| Widowed | 0 | 1 (5) | 0 | 0 | ||
| Relationship to patient, no. (%) | .078 | |||||
| Spouse/Partner | 10 (50) | 12 (60) | ||||
| Son or daughter | 4 (20) | 2 (10) | ||||
| Parent | 1 (5) | 2 (10) | ||||
| Sibling | 5 (25) | 3 (15) | ||||
| Other | 0 | 1 (5) | ||||
| Education level, no. (%) | .202 | .147 | ||||
| < High school | 0 | 1 (5) | 0 | 0 | ||
| High school graduate | 5 (25) | 6 (30) | 4 (20) | 7 (35) | ||
| Technical school, some college, or associate degree | 8 (40) | 7 (35) | 7 (35) | 6 (30) | ||
| Bachelor’s degree | 5 (25) | 4 (20) | 7 (35) | 3 (15) | ||
| Graduate/Professional degree | 2 (10) | 2 (10) | 2 (10) | 4 (20) | ||
| Work, no. (%) | .554 | .307 | ||||
| Not currently working | 16 (80) | 11 (55) | 13 (65) | 10 (50) | ||
| Home life, no. (%) | 0 | |||||
| Living alone | 3 (15) | 3 (15) | ||||
| Insurance, no. (%)* | .612 | |||||
| Private | 15 (75) | 9 (45) | ||||
| Medicaid | 5 (25) | 6 (30) | ||||
| Medicare | 6 (30) | 10 (50) | ||||
| Other (Cobra) | 1 (5) | 0 | ||||
| VAD Device, no. (%)* | .594 | |||||
| HeartMate 3™ | 3 (15) | 0 | ||||
| HeartWare™ | 17 (85) | 20 (100) | ||||
| Implant strategy, no. (%) | .447 | |||||
| BTT | 8 (40) | 4 (20) | ||||
| DT | 12 (60) | 16 (80) | ||||
| Ventricle(s) supported, no. (%) | 0 | |||||
| BiVAD | 1 (5) | 1 (5) | ||||
| LVAD | 19 (95) | 19 (95) | ||||
| INTERMACS Profile, no. (%) | 0 | |||||
| 1 | 5 (25) | 3 (15) | ||||
| 2 | 10 (50) | 13 (65) | ||||
| 3 | 4 (20) | 4 (20) | ||||
| 4 | 1 (5) | 0 | ||||
| Etiology of heart failure, no. (%) | .085 | |||||
| Dilated-included non-ischemic | 13 (65) | 12 (60) | ||||
| Ischemic | 6 (30) | 7 (35) | ||||
| Other | 1 (5) | 1 (5) | ||||
| NYHA Class, no. (%) | 0 | |||||
| Class IV | 20 (100) | 20 (100) | ||||
| Surgical approach, no. (%) | .211 | |||||
| Sternotomy | 12 (60) | 14 (70) | ||||
| Thoracotomy | 8 (40) | 6 (30) | ||||
| GFR, no (%) | .275 | |||||
| <15 | 0 | 1 (5) | ||||
| 15–29 | 1 (5) | 2 (10) | ||||
| 30–59 | 8 (40) | 7 (35) | ||||
| ≥60 | 11 (55) | 10 (50) | ||||
| Diabetes, no. (%) | 9 (45) | 11 (55) | .201 | |||
| Lung disease (COPD, emphysema, chronic bronchitis), no. (%) | 3 (15) | 5 (25) | .252 | |||
VAD= ventricular assist device
BTT= bridge to transplant
DT= destination therapy
LVAD= left ventricular assist device
BiVAD= Biventricular assist device
INTERMACS= Interagency Registry for Mechanically Assisted Circulatory Support
NYHA= New York Heart Association
GFR= Glomerular filtration rate
COPD= Chronic obstructive pulmonary disease
SBML trained patients and caregivers significantly improved their performance on all 3 skills from pre- to posttest (all p<.001). Three participants (8%; 1 patient and 2 caregivers) were unable to meet or exceed the MPS for the controller change skill at first attempt. All 3 required less than 1 hour of additional deliberate practice to meet the MPS within one retesting session. As shown in Table 2, all patients and caregivers randomized to the SBML intervention eventually met the MPSs for each skill at posttest.
Table 2.
Median (Interquartile Range) Checklist Scores (% Correct) for Simulation-based Mastery Learning (SBML)-trained Patients (n=20) and Caregivers (n=20) before and after the SBML Intervention on Three VAD Self-care Skills.
| Skill | Patients n=20 | Caregivers n=20 | Total n=40 | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Pretest | Posttest | p-value | Pretest | Posttest | p-value | Pretest | Posttest | p-value | |
| Controller change | 21.4 (7.1, 51.8) | 100 (100, 100) | <.001 | 28.6 (17.9, 71.4) | 100 (100, 100) | <.001 | 22.3 (9.8, 67.9) | 100 (100, 100) | <.001 |
| Power source change | 33.3 (0, 66.7) | 100 (100, 100) | <.001 | 88.89 (36.1, 88.9) | 100 (100, 100) | <.001 | 55.6 (3.9, 88.9) | 100 (100, 100) | <.001 |
| Dressing change* | 20.31 (9.4, 38.3) | 100 (100, 100) | <.001 | ||||||
Caregiver only
More participants in the SBML group met or exceeded the MPS on the written examination than in the usual training group [40 (100%) vs. 36 (90%), p=0.04], although median differences between scores were not significant. Median discharge test scores are reported in Table 3. Overall, the SBML group significantly outperformed the usual training group in all 3 skills tests. Significantly more participants in the SBML group compared to the usual training group met the MPS for controller, power source, and dressing changes [Table 4]. These significant differences persisted when evaluating patients and caregivers separately, except in one instance where caregivers in the SBML and usual training groups did not differ on controller change assessments. Overall, 52/80 (65%) of power source and 22/40 (55%) of dressing change assessments in both the SBML trained and usual care groups were performed on the actual patient. The others were performed on the simulator. The interrater reliability on the pre/post and discharge tests was extremely high (mean Kn=0.98 pre/post tests and Kn=0.94 discharge tests).
Table 3.
Median (Interquartile Range) of Patients’ and Caregivers’ Discharge Tests for Simulation-based Mastery Learning (SBML) Trained and Usual Trained groups on the Knowledge-based Written Examination and Three VAD Self-care Skills (% Correct).
| VAD skill | Patients n=40 | Caregivers n=40 | Total n=80 | ||||||
|---|---|---|---|---|---|---|---|---|---|
| SBML trained n=20 | Usual trained n=20 | p-value | SBML trained n=20 | Usual trained n=20 | p-value | SBML trained n=40 | Usual trained n=40 | p-value | |
| Written examination | 100 (97.1, 100) | 100 (97.1, 100) | .39 | 100 (97.1, 100) | 97.1 (97.1, 100) | .26 | 100 (97.1, 100) | 98.6 (97.1, 100) | 0.16 |
| Controller change | 100 (100, 100) | 92.9 (78.6, 100) | .005 | 100 (100, 100) | 100 (100, 100) | .80 | 100 (100, 100) | 100 (87.5, 100) | 0.003 |
| Power Source change | 100 (100, 100) | 88.9 (88.9, 88.9) | <.001 | 100 (100, 100) | 88.89 (88.9, 97.2) | <.001 | 100 (100, 100) | 88.9 (88.9, 88.9) | <0.001 |
| Dressing change* | 100 (100, 100) | 79.7 (64.1, 84.4) | <.001 | ||||||
Caregiver only
Table 4.
Number (Percent) of Simulation-based Mastery Learning (SBML) Trained and Usual Trained Patients and Caregivers Meeting the Minimum Passing Standard (MPS) on the Knowledge-based Written Examination and Three VAD Self-care Skills at Discharge Testing.
| VAD skill | Patients n=40 | Caregivers n=40 | Total n=80 | ||||||
|---|---|---|---|---|---|---|---|---|---|
| SBML trained n=20 | Usual trained n=20 | p-value | SBML trained n=20 | Usual trained n=20 | p-value | SBML trained n=40 | Usual trained n=40 | p-value | |
| Written examination | 20 (100) | 19 (95) | .31 | 20 (100) | 17 (85) | .07 | 40 (100) | 36 (90) | .04 |
| Controller change | 18 (90) | 7 (35) | <.001 | 19 (95) | 18 (90) | .55 | 37 (93) | 25 (63) | .001 |
| Power Source change | 17 (85) | 4 (20) | <.001 | 19 (95) | 5 (20) | <.001 | 36 (90) | 9 (23) | <.001 |
| Dressing change* | 19 (95) | 0 | <.001 | ||||||
Caregiver only
Patient and caregiver self-confidence on performing VAD related self-care tasks was very high in both groups (Table 5), with no significant between group differences. Total training time ranged from 2.1 to 14.9 hours per participant in the SBML group [mean=5.6 hours (SD 2.34)] and 2.0 to 12.0 hours per participant in the usual training group [mean=6.2 hours (SD 2.44), p>.05]. There were no significant associations between sociodemographic and clinical characteristics, self-confidence, and training time with discharge test skills performance.
Table 5.
Median (Interquartile Range) of Simulation-based Mastery Learning (SBML) Trained and Usual Trained Patient and Caregiver Reported Self-Confidence (0=very low confidence to 100=very high confidence) before Discharge Testing on Five VAD Skills.
| Item | Patients n=40 | Caregivers n=40 | ||||
|---|---|---|---|---|---|---|
| SBML trained (n=20) | Usual trained (n=20) | p-value | SBML trained (n=20) | Usual trained (n=20) | p-value | |
| VAD Driveline exit site sterile dressing changes | 50 (0, 90) | 40 (10, 80) | .73 | 100 (100, 100) | 90 (80, 100) | .10 |
| Changing the VAD controller during emergencies | 100 (80, 100) | 90 (80, 100) | .62 | 100 (100, 100) | 100 (90, 100) | .60 |
| Changing power sources on the VAD | 100 (90, 100) | 100 (100, 100) | .45 | 100 (100, 100) | 100 (92.5, 100) | .76 |
| Troubleshooting emergency VAD-related malfunction | 80 (90, 100) | 90 (80, 100) | .95 | 100 (90, 100) | 90 (70, 100) | .14 |
| Recognizing VAD-specific signs and symptoms requiring immediate contact with the VAD team | 100 (100, 100) | 100 (82.5, 100) | .17 | 100 (90, 100) | 90 (72.5, 100) | .17 |
DISCUSSION
We report the first study using SMBL to train patients and caregivers. Our study demonstrates that SBML results in superior learning outcomes compared to usual training for patients and caregivers performing VAD self-care. No patients or caregivers in the usual training group were able to meet the MPS on all of the skills assessments at discharge as no participants passed the dressing change skills assessment. This is especially important because many of the discharge tests (power source and dressing changes) were performed on the actual patient not the simulator. The small differences in written examination scores were likely due to the fact that the examinations were taken with an open book/source policy, and participants could verify their responses prior to submission.
Mastery learning is a strict form of competency-based education requiring high levels of skill acquisition. The MPSs used in our study were set by VAD patients, caregivers, VAD coordinators, and physicians using the Mastery Angoff standard setting method. This method asks experts to define the MPS as the level of skill at which a well prepared VAD patient and their caregiver would perform each skill independently and safely.22 This rigor has important downstream patient care implications because improved VAD self-care may reduce the incidence of adverse events including driveline site infections from improper dressing changes, and death from improper controller5 or power source changes.
A growing body of evidence documents that SBML produces superior learning outcomes to more traditional simulation educational methods.17 An argument against the use of SBML is that it takes more time for participants and instructors.17 However, there is no evidence that such a time differential exists. In fact, our study found no significant difference in training time between groups, yet educational outcomes were superior in the SBML group. We believe the SBML group outperformed the usual training group because SBML uses very effective “adult learning methods” including a flipped classroom (e.g., participant time to watch videos on their own with opportunity to follow up with VAD coordinators if questions arise),27 deliberate practice with expert feedback on a simulator, and multiple rigorous assessment sessions (wherein learners were required to complete the entire skill to mastery). In addition, the standardization of the SBML curriculum may have allowed VAD coordinators to become more efficient and focused because they knew exactly what the protocol required at each stage of training.
Usual care at our institution (and likely many others) incorporates informal measures of skills proficiency for VAD patients and their caregivers. Patients and caregivers need to report a high level of self-confidence before training is complete, and VAD coordinators use their personal judgment and a global rating to determine VAD patient and caregiver readiness for independent self-care. Despite a high level of self-confidence in VAD skills by participants in the usual care group, many were unable to safely perform self-care tasks when tested rigorously. This is notable because all participants in the usual care group had been assessed by experienced VAD coordinators as ready for discharge testing. These findings support previous research findings that self-efficacy and informal assessment do not predict performance,8,10,16,23–25,28 and confirm that these measures should not be used as a proxy for skill. Instead rigorous assessment (such as SBML) which demonstrates a high level of proficiency of patient and caregiver self-care tasks is preferred to ensure safe discharge of patients after VAD implantation.
This study has several limitations. First, it was performed at one institution with a small number of patients and caregivers. It is not possible to compare these results to training practices at all institutions where VADs are implanted. Second, 33% of participants eligible for the study were unable to complete discharge testing. This is consistent with the significant morbidity and mortality in the VAD patient population.1 Third, we are unable to make any conclusions about SBML training for HeartMate™ devices. We were only able to enroll 3 patients with the HeartMate 3™ device because the device was not FDA-approved for bridge to transplant until August 2017 and destination therapy until October 2018. Nonetheless, we expect that performance on the dressing change skill would be the same with HeartMate™ devices because the procedure is identical for HeartWare™ devices. Fourth, participants in the VAD SBML curriculum took 2 tests (pretest and posttest) before discharge testing which may raise a concern that they were “trained to the test.” However, we do not believe this to be the case because checklist items were not shared with participants and posttest results were actually higher than discharge test results. Discharge testing was necessary to compare groups in a more similar context during the potential heightened anxiety before discharge and the expectation that participants could perform independent care while their skills were primarily assessed on the patient (not the simulator). Fifth, we did not evaluate participants’ numeracy or literacy at the time of enrollment in our study. Therefore, it is possible that there were differences between the SBML and usual care groups. However, all written materials used in the study were written at an 8th grade reading level,22 and only one patient in the usual care group (no caregivers) had less than a high school education. Additionally, we believe most of the required VAD skills can be learned through visual demonstration and deliberate practice, which is independent of numeracy and literacy. Sixth, we were not able to determine whether the SBML intervention was successful because of more practice time, more assessment time, or the content of the curriculum. We believe SBML is a package consisting of several tightly linked components including didactics, deliberate practice, individualized feedback, and rigorous assessment protocols. Finally, this study was not powered to demonstrate change in clinical outcomes such as driveline infections or death. However, SBML has been shown to improve clinical outcomes for other procedural skills including central venous catheter insertion,13 thoracentesis,26 paracentesis9 and laparoscopic surgery.29,30 We plan to perform a multicenter study with a larger sample size to evaluate clinical outcomes after VAD SBML self-care training. This trial will allow us to also study dissemination into practice at multiple clinical sites.
CONCLUSIONS
A rigorous SBML curriculum significantly improved VAD self-care skills compared to usual training. This finding is important because VAD self-care is high-stakes and errors can result in significant morbidity and mortality. The SBML curriculum used in our study provides reliable evidence that patients and caregivers are ready for independent VAD self-care. Therefore, our institution now requires all VAD patients and their caregivers to complete VAD SBML self-care training. Further research is needed to demonstrate successful transfer of our SBML curriculum to other institutions and to evaluate whether the curriculum improves downstream patient outcomes after VAD self-care training.
What is Known:
There are no recognized standards for teaching patients and caregivers ventricular assist device (VAD) self-care skills and knowledge
There is no accepted approach to assessing whether patients and caregivers acquire and maintain VAD self-care skills and knowledge
Simulation-based mastery learning improves skills and patient outcomes in a range of clinical domains.
What the Study Adds:
Simulation-based mastery learning results in superior VAD self-care skills and knowledge for patient and caregivers compared to usual training.
ACKNOWLEDGEMENTS
We would like to acknowledge Drs. Douglas Vaughan, Kevin O’Leary, Duc Pham, and Clyde Yancy for their support and encouragement of this work. We thank the patients, caregivers, VAD coordinators, and physicians at Northwestern Memorial Hospital for their dedication to education and patient safety.
SOURCES OF FUNDING
This study was supported by the National Institutes of Health, National Institute of Nursing Research [grant number 1R21NR016745-01].
Footnotes
DISCLOSURES
None.
REFERENCES
- 1.Kormos RL, Cowger J, Pagani FD, Teuteberg JJ, Goldstein DJ, Jacobs JP, Higgins RS, Stevenson LW, Stehlik J, Atluri P, Grady KL and Kirklin JK. The Society of Thoracic Surgeons Intermacs database annual report: Evolving indications, outcomes, and scientific partnerships. J Heart Lung Transplant 2019;38:114–126. [DOI] [PubMed] [Google Scholar]
- 2.Mehra MR, Goldstein DJ, Uriel N, Cleveland JC Jr., Yuzefpolskaya M, Salerno C, Walsh MN, Milano CA, Patel CB, Ewald GA, Itoh A, Dean D, Krishnamoorthy A, Cotts WG, Tatooles AJ, Jorde UP, Bruckner BA, Estep JD, Jeevanandam V, Sayer G, Horstmanshof D, Long JW, Gulati S, Skipper ER, O’Connell JB, Heatley G, Sood P, Naka Y, Momentum Investigators. Two-year outcomes with a magnetically levitated cardiac pump in heart failure. N Engl J Med 2018;378:1386–1395. [DOI] [PubMed] [Google Scholar]
- 3.Grady KL, Sherri W, Naftel DC, Myers S, Gelijins A, Moskowitz A, Pagani FD, Young JB, Spertus JA, Kirklin JK. Age and gender differences and factors related to change in health-related quality of life from before to 6 months after left ventricular assist device implantation: Findings from Interagency Registry for Mechanically Assisted Circulatory Support. J Heart Lung Transplant 2016;35:777–788. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Kato N, Jaarsma T, Ben Gal T. Learning self-care after left ventricular assist device implantation. Curr Heart Fail Rep 2014;11:290–298. [DOI] [PubMed] [Google Scholar]
- 5.U.S. Food and Drug Administration. Medical Device Recalls 2017; https://www.fda.gov/MedicalDevices/Safety/ListofRecalls/ucm560200.htm. Accessed March 7, 2019.
- 6.Yarboro LT, Bergin JD, Kennedy JL, Ballew CC, Benton EM, Ailawadi G and Kern JA. Technique for minimizing and treating driveline infections. Ann Cardiothorac Surg 2014;3:557–562. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Barsuk JH, Cohen ER, Vozenilek JA, O’Connor LM, McGaghie WC, Wayne DB. Simulation-based education with mastery learning improves paracentesis skills. J Grad Med Educ 2012;4:23–27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Barsuk JH, Ahya SN, Cohen ER, McGaghie WC, Wayne DB. Mastery learning of temporary hemodialysis catheter insertion by nephrology fellows using simulation technology and deliberate practice. Am J Kidney Dis 2009;54:70–76. [DOI] [PubMed] [Google Scholar]
- 9.Barsuk JH, Cohen ER, Feinglass J, McGaghie WC, Wayne DB. Clinical outcomes after bedside and interventional radiology paracentesis procedures. Am J Med 2013;126:349–356. [DOI] [PubMed] [Google Scholar]
- 10.Barsuk JH, Cohen ER, Mikolajczak A, Seburn S, Slade M, Wayne DB. Simulation-Based Mastery Learning Improves Central Line Maintenance Skills of ICU Nurses. J Nurs Adm 2015;45:511–517. [DOI] [PubMed] [Google Scholar]
- 11.Barsuk JH, Cohen ER, Potts S, Demo H, Gupta S, Feinglass J, McGaghie WC, Wayne DB. Dissemination of a simulation-based mastery learning intervention reduces central line-associated bloodstream infections. BMJ Qual Saf 2014;23:749–756. [DOI] [PubMed] [Google Scholar]
- 12.Barsuk JH, McGaghie WC, Cohen ER, Balachandran JS, Wayne DB. Use of simulation-based mastery learning to improve the quality of central venous catheter placement in a medical intensive care unit. J Hosp Med 2009;4:397–403. [DOI] [PubMed] [Google Scholar]
- 13.Barsuk JH, Cohen ER, Feinglass J, McGaghie WC, Wayne DB. Use of simulation-based education to reduce catheter-related bloodstream infections. Arch Intern Med 2009;169:1420–1423. [DOI] [PubMed] [Google Scholar]
- 14.Didwania A, McGaghie WC, Cohen ER, Butter J, Barsuk JH, Wade LD, Chester R, Wayne DB. Progress toward improving the quality of cardiac arrest medical team responses at an academic teaching hospital. J Grad Med Educ 2011;3:211–216. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Butter J, McGaghie WC, Cohen ER, Kaye M, Wayne DB. Simulation-based mastery learning improves cardiac auscultation skills in medical students. J Gen Intern Med 2010;25:780–785. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Barsuk JH, Cohen ER, Nguyen D, Mitra D, O’Hara K, Okuda Y, Feinglass J, Cameron CA, McGaghie WC, Wayne DB. Attending physician adherence to a 29 component central venous catheter bundle checklist during simulated procedures. Crit Care Med 2016;44(10):1871–1881. [DOI] [PubMed] [Google Scholar]
- 17.Cook DA, Brydges R, Zendejas B, Hamstra SJ, Hatala R. Mastery learning for health professionals using technology-enhanced simulation: a systematic review and meta-analysis. Acad Med 2013;88:1178–1186. [DOI] [PubMed] [Google Scholar]
- 18.Tofil NM, Rutledge C, Zinkan JL, Youngblood AQ, Stone J, Peterson DT, Slayton D, Makris C, Magruder T, White ML. Ventilator caregiver education through the use of high-fidelity pediatric simulators: a pilot study. Clin Pediatr (Phila) 2013;52:1038–43. [DOI] [PubMed] [Google Scholar]
- 19.Arnold J, Diaz MC. Simulation training for primary caregivers in the neonatal intensive care unit. Semin Perinatol 2016;40:466–472. [DOI] [PubMed] [Google Scholar]
- 20.Heiser Rosenberg CE, Terhaar MF, Ascenzi JA, Walbert A, Kokoszka KM, Perretta JS, Miller MR. Becoming parent and nurse: High-fidelity simulation in teaching ambulatory central line infection prevention to parents of children with cancer. Jt Comm J Qual Patient Saf 2017;43:251–258. [DOI] [PubMed] [Google Scholar]
- 21.Prickett K, Deshpande A, Paschal H, Simon D, Hebbar KB. Simulation-based education to improve emergency management skills in caregivers of tracheostomy patients. Int J Pediatr Otorhinolaryngol 2019;120:157–161. [DOI] [PubMed] [Google Scholar]
- 22.Barsuk JH, Harap RS, Cohen ER, Cameron KA, Grady KL, Wilcox JE, Shanklin KL, Wayne DB. The effect of judge selection on standard setting using the Mastery Angoff method during development of a ventricular assist device self-care curriculum. Clin Simul Nurs 2019;27:39–47.e4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Barsuk JH, Cohen ER, Caprio T, McGaghie WC, Simuni T, Wayne DB. Simulation-based education with mastery learning improves residents’ lumbar puncture skills. Neurology 2012;79:132–137. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Wayne DB, Barsuk JH, O’Leary KJ, Fudala MJ, McGaghie WC. Mastery learning of thoracentesis skills by internal medicine residents using simulation technology and deliberate practice. J Hosp Med 2008;3:48–54. [DOI] [PubMed] [Google Scholar]
- 25.Wayne DB, Butter J, Siddall VJ, Fudala MJ, Wade LD, Feinglass J, McGaghie WC. Graduating internal medicine residents’ self-assessment and performance of advanced cardiac life support skills. Med Teach 2006;28:365–369. [DOI] [PubMed] [Google Scholar]
- 26.Barsuk JH, Cohen ER, Williams MV, Scher J, Jones SF, Feinglass J, McGaghie WC, O’Hara K, Wayne DB. Simulation-based mastery learning for thoracentesis skills improves patient outcomes: A randomized trial. Acad Med 2018;93:729–735. [DOI] [PubMed] [Google Scholar]
- 27.Chen KS, Monrouxe L, Lu YH, Jenq CC, Chang YJ, Chang YC, Chai PY. Academic outcomes of flipped classroom learning: a meta-analysis. Med Educ 2018;52:910–924. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Ericsson KA, Prietula MJ, Cokely ET. The making of an expert. Harv Bus Rev 2007;85:114–121,193. [PubMed] [Google Scholar]
- 29.Schwab B, Teitelbaum EN, Barsuk JH, Soper NJ, Hungness ES. Single-stage laparoscopic management of choledocholithiasis: An analysis after implementation of a mastery learning resident curriculum. Surgery 2018;163:503–508. [DOI] [PubMed] [Google Scholar]
- 30.Zendejas B, Cook DA, Bingener J, Huebner M, Dunn WF, Sarr MG, Farley DR. Simulation-based mastery learning improves patient outcomes in laparoscopic inguinal hernia repair: a randomized controlled trial. Ann Surg 2011;254:502–509. [DOI] [PubMed] [Google Scholar]