Simulation versus live tissue training randomised trial for ECMO proficiency: is one better than the other?

Abstract

Introduction

Extracorporeal membrane oxygenation (ECMO) is a classic low-volume high-risk procedure that requires just in time and/or refresher training through animal or simulation modalities. This manuscript evaluated the performance of ECMO personnel trained with both modalities to determine which is better suited for ECMO skills training.

Methods

Participants (physicians, nurses and respiratory/medical technicians) completed a series of ECMO scenarios with synthetic tissue cannulation task trainer as well as a live tissue model. Objective performance quality was based on task completion using a validated ECMO skills assessment tool.

Results

Thirty-eight individuals completed this study. Participants completed individual scenario tasks 3 min faster using the simulator (26 min vs 29 min; p=0.03). No differences were seen in percentage of individual tasks completed. In the group scenarios, participants completed a higher percentage of critical tasks using the simulator (97%) versus the animal model (91%; p=0.05), but no differences were seen in task completion times. Additionally, no differences were seen in either lab-based or participants’ prelab cognitive scores.

Conclusions

Regardless of their self-assessment or experience, participants’ objective performances were similar among both animal and simulation labs. Task completion times were quicker with simulation model. The distinction between simulation versus animal model may be less important as both demonstrate benefit in development of and/or maintaining skill competency. In the era of questioning the need for and costs of live tissue training, expanding the role of simulation may achieve similar training goals.

INTRODUCTION

Extracorporeal membrane oxygenation (ECMO) supports individuals with life-threatening heart and/or lung failure caused by birth defects, infection, trauma, heart surgery and other severe illnesses. ECMO is a classic low-volume high-risk procedure, where complications can have detrimental effects on survival rates. ¹ Low-volume, high-risk skills require just in time and/or refresher training either through animal or through simulation modalities. Animal training laboratories are thought to best replicate the physiological impact of ECMO initiation, management, crisis prevention/resolution and ECMO patient transport. ² Therefore, animal training has historically augmented ECMO didactic education to facilitate the acquisition and sustainment of the broad repertoire of skills. ³ Since 2006, ECMO simulation models have been used with increasing sophistication and promise. ^{4 5} Simulation provides a realistic yet controlled setting tailored to the trainee’s experience level and increases their comfort level rather than reliance solely upon traditional lecture and patient care alone. ^{6 7} Improved knowledge of the efficacy of both training modalities could help all ECMO training programmes optimise their education systems. However, no study to date has directly compared animal versus simulation for individual and team ECMO emergency skills proficiency.

This manuscript aims to evaluate the performance assessments of physicians, perfusion-trained nurses and technicians trained on animal and simulation modalities to determine which modality is better suited for ECMO skills training.

METHODS

This study was jointly conducted at Wilford Hall Ambulatory Surgical Center (WHASC) 59th Clinical Research Division vivarium and Brooke Army Medical Center (BAMC) simulation centre located in an intensive care unit (ICU) room environment. The animal study training protocol was approved by WHASC’s Institutional Animal Care and Use Committee. Investigators complied with polices as prescribed in the United States Department of Agriculture (USDA) Animal Welfare Act and the National Research Council’s ‘Guide for the Care and Use of Laboratory Animals’. Facilities were fully accredited by the Association for Assessment and Accreditation of Laboratory Animal Care. BAMC’s simulation centre conducted simulation model training under an exempt human subjects research protocol.

Participants

Participants representing four distinct disciplines were recruited voluntarily from the US Army Institute of Surgical Research, BAMC and Children’s Hospital of San Antonio neonatal, pediatric and adult ECMO programmes from March 2016 to August 2017. To ensure appropriate stratification by skillset and experience, the following criteria were established for the four disciplines:

1. Attending physician: attended an ECMO orientation course in the last 5 years, cared for an ECMO patient within the last year or 5 ECMO patients within the last 5 years and actively assisted in the cannulation of an ECMO patient;

2. ICU fellow/resident: attended an ECMO orientation course in the last 3 years and involved in the care of at least one ECMO patient;

3. ECMO specialist (perfusionist/perfusion-trained nurse): attended an ECMO orientation course in the last 3 years and a minimum of 2 years of ECMO specialist experience in management of the ECMO circuit;

4. Technicians (ICU nurse, respiratory therapist, military medic/technician): attended an ECMO orientation course and minimum of 1 year of ICU experience.

Animal lab

Yorkshire/Landrace cross (Sus scrofa) piglets (10–20 kg) were sedated, anesthetised and mechanically ventilated to maintain appropriate tidal volume and end-tidal CO₂. ECMO cannulae were placed for veno-venous (VV) and veno-arterial (VA) delivery with 8–14Fr arterial ECMO cannula in the right common carotid artery and 10–16Fr double lumen venous ECMO cannula in the right external jugular vein per standard protocol. ECMO flow was achieved via centrifugal pump (Jostra-Rotaflow HL20, Maquet, Rastatt, Germany) through a hollow fibre membrane oxygenator (Quadrox, Josta, Maquet, Rastatt, Germany). All piglets received an initial 100 units/kg intravenous heparin bolus before cannulation, followed by a continuous heparin infusion while on ECMO, to maintain an activating clotting time between 160 and 220 s. Epinephrine boluses, as well as crystalloid or whole blood infusions, were used to manage cardiac instability during cannulation or while on ECMO. A training protocol investigator supervised the animal labs for the entire study period, and animals were euthanised at the end of the training lab.

Simulation lab

The ECMO simulator system consisted of a high-fidelity synthetic tissue cannulation task trainer (SynDaver Labs, Tampa, Florida), connected to a water-tight closed reservoir pump, embedded into a low-fidelity newborn infant manikin (Laerdal Medical Corporation, Wappingers Falls, New York, USA) system. This prototype was optimised to advance the realism of the simulation model for this study and consisted of life-like skin and subcutaneous fat overlaying the sternocleidomastoid muscle which was retracted to reveal a medially placed carotid sheath containing the internal jugular vein, the vagus nerve and the common carotid artery. The internal jugular vein (13Fr vessel calibre) and common carotid artery (8Fr vessel calibre) were isolated, incised and cannulated with ECMO cannulae. The major vessels are ‘dead-ended’ peripherally into a water-tight closed reservoir system. Once the cannulae were in place, they were connected to a similar ECMO circuit described above except primed with simulated blood product solution. Simulation operators controlled pressure transducers (Fogg System Company, Aurora, Colorado, USA) and adjusted patient display software (Laerdal Medical Corporation) to display ‘real-time’ patient vital sign and circuit pressure measurements.

ECMO training scenarios

Participants completed six scenarios during each lab. Study co-ordinators scheduled each lab with a multidisciplinary team composition of four participants, that is, from each discipline. ‘Patient’ describes either a pig or manikin unless otherwise specified:

1. Cannulation: participants, functioning in their respective roles in this group scenario, cannulated vessels via cut down of the replaceable skin and vessel tissue pad and pig anterolateral cervical region, respectively. The patient was initially placed on VV ECMO for persistent pulmonary hypertension secondary to meconium aspiration syndrome. Subsequently, the scenario changed to describe inadequate support on VV ECMO, necessitating conversion to VA ECMO. The patient was taken off VV ECMO and placed on emergency ventilator settings, and circuit attachments were reconfigured prior to placing on VA ECMO.

2. Poor venous return: this individual scenario required participants to recognise bleeding at the cannulation site, contributing to poor venous return. Fogg transducers increased the negative venous pressures that shut down the ECMO pump when pressure alarm limits were exceeded. Participants were required to recognise increasingly negative venous pressure and to provide appropriate intervention.

3. Gas failure: this individual scenario required participants to troubleshoot the patient’s oxygen desaturations to identify a disconnected gas source and reconnect it.

4. Pump failure: a simulated power outage and subsequent surge disabled the ECMO pump and required an individual participant to initiate manual pumping of the circuit with the hand crank and place the patient back on ECMO with a new ECMO pump.

5. Circuit rupture: this group scenario required the participants to function within their respective roles to identify circuit rupture, take the patient off ECMO in order to isolate and repair the rupture, and return the patient back on ECMO.

6. Arterial air: this individual scenario required the participant to recognise air on the arterial side of the ECMO, take the patient off ECMO and effectively remove all air from the circuit. Participants were also required to describe how to correct an air embolism in the venous side of the ECMO circuit.

Study design

This study used a block randomised crossover training protocol (AB/BA) comparing an animal training model (A) versus a simulation training model (B). Participants underwent sequential randomisation by the four discipline groups to either Track A (animal lab 1st; simulation lab 2nd) or Track B (simulation lab 1st; animal lab 2nd).

Participants completed a 4-hour prelab didactic session, adapted from BAMC’s core ECMO training curriculum, consisting of multiple video webinars covering ECMO cannulation, emergency procedures, clinical checklists, as well as a demonstration of appropriate cannulation technique and orientation to both training environments. Prior to the lab, participants completed a demographic questionnaire delineating their clinical discipline, duration of professional experience, duration and type of ECMO experience (neonatal, paediatric or adult), ECMO cannulation experience and prior ECMO training (lecture only, simulation lab, animal lab or on-the-job training) as well as a 25 multiple-choice question examination on ECMO fundamentals. Each session lasted about 6 hours. Subsequent labs were spaced a median of 56 (IQR 35–89) days apart to allow for a standardised washout period (figure 1).

Figure 1.

Protocol timeline.

At the start of each lab, participants rewatched the cannulation and orientation videos as a standardised refresher to reduce variation in participant preparation and were additionally provided one opportunity to practice pump failure manoeuvres.

The subjective self-assessment survey and the objective ECMO skills assessment tool were created after a thorough review of the ECMO clinical and simulation literature as well as expert consultation. ^8–10 In order to obtain validity evidence for the training session, participants completed the self-assessment survey before and after each training session to assess self-perception of competency in emergent ECMO skills, evaluation of the training session, problems encountered with the training modality and modality preference. Objective performance was based on a total weighted performance score derived from an ECMO skills assessment tool created for this study by the research team, and content validity was established via review by three ECMO subject matter experts not involved in the study. The tool was constructed in a weighted Go/No Go checklist format for each step, with a preassigned priority value (basic or critical) for weighted scoring. Additionally, these performance skills were tailored for each of the four disciplines. Critical performance measures were also timed for task completion.

All facilitators and instructors completed a simulation education course, received extensive training on both the simulator and the animal model and watched a sample videotaped performance of optimal execution of the six scenarios enacted by an ECMO clinician not involved in the study. Facilitators and instructors were also responsible for prestaging each scenario. To achieve appropriate inter-rater reliability, facilitators scored the sample videotape performance and were required to achieve 80% concordance before evaluating any study participants. To eliminate variability in the degree of assistance provided to each participant, instructors followed a precise script for each scenario (online appendix A), which provided the simulated patient’s clinical history and allowable responses to clinical queries (not evident on patient exam or disabled on the monitoring equipment) and permitted prompting to elicit knowledge check items not spontaneously demonstrated. Instructors debriefed each scenario ranging from 2 to 5 min to provide individually tailored educational feedback encompassing teamwork, clinical as well as technical skills. During the study labs, two facilitators evaluated the participants. Following each scenario, facilitators reviewed their scores to ensure concordance and adjudicate any discrepant scoring. While they were not blinded to the participants’ identities, both facilitators and instructors were blinded to participants’ self-assessment profiles, track assignment (A vs B), and 1st versus 2nd training iteration. Female and male facilitators and instructors were critical care attending physicians and perfusion-trained ICU nurses. Video recordings of the participants were not used.

Statistical analysis

As there were no previous studies directly comparing simulation versus live tissue training modalities, our study team determined that 16 participants per discipline (64 total participants) would be sufficient for this pilot prospective randomised trial given constraints based on laboratory space/personnel availability and funding timelines. In unadjusted analyses by lab type and prelab cognitive scores, study participants were compared using the Wilcoxon rank-sum tests or Fisher’s exact tests. Participant lab type preferences were assessed from the modality preference section of the self-assessment survey. For both individual and team scenarios, repeated measures generalised linear models were used to examine differences in animal versus simulation models by (1) task time (time from start to end of scenario), (2) critical task time, (3) task completion (per standardised definition) and (4) critical task completion, while simultaneously adjusting for prelab cognitive scores and lab track. Both task time and completion included measurements on the total scenario and on critical tasks within each scenario. Critical task/time completion were defined within the six training scenarios for separate analyses. Within cannulation, critical tasks included initial placement on VV ECMO as well as the transition to VA ECMO. The critical task within the poor venous return scenario was initial alarm recognition. For gas failure, the critical task was time to reconnection. Within pump failure, critical tasks included transition to hand crank operation as well as placement onto the new pump. For circuit rupture, the critical task time recorded was time off the ECMO circuit for repair. Initial clamping of neck lines as well as resumption of ECMO were two critical tasks identified within the arterial air scenario. A prelab cognitive score calculated from the ECMO fundamentals examination was used as an objective proxy of participant knowledge and experience. All analyses were completed using SAS version 9.4 (SAS Institute). For all statistical tests, a p<0.05 was considered significant.

RESULTS

Fifty-one individuals participated in this study and 38 completed both animal and simulation labs from March 2016 to August 2017 (table 1) and included in the study analyses. Participants included 6 attending physicians, 11 fellows/residents, 11 specialists and 10 technicians. The majority of participants (61%) had experience caring for adult ECMO patients, compared with 29% having experience managing pediatric/neonatal ECMO patients. Attending physicians and specialists reported more ECMO experience (18% reported ≥6 years and 71% ≥6 ECMO patients) than residents and technicians (86% reported <1 year and 71% ≤2 ECMO patients). Except for specialists, the majority of participants had not participated in lecture-based ECMO certification courses (55%), ECMO simulation (79%) or animal training sessions (66%) prior to this training opportunity. Participants worked in their respective positions a median of 4.3 years, with specialists being outliers (median 20 years). While 10 of 11 specialists reported over 36 hours of ECMO contact time, 22 of the remaining 27 participants reported 10 or less hours of contact time (4 attendings, 9 residents/fellows and 9 technicians).

Table 1.

Study participant demographics

	Total	P value
Medical position (%)		0.6172
Attending	6 (15.8)
Fellow/resident	11 (28.9)
Specialist	11 (28.9)
Technician	10 (26.3)
ECMO experience (%)		<0.0001
<1 years	22 (57.9)
1–2 years	7 (18.4)
3–5 years	6 (15.8)
6–10 years	1 (2.6)
>10 years	2 (5.3)
Number of previous ECMO patients (%)		0.6399
None	11 (28.9)
1–2	7 (18.4)
3–5	5 (13.2)
6–10	7 (18.4)
>10	8 (21.1)
Number of previous ECMO cannulations (%)		<0.0001
None	22 (57.9)
1–2	6 (15.8)
3–5	7 (18.4)
6–10	1 (2.6)
>10	2 (5.3)
Role in ECMO cannulation (%)		<0.0001
Primary provider responsible for the procedure	2 (5.3)
Procedural assistant in the surgical/sterile field	2 (5.3)
Team member present in the patient room	22 (57.9)
Other	12 (31.6)
No. of previous ECMO lecture-based certification courses (%)		0.0006
None	21 (55.3)
1–2	15 (39.5)
>10	2 (5.3)
No. of previous ECMO manikin-based simulation training sessions (%)		<0.0001
None	30 (78.9)
1–2	3 (7.9)
3–5	3 (7.9)
6–10	1 (2.6)
>10	1 (2.6)
No. of previous ECMO animal lab training sessions (%)		<0.0001
None	25 (65.8)
1–2	10 (26.3)
3–5	2 (5.3)
>10	1 (2.6)
Hours of ‘on-the-job training’ caring for an ECMO patient (%)		0.0439
None	16 (42.1)
1–10	6 (15.8)
11–35	5 (13.2)
>36	11 (28.9)
Median years of experience in current medical position (IQR)	4.3 (2,15)	0.0014
Attending	4.8 (3,6)
Fellow/resident	2 (1,3)
Specialist	20 (9,25)
Technician	5.5 (4,20)
Median cognitive assessment score as % (IQR)	84 (72,88)	0.0002
Attending	92 (88,96)
Fellow/resident	88 (80,92)
Specialist	84 (80,88)
Technician	64 (24,68)

ECMO, extracorporeal membrane oxygenation; ICU, intensive care unit.

Comparing the live animal to simulation lab, participants completed individual scenario tasks 3 min faster using the simulator (26 min vs 29 min; p=0.03). No significant differences were seen in percentage of individual tasks completed or ability or speed in completing the subset of critical tasks. In the group scenarios, participants also completed a higher percentage of critical tasks using the simulator (97%) versus the animal model (91%; p=0.05). However, no differences were seen in critical task time or overall task completion and time. Table 2 provides a comprehensive review of participant performance by lab type.

Table 2.

Animal versus simulation training model

		Animal		Simulation
		n	Median (Q1, Q3)	n	Median (Q1, Q3)	P value
Individual scenarios	Task time, min	38	28.5 (25.7,33.4)	38	25.8 (21.0,29.0)	0.03
	Critical task time, min	25	18.0 (12.9,23.7)	19	16.0 (12.6,18.1)	0.30
	Task completion, %	38	59.0 (56.0,65.0)	38	61.0 (56.0,64.0)	0.60
	Critical task completion, %	38	98.0 (94.0,100.0)	38	96.0 (92.0,98.0)	0.42
Group scenarios	Task time, min	37	87.7 (82.2,95.9)	32	85.7 (67.4,107.2)	0.70
	Critical task time, min	33	71.9 (56.0,82.1)	24	57.6 (49.8,82.7)	0.30
	Task completion, %	38	83.0 (75.0,89.0)	32	86.0 (80.0,92.0)	0.20
	Critical task completion, %	38	91.0 (83.0,100.0)	32	97.0 (91.0,100.0)	0.05
All scenarios	Task time, min	37	115.8 (102.2,129.9)	32	118.9 (95.5,131.6)	0.57
	Critical task time, min	23	95.3 (82.0,107.3)	16	73.7 (58.5,97.2)	0.01
	Task completion, %	38	67.0 (62.0,71.0)	32	70.0 (65.0,74.0)	0.10
	Critical task completion, %	38	96.0 (88.0,97.0)	32	96.0 (94.0,99.0)	0.33

Psychomotor task performance in the two modalities was also analysed by prelab cognitive score. Participants’ training outcomes were compared by their prelab cognitive assessment scores as an objective proxy for previous training and experience. No differences were seen in task time, task completion, critical task time or completion in either lab-based on prelab cognitive test performance: <68% (n=10), 68–92% (n=16) and ≥92% (n=12). Additionally, there were no differences based on Track A versus Track B assignment or following the second iteration of training.

When adjusting for covariates, participants completing individual simulation-based scenarios had faster total task times compared with the animal model (p=0.02). However, there were no differences in critical task times or completion scores. During group scenarios, there were no differences in total task times between simulation and animal models, but a trend towards faster critical task completion times (p=0.05) and a higher total critical task completion score (p=0.02) for the simulation model was noted. Also adjusting for covariates, participants with higher cognitive scores did not complete individual tasks more quickly; however, their total task and critical task scores were higher than those with lower cognitive scores (p=0.004 and 0.01, respectively). During group scenarios, there were no differences in any of the outcomes by cognitive score. Finally, track assignment did not demonstrate any outcome differences. Table 3 summarises these results.

Table 3.

Skill performance by lab track

		Total		Track A		Track B		p value
		n	Median (Q1, Q3)	n	Median (Q1, Q3)	n	Median (Q1, Q3)
Individual scenarios	Task time (Lab 1-2), min	38	5.6 (0.5,10.3)	20	8.0 (2.3,12.7)	18	2.0 (−1.3,6.7)	0.19
	Critical task time (Lab 1-2), min	13	6.5 (5.2,10.6)	8	8.4 (5.4,12.2)	5	6.2 (3.1,8.9)	0.90
	Task completion difference (Lab 2-1), %	38	2.0 (−6.0,7.0)	20	3.0 (−3.0,8.0)	18	−1.0 (−6.0,4.0)	0.59
	Critical task completion difference (Lab 2-1), %	38	0.0 (−2.0,4.0)	20	−1.0 (−4.0,2.0)	18	0.0 (−2.0,4.0)	0.65
Group scenarios	Task time (Lab 1-2), min	31	4.3 (−11.3,26.3)	14	5.8 (−8.7,40.0)	17	4.3 (−14.8,22.0)	0.53
	Critical task time (Lab 1-2), min	20	4.6 (−8.3,26.1)	7	27.3 (7.9,39.7)	13	0.8 (−14.2,7.9)	0.88
	Task completion difference (Lab 2-1), %	32	−2.0 (−5.0,8.0)	14	2.0 (−4.0,13.0)	18	−4.0 (−6.0,-2.0)	0.15
	Critical task completion difference (Lab 2-1), %	32	1.0 (−5.0,10.0)	14	9.0 (0.0,25.0)	18	0.0 (−14.0,6.0)	0.47
All scenarios	Task time (Lab 1-2), min	31	10.4 (−8.0,29.5)	14	17.5 (2.7,48.3)	17	8.8 (−12.6,19.2)	0.71
	Critical task time (Lab 1-2), min	8	12.8 (−6.0,41.6)	4	41.6 (20.7,49.5)	4	−6.0 (−10.6,8.6)	0.83
	Task completion difference (Lab 2-1), %	32	0.0 (−5.0,7.0)	14	4.0 (−1.0,8.0)	18	−2.0 (−6.0,4.0)	0.70
	Critical task completion difference (Lab 2-1), %	32	1.0 (−3.0,8.0)	14	2.0 (−1.0,8.0)	18	0.0 (−7.0,7.0)	0.80
	Cognitive score (Lab 2-1), %	38	0.0 (−4.0,8.0)	20	0.0 (−6.0,4.0)	18	2.0 (−4.0,8.0)	0.61

Track A (animal lab 1st): Track B (simulation lab 1st).

There were no significant outcome differences based upon participant prelab self-assessment scores. Participants also indicated preferences towards the animal lab (n=19) or the combination of both (n=16). Only two participants preferred a simulation model. One participant did not provide a response. Technical challenges, such as animal demise or equipment malfunction, impacted two sessions (one animal and one simulation) to the extent that participants reported these obstacles moderately affected their learning experience.

DISCUSSION

We found that although the majority of participants preferred animal or the combination of both animal and high-fidelity simulation training, there was no difference in skills proficiency attained by either technique.

Animal training for adequate hands-on exposure is endorsed by Extracorporeal Life Support Organization for all members of the ECMO team particularly for low-volume programmes in which clinical patient hours are limited. ³ There are no studies comparing simulation and animal modalities for ECMO skills maintenance. In a similar trial randomising medics to simulation or animal training on cricothyroidotomy, there were no differences among success rate or task completion time among participants, with authors concluding no objective benefit for animal training. ¹¹ More broadly, all disciplines in our ECMO programme currently attend animal labs to maintain certification. While the majority of participants in all disciplines preferred the animal training to the simulation training, our results do not definitively favour one modality over the other, suggesting that both modalities may achieve training aims for acquisition and maintenance of ECMO skills for participants of varying experience and skillset.

As previously reported, we noticed a trend for improvement over time with any form of repeated scenario training. Studies by Anderson et al and Allan et al showed that simulator-based training improved provider comfort level more than traditional lecture and patient care alone. ^{6 7} In a follow-up study, Anderson et al assessed time to emergency task completion and rated behavioural markers of performance for nursing ECMO circuit specialists after completion of two consecutive simulation scenarios. Behavioural markers including communication, anticipation, resource use and environmental awareness all improved significantly from baseline. Time to task completion was not significantly improved; however, successful technical performance rates improved after simulation-based debriefing. Therefore, the authors concluded that simulation training creates a learning environment that readily supports the acquisition of technical and behavioural skills important in solving life-threatening problems on ECMO. ⁸ Similarly, Burton et al found that simulation training for ECMO nurses and respiratory therapists trained to manage the ECMO system improved safety, attitudes and teamwork surrounding ECMO emergencies and helped identify sources of error which remain latent safety threats. ⁹ Allan et al found that a contextualised ECMO cannulation during cardiopulmonary resuscitation simulation model resulted in improvement in time to cannulation, surgical technique and a validated composite ECMO cannulation score compared with baseline performance. ⁷ Zackhary et al reported in a randomised clinical trial conducted among novice critical care fellows that simulation training (high fidelity manikin connected to ECMO circuit in a medical ICU setting with ventilator and patient monitors) was superior to traditional ‘water’ drills (ECMO cannula connected to bladder reservoir hidden behind a low-fidelity manikin) in terms of skills maintenance at 6 weeks and 1 year. ¹²

We observed faster completion times and slightly higher critical tasks completion percentages with the simulation model as compared to animal training. However, we speculate that this is in part due to the decreased variability and complexity inherent to artificial models compared with live physiologic responsiveness which requires more intellectual processing on the part of the learner, rather than actual educational superiority of the simulation model. Our study design compared simulation versus animal training, incorporated a greater number of training scenarios and included participants from various disciplines, to support the value of simulation-based approach to training and education on ECMO concepts.

Our study has several strengths. Foremost, this is the first and most comprehensive randomised controlled trial comparing live tissue with simulation platform for ECMO skills maintenance in a multidisciplinary cohort of participants. Second, we developed and incorporated six complex team and individual skill scenarios with learner feedback into our design as part of a comprehensive training curriculum that incorporated key elements for simulation-based research. ¹³ Third, we obtained validity evidence for an ECMO skills assessment following Kane’s validity framework for educational assessments. ^{14 15} Scoring and decision/interpretation inferences were achieved as the tool was developed with expert consultation. Generalisation inference was accomplished by inter-rater reliability testing, and extrapolation inference was assessed by expert–novice comparisons. Fourth, we advanced the realism of a simulation model available to the ECMO community for ongoing education by integrating cutting edge high-fidelity synthetic tissue cannulation task trainer technology to more closely approximate real-world interactions. Additionally, we were able to better delineate the potential cost savings of recurrent high-fidelity ECMO simulator training compared to sustaining animal training labs. The costs associated with running a six-hour long animal lab are approximately US$12 600 to cover animal, personnel, facilities and supply expenses. The same costs associated with running a 6-hour-long simulation lab are approximately $9500 which includes the cost for the synthetic tissue cannulation task trainers that are now commercially available. While the largest budget contributor to expense is the ECMO circuit tubing (US$8400), US$3000 could represent significant savings in addition to reducing reliance on animals and veterinary facilities to support the training.

There were several limitations in this study. Our initial goal had been to recruit 16 participants from each group so that we could additionally compare our findings among different disciplines and experience. Our enrollment limitations resulted in a greater number of introductory-level participants than senior experienced participants. It was not possible to recruit participants who met every required ECMO experience by designated skillset. Therefore, when no other alternatives were available, the research team waived some of these requirements. While the scenario was based on a neonatal clinical case unfamiliar to participants with adult clinical backgrounds, we felt that the underlying ECMO principles and concepts were still achieved. Furthermore, participant evaluations did not report any diminished educational value. Scheduling clinicians for 2 full-day training proved to be difficult, and only 38 of the 51 recruits completed both training labs. Additionally, technical difficulties impacted some training and subsequent data collection. These limitations are reflected in the different ‘n’ values reported in the tables. Last, our study was not designed to assess long-term retention of ECMO skills. However, our study reinforces the value of multimodal educational and simulation techniques to provide skills proficiency improvement through repeated scenario-based exposure for low-volume high-risk procedures.

CONCLUSIONS

Regardless of self-assessment or experience, participants’ objective performances were similar among both animal and simulation labs. Task completion times were quicker with simulation model. The distinction between simulation versus animal model may be less important as both demonstrate benefit in acquisition and/or maintenance of skills. However, in this era of continuously questioning the need for, costs and benefits of live tissue training, expanding the role of simulation may achieve similar training goals at a lower expense especially when there is a demonstrated benefit to an increased frequency of training events.

What is already known on this subject.

• Animal models have been the gold standard for ECMO training due to their ability to replicate complex physiology and anatomic variation.

• As simulation models have become more sophisticated, less costly and less manpower-intensive, they are increasingly incorporated into routine training.

• No study has compared which modality (animal vs simulation) is best for optimal emergency ECMO skills proficiency, safety and teamwork training.

What this study adds.

• This study contributes to the value of a simulation-based approach to ECMO education and training to potentially improve provider skills proficiency.

• This research used advanced high-fidelity synthetic tissue cannulation task trainer technology to provide a more realistic simulation model for ECMO training purposes.

• This study supports that recurrent high-fidelity ECMO simulator training may be more cost-effective than an animal modality because simulator training reduces reliance on animals and veterinary support.