Is a haptic simulation interface more effective than computer mouse-based interface for neonatal intubation skills training?

Abstract

Objective

To compare the efficacy of a three-dimensional (3D) haptic interface to a two-dimensional (2D) mouse interface for a screen-based simulation (SBS) neonatal intubation (NI) training intervention. Primary hypothesis: a haptic interface is more effective than a mouse interface for SBS training intervention for NI. Secondary hypothesis: SBS training, regardless of interface, will result in improved NI performance on a neonatal airway simulator.

Methods

45 participants were randomised to either a haptics or a mouse interface to complete an identical SBS training intervention for NI over a five-month period. Participants completed pre- and post-training surveys to assess demographics, experience, knowledge and attitudes. The primary outcome of participants’ NI skills performance was assessed on a neonatal manikin simulator. Skills were measured pre- and post- training by number of attempts and time to successfully intubate, and airway visualization.

Results

The demographics, training and experience were similar between groups. There was no difference in the improvement in skills, knowledge, attitudes or satisfaction ratings pre- and post-training between the groups. There was a significant decrease in number of attempts to intubate a neonatal airway simulator (2.89 vs 1.96, p<0.05) and improvement in the percent of subjects intubating in <30 seconds (22% vs 27%, p=0.02) from pre- to post-training in the study population overall.

Conclusion

Using a haptic interface did not have an advantage over a mouse interface in improving NI skills, knowledge, attitudes, or satisfaction. Overall, a SBS training intervention for NI improved skills measured on a neonatal airway simulator.

Introduction

Medical training has historically involved didactic and procedural training. Procedural skills are necessary for a variety of medical specialties.^1–3 The traditional training paradigm employed by the majority of medical professionals involves the well-known Halstedian apprenticeship model of developing skills during patient care activities, or ‘see one, do one, teach one.’ ^{1 3 4} Medical education is currently shifting away from this training paradigm in which the stakes are high and patient safety may suffer from inexperienced providers who are developing their skills while practicing on patients.⁵ Recent research has demonstrated that technology-enhanced simulators, broadly divided into manikin-based and computer-based simulators, are effective for procedural skills training in a variety of specialties including orthopaedics, neurosurgery and general surgery.^{6 7} They have been gaining favour for use in procedural training in recent years and in response to this research the use of technology-enhanced simulators in paediatrics has rapidly increased incorporating new technologies and manikins to simulate real-life situations.^{3 8–10} Simulation-based techniques and technologies are components of novel paradigms of training such as Sawyer's ‘Learn, See, Practice, Prove, Do, Maintain’.¹¹ These paradigms shift the steep part of the learning curve, where many errors occur, into the simulation laboratory and protect patients from harm.

Endotracheal (ET) intubation is a critical procedural skill for paediatric providers, and suboptimal performance can lead to significant patient harm. In order to develop and maintain neonatal intubation (NI) skills, providers need to engage in deliberate and repeated practice. However, there is an inherent risk to patients in allowing novice providers perform procedures on critically ill infants. Among paediatric patients, neonates require more frequent ET intubation and can be more technically challenging due to patient size, airway anatomy and baseline haemodynamic instability.¹² Multiple authors have demonstrated that paediatric trainees currently have suboptimal success rates in NI, ranging from 20% to 69% on first attempt.^12–15 This may be due to a variety of circumstances, including limitations on trainee work hours and decreased time spent in intensive care settings, changes in the Neonatal Resuscitation Program (NRP) guidelines for the resuscitation of infants with meconium-stained amniotic fluid, increased presence of mid-level providers, and frequent utilisation of non-invasive forms of ventilation. Unsuccessful repeated intubation attempts may lead to airway trauma including soft tissue bleeding, gum or lip trauma, injury to the vocal cords, or perforation of the trachea or oesophagus.^{16 17} Delays in successfully intubating an infant can result in clinical decompensation, morbidity or mortality. These risks validate the need for realistic airway simulators and training paradigms that allow trainees to develop a baseline competency prior to performing procedures on patients.¹⁸

Haptics is a tactile feedback technology, which, along with virtual reality simulators, can be used as a feature of instructional design for procedural skills training.^{6 10 19} Medical professionals’ NI skill acquisition may be enhanced using a haptic three-dimensional (3D) simulator interface compared with a traditional computer mouse-based 2D simulator interface. The ability to move in a 3D environment may help trainees understand the relationship between the anatomical landmarks, develop psychomotor skills required to intubate and gain confidence in their procedural skills. Ideally, these advances will translate into improved procedural success rates and higher quality patient care. Haptic technologies have been trialled for simulation-based procedural skills training in various surgical specialties including orthopaedics, neurosurgery and minimally invasive surgeries.^{6 20 21} Additionally, haptic technology has recently been developed for procedure-based skills in paediatrics, specifically NI.¹⁰ However, there is insufficient data to support its ease of use or efficacy as a teaching modality, nor has its usefulness been assessed compared with 2D computer mouse-based simulators.^{2 7}

This study compares the efficacy of a 3D haptic simulator interface with a 2D computer mouse-based simulator interface to train students, residents and fellows for NI. Our primary hypothesis is that a haptic interface is more effective than a computer mouse-based interface for NI, and results in improved airway views, faster intubation times, and reduced number of attempts needed to successfully intubate a neonatal airway simulator. Our secondary hypothesis is that screen-based simulation (SBS) training results in improved neonatal airway simulator intubation skills across the measured domains compared with baseline, regardless of whether a haptic or computer mouse-based interface is used.

Methods

A convenience sample of 45 participants was enrolled in the neonatal ET intubation study over a 5-month period from November 2013 to March 2014 (figure 1). They consisted of undergraduate students, medical students, residents (paediatrics, emergency medicine, and family medicine) and pediatric emergency medicine fellows. Verbal consent was obtained from each individual. Participants were randomised into two groups, the intervention group and the control group using a random number generator (Random.org).

Figure 1.

Study flow diagram.

Prior to beginning the training, each participant completed a confidential pretraining survey to obtain information about (1) demographics, (2) experience with NI, (3), knowledge of the NI procedure and (4) attitude towards procedural skills training. Knowledge was assessed using multiple-choice questions about airway anatomy, intubation technique and equipment required for intubation (see online supplementary appendix 1). Attitudes were assessed by asking the participant to rate his or her comfort level, confidence and understanding of NI on a five-point Likert scale. In addition to the survey, participants completed a pretraining neonatal airway simulator intubation to assess baseline procedural skill. This was done using the SimNewB neonatal airway simulator and Storz C-MAC Laryngoscope. SimNewB Neonatal airway simulator is an interactive neonatal simulator designed by Laerdal with the American Academy of Pediatrics to meet the training requirements of the NRP. It effectively mimics the experience of NI through an anatomically correct airway. The Storz C-MAC Endoskope (Karl Storz, Tuttingen, Germany) is a video laryngoscope which captures photographic images or videos of the provider's view of the airway during intubation attempts using a camera attached at the tip of the blade. One of the main advantages of this device, when compared with other available video-laryngoscopes, is that the design is very similar to the Miller 1 blade. This allows trainees to learn and practice the procedure using identical procedural technique to traditional direct laryngoscopy, while providing the supervisor or instructor with a real-time image of the trainee's view of the airway.

All of the necessary equipment (including ET tube, stylet, shoulder roll, C-MAC video-laryngoscope, bag and mask) required to intubate the neonatal airway simulator was provided to the participant. Each participant had a maximum of eight attempts (30 s maximum duration each) to intubate without the guidance of the video monitor. The timer began when the laryngoscope blade was inserted into the mouth. After 30 s, the examiner would end the current intubation attempt and request for the participant to start the procedure again from the beginning. The participants were instructed to prompt the examiner when (1) they thought they had the best possible view of the vocal cords and (2) they felt they had inserted the tube into the trachea. At each of these prompts, the examiner captured the image on the C-MAC monitor.

After the pretraining survey and manikin intubation, each participant was required to complete the SBS NI training programme (figures 2–4) using either a 3D haptic device (figure 4) or a 2D computer mouse, based on randomisation group. The training programme was standardised for each participant with voice-over and instructions. The intervention group was trained in NI with the 3D haptic simulation interface-Clinical tvr. Clinical tvr by MySmartSimulations uses ‘sense of touch’ through the haptic technology along with computer-based didactic procedural training. The software includes a haptic tutorial that orients the user to the technology, followed by a programme that trains the user to intubate a virtual neonate using the haptic device. The control group was trained in NI using a 2D computer mouse-based simulation interface using the same computer-based didactic procedural training programme used for the haptic group. Both of the programmes had the same graphics, voice-over, steps of training and instructions, and only differed by the inclusion of the steps incorporating the haptic device.

Figure 2.

Screen-based simulation training program: anatomy.

Figure 3.

Screen-based simulation training program: equipment.

Figure 4.

Haptic technology.

After completing the training, intubation skills were reassessed using the SimNewB manikin following the same sequence of steps as was used in the pretraining assessment. Finally, the participants completed the post-training survey. The images obtained using C-MAC during intubation were used for Cormack-Lehane (C-L) scoring and percentage of glottic opening (POGO) score.^{17 22} Both of these systems are used to quantify the laryngoscopic view of the glottis and vocal cords. The C-L score assigns Grade I as complete exposure of the glottis, and Grade IV as visualisation of neither the glottis nor epiglottis. The POGO score represents the percentage of the glottis visualised, with 0% being no visualisation and 100% being complete visualisation. Scores were assigned by a single experienced neonatology attending who was blinded to the randomisation group. This attending has served on faculty for over 10 years in an academic medical centre neonatal fellowship training programme.

The intervention and control groups were compared with each other pre training and post training using the following metrics: (1) Number of baseline attempts for successful intubation (maximum attempt duration=30 s); (2) percentage of each group completing intubation attempts in fewer than 30 s; (3) mean POGO score and (4) percentage of subjects with a C-L score of 1. Data was analysed using IBM SPSS Statistics software (V.19.0). Two-tailed paired t-test (for ordinal variables) and χ² test (for nominal variables) were used to compare pretraining and post-training data for the control and intervention groups.

Results

The demographics of the two groups were similar in terms of profession, experience, previous resuscitation training and exposure to intubation, although there was some heterogeneity (table 1). However, there was no significant difference in baseline metrics between the control and intervention groups, including the mean number of baseline intubation attempts (2.29 vs 3.57; p=0.12), subjects who intubated in fewer than 30 s, (50% vs 48%; p=0.87), mean POGO score (49 vs 33; p=0.36) and per cent of subjects with C-L score of 1 (38% vs 29%; p=0.53). There was also no significant difference in baseline knowledge (mean score of 6 out of 9 questions correct for both groups), comfort in intubation (17% vs 14%; p=0.83) or confidence in intubation (13% vs 5%; p=0.63).

Table 1.

Demographics

	Computer mouse (control) (n=24)	Haptics (intervention) (n=21)
Gender
Male	12 (50%)	4 (19%)
Female	12 (50%)	17 (81%)
Profession
Undergraduate	4 (17%)	4 (19%)
Medical student	9 (37.5%)	6 (28%)
Resident, paediatrics	3 (12.5%)	7 (34%)
Resident, emergency	4 (17%)	4 (19%)
Other (fellow, PA, FM resident)	4 (17%)	0 (0%)
Years in profession/training
0	13 (54%)	10 (48%)
1	2 (8%)	6 (28%)
2	3 (12.5%)	3 (14%)
3	2 (8%)	2 (9%)
4	4 (17%)	0 (0%)
Resuscitation training completed
PALS	9 (37.5%)	9 (43%)
NALS	5 (21%)	6 (28%)
Exposure to intubation
None	9 (37.5%)	5 (24%)
Video	0 (0%)	0 (0%)
Game	0 (0%)	1 (5%)
Didactic lecture	9 (37.5%)	9 (43%)
Performance on simulator	14 (58%)	15 (71%)
Performance on patient	10 (42%)	7 (34%)
Other
Average number of times performing intubation on simulator or patient
Simulator (range)	7 (0, 80)	4 (0, 20)
Patient (range)	9 (0, 70)	7 (0, 50)
Exposure to neonatal intubation
None	16 (67%)	12 (57%)
Video	1 (4%)	1 (5%)
Game	0 (0%)	0 (0%)
Didactic lecture	3 (12.5%)	4 (19%)
Performance on simulator	6 (25%)	7 (34%)
Performance on patient	5 (21%)	3 (14%)
Other	0 (0%)	0 (0%)
Mean of number of times neonatal intubation performed on simulator or patient
Simulator	3 (0, 40)	1 (0, 3)
Patient	1 (0, 10)	1 (0, 12)

FM, family medicine; NALS, neonatal advanced life support; PA, physician assistant; PALS, pediatric advanced life support.

There was no significant difference in improvement in knowledge, attitudes or satisfaction between the two groups from pre training to post training (table 2). Additionally, there was no significant difference in the change in mean number of attempts, the number of subjects improving to achieve the NRP recommended 30 s maximum attempt duration, the change in time to intubate, the change in POGO score or the number of subjects whose C-L score changed from ‘fail’ (or Grade II, III, IV) to ‘pass’ (or Grade I) during pretraining versus post-training intubation between the two study groups (table 2). The 3D haptic training did not result in additional benefit in improved airway views, faster intubation times or reduced number of attempts compared with the 2D computer mouse-based training.

Table 2.

Comparison of effects of intervention on skills, knowledge, attitude, satisfaction

	Computer mouse (control)	Haptics (intervention)	p Value
Change in skills
Change in number of attempts—mean (SD)	0.67 (0.31)	1.24 (0.52)	0.352
Subjects change from intubation in >30 s to <30sec (%)	6 (25%)	4 (19%)	0.23
Change in POGO score—mean (SD)	12.7 (11.4)	26.7 (7.79)	0.332
Subjects change C-L score from fail to pass (%)	6 (25%)	3 (14%)	0.37
Knowledge
Per cent improvement in airway view knowledge (SD)	14% (4.3%)	5.7% (4.2%)	0.166
Per cent improvement in total intubation knowledge (SD)	19% (3.2%)	14% (3.4%)	0.28
Attitudes
Subjects with improvement in understanding of intubation	14 (93%)	10 (77%)	0.472
Subjects with improvement in comfort to intubate	8 (40%)	4 (22%)	0.323
Subjects with improvement in confidence to intubate	5 (24%)	2 (10%)	0.296
Satisfaction
This technology helped improve my skill	14 (58%)	9 (43%)	0.378
This technology helped improve my knowledge	22 (92%)	19 (90%)	0.662
I would use this technology for continued training if it was available	16 (67%)	11 (52%)	0.429
I would recommend this technology to others	15 (62.5%)	13 (62%)	0.864
The technology was easy to use	18 (75%)	10 (48%)	0.086

L score, Cormack-Lehane score; POGO score, percentage of glottic opening.

Examining the control and intervention study groups as one population as per our secondary hypothesis, there was a significant improvement in the evaluative metrics (number of attempts, subjects achieving intubation in fewer than 30 s, POGO score and knowledge score) from pretraining to post-training intubation (table 3). It also resulted in improved comfort, confidence and understanding in both groups.

Table 3.

Comparison of effects of both interventions combined on skills and knowledge

	Pretraining	Post-training	p Value
Skills and knowledge combined computer mouse and haptics
Attempts (SD)	2.89 (0.400)	1.96 (0.275)	<0.001
C-L score=1 (%)	15 (33%)	18 (45%)	0.053
POGO score (SD)	43.9 (5.92)	63.1 (5.61)	0.01
Successful intubation in <30 s (%)	22 (48.9%)	27 (60.0%)	0.021
Airway view knowledge score (mean per cent correct (SD))	63.6% (4.96)	74.0% (4.44)	0.002
Total knowledge score (mean per cent correct (SD))	65.1% (3.42)	82.5% (2.69)	<0.001

L score, Cormack-Lehane score; POGO score, percentage of glottic opening.

Discussion

Haptic technology is an exciting area of novel research in the field of simulation and provider training.^{5 10} Simulations are an important part of psychomotor skills development, and haptics may be used as a an added instructional design feature in this arena.¹⁹ Our results indicate that a 3D haptic training interface was not superior to 2D computer mouse-based training interface in developing NI skills measured on a neonatal airway simulator in this study population. Nor was the haptic interface superior in the acquisition of knowledge or changes in attitudes compared with the mouse-based interface. Additionally no differences were noted in the participants’ satisfaction with training between groups. Significant improvement was noted in the skills, knowledge, comfort and confidence of the population as a whole (across both groups) after completion of the SBS training session. However, although there was an improvement in skill, the population continued to have significant room for improvement with only 60% of providers intubating in 30 s.

While technology enhanced manikin-based simulator use has increased at a rapid pace in the past decade, there are several merits of computer-based simulators that are important for the simulation community to consider. Unlike many other types of simulators, a single computer-based simulator can have multiple case scenarios, multiple levels of difficulty and variations within the same scenario. Computer-based simulators can also be equipped with a variety of technologies, including development into 3D programmes such as haptics and virtual reality.

Participants in our study were captivated by the somatosensory experiences that haptics offered such as palpating a pulse, and feeling resistance of varying materials in the 3D environment. Haptic simulation has several other advantages compared with traditional training methods. It may be more cost-effective in that a computer with a 3D monitor and haptic device is a durable piece of equipment that can be constructed to last for long-term usage and placed in the clinical environment (in contrast to other simulators that may reside in a simulation-centre with limited access). Haptic technology facilitates self-training in several ways. First, all necessary intubation equipment is included ‘virtually’ within the programme, unlike with a manikin simulator. Since learners do not need to schedule an appointment to work with an instructor, this method of training allows for great flexibility and can be available for use at any time, including during nights and weekends. Second, trainees may modify the module's difficulty level or change the scenario to provide variety to their training and to further enhance their skills.¹⁰ Third, the haptic equipment could be modified to provide quantitative feedback on the directional and force vectors applied to the virtual patient using the haptic device, thus providing the learner with objective, formative feedback to improve their procedural skills. As with other simulation technologies, this technology avoids the ethical and safety issues associated with trainees making mistakes while developing their skills on real patients.

An important aspect of this study design is the use of a computer-based simulator for training while using a high-fidelity neonatal airway simulator to assess procedural performance. While simulators are widely used for instruction and assessment of procedural skill acquisition and performance, there is variable evidence to conclude that the skills learned and practiced on simulators do, in fact, directly translate to improved skills performance on patients. Cook et al and Zendejas et al^{23 24} conducted systematic reviews exploring this question and found a small beneficial effect of simulation on patient care. Zendejas et al²³ also noted many positive elements of simulation training including clinical variation, more learning strategies and longer time spent learning, all of which were associated with improved clinical outcomes. McGaghie et al,²⁵ in a critical review, conclude that with appropriate features, simulation based medical education can improve clinical outcomes. Of note, none of these studies focused specifically on intubation; all included other procedures in their analysis. Overall, more research is needed to evaluate the impact of simulation-based training on clinical performance and downstream patient outcomes, particularly in the field of intubation. However, it is generally agreed that the elements of simulation training are beneficial, and that simulation technology is a powerful tool for skill straining.

There were various limitations of the study. The small size and heterogeneity of our convenience sample of participants limits the power and generalisability of the study. Another limitation is that the pretraining intubation experience on the neonatal airway simulator may have contributed to the improvement in subsequent performance that was noted post training. Furthermore, since participants completed between 1 and 8 pretraining intubation attempts, this experience might have affected results inadvertently, by providing differing amounts of practice. An alternative may be to standardise the number of attempts that each participant is permitted before training. A third limitation is that the study intervention had only one session of training, thus is representative of skill improvement over a small amount of time. NI, however, like other procedures, is more often mastered over time through exposure and practice in varying settings. Our study did not assess the temporal relationship between training and mastery of NI. Fourth, it is unclear whether the haptic device itself or simply the computer-based training video was responsible for improvement in these groups. Future studies may include a third group as a control, which will have a half an hour gap between preintubation and postintubation without any intervention. Lastly, the objectives measured in this study focused on the completion of intubation, but not on the quality of technique demonstrated in this procedure. Subsequent studies may incorporate a procedural skills checklist rather than a dichotomous assessment of procedural success.

Several limitations of the haptic technology itself were discovered through this study. One of the main limitations was that the haptic device we studied required the use of only one hand, whereas intubation is a procedure that requires the use of both hands (ie, manipulation of the laryngoscope and insertion of the ET tube). Furthermore, the ‘laryngoscope handle’ on the haptic device was mobile at points that would be fixed in an actual laryngoscope. Thus, the process was unable to simulate this experience with precision and may limit transfer of learned skills to clinical practice. It is possible that these limitations contributed to the fact that the motor skills acquired on the device did not translate in this study to a significantly greater amount of improvement over the mouse-based training. Future haptic devices should incorporate several features suggested by the authors of this study, such as improved tissue compliance consistent with an infant's airway and increased fidelity to airway anatomy. These devices should aim to incorporate more realistic movement of the haptic device to simulate the trajectory of path of laryngoscope to help improve muscle memory and improve transfer to clinical intubations. Additionally, the next generation could also include a bimanual haptic device to more accurately simulate intubation. It is possible that with future advancements in the technology, the motor skills acquired on the device will simulate more precisely the skills needed for the true procedure, and that these will translate into a greater improvement.

Another disadvantage of haptic training is that it takes time to get acquainted with haptic software and technology. Additionally, at this time the current cost of the haptic interfaces may be prohibitive in some resource-poor settings⁴ and may only be purchased at larger institutional centres with adequate funding and resources for training. In future years, however, haptics may become available for smaller institutions and resource-limited settings since this technology will likely become less expensive with time.

Most of the participants felt that the module as a whole provided an adequate review of the intubation procedure, but did not view the haptics or the mouse-based interactions as an effective tool for self-improvement in intubation technique in the current delivery system. It is possible that they felt this way because the module spent the majority of time on the periprocedural aspects of intubation such as hand hygiene, positioning the patient for intubation, providing effective bag-valve mask ventilation for preoxygenation before intubation, preparing the appropriate equipment, and securing and confirming the position of the ET tube. The time and steps dedicated to airway view and inserting the ET tube were perhaps not given adequate weightage. Simulation educators should focus their use of haptics to tasks that benefit most from this instructional design feature. In reflecting on the qualitative and quantitative feedbacks from this work the authors suggest considering a blended interface (computer mouse-based and haptic) that selectively uses haptics. For example, a progressive approach could be taken where the learner spends time developing their skills using a computer mouse-based interface and then refines key elements of their skills using a haptic interface targeted at specific steps of the procedure such as visualising the trachea.

The haptic device used in this study is the first known attempt to incorporate haptics modality for NI training. While it currently has several limitations and has not yet been shown to be superior to other technologies in its effectiveness,⁷ it is still a developing technology. With revisions, it may eventually be used as a standardised teaching and training tool. Further research is necessary to compare 3D haptic technologies with the 2D environment (ie, computer mouse models) and other devices, as well as haptic simulations in other fields such as surgical specialties. As simulation technologies advance, we hope that medical providers will have more realistic training experiences and thus more successful outcomes in live situations.