Abstract
Background
Simulation-based technologies are emerging to enhance medical education in the digital era. However, there is limited data for the use of virtual reality simulation in pediatric medical education. We developed Virtual Doc as a highly immersive virtual reality simulation to teach pediatric cardiopulmonary resuscitation skills to medical students.
Objective
The primary objectives of this study were to evaluate participant satisfaction and perceived educational efficacy of Virtual Doc. The secondary aim of this study was to assess the game play features of Virtual Doc.
Methods
We conducted a prospective closed beta-testing study at the University of New South Wales (Sydney, Australia) in 2018. All medical students from the 6-year undergraduate program were eligible to participate and were recruited through voluntary convenience sampling. Participants attended a 1-hour testing session and attempted at least one full resuscitation case using the virtual reality simulator. Following this, participants were asked to complete an anonymous postsession questionnaire. Responses were analyzed using descriptive statistics.
Results
A total of 26 participants were recruited, consented to participate in this study, and attended a 1-hour in-person closed beta-testing session, and 88% (23/26) of participants completed the anonymous questionnaire and were included in this study. Regarding participant satisfaction, Virtual Doc was enjoyed by 91% (21/23) of participants, with 74% (17/23) intending to recommend the simulation to a colleague and 66% (15/23) intending to recommend the simulation to a friend. In assessment of the perceived educational value of Virtual Doc, 70% (16/23) of participants agreed they had an improved understanding of cardiopulmonary resuscitation, and 78% (18/23) agreed that Virtual Doc will help prepare for and deal with real-life clinical scenarios. Furthermore, 91% (21/23) of participants agreed with the development of additional Virtual Doc cases as beneficial for learning. An evaluation of the game play features as our secondary objective revealed that 70% (16/23) of participants agreed with ease in understanding how to use Virtual Doc, and 74% (17/23) found the game play elements useful in understanding cardiopulmonary resuscitation. One-third (7/23, 30%) found it easy to work with the interactive elements. In addition, 74% (17/23) were interested in interacting with other students within the simulation.
Conclusions
Our study demonstrates a positive response regarding trainee satisfaction and perceived educational efficacy of Virtual Doc. The simulation was widely accepted by the majority of users and may have the potential to improve educational learning objectives.
Keywords: pediatrics, cardiopulmonary resuscitation, virtual reality, medical education
Introduction
Simulation-based education plays a significant role in developing vital clinical assessment and management skills, especially in emergency scenarios such as a sudden cardiac arrest (SCA). SCA is an uncommon but life-threatening condition within the pediatric population [1,2] that includes the potential catastrophic consequences of mortality and poor neurological morbidity. An estimated 16,000 [3,4] children in the United States experience out-of-hospital SCAs and 5800 [4] experience in-hospital SCAs every year. Overall, survival rates are better for in-hospital arrests compared with out-of-hospital arrests [5], which is unsurprising considering that the most significant predictor of survival is early initiation of adequate cardiopulmonary resuscitation (CPR) [6]. However, junior doctors often have limited real-world learning experience managing an acutely ill and vulnerable child [7], especially in emergent scenarios, where active participation is often reserved for more experienced practitioners. In addition, many pediatric health care professionals feel inadequately prepared to manage a critically ill child, increasing the risk of medical errors [8]. Therefore, simulation training forms the cornerstone of resuscitation education, providing an opportunity for trainees to prepare for real-world clinical practice without risking patient safety [9-11]. However, in-person simulation training sessions are expensive and resource-intensive [12], and, therefore, the efficacy of alternative modalities should be explored.
The revolutionary development of highly immersive digitized learning resources such as virtual reality simulators harness the advantages of simulation while offering a cost-effective and widely accessible educational platform. Several studies have provided evidence for the use of virtual reality simulators for procedural skills training in the surgical field, with improved intraoperative performance [13] and accuracy [14] and a reduction in operating times [13] and errors [15-18]. One study investigated the use of virtual reality for CPR education compared with traditional mannequins and found virtual reality could be a valid and acceptable training method [19]. Furthermore, in a study by Wong et al [20], clinical CPR instructors outlined the limits of traditional education and noted the potential beneficial features of virtual reality as an additional learning tool. The experiential nature of virtual reality may also be conducive to developing procedural skills such as CPR, as repetitive hands-on practice permits the development of muscle memory [21], leading to technical skill competence. CPR involves the activation of declarative memory to recall the sequential steps and procedural memory to perform the active steps such as chest compressions [22], both of which may be developed and reinforced through the use of virtual reality.
To the best of our knowledge, there are no highly immersive virtual reality simulators for pediatric CPR training. As such, we developed Virtual Doc, a highly immersive and experiential 3D multimedia sensory virtual environment designed to teach pediatric CPR skills to medical students and junior doctors. We performed a prospective closed beta-testing study on undergraduate medical students with the primary aim to evaluate participant satisfaction and perceived educational efficacy of Virtual Doc. The secondary aim of this study was to assess the gameplay features of our simulation. We hypothesized that Virtual Doc would be an enjoyable and highly educational learning experience. We also hypothesized that the gameplay features would be beneficial in the facilitation of a realistic hands-on clinical learning experience.
Methods
Study Design
We conducted a prospective, uncontrolled, interventional closed beta-testing study in 2018 to assess the usability, acceptability, and perceived educational effectiveness of Virtual Doc as a method for teaching pediatric CPR to medical students. All participants provided implied consent by expressing interest in the study, attending a 1-hour closed beta-testing session, and completing an anonymous postsession questionnaire (Multimedia Appendix 1). The study was approved by the University of New South Wales Human Research Ethics Committee (HC180484).
Participants
Participants were eligible to partake in this study if they were currently enrolled in the 6-year undergraduate medical program at the University of New South Wales, Sydney, Australia. All medical students from the 6-year program were eligible and invited to participate in this study through an information email sent out by the University of New South Wales Medical Society administration. Students were instructed to contact the study investigators with their expression of interest. They were subsequently allocated to a 1-hour session in which they would engage with Virtual Doc and complete the postsession questionnaire. Participants were excluded from the analysis if they did not complete the questionnaire.
Virtual Doc
Virtual Doc is a virtual reality simulation developed for Oculus VR (Facebook Technologies LLC; Figure 1). The Virtual Doc app was developed using Unity3D software (Unity Technologies), and models were developed using open-source Blender modeling software. The simulation was conducted using the hardware of the Oculus Rift system linked to Alienware laptops (Alienware Corp). Virtual Doc is a first-person active learning experience through immersion within a multimedia sensory environment. Users wear a headset and use 2 hand controllers and are able to use these controllers, which are equipped with haptic technologies, to interact with the virtual environment. This simulation enables users to perform a series of actions including picking up objects, pushing buttons, and turning dials. They are also fully immersed with audiovisual aids such as auscultation of the heart and lungs.
Figure 1.
The Virtual Doc environment.
The scenario in this study involved an SCA in a young child that required appropriate and timely management. The required cardiac arrest management was in keeping with the Advanced Pediatric Life Support Australia guidelines [23]. Students were provided with a brief written clinical history on the main screen in the virtual environment and were then instructed to begin CPR. To complete the case, participants were required to assess the surroundings, check for a response, signal for senior help by pressing a call button, and then begin resuscitation by assessing the airway, breathing, and circulation; bagging and masking the patient (Figure 2); performing adequate chest compressions; and defibrillating the patient. As this scenario featured a shockable rhythm, this patient was defibrillated as part of the SCA management algorithm; however, this step may not be required for all pediatric SCA presentations. Shortly after the participant signaled for help, a senior physician arrived in the virtual environment to resume chest compressions while the user prepared for defibrillation. For successful completion of the case, all steps must have been completed correctly within 10 minutes.
Figure 2.
User is ventilating the patient with a bag-valve mask as per the resuscitation algorithm.
Outcome Measures
Validating the use of highly immersive and experiential virtual reality simulators as a form of active learning can be achieved by assessing various outcomes through the Kirkpatrick model of evaluation [24]. This 4-tiered hierarchy includes a scaffolding evaluation of reaction (level 1), learning (level 2), behavior (level 3), and results (level 4) [24]. As the levels progress, there is an increase in conclusively defining the efficacy of the educational activity [25].
The primary outcomes of this study were participant satisfaction (Kirkpatrick level 1) and the perceived educational value of Virtual Doc (Kirkpatrick level 2). Participant satisfaction was assessed through 1 Likert-style question and 2 trichotomous questions regarding enjoyment of the game and whether the participant would recommend the game to a colleague or friend. The perceived educational efficacy was assessed through 3 Likert-style questions including whether participants improved their understanding of the simulation objectives, whether engaging with Virtual Doc prepares students for a similar real-life clinical scenario, and whether different clinical cases would be beneficial for learning.
The secondary outcome was an evaluation of the gameplay features. Here, subjects were asked to respond to 3 Likert-style questions and 1 dichotomous question regarding the ease of understanding how to engage with the simulation, the level of ease to work with the interactive elements, whether the interactive elements were useful for learning, and whether the user would like to interact with other students in the virtual world.
Testing Session
All interested participants were allocated to a 1-hour in-person session. Attendees were invited to attempt at least one full clinical scenario using the virtual reality simulator. They were offered an unlimited number of attempts within the 1-hour session. After engaging with Virtual Doc, participants were invited to complete an anonymous questionnaire that evaluated the primary and secondary outcomes.
Data Collection and Statistical Analysis
Each postsession questionnaire was assigned a unique identification number and was completed anonymously by participants. The responses were collated and outcome measures were analyzed using descriptive statistics. The 7-point Likert scale responses were streamlined to 3 categories of agree (included all responses indicating strongly agree, agree, and mildly agree), neutral, and disagree (included all responses indicating mildly disagree, disagree, and strongly disagree) for the data analysis. Trichotomous questions requiring a response of yes, not sure, or no and dichotomous questions requiring a response of yes or no were analyzed as such. The data were analyzed using the statistical software R (version 1.1.423, R Foundation for Statistical Computing) [26].
Results
A total of 26 participants were recruited and attended 1-hour in-person closed beta-testing sessions. A total of 88% (23/26) of participants completed the anonymous questionnaire and were included in the analysis, and 12% (3/26) attended the testing session but declined to complete the questionnaire and were therefore excluded from the analysis. They were not required to provide a reason for declining to participate.
Study Population
All 23 of the included participants completed the demographic characteristics section of the questionnaire. Overall, the median age of participants was 22.0 (interquartile range 21.5-23.0) years, and 70% (16/23) of participants were female. In 2018, 61% (14/23) of participants were in year 4, 13% (3/23) were in year 5, and 26% (6/23) were in year 6. A total of 87% (20/23) of participants were local students, while 13% (3/23) were international students.
Outcome Measures
Participants responded to 10 Likert-style, trichotomous, or dichotomous questions regarding their experience with Virtual Doc. The results of these questions are summarized by question type in the horizontal stacked bar graphs in Figures 3-5.
Figure 3.
Results of the Likert-style questions.
Figure 5.
Results of the dichotomous question.
Figure 4.
Results of the trichotomous questions.
Participant Satisfaction
Respondents completed 1 Likert-style question and 2 trichotomous questions to evaluate satisfaction with Virtual Doc as an assessment of Kirkpatrick level 1. Virtual Doc was enjoyed by 91% (21/23) of participants. Furthermore, 74% (17/23) would recommend this simulation to a colleague, and 66% (15/23) would recommend this simulation to a friend.
Perceived Educational Value
Participants completed 3 Likert-style questions to assess the perceived educational efficacy of Virtual Doc, evaluating Kirkpatrick level 2, and 70% (16/23) of participants agreed they had an improved understanding of CPR following the use of Virtual Doc. In addition, 78% (18/23) agreed that Virtual Doc will help them prepare for and deal with real-life clinical scenarios. Furthermore, 91% (21/23) agreed there was a learning benefit to developing different cases within Virtual Doc, and 9% (2/23) were neutral.
Gameplay Features
Respondents completed 3 Likert-style questions and 1 dichotomous question evaluating the gameplay features of Virtual Doc. In terms of usability and gameplay, 70% (16/23) of participants agreed they had ease in understanding how to use Virtual Doc, and 74% (17/23) found the gameplay elements useful for understanding CPR. Furthermore, 30% (7/23) of participants found it easy to work with the interactive elements, which provides a template for future software improvements. In addition, 74% (17/23) would like the option to interact with other students within the game.
Discussion
Principal Findings
To the best of our knowledge, a prospective study on the use of a virtual reality simulator to teach pediatric CPR to medical students has yet to be conducted. In accordance with the Kirkpatrick model [24], the results of our study demonstrated high trainee satisfaction (Kirkpatrick level 1) and a perceived improvement in learning (Kirkpatrick level 2). In evaluating our secondary outcome of assessing the response to the virtual reality gameplay features, the majority of participants agreed with the ease in understanding how to use the simulation and the usefulness of gameplay features in improving their understanding of CPR and expressed a desire for interacting with others in the virtual environment. Although some participants found it difficult to work with the interactive elements, this closed beta test provided an opportunity to identify elements for future improvements, including the refinement of the interactive elements and future considerations such as a multiplayer design which may further enhance the learning experience and element of realism given the significant role of teamwork in resuscitation. Current data on trainee satisfaction, educational efficacy, and gameplay elements of virtual reality simulators for pediatric medical education is limited, and the encouraging results of this study will address the gap in the literature and could have an integral role in transforming the culture of future medical education.
Immersive technologies provide users with an engaging and enjoyable learning platform to train skills to proficiency without risking patient safety. Our study illustrated high participant satisfaction as Virtual Doc was enjoyed by almost all participants. This result parallels a study by Kron et al [27] in which an evaluation of medical student attitudes toward technology-based education was conducted and revealed that almost all students liked the idea of using technology to enhance their educational experience and thought that education should make better use of new media technologies. Beyond satisfaction, a previous study reported that experiential simulations offer a high level of interactivity and engagement, which may increase user motivation to learn [28]. Providing a highly engaging environment, such as that of Virtual Doc, is important for trainees as increased interest and satisfaction with the learning modality may result in an increased level of motivation to learn; one study showed that highly motivated students are more effective learners [29]. The importance of satisfaction and effective learning is reinforced by a study evaluating the effectiveness of virtual reality versus traditional or other forms of digital learning in educating health care professionals, in which a higher level of interactivity was more effective for the development of postintervention knowledge and skills [30]. The level of satisfaction with a learning experience also affects the probability of a user recommending such an experience to their peers [31]. The results of our study are congruent with this notion, as indicated by most students enjoying the simulation and agreeing with the intention of recommendation.
In this study, Virtual Doc was perceived as educationally efficacious by the majority of the participants, as indicated by a positive response toward improved understanding, preparation for real-world clinical scenarios, and support for designing additional cases. An important component of effective education in the adult learner involves the psychological concept of flow. This term is defined as a state in which a learner is “fully engaged, focused, and committed to the success of the activity” [32], and this can be achieved through active engagement in an experiential learning process. The use of highly immersive and interactive virtual reality simulators can improve flow, which may lead to superior learning outcomes and performance as echoed by the majority of participants reporting an improved understanding of CPR. Additionally, engaging sensory modalities including vision, haptics, and audition also promotes active learning, which can improve memory retention [33]. Virtual Doc includes sensory features through auscultation of heart and lung sounds and palpation of the peripheral pulses, which the user will hopefully be able to translate into the real-world clinical setting when assessing or managing a similar patient presentation.
Being immersed in the virtual yet realistic hospital environment of Virtual Doc provides users with a lifelike clinical experience. This digital-based environment is an innovative tool with realistic features supporting the recommendations of adult learning theories such as Knowles’ theory [34,35]. The responses in this study regarding the Virtual Doc environment and its ability to prepare students for real-life clinical scenarios showcases a facility in which the elements of adult learning theories can be included. Active experimentation within a realistic and immersive learning environment allows users to make clinical decisions and experience the consequences of their actions in real time, mimicking a real-world clinical environment. This element of realism is important in encouraging intrinsic motivation to support the needs of an adult learner.
Additionally, the Kolb cycle of learning encourages repetition, reflection, and correction to improve learning outcomes [36]. Virtual Doc supports this cyclical relationship as learners have the opportunity to engage in repetitive practice until they achieve educational mastery in a safe and controlled environment, which is an important benefit of simulation as this is severely limited in real-world emergency patient encounters, especially in the pediatric population [5]. The flexible design of virtual reality may enable health care professionals to have more convenient and frequent opportunities to practice and refine skills.
Several studies have demonstrated that junior doctors feel inadequately prepared and lack confidence in their early training years [37-41]. This is further supplemented by less favorable outcomes for neonatal resuscitation being described in the summer when senior staff physicians are less likely to be present and the overall volume of staff members is reduced [42]. Lacking confidence and inadequate preparation may translate into unsafe practice and poor patient outcomes and is an especially important consideration when managing emergency scenarios such as an SCA. Virtual reality simulators such as Virtual Doc may provide an avenue to bridge the gap between trainee and doctor through immersive gameplay by enabling users to master foundational principles and basic management algorithms and apply this knowledge to challenging simulated cases. In our study, the majority of respondents expressed that this simulation game would aid in developing clinical skills and preparing the user for dealing with real-life clinical scenarios. In addition, almost all participants expressed benefit in the development of different cases, indicating a positive response toward the clinical educational value of the simulation.
The gameplay elements involved in virtual reality add entertainment to the learning process, thereby motivating learners to engage in study. The triad of immersion, interaction, and imagination, as described by Burdea and Coiffet [43], are important factors of virtual reality technology. Coupled with andragogical principles, a highly interactive game design improves intrinsic motivation and creates a more enjoyable and engaging learning experience with a consequential potential to illustrate superior examination scores [44]. In our study, the vast majority of respondents agreed that the interactive gameplay elements were useful in understanding how to perform CPR and found the process of the virtual reality simulation easy to understand. This was further supplemented by almost all participants enjoying the experience in the assessment of participant satisfaction. However, some respondents did not experience ease in using the interactive elements within the virtual environment. As this was a closed beta test, the software was able to deliver its intentional value to the participants, but there were some minor issues reported with ease in fully integrating into the environment such as accurately picking up a stethoscope or pushing the assistance button. This feedback is being considered as we progress toward the final product and acts as identified areas for future developments.
Finally, nontechnical skills such as teamwork and leadership are paramount in successful resuscitation. The collegiality of the responding team directly impacts predictors of postarrest survival as a coordinated course of action is associated with improved patient outcomes [45]. Technology-based games such as Virtual Doc have the potential to facilitate an atmosphere of teamwork, and in the digital era, medical students have agreed with introducing multiplayer simulations if they were fun and developed clinical skills [27]. Likewise, our study demonstrated that the majority of participants were interested in interacting with other players. A multiplayer design within the virtual environment could afford an opportunity to rotate through different resuscitation team roles, supporting an active and immersive learning experience, with the potential to equip learners with the crucial teamwork skills required for the effective management of a pediatric SCA.
Limitations
Our study was limited by the small sample size and monocentric design, which limits its generalizability and external validity. However, participants from all years of study were eligible to participate, providing a degree of generalizability. This trial is also limited by voluntary convenience sampling, leading to a potential selection bias. Furthermore, this study used a single intervention with no active comparator, and therefore, the participants were not randomized and were unblinded to the intervention, which may formulate a measurement bias. However, the questionnaires were anonymized, providing a layer of security for participants to provide uncoerced feedback. Moreover, the final version of Virtual Doc will differ from the prototype used in this study as this was closed beta-testing and we are improving the software to address the feedback provided by the participants in this study. In addition, the prototype design of Virtual Doc in this closed beta test did not assess user hand technique or the quality of chest compressions. This limitation will guide future software improvements to ensure users are equipped with real-time visual and haptic feedback regarding the technique and quality of chest compressions.
Future Research
Given the encouraging results of this study, future research should include an investigator-blinded randomized controlled trial to objectively evaluate the educational efficacy of Virtual Doc against traditional simulation-based education or other digitized modalities such as serious games or e-learning modules. Furthermore, an evaluation of translation of knowledge into clinical practice (Kirkpatrick level 3) and the impact of learned knowledge on changes within the organizational practice to improve patient outcomes (Kirkpatrick level 4) should be performed, with emphasis on a reduction in medical errors and improved patient survival rates and overall health outcomes. This study does not assess the higher levels of the Kirkpatrick model, and this limitation should be the focus of future research to contribute to the quality of data for the efficacy of Virtual Doc.
Conclusion
In summary, our study demonstrates a positive response regarding the satisfaction and educational efficacy of Virtual Doc. Our findings reveal that this virtual reality simulation was widely accepted by the majority of users and has the potential to improve educational learning objectives. As such, our results provide a promising contribution to the educational revolution and may encourage the use of this emerging and versatile technology in the transformation of the 21st century medical curricula.
Acknowledgments
The study team would like to acknowledge the participating medical students for their involvement in this study, Reece Pahn (University of New South Wales Medical Society secretary) for facilitating the invitation of students, and Seiya Takeda and Zheyu Li for assisting with game development.
Abbreviations
- CPR
cardiopulmonary resuscitation
- SCA
sudden cardiac arrest
Appendix
Questionnaire instrument.
Footnotes
Authors' Contributions: The study was conceived by CYO and MJC. The study design and analysis were developed by CYO and MJC. CYO, MJC, ALS, and LD contributed to the conception and design of Virtual Doc. Subject recruitment and trial coordination were performed by JEP and MJC. LD sourced and coordinated the required technological equipment for the testing sessions. Data collection and statistical analysis were performed by JEP, MJC, and CYO. The paper was first drafted by JEP and revised in subsequent drafts by MJC, MEK, and CYO. All authors reviewed and approved the final manuscript.
Conflicts of Interest: None declared.
References
- 1.Ilina MV, Kepron CA, Taylor GP, Perrin DG, Kantor PF, Somers GR. Undiagnosed heart disease leading to sudden unexpected death in childhood: a retrospective study. Pediatrics. 2011 Sep;128(3):e513–e520. doi: 10.1542/peds.2010-2307. [DOI] [PubMed] [Google Scholar]
- 2.Meyer L, Stubbs B, Fahrenbruch C, Maeda C, Harmon K, Eisenberg M, Drezner J. Incidence, causes, and survival trends from cardiovascular-related sudden cardiac arrest in children and young adults 0 to 35 years of age: a 30-year review. Circulation. 2012 Sep 11;126(11):1363–1372. doi: 10.1161/CIRCULATIONAHA.111.076810. [DOI] [PubMed] [Google Scholar]
- 3.Topjian AA, Nadkarni VM, Berg RA. Cardiopulmonary resuscitation in children. Curr Opin Crit Care. 2009 Jun;15(3):203–208. doi: 10.1097/mcc.0b013e32832931e1. [DOI] [PubMed] [Google Scholar]
- 4.Marino BS, Tabbutt S, MacLaren G, Hazinski MF, Adatia I, Atkins DL, Checchia PA, DeCaen A, Fink EL, Hoffman GM, Jefferies JL, Kleinman M, Krawczeski CD, Licht DJ, Macrae D, Ravishankar C, Samson RA, Thiagarajan RR, Toms R, Tweddell J, Laussen PC, American Heart Association Congenital Cardiac Defects Committee of the Council on Cardiovascular Disease in the Young. Council on Clinical Cardiology. Council on Cardiovascular and Stroke Nursing. Council on Cardiovascular Surgery and Anesthesia. Emergency Cardiovascular Care Committee Cardiopulmonary resuscitation in infants and children with cardiac disease: a scientific statement from the American Heart Association. Circulation. 2018 May 29;137(22):e691–e782. doi: 10.1161/CIR.0000000000000524. [DOI] [PubMed] [Google Scholar]
- 5.Nadkarni VM, Larkin GL, Peberdy MA, Carey SM, Kaye W, Mancini ME, Nichol G, Lane-Truitt T, Potts J, Ornato JP, Berg RA. First documented rhythm and clinical outcome from in-hospital cardiac arrest among children and adults. JAMA. 2006 Jan 4;295(1):50–57. doi: 10.1001/jama.295.1.50. [DOI] [PubMed] [Google Scholar]
- 6.Goto Y, Maeda T, Goto Y. Impact of dispatcher-assisted bystander cardiopulmonary resuscitation on neurological outcomes in children with out-of-hospital cardiac arrests: a prospective, nationwide, population-based cohort study. J Am Heart Assoc. 2014 Apr 30;3(3):e000499. doi: 10.1161/JAHA.113.000499. https://www.ahajournals.org/doi/10.1161/JAHA.113.000499?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%3dpubmed. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Clerihew L, Rowney D, Ker J. Simulation in paediatric training. Arch Dis Child Educ Pract Ed. 2016 Feb;101(1):8–14. doi: 10.1136/archdischild-2015-309143. http://europepmc.org/abstract/MED/26614805. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Cheng A, Duff J, Grant E, Kissoon N, Grant VJ. Simulation in paediatrics: an educational revolution. Paediatr Child Health. 2007 Jul;12(6):465–468. doi: 10.1093/pch/12.6.465. http://europepmc.org/abstract/MED/19030409. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Ten Eyck RP. Simulation in emergency medicine training. Pediatr Emerg Care. 2011 Apr;27(4):333–341. doi: 10.1097/PEC.0b013e3182131fe0. [DOI] [PubMed] [Google Scholar]
- 10.Issenberg SB, McGaghie WC, Petrusa ER, Lee GD, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005 Jan;27(1):10–28. doi: 10.1080/01421590500046924. [DOI] [PubMed] [Google Scholar]
- 11.Gaba DM. The future vision of simulation in health care. Qual Saf Health Care. 2004 Oct;13 Suppl 1:i2–i10. doi: 10.1136/qhc.13.suppl_1.i2. http://qhc.bmj.com/cgi/pmidlookup?view=long&pmid=15465951. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Drummond D, Arnaud C, Thouvenin G, Guedj R, Grimprel E, Duguet A, de Suremain N, Petit A. An innovative pedagogic course combining video and simulation to teach medical students about pediatric cardiopulmonary arrest: a prospective controlled study. Eur J Pediatr. 2016 Jun;175(6):767–774. doi: 10.1007/s00431-016-2702-1. [DOI] [PubMed] [Google Scholar]
- 13.Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, Schroeder TV, Ottesen BS. Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ. 2009 May 14;338:b1802. doi: 10.1136/bmj.b1802. http://www.bmj.com/cgi/pmidlookup?view=long&pmid=19443914. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Gallagher AG, Ritter EM, Champion H, Higgins G, Fried MP, Moses G, Smith CD, Satava RM. Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg. 2005 Feb;241(2):364–372. doi: 10.1097/01.sla.0000151982.85062.80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Munz Y, Kumar BD, Moorthy K, Bann S, Darzi A. Laparoscopic virtual reality and box trainers: is one superior to the other? Surg Endosc. 2004 Mar;18(3):485–494. doi: 10.1007/s00464-003-9043-7. [DOI] [PubMed] [Google Scholar]
- 16.Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg. 2004 Feb;91(2):146–150. doi: 10.1002/bjs.4407. [DOI] [PubMed] [Google Scholar]
- 17.Seymour NE, Gallagher AG, Roman SA, O'Brien MK, Bansal VK, Andersen DK, Satava RM. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg. 2002 Oct;236(4):458–463. doi: 10.1097/01.SLA.0000028969.51489.B4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.McClusky DA, Gallagher AG, Ritter E, Lederman AB, Van Sickle KR, Baghai M, Smith C. Virtual reality training improves junior residents’ operating room performance: results of a prospective, randomized, double-blinded study of the complete laparoscopic cholecystectomy. J Am Coll Surg. 2004 Sep;199(3):73. doi: 10.1016/j.jamcollsurg.2004.05.157. [DOI] [Google Scholar]
- 19.Semeraro F, Ristagno G, Giulini G, Gnudi T, Kayal JS, Monesi A, Tucci R, Scapigliati A. Virtual reality cardiopulmonary resuscitation (CPR): comparison with a standard CPR training mannequin. Resuscitation. 2019 Feb;135:234–235. doi: 10.1016/j.resuscitation.2018.12.016. [DOI] [PubMed] [Google Scholar]
- 20.Wong M, Chue S, Jong M, Benny H, Zary N. Clinical instructors' perceptions of virtual reality in health professionals' cardiopulmonary resuscitation education. SAGE Open Med. 2018;6:2050312118799602. doi: 10.1177/2050312118799602. https://journals.sagepub.com/doi/10.1177/2050312118799602?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%3dpubmed. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Hunt EA, Duval-Arnould JM, Nelson-McMillan KL, Bradshaw JH, Diener-West M, Perretta JS, Shilkofski NA. Pediatric resident resuscitation skills improve after "rapid cycle deliberate practice" training. Resuscitation. 2014 Jul;85(7):945–951. doi: 10.1016/j.resuscitation.2014.02.025. [DOI] [PubMed] [Google Scholar]
- 22.Sense F, Maass S, van Rijn H. Interactions of declarative and procedural memory in real-life tasks: validating CPR as a new paradigm. Proceedings of the 14th International Conference on Cognitive Modeling; 2016; University Park. 2016. [Google Scholar]
- 23.Advanced Life Support Group . Advanced Paediatric Life Support, Australia and New Zealand: A Practical Approach to Emergencies. 6th edition. West Sussex: John Wiley & Sons; 2017. [Google Scholar]
- 24.Kirkpatrick DL, Kirkpatrick J. Evaluating Training Programs: The Four Levels. 3rd Edition. San Francisco: Berrett-Koehler Publishers, Inc; 2006. [Google Scholar]
- 25.Drummond D, Hadchouel A, Tesnière A. Serious games for health: three steps forwards. Adv Simul (Lond) 2017;2:3. doi: 10.1186/s41077-017-0036-3. https://advancesinsimulation.biomedcentral.com/articles/10.1186/s41077-017-0036-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Hornik K. The R FAQ. [2019-09-13]. https://cran.r-project.org/doc/FAQ/R-FAQ.html.
- 27.Kron FW, Gjerde CL, Sen A, Fetters MD. Medical student attitudes toward video games and related new media technologies in medical education. BMC Med Educ. 2010;10:50. doi: 10.1186/1472-6920-10-50. http://www.biomedcentral.com/1472-6920/10/50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Limniou M, Roberts D, Papadopoulos N. Full immersive virtual environment CAVETM in chemistry education. Comput Educ. 2008 Sep;51(2):584–593. doi: 10.1016/j.compedu.2007.06.014. [DOI] [Google Scholar]
- 29.Sutcliffe A. Multimedia and Virtual Reality: Designing Multisensory User Interfaces. Mahwah: Psychology Press; 2003. [Google Scholar]
- 30.Kyaw BM, Saxena N, Posadzki P, Vseteckova J, Nikolaou CK, George PP, Divakar U, Masiello I, Kononowicz AA, Zary N, Tudor Car L. Virtual reality for health professions education: systematic review and meta-analysis by the digital health education collaboration. J Med Internet Res. 2019 Jan 22;21(1):e12959. doi: 10.2196/12959. http://www.jmir.org/2019/1/e12959/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Hyejin C, Woonhan K, Hwayeol C. Effects of smartphone-based learning properties on user satisfaction and recommendation intention. Indian J Sci Technol. 2015;8:26. doi: 10.17485/ijst/2015/v8i26/80990. [DOI] [Google Scholar]
- 32.Taekman JM, Shelley K. Virtual environments in healthcare: immersion, disruption, and flow. Int Anesthesiol Clin. 2010;48(3):101–121. doi: 10.1097/AIA.0b013e3181eace73. [DOI] [PubMed] [Google Scholar]
- 33.Freeman S, Eddy SL, McDonough M, Smith MK, Okoroafor N, Jordt H, Wenderoth MP. Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci U S A. 2014 Jun 10;111(23):8410–8415. doi: 10.1073/pnas.1319030111. http://www.pnas.org/cgi/pmidlookup?view=long&pmid=24821756. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Akl EA, Sackett K, Pretorius R, Erdley S, Bhoopathi PS, Mustafa R, Schünemann HJ. Educational games for health professionals. Cochrane Database Syst Rev. 2008 Jan 23;(1):CD006411. doi: 10.1002/14651858.CD006411.pub2. [DOI] [PubMed] [Google Scholar]
- 35.Knowles M. The Modern Practice of Adult Education: Andragogy Verus Pedagogy. New York: Association Press; 1970. [Google Scholar]
- 36.Kolb AY, Kolb DA. Learning styles and learning spaces: enhancing experiential learning in higher education. Acad Manag Learn Educ. 2005 Jun;4(2):193–212. doi: 10.5465/amle.2005.17268566. [DOI] [Google Scholar]
- 37.Tallentire VR, Smith SE, Skinner J, Cameron HS. The preparedness of UK graduates in acute care: a systematic literature review. Postgrad Med J. 2012 Jul;88(1041):365–371. doi: 10.1136/postgradmedj-2011-130232. [DOI] [PubMed] [Google Scholar]
- 38.Goldacre MJ, Taylor K, Lambert TW. Views of junior doctors about whether their medical school prepared them well for work: questionnaire surveys. BMC Med Educ. 2010 Nov 11;10:78. doi: 10.1186/1472-6920-10-78. https://bmcmededuc.biomedcentral.com/articles/10.1186/1472-6920-10-78. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Bligh J. The first year of doctoring: still a survival exercise. Med Educ. 2002 Jan;36(1):2–3. doi: 10.1046/j.1365-2923.2002.01129.x. [DOI] [PubMed] [Google Scholar]
- 40.Wall D, Bolshaw A, Carolan J. From undergraduate medical education to pre-registration house officer year: how prepared are students? Med Teach. 2006 Aug;28(5):435–439. doi: 10.1080/01421590600625171. [DOI] [PubMed] [Google Scholar]
- 41.Drummond D, Arnaud C, Thouvenin G, Guedj R, Duguet A, de Suremain N, Petit A. [Newly formed French residents in pediatrics are not well prepared for conducting pediatric resuscitation after medical school] Arch Pediatr. 2016 Feb;23(2):150–158. doi: 10.1016/j.arcped.2015.11.012. [DOI] [PubMed] [Google Scholar]
- 42.Stewart JH, Andrews J, Cartlidge PH. Numbers of deaths related to intrapartum asphyxia and timing of birth in all Wales perinatal survey, 1993-1995. BMJ. 1998 Feb 28;316(7132):657–660. doi: 10.1136/bmj.316.7132.657. http://europepmc.org/abstract/MED/9522787. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Burdea GC, Coiffet P. Virtual Reality Technology. 2nd Edition. Hoboken: John Wiley & Sons; 2003. [Google Scholar]
- 44.Kiili K. Digital game-based learning: towards an experiential gaming model. Internet Higher Educ. 2005 Jan;8(1):13–24. doi: 10.1016/j.iheduc.2004.12.001. [DOI] [Google Scholar]
- 45.Hunziker S, Johansson AC, Tschan F, Semmer NK, Rock L, Howell MD, Marsch S. Teamwork and leadership in cardiopulmonary resuscitation. J Am Coll Cardiol. 2011 Jun 14;57(24):2381–2388. doi: 10.1016/j.jacc.2011.03.017. http://linkinghub.elsevier.com/retrieve/pii/S0735-1097(11)01072-2. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Questionnaire instrument.