Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Feb 1.
Published in final edited form as: Clin Simul Nurs. 2018 Feb;15:34–41. doi: 10.1016/j.ecns.2017.09.008

Interactive Anatomy-Augmented Virtual Simulation Training

Michelle Aebersold a,*, Terri Voepel-Lewis d, Leila Cherara b, Monica Weber c, Christina Khouri e, Robert Levine e, Alan R Tait f
PMCID: PMC5978424  NIHMSID: NIHMS926533  PMID: 29861797

Abstract

Background

Traditionally, clinical psychomotor skills are taught through videos and demonstration by faculty which does not allow for the visualization of internal structures and anatomical landmarks that would enhance the learner skill performance.

Methods

Sophomore and junior nursing students attending a large Midwestern Institution (N=69) participated in this mixed methods study. Students demonstrated their ability to place a nasogastric tube (NGT) after being randomly assigned to usual training (Control group) or an iPad anatomy-augmented virtual simulation training module (AR group). The ability of the participants to demonstrate competence in placing the NGT was assessed using a 17-item competency checklist. After the demonstration, students completed a survey to elicit information about students’ level of training, prior experience with NGT placement, satisfaction with the AR technology, and perceptions of AR as a potential teaching tool for clinical skills training.

Results

The ability to correctly place the NGT through all the checklist items was statistically significant in the AR group compared with the control group (P = 0.011). Eighty-six percent of participants in the AR group rated AR as superior/far superior to other procedural training programs to which they had been exposed, whereas, only 5.9% of participants in the control group rated the control program as superior/far superior (P < 0.001).

Conclusions/Implications

Overall the AR module was better received compared with the control group with regards to realism, identifying landmarks, visualization of internal organs, ease of use, usefulness, and promoting learning and understanding.

Keywords: Augmented Reality, Virtual Reality, Nursing Skills, Simulation, Learning, Nursing Education, Situated Learning Theory

Background

The U.S. is currently experiencing a shortage of qualified nurses, which will likely intensify. In 2014 almost 68,938 qualified applicants applying to baccalaureate and graduates nursing programs were turned away due to faculty shortage, insufficient clinical sites and clinical preceptors, limited classroom space, and budget constraints with two-thirds of them citing faculty shortages as a main reason (AACN, 2015). To address this shortfall, we need to find smarter and more efficient ways to provide clinical training to our students including leveraging simulation and virtual training methods that are not only more efficient but are more effective as well.

Simulation-based education (SBE) has become a cornerstone of undergraduate nursing education. The phases of simulation include preparation, briefing, simulation activity, debriefing, reflection, and evaluation (INACSL Standards Committee, 2016). A recent study sponsored by the National Council of State Boards of Nursing found that simulation can effectively replace up to 50% of clinical hours with no change in student critical thinking skills, knowledge or NCLEX licensure exam pass rates (Hayden et al., 2015). However, the study also recommended that the simulation experiences need to be of a high quality to ensure the results are reproducible. SBE increases student’s clinical judgment skills (Lasater, 2007), psychomotor skills (Shin et al., 2015) and non-technical or interpersonal skills (Searl et al., 2014).

In general, psychomotor skill proficiencies are practiced and demonstrated on task trainers or low fidelity simulation mannequins (Decker et al., 2008); with the initial training done through skills videos or faculty/instructor demonstrations. Additionally, nursing students often need to learn multiple skills in a short period and have limited access to the skills lab to practice those skills (Gonzalez & Kardong-Edgren, 2017). This creates a burden on faculty time and does not allow students optimal opportunities to practice skills or to practice them again in a just-in-time learning method when they need to perform them in the clinical area. For example, the student finds out they may be assigned a patient with a tracheostomy so they would like to review tracheostomy suctioning and other relevant skills. If the lab is not open, videos do not always offer an optimal learning method. If students had an interactive virtual training app on their mobile device at home, they could ‘practice’ suctioning techniques prior to the clinical setting.

Further difficulties in assessing student performance or competency occur when rubrics are used that are not validated or when inter-reliability between faculty is lacking (Watson et al., 2002). This indicates the need to use validated tools to assess students and perform inter-rater reliability among the faculty to improve consistency. All of these issues can lead to inconsistencies in skill development in student learners.

On the other hand, the use of videos and skill demonstrations does not always allow for the interactive visualization of internal structures and anatomical landmarks that would assist the student learner in performing the skill in a way that they could practice in real time with feedback. This research study focused on the preliminary evaluation of an anatomy-augmented procedure training video with interactive virtual simulation exercises to determine the impact on naso-gastric tube (NGT) placement skills, while also assessing qualitative metrics.

Literature Review

Simulation-based Nursing Education

The use of simulation to improve skills has a long history in a variety of areas (Aebersold, 2016). SBE provides the opportunity to practice skills in a safe, risk-free environment in a repetitive, deliberate and structured manner (Lateef, 2010). In nursing, SBE has proven to enhance technical (Jacobson et al., 2010) as well as non-technical skills (Hsu et al., 2015), though it is a costly technique. High-fidelity mannequins cost approximately $70,000 (U.S.) or more depending on options (Schiavenato, 2009). However, this does not include the estimated cost of many elements including faculty time, building space, and equipment (Aebersold & Tschannen, 2013). Evidence from recent studies supports the use of SBE in addition to technology-enhanced simulation as a way to supplement learning for procedural training and to develop clinical reasoning, which could offer a less costly solution (Cant & Cooper, 2014).

Augmented Reality

Augmented Reality (AR), as a technology-enhanced element, was coined by Tom Caudell in 1990, yet, the concept has been in use for several decades (Lee, 2012). AR was defined as “technology that allows a live real-time direct or indirect real-world environment to be augmented/enhanced by computer-generated virtual imagery information” (Zhu et al., 2014, p.3). These augmented realities are complementary to human cognitive processes in many areas including enhanced motivation and performance (Neumann & Majoros, 1998). AR, which combines virtual with physical reality and feedback, is increasingly being used in medical and nursing training to make simulations even more realistic and results in significantly improved skills transfer in students (Kilmon et al., 2010; Barsom et al., 2016). However, the difference between augmented reality and virtual reality is that the former enhances the learner's perception of the real world; while, the latter replaces it with a simulated one (Ferguson et al., 2015).

In a systematic review by Bacca et al. (2014), AR use in education, compared to traditional teaching materials, was found to be effective for better learning performance and motivation, student engagement, and positive attitudes. Also, in a meta-review by Radu (2014) findings highlighted the positive impact AR versus non-AR applications had on increased student understanding of content as well as memory retention and improved collaboration and motivation. Ferguson and colleagues (2016) recommend using teaching methods that accommodate students and the way they learn. They argue that AR is an immersive experience that builds on easily available and cost effective technology to enhance the learning process and its accessibility. The ease of access of AR from any mobile device, which could be accessed on or off-campus as well as in any clinical setting, can transform any context into a learning opportunity.

Empirical studies of augmented reality technology show promising results in learning outcomes. Garrett et al. (2015) tested the AR technologies’ potential for enhancing the learning of clinical skills in the lab. These authors found that students’ knowledge acquisition, as well as, access to resources and self-directed learning, improved. Students' overall views regarding AR application were positive. In a study by Vaughn et al. (2016), developed to increase the perception of realism in SBE, an educational application of Google Glass was used in conjunction with simulation mannequins. Google Glass is a wearable head mounted display that allows the wearer to see images projected through the glasses. Students reported that the integration of Google Glass gave them confidence in their skills and knowledge that can be transferred into the clinical setting (Vaughn et al., 2016). Some of the challenges of AR, however, include the general resistance to new technologies (Lee, 2012) and technical thresholds (Garrett, Jackson & Wilson, 2015).

Simulation-based education is an effective teaching and learning method for training and ascertaining competency of students before clinical experiences (Cant & Cooper, 2010). However, it is costly in terms of resources (personnel and equipment) and physical boundaries limitations. AR, using unique capabilities of mobile devices will provide a whole new set of affordable possibilities (Ferguson et al., 2015) to improve proficiency in the workplace with just-in-time learning that promotes active learning of nursing concepts and skills. Just-in-time training, which usually occurs immediately prior to skill performance in the actual clinical environment, has resulted in improved clinical performance in five of eight studies in a systematic review by Braga et al. (2015). The anatomy-augmented nursing procedure training video used in this study were developed using a hybrid approach. This approach combines video with computer graphics and can be manipulated to incorporate 3-D organ systems to allow learneers to better conceptualize an invasive procedure.

Situated Learning Theory

Situated learning theory (SLT), with its view of learning as a social process that is enhanced within the real environment, was used most frequently in AR studies, in addition to, inquiry-based learning and game-based learning theories (Saltan & Arslan, 2016). SLT, also known as situated cognition theory, has been used more frequently in nursing education, in recent years, and in particular within SBE, due to its focus on participative teaching methods (Onda, 2011). SLT is suited for the AR interactive learning in the lab, as well as in-situ context. The study’s propositions are rooted in SLT, presenting the anatomy-augmented nursing procedure training video as a situated interactive learning activity, taking place in a natural environment, with learners working on real-life activities (Onda, 2011).

Methods

Study Design

This study used a mixed methods approach employing techniques for quantitative data collection followed by qualitative data collection. The quantitative and qualitative data were collected and analyzed separately. The study was deemed exempt by the institutional review board (IRB). After receiving IRB approval, students were recruited through course announcements. Written informed consent was obtained prior to inclusion. Participants received a $10 gift card for their participation.

Sample/Intervention

The study population included sophomore and junior nursing students attending a baccalaureate nursing program at a large Midwestern Institution. The volunteers were randomly assigned to receive information regarding placement of a nasogastric tube (NGT) using either a video and didactic content per the Institution’s standard curriculum (Control group) or the iPad anatomy-augmented virtual simulation training module (AR group). The educational content was similar for each training method. Each student was given 20–25 minutes to review the educational material in a separate room. After reviewing the educational material, students were taken to the simulation suite where they demonstrated their ability to place an NGT on a mannequin. The evaluators were blinded to which education the students had received. After placing the NGT, participants were given feedback and then asked to fill out a survey on their experience.

Simulation Implementation

The participants had a briefing in the form of orientation to the study and its purpose. The Research Assistant briefly explained the iPad anatomy-augmented virtual simulation and demonstrated use to those participants who were randomized to this option. The simulation preparation varied based on the group to which the participant was randomly assigned. The preparation for the skills demonstration included either watching an animated video and reading didactic content in a module format per the Institution’s standard curriculum or the iPad anatomy-augmented virtual simulation training module. The participants were debriefed individually with feedback on their general performance and clarifications of misconceptions. A more extensive debriefing was not done for fear of contaminating the results because the study took place over four weeks.

Description of iPad Anatomy-Augmented Virtual Simulation

Based on a review of existing Nursing textbooks, the current Nursing Education Guidelines, peer-reviewed literature, and accepted medical treatment protocols the anatomy-augmented procedure training video for NGT insertion was developed by an outside company. The training video used a hybrid approach that combines video with 3-D computer graphics which allows trainees to visualize the invasive procedure by seeing the underlying anatomical structures (see figure 1). The training video was followed by interactive virtual simulation exercises that reinforce critical learning objectives for each subtask (see figure 2) as well as highlighting the safety measures to prevent complications (see figure 3). The application interaction took advantage of the three different types of touches the mobile device recognizes: taps, moves, and gestures. The company provided the iPads for use in the study with the application already downloaded and ready for use.

Figure 1.

Figure 1

Open in a new tab

Figure 2.

Figure 2

Open in a new tab

Figure 3.

Figure 3

Open in a new tab

Skill Competency Evaluation

The ability of the participants to correctly place the NGT in a high-fidelity mannequin was assessed using a checklist approach. The checklist included 21 activities that are required to demonstrate competency in successful NGT placement. This 21-item checklist was developed and reviewed by expert faculty for use in the institution’s skill checklist program. Content validity was established as part of the development process. However, because the checklist used at the institution includes two items that were not covered in either training module we excluded them from the analysis to avoid bias. These items were “Check the medical record” and “Coil NGT prior to insertion.” “Assure privacy” was also excluded since none of the participants had the ability to demonstrate this, i.e., no curtains to draw around the patient. The requirement to “listen at the tube for air exchange” was also excluded since this activity is no longer recommended at the institution and is, therefore, not taught routinely. This reduced the tool to 17 items.

The checklist items reflected each sequential step of the procedure and were scored using a 0 (incorrect), 0.5 (partially correct) or 1 (correct) rating system. For example, points were awarded based on the ability of the participant to gather supplies, perform appropriate hand hygiene, examination of the patency of the nostrils, etc. This scoring system is currently used at the institution and was agreed upon by the faculty assessors before the start of the study. Prior to using the tool inter-rater reliability was established between the two raters at .95. The raters were blinded to the training assignment of the students.

Feasibility Evaluation Using Quantitative and Qualitative Methods

Following placement of the NGT, all participants completed an on-line Qualtrics® survey to elicit information about their level of training, and prior experience with NGT placement. Participants who received the AR training were asked additional questions about their satisfaction with the educational training they received, and perceptions of AR as a potential teaching tool for nursing procedural skills training. The responses were measured using a 5-point Likert scale of strongly disagree, disagree, neither agree nor disagree, agree and strongly agree. Other questions used open-ended responses regarding their likes and dislikes, perceived potential barriers to successful adoption of AR technology, and any suggestions for improvement. The survey used was adapted from a survey used in previous studies to evaluate similar AR technology (Rochlen, Levine & Tait,).

Statistics

Descriptive statistics were described using frequency distributions. Competency scores between groups were compared using independent t-tests and Mann-Whitney U, as appropriate. Comparisons of categorical responses were analyzed using Chi-Square and Fisher’s exact tests. Significance was accepted at the 5% level (P <0.05).

Results

Sixty-nine nursing students comprised the study sample (Control = 34, AR = 35). There were no statistically significant differences in the demographics between the control and AR groups, both in terms of prior education regarding NGT placement (76.5% vs. 77.1%, respectively, P = 0.95), and a median number of clinical rotations completed (1.50 vs. 1.00, respectively, P = 0.24). Twenty-one percent of participants in the control group had received prior training in NGT placement using a simulator compared with 31.4% in the AR group (P = 0.31). None of the participants had previously attempted NGT placement on a patient.

Table 1 compares the competency scores between the groups for each of the checklist items. As shown, competency was significantly better among participants in the AR group for placement of towel/access to emesis basin/establishing stopping cue, correct tube insertion, and flexion of the head forward compared with the control group. Furthermore, results showed that the ability to correctly place the NGT through all the checklist items was significantly better in the AR group compared with the control group (15.96 ± 0.75 vs. 15.39 ± 1.01 out of 17, respectively, P = 0.011).

Table 1.

Comparison of checklist scores between Augmented Reality and Control Groups

Checklist item* Control (n = 34) AR (n = 35) P value
Assemble NG supplies 0.99 ± 0.09 1.00 ± 0.00 0.325
Hand hygiene 0.97 ± 0.17 1.00 ± 0.00 0.325
Introduction, explain procedure 1.00 ± 0.00 1.00 ± 0.00 1.000
Check patient ID 0.91 ± 0.29 0.77 ± 0.43 0.113
Don clean gloves 0.91 ± 0.29 0.77 ± 0.43 0.113
Position patient at 45° angle 0.96 ± 0.19 1.00 ± 0.00 0.184
Examine nostrils, select most patent 0.85 ± 0.34 0.94 ± 0.20 0.187
Towel, basin, establish stopping cue 0.59 ± 0.23 0.77 ± 0.28 0.004
Measure for correct tube length 0.97 ± 0.12 1.00 ± 0.00 0.160
Mark determined distance 1.00 ± 0.00 1.00 ± 0.00 1.000
Lubricate tube 0.97 ± 0.17 1.00 ± 0.00 0.325
Insert tube, head extension, direction 0.82 ± 0.24 0.96 ± 0.19 0.013
Flex head forward 0.47 ± 0.51 0.79 ± 0.41 0.006
Sip water during advancement 1.00 ± 0.00 1.00 ± 0.00 1.00
Advance tube to marked position 0.99 ± 0.09 1.00 ± 0.00 0.325
Radiographic confirmation 1.00 ± 0.00 0.97 ± 0.17 0.324
Verify placement, aspirate 1.00 ± 0.00 0.99 ± 0.08 0.324
Total score (out of 17) 15.39 ± 1.01 15.96 ± 0.75 0.011
Open in a new tab
*

Competency scored as 0 = incorrect, 0.5 = partially correct, 1 = correct

Data are mean ± SD, NG = nasogastric

Overall the AR module was significantly better received compared with the control video (see Table 2). Eighty-six percent of participants in the AR group rated AR as superior/far superior to other procedural training programs to which they had been exposed whereas only 5.9% of participants in the control group rated the control program as superior/far superior (P < 0.001). Participant perceptions of the media to which they were randomized had a statistical significance of p < 0.01 for AR modality as realistic, easy to use, and enjoyable,. The AR group also perceived AR as significantly more helpful in identifying landmarks and visualization of the internal structures as well as interactive and novel (p < 0.01). Compared to the control group, the AR group found the technology to be a useful tool for procedural skills training (p = 0.015).

Table 2.

Participants’ perceptions of the media to which they were randomized

Control (n = 34) AR (n = 35)
Agree/ Strongly agree P value
Anatomy was realistic 20 (58.8) 34 (97.1) < 0.001
Helped identify landmarks 20 (58.8) 35 (100) < 0.001
Better able to visualize internal structures 10 (29.4) 35 (100) < 0.001
Easy to use 30 (88.2) 33 (94.3) < 0.001
Enjoyable to use 9 (26.5) 30 (85.7) < 0.001
Improved my confidence in NGT placement 28 (82.4) 33 (94.3) 0.124
Features supported learning 31 (91.2) 35 (100) 0.063
Interactivity promoted understanding 11 (32.4) 33 (94.3) < 0.001
The interactive quiz helped prepare me for NGT placement 13 (38.2) 34 (97.1) < 0.001
Novel features promoted learning 17 (50.0) 30 (85.7) 0.026
Useful tool for procedural skills training 25 (73.5) 33 (94.3) 0.015
Open in a new tab

Data are n (%), AR = Augmented reality program, NGT = Nasogastric tube

Open-ended comments regarding the participants’ “likes” and “dislikes” of the augmented reality prototype and its potential as a training tool for nursing procedural skills, as well as perceived barriers, were recorded as unedited comments and sorted by themes (see Table 3). The comments regarding “likes” were primarily related to the interactivity, the ability to visualize the internal structures, and the interactive quizzes whereas the comments regarding “dislikes” were largely related to the access and pace.

Table 3.

Participants’ Comments of the Augmented Reality Prototype Presented in Themes

Theme
Likes
Usability

  • “I liked the way that it guided you step by step in an interactive way through the insertion of the NGT and then also quizzed you in an interactive way at the end.”

  • “It was more interactive and held my attention.”

  • “It kept me engaged and was easy to follow.“

  • “I liked that it was easy to use and understand. The bright colors made it more interesting to look at. I liked that it wasn't just a video
Visualization

  • “Being able to visualize the internal structured and what was actually happening inside the patient. I also liked the quiz a lot.”

  • “I liked the ability to visualize the structures and see where the NG tube actually was as we inserted it.”

  • “I liked the graphics and the interactive quiz at the end.”

  • “I loved how it caught my attention, had realistic graphics, and had
Fidelity

  • “It was very realistic and was very interactive.”

  • “Realistic, step by step and interactive!”
Dislikes
Usability

  • “ It was too slow paced.”

  • “ Some of the questions were redundant and the pace was slow for some of the videos.”
Access

  • “ If no access to iPad.”

  • “The only issue I can think of relates to the accessibility of this program.”
Open in a new tab

Discussion

Results of this study demonstrated the usability and feasibility of anatomy-augmented virtual reality for nursing procedural training among nursing students. The AR trainer used in this study resulted in improved competency in simulated NGT placement compared with a standard video and didactic training. This is consistent with the work done by Radu (2014) in which students had better memory retention and understanding of content. In the AR application the visualization of the internal structures may have provided a stronger memory trigger for those students. They performed significantly better on the three checklist items relating to head extension, head flexion and ensuring the patient had a cue to stop than the control group. Furthermore, the nursing students reported that the AR program provided a superior educational interface as compared with their standard training methods. Of note was the reported unique ability of the AR program to provide visualization of internal anatomical landmarks to augment their hands-on experience. This is consistent with the study by Bacca et al. (2014) in which AR use in education had positive student engagement and attitudes. Students also commented on the interactive quiz, the graphics being colorful, and attention grabbing. One of the benefits faculty valued was the ability of the students to observe the complications of doing the procedure incorrectly.

The ease of access for mobile technology such as AR programs can provide feasible options for faculty to use as training methods for students. The AR NGT trainer demonstrated that students were able to learn how to place an NGT with very little faculty intervention. This can save faculty time and lab space. The portability of this type of mobile technology application can be developed so students could have it accessible on any smart phone, tablet or computer as a just-in-time training application. The concept of just-in-time training allows a student or practitioner to practice a skill (in which they have already demonstrated competence) on the mobile technology just before doing it on a ‘real patient’ in the practice setting. This allows the person to make sure they are comfortable, remember all the steps and can perform the procedure effectively and safely.

Conclusions

Results of this study reinforce the effectiveness of AR technology for procedural skills training among nursing students. Further studies comparing AR technology with standard simulation trainers for a variety of procedural skills will be important prior to implementation into medical and nursing education curricula. . If we can supplement faculty time and space resources by the development of mobile technology applications students can use to practice before coming to the lab it will benefit students and others.

Limitations

This was a small study conducted in one school with students who had some prior exposure to this skill. More studies are required to determine the effectiveness of this application in users who have not received any training or in other settings.

Highlights.

  1. The Anatomy-augmented nursing procedure training video used in this study included virtual elements (combines computer models of anatomical structures) in real time to render mixed reality interactive experiences that enhance the perception learners have in the real world.

  2. The Anatomy-augmented reality group also perceived this training method as significantly more helpful in identifying landmarks and visualization of the internal structures which is critical to correctly placing nasogastric tubes.

  3. The students liked the graphics, the fun games and the interactive activities and quizzes at the end of the Anatomy-augmented virtual reality trainer.

Acknowledgments

Funding: This work was supported by a grant from the National Institutes of Health (NIGMS), Bethesda MD. to Dr. Levine (grant number R43GM113376).

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Disclaimer:

Dr. Levine is the President and Chief Medical Officer and President of ArchieMD, Inc. but was funded independently for this project by a grant from the National Institutes of Health. Dr. Levine was responsible for the development of the interactive program and read the final manuscript but had no involvement in subject recruitment, data collection, scoring, statistical analysis, or interpretation of the data. None of the other investigators have any financial, commercial, or other interests in ArchieMD, Inc. to declare.

References

  1. Aebersold M. The history of simulation and its impact on the future. AACN. Advanced Critical Care. 2016;27(1):56–61. doi: 10.4037/aacnacc2016436. [DOI] [PubMed] [Google Scholar]
  2. Aebersold M, Tschannen D. Simulation in nursing practice: The impact on patient care. The Online Journal of Issues in Nursing. 2013;18(2) doi: 10.3912/OJIN.Vol18. Manuscript 6. [DOI] [PubMed] [Google Scholar]
  3. American Association of Colleges of Nursing (AACN) Fact Sheets. Nursing Faculty Shortage. 2015 Retrieved from http://www.aacn.nche.edu/media-relations/fact-sheets/nursing-faculty-shortage.
  4. Bacca J, Baldiris S, Fabregat R, Graf S. Augmented reality trends in education: a systematic review of research and applications. Journal of Educational Technology & Society. 2014;17(4):133–149. [Google Scholar]
  5. Barsom EZ, Graafland M, Schijven MP. Systematic review on the effectiveness of augmented reality applications in medical training. Surgical Endoscopy. 2016;30(10):4174–4183. doi: 10.1007/s00464-016-4800-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Braga MS, Tyler MD, Rhoads JM, Cacchio MP, Auerbach M, Nishisaki A, Larson RJ. Effect of just-in-time simulation training on provider performance and patient outcomes for clinical procedures: a systematic review. BMJ Simulation and Technology Enhanced Learning. 2015;1(3):94–102. doi: 10.1136/bmjstel-2015-000058. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Cant RP, Cooper SJ. Simulation-based learning in nurse education: Systematic review. J Adv Nurs. 2010;66(1):3–15. doi: 10.1111/j.1365-2648.2009.05240.x. [DOI] [PubMed] [Google Scholar]
  8. Cant RP, Cooper SJ. Simulation in the internet age: The place of web-based simulation in nursing education. An integrative review. Nurse Educ. Today. 2014;34(12):1435–1442. doi: 10.1016/j.nedt.2014.08.001. [DOI] [PubMed] [Google Scholar]
  9. Decker S, Sportsman S, Puetz L, Billings L. The evolution of simulation and its contribution to competency. The Journal of Continuing Education in Nursing. 2008;39(2):74–80. doi: 10.3928/00220124-20080201-06. [DOI] [PubMed] [Google Scholar]
  10. Ferguson C, Davidson PM, Scott PJ, Jackson D, Hickman LD. Augmented reality, virtual reality and gaming: An integral part of nursing. Contemporary Nurse. 2015;51(1):1–4. doi: 10.1080/10376178.2015.1130360. [DOI] [PubMed] [Google Scholar]
  11. Garrett BM, Jackson C, Wilson B. Augmented reality m-learning to enhance nursing skills acquisition in the clinical skills laboratory. Interactive Technology and Smart Education. 2015;12(4):298–314. [Google Scholar]
  12. Gonzalez L, Kardong-Edgren S. Deliberate practice for mastery learning in nursing. Clinical Simulation in Nursing. 2017;13(1):10–14. [Google Scholar]
  13. Hayden JK, Smiley RA, Alexander M, Kardong-Edgren S, Jeffries PR. The NCSBN National Simulation Study: A Longitudinal, Randomized, Controlled Study Replacing Clinical Hours with Simulation in Prelicensure Nursing Education. Journal of Nursing Regulation. 2014;5(2 Suppl):S1–S64. [Google Scholar]
  14. Hsu LL, Chang WH, Hsieh SI. The effects of scenario-based simulation course training on nurses' communication competence and self-efficacy: A randomized controlled trial. Journal of Professional Nursing. 2015;31(1):37–49. doi: 10.1016/j.profnurs.2014.05.007. [DOI] [PubMed] [Google Scholar]
  15. INACSL Standards Committee. INACSL standards of best practice: SimulationSM Simulation design. Clinical Simulation in Nursing. 2016;12(S):S5–S12. http://dx.doi.org/10.1016/j.ecns.2016.09.005. [Google Scholar]
  16. Jacobson T, Belcher E, Sarr B, Ruitta E. Clinical scenarios: Enhancing the skill set of the nurse as a vigilant guardian. The Journal of Continuing Education in Nursing. 2010;41(8):347–353. doi: 10.3928/00220124-20100503-06. [DOI] [PubMed] [Google Scholar]
  17. Kilmon CA, Brown L, Ghosh S, Mikitiuk A. Immersive virtual reality simulations in nursing education. Nurs Educ Perspect. 2010;31(5):314–317. [PubMed] [Google Scholar]
  18. Klipfel J, Carolan B, Brytowski N, Mitchell C, Gettman M, Jacobson T. Patient safety improvement through in situ simulation interdisciplinary team training. Urologic Nursing. 2014;34(1):39–46. [PubMed] [Google Scholar]
  19. Lasater K. High-fidelity simulation and the development of clinical judgment: Student’s experiences. Journal of Nursing Education. 2007;46(6):269–276. doi: 10.3928/01484834-20070601-06. [DOI] [PubMed] [Google Scholar]
  20. Lateef F. Simulation-based learning: Just like the real thing. Journal of Emergencies, Trauma, and Shock. 2010;3(4):348. doi: 10.4103/0974-2700.70743. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Lee K. Augmented reality in education and training. TechTrends. 2012;56(2):13–21. [Google Scholar]
  22. Neumann U, Majoros A. Virtual Reality Annual International Symposium, 1998. Proceedings., IEEE 1998. IEEE; 1998. Cognitive, performance, and systems issues for augmented reality applications in manufacturing and maintenance; pp. 4–11. [Google Scholar]
  23. Onda E. Situated cognition: Its relationship to simulation in nursing education. Clinical Simulation in Nursing. 2011;8(7):e273–e280. [Google Scholar]
  24. Rochlen LR, Levine R, Tait AR. First person point of view augmented reality for central line insertion training: A usability and feasibility study. Simulation in Healthcare. 2017;12(1):57–62. doi: 10.1097/SIH.0000000000000185. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Radu I. Augmented reality in education: a meta-review and cross-media analysis. Personal and Ubiquitous Computing. 2014;18(6):1533–1543. [Google Scholar]
  26. Schiavenato M. Re-evaluating simulation in nursing education: beyond the human patient simulator. Journal of Nursing Education. 2009;48(7):388e–394e. doi: 10.3928/01484834-20090615-06. [DOI] [PubMed] [Google Scholar]
  27. Searl KR, McAllister M, Dwyer T, Krebs KL, Anderson C, Quinney L, McLellan S. Little people, big lessons: An innovative strategy to develop interpersonal skills in undergraduate nursing students. Nurse Education Today. 2014;34(9):1201–1206. doi: 10.1016/j.nedt.2014.04.004. [DOI] [PubMed] [Google Scholar]
  28. Shin S, Park JH, Kim JH. Effectiveness of patient simulation in nursing education: meta-analysis. Nurse Education Today. 2015;35(1):176–182. doi: 10.1016/j.nedt.2014.09.009. [DOI] [PubMed] [Google Scholar]
  29. Vaughn J, Lister M, Shaw RJ. Piloting Augmented Reality Technology to Enhance Realism in Clinical Simulation. CIN. Computers, Informatics, Nursing. 2016;34(9):402–405. doi: 10.1097/CIN.0000000000000251. [DOI] [PubMed] [Google Scholar]
  30. Watson R, Stimpson A, Topping A, Porock D. Clinical competence assessment in nursing: A systematic review of the literature. Journal of Advanced Nursing. 2002;39(5):421–431. doi: 10.1046/j.1365-2648.2002.02307.x. [DOI] [PubMed] [Google Scholar]
  31. Zhu E, Hadadgar A, Masiello I, Zary N. Augmented reality in healthcare education: An integrative review. PeerJ. 2014;2:e469. doi: 10.7717/peerj.469. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES