Exploring facilitation in virtual simulation in nursing education: A scoping review

Abstract

Objective

To systematically map existing research regarding the reflective feedback in virtual simulation in undergraduate nursing education.

Methods

A scoping review was conducted based on the Arksey and O'Malley framework and the PRISMA-ScR.

Results

We included 41 studies from 15 different countries. The simulation interventions allowed for interaction between the student and the virtual patient, the software, faculty, peers, or a combination of two or more of these. Students valued reflective feedback during and after the simulation.

Conclusions

Our review emphasizes the importance of a human in the loop. Feedback before, during, and after the simulation is possible in virtual simulation where the facilitator can pause the virtual scenario and stimulate reflections during the simulation to obtain deep learning. Virtual simulation provides opportunities to give feedback from the software, such as cues or direct feedback.

Innovation

There is a lack of focus on the feedback process and there is a need to revitalize the role of facilitators in a virtual simulation to determine their relative contribution in this process. Several studies reported the usefulness or the effect of virtual simulation on learning processes, but most lacked emphasis on investigating the significance of including a human in the loop.

Highlights

• •A variety of different software for virtual simulation have been tested or evaluated.
• •Nursing students learn from reflective feedback throughout the simulation session.
• •The feedback could be from the software or from a human in the loop (i.e. faculty).

1. Background

In undergraduate nursing, simulation is a well-established pedagogical method to prepare students for clinical rotation and to support them in transferring skills training from an academic context to a clinical context [1]. Simulation allows for scaffolding of skills, starting with particular sets of skills and building to more complex scenarios. Traditionally, simulation has involved using mannequins or standardized patients for scenario-based training. More recently, technologically enhanced simulation has proven to be a successful alternative offering clinical reality with virtual patients and a more immersive experience [2,3]. There is an inconsistency in the literature regarding the terminology of technology-enhanced simulation. Frequently used terms include digital simulation [4], virtual simulation [5], gaming [6], virtual reality [7], immersive simulation [8] and e-simulation [9]. In this review, we use the term virtual simulation as an umbrella term for the different, more specific, types of simulation present in the literature that are different from physical simulation (e.g., mannequins). Where ‘immersive’ or ‘gaming’ focus on the user perspective, virtual stays at the delivery level and can encompass a wider range of experiences. Virtual simulation is defined as clinical simulation offered on a computer, the Internet, or in a digital learning environment involving single or multiuser platforms [5]. According to the International Nursing Association for Clinical Simulation and Learning Standards Committee (INACSL) standards of best practice, virtual simulation allows learners to experience various auditory and visual stimuli [10]. In this review we understand the virtual simulation as being 2-dimensional computer software providing the participants with immersive experiences by engaging and interacting with the software on a physical, emotional, and cognitive level to achieve learning. Virtual simulations have shown to have several benefits, including preparing students for clinical practice at no risk for the patient, providing all students with similar learning situations, and the possibility of repeated training [2]. Virtual simulation is also less constrained by time and space. Taken together, virtual simulation can be valuable as a supplement for students entering higher education today [11,12].

The design of simulation-based training in nursing education may be based on international standards of best practice. The INACSL clearly states that learning outcomes in simulation-based training depend on the response from the facilitator on student performance and recommend pre-briefing and debriefing to optimize learning [10]. With simulation-based training in real-time, facilitators can stimulate students to reflect retrospectively on actions made in the simulation session. The response can be given in different forms, such as structured feedback or debriefing. Feedback in simulation training is usually understood as direct responses from the facilitator, faculty, or simulation software to the student, while debriefing becomes a more interactive, bidirectional, and reflective discussion [13]. However, since many virtual simulation software offers the possibility to stop and pause the simulation or to communicate continuously between the participants throughout the virtual scenarios, the essence of the facilitation can involve deeper reflection in action and exploration of the actions made [14]. The literature has shown that structured debriefing in virtual simulation is important for the students´ learning process and development of clinical reasoning skills [15]. Hence, debriefing is considered as the key for increased learning in all types of simulation-based training, including virtual simulation, because it combines the students´ prior knowledge and experience with reflective practice [5]. Many debriefing methods are concerned with exploring what went right, what went wrong, and what could have been done differently in a particular scenario. The degree of facilitation can vary between low, intermediate, and high degree, based on the educational level of the students, how engaged or involved the facilitator becomes in the discussion, reflection, or the cognitive reasoning processes [16]. The reflective feedback and the debriefing process can be performed in different ways and from different agents, such as a video-conferencing system (such as Zoom), in person from a facilitator, peer feedback, or self-debrief.

Despite the importance of facilitation and debriefing, previous reviews on virtual simulation in nursing education [5,17] have not described the specific function of the role of the faculty in virtual simulation, nor the similarities and differences with traditional simulation. The aim of this scoping review was to systematically map existing research regarding simulation with different virtual simulation software in undergraduate nursing education with a specific focus on the reflective feedback and identify any existing gaps in knowledge. The specific research questions were:

• •What are the reported findings for virtual simulation in nursing education?
• •How is facilitation and debriefing described in relation to learning outcomes for nursing students?

2. Methods

2.1. Study design

As the aim was to address a broad topic including different study designs, a Scoping Review was most appropriate. As the literature on virtual simulation in nursing is heterogenous and lack of a clear body of work focused on reflective feedback during the simulation, a scoping review was considered the best method to map the existing literature while not restricting included articles to certain methods or quality criteria. This allowed for identification of gaps in knowledge and determine trends in research [18]. The protocol was developed based on the Joanna Briggs Institute's framework [18], which is based on Arksey and O'Malley five step framework [19] and the Preferred Reporting Items for Systematic Reviews and Meta-analysis extension for Scoping Reviews (PRISMA-ScR).

2.2. Identifying the relevant studies

The search strategy was developed by the research team, which included an information specialist and several nurse educators with experience in simulation with virtual patients. Terms related to nursing education, virtual simulation, and facilitator role were combined with no restrictions on study design or language. The search strategy in MEDLINE is documented in Supplementary file 1. This search strategy was adapted and used in all the databases. Systematic and comprehensive literature searches were performed in MEDLINE, CINAHL, ERIC, EMBASE, and PsycInfo from January 2010 to September 2020. The reference lists of included articles were also screened to retrieve additional relevant references. The search was updated in March 2022, following the procedure described by Bramer & Bain [20].

2.3. Eligibility criteria and study selection

The screening and study selection was performed by eight authors working in four independent pairs, using the online review software Covidence (covidence.org). This was a two-step process; first to screen titles and abstracts of the retrieved articles, then the full-text papers. Using Covidence allowed for independent screening blinded, and each paper was examined by two members of the research team. The inclusion criteria were empirical studies investigating virtual simulation as a learning method in undergraduate nursing, focus on facilitation / debriefing / reflection (not necessarily in the aim, but clear and detailed described in the methods), nursing students alone or as part of multidisciplinary teams, written in Scandinavian or English language, and published within the last ten years. Studies were excluded if they were reviews, pilot studies, if they included post-graduate nursing students only, if the study reported the development of a software without an evaluation, or if the study investigated virtual simulation as a learning method with no mentioning of facilitation or debriefing. In case of doubt or disagreement within the pairs, the study was included in the first step. The inclusion and exclusion criteria were refined before the full text screening. The first author double-checked the final screening, and any disagreements were solved by discussion with a second author.

2.4. Charting the data

The following data was extracted into a data charting form using Excel: paper information (author(s), year of publication, journal, and study location), aim of the study, methodology, intervention type and comparison (if any), sample, outcome measures, and results. Descriptions of facilitation were also extracted, in order to answer the research questions. Each paper was read by two authors, but the first, second, and last author worked together with charting the data. This data charting form was the foundation for the analysis.

2.5. Collating, summarizing, and reporting the results

In line with the framework of a scoping review, the aim is to present an overview of the literature rather than to synthesize or aggregate findings [19]. According to the PRISMA-ScR quality appraisal is optional. As findings in scoping reviews can be problematic to implement in practice with no critical assessment [21], this step was included in this study. The quality of the included studies was assessed independently using the Mixed methods appraisal tool (MMAT) [22]. This tool was used due to the heterogeneous nature of the included studies. All authors were involved in the quality assessment, in pairs. Critical appraisal outcomes were compared once both reviewers had completed their appraisal. A third author assisted in resolving any disagreements in each pair. None of the studies were excluded based on the quality appraisal.

3. Results

3.1. Screening process and characteristics of the included studies

The first search in February 2021 identified 3243 potentially relevant studies, 1813 of which were unique. Abstract screening eliminated 1696 studies. From the full-text screening of the 117 remaining studies, 96 were excluded, most frequently because of: 1) not a virtual simulation, or 2) not facilitation / debriefing. This left 21 studies for the analysis. After a cross-reference search of the included initial search, another 6 studies were included. The updated searches in March 2022 identified 43 relevant studies. After screening and the quality assessment, 13 new studies were included. In total, 41 studies were included in the analysis. The selection process is shown in Fig. 1.

Fig. 1.

PRISMA 2020 flow diagram for new systematic reviews which includeed searches of databases and registers only.

From: Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. doi: https://doi.org/10.1136/bmj.n71.

For more information, visit: http://www.prisma-statement.org/.

All studies were published after 2012, 32 of the 41 studies (78%) were published within the last five years. Altogether, the studies included 3989 participants, mainly nursing students, from 15 different countries. All continents but Africa is represented. See Table 1 for more details.

Table 1.

Included studies.

Author(s), year (Country)	MMAT	Study aim	Research design and methods	Main results
Randomized Controlled Studies
Blanié et al., 2020 (France)	****	To compare the respective educational value of simulation by serious gaming with debriefing and a traditional teaching method to improve the clinical reasoning skills necessary to detect patient deterioration in nursing students.	2nd year nursing students were included and randomized into two groups: the simulation by gaming group (SG group) and the traditional teaching group (TT group). The primary outcome measure was the student's clinical reasoning skills regarding detection of clinical deterioration as measured by script concordance tests (SCTs) immediately after the session. N = 146	No significant difference between a serious game-based simulation format and a traditional teaching method immediately and 1 month after training. However, the students expressed more satisfaction and motivation with the innovative teaching method.
Gu et al., 2017 (China)	***	To report the effectiveness of vSIM for Nursing as a supplemental teaching strategy on performance of undergraduate students in a Fundamentals of Nursing course.	RCT posttest design with students randomly assigned to an experimental group with virtual cases in addition to regular course procedure. N = 28	Students in the experimental group had significantly higher knowledge scores than students in the control group.
Haerling, 2018 (USA)	****	To: 1) compare cognitive, affective, and psychomotor learning outcomes, and 2) describe cost-utility analysis comparing the two types of simulation activities in terms of costs and multiple measures of effectiveness.	A pre and post assessment. A random sample of participants from each group completed a post simulation performance assessment with a standardized patient. N = 84	No difference in learning or performance between the groups were found.
LeFlore et al., 2012 (USA)	*****	To compare achievement of learning outcomes of undergraduate nursing students when a virtual patient trainer or a traditional lecture was used to teach pediatric respiratory content.	RCT posttest design with senior nursing students. Students performed multiple choice questions to assess knowledge acquisition. N = 93	The students in the intervention group had higher knowledge acquisition and better knowledge application.
Liaw et al., 2014 (Singapore)	***	To describe the development of the virtual patient simulation and evaluate its efficacy, by comparing with a conventional mannequin-based simulation, for improving the nursing students' performances in assessing and managing patients with clinical deterioration.	A prospective, randomized controlled trial with a pretest-posttest design. N = 57	Using a self-directed learning approach, the virtual patient simulation provided learners control of their training agenda, allowing repeated “deliberate practice”, and receiving standardized feedback.
Liaw et al., 2020 (Singapore)	****	To evaluate the effectiveness of virtual reality in comparison with live simulations on medical and nursing students' communication skill performances and teamwork attitudes.	A prospective RCT with a pretest-posttest study design. N = 120	No significant differences in the communication posttest scores between virtual and simulation groups. Both groups increased interprofessional attitudes posttest.
Padihla et al., 2019 (Portugal)	*****	To evaluate the effect of clinical virtual simulation regarding knowledge retention, clinical reasoning, self-efficacy, and satisfaction with the learning experience among nursing students.	Randomized controlled trial with a pretest and 2 posttests. N = 42	The experimental group made more significant improvements in knowledge after the intervention (P = 0.001; d = 1.13) and 2 months later (P = 0.02; d = 0.75), and it also showed higher levels of learning satisfaction (P < 0.001; d = 1.33). We did not find statistical differences in self-efficacy perceptions (P = 0.9; d = 0.054)
Verkuyl et al., 2018 (Canada)	***	To compare three types of debriefing (in person, virtual, and self-debrief) after a virtual gaming simulation (VGS) on three student outcomes: self-efficacy (SE), knowledge, and the debriefing experience.	An experimental design using surveys was used to test the differences between the three VGS debriefing methods. N = 200	Within groups, students made significant knowledge and self-efficacy gains, and all groups rated their debriefing experience highly. There were no significant differences in outcomes between groups.
Non Randomized Studies
Atthill et al., 2021 (Canada)	*****	To explore the impact of virtual asynchronous debriefing after a virtual simulation game on nursing students' perceived anxiety and self-confidence for engaging in clinical decision-making (CDM).	An experimental design compared virtual asynchronous debriefing with traditional face-to-face debriefing. N = 64	Virtual asynchronous debriefing resulted in increased self-confidence and reduced anxiety for CDM related to gathering data, seeing the big picture, and knowing and acting. Asynchronous debriefing was comparable with face-to-face debriefing and resulted in a significantly greater reduction of anxiety in the CDM dimension of data gathering.
Bogossian et al., 2015 (Australia)	*****	To investigate undergraduate nursing students` theoretical and applied learning in response to the e-simulation program FIRST2ACT WEB™ and explore predictors of virtual clinical performance.	A multi-center trial with a posttest design. Participants proceeded through three phases: (i) pre-simulation, (ii) e-simulation, and (iii) post-simulation feedback and evaluation. N = 489	The students improved performance from first to third e-simulation. Students' knowledge improved significantly from pre to post e-sim.
Campbell et al., 2021 (USA)	****	To evaluate perceptions of awareness, knowledge and sensitivity of future nurses concerning Alzheimer's Disease patients before and after participation in a simulated virtual reality dementia experience.	A quasi-experimental repeated measure pre-post design was used with a convenience sample of undergraduate baccalaureate nursing students from three different courses at large public university in the Midwest. N = 163	Statistically significant changes (p < 0.001) were noted in the pre/post DAS survey and the Healthcare Tour Survey. No significant change was noted on the KAML-C, although there was a trend toward improvement. No differences were noted between courses on any of the surveys
Dubovi, 2018 (Israel)	***	To compare the effectiveness of online computer-based simulations designed using two alternative instructional approaches, Productive Failure and Simple-to-Complex sequencing, on learning of clinical reasoning skills.	A pre- and post-test of clinical skills was tested after students completed two online simulations using Productive Failure and two online simulations with Simple-to-Complex approach. N = 103	Clinical reasoning learning gains were significantly higher for online simulation designed with the Simple-to-Complex approach than simulation designed with Productive Failure approach (F (1,102) =260.15, P < 0.001) and attempts (F (1,102) =167.39, p < 0.001) in learning with StC simulations than they did with PF simulations. The amount of time that students were engaged in learning with simulations was significantly associated with learning gains scores.
Hanson et al., 2020 (Australia)	****	To compare the effect on student learning, satisfaction and comfort following exposure to a three-dimensional artefact in a virtual facility (CAVE2™) with viewing of the same artefact using a mobile handheld device with stereoscopic lenses attached.	Pretest posttest design with nursing or midwifery students performing multiple choice online tests before and after to measure knowledge. N = 249	No disadvantaged in terms of knowledge acquisition by using either CAVE2 or the handheld visualization mode. Students were more satisfied (p = 0.013) and clinical learning (p < 0.001. No significant difference in the satisfaction with debriefing and reflective practice process.
Heinrich et al., 2012 (USA)	****	To examine the impact of MicroSim® classroom-based simulation on senior nursing students' perception and content knowledge.	This study consisted of a pre post test design intended to determine whether students' cognitive knowledge increases after MicroSim compared with typical classroom experiences. MicroSim learning experiences were integrated into the class content on two different days, 1 week apart. The convenience sample included 56 senior baccalaureate nursing students. N = 56	The mean difference between pretest 1 and pretest 2 for the pulmonary emboli case study was not significant. The mean difference between pretest 1 and posttest for pulmonary emboli was significant. The study also found a significant difference between pretest 2 and posttest. The mean difference between pre-test 1 and pretest 2 for diabetic ketoacidosis was significant.
Kang et al., 2020 (Korea)	***	To compare critical-thinking disposition and self-directed learning activity before and after virtual simulation among Korean nursing students.	One-group pre-post design to investigate critical-thinking disposition, self-directed learning ability, and simulation effectiveness of 47 senior nursing students. N = 47	No statistically significant differences emerged between the assessment on critical thinking or self-directed- learning ability before and after virtual simulation. However, one subscale on self-directed learning ability, gathering resources for learning, showed statistical significance.
Liu, 2021 (USA)	****	To examine the effect of virtual simulation as an educational tool on undergraduate nursing students` mental health literacy.	A prospective cohort study measuring beliefs about the effectiveness of specific interventions for managing depression and schizophrenia. N = 299	Overall, nursing students in the simulation and non-simulation cohorts shared agreement on a wide range of help-seeking behaviours and treatment options for managing depression and schizophrenia. Virtual simulation moderated students` uncertainty toward some treatment intervention for managing depression. Virtual simulation increased students` perception of usefulness of some self-help strategies for managing schizophrenia.
O'Flaherty and Costabile, 2020 (Australia)	*****	To explore the impact of embedding a desktop simulation about the hemolytic disease of the newborn (HDN) in a first-year, undergraduate nursing course on students` active learning, knowledge acquisition, self-confidence and development and application of critical thinking and academic writing.	A pre-test post-test design study. N = 60	The students were mainly satisfied with the simulation and reported an increase in self-confidence after the simulation.
Roh and Kim, 2014 (South Korea)	****	To assess the effects of computer-based simulation on nursing students' performance, self-efficacy, post-code stress, and satisfaction between computer-based simulation plus instructor-led cardiopulmonary resuscitation training group and instructor-led resuscitation training-only group.	A nonequivalent control group posttest-only design study. N = 213	There were no significant differences in results in the two groups
Sapiano et al., 2018 (Malta)	*****	To investigate the effectiveness of virtual simulation in improving student nurses` knowledge and performance during rapid patient deterioration	A pretest-posttest design with knowledge tests and performance registration. N = 166	There was a significant improvement in post-scenario knowledge. Knowledge was not a predictor of students` performances.
Tschannen et al., 2012 (USA)	****	To determine the impact of virtual reality simulations on knowledge transfer of nursing students.	A quasi-experimental design with 115 students N = 115	Total CRI score for the intervention group was significantly higher than the score for the control group.
Weston and Zauche, 2021 (USA)	*****	To compare the Assessment Technologies Institute (ATI) scores of prelicensure nursing students within a prelicensure program who completed their pediatric clinical practicum in person in the clinical and simulation practice setting versus virtually.	Cross-sectional descriptive evaluation approach. N = 186	No significant difference in ATI scores between students who completed clinically versus virtually.
Mixed Methods
Choi et al., 2020 (China)	****	To (1) compare the efficacy of a computer simulation-based, interactive communication education (ComEd) program and an attention control (AC) program on communication knowledge, learning self-efficacy, and communication efficacy at baseline and twice after the intervention; and (2) assess the acceptability and satisfaction of the ComEd program.	Mixed method, randomized controlled study design with repeated measures. N = 131	The ComEd program, compared to the AC program, significantly improved communication knowledge, learning self-efficacy, and communication efficacy among nursing students, and the effects of the program were maintained at two weeks. Both quantitative and qualitative findings of the study confirmed that the participants experienced cognitive, emotional, and behavioral changes along with improved self-efficacy.
Donovan et al., 2018 (USA)	*****	To examine undergraduate nursing student perceptions and experiences when given a computer-based simulation program as a preparation prior to their simulated lab experience.	Mixed method with retrospective/pretest survey of past simulated experience and posttest survey with focus groups, after the first and last simulation debriefing of the semester. N = 82	Analysis of the numeric data suggested the introductory program positively enhanced learning. Narrative data elicited six themes: improved prioritization, role modeled nursing care, individualized preparedness, engaged critical thinking, decreased level of anxiety and increased confidence in the lab. Quantitative and qualitative perspectives suggested that reinforcement of learned concepts through the computer-based simulation scenarios were central to positive student performance during the simulation lab experience.
Flo et al., 2021 (Norway)	*****	To explore how second-year nursing students experienced learning through virtual simulations during the pandemic.	Mixed method. Students performed two sessions of virtual simulations and completed an online questionnaire after each simulation. Students were also invited to focus group interviews. N = 69	Students completed 79 virtual simulations individually in preparation for the group simulations in the first cohort, and 104 in the second cohort. Four themes emerged from the interviews; learning from self-training, learning from the software, learning from peers, and learning from faculty.
Foronda et al., 2016 (USA)	***	To determine the impact of a virtual simulation to teach nursing students concepts of triage	Mixed methods approach with six nursing students participating in a web-based virtual simulation of ana earthquake. The students took a 20-item MC test before and after and participated in a debriefing setting N = 6	No improvement in the post-test. The qualitative data found four themes: fun, appreciation for immediate feedback, better than reading, and technical issues.
Lee et al., 2021 (Korea)	***	To (1) identify non-therapeutic communication patterns that nursing students, exhibit in simulated situations; and (2) explore students` responses in challenging situations	Mixed-method research design to analyze the communication patterns exhibited by nursing students participating in the ComEd program N = 66	False reassurance was the most common non-therapeutic communication pattern used by nursing students.
Liaw et al., 2019 (Singapore)	*****	To describe the development of a 3D-VE for interprofessional team care, and to evaluate healthcare students` perceptions and experiences of the virtual environment for collaborative learning.	Mixed method with pretest - posttest and focus groups N = 29	The students demonstrated significant improvements in their attitudes toward healthcare team and interprofessional collaboration after the collaborative learning. Four themes emerged from the focus groups: feeling real, less threatening, understanding each other's role, and technical hiccups.
Redmond et al., 2020 (Ireland)	****	To (1) develop a virtual patient with a chronic wound and evaluate the perceived learning gains of engaging with this source; and (2) evaluate the perceived impact of using a VP on future nursing wound care performance.	Cross-sectional survey design N = 148	The virtual patient also provided the students the opportunity to develop their clinical reasoning skills.
Rim and Shin, 2022 (Korea)	***	To develop a multi-user virtual simulation program for metacognition and evaluate the students` satisfaction, clinical judgment, and nursing competence.	A mixed methods with 45 senior nursing students. Focus group interviews (12 students) and pre- post-test survey N = 57	Total scores for nursing competency before and after the simulation program had a significant increase (from 243.6 to 264.0), p < 0.001. The major theme from the focus group were democratic participation, scaffolding mechanisms that facilitate learning process, leap to be like a nurse.
Tjoflåt et al., 2018 (Norway)	*	To evaluate second year nursing students' experiences with a virtual clinical simulation scenario in surgical nursing from vSim® for Nursing.	The study follows a descriptive and convergent mixed method design QUAN/QUAL, in which quantitative and qualitative data are collected simultaneously, with equal priority. N = 65	The majority of Norwegian nursing students evaluated the virtual clinical scenario in surgical nursing from vSim® for Nursing useful, realistic and educational in preparing for clinical placement in surgical care. However, a small portion of the nursing students had trouble understanding and navigating the American vSim® for Nursing program. Conclusions: Introducing virtual simulation tools into the nursing education encompasses faculty and student preparation, guidance from faculty members during the simulation session and support for students who are facing difficulties with the simulation program.
Verkuyl et al., 2019 (Canada)	***	To examine the impact of three different debriefing methods (self-debrief only, self-debrief followed by a small-group debrief, and self-debrief followed by a large-group debrief) on nursing students' knowledge and debriefing experience after playing a VGS.	A sequential mixed methods study was conducted. The quantitative component included a pretest posttest quasi experimental design with a control and two intervention groups (Figure). The study also included a qualitative component, with focus groups to deepen our understanding of the quantitative data N = 254	All groups made significant knowledge gains. The self-debrief only participants had the lowest debriefing experience scores. The small- and large-group debriefing after self-debriefing offered many benefits. One important finding was that undergraduate students value and benefit from a self-debrief before a group debrief. Students appreciated the opportunity to collect and analyze their thoughts in the self-debrief before participating in a group debrief
Qualitative design
Buijs-Spanjers et al., 2020 (Netherlands)	*****	To explore aspects essential to enhance students' attitudes and learning experiences regarding delirium by making use of a video-based serious games' narrative.	Semi-structured interviews were performed with seven nursing and nine medical students about their attitudes and learning experiences, after they had played the game. A qualitative descriptive design and inductive content analysis with constant comparison were used. N = 16	The patient's and nurse's perspective, interactivity to experiment, realistic views on care options, and feedback on care actions were important for enhancing students' attitudes and learning experiences regarding delirium. Students felt these aspects encouraged them to get actively involved in and experiment with the study material, which in turn led to enhanced reflection on delirium care and education. Our findings highlight the importance of a more patient-oriented focus to delirium education to drive attitudinal change. Students' learning experiences were further enhanced through their affective responses provoked by the perspectives, interactivity, realism, and feedback.
Johannesson et al., 2013 (Sweden)	*****	To investigate students' perceptions of how they learn manual clinical skills, i.e., to discover what students are experiencing and what they think about their learning in simulated skills training	Qualitative design with ten individual interviews. The material was analyzed using inductive content analysis, with both manifest and latent content considered. N = 10	Three themes emerged: what the students learn, how the students earn, and the simulator's contribution to the students` learning
Johnsen et al., 2021 (Norway)	*****	To explore nursing students' perceptions of combining hands-on simulation with simulated patients and a video-based Serious Game in preparation for a home healthcare clinical placement.	An exploratory qualitative design with five focus groups with a convenience sample of 26 students. Analyzed with B&C thematic analysis. N = 26	Four main themes that influenced students` perceptions of combining the two simulations: personal engagement (being actively engaged, being an observer, acquiring experiential learning), contextual and environmental factors (authentic patient actors, visual and contextual factors), a structured and safe learning environment (the teacher's role as a facilitator, group dynamics), and organizational and technical factors (organizational factors, technical factors).
Koivisto et al., 2017 (Finland)	*****	To investigate nursing students' experiential learning processes during a 3D simulation game and to determine which game characteristics support experiential learning.	Qualitative methods were used to produce new theoretical knowledge about nursing students' experiential learning processes during gaming. Data consisted of audio and video recordings of gaming sessions and focus group interviews N = 8	Characteristics of the 3D simulation game supporting students` concrete experiences. Patient-related experiences were supported by audiovisual authenticity, the authenticity of the scenarios, and interactivity.
Liaw et al., 2021 (Singapore)	*****	To explore the experiences of healthcare students and facilitators on the use of 3D virtual world for interprofessional team-based virtual simulation.	A qualitative descriptive study was conducted on a purposive sample of 30 healthcare students and 12 facilitators using focus group discussions and individual interviews. N = 42	Four themes emerged from their experiences: the “wow experience”, authentic experience on collaborative care, ease of learning, and preeminent role of the facilitator. The simulation provided the “wow” experiences through contextual, collaborative, and experiential learning approaches. Despite technical challenges, the participants were wooed by the comforts of learning from home and the psychological safety in virtual environment. The facilitators played a critical role in optimizing learning engagement to win learners over.
Peddle et al., 2019 (Australia)	*****	To investigate how students' interactions with virtual patients influenced learning and practice of non-technical skills.	This paper reports part of a larger multisite exploratory, qualitative research project using case study methodology with focus groups and individual interviews N = 76	Findings indicated that the different years interacted differently with the virtual patients. Four themes were recognized in the data: how the virtual patients enabled learning non-technical skills, learning surrounding the virtual patient encounter, changing the way students perceive practice and potential limitations to learning.
Saunder and Berridge, 2015 (United Kingdom)	*****	To evaluate the implementation of Shareville in the undergraduate and postgraduate pre-registration nursing curricula by students and staff.	An exploratory, qualitative research design. Semi-structured interviews with 19 students (5 undergrad, 14 postgrad) N = 19	The study found five main themes relating to: learning, clinical practice, functionality, fidelity, and pedagogy
Shorey et al., 2020 (Singapore)	****	To examine student users' attitudes and experiences and clinical facilitators' perspectives on student performance in the clinical setting post-virtual patient training.	Descriptive qualitative study with focus group discussions and individual interviews. N = 30	Themes from student perspectives: Attitudes toward virtual patient training, virtual patients` role in student development, lack of realism, enhanced features and implementation suggestions, value of technology in teaching communication.
Singleton et al., 2021 (United Kingdom)	*****	To explore nursing students', simulation technicians', and lecturers' experience of using a virtual reality simulation to support learning about the recognition and management of an acute diabetic emergency.	A descriptive qualitative study with focus group discussions. N = 21	Five themes emerged: engagement, immersion, confidence, knowledge and challenges
Verkuyl et al., 2020 (Canada)	*****	To explore the following debriefing practices: self-debrief only; self-debrief followed by a small-group debrief; and self-debrief followed by a large-group debrief.	A focus group study methodology with 19 first year students N = 19	Study results included four themes: developing process; promoting safe debriefing spaces; fortifying knowledge; and engaging in reflection.

The 41 studies included had different research designs (Table 1). Most of the studies were intervention studies testing a large variety of virtual simulations. Eighth studies were randomized controlled trials, 13 were non-randomized studies, 10 used mixed methods, and 10 were qualitative studies. Of the qualitative studies, two employed individual interviews [23,24], three focus group interviews [[25], [26], [27]], three combined focus groups and individual interviews [[28], [29], [30]], one analyzed audio-recordings [31], and one employed stimulated recall interviews [32]. Of the quantitative and mixed method studies, 11 used self-report questionnaires, 8 had objective measures, such as observation or skills test, and 12 had a combination of both.

3.2. Study findings

All included studies tested or evaluated virtual simulation in some form (Table 1). Several studies compared virtual simulation to traditional simulation [[33], [34], [35], [36], [37], [38], [39], [40]], or tested effects of virtual simulation without comparison [12,[41], [42], [43], [44], [45], [46], [47]]. Some studies investigated different models of debriefing [4,33,48,49] or students´ experiences [[50], [51], [52]]. The qualitative studies investigated students' perceptions of how they learn manual clinical skills, one study [24] also discussed the implementation process.

The quantitative and mixed method studies assessed clinical reasoning skills [33,34,41,44,49], knowledge acquisition by performance [9,12,[35], [36], [37], [38], [39], [40], [41], [42], [43],47,48,59,53,54,[55], [56], [57]] or self-perceived competence [9,12,44,50,52,58], attitude to learning [36,50,51], satisfaction with virtual simulation [34,35,37,39,47,54,55,58], self-efficacy [35,39,48,55] / self-confidence [33,37,58], ease of use [51,52], specific learning outcome [35,37,40,43,60], stress [39], comfort [54], and cost [37].

Most of the qualitative studies interviewed nursing students, but some interviewed students and faculty [24,26,28,29] and some included other healthcare students [23,26]. The main themes from these studies were: perspective of the characters, interactivity to the experiment, realism, and feedback [23]; what the students learn, how the students learn, and the simulator's contribution to students` learning [32]; personal engagement, contextual and environmental factors, a safe and structured learning environment, and organizational and technical factors [25]; audiovisual authenticity, the authenticity of scenarios, and interactivity [31]; the “wow” experience, authentic experience on collaborative care, ease of learning, and preeminent role of the facilitator [26]; how the virtual patients enabled learning of non-technical skills, learning surrounding the virtual patient encounter, changing the way students perceive practice, and potential limitations to learning [30]; learning, clinical practice, functionality, fidelity and pedagogy [24]; attitudes toward virtual patient training, virtual patient's role in student development, lack of realism, and enhanced features and implementation suggestions [29]; engagement, immersion, confidence, knowledge, and challenges [28]; developing process, promoting safe debriefing spaces, fortifying knowledge, and engaging in reflection [27].

3.3. Different forms for facilitation and debriefing

Table 2 gives an overview of the interventions in the included studies, and in what way students could interact with peers, the software, or the facilitator. In most of the studies, the simulation intervention tested allowed for interaction between the student and a virtual patient, the software, a facilitator, peers, or a combination of two or more of these. The interaction with the software could either be scoring [9,12,37,55,56], warning prompts [28,40] or direct feedback [9,32,34,38,50,60,61]. Some of the interventions also included demonstration videos [35,38], or gave students the opportunity to watch themselves on video [32], to read a log with individual feedback on their performance [12,35,37,53,55,59], or access to direct guidance [43].

Table 2.

Feedback from the different simulations.

Study	Type of intervention (Name of software)	Content of educational intervention	Student interaction in the simulation activity with				Time spent in the simulation activity	Description of debriefing and/or facilitation
			No interaction	Peers	The system	A facilitator
RCT
Blaine et al., 2020	Serious game (LabForGames Warning)	In the game, a nurse was expected to identify clinical deterioration in different clinical situations (four different cases available) and warn the medical team appropriately (using EWS and ISBAR). Three consecutive steps were constructed to reproduce complications of increasing severity and introduce early warning signs. Students solved two cases, either in the game (intervention) or in text paper format (control).			X	X	Two hours of lecture prior to the simulation on clinical deterioration, no mention of simulation time	Automatic feedback from the software after the scenario involving main guidelines and key messages, global and detailed scoring according to the grid. Some of the actions in the case got negative, positive or neutral points. Group debriefing by trained instructor based on reaction, analysis and synthesis phases promoted by positive interaction between students and instructor.
Gu et al., 2017	Virtual simulation (vSIM for Nursing)	All the students completed regularly scheduled learning activities together. In the experimental group, the students received access to ten virtual cases for fundamentals of nursing. One week before the regular lecture the students were asked to complete the designated virtual case that corresponded to the lecture. Before the lecture day, the students handed in their feedback screen shots and reported the time spent on completing the virtual case. At the end of the semester, all ten virtual cases had been completed.			X		Average time spent on each virtual patient case was about 29 min	Clear introduction, specific guidance, and personalized feedback were given immediately after the scenario by the platform to facilitate the students' active learning process. Each patient-care decision during the simulation was tracked and measured.
Haerling, 2018	Virtual simulation (vSim)	Both groups prepared for the simulation by completing an independent, computer-based learning module on COPD exacerbation. Then, the control group performed live, facilitated manikin-based simulation in groups, and the intervention group performed virtual simulation independently. The same patient problem (COPD) was used in both simulation methods.			X		30 min	Participants in the control group interacted with the manikin, their facilitator, and peers and then participated in a facilitated debriefing session based on the plus/delta model. Participants in the intervention group interacted independently with the virtual patient and got computer-generated feedback from the software, including opportunities for improvement, a detailed log of actions during the scenario, and a numeric score. Both groups completed a written reflection that also was a part of the debrief process.
LeFlore et al., 2012	Virtual Pediatric Patients (VPP) and Virtual Pediatric Unit (VPU)	The control group received the traditional 3-h lecture by faculty, and the intervention group participated in the virtual patient trainer experience. Students in both groups were assigned textbook readings related to the topic (respiratory diseases). Students in both groups also received a standard medium- and high-fidelity manikin simulation.			X		Maximum 3 h for each student to finish, most finished in less than two hours	Before the virtual simulation, instructions were given by video and on paper, and technology experts were available for computer/software issues. Feedback from the system was given in the form of different coloured smiley faces on the screen: green indicated appropriate performance, yellow indicated caution and reassess actions, red indicated incorrect or delayed actions. If the face turned red, a virtual nurse would appear and coach the student to the next step.
Liaw et al., 2020	CREATIVE	Students simulated in teams, and each team was facilitated by a simulation-trained faculty member. The computer based virtual reality used avatars in a 3D virtual hospital environment. Instructions were given on how to navigate between the tutorial room and the ward setting, how talk among themselves using headsets, and how to perform assessments on the avatar.		X	X	X	3-h team training before the simulation and then 20 min simulation	Each scenario was followed by a debriefing facilitated by a facilitator using a guide focusing on communication skills and lasting for 30 min.
Padihla et al., 2019	Body Interact	Both groups received a 45-min lecture to activate knowledge and develop clinical reasoning skills. The intervention group performed a simulation using a clinical virtual simulator that presents virtual patients backed up by a physiological algorithm that recreates a dynamic health condition that responds to user interventions. The control group used the same case-based learning approach using a low-fidelity simulation and a realistic environment.			X	X	45 min laboratory class	The clinical scenario was initiated by a briefing, then the user could interact with the virtual patient. Immediately after the simulation a differential diagnosis interface was presented. The simulator provided a debriefing tool which included a simulation report, a simulation timeline, and a performance report.
Verkuyl et al., 2018	Virtual Gaming Simulation (VGS)	After a virtual gaming simulation, students were divided into three groups to test different debriefing methods; 1) In-person debrief (facilitated in-person debrief during scheduled class, 2) Self-debrief (provided debrief questions via LMS and wrote down their reflections without facilitator), or 3) Synchronous debrief (using Zoom to simultaneously communicate with each other and the facilitator.			X	X	30–60 min to play	Feedback from the system informed students of their decisions at each decision point. Debriefing in groups (max 10 students) after the simulation – within 72 h. Faculty were experts in debriefing and had completed debriefing workshop prior to the student debrief. All the debriefers followed the same script and questions.
Mixed methods
Choi et al., 2020	ComEd	The program consisted of a brief patient medical history video, an interactive clinical performance with virtual patients, and a debriefing session. The control group watched an educational video on the same topic as the virtual patient cases (mental health issues and communication skills).			X		Varied between students, approximately 40–50 min	The intervention included a debriefing session with model videos demonstrating therapeutic communication skills, student review of their own clinical performance, and tailored feedback on each performance. There is no mention of debriefing for the control group.
Donovan et al., 2018	vSim for Nursing (Laerdal)	A computer-based simulation program providing medical-surgical simulation scenarios that creates an active learner-centric environment was introduced to the students prior to their first simulated lab experience. Ten cases were available. Students began the first simulation two weeks after the orientation, and eight weeks elapsed between the first and the last simulation.			X		15–30 min per scenario	Debriefing with a faculty member occurred immediately after the lab simulation experience and included a discussion of student reactions and content learned during the simulation.
Flo et al., 2021	BodyInteract™	Students interacted with the virtual patient through dialogues (by choosing among specific questions), by monitoring different physiological parameters, by performing a physical examination and by initiating nursing interventions. The students were divided in groups. Three students in each group were appointed by the faculty to actively take part in the simulation. Students did not know beforehand who would be chosen to ensure that all students who participated were prepared. The rest of the students observed the simulation and could contribute with questions or input through a chat function.		X	X	X	90 min	Each patient case started with a short briefing and ended with a debriefing session. Immediately after the simulation a differential diagnosis interface was presented. The simulator provided a debriefing tool which included a simulation report, a simulation timeline, and a performance report. The report gave the students feedback on their level of knowledge and the competencies they had achieved. In the debriefing, the students reflected upon their assessments and actions coached by the facilitator who also encouraged them to consider other possible actions that could have benefited the situation. Two experienced faculty members participated: one navigating the software and the dialogue with the students, the other helping stimulate the discussions.
Foronda et al., 2016	V-CAEST	Students could perform a variety of interventions, such as assessing mental status, taking a pulse, and auscultating breath sounds. Once a triage determination was made, students would attempt to select the appropriate colour on the triage tag. Multiple participants could participate at the same time and communicate with each other, providing opportunities for collaborative problem solving, peer feedback and constructive discussions.		X	X		1 h and 15 min	15 min pre-simulation orientation by a facilitator. If a student selected the wrong colour, the program would not allow them to progress. A 15 min debriefing session occurred to elicit students' reflections about the virtual simulation experience. The facilitator typed field notes during the debriefing sessions to capture the students' comments.
Lee et al., 2021	Computer simulation-based, interactive, communication education (ComEd)	The ComEd program consists of a patient medical history video, an interactive clinical performance with two virtual patients (a depressed and a psychotic patient), and a debriefing session. The user first watches the patient's medical history video and starts interacting with a virtual patient in various clinical situations. During the interaction, the user chooses a response they believe is appropriate for the situation among multiple-choice questions and records it with their voice or provides an appropriate response to the open-ended questions.			X	X	40–50 min	The debriefing session provides tailored feedback based on the answers given by the program user. During debriefing, the user watches model videos demonstrating therapeutic communication skills and reviews their virtual clinical performance. The program proceeds if the user chooses an appropriate response using therapeutic communication skills (a total of 13 nurse–patient interactions for each scenario). The program is terminated if the user chooses an inappropriate response using non-therapeutic communication skills.
Liaw et al., 2014	e-RAPIDS	A virtual patient simulator with demonstration of ABCDE and SBAR. The learner enters the virtual ward, receives a handover report, and meets the virtual patient. The learner could choose to participate in any scenarios, receive handover and manage patient clinical deterioration. 5 scenarios with acute medical conditions			X		2 h	Immediate feedback, including information and physiological changes, was programmed into the system to respond to the student's actions. The feedback was given by a virtual nurse verbally and through a text display and a debriefing screen when completed. Feedback was provided by the software in the form of 1) 5 debriefing questions, 2) an evaluation tool and 3) a performance score.
Liaw et al., 2019	3D-VE	The software program supports multi-user real-time interactions in a virtual hospital environment, including an intensive care unit, a general acute ward, and a community care ward. Tutorial and family conference rooms were built within each of these areas. The scenario focus was an elderly man who was admitted for right knee replacement. The intervention was implemented over three days among six healthcare teams in a computer lab and the students were supported by two facilitators and standardized patients. The student also received asynchronous online video describing the use of communication tools and team care models.		X	X	X	Not specified	The participants were able to communicate with each other with headsets, only one speaker at time by activating the speaker button. The facilitator could move freely inside the hospital and teleport from one place to another. The facilitator could also interject or pause any ongoing conversation and adjust the physiological parameters and responses of the virtual patient. After an interprofessional round in the patient room the students returned to the tutorial room for a debriefing session. After a break the students received an online video instruction on the discharge team before they went again into the patient room to deal with discharge and caregiving issues.
Redmond et al., 2020	Virtual Patient	The simulated patient was played by an actor and the students watched a video of that person. Guided by a linear pathway design, students acquired data sequentially to parallel real clinical experiences. Students were presented with some clinical information and could choose from a designated list of options.			X		Not described	Feedback on correct, partially correct and incorrect actions was provided by the system. The feedback was continuous and immediate. At some points, students were asked to reflect on their assessment, to propose hypotheses and to provide summary statements or rationales for their own clinical decisions.
Rim & Shin, 2021	Multi-User Virtual Environment (MUVE) including Second-Life and Unity 3D	The MUVEs comprised five scenarios. Virtual spaces included pediatric outpatient department, pediatric ward, and neonatal intensive care unit settings. The learner was able to communicate with characters such as doctors, patients, caregivers, and laboratory staff appearing in the scenario. For each scenario, learners had two chances in practice mode and one in test mode. The simulation training was divided into groups of six to eight students each. The students attended five sessions each for two weeks.		X	X	X	2.5 h per scenario	Colleagues and professors left feedback on the learner's nursing activities based on a clinical judgment rubric while the learner was participating in the simulation in test mode. The program operation consisted of pre-exploration, pre-briefing, simulation running, debriefing using self-reflection with peer feedback, and group reflection. Debriefing took place on a virtual conference system as well as offline including reflective debriefing and guided structured reflections both individually and in groups.
Tjoflåt et al., 2018	vSim for Nursing (Laerdal)	A web-based simulation platform linked to the curriculum to simulate clinical scenarios, including fully integrated learning resources. The students worked in pairs to allow for discussion.		X (Worked in pairs)		X	2 h session	A faculty member and a clinical tutor guided the students. The students received real-time feedback on what they did.
Verkuyl et al., 2019	Virtual Gaming Simulation (VGS)	The game was constructed using video clips of standardized patients acting their assigned role in typical clinical situations. Students collected data, made intervention decisions and received immediate feedback. Three debriefing methods were tested: 1) Self-debrief, 2) Self-debrief followed by small group debrief sitting in a circle, and 3) Self-debrief followed by large group debrief in a traditional classroom.		X	X	X	1 h	Students received formative and summative feedback in a summary report on the consequences of their decisions. Faculty leading the debriefing were experts in debriefing. All the debriefers followed the same script and questions.
Non-randomized
Atthill et al., 2021	Virtual simulation game (VSG)	Students were randomized to either an asynchronous or face-to-face debriefing strategy. Students in the face-to-face debriefing strategy completed pre-simulation activities before attending the in-person class, where they first engaged in debriefing about the virtual simulation games (VSG) before beginning the live simulation. Participants in the asynchronous debriefing strategy completed the pre-simulation activities including the VSG and a self-reflection within 12 h, followed by an asynchronous debrief which occurred 48 h after the VSG but before the in-person class.			X		Not specified	The debriefing guidelines were structured according to the INACSL 3D model of debriefing. The asynchronous debrief was accessed by students through their learning management software and required students to respond to a series of group discussion posts related to the VSG. The nursing faculty was responsible for posting the initial discussion questions and responding to students' posts during the asynchronous debrief.
Bogossian et al., 2015	FIRST2ACT™	Final year nursing students could complete the FIRST2ACT program at their own pace and convenience. The program consisted of three stages: pre–e-simulation, the e-simulation intervention and post e-simulation (all electronically). In the pre–e-simulation the students could access a power point presentation on managing a deteriorating patient. The e-simulation consisted of three scenarios, each commenced with a video handover. In the virtual patient room, participants were able to click selected actions and respond to cues in the environment. They could call for assistance from other team members.			X		Varied from 27 min to 1604 min to complete 3 scenarios	On completion of each scenario, the participants were awarded a numeric score for performance and provided with general formative feedback incorporating patient diagnosis and best practice management. When finished with the e-simulation component, the participants received a generic debriefing of the experience and were provided with links to resources for further self-directed learning.
Campell et al., 2021	The Virtual Dementia Tour (VDT)	Students from different courses were placed into teams of four during the sessions. The VDT® experience began with a 15-min team pre-briefing session where the students were orientated to the space and equipment, the simulation objectives were reviewed, and an environment of trust and respect was established. Then each student participated individually for approximately 10 min. During the experience, each student was provided with equipment that altered their vision, hearing and touch. Prior to entering the patient room, students were given five tasks to try to complete during their 10 min in the simulated environment.		X	X		45 min	Trained facilitators guided participants and one facilitator participated as an observer, recording participants' actions and behaviours. While the students were in the patient room, a faculty facilitator was in the corner of the room observing students for any notable behaviours. Immediately after all four students finished, a 30-min team debriefing session facilitated by faculty using the Plus/Delta debriefing technique was held. The Plus/Delta strategy is a straightforward method for debriefing that facilitates discussion focused on describing the successful actions and results (plus) and the areas needing improvement (delta) during the VDT experience.
Dubovi, 2018	SimNurse	A computer-based simulation platform integrating a variety of educational e-learning tools designed to provide multiple clinical online experiences (case study scenarios with virtual patients, games, virtual mentoring, self-assessment tools, 3D visualization, interactive videos, digital-dynamic tools, and biochemical models for discovery learning. Simple-2-complex or productive failure approaches were tested.	Not stated	Not stated	Not stated	Not stated	Not stated	The two approaches tested provided different feedback. Due to the complexity of clinical problems in the productive failure approach, support such as reflection prompts, question prompts, content support, and opportunity for knowledge assembling was provided.
Hanson et al., 2020	3D CAVE2	An immersive learning experience with an artefact that visualizes the effect of the autonomic nervous system on heart rate. There were two different visualization modes: CAVE2 facility or mobile handheld device with stereoscopic lenses attached to it.	X				Not specified	Not specified.
Heinrich et al., 2012	MicroSIMR	MicroSIM provided complex patient cases and was integrated into the class content on two different days, one week apart. The software could be used on an individual basis or in a group activity in the classroom environment. The scenarios begun in the emergency department and all nursing interventions were recorded in the medical record and could be viewed at all times. The simulation was projected onto a large screen at the front of the class. At the end of the simulation the students were asked to transfer the patient either to a critical care setting, a medical unit or home.		X	X	X instructor led activity	Not specified	The student group needed to agree on the appropriate nursing care and the faculty acted as a resource as needed. On completion of the patient scenario, the debriefing process started. An overall score was reported, and then a detailed step-by-step analysis of the students' performance was provided. The care provided was compared with accepted standards. Correct interventions were listed, and incorrect interventions were also identified (with rationale). Students' response system was used during each simulation to validate participation under the virtual patient scenario.
Kang et al., 2020	vSim for Nursing™	Students took part in virtual simulation scenarios in groups. The instructor explained the scenario (15 min) and then conducted a pre-simulation quiz in groups (30 min). The instructor immediately showed students the answers to facilitate their learning. Students entered the simulation room with the instructor. After the simulation, the students were informed of the percentage of nursing interventions performed with detailed information and could go back to the simulation room to finish incomplete interventions. The instructor provided 10 post-simulation quizzes.		X (collaborating in groups)	X	X	30 min	Students self-studied for the post-simulation quiz and instructors reviewed the quiz results. The instructors debriefed the simulation with a class of 14 to 21 students who experienced the same scenario. Debriefing was based on the “debriefing overview” provided by the program. Debriefing questions like “On what care practices do you think did well in this scenario?” were the starting point for students to discuss their ideas freely, and after sufficient student discussion, the instructor summarized students' ideas.
Liu, 2021	vSim for Nursing Mental Health	The students were required to complete one virtual simulation scenario every week on five specific mental disorders as individual homework before receiving theoretical education on each disorder. Before launching the actual simulation, students were assigned to complete the suggested readings concerning the scenario and take a pre-simulation quiz to assess their knowledge.	X				Each simulation timed out in 30 min	Students received real-time feedback on their performance, and post-quiz remediation links were provided for immediate access to remediation resources. The students could repeat the simulation multiple times to reach a satisfactory score required by the course (80% on each case). No debriefing sessions were held by faculty.
O'Flaherty & Costabile, 2020	Desktop simulation about the hemolytic disease of the newborn (HDN)	All students attended an online course covering key topics related to hemolytic disease of the newborn (HDN). Students then watched and participated in an interactive simulation and then answered several questions built into the simulation on the covered content. Students received instant feedback on their answers. Students who did not achieve a passing grade were given a second opportunity to watch the simulation and resubmit their answers.			X	X	Not specified. Students were allocated two weeks to complete the three phases	At the end of the simulation, as part of a post simulation debriefing, students tested their understanding of HDN by answering 12 MCQs with immediate feedback provided for any incorrect choices. An informal, virtual debriefing at the end of the simulation which included several guided reflection questions where students were invited to describe their simulation experience, comment on how the simulation could be improved, and leave comments on anything else they would like to discuss.
Roh & Kim, 2014	MicroSim In-Hospital Self-Directed Learning System	All students performed a manikin-based simulation. The intervention group also completed a computer-based simulation as self-directed resuscitation training before a simulation testing session.			X		Not specified	A detailed evaluation log of students' performance was provided by the MicroSim program, pointing out participants' “correct” and “incorrect” actions. There was no debrief after the virtual simulation, but one was held after the manikin-based simulation.
Sapiano et al., 2018	FIRST2ACTweb	The students participated in three scenarios involving deteriorating patients within a four-week period using their own computers. The simulation started with a brief virtual handover by faculty, and then the patient (an actor) explained how he was feeling (prerecorded video clips). The student clicked buttons on the screen to indicate the most appropriate actions.			X		Max 8 min to complete each scenario	Computer-generated performance feedback was provided at the end of each scenario. A weighted score was applied for correctness of each action chosen. A score out of a possible total of 30 points for each scenario was provided to each student together with written feedback. A score under 10 was classified as poor, scores above 20 constituted a distinction and scores in between were considered good.
Tschannen et al., 2012	Second Life	The students participated in three virtual simulation sessions in Second Life. The students got handouts explaining the basic skills needed to use the software. The focus of the simulations for this study included: (1) communication/ conflict management, (2) priority setting, and (3) problem solving related to a patient safety issue. Between 12 and 15 students participated in each group in the simulation. Students had the opportunity to practice for 5 min before the scenario started.		X	X	X	The total time was 35 min (15–20 min for simulation and 15 min for debriefing)	The students who were active met the facilitator in the virtual conference room and were pre-briefed there. A debriefing was led by a faculty member at the simulation center. All the students participated in the debriefing session, which included a discussion of the scenario (e.g. ‘What went well?’) and ways to improve performance in the future (e.g. ‘How might you have done this differently?’).
Weston & Zauche, 2021	i-Human	i-Human is a virtual simulation platform with interactive medical patient encounters. Each case includes an animated avatar in which students take patient history, perform physical assessment, identify health problems, and prioritize interventions. The weekly virtual simulation included a 90-min pre-brief.			X	x	4 h per case (total 35 h over 5 weeks including pre- and debrief)	i-Human tracked each decision that students made and provided them with immediate feedback about their decision making. Performance reports were available to faculty for use in debrief. After each session, students met online with their clinical faculty for a one-hour debrief. Debrief guidelines were followed using the recommended best practice standards from INACSL
Qualitative methods
Buijs-Spanjers et al., 2020	The Delirium Experience	The Delirium Experience is a serious game that makes use of video simulation. The game represents four working days of a healthcare professional. The game includes the narratives of an older patient undergoing hip surgery and a healthcare professional who must provide care to this patient. After surgery, the patient experiences delirious episodes, showing the mixed subtype of delirium. The episodes differ in severity depending on the care provided. All the player's actions will change the narrative.			X		20 min to complete the game once	After each play day of the game, players receive tailored feedback on how their care can be improved. In addition, players receive feedback through the patient's responses to their actions and the impact of their actions on the delirium symptoms. At the end of the game, final written feedback on all chosen care options is provided within the game.
Johannesson et al., 2013	UrecathVision	A portable virtual reality task trainer with haptic properties for providing training in urethral catheterization. The first module explained procedures using multimedia techniques. When students inserted the catheter, they could follow their performance on the computer screen and anatomical features were seen as anatomic cross-section features. Resistance could be felt.		X	X	After the simulation in the debriefing session (SRC method)	Sessions of 15–20 min	The simulation was videotaped, and stimulated recall was used following interviews. Students worked in pairs and the assisting student was a discussion partner.
Johnsen et al., 2021	Hands-on simulation with simulated patients and a serious video-based game	Over the course of two days, 26 nursing students were exposed to five different patient scenarios (Day 1: three hands-on scenarios, Day 2: two computer-based serious game (SG) scenarios). The scenarios concerned a COPD patient with a noninfectious deterioration and another with an infectious deterioration. Actors were provided with scripts but were encouraged to act as themselves. Quiz-based questions and tasks were presented during each scenario (interactive design).			X	X	60–120 min	All simulation sessions involved participant briefing, simulation, and a final debriefing. The facilitating teacher was available the entire time the SG was being played. Students received feedback on incorrect and correct answers, including a demonstration by the RN of the proper care. Plenary debriefing was held with all students in an auditorium to facilitate group discussion of answers to the quiz-based tasks.
Koivisto et al., 2017	CareMe	The 3D simulation game with modular, customizable, graphically pleasing and platform independent intervention included two gaming sessions: both a single player and multiplayer patient scenario.		X In the multiplayer scenarios	X	X	10–20 min to solve the cases	The debriefing focused on the learning aspect of the game in general without a specific focus on any learning outcomes.
Liaw et al., 2021	Three-dimensional virtual world (3DVW)	Using the Unity 5 games engine, a virtual hospital environment, avatar roles and head-up displays for different healthcare professions, facilitators and simulated patients were developed in the 3DVW to support multi-user real-time interaction. The initiation of the simulation involved gathering each interprofessional team in a virtual tutorial room to introduce the team members and receive a briefing from the facilitator on the learning objectives and activities. Students were then provided with an electronic health record of an 80-year-old patient who was admitted for a right knee replacement. The team then navigated to the ward setting to participate in two scenarios and an interprofessional family conference with the patient and his daughter.		X	X	X	120 min	Each team was supported by a simulated patient who was trained to act as the role of the patient's avatar. Each scenario ended with a debriefing session for the team to reflect on their experiences.
Peddle et al., 2019	VSPR (virtual simulated patient recourse) E-learning modules and branching narrative VP	Students engaged in the simulation as themselves. The narrative VP used a “choose your own adventure game” approach with short video vignettes depicting a patient's story over time. The simulation progressed when students selected from two choices appearing on the screen following the video vignette. A decision tree using a branching algorithm determined the next video in the simulation sequence. Seven VP scenarios are available in the VSPR.		X	X		Not specified	Consequences of decisions resulted in positive or negative feedback on the patient's outcome, providing intrinsic feedback to students on their actions. Learning was supported through small group activities, by group discussions at the end of each vignette and by a concluding facilitator-guided debriefing.
Saunder & Berridge, 2015	Shareville	Students started a three-hour classroom session with a group discussion of their personal and professional experience of people with learning disability. Students were encouraged to share positive and negative experiences and to explore their own values and attitudes. They were then fully briefed on how to use Shareville. They worked at their own pace, experiencing the simulated reality scenarios and collating information and reflections in an electronic workbook. Group size varied from 13 to 23 students.		X	X	X	Approx. 60 min	To optimize participation, students used Shareville in classroom sessions facilitated by teaching staff, blending both independent and group work. Each session was led by faculty – one with expertise in learning disability and the other with expertise in adult, child, or mental health nursing, as appropriate. Students discussed their experience in small groups and identified key considerations that they would need to incorporate into practice to enhance the quality of care delivered to people with learning disabilities.
Shorey et al., 2020	Virtual Counseling Application using Artificial Intelligence (VCAAI)	The VCAAI is visualized as a three-dimensional (3D) avatar. Ninety-three undergraduate nursing students received communication training using Virtual Patient (VP) simulations before their clinical postings for each semester (two semesters per year) in educational years two and three. The VP trainings took place once per semester at a computer lab in the university, where different case scenarios were presented each time.			X		Not specified	No debriefing was described.
Singleton et al., 2021	Virtual reality (VR) diabetes simulation	Students played a nurse avatar who stayed within the private room. A virtual handover gave the nursing student knowledge about the patient's condition, current medication, and observations and provided extensive details relating to the patient's history and condition. The student had to make a safe clinical decision about how to react and communicate with the patient. Students were presented with clinical decisions in multiple-choice questions via pop-up text boxes.			X		60 min	If the student made unsafe decisions, instant feedback was given via text boxes asking them to reconsider quickly because the patient was deteriorating. The patient was programmed to look unwell if the student did not correct the patient's blood glucose quickly. No debrief was described. Each clinical decision was followed up with instant feedback so that the student could learn and improve. The student could complete the simulation multiple times. The lecturer was sent data analytics about each student's performance.
Verkuyl et al., 2020	Virtual Gaming Simulation (VGS)	The game is a branching scenario where the user has options (i.e., clinical decision-making based on the simulation) and can control the pace of play. Students assumed the role of the nurse during the simulation and were required to choose appropriate questions, respond to specific assessments and provide health teaching to the client. Three debriefing methods were tested, 1) self-debrief, 2) self-debrief followed by small group debrief (max 12 students), and 3) self-debrief followed by a large group debrief (max 30 students).		Interaction in the debriefing phase	X		Max 60 min	Feedback on the user's decisions was provided throughout the experience along with a final score and an autogenerated summary report of each decision they made in relation to the correct response. The self-debrief was finished within one hour of completing the simulation and then submitted. The group debriefing occurred within one to two weeks from completing the simulation. All the facilitators had completed comprehensive simulation courses and two hours of training. They used the same facilitation guide.

Feedback from the virtual patient, or from the software, was one feedback that several students in the included studies emphasized as useful [30,50]. In fact, students reported that they deliberately chose the wrong action to learn from it [24,30]. Several studies pointed out the importance of debriefing, also referred to as feedback or reflective feedback, for learning [31,32]. One study indicated that feedback provided at the end was necessary, but not sufficient. For increased learning, feedback must also be provided during the simulation [31]. Other studies also indicated that reflections in the form of self-debrief should be performed just after the simulation, while the memory was fresh [27]. In two of the studies [48,59], the main aim was to investigate different models of debriefing. These studies concluded that self-debrief prepared the students better for structured debriefing in groups. Students expressed that they learned more when they were given time to reflect prior to the debriefing session.

4. Discussion and conclusion

4.1. Discussion

This review investigated the existing literature on facilitation and debriefing in virtual simulation in undergraduate nursing education and comprised 41 studies from 15 different countries. Although several more studies on virtual simulation have been conducted, the specific aim for this scoping review was those with a clear description of direct feedback to the students, as this focus is lacking in the literature. In the included studies there were large variations regarding the emphasis they placed on the importance of reflective dialogue for student learning. Most of the studies described debriefing, which is acknowledged as a significant factor for learning in simulation-based training. In addition, some studies also described pre-briefing or feedback from the software, peers, or a facilitator during simulation. However, there was little reflection from the authors about the impact of having a human in the loop, meaning the reflective dialogue with a facilitator that potentially stimulated the learning. This might imply that the design of virtual simulation sessions fails to recognize that the key to optimal technology use lies in the feedback and is in line with virtual simulation in other fields [62]. In the qualitative studies, several of the students talk about guided reflection and how it promoted understanding with immediate feedback. Previous studies have also shown that debriefing in various forms increases learning [63].

This review shows that feedback that enhances learning not only occurs in the debriefing phase after the simulation, but in fact during the entire virtual simulation session. This aspect has not been emphasized before in the context of virtual simulation. In the virtual simulation software it is possible to stop and pause as many times as needed in the virtual patient scenario. This function creates the space to extend the debriefing to an ongoing discussion and in-depth reflections between participating students and the facilitator. The benefits of this kind of reflections and the discussions are to explore cognitive thinking processes and reasoning to strengthen student understanding of professional knowledge [10]. This can contribute to strengthening the development of clinical reasoning and clinical decision-making skills. Exploration of professional knowledge and cognitive processes appears to be essential to better understand virtual patients' clinical cues to be capable of acting accordingly and with action that promote patient safety [64]. This way of facilitation for reflective debriefing throughout the virtual simulation can also stimulates critical thinking and can increase awareness about own clinical practices [5]. Furthermore, this kind of facilitation exemplifies the concept human in the loop and how optimize the possibilities technology.

The majority of the quantitative studies used a pretest-posttest design or included a control group that received less educational content than the intervention group. The students improved their knowledge or performance after simulation. Only a few studies compared traditional teaching to virtual simulation. None of these studies found significant improvement in the intervention group over the control group. In comparison, half of the studies included in a recent review about virtual simulation from medical education showed significant improvement in students' skills [62]. This could imply that virtual simulation in nursing education should improve the educational content, i.e., by adjusting the curriculum design or to include sufficient instructional strategies. The implementation of virtual simulation or other technology enhanced devices need to be anchored in solid pedagogical consideration or assessment to argue for the benefit of the technology to obtain the intended learning outcomes. The description of the educational content in the studies included varied (Table 2) from no mentioning of the pedagogical design to a detailed description of the content, including training of facilitators and technological support for students. Several studies in the current review found that students enjoyed virtual simulation. None of the studies in this review applied a longitudinal design to explore learning retention over time, nor did the included studies compare the overall competence among the students participating in the virtual simulation and those that did not at the point of graduation. This is a gap in the literature, and it would be interesting to investigate further. A recent review found that simulation positively impacts nursing performance, patient safety, and patient outcomes [64]. The fact that virtual simulation in this study achieves similar outcomes as traditional simulation is a positive and interesting finding. Most of the simulation-based training in nursing education today are based on traditional approach rather than virtual simulation. This is despite the fact that students can struggle to engage with and to stay motivated during traditional methods, even when active learning elements are used [65]. The different approaches for simulation-based learning should be viewed as complementary activities with different strengths and limitations instead of choosing one over the other.

All the qualitative studies have findings that can be summarized in the following themes: safe space, feedback and guided reflection, perceived learning, and technical functionality. The students explained how they could learn from the safe spaces provided by the virtual simulations by exploration [27], repetition [29,32], and by making mistakes [[23], [24], [25], [26],30,31]. The safe space indicated a sense of competence [24,30]. Some students found the virtual patient realistic [23,26,28], whereas others found it to be less realistic [29,31]. Several students highlighted the crucial role of the facilitator related to learning [25,26]. They also found feedback during the simulation to stimulate learning and increase confidence [23,25,28,30,32]. In one study that did not provide feedback from the virtual patient during gameplay, this was reported as a feature students felt lacking [31]. Some students found the virtual simulation helped their learning by being so visual [28], particularly students with dyslexia [24].

Except for one study that reported on comfort [54], none of the included studies reported on cyber sickness. This has been repeatedly mentioned as a side effect of virtual reality (VR). As this is particularly common among young female, it is a bit surprising given that the majority of nursing students are young and female. This can be because we included other simulations than VR, but also because we excluded pilot studies. One of the pilot studies we found in the screening process, that was not included in this review, did in fact investigate cybersickness [66]. It is possible that the side effects are controlled for when piloting the VR.

One limitation of this scoping review is the restriction of papers to be written in English. This could mean that studies in other languages could have been missed. Still, the included papers were from almost all continents. As with all reviews, there is a possibility that relevant studies might have been missed despite systematic searches and screening of all retrieved studies in pairs.

4.2. Innovation

The innovation in this study is the focus on the value of having a human in the loop during virtual simulation and on reflective feedback throughout the virtual simulation session rather than restricted to the debriefing at the end of the simulation. The purpose of introducing virtual simulation is not to fully replace traditional simulation, but rather to be a complementary supplement. To succeed, there is a need to better understand how to facilitate for reflection by pausing the virtual simulation or providing structured feedback.

Although virtual simulation might not be innovative in 2023, the standard in most nursing educations today seems to be traditional high-fidelity simulation. Therefore, this review contributes to an evidence-based overview of the findings reported in the literature regarding the use of virtual simulation, the variety of software used, and how the facilitation can and should be different in virtual simulation compared to traditional simulation.

4.3. Conclusions

This scoping review described simulation-based simulation with a variety of software in nursing education. Virtual simulation produced similar effects on learning as traditional simulation. Virtual simulation was well accepted by the students, and often preferred over traditional simulation. This review emphasizes the importance of reflective feedback before, during, and after the simulation. The use of reflective feedback led by the facilitator is important in the virtual simulation, i.e., by pausing the virtual case to reflect. Virtual simulation also provides opportunities to give feedback from the software, such as cues or direct feedback.

Virtual simulation can be used as a complementary activity to traditional simulation to facilitate student engagement and learning. Future research should focus on exploring the long-term impact of the virtual simulation on knowledge and skills retention, as none of the included studies are longitudinal. The pedagogical structure, i.e. different approach to facilitation and reflective debriefing during virtual simulation sessions can influence whether the students attain the intended learning outcomes. Further, a combination of reflective feedback during and after the simulation seems to increase learning.

Funding sources

No external funding.

Declaration of Competing Interest

The study has received no external funding.

None of the authors have any conflict of interest to declare.

Appendix A. Supplementary data

Supplementary material