Virtual reality vs. Manikin based training on emergency life saving basic rescue skills: a summative evaluation

Abstract

Ongoing professional development constitutes an essential and necessary process for healthcare practitioners, thereby ensuring their proficiency and relevance within the field. This quantitative summative program evaluation compared the effectiveness of Virtual Reality (VR) technology versus traditional manikin-based simulation in teaching basic rescue skills to first responders in Saudi Arabia. The study, conducted through the Academy of Sciences (AOS), addressed the growing need for innovative, engaging, and efficient healthcare education methods, particularly in the context of the post-COVID-19 landscape. The research evaluated the “Mongeth” VR program against conventional manikin simulation training, focusing on key rescue skills including CPR, AED usage, stop the bleed, and choking management. Participants (N = 98) were divided into VR (n = 58) and Manikin (n = 40) groups. Performance was assessed using standardized checklists for various rescue skills. Results demonstrated significantly higher performance scores in the VR group for AED usage (p < 0.001), adult choking management (p = 0.009), infant choking management (p = 0.009), and CPR (p < 0.001). Only the Stop the Bleed assessment showed no significant difference between groups (p = 0.096). Through the lens of Social Cognitive Theory, the study revealed that VR training enhanced learner engagement and self-efficacy while reducing time investment. The findings suggest that VR technology provides a more time-efficient and engaging alternative to traditional manikin-based training, with significant improvements in skills like CPR and choking management, particularly beneficial for busy healthcare professionals and community members requiring basic rescue skills certification. This research contributes to the growing body of evidence supporting the integration of VR technology in healthcare education and has implications for improving population health through more accessible and effective training methodologies.

Section 1 - Background and context

Ongoing professional development is essential healthcare practitioners to maintain proficiency and relevance in the field. The implementation of advanced educational standards in healthcare substantially bolsters the quality of patient care and the associated health outcomes [1, 2]. The quality of healthcare education relies on various elements, including the inherent excellence of the educational material, alongside the critical need for modern, engaging, accessible, and cost-effective technologies and teaching methodologies that enhance effective learning experiences [3]. The ongoing education of healthcare professionals enhances their skill set, and significantly broadens their knowledge base and improves their overall performance in clinical settings. The application of diverse pedagogical approaches enhances information retention among learners and stimulates their engagement in ongoing educational pursuits. In the contemporary technological era, healthcare education institutions are presented with diverse opportunities to enhance the professional growth of healthcare practitioners via the strategic integration of technological innovations that facilitate the provision of superior, engaging, and accessible educational materials.

Rational for the Study

The COVID-19 pandemic has significantly transformed how healthcare professionals earn their necessary continuing professional education credits, compelling them to adapt to new methods. As the landscape of in-person training experienced substantial restrictions and limitations, healthcare providers were faced with the urgent necessity to explore alternative avenues for acquiring their educational credits amidst the backdrop of an accelerating rate of innovations in the healthcare sector and the culmination of vital updates that have been developing over the past three years [4]. The intricate balancing act of finding adequate time in their demanding schedules to fulfill the obligations associated with mandatory training presented a challenge for healthcare practitioners, particularly in light of the imperative to uphold their basic skills certifications, which are crucial for the continued validity of their professional licensures. Consequently, these multifaceted challenges prompted a significant transition in the training methodologies employed, leading to a pronounced shift towards self-learning approaches within the healthcare education landscape.

Self-learning, particularly through Virtual Reality (VR), emerged as a crucial solution for healthcare providers to maintain their certifications and addressed population health needs, especially during the COVID-19 pandemic [5, 6]. While VR technology has become increasingly sophisticated yet user-friendly, making it accessible for learners at different levels, its application in training first responders and the general population remains understudied [7, 8]. The focus on population health, which encompasses patients of all ages and conditions [9], has intensified due to global changes, particularly as first responders faced unfamiliar emergencies during quarantine periods. Although healthcare education organizations have the opportunity to bridge knowledge gaps and enhance community safety through VR simulation education, limited research has evaluated the effectiveness of VR modalities compared to traditional manikin simulation for training first responders [10], particularly in terms of benefits, time efficiency, information retention, satisfaction, and performance outcomes.

The accelerated adoption of virtual training modalities during the COVID-19 pandemic (2019–2023) addressed an immediate system vulnerability: the disruption of traditional recertification pathways for healthcare professionals’ basic skills maintenance [4]. Our study emerges from this context not as pandemic-driven research, but as an investigation of solutions for an enduring educational challenge that the pandemic exposed - the critical need for flexible, time-efficient training systems that can sustain provider competency despite systemic shocks [5]. Virtual Reality training represents a maturation of three pre-existing trends in medical education: (1) the shift towards competency-based assessment [6] (2), the integration of simulation technologies [7, 8], and (3) the demand for scalable continuing education solutions [7, 8]. While our research demonstrates VR’s particular utility for maintaining basic rescue skills during periods of constrained in-person training, its potential extends beyond pandemic conditions. The technology’s inherent advantages - standardized content delivery, repeatable scenarios, and objective performance metrics [9] - address chronic challenges in healthcare professional development that predate COVID-19.

Organizational context

In this summative program evaluation, we studied the VR “Mongeth” system compared to the hands-on manikin simulation training for first responders. The word “Mongeth” in Arabic means “Rescuer”. The training modules covered are Cardio-Pulmonary Resuscitation (CPR) Adult, Choking Adult, Choking Pediatric, Automated External Defibrillator (AED) use, and Stop the Bleed. This study will refer to these modules as Basic Rescue Skills (BRS). The Program was presented at the 5th Saudi Health Simulation Conference (2022) and was awarded a prize for further research. This award opened the opportunity for the AOS to collaborate with the Clinical Skills and Simulation Center (CSSC) at Princess Noura Bint Abdulrahman University (PNU) to conduct an in-depth study on the VR Modules (the Program). Because this study’s outputs and outcomes are varied and complex, a Summative Program evaluation methodology is conducted.

The Academy of Sciences (AOS), a small for-profit private healthcare education organization, provides educational programs, events, and consulting services to clinicians and managers across Saudi Arabia and the Middle East and North Africa (MENA) Region. The organization’s commitment to delivering innovative, high-quality continuing education for healthcare providers led to the creation of the Mongeth Program, which aims to teach basic rescue skills to both healthcare providers and community members (Chief Executive Officer, direct meeting, March 20, 2022). Recognizing the uniqueness of this healthcare education innovation and its absence in Saudi Arabia and globally, the Organization’s stakeholders sought to validate and authenticate this initiative, leading AOS to apply for a patent.

The goals achieved through this summative program evaluation were to assess the existing operational Program and validate the reasons for its success and areas for improving its effectiveness in healthcare education. The “Mongeth” Program evaluated is compared with the manikin simulation training on the Basic Rescue Skills with the goals to (a) assess the effectiveness of each of the modalities, (b) Assess the time spent in the teaching and returned demonstration of both modalities, (c) compare the retention of information and improvement of performance in both modalities, and (d) assess the satisfaction with the VR experience.

Problem statement

Despite the widespread use of manikin-based simulations in training basic rescue skills such as Cardiopulmonary Resuscitation (CPR) and Automated External Defibrillator (AED) use, growing evidence highlights their limitations—particularly in learner engagement, perceived realism, and the transfer of skills to real-life scenarios. Manikin simulations often lack the complexity and interactivity required to develop situational awareness and critical decision-making, which are essential for effective emergency response. Studies have shown a decline in performance when trainees transition from manikin-based practice to real clinical settings, underscoring the gap between traditional simulation and practical applicability [11, 12].

These challenges are especially relevant in healthcare systems like that of Saudi Arabia, where there is increasing demand for evidence-based, scalable, and high-impact training methods for first responders. Immersive Virtual Reality (VR) technologies have emerged as promising alternatives, offering interactive environments that may enhance learner engagement, satisfaction, and practical skill acquisition [13].

To explore this potential, this study conducts a summative program evaluation of Mongeth, an innovative VR training platform, comparing it to conventional manikin-based simulation for basic rescue skills training in Saudi Arabia. The evaluation focuses on three key outcomes: training satisfaction, improvement in performance/competency, and training time. By examining these dimensions, the study aims to determine whether immersive VR offers a more effective and engaging training modality—and whether it can bridge the gap between theoretical learning and real-world readiness. The findings aim to inform educators and decision-makers seeking to enhance the effectiveness and scalability of emergency preparedness training.

Purpose statement

This quantitative summative program evaluation aimed to assess the efficiency of the Mongeth VR program compared to the manikin simulation modality training on basic rescue skills. The study’s findings may help develop an understanding of the learners’ preferences and behaviors toward the VR and manikin simulation. The implications for positive social change will include the potential for an improved customer experience and identifying best practices that positively affect the first responders’ performance and quality of care, which will positively impact the population’s health and the community’s safety.

Research question

The below summative evaluation research questions (program’s research question) were used to explore the research problem.

Program’s research question

1. What is the satisfaction in the basic rescue skills training for the participants in the VR versus the manikin simulation modality?

2. What is the percentage of gap improvement in the competences/performance of the basic rescue skills in the VR versus the manikin simulation modality?

3. What is the average time spent in the basic rescues kills training for the participants in the VR versus the manikin simulation modality?

4. What is the level of satisfaction and engagement in the basic rescues kills training for the participants in the VR versus the manikin simulation modality?

Hypotheses

H1: Participants receiving VR training will report significantly more positive training satisfaction compared to those receiving traditional training methods.

H2: Participants completing VR training will demonstrate significantly greater performance improvements in basic rescue skills than those receiving traditional training methods.

H3: VR training will require significantly less time to achieve competency benchmarks in basic rescue skills compared to those receiving traditional training methods.

H4: Participants in the VR training group will report significantly higher satisfaction and engagement levels than those receiving traditional training methods.

Conceptual Framework and Program Model

Conceptual framework

This study employed Social Cognitive Theory (SCT) as its conceptual framework to analyze the complex interplay between virtual reality-based learning and traditional simulation methods in healthcare education. Originally developed by Bandura (1986), SCT provides a comprehensive lens through which to examine how personal factors (such as self-efficacy and prior experience), behavioral elements (including engagement and skill performance), and environmental conditions (like the learning modality and setting) interact to influence learning outcomes (refer to Fig. 1 for the framework’s constructs). This theoretical framework was particularly relevant for evaluating the “Mongeth” VR program as it helped illuminate how the immersive virtual environment might affect learners’ self-efficacy, motivation, and subsequent performance compared to traditional manikin-based training. By applying SCT’s core constructs, we could systematically analyze how different learning modalities impact skill acquisition, retention, and transfer, while also considering the unique psychological and environmental factors that contribute to successful healthcare education outcomes.

Fig. 1.

SCT’s Constructs

Self-efficacy, which constitutes a central element of SCT, can be defined as an individual’s subjective belief in their own capability to achieve success in particular tasks or challenges that they may encounter throughout their educational journey. This construct has a profound impact on motivation levels, as students who possess elevated levels of self-efficacy are demonstrably more inclined to engage in tasks that are challenging in nature, exert considerable effort in their pursuits, and persist resolutely in the face of obstacles and difficulties that may arise during the learning process [14, 15].

The concept of reciprocal determinism, as articulated within SCT, offers a comprehensive explanation for the manner in which self-efficacy interacts dynamically with both environmental and behavioral factors that permeate the educational experience. This notion posits that the beliefs held by students regarding their own capabilities possess the potential to influence not only their learning environment but also the behaviors they exhibit, which, in turn, can have reciprocal effects on their levels of self-efficacy [15, 16].

Another critical construct within the SCT framework is outcome expectancy, which refers to the belief that certain behaviors will inevitably lead to particular outcomes or results. Although this construct has been studied less extensively than self-efficacy, it nonetheless plays an important role in shaping motivation and learning by influencing students’ anticipations regarding their own success [14].

Self-regulation, which represents a crucial process emphasized in SCT, involves a triadic system of self-observation, self-judgment, and self-reaction that students utilize to navigate their learning experiences. This process is closely intertwined with the concept of self-efficacy, as students who exhibit high levels of self-efficacy are typically more likely to employ effective self-regulatory strategies, which in turn lead to improved learning outcomes [17].

While SCT indeed provides a robust and comprehensive framework for understanding the multifaceted relationship that exists between self-efficacy and various learning outcomes, it is essential to recognize that possessing high levels of self-efficacy in isolation does not automatically guarantee success in educational endeavors. A multitude of other factors, including the accessibility of resources, the quality of instruction provided, and the individual variances in cognitive and emotional factors among students, also play crucial and significant roles in influencing the overall educational outcomes that students ultimately achieve [18]. The authors in this study conducted a summative evaluation and analyzed the findings through the lens of the SCT and its constructs.

The logic model

A program logic model (PLM) is a simple graphic and diagrammatical representation of the program’s various elements and shared relationships [19, 20]; they provide a map of how a program should work. The PLM is a visual aid that assisted us in determining how the components of the program under evaluation should flow and how to reach the desired outputs, outcomes, and impacts [3].

The PLM is a pivotal step to articulate and use the framework in the program evaluation. The visual representation of the PLM is in Fig. 2. The situation is the historical background and organizational context. The priorities are the decisions that the research team decided to work on in the study to identify the goals, outputs and outcomes. The inputs of the model address the resources invested in this study. In this study the AOS and PNU teams partnered to conduct the study. The roles and responsibilities of each team were identified and distributed. The competition was analyzed and it was a fact, as mentioned before, that this VR modality and content of healthcare education is unique and a newly established modality with a patent of creation. The VR system is available in multiple units at the AOS, as well as a well-developed Simulation center and appropriate equipment. The AOS team is formed by experts in healthcare education and program evaluation and are leading the study. The PNU’s team are experts in statistical analysis, tool validation, and research, as well as simulation in healthcare education.

Fig. 2.

Virtual reality basic rescue skills program’s logic model

The outputs of the study are immediate activities performed and executed during or immediately after the program evaluation to fulfill the mission and reach the outcomes. The outputs of this study are multiple, and will reach the staff, participants, faculty, and population. Figure 3 is a visual representation of the actions taken in this study, which we explain more in Sect. 3 of the study.

1. Assessment of time per module per student in the VR training module. The Manikin based modality has a fixed average training time that was also monitored and documented.

2. The time motion of the VR modality teaching conducted during this study was utilized for comparison with the manikin simulation modality. It was also used for the accreditation process study for the VR modality.

3. The authors found no similar studies in Saudi Arabia or the Region. This study is the first of its kind in Saudi Arabia and the Region.

4. Train employees and faculties to conduct the research properly and impartially, and make sure the recruitment questionnaire is filled for exclusion criteria, and the informed consent is signed before conducting the study.

5. Develop processes and strategies for the VR modality training, and accreditation process in AOS.

6. Work on incorporating the VR Program in the internationally accredited courses; ATLS, ATCN and PHTLS for Stop the bleed module, and BLS for the other modules.

7. Create a comprehensive curriculum for the basic rescue skills course that will be offered for this study.

Fig. 3.

Data collection process flow for the summative evaluation

The short term outcomes of this study addressed the learning aspect and aimed to improve the participants’ knowledge. The Intermediate outcome addressed the action aspect and aimed to improve the skills (performance) of the participants. The Long term outcome is to have a positive impact on the condition of rescuing skills in the population which will affect and improve the population health. The Long Term Outcome cannot be achieved in the study’s context, however, it should be a project for the AOS on the long term to affect and positively impact population health and their knowledge and skills about basic rescue skills, as part of a corporate social responsibility.

The PLM shows the general assumptions we have about the favorable conditions and the beliefs about the program and how we think it will work. We know that we have no rivals in the VR Program or in the study itself. We are sure that the inputs are available and adequate. The researchers followed the PLM to make sure the Program’s study is successful and we followed what the evidence showed us. The staff of AOS and PNU are engaged and aware of the every update all the way. The staff is available for data collection and analysis.

The contextual factors (internal and external) were addressed and mitigated to minimize their influence on the Program’s outcomes. Involving the team from the start, explaining in parts each phase, and having the input of the researchers on the decisions of the process led to minimizing the resistance. The financial limitation is related to cost of upgrading the VR to be able to extract the reports of performance; we mitigated this cost at the time of the study by doing a manual time motion and taking screenshots of the performance. The challenge of the duration of the study was dependent on the number of participants who would agree to help in the validation process, and then we had to time and schedule the two groups to conduct the study. Another challenge was in finding the participants with the appropriate backgrounds and form 2 groups with similar percentage distribution of demographics and backgrounds. The language challenge was addressed by having the participants decide from the start if they wanted to be trained in Arabic or English, they were provided evaluations in the language of their choice at the end of the training.

Section 2 - Representative Literature Review

Technology-enhanced learning (TEL) within healthcare education represents a complex and multifaceted modality that strategically utilizes a diverse array of digital tools and platforms, with the primary objective of significantly enhancing and enriching the educational experiences of healthcare providers. This innovative approach is of paramount importance in adequately equipping healthcare professionals with the necessary skills and knowledge to effectively navigate and respond to the exigent demands posed by a rapidly evolving digital health ecosystem that is constantly transforming. Several pivotal factors that contribute to the successful implementation of TEL in healthcare education encompass the seamless integration of cutting-edge and innovative technologies, the critical role played by user engagement and acceptance, the indispensable nature of digital literacy, and the imperative need to adapt to the continuously shifting landscape of educational requirements.

The deployment and utilization of advanced technologies such as virtual reality (VR), augmented reality (AR), simulation tools, and mobile applications have emerged as increasingly vital components in the domain of healthcare education, reflecting an ongoing trend towards technological integration. These sophisticated technologies afford experiential, self-directed, and practical learning opportunities that are absolutely essential for the comprehensive training and education of future healthcare professionals [21, 22]. Notably, the incorporation of virtual anatomy platforms, exemplified by tools such as Anatomage, has been strategically integrated into educational curricula to substantially enhance the learning experience; however, the ultimate success of these platforms is contingent upon effectively addressing user concerns regarding their inherent complexity and overall ease of use, as highlighted by Singer et al. [23]. The incorporation of artificial intelligence (AI) tools, such as ChatGPT, is being actively pursued to boost digital literacy and to adeptly adapt to the increasing significance of digital environments within the healthcare sector [24].

The level of user engagement emerges as a critical determinant influencing the successful adoption and implementation of technology-enhanced learning, thereby emphasizing its importance within the healthcare education domain. Factors such as enjoyment derived from the learning experience, perceived usefulness of the technology, and the overall ease of use of the learning platforms play a significant role in shaping healthcare students’ engagement with technology-enhanced learning initiatives [23]. Abdekhoda et al. (2022) delineated that the theoretical frameworks of the Unified Theory of Acceptance and Use of Technology (UTAUT) and the Task-Technology Fit (TTF) models serve to illuminate the vital importance of various elements, including social influences, effort expectancy, and performance expectancy, in facilitating the adoption and effective utilization of e-learning methodologies [25].

In light of the ongoing digital transformation within the healthcare sector, there exists an urgent and pressing necessity for healthcare providers to cultivate and develop essential digital literacy skills, which encompass not only an understanding but also the effective utilization of a wide array of digital tools and platforms that are instrumental in enhancing both learning outcomes and professional practice [24]. Despite the presence of certain concerns, mobile devices offer substantial advantages in clinical practice by granting healthcare professionals access to a plethora of useful applications and resources that can significantly augment their educational and professional capabilities, as discussed by Junk et al. (2020) [26].

The unprecedented circumstances surrounding the COVID-19 pandemic have plainly highlighted the critical necessity for flexible and scalable solutions in technology-enhanced learning, which are essential for maintaining continuity in healthcare education during times of crisis and disruption [27]. The advent of simulation-based learning methodologies, alongside the proliferation of Massive Open Online Courses (MOOCs) and other online educational resources, has notably expanded access to educational opportunities, thereby facilitating the provision of more personalized and learner-centered experiences that cater to individual learning needs [21].

While the implementation of technology-enhanced learning undoubtedly offers a multitude of benefits, it concurrently presents a series of challenges that must be addressed, including the imperative to ensure equitable access to technology, the necessity of addressing privacy and security concerns, and the need to overcome resistance to change that may be encountered among both educators and learners. Moreover, the overall effectiveness of TEL is intrinsically linked to the capacity to seamlessly integrate these advanced technologies into existing educational curricula while also ensuring that adequate support and comprehensive training are provided for both educators and students alike. As the field of healthcare education continues to evolve and adapt to new realities, it is of utmost importance to proactively address these challenges in order to fully harness and realize the transformative potential that technology-enhanced learning has to offer.

Section 3 - Research method and design

Research method

The purpose of this quantitative summative program evaluation was to assess the efficiency of the Mongeth VR program in comparison with the manikin simulation modality training on the basic rescue skills. Quantitative researchers aim to systematically outline the elaborate properties of different variables and meticulously explore the sophisticated correlations that exist between causative elements and their subsequent ramifications. The process of quantitative analysis significantly enhances the ability to generalize the findings derived from an extensive sample population, as evidenced by the scholarly contributions of Ahmad et al. (2019) and Manzoor (2020) [28, 29].The effective integration of a robust and well-structured quantitative methodology substantially elevates the overall value and credibility of the study; the quantitative approach empowers researchers to clarify, regulate, quantify, and maintain a focus on outcomes; nonetheless, it is imperative to acknowledge that the quantitative method is inherently ill-suited for yielding precise answers that pertain to the nuances of human behavior, as articulated by Henson et al. (2020) and Manzoor (2020) [29, 30].

The quantitative methodology was best suited to answer the following summative evaluation research questions:

1. What is the experience in the basic rescues kills training for the participants in the VR versus the manikin simulation modality?

2. What is the percentage of gap improvement in the competences/performance of the basic rescue skills in the VR versus the manikin simulation modality?

3. What is the average time spent in the basic rescues kills training for the participants in the VR versus the manikin simulation modality?

4. What is the level of satisfaction and engagement in the basic rescues kills training for the participants in the VR versus the manikin simulation modality?

Population

This program evaluation targeted first responders from the public/community mainly, and included a percentage of healthcare providers who did not have a BLS or ACLS certification, or recertification within the past 2 years prior to the study’s observations. The age threshold for both groups was 18 year old. All participants were proficient in English and/or Arabic.

For the validation of the checklists, included any instructors from the AOS pool of Life Support Instructors, willing to volunteer and help in the observations; however, the focus was to have 8 different instructors using the checklists to validate their understanding of the tool. For the validation of the VR evaluation Arabic version, asked previous users of the VR to fill the evaluation, or we administered this tool to new VR users, collecting 66 observations for validation of the tool. Details about the validation inclusion will be published in a separate paper.

The served or affected stakeholders are; student clients who are healthcare providers or from the community, attending basic rescue skills events offered by the AOS, selected external faculty contracted by the organization, and employees. The decision-makers in the organization under study are the CEO of the organization, the VR Project Manager, the Program Evaluator, the Director of the Simulation and Life Support, and the contracted instructors who are members of the scientific committee. The decision makers from PNU-CSSC are the assessment unit supervisor and the statistician. Based on the variables at hand and the G-Power, it was determined that the minimum number of observations per modality should be 38. We were able to collect a total of 58 valid observations in the VR modality and 40 valid observations in the Manikin based Modality.

Research design and strategy

Design

To effectively respond to the research inquiries posed in this quantitative study, a specific research framework was employed that featured an evaluative design [31]. As articulated by Saunders et al. (2015) [31], an evaluative research design is primarily concerned with measuring the efficacy or worth of a certain process, intervention, or program, typically within a defined organizational or social milieu. This approach necessitates a systematic examination of the results and repercussions of a program to ascertain whether its goals have been achieved. Frequently, this research design integrates mixed methodologies, leveraging both qualitative and quantitative data to furnish a thorough assessment of the intervention’s effectiveness. Furthermore, evaluative research prioritizes the comprehension of the contextual elements that affect the outcomes, such as participant perceptions or organizational limitations, rendering it particularly advantageous for evaluating practical programs or policies. Its application is prevalent in sectors such as business, healthcare, and education, providing insights to inform decision-making and enhance operational practices.

This type of research can also involve a cyclical process, where findings are used to refine the program or policy being evaluated, allowing for continuous improvement and adaptation to changing conditions. The authors started this study to check if the new VR Mongeth Program served its purpose and compared it to the manikin based modality. The outcomes from this study can serve as input for further program evaluations for this or similar studies. Refer to Fig. 2, for the study’s logic model.

Strategy

This study’s strategies were Experimental and survey. Saunders et al. (2015) discussed the experimental research and survey research strategies in their “Research Onion” model [31]. The experimental strategy is primarily focused on determining cause-and-effect relationships between variables [32, 33]. In this approach, the researcher manipulates an independent variable to observe its effect on a dependent variable, often using control groups and randomization to enhance the reliability of the results. This method is commonly used in highly controlled environments, such as laboratories, to ensure the manipulation of variables is isolated and observable. On the other hand, the survey strategy is aimed at answering questions like “what,” “who,” “where,” “how much,” or “how many” [34, 35]. Surveys typically involve collecting data from a sample of a population through questionnaires or interviews to understand trends, opinions, or characteristics of the population. Unlike experiments, surveys are often applied in real-world settings and are particularly useful for gathering large amounts of data efficiently. Both strategies offer distinct advantages depending on the research objectives—experiments are ideal for establishing causality, while surveys are useful for descriptive or exploratory research where understanding broader patterns is essential.

The research protocol for this study is grounded in an experimental design, as illustrated in Fig. 4, detailing the protocol’s structure. Participants were randomly assigned to two groups: the experimental group, which trained using virtual reality (VR)-based simulation for Basic Resuscitation Skills (BRS), and the control group, which received traditional manikin-based training. Both groups underwent a pre-assessment of their skills using manikins, with competency checklists completed by designated instructors. Importantly, no instructions or training were provided at this stage to ensure an unbiased baseline assessment of their competencies.

Fig. 4.

Study process – randomized controlled experimental and survey strategies

Following the pre-assessment, each group participated in their respective training modality—VR or manikin-based. The distribution of participants was randomized, and group, instructors, and room availability were taken into consideration while ensuring that all participants met the inclusion criteria. In the third phase, participants’ psychomotor skills were re-assessed using manikins and the same competency checklists as in the pre-assessment were filled.

The fourth phase involved evaluating participants’ cognitive skills through multiple-choice questions (MCQs). For the experimental group, these questions were embedded within the VR module, requiring participants to answer them in order to complete their training. The control group answered the same set of MCQs in written form after completing their manikin-based training.

The final phase involved collecting feedback on participant satisfaction through the AOS evaluation form for both groups. Additionally, participants in the experimental group completed the Virtual Experience Tool to assess the VR program.

Tools development and validation

The development of pre- and post-skills assessment checklists was essential to this study, based on internationally accredited courses from the American Heart Association (AHA) and the American College of Surgeons (ACS). Due to the adaptation for a VR platform and the AHA’s restrictions on checklist use, healthcare experts from AOS created custom checklists. These underwent face validation by a statistician at PNU, leading to the removal of one irrelevant item and revisions for clarity. To ensure uniform application, the checklists were distributed to a diverse group of instructors who would score participants consistently. This work was part of a broader study on the Virtual Reality First Responder Course, divided into two phases: creating course content and assessment tools, and piloting the VR module while evaluating user experience and tool reliability. Participants’ skills were assessed using a standardized checklist that underwent rigorous validity testing. Content validity was established through expert feedback and calculating the item content validity index (I-CVI); items with an I-CVI below 0.8 were removed. The scale’s overall validity was assessed using the average I-CVI scores (S-CVI/Ave). For construct validity, confirmatory factor analysis (CFA) models were utilized. The assessment of construct validity was performed utilizing Principal Axis Factoring with Kaiser normalization. Widely employed fit measures encompass chi-square statistics, Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), and Root Mean Square Error of Approximation (RMSEA). These measures offer insights into the extent of data fit for a particular model. Various benchmarks for fit measures have been suggested to evaluate the adequacy of model fits. Hu and Bentler (1999) posited that an RMSEA value below 0.06, along with CFI and TLI fit measures exceeding 0.95 (RMSEA < 0.06, CFI ≥ 0.95, and TLI ≥ 0.95), signifies a good fit [9]. Conversely, Marsh, Hau, and Wen (2004) proposed less rigorous criteria, where CFI ≥ 0.90, TLI ≥ 0.90, and RMSEA ≤ 0.08 denote an acceptable model-data fit [10]. The lavaan R package was employed to carry out CFA analyses. All course content and checklists were independently developed and validated in English and Arabic. Detailed validation processes will be reported in a separate paper by the same lead authors.

Data collection and analysis techniques

This summative evaluation focuses on the short and intermediate outcomes; aim to improve the outcomes of a program on an ongoing basis, help in the continuous evaluation and implementation of corrections throughout the process, and help describe the quality and effectiveness of the program.

Data collection process

Prior to the data collection, we have received an IRB approval for the study from the PNU with the IRB approval number 22–0887. Clinical trial number: not applicable. After IRB approval, we developed or purchased the following tools and validated them for use. The whole validation process will be published in a separate paper as an addendum to this paper.

1. Development and validation of the checklists for the pre- and post- assessment for the identified program’s skills.

2. Purchasing of the VR experience assessment tool, translation to Arabic, and validation of the Arabic version.

3. Validation of the AOS evaluation form in English and in Arabic.

The study, spanning 18 months from December 2022 to May 2024, involved both the development and validation of tools, as well as the actual observational phase. This timeframe allowed for a comprehensive approach to refining the study’s instruments before data collection began. The VR modality observations were completed with relative ease, requiring only one technician to oversee the training sessions, facilitated by an expert healthcare provider from the AOS author team to ensure reliability and validity. Recruitment was achieved by announcing a free 2 h Basic Rescue training offered by AOS, which attracted numerous volunteers on designated training days. Ultimately, the study collected data from 64 participants; however, after eliminating those who did not meet the study requirements, 58 observations were deemed valid for analysis.

The manikin-based training modality required additional resources, including the presence of an AOS author team expert, three certified life support educators, and dedicated room availability at the AOS facility. These logistical demands created challenges, as it was necessary to coordinate instructor availability and secure training space before scheduling each session. Furthermore, the manikin-based training was a full-day commitment, limiting participant enrollment since it required individuals to dedicate an entire day to a skill not essential for their current career roles. These constraints extended the data collection timeline by six months beyond the completion of the validation and VR modality training phases. The total number of valid observations for analysis in this modality was 40 observations. During the Manikin based training, the instructors were assigned randomly so where the participants to avoid biases. The Instructors did not know the participants and conducted the training during the assigned days on a first-come first served basis.

Both groups trained through the Basic Rescue Course training course (either in the VR modality or the manikin based modality) and then were assessed on the cognitive (Knowledge) domain by solving multiple choice questions (MCQs). Psychomotor skills using checklists were based on completing the competencies required for the skills.

VR basic rescue training additional information

For each skill, the participants had the ability to choose the setting of the incident, whether a school, workshop, house, or garden. Furthermore, each environment is designed to replicate the actual location with adjustable dimensions based on the learners. The model was developed using Unity software and the programming language C++. Oculus Quest 2 devices were used for the CR hardware components.

To introduce the trainee to the VR world, a tutorial was provided to demonstrate the usage of the VR component and hardware, allowing them to practice as necessary. Subsequently, the trainee was presented with the main menu of the module, displaying all the skills to be acquired, and given the freedom to select the initial skill to focus on. Upon choosing a skill, the trainee was presented with a video explaining the content and practical execution of the skill, equipping them with the necessary theoretical knowledge. Following this, the trainee was immersed in the primary environment or context related to the skill, introduced to the scenario, and guided step by step by a virtual instructor dressed in medical attire (scrubs), providing demonstration and instruction. The trainees were also allowed to repeat this process until they were satisfied with their performance.

After they finish their training, they were directed to the multiple-choice questions, where the trainees were asked to answer correctly based on what they learned in the module. After completing and passing the exam, they proceeded to the practical exam, where they were assessed on the exact scenarios they trained in during the course. After they finished both the practical exam and the knowledge exam, they are shown a report card that shows their performance what they achieved, and their performance on both exams.

Manikin-based rescue training additional information

The manikin-based first responder course was developed by ten healthcare providers with experience teaching BLS and ACLS or more than three years of experience and training in simulation-based education. The course included the same skills as the BRS VR modality. It consisted of a 45-minute lecture delivered by the experts on how to perform the skills, followed by 1-hour training for the participants where the experts first demonstrated skills and then allowed the participant to practice the skills with the expert observing and giving feedback and correcting techniques until the participant perfect the skills.

The training is then followed by an assessment of the participant’s knowledge using MCQs that are the same as the VR model. The same steps were then followed as for the VR modality to conduct the post-course reassessment.

Study’s parameters

Demographic data

The following demographic data were collected from the participants: age, gender (male, female), nationality, and education level (high school and below, bachelor, higher degree), employment (employed (working 1–39 h per week), employed (working 40 or more hours per week), not employed (looking for work), not employed (NOT looking for work), retired, and disabled (not able to work)), and first language. Participants’ data will be kept in a password-protected laptop and PC; only the investigator and co-investigators will access the forms. Data entry on computers will not contain any information about the respondent.

Knowledge retention (Cognitive domain)

Participants’ knowledge retention in both the VR and manikin-based training groups was assessed using multiple-choice questions (MCQs) upon completion of their respective training sessions. For the VR group, participants completed the MCQs after each skill session and before undertaking the practical test, allowing for immediate assessment following each training component. In contrast, those in the manikin-based training took the MCQ test only at the end of the full-day course. Each participant answered a set of 10 questions to gauge retention in the Cognitive Domain. However, a comprehensive comparative study specifically focused on this domain will be undertaken as part of a future study, which will be published separately to provide a detailed analysis that extends beyond the scope of the current research.

Skill acquisition (Psychomotor domain)

Participants’ skills acquired during the course were evaluated with a standardized checklist, meticulously developed and validated by content experts to ensure both reliability and validity. The expert observer (one of the expert healthcare instructors from the author’s team in AOS) conducted the pre-assessment checklist while observing and scoring the performance of the participant. In the VR module, the same expert conducted the post-assessment after the participants concluded the VR modality training. The same process was done in the manikin-based modality, however, for time limitations, all 3 educators conducted the pre- then the post- assessment for the groups they were assigned to train.

Section 4 - Analysis and findings

Statistical analysis

The quantitative data analyses was performed using the Statistical Package for Social Sciences version 21 (SPSS 21.0) [36] and the VR Immersion Experience Evaluation Survey was analyzed with Pivot Tables in the Excel tool. The Independent variable was the training modality used (manikin and VR), and the dependent variable was the performance outcome and competencies scores. Comparisons between the two groups was completed using a non-parametric test (Mann Whitney) as the normality test Shapiro-Wilk result were statistically significant (p < 0.05) showing that the data are not normally distributed.

Demographic results

In the study, participants were categorized based on various characteristics across two groups: the VR group (n = 58) and the Manikin group (n = 40). Gender distribution in the VR group revealed that 29 participants (70.7%) were female, while 28 participants (51.9%) were male. Some demographic information such as education level and employment status was missing and we plan to mitigate to overcome in the future study we are preparing. The main reason for the missing data was that the instructors focused on having the informed consent signed by the participants, they did not focus on collecting the demographic data pertinent to this group. This is a limitation that we acknowledge and plan to mitigate in future studies.

In the study, participants were categorized based on various characteristics across two groups: the VR group (n = 58) and the Manikin group (n = 40). Gender distribution in the VR group revealed that 29 participants (70.7%) were female, while 28 participants (51.9%) were male. Table 1 shows the study participants’ demographic information. Gender did not affect the distribution in the methods and were not factors in the outcomes (p > 0.05).

Table 1.

Skills performance comparison

Skill	Virtual Reality Group (VR) (n = 58)		Manikin group (n = 40)		95% Confidence interval (CI)	Effect size (Cohen’s d)	P-value*
	Mean ± Standard deviation (SD)	(25th percentile – 95th percentile) Median	Mean ± Standard deviation (SD)	(25th percentile – 95th percentile) Median
Automated external defibrillator (AED) Checklist score (Total score = 21)	20.21 ± 1.96	(21.00–21.00) 21.00	18.00 ± 4.01	(17.00–21.00) 20.00	−9.29, −4.27	0.7	< 0.001*
Choking management of adult checklist score (Total score = 27)	24.67 ± 5.95	(25.00–27.00) 27.00	23.75 ± 4.11	(23.50–27.00) 25.00	−4.80, −0.32	0.18	0.009*
Choking management of infant checklist score (Total score = 27)	26.30 ± 1.75	(26.00–27.00) 27.00	24.20 ± 4.14	(21.00–27.00) 26.00	−11.28, −6.07	0.66	0.009*
Cardiopulmonary resuscitation of adult checklist score (Total score = 30)	28.82 ± 1.99	(28.00–30.00) 30.00	26.39 ± 3.94	(24.00–30.00) 28.00	−7.76, −2.91	0.78	< 0.001*
Stop the bleed checklist score (Total score = 27)	13.56 ± 1.83	(12.00–15.00) 14.00	14.06 ± 1.88	(14.00–15.00) 15.00	−1.04, −0.48	−0.27	0.096

*Mann-Whitney Test

Skill performance results

Skill performance was compared between the VR group (n = 58) and the Manikin group (n = 40). The AED checklist score, with a maximum total score of 21, yielded a mean score of 20.21 ± 1.96 (25th percentile – 95th percentile: 21.00–21.00; median: 21.00) for the VR group, compared to a mean of 18.00 ± 4.01 (25th percentile – 95th percentile: 17.00–21.00; median: 20.00) for the Manikin group, with a statistically significant difference (p < 0.001). For choking management of adults, the VR group scored a mean of 24.67 ± 5.95 (25th percentile – 95th percentile: 25.00–27.00; median: 27.00), while the Manikin group had a mean score of 23.75 ± 4.11 (25th percentile – 95th percentile: 23.50–27.00; median: 25.00), resulting in a p-value of 0.009. In choking management of infants, the VR group achieved a mean score of 26.30 ± 1.75 (25th percentile – 95th percentile: 26.00–27.00; median: 27.00), significantly higher than the Manikin group’s mean of 24.20 ± 4.14 (25th percentile – 95th percentile: 21.00–27.00; median: 26.00), with a p-value of 0.009. The Cardiopulmonary Resuscitation (CPR) checklist score also demonstrated a significant difference, with the VR group scoring 28.82 ± 1.99 (25th percentile – 95th percentile: 28.00–30.00; median: 30.00) compared to 26.39 ± 3.94 (25th percentile – 95th percentile: 24.00–30.00; median: 28.00) in the Manikin group (p < 0.001). Lastly, the Stop the Bleed checklist score showed no significant difference, with the VR group scoring 13.56 ± 1.83 (25th percentile – 95th percentile: 12.00–15.00; median: 14.00) and the Manikin group scoring 14.06 ± 1.88 (25th percentile – 95th percentile: 14.00–15.00; median: 15.00), resulting in a p-value of 0.096. Table 1 shows the Comparison of the performance scores among groups. Medium effect size was observed in the AED, choking management, and CPR performance (d = 0.7, d = 0.66, d = 0.78, respectively). Because different instructors conducted assessments in each group, formal inter-rater reliability was not established, which is a recognized limitation of the study.

Knowledge assessment in the VR modality

Knowledge assessment was performed for the VR groups’ participants and the results are shown in Table 2. For the “Stop the Bleed” skill, participants answered Question 1 incorrectly 10 times (41.7%), while 14 participants (58.3%) provided the correct answer. All 24 participants (100.0%) answered Question 2 correctly. In the Adult CPR skill, Question 1 received 8 incorrect responses (32.0%) and 17 correct responses (68.0%). For Question 2, 6 participants (24.0%) answered incorrectly, while 19 (76.0%) answered correctly. For the AED skill, Question 1 had 18 incorrect answers (69.2%) and 8 correct answers (30.8%). Question 2 yielded 9 incorrect responses (34.6%) and 17 correct responses (65.4%). In Question 3, 10 participants (41.7%) answered incorrectly, while 14 (58.3%) answered correctly. Lastly, in the “Choking Management of Infant” skill, 12 participants (42.9%) answered Question 1 incorrectly, whereas 16 participants (57.1%) provided the correct answer. Table 3 shows the knowledge of participants who trained using VR technology to train on first responder skills.

Table 2.

Knowledge of participants who trained using virtual reality technology to train on first responder skills

Skill	Question	Participant answer	n	%
Stop the bleed	Question 1	Wrong answer	10	41.7%
		Correct answer	14	58.3%
	Question 2	Correct answer	24	100.0%
Cardiopulmonary resuscitation of adult	Question 1	Wrong answer	8	32.0%
		Correct answer	17	68.0%
	Question 2	Wrong answer	6	24.0%
		Correct answer	19	76.0%
Automated external defibrillator	Question 1	Wrong answer	18	69.2%
		Correct answer	8	30.8%
	Question 2	Wrong answer	9	34.6%
		Correct answer	17	65.4%
	Question 3	Wrong answer	10	41.7%
		Correct answer	14	58.3%
Choking management of infant	Question 1	Wrong answer	12	42.9%
		Correct answer	16	57.1%

Table 3.

Training time for the VR modality in minutes

Skill	Mean	Standard Deviation	Percentile 25th	Percentile 95th
Introduction and orientation to the virtual reality model	1.88	2.94	0.25	7.83
Stop the bleed	9.97	4.10	7.95	17.75
Cardiopulmonary resuscitation of adult	10.00	10.88	8.07	24.60
Automated external defibrillator	4.52	2.15	3.10	9.43
Choking management of adult	4.93	1.88	3.93	8.52
Choking management of infant	21.99	16.86	5.93	48.43

Participant’s performance time in the VR group

Participants’ performance time in the VR group across various skills was measured in minutes (min) and is shown in Table 3. For the “Introduction and Orientation to the Virtual Reality Model,” the mean score was 1.88 min, with a standard deviation of 2.94 min; the 25th percentile was 0.25 min, while the 95th percentile reached 7.83 min. The “Stop the Bleed” skill had a mean score of 9.97 min and a standard deviation of 4.10 min, with the 25th percentile at 7.95 min and the 95th percentile at 17.75 min. Participants scored a mean of 10.00 min and SD 10.88 in “CPR of Adult,” with the 25th and 95th percentiles at 8.07 min and 24.60 min. Additionally, The AED skill time mean was 4.52 ± 2.15 min, with the 25th percentile at 3.10 min and the 95th percentile at 9.43. For “Choking Management of Adults,” the mean score was 4.93 min, with a standard deviation of 1.88 min; the 25th percentile was 3.93 min, while the 95th percentile was 8.52 min. Finally, in “Choking Management of Infants,” participants achieved 21.99 ± 16.86 min, with the 25th percentile at 5.93 min and the 95th percentile at 48.43 min.

Satisfaction level with the VR experience

The Confirmatory Factor Analysis (CFA) was executed to assess the construct validity of the finalized Virtual Experience Test (VET) version. The CFA was carried out in accordance with the ultimate factorial structure of VET, which encompassed 17 items associated with five domains The CFA findings of the five-factor model revealed a satisfactory fit for all items, with factor loadings exceeding 0.30. Table 4 presents the outcomes of the CFA models. The Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI) values exceeded 0.95 for all CFA models. Furthermore, the Root Mean Square Error of Approximation (RMSEA) value for the five-factor model was below 0.05 (RMSEA = 0.052), indicating a favorable fit for the data. Additionally, an examination of the factor loadings for the five-factor model demonstrated that all items possessed factor loadings greater than 0.30. Moreover, given that the ten-factor model exhibited CFI and TLI fit indices surpassing 0.95 and RMSEA values below 0.06, a strong fit was attained between the five-factor CFA model and the data according to the criteria proposed by Hu and Bentler (1999). These findings suggest that the five-factor CFA model achieved robust construct validity.

Table 4.

VR immersive environment experience evaluation

Question	Strongly Agree (5)	Agree (4)	Neutral (3)	Disagree (2)	Strongly Disagree (1)
The VR equipment was of high quality	75	19.64	3.57	1.79	0
The interaction with the VR environment was of high quality	73.21	23.21	3.57	0	0
The visual content of the VR was high quality	75	21.43	3.57	0	0
I think that the VR environment was able to support multiple users at the same time	67.86	19.64	7.14	3.57	1.79
In the presence of an emotional situation, I felt that my emotional status aligned with the VR’s scenario	51.79	37.5	7.14	3.57	0
The VR was successful in providing clear instructions of the scenario	75	21.43	1.79	1.79	0
I had multiple emotions while working through the scenario provided in the VR	64.29	25	10.71	0	0
Completing tasks in the virtual environment requires high-level interaction with other users or a virtual assistant	71.43	16.07	10.71	1.79	0
Virtual assistant (AI) has been well used in the virtual environment	71.43	25	1.79	1.79	0
There were emotional reactions while performing tasks in the virtual environment	51.79	30.36	16.07	1.79	0
I really felt like I was the character I was controlling in the virtual environment	69,364	19.64	10.71	0	0
The content in the virtual environment was helpful in introducing me to my current role	71.43	23.21	5.36	0	0
I feel like I can create a story from my actions in the virtual environment	57.14	28.57	14.29	0	0
The user interface was helpful in getting me familiar with my current task	71.43	19.64	8.93	0	0
The sensory content (tactile simulating content) in the virtual environment was of high quality	64.29	25	8.93	1.79	0
The guidelines in the virtual environment were clear about what was allowed and what was not allowed	71.43	21.43	5.36	0	0
The sound system was of high quality	78.57	14.29	5.36	0	0
There are multiple techniques/methods/means in the virtual environment to evoke emotions	69.64	21.43	8.93	0	0
The audio content of the virtual environment was of high quality	78.57	16.07	5.36	0	0
The tasks I was able to complete in the virtual environment were enjoyable	83.93	10.71	5.36	0	0
The virtual environment allowed me to complete my assignment in different ways	75	14.29	8.93	0	1.79
I was able to reuse techniques I learned in previous assignments in subsequent assignments on an ongoing basis	73.21	19.64	5.36	1.79	0
The sensory devices (simulated and related to the sense of touch) in the virtual environment were of high quality	67.86	23.21	8.93	0	0
The sensory information of the virtual environment was consistent and in line with reality, for example the sound of two metal objects colliding, and the smooth texture of smooth objects	66.07	23.21	10.71	0	0

Table 4 shows the percentage distribution of the VR experience using the validated tool for VR Immersive experience evaluation form. The survey used for the VR immersive experience evaluation was assessed based on a 5-level Likert Scale where 5 was the highest satisfaction scoring and 1 the lowest. The details in Table 4 clearly show a high level of satisfaction with the VR experience where the majority of participants either agreed or strongly agreed with the statement.

Section 5 - Results and discussions

Limitations

This study encountered several methodological constraints that warrant consideration. First, significant logistical challenges emerged in the manikin-based training arm, including requirements for dedicated instructors, specialized facilities, and full-day participant commitments. These resource-intensive demands resulted in a six-month recruitment delay, though we ultimately exceeded the minimum sample size (N = 38) required for adequate statistical power (G*Power, α = 0.05, power = 0.80). Second, operational constraints during manikin-based training led to incomplete documentation of two key parameters: [1] participant demographic characteristics and [2] time-on-task metrics per trainee [3] satisfaction and engagement. While these data gaps limit our ability to conduct supplementary analyses of potential confounding variables, they provide important lessons for future implementations. We are currently designing a follow-up study with enhanced procedural controls to address these limitations, including standardized data collection protocols and balanced resource allocation across training modalities. These methodological challenges, while not affecting our primary outcome measures, highlight the real-world complexities of comparative training studies in emergency response education.

Consideration of novelty effects

While our results demonstrate the effectiveness of VR training compared to traditional methods, we acknowledge the potential influence of novelty effects on participant performance and satisfaction metrics. The inherent appeal of emerging technologies may introduce temporary performance enhancements through increased learner engagement or motivation bias. To mitigate this confounding factor, our study design incorporated three safeguards: (1) standardized pre-training orientation sessions to reduce initial technology-related excitement (2), repeated skill assessments over to evaluate sustained (versus initial) performance gains, and (3) control group participants who received equivalent attention time from instructors. Furthermore, our satisfaction metrics specifically distinguished between technology novelty (e.g., “The VR system was exciting to use”) and perceived training efficacy (e.g., “This method improved my skills”), allowing for differential analysis of these constructs. These methodological choices strengthen our conclusion that observed differences primarily reflect pedagogical advantages rather than transient novelty responses, though longitudinal studies with repeated VR exposures remain necessary to fully disentangle these effects.

General results based on the conceptual framework

Through the lens of the Social Cognitive Theory (SCT) framework, the results reveal that the VR immersive experience contributed significantly to participant satisfaction and supported self-directed learning, which in turn enhanced knowledge retention. In alignment with SCT’s constructs, the feedback indicates that participants who engaged with the VR modality demonstrated high levels of satisfaction, which positively influenced their self-efficacy. Notably, despite varied personal and cognitive factors, participants from non-healthcare backgrounds—who were unfamiliar with the skills—achieved high levels of confidence and competence. This suggests that the VR environment successfully mitigates potential self-efficacy barriers by providing an engaging and supportive learning context.

These outcomes suggest even greater benefits when applied to healthcare providers, particularly those renewing their Basic Life Support (BLS) certifications. For healthcare professionals, the Mongeth VR module could streamline the recertification process, making it more effective and less time-consuming. The immersive and interactive nature of VR allows busy providers to quickly refresh their skills without compromising the quality of learning. The experience suggests a promising future for VR in healthcare training, where recertification can be completed efficiently without pulling professionals away from their demanding daily responsibilities. More importantly, the results promise a high community involvement and capability to achieve this study’s long term outcome which is to improve population health through the training of key members in the community on basic rescue skills that will save lives while waiting for the professionals to arrive on the scene.

Importantly, individual behavioral factors did not hinder learning outcomes, as all participants—regardless of background—successfully completed the skills training and assessments in both the VR and manikin-based modalities. Observations showed that participants in the VR modality exhibited a strong focus and determination, benefiting from the self-paced nature of the training. Remarkably, the VR course, including both pre- and post-assessments, took less than two hours, highlighting VR’s potential to deliver intensive skills training in a condensed timeframe. This self-paced, time-efficient approach effectively supports skill mastery, making VR a practical training tool for both new learners and busy healthcare professionals.

Interpretation and results from the statistical analysis

Three key limitations contextualize our findings. First, while randomization occurred among volunteers, our exclusively civilian professional sample (100% voluntary participants seeking supplemental certification) may over represent individuals with higher intrinsic motivation compared to mandatory trainees [37]. This self-selection bias suggests caution when generalizing results to compulsory training programs, particularly for time-sensitive skill acquisition in professional responders. Second, the participant imbalance (58 VR vs. 40 manikin, 31% discrepancy) was addressed through Welch’s t-tests and post-hoc equivalence testing, with sensitivity analyses confirming effect size stability for group differences < 35% in simulation studies [38]. Finally, while instructor blinding and standardized protocols minimized observation biases, the Hawthorne effect remains a consideration in unblinded designs [39].However, our naturalistic assessment environment - where participants focused on practical skill demonstration rather than study participation - likely attenuated these effects more effectively than controlled lab settings [40]. These factors particularly warrant replication in: (1) mandatory training cohorts, and (2) more balanced recruitment contexts, while supporting VR’s immediate applicability for voluntary continuing education programs.

Interpretation linked to hypothesis

The interpretation of Table 2 reveals key insights into the impact of learning modality on training outcomes. Specifically, for Stop-the-Bleed training, there was no significant difference between VR and Manikin-based modalities, as indicated by the P-value of 0.096, which is above the acceptable threshold of 0.005. This suggests that the learning modality did not influence outcomes for this skill, with both groups achieving similar results pre- and post-training. However, for other critical skills—such as AED use, adult CPR, and adult and pediatric choking response—the VR modality demonstrated a statistically significant advantage over the Manikin-based approach.

Interestingly, the data challenges the conventional belief that hands-on demonstration by instructors, followed by immediate feedback, would yield the best results. Instead, participants in the VR modality performed better, likely due to the opportunity for self-paced learning and unlimited practice without the pressure of an in-person audience. The VR environment allowed learners to focus intensively on instructions and refine their technique through repeated practice, ensuring they could execute each skill with precision and efficiency. This suggests that VR not only provides a robust learning platform but also facilitates mastery by empowering participants to engage in deliberate practice until they reach optimal proficiency. This result supports Hypothesis H2: VR training modality will be positively associated with the improved performance of the participants in the basic rescue skills training.

The findings in Table 3 further support Hypothesis 2 (H2), particularly in terms of knowledge retention and passing scores on the MCQs for participants in the VR modality. Due to constraints on the availability of experts to assess MCQ responses immediately in the Manikin-based training, this analysis could not be completed for the Manikin group, highlighting a recommendation for future studies to explore comparative information retention across both modalities. For this study’s scope, the VR modality demonstrated strong retention, as most participants accurately answered the MCQs.

However, a limitation in the VR approach is the absence of real-time monitoring of participants’ performance, which means instructors are not able to provide immediate, hands-on guidance as they would in a traditional setting. Despite not achieving a 100% correctness rate on the MCQs, participants in the VR modality showed a significant improvement in their practical skills. This improvement suggests that our objective was largely met, as the VR training enabled participants to enhance their practical abilities substantially, a core focus of the study’s goals.

Hypothesis 1 (H1) positing that the VR training modality would positively impact participants' experiences in basic rescue skills training, and Hypothesis 4 (H4) suggesting a positive association between VR training and participant satisfaction and engagement, were not fully assessed due to limited resources for monitoring and simultaneous feedback collection across both training modalities. Evaluation forms were provided via a QR code for easy access, but due to time constraints and personnel shortages, we could not ensure consistent completion of the AOS evaluation form. Ultimately, we obtained only 9 responses from the VR modality and 2 from the Manikin-based modality, limiting our comparative insights.

To address this gap, future studies are needed to thoroughly assess satisfaction levels across both training methods using a standardized tool to allow for direct comparison. Despite these challenges, we successfully gathered 56 responses specifically on the VR Immersive Experience assessment. These responses indicate a very high level of satisfaction with the VR immersive experience where the participants felt they were living the scenario from a visual, audio, and emotional perspectives. The participants also were able to follow the clear instructions provided to them in the scenario and accurately apply the skills taught in the modules.

Hypothesis 3 (H3), suggesting that the VR training modality would positively impact the time efficiency in basic rescue skills training, was validated by the study results. The VR modality required just two hours to complete, including observations, in contrast to the full day needed for the Manikin-based training. As shown in Table 4, the time efficiency in VR training was most notable in skill-based tasks: for instance, Choking Management for infants averaged around 22 minutes for correct completion, whereas all other skills averaged under 10 minutes.

The overall mean time for VR training, covering introduction, orientation, MCQs, and correct practice, was 53.29 min. This significant time-saving aspect of VR training highlights an advantage over the Manikin-based modality, where instructors must work individually with participants, making it more time-intensive. This finding underscores VR training’s potential for offering a streamlined, focused learning experience that maximizes the effective use of training time.

Comparative efficacy of VR training across emergency response skills

VR training has demonstrated significant advantages in teaching critical emergency skills, including AED usage, adult and infant choking management, and CPR, by leveraging immersive, interactive learning principles. For AED application, VR enhances spatial awareness and electrode placement accuracy through 3D anatomical visualization and real-time corrective feedback [41]. Augmented reality (AR) systems provide real-time feedback on electrode positioning, improving repeatability compared to traditional methods [42]. For CPR, VR outperforms traditional methods in maintaining proper compression depth and rate, attributed to biomechanical feedback and stress inoculation in controlled yet realistic environments [43, 44]. These benefits align with cognitive load theory [45] and deliberate practice frameworks [46], as VR reduces extraneous cognitive demands while enabling repetitive, structured skill refinement. The technology’s capacity to engage emotional memory [47] further enhances long-term retention. Future research should prioritize longitudinal retention studies, cost-effective haptic solutions for low-resource settings, and AI-driven adaptive learning systems to personalize training. Additionally, validating VR’s efficacy in diverse populations, including low- and middle-income countries, will be critical for global scalability. By addressing these gaps, VR can solidify its role as a transformative tool in emergency response training, combining pedagogical rigor with practical accessibility.

The differential effectiveness of VR training across various emergency skills can be attributed to fundamental distinctions in skill acquisition requirements. For AED usage, CPR, and choking management, VR demonstrates particular efficacy due to the primacy of visual-spatial learning and procedural sequencing in these domains. The technology excels in simulating anatomical relationships (e.g., optimal AED pad placement relative to cardiac anatomy), rhythm maintenance (e.g., CPR compression rate), and technique precision (e.g., infant back blow positioning) through immersive 3D visualization and instantaneous performance feedback. These skills share common characteristics that align with VR’s strengths: they require consistent, standardized motions rather than adaptive tactile responses, and success depends more on cognitive mapping than on nuanced physical manipulation.

In contrast, hemorrhage control skills such as those taught in Stop the Bleed training present unique challenges for VR implementation. Effective tourniquet application and wound packing demand sophisticated haptic feedback - the ability to perceive and adjust pressure, tension, and packing density through tactile and proprioceptive cues. Current VR systems cannot fully replicate the resistance of a tightening tourniquet, the texture of wound dressings, or the variable compliance of different tissue types. This technological limitation creates a fidelity gap that may explain the observed performance parity between VR and traditional training methods for these particular skills.

The cognitive-psychomotor spectrum of emergency skills further clarifies this divergence. AED application and choking interventions occupy the more cognitive end of the spectrum, where decision-making and spatial reasoning predominate. CPR represents an intermediate position requiring both cognitive timing and physical execution, but where the physical component (chest compressions) follows a predictable, repetitive motion pattern. Hemorrhage control, however, resides firmly at the psychomotor extreme, demanding real-time tactile adaptation to unpredictable tissue resistance and bleeding dynamics.

Section 6 - Conclusion

Summative program evaluation interpretation

The evaluation demonstrated significant achievement of both short-term and intermediate outcomes, with the Virtual Reality modality yielding enhanced knowledge acquisition coupled with high participant satisfaction levels and marked improvement in skill performance. While the long-term impact on population health requires further longitudinal investigation, the implementation of the Mongeth program represents a pivotal advancement toward comprehensive community health enhancement. The program evaluation successfully fulfilled the predetermined outputs outlined in the logic model and generated innovative procedural frameworks. Additionally, the Academy of Sciences (AOS) established foundational accreditation guidelines, although current limitations preclude immediate presentation to accrediting bodies.

Given the compelling evidence supporting the efficacy of Virtual Reality in basic rescue skills training, particularly its significant impact on first responder preparation, this summative evaluation strongly advocates for the expansion of VR implementation in healthcare provider education, specifically in Basic Life Support (BLS) recertification processes. The demonstrated time efficiency and high satisfaction rates further reinforce this recommendation. Moreover, based on the program’s validated success and effectiveness, we propose integrating this VR methodology across the spectrum of life support training programs, including Advanced Cardiac Life Support (ACLS), Advanced Trauma Care Nursing (ATCN), Prehospital Trauma Life Support (PHTLS), and Advanced Trauma Life Support (ATLS), to maximize its potential impact on healthcare education.

Global implementation considerations for VR-Based training

This study’s findings demonstrate that VR-based training offers significant logistical advantages for basic rescue skills education, including reduced instructor requirements, minimal space needs, and time-efficient implementation. While these benefits suggest particular promise for resource-constrained settings, several implementation challenges must be addressed before widespread adoption in low- and middle-income countries (LMICs) can be recommended. The upfront costs of VR technology, while potentially offset by long-term savings from eliminated consumable expenses and reduced personnel requirements, remain a substantial barrier in many LMIC contexts. Furthermore, successful implementation would require reliable electricity infrastructure, basic technical support systems, and cultural adaptation of training content. We propose a phased implementation approach beginning with tertiary referral centers, where: (1) rigorous cost-benefit analyses could quantify the total 5-year expenditure compared to traditional methods (2), local acceptability studies could assess cultural and educational appropriateness, and (3) train-the-trainer programs could establish sustainable local expertise. Future research should prioritize the development of ruggedized, low-cost VR systems specifically designed for LMIC environments, while public-private partnerships could help overcome initial investment barriers. These considerations highlight VR’s potential to transform emergency skills training globally while acknowledging the practical constraints that must be addressed to ensure equitable implementation.

Recommendations for future studies

Based on the findings and the limitations in this study, we recommend some future studies to be conducted:

1. Experimental focused study on the cognitive domain (MCQs) before and after learning in the two different modalities to accurately be able to compare the retention of information and to compare the results between modalities.

2. Expand the study Nationally in Saudi Arabia, Regionally in the MENA region, and if possible, internationally.

3. Conduct a summative evaluation to address the long-term outcome of this study’s Logic model to study the impact on population health and changes in their behavior in emergencies.

4. Time motion study with an accurate oversight on the time spent in each module for both the VR and Manikin-based modalities, and then compare the time spent in each with the outcome in performance.

5. Propose a study to international accrediting bodies to explore new methods for awarding continuing education credits in VR based healthcare education.

6. Further study on retention of information where in the VR modality, we can have the technician monitor in real time the completion of the modules to ensure we have everything completed in time. In addition to having the appropriate process in place to collect data from the Manikin-based modality.

7. Conduct a study that focuses on the learner’s experience and engagement in both modalities.

8. Conduct a study on the data collected in the recruitment questionnaire to.

   • a
     Check if previous gaming, age, or VR use experience allowed for a faster completion of the VR modality and dragging or dissatisfaction in the manikin-based modality.
   • b
     Check if wearing eye glasses hindered the participants’ experience and satisfaction in the VR modality.
   • c
     Check if having previous injuries (back or joints) affected the levels of satisfaction in both modalities.

Future directions for VR training development

Advancing VR’s effectiveness for tactile-dependent skills will require technological innovations in haptic feedback systems and training paradigm redesign. Promising avenues include the development of hybrid simulation platforms that combine VR visualization with physical task trainers, allowing learners to simultaneously benefit from immersive scenarios and authentic tactile experiences. Adaptive algorithms could modulate virtual wound properties based on user interventions, creating more dynamic training environments. Additionally, research should explore cognitive scaffolding strategies that use VR for initial cognitive skill acquisition before transitioning to physical practice for psychomotor refinement.

Implications for training program design

These findings suggest a tiered approach to emergency skills training technology adoption. VR currently offers superior training solutions for spatially complex but physically consistent skills like AED use, while traditional methods remain essential for tactile-intensive skills like hemorrhage control. As haptic technologies mature, the boundary between these approaches will likely blur, enabling comprehensive VR training for the full spectrum of emergency response skills. Training programs should strategically deploy VR where it demonstrates clear advantages while maintaining hands-on components for skills requiring high-fidelity tactile experience, with periodic reassessment as VR capabilities evolve.

Authors’ contributions

S.A.T. conceptualized the study, led the project development, and provided expert oversight throughout the study implementation. D.S. led the program evaluation design, logic model development, and contributed substantially to the analysis and writing of the manuscript. G.M. contributed to tools development and validation, and participated in data analysis and manuscript drafting. B.O. contributed to the data analysis process, manuscript writing, and manuscript editing..M. supported the methodological design and provided expert review of emergency medicine content. L.I. contributed to project management, manuscript review, and critical revisions. C.D. directed simulation implementation, coordinated training logistics, and supported evaluation tool use. M.N.A. supervised quality assurance and checklist development and contributed to validation. A.A.M. reviewed clinical content and contributed to the development of the training curriculum. A.O. served as VR project manager, coordinated software implementation, and supported user training. A.A.G. and I.A. contributed to data collection oversight, study design and conceptualizations, and supported the analysis of data. M.A., N.A., D.A., and A.A.D. participated in data review, data cleaning, and data analysis. A.A. provided institutional support, supervised site logistics, and reviewed the manuscript. All authors reviewed and approved the final version of the manuscript.

Funding

This research has not received funding.

Data availability

The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

The study received approval from the Princess Nourah University Institutional Review Board (IRB), with the IRB log number 22–0887. All methods and procedures conducted in this study complied with relevant guidelines and regulations. Informed consent was obtained from all subjects who participated in the study. All procedures adhered to the Declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.