Skip to main content
JMIR Serious Games logoLink to JMIR Serious Games
. 2024 May 23;12:e54188. doi: 10.2196/54188

Adoption of Augmented Reality in Educational Programs for Nurses in Intensive Care Units of Tertiary Academic Hospitals: Mixed Methods Study

Suyoung Yoo 1,#, Sejin Heo 2,#, Soojin Song 3, Aeyoung Park 3, Hyunchung Cho 1, Yuna Kim 3, Won Chul Cha 1,2, Kyeongsug Kim 4, Meong Hi Son 1,2,
Editor: Anna McDaniel
Reviewed by: Gamal Alkawsi, George Evangelinos
PMCID: PMC11157172  PMID: 38780998

Abstract

Background

In the wake of challenges brought by the COVID-19 pandemic to conventional medical education, the demand for innovative teaching methods has surged. Nurse training, with its focus on hands-on practice and self-directed learning, encountered significant hurdles with conventional approaches. Augmented reality (AR) offers a potential solution to addressing this issue.

Objective

The aim of this study was to develop, introduce, and evaluate an AR-based educational program designed for nurses, focusing on its potential to facilitate hands-on practice and self-directed learning.

Methods

An AR-based educational program for nursing was developed anchored by the Kern six-step framework. First, we identified challenges in conventional teaching methods through interviews and literature reviews. Interviews highlighted the need for hands-on practice and on-site self-directed learning with feedback from a remote site. The training goals of the platform were established by expert trainers and researchers, focusing on the utilization of a ventilator and extracorporeal membrane oxygenation system. Intensive care nurses were enrolled to evaluate AR education. We then assessed usability and acceptability of the AR training using the System Usability Scale and Technology Acceptance Model with intensive care nurses who agreed to test the new platform. Additionally, selected participants provided deeper insights through semistructured interviews.

Results

This study highlights feasibility and key considerations for implementing an AR-based educational program for intensive care unit nurses, focusing on training objectives of the platform. Implemented over 2 months using Microsoft Dynamics 365 Guides and HoloLens 2, 28 participants were trained. Feedback gathered through interviews with the trainers and trainees indicated a positive reception. In particular, the trainees mentioned finding AR particularly useful for hands-on learning, appreciating its realism and the ability for repetitive practice. However, some challenges such as difficulty in adapting to the new technology were expressed. Overall, AR exhibits potential as a supplementary tool in nurse education.

Conclusions

To our knowledge, this is the first study to substitute conventional methods with AR in this specific area of critical care nursing. These results indicate the multiple principal factors to take into consideration when adopting AR education in hospitals. AR is effective in promoting self-directed learning and hands-on practice, with participants displaying active engagement and enhanced skill acquisition.

Trial Registration

ClinicalTrials.gov NCT05629663; https://clinicaltrials.gov/study/NCT05629663.

Keywords: augmented reality, AR, clinical skills education, nurse education, technology-based education, education, nurse, nursing, allied health, technology-enhanced learning, interview, training, usability, acceptability, educational, teaching, ICU, intensive care unit, self-guided, self-directed, hands-on, adoption, TAM, Technology Acceptance Model, skill, acquisition

Introduction

In recent years, conventional education, especially in the medical field, has been challenged by the introduction of new technologies [1]. The COVID-19 pandemic further highlighted the limitations of conventional teaching methods [2]. Nurse training, with its emphasis on hands-on practice and self-directed learning, was particularly affected by the pandemic, making it evident that conventional training methods could not sustain the demands of the situation [3,4]. Given these constraints, the search for alternative, technology-driven educational methods intensified, aiming to address both physical resource and time challenges without compromising education quality [5,6]. In this context, an immersive learning environment, based on a computer-generated environment enabling real-time user interactions [7], has emerged as a promising solution.

Such an immersive platform merges augmented reality (AR) and virtual reality (VR), offering a dynamic 3D space for learners. This integration not only enhances the learning experience by providing a rich, immersive environment [8,9] but also reshapes the boundaries between reality and the virtual realm, paving the way for innovative learning avenues [10,11]. Exploring AR’s potential reveals that its uses surpass merely aiding in remote education. AR also introduces real-time feedback mechanisms, empowering trainees to obtain immediate insights about their actions and performance through virtual aids [12,13]. This immediacy in feedback is invaluable, as it allows errors to be addressed promptly, fostering continuous improvement in learning [14,15].

In the field of critical care, there has been an exploration of the use of AR and VR in educational applications [16]. VR-enhanced training for tracheostomy care in the intensive care unit (ICU) setting has demonstrated the potential of education in an immersive learning environment [17,18]. Studies have been conducted for training mechanical ventilator settings and central line insertion, showing improvements in self-efficacy, increased familiarity, confidence, and reduced anxiety compared to conventional methods [13,17,19]. However, while the advantages are evident, existing research into AR and VR remains limited. These studies are usually one-time or short-term investigations, mainly focusing on the effectiveness of the immersive learning environment [20]. Moreover, integrating these technologies into a nursing curriculum represents an area yet to be fully explored [21].

In this study, we aimed to derive key considerations for each phase of implementation based on our experience of introducing an AR nursing program within an ICU in a tertiary hospital setting.

Methods

Experimental Design

Overview

Our methodology was refined based on the Kern six-step approach to transition nursing education into an immersive AR-based format [22], focusing on the following key stages: (1) problem identification, (2) needs assessment, (3) setting goals and objectives, (4) choosing educational strategies, (5) implementation, and (6) evaluation [23] (Figure 1).

Figure 1.

Figure 1

Overall process of the adoption of augmented reality into nursing education based on the Kern 6-step framework [22]. AR: augmented reality; ECMO: extracorporeal mechanical oxygenation.

Our initial steps involved conducting interviews with trainer nurses to discern existing issues and identify a procedure amenable to transition into an AR format [24]. We then developed and implemented an AR-based educational program encapsulating two distinct procedures. We surveyed and interviewed trainees, focusing on technology acceptance and usability.

Stage 1: Problem Identification

In this foundational stage, we endeavored to identify core problems through a general needs assessment, employing two primary approaches: engaging in interviews with training nurses and reviewing the extant literature [24-27].

The interviews, conducted with nurses from the nursing education department, were based on a semistructured format and were held online or offline depending on participant preference. Voice recordings ensured the precise capture of data shared during these interactions.

In addition to the interviews, we explored previous studies with the aim of harvesting insights and identifying common issues found within the existing research landscape, thereby anchoring our findings in a robust context of existing knowledge [28].

Stage 2: Needs Assessment

Upon conducting the interviews in stage 1, we recognized the challenges endemic to conventional educational methods as identified by educators. We investigated possible solutions to these challenges, and found the need for feedback through remote supervision, especially regarding certain devices. We further identified key elements that should be considered in the development of the educational program. This stage also served to validate our problem identification process.

Stage 3: Goals And Objectives

We defined our goals through collaboration between trainers and researchers, focusing on improving access and proficiency with complex medical equipment.

Stage 4: Educational Strategies

Step 1: Selecting Appropriate Technology

With the imperative for hands-on practice in nursing education, AR was chosen to enable nurses to interact with virtual medical devices within a realistic clinical setting [29]. AR’s ability to superimpose digital models onto the physical world allows for a highly interactive and immersive learning experience without the traditional constraints of location, time, or physical resources [30]. This aligns with our goal to empower self-directed learning, permitting nurses to engage in practical education at their own pace and convenience [31].

Step 2: Developing Educational Material

Navigating through the lens of self-directed learning and hands-on practice, we considered AR options that could facilitate tangible interaction with 3D objects. The development of materials required a detailed comprehension of the unique needs of nurses and the incorporation of AR content to support self-directed learning.

Step 3: Preparation of the Educational Environment

To facilitate a high-fidelity learning experience, our AR-based educational program was set within the hospital’s simulation laboratory. A designated area within the lab was prepared, encompassing a minimum of 3×3 meters to provide trainees with sufficient room to maneuver and interact with the virtual elements without spatial constraints. To ensure uninterrupted delivery of our AR-based educational program, we utilized five HoloLens 2 devices. This approach was adopted to mitigate against battery and overheating issues that could disrupt the learning process. This setup was optimized to allow multiple nurses to receive training at the same time, promoting efficient learning throughout while maintaining an individualized, hands-on experience.

Stage 5: Implementation

We recruited the participants through an advertisement posted on the hospital’s internal internet network. Our goal was to enroll a minimum of 10 participants for each session to ensure a dynamic and interactive learning environment while still allowing for personalized instruction. We designed the sessions to accommodate up to five nurses at a time, which was determined as the optimal number for both effective learning and space utilization within our AR setup. This small-group approach not only facilitated focused attention from the instructors but also ensured that each participant could engage deeply with the AR modules.

To maintain a high standard of education and safety, we appointed two experienced supervisors for each training session. These supervisors were selected based on their expertise in intensive care procedures and their familiarity with AR technology. Their role was to provide immediate assistance and feedback, ensuring that any technical issues could be addressed without disrupting the learning process. They were also tasked with observing the sessions to gather informal feedback, contributing to the continuous improvement of the program.

Stage 6: Evaluation

This study was conducted at a large academic tertiary hospital in Seoul, Korea, which accommodates more than 3100 nurses and 2000 inpatient beds. The research provided education to intensive care nurses and included a postsession survey and semistructured interview.

Ethical Considerations

The Institutional Review Board of Samsung Medical Center approved the study design (SMC-2022-08-058 and SMC-2022-08-079), and all trainers and trainees provided written informed consent before participating in the study, ensuring ethical adherence throughout the research. To protect the participants’ privacy and confidentiality, all data collected during this study were anonymized or deidentified. Stringent data protection measures are in place, including the use of secure, encrypted data storage systems accessible only by authorized personnel. These precautions are designed to safeguard sensitive information and maintain the integrity of the data. Participants were compensated for their time and contribution. Each participant received 30,000 KRW (~US $22) upon completion of their involvement in the study. This compensation was intended to acknowledge their valuable time and effort and to offset any inconvenience associated with participation. The compensation structure was clearly communicated to all participants prior to their enrollment and was administered transparently to ensure fairness.

Outcome Measures of the Survey

Upon completion of the educational program, participants were asked to fill out a questionnaire evaluating their user experience. This evaluation was based on the theories of self-directed learning and hands-on practice, including questions on personal characteristics, job satisfaction, and appropriateness. To evaluate the AR program’s acceptability and usability, we employed the System Usability Scale (SUS) and the Technology Acceptance Model (TAM) [20,32,33].

We chose the SUS for its proven reliability and efficiency across various technologies. The SUS is widely used across various domains, including software, websites, and medical devices, to assess overall user experience. Moreover, it has been validated in hospital environments and shown effectiveness with small sample sizes. The SUS consists of 10 simple questions presented in a 5-point Likert-scale format, assessing both positive and negative aspects of the system, with total scores ranging from 0 to 100 [34,35]. The TAM was selected for its emphasis on understanding user acceptance of information technology [16,36,37]. We adapted the TAM-based survey questions to fit the context of AR nurse education, informed by various relevant studies [24,26,27,38]. In contrast, the SUS was employed in its unmodified form. In addition, we conducted a correlation analysis between the TAM and SUS elements [37,39].

Outcome Measures of Interviews

Nurses who responded to the questionnaire were selectively screened for their willingness to participate in further interviews. These interviews were semistructured and guided by nursing education theories from previous studies. The format allowed flexibility, permitting up to two additional questions based on the responses of the interviewees.

Statistical Analysis

The statistical analysis was performed using R software (version 4.3.2). Continuous variables are expressed as either mean (SD) or median (IQR), depending on their distribution, while nominal variables are expressed as counts (n) and percentages (%). We performed a correlation analysis to examine the relationship between the TAM and SUS using survey data.

Results

Stage 1: Problem Identification

Overview

We obtained interview results from four nurses who are trainers and operators in the nursing education department. The insights garnered from the interviews are summarized below and detailed in Table 1.

Table 1.

Key considerations in augmented reality (AR) education development as expressed during trainer interviews.

Category Key details Core implications
Educational needs and challenges Training requirements necessary for handling advanced medical equipment; aligning AR educational content to complement the features of specific medical devices; adapting training modules to meet the unique demands precipitated by the COVID-19 pandemic; implementing streamlined training processes for the rapid acclimatization of new nurses; addressing the limitations inherent in conventional training techniques; tackling the deficit of hands-on training equipment in nursing training Imperative for AR solutions in bridging training disparities and responding to progressive requirements
Program development and strategies Tailoring educational programs to align with the diverse experience levels of nursing professionals; standardizing the phases of training to ensure uniformity and consistency in educational outcomes; formulating well-structured and strategic plans for nursing training; ensuring efficient and effective distribution and management of training equipment and resources Criticality of a holistic design and meticulous implementation in AR training for optimal efficacy
Challenges and future concerns Addressing trainees’ physical challenges, such as the necessity to wear glasses or masks, in the training environment; guaranteeing the safety and appropriateness of both the devices and venues utilized for training; modifying AR training methods to be inclusive and effective for older nursing personnel; integrating strategies within training programs to manage and reduce trainee fatigue effectively Recognition of and addressing present and prospective hurdles for continuous advancement in AR training

Requirement of Education in Difficult-to-Use Devices

Using difficult-to-use medical devices in health care can pose a significant challenge for medical staff due to the increased risk of errors, negatively impacting patient outcomes. Proper education on these devices is essential to ensure that medical staff can use them safely and effectively.

Lack of Resources: Space, Instructor, Time, and Cost

The lack of education resources in health care can be a significant challenge for health care organizations and medical staff [15]. Education is essential to ensure that health care professionals possess the necessary knowledge, skills, and abilities to provide safe and effective care to patients. However, the interviewees mentioned that many health care organizations face barriers in providing adequate education resources to their staff, which can negatively impact patient outcomes.

Another challenge is the lack of adequate education time. Health care professionals are often required to work long hours, and finding time to attend education sessions and complete the necessary course work can be difficult. This lack of time can make it difficult for health care organizations to provide education tailored to the specific needs of their staff.

Stage 2: Needs Assessment

Following the interviews, we identified crucial factors to consider when selecting educational topics. Educators highlighted the significant challenges of limited access to educational devices and instructors. Additionally, they emphasized the necessity for education in technically demanding skills. Trainers expressed a preference for educational topics that required hands-on practice. Their reasoning is grounded in the knowledge that complex devices are frequently used in treating patients with critical illnesses. The competence of nurses in operating these devices directly impacts patient outcomes. The responses from trainers and operators related to needs are summarized in Table 1.

Stage 3: Goals and Objectives

Selected Objectives

Based on the interview results, extracorporeal membrane oxygenation (ECMO) machines and ventilators were selected as the objectives for training owing to their complexity and difficulty of use. Ventilators were selected as important yet challenging devices to master. The complexity of ventilators, compounded by the multitude of lines and connections involved, can pose challenges for nurses with limited experience.

By contrast, an ECMO machine is a high-risk medical device that is essential for patients with COVID-19. When the alarm of an ECMO machine sounds, nurses must promptly find a solution. However, given its rarity, even experienced nurses may not have encountered this situation. Nevertheless, as this could pose a risk to the patient, appropriate education was deemed necessary, and therefore use of the ECMO machine was selected as the problem scenario for this evaluation.

Selected Goals

Our aim was to develop a sustainable AR-based educational program that could offer numerous benefits to trainers and trainees. These benefits include enhanced engagement and motivation, interactive and immersive learning experiences, and the facilitation of personalized learning. Key considerations for developing such a sustainable AR-based educational program encompass designing for scalability and accessibility, and incorporating user feedback to enhance the program in its maintenance and operation over time.

Stage 4: Educational Strategies

Step 1: Development of an AR-Based Educational Program

The ECMO machine educational program comprises 45 slides divided into four parts, each detailing the operation of the machine, managing machine disruptions, responding to “low battery” alarms, and addressing the loss of flow signal (“SIG”) alarm. Each part includes approximately 8-10 steps, guiding trainees on how to manage each situation effectively.

The ventilator program is composed of three parts encompassing a total of 46 slides. Each part involves 26 steps related to ventilator settings and preuse inspections, as well as seven steps for application, alarm configuration, and educational content evaluation. The education process involves checking supplies, power sources, wall oxygen and medical gas connections, and exhalation cassette connections; powering on the device; performing preuse checks; connecting test tubes; selecting the target and application method; turning on the humidifier; configuring the mode and parameters; connecting the patient to the system; monitoring after patient application; and setting alarms.

Step 2: Adoption of Innovative Technology

The AR-based education was performed with a Maquet Servo-I mechanical ventilator and the RotaFlow II System Permanent Life Support ECMO machine from Getinge. We attempted to incorporate a 3D guide for hands-on practice and used videos to enhance understanding. The AR-based educational program was developed using the Microsoft Dynamics 365 Guides program. The program’s content was delivered to users through a Microsoft HoloLens 2 device.

Step 3: Operation Plan

The previously designated simulation laboratory was successfully used during the AR-based educational program. The allocated space for the program proved sufficient, with each trainee having access to the minimum 3×3 meters space as planned [40]. This spatial arrangement allowed for unimpeded movement and interaction with the AR components, which was critical for the immersive learning experience.

In practice, the ventilator and ECMO machine simulations were conducted without any spatial constraints, enabling a total of 28 trainees to complete the training per the session schedules. The effective use of space was evidenced by the trainees’ ability to perform the necessary tasks and their reported comfort level during the training sessions.

Stage 5: Implementation

The AR-based educational program platform was operational for a period of 2 months, with education sessions scheduled from 9 AM to 5 PM. This schedule allowed nurses to select their preferred date and time within this interval. To facilitate the program’s implementation, we used five HoloLens 2 devices, along with two laptops for supervisor screen connections and two large screens for the research environment. Throughout the research, a total of 22 trainees actively engaged in the education sessions. The trainees’ screens, as viewed through the HoloLens 2 devices, were immediately visible to the trainer, enabling real-time progress monitoring. Additionally, trainees were encouraged to request assistance if they encountered any difficulties during the session.

Stage 6: Evaluation

Participants

Training sessions were conducted by two trainers and two operators for the 28 nurses in the ICU from January 1 to February 3, 2022. Twenty-four nurses participated in the survey, 11 of whom took part in an in-depth interview. They were trained either in ventilator or ECMO machine usage with an even distribution across both groups. The participants’ baseline characteristics are presented in Table 2. The median work experience was 3 (IQR 0-6.25) years with a mean of 3.75 (SD 3.90) years. There was a predominance of female participants (17/24, 71%). All participants belonged to the general nursing field with a slight majority working in the medical ICU compared to the surgical ICU (Table 2).

Table 2.

Demographic and clinical characteristics of the surveyed nurse trainees (N=24).

Characteristics Trainees, n (%)
Method trained on

ECMOa 12 (50)

Ventilator 12 (50)
Sex

Male 7 (29)

Female 17 (71)
Medicine specialty

Internal medicine 15 (63)

Surgical department 9 (38)
Experience (years)

<1 5 (25)

1-2 3 (15)

3-4 2 (10)

5-6 2 (10)

≥7 3 (15)

aECMO: extracorporeal membrane oxygenation.

Additionally, all participants (24/24, 100%) owned smartphones and the majority (23/24, 95.83%) possessed either a tablet PC or laptop. Prior to the instruction, 13 (54%) nurses had previous experience with a head-mounted display.

Comparison of SUS and TAM Scores

In the usability test, the items “I think that I would like to use this system frequently” and “I don’t think the system is unnecessarily complex” received the highest rating of 4.38 out of a possible 5, while the lowest-rated item, “I thought there was too much inconsistency in this system,” received an average score of 1.83. The responses concerning technology acceptance were categorized into four areas according to the TAM: perceived usefulness (PU), perceived ease of use (PEU), perceived enjoyment (PE), and intention to use (IU). The survey included 15 questions scored on a 7-point scale. The item with the highest score was “It is fun to use,” scoring 6.71, while the lowest-rated item was “It is easy to use,” scoring 5.17. In further survey results, factors such as age, sex, department of work, and years of work did not impact satisfaction with the education or usability. All response results for the survey are provided in Multimedia Appendix 1.

Correlation of Usability and Acceptance

Our correlation analysis revealed varying degrees of association between SUS and TAM factors. For instance, there was a strong correlation between PU and IU and a moderate correlation between PE and PU. However, the correlation between PEU and IU was not significant (Table 3).

Table 3.

Correlation between the Technology Acceptance Model–based survey items and System Usability Scale (SUS).

Variable UEa (SUS) PUb PEUc PEd IUe
UE (SUS)

r 1 0.2 0.34 0.15 0.22

P value f .50 .12 .45 .30
PU

r 0.2 1 0.2 0.69 0.76

P value .50 .50 <.001 <.001
PEU

r 0.34 0.20 1
0.31

P value .12 .50 .34 .33
PE

r 0.15 0.69 0.32 1 0.72

P value .45 <.001 .34 <.001
IU

r 0.22 0.76 0.31 0.72 1

P value .30 <.001 0.33 <.001

aUE: user experience.

bPU: perceived usefulness.

cPEU: perceived ease of use.

dPE: perceived enjoyment.

eIU: intention to use.

fNot applicable.

Insights From Participant Interviews

Four participants completed interviews related to their experiences with the AR-based educational program. Key insights from these interviews have been collated and are summarized in Table 4. We present a curated selection of interview responses that most effectively capture the key insights. These selections were thoroughly chosen for their relevance and ability to represent the broader findings of the study.

Table 4.

Trainees’ feedback after augmented reality (AR) implementation in the training program.

Category and subcategory Details
Motivation

Intrinsic motivation Interest in the integration of AR into educational settings; desire among learners for practical experience with medical devices; expectations that AR technology will significantly improve learning efficacy

Reasons for participation Influence of colleague recommendations; curiosity about AR teaching methods; specific needs related to own job

Existing issues Noticing varying standards in educational quality; underscoring the need for improvement; necessity to establish standardized training procedures and protocols; need for training programs to be customized to individual learning styles and needs
Learning preferences

Preferred learning method Balancing traditional (54.5%) and self-directed (45.5%) learning approaches; valuing feedback and interaction in traditional learning; preference for flexibility and pace in self-directed learning

Face-to-face versus nonface-to-face Equilibrium between face-to-face (54.5%) and remote learning (45.5%); diverse preferences shaped by feedback, comfort, and flexibility
Feedback

Practical use Majority opinion holding that AR technology is beneficial for skill development; mixed opinions regarding the real-world applicability of AR in professional settings; varied levels of expectation regarding the use of AR devices in educational contexts

Training experience Recognizing the benefits of AR in providing realistic scenarios, allowing for self-directed learning, and enabling repeated practice; challenges include unfamiliarity with AR, focus on operation over content, and limited interaction

Content and support General satisfaction with AR content amid comparisons to traditional methods; requirement for technical support and assistance in AR training

Comparative analysis and outlook AR’s superiority in learning pace, error identification, and training repetition over conventional methods; challenges in mastering AR operation and content depth; mixed perspectives on AR replacing traditional methods (viewed as supplementary); AR’s efficacy in specific scenarios; considered resource-intensive for broad implementation; potential for enhancing self-directed and iterative learning
Future considerations

Target demographics for AR training Target new nurses, individuals lacking device experience, and department transfers; however, limited relevance for experienced nurses

Benefits of self-learning with AR Reduced pressure, time efficiency, review flexibility; utility in learning uncommon scenarios and repeatable sessions

Concerns with self-learning Limitations of AR training in actual clinical settings; lack of communal learning opportunities in AR environments; concerns over system errors and device quantity limitations

Overall, the evaluation of the AR-based education was positive, with participants indicating that AR could enhance their actual clinical performance. AR technology is particularly well-suited for individuals interested in self-directed or hands-on learning theories. Nurses were found to be open to education using innovative technology. When asked if they needed assistance with the curriculum, no participant responded negatively regarding the content. However, some participants did express a need for help in adapting to new devices and technologies.

Discussion

Principal Results

The principal findings of this study provide valuable insights into the strategic translation of conventional critical nursing education to AR-based education platforms in the use of difficult-to-use medical devices [41]. Through interviews conducted with trainers before program development, the study successfully identified the specific needs and requirements of trainers in critical care nursing education. The study employed AR-based educational technology to enhance self-directed learning and hands-on practice.

In the educational strategies employed, the study leveraged the unique features of AR to facilitate self-directed learning. By offering interactive and self-controlled learning experiences, AR empowered trainees to take ownership of their learning process. The program incorporated instructional materials and modules that allowed learners to explore and acquire knowledge at their own pace, fostering a sense of autonomy and self-guided learning. The use of AR also enabled real-time feedback and assessment, allowing learners to track their progress and identify areas for improvement [29].

Through the overlay of 3D objects and virtual models onto real-world settings, trainees engaged in simulated scenarios closely resembling authentic ICU environments. This hands-on component of the program enabled learners to apply their knowledge and skills in realistic contexts, promoting an understanding of the subject matter and the development of critical thinking and problem-solving abilities.

This study highlights how AR technology significantly contributes to the success of self-directed learning and hands-on practice. The utilization of AR technology facilitates active engagement, learner-centeredness, and skill development, thereby enhancing the overall effectiveness of critical care nursing education. Moreover, we provide useful insights based on the perspectives of trainers and operators of the platform. The inherent nature of education often necessitates a lower number of educators compared to learners. The main strength of our study thus lies in presenting an infrequent perspective of educators, a viewpoint seldom encountered within the large-scale hospital setting.

Comparison With Prior Work

AR technology has been extensively explored in areas related to nursing education [42] such as surgical simulation [43], anatomy education [20], and patient safety education [44]. However, it is worth noting that this study represents the first investigation into the use of AR for replacing a conventional educational program in the use of difficult-to-use devices such as an ECMO machine and mechanical ventilator specifically within the critical care nursing field. By incorporating AR into these fields, this study pioneers the integration of innovative approaches in nursing education.

Limitations

This study, being characteristic of a pilot study to identify and apply new educational methods, has the limitation of a restricted number of participants. In further research, a larger sample size could be recruited to identify factors influencing user acceptability and to enhance usability, leveraging insights for more effective implementation.

Implications

This study highlights the implications of AR in future research and practice. The findings suggest the need for longitudinal studies to assess AR’s long-term impact on clinical performance and patient outcomes, and to explore its scalability and cost-effectiveness compared to traditional training. Practically, the results of our study indicate that institutions adopting AR should invest in technical support and training and consider integrating AR as a supplementary tool in curricula for a blended learning approach.

Conclusions

This study provides insights on the development, launch, and operation of an AR-based medical educational program. The study suggests that an AR-based educational program can be an alternative to compensate for insufficient resources for conventional critical care nursing education. Further research can be conducted to compare the effectiveness and feasibility of this program with other AR-based educational programs and traditional nursing educational programs.

Acknowledgments

This research was supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HI23C038700). We would like to express our sincere gratitude to all the individuals who contributed to the successful completion of this study. Primarily, we would like to thank the nursing education chiefs for their valuable insights and participation in the interviews. We are also grateful to the nurses who willingly participated in the study, sharing their experiences and providing valuable feedback. Additionally, we extend our appreciation to the operators who assisted in the implementation of the augmented reality–based educational program. The authors would like to thank Seungheon Choo for his invaluable assistance with the English proofreading of this manuscript.

Abbreviations

AR

augmented reality

ECMO

extracorporeal membrane oxygenation

ICU

intensive care unit

IU

intention to use

PE

perceived enjoyment

PEU

perceived ease of use

PU

perceived usefulness

SUS

System Usability Scale

TAM

Technology Acceptance Model

VR

virtual reality

Multimedia Appendix 1

All response results for the System Usability Scale and survey based on the Technology Acceptance Model.

Footnotes

Conflicts of Interest: None declared.

References

  • 1.Lockee BB. Online education in the post-COVID era. Nat Electron. 2021 Jan 25;4(1):5–6. doi: 10.1038/s41928-020-00534-0. [DOI] [Google Scholar]
  • 2.Gause G, Mokgaola IO, Rakhudu MA. Technology usage for teaching and learning in nursing education: an integrative review. Curationis. 2022 Jun 15;45(1):e1–e9. doi: 10.4102/curationis.v45i1.2261. https://europepmc.org/abstract/MED/35792609 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Robbins DE. The effect of hands-on practice on the skills proficiency of nursing home nurses. Int J Stud Nurs. 2020;5(1):39. doi: 10.20849/ijsn.v5i1.717. [DOI] [Google Scholar]
  • 4.Örs M. The self-directed learning readiness level of the undergraduate students of midwife and nurse in terms of sustainability in nursing and midwifery education. Sustainability. 2018 Oct 07;10(10):3574. doi: 10.3390/su10103574. [DOI] [Google Scholar]
  • 5.Johnsen HM, Briseid HS, Brodtkorb K, Slettebø Å, Fossum M. Nursing students' perceptions of combining hands-on simulation with simulated patients and a serious game in preparing for clinical placement in home healthcare: a qualitative study. Nurse Educ Today. 2021 Feb;97:104675. doi: 10.1016/j.nedt.2020.104675. https://linkinghub.elsevier.com/retrieve/pii/S0260-6917(20)31525-2 .S0260-6917(20)31525-2 [DOI] [PubMed] [Google Scholar]
  • 6.Noble C, Billett S, Hilder J, Teodorczuk A, Ajjawi R. Enriching medical trainees' learning through practice: a video reflexive ethnography study protocol. BMJ Open. 2019 Aug 22;9(8):e031577. doi: 10.1136/bmjopen-2019-031577. https://bmjopen.bmj.com/lookup/pmidlookup?view=long&pmid=31444194 .bmjopen-2019-031577 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Moro C. Utilizing the metaverse in anatomy and physiology. Anat Sci Educ. 2023 Jan 05;16(4):574–581. doi: 10.1002/ase.2244. [DOI] [PubMed] [Google Scholar]
  • 8.Patel ST, Shah S, Sood RP, Siddiqui Z, McKay-Davies I. The implementation of virtual clinical skills teaching in improving procedural confidence in ENT trainees. Adv Med Educ Pract. 2021;12:965–969. doi: 10.2147/AMEP.S322965. https://europepmc.org/abstract/MED/34475794 .322965 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Ordov K, Madiyarova A, Ermilov V, Tovma N, Murzagulova M. New trends in education as the aspect of digital technologies. Int J Mech Eng Technol. 2019;10(2):1319–1330. https://tinyurl.com/2vjhw84y . [Google Scholar]
  • 10.Dwivedi YK, Hughes L, Baabdullah AM, Ribeiro-Navarrete S, Giannakis M, Al-Debei MM, Dennehy D, Metri B, Buhalis D, Cheung CM, Conboy K, Doyle R, Dubey R, Dutot V, Felix R, Goyal D, Gustafsson A, Hinsch C, Jebabli I, Janssen M, Kim Y, Kim J, Koos S, Kreps D, Kshetri N, Kumar V, Ooi K, Papagiannidis S, Pappas IO, Polyviou A, Park S, Pandey N, Queiroz MM, Raman R, Rauschnabel PA, Shirish A, Sigala M, Spanaki K, Wei-Han Tan G, Tiwari MK, Viglia G, Wamba SF. Metaverse beyond the hype: multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. Int J Inf Manage. 2022 Oct;66:102542. doi: 10.1016/j.ijinfomgt.2022.102542. [DOI] [Google Scholar]
  • 11.Zhang X, Chen Y, Hu L, Wang Y. The metaverse in education: definition, framework, features, potential applications, challenges, and future research topics. Front Psychol. 2022 Oct 11;13:1016300. doi: 10.3389/fpsyg.2022.1016300. https://europepmc.org/abstract/MED/36304866 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Dinh A, Tseng E, Yin AL, Estrin D, Greenwald P, Fortenko A. Perceptions about augmented reality in remote medical care: interview study of emergency telemedicine providers. JMIR Form Res. 2023 Mar 28;7:e45211. doi: 10.2196/45211. https://formative.jmir.org/2023//e45211/ v7i1e45211 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Rochlen LR, Levine R, Tait AR. First-person point-of-view-augmented reality for central line insertion training: a usability and feasibility study. Simul Healthc. 2017 Feb;12(1):57–62. doi: 10.1097/SIH.0000000000000185. https://europepmc.org/abstract/MED/27930431 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Lateef F. Simulation-based learning: just like the real thing. J Emerg Trauma Shock. 2010 Oct;3(4):348–352. doi: 10.4103/0974-2700.70743. http://www.onlinejets.org/article.asp?issn=0974-2700;year=2010;volume=3;issue=4;spage=348;epage=352;aulast=Lateef . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Real FJ, Whitehead M, Rosen BL, Meisman A, Crosby LE, Klein MD, Herbst RB. Assessment of competency-based behavioral health anticipatory guidance skills among pediatric residents: the role of virtual reality. J Technol Behav Sci. 2021 Mar 18;7(1):115–124. doi: 10.1007/s41347-021-00202-w. [DOI] [Google Scholar]
  • 16.Samadbeik M, Aslani N, Maleki M, Garavand A. Acceptance of mobile health in medical sciences students: applying technology acceptance model. Inform Med Unlocked. 2023;40:101290. doi: 10.1016/j.imu.2023.101290. [DOI] [Google Scholar]
  • 17.Chiang D, Huang C, Cheng S, Cheng J, Wu C, Huang S, Yang Y, Yang L, Kao S, Chen C, Shulruf B, Lee FY. Immersive virtual reality (VR) training increases the self-efficacy of in-hospital healthcare providers and patient families regarding tracheostomy-related knowledge and care skills: a prospective pre-post study. Medicine. 2022 Jan 14;101(2):e28570. doi: 10.1097/MD.0000000000028570. https://europepmc.org/abstract/MED/35029229 .00005792-202201140-00059 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Radianti J, Majchrzak TA, Fromm J, Wohlgenannt I. A systematic review of immersive virtual reality applications for higher education: design elements, lessons learned, and research agenda. Comput Educ. 2020 Apr;147:103778. doi: 10.1016/j.compedu.2019.103778. [DOI] [Google Scholar]
  • 19.Heo S, Moon S, Kim M, Park M, Cha WC, Son MH. An augmented reality-based guide for mechanical ventilator setup: prospective randomized pilot trial. JMIR Serious Games. 2022 Jul 22;10(3):e38433. doi: 10.2196/38433. https://games.jmir.org/2022/3/e38433/ v10i3e38433 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Baashar Y, Alkawsi G, Ahmad WNW, Alhussian H, Alwadain A, Capretz LF, Babiker A, Alghail A. Effectiveness of using augmented reality for training in the medical professions: meta-analysis. JMIR Serious Games. 2022 Jul 05;10(3):e32715. doi: 10.2196/32715. https://games.jmir.org/2022/3/e32715/ v10i3e32715 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Aebersold M, Dunbar D. Virtual and augmented realities in nursing education: state of the science. Annu Rev Nurs Res. 2020 Dec 01;39(1):225–242. doi: 10.1891/0739-6686.39.225.39/1/225 [DOI] [PubMed] [Google Scholar]
  • 22.Heidari H, Beni ZHM, Deris F. Using Kern model to design, implement, and evaluate an infection control program for improving knowledge and performance among undergraduate nursing students: a mixed methods study. BMC Med Educ. 2023 Oct 25;23(1):795. doi: 10.1186/s12909-023-04775-3. https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-023-04775-3 .10.1186/s12909-023-04775-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Sweet LR, Palazzi DL. Application of Kern's six-step approach to curriculum development by global health residents. Educ Health. 2015;28(2):138–141. doi: 10.4103/1357-6283.170124.EducHealth_2015_28_2_138_170124 [DOI] [PubMed] [Google Scholar]
  • 24.Uymaz P, Uymaz AO. Assessing acceptance of augmented reality in nursing education. PLoS One. 2022 Feb 17;17(2):e0263937. doi: 10.1371/journal.pone.0263937. https://dx.plos.org/10.1371/journal.pone.0263937 .PONE-D-21-32878 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Elo S, Kääriäinen M, Kanste O, Pölkki T, Utriainen K, Kyngäs H. Qualitative content analysis. SAGE Open. 2014 Feb 11;4(1):215824401452263. doi: 10.1177/2158244014522633. [DOI] [Google Scholar]
  • 26.Dumpit DZ, Fernandez CJ. Analysis of the use of social media in Higher Education Institutions (HEIs) using the Technology Acceptance Model. Int J Educ Technol High Educ. 2017 Mar 1;14(1):5. doi: 10.1186/s41239-017-0045-2. [DOI] [Google Scholar]
  • 27.Papakostas C, Troussas C, Krouska A, Sgouropoulou C. Exploring users’ behavioral intention to adopt mobile augmented reality in education through an extended Technology Acceptance Model. Int J Hum Comput Interact. 2022 Apr 17;39(6):1294–1302. doi: 10.1080/10447318.2022.2062551. [DOI] [Google Scholar]
  • 28.Carroll JK, Yancey AK, Spring B, Figueroa-Moseley C, Mohr DC, Mustian KM, Sprod LK, Purnell JQ, Fiscella K. What are successful recruitment and retention strategies for underserved populations? Examining physical activity interventions in primary care and community settings. Transl Behav Med. 2011 Jun 12;1(2):234–251. doi: 10.1007/s13142-011-0034-2. https://europepmc.org/abstract/MED/24073048 .34 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Chaghari M, Saffari M, Ebadi A, Ameryoun A. Empowering education: a new model for in-service training of nursing staff. J Adv Med Educ Prof. 2017 Jan;5(1):26–32. https://europepmc.org/abstract/MED/28180130 . [PMC free article] [PubMed] [Google Scholar]
  • 30.Al-Ansi AM, Jaboob M, Garad A, Al-Ansi A. Analyzing augmented reality (AR) and virtual reality (VR) recent development in education. Soc Sci Humanit Open. 2023;8(1):100532. doi: 10.1016/j.ssaho.2023.100532. [DOI] [Google Scholar]
  • 31.Lau ST, Siah CJR, Loh WL, Rusli KDB, Schmidt LT, Lim FP, Liaw SY. Enhancing professional competency in clinical procedures using head-mounted display virtual reality - a mixed method study. Med Educ Online. 2023 Dec 05;28(1):2232134. doi: 10.1080/10872981.2023.2232134. https://europepmc.org/abstract/MED/37406175 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Davis FD. A technology acceptance model for empirically testing new end-user information systems: theory and results. Doctoral dissertation. MIT Sloan School of Management, MIT Libraries. 1986. [2024-04-17]. https://dspace.mit.edu/handle/1721.1/15192 .
  • 33.Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quart. 1989 Sep;13(3):319. doi: 10.2307/249008. [DOI] [Google Scholar]
  • 34.Brooke J. SUS: a quick and dirty usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, McClelland AL, editors. Usability Evaluation In Industry. Boca Raton, FL: CRC Press, Taylor & Francis Group; 1996. pp. 189–194. [Google Scholar]
  • 35.Cheah W, Mat Jusoh N, Aung MMT, Ab Ghani A, Mohd Amin Rebuan H. Mobile technology in medicine: development and validation of an adapted System Usability Scale (SUS) questionnaire and modified Technology Acceptance Model (TAM) to evaluate user experience and acceptability of a mobile application in MRI safety screening. Indian J Radiol Imaging. 2023 Jan 07;33(1):36–45. doi: 10.1055/s-0042-1758198. http://www.thieme-connect.com/DOI/DOI?10.1055/s-0042-1758198 .IJRI-22-3-1470 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Wahdain EA, Ahmad MN. User acceptance of information technology: factors, theories and applications. J Inf Syst Res Innov. 2014 Jan;6(1):17–25. https://seminar.utmspace.edu.my/jisri/download/F_Vol6Feb2014_FullPaper/Pub3_UserAcceptanceofIT.pdf . [Google Scholar]
  • 37.Kim S, Garrison G. Investigating mobile wireless technology adoption: an extension of the technology acceptance model. Inf Syst Front. 2008 Apr 9;11(3):323–333. doi: 10.1007/s10796-008-9073-8. [DOI] [Google Scholar]
  • 38.Martínez-Caro E, Cegarra-Navarro J, Solano-Lorente M. An extension of the technology acceptance model in hospital-in-the-home units. In: Cruz-Cunha MM, Miranda IM, Gonçalves P, editors. Handbook of Research on ICTs and Management Systems for Improving Efficiency in Healthcare and Social Care. Hershey, PA: IGI Global; 2013. pp. 1191–1207. [Google Scholar]
  • 39.Chuenyindee T, Montenegro LD, Ong AKS, Prasetyo YT, Nadlifatin R, Ayuwati ID, Sittiwatethanasiri T, Robas KPE. The perceived usability of the learning management system during the COVID-19 pandemic: integrating system usability scale, technology acceptance model, and task-technology fit. Work. 2022;73(1):41–58. doi: 10.3233/WOR-220015.WOR220015 [DOI] [PubMed] [Google Scholar]
  • 40.Zorzal ER, Campos Gomes JM, Sousa M, Belchior P, da Silva PG, Figueiredo N, Lopes DS, Jorge J. Laparoscopy with augmented reality adaptations. J Biomed Inform. 2020 Jul;107:103463. doi: 10.1016/j.jbi.2020.103463. https://linkinghub.elsevier.com/retrieve/pii/S1532-0464(20)30091-5 .S1532-0464(20)30091-5 [DOI] [PubMed] [Google Scholar]
  • 41.AlGerafi MAM, Zhou Y, Oubibi M, Wijaya TT. Unlocking the potential: a comprehensive evaluation of augmented reality and virtual reality in education. Electronics. 2023 Sep 20;12(18):3953. doi: 10.3390/electronics12183953. [DOI] [Google Scholar]
  • 42.Tang YM, Chau KY, Kwok APK, Zhu T, Ma X. A systematic review of immersive technology applications for medical practice and education - Trends, application areas, recipients, teaching contents, evaluation methods, and performance. Educ Res Rev. 2022 Feb;35:100429. doi: 10.1016/j.edurev.2021.100429. [DOI] [Google Scholar]
  • 43.Williams MA, McVeigh J, Handa AI, Lee R. Augmented reality in surgical training: a systematic review. Postgrad Med J. 2020 Sep 30;96(1139):537–542. doi: 10.1136/postgradmedj-2020-137600.postgradmedj-2020-137600 [DOI] [PubMed] [Google Scholar]
  • 44.Chen P, Liou W. The effects of an augmented reality application developed for paediatric first aid training on the knowledge and skill levels of nursing students: An experimental controlled study. Nurse Educ Today. 2023 Jan;120:105629. doi: 10.1016/j.nedt.2022.105629.S0260-6917(22)00365-3 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Multimedia Appendix 1

All response results for the System Usability Scale and survey based on the Technology Acceptance Model.


Articles from JMIR Serious Games are provided here courtesy of JMIR Publications Inc.

RESOURCES