Enhancing professional competency in clinical procedures using head-mounted display virtual reality - a mixed method study

ABSTRACT

Background

The maintenance of nursing professional competency is essential to ensure patients’ health outcomes. With the current shortage of nursing workforce, a novel approach is necessary to refresh clinical skills and update practice.

Objective

This study aims to examine the effectiveness of using head-mounted display virtual reality to refresh knowledge and skills and explore nurses’ perceptions towards using this technology for refresher training.

Design

A pre-test post-test mixed-method experimental design was employed.

Results

Participants (n = 88) were registered nurses with a diploma in nursing. The intravenous therapy and subcutaneous injection procedures were implemented using head-mounted display virtual reality. The study showed significant improvement in knowledge for the procedures, cognitive absorption, online readiness, self-directed learning, and motivation for learning. In the qualitative focus group discussions, three themes were identified using thematic analyses: enjoyable way to refresh clinical knowledge; learning outside classroom and limitations in maneuver.

Conclusion

Using head-mounted display virtual reality is promising in refreshing clinical skills for nurses. Training and refresher courses can explore using this novel technology, which may be a viable alternative to ensure professional competence with reduced manpower and resources used by the healthcare institution.

Introduction

Immersive technologies, such as virtual reality and mixed reality, have been increasingly used in recent years as novel tools for teaching and training in medical education [1]. Immersive head-mounted devices (HMDs) are beneficial based on their flexibility, affordability, and portability nature [1]. Immersive virtual reality (IVR) devices consist of interactive computer simulators that sense the user’s position and actions and provides sensory stimuli. It allows the user to explore and manipulate a computer-generated 3D multimedia sensory environment to augment feedback [2]. VR utilizes 3D graphic systems in combination with various interface devices to allow learners to immerse within authentic contexts with more intuitive, real-time interactivity and problem solving of real situation capability [3]. The use of VR technology has emerged as a pedagogy to teach and improve the performance of clinical procedures in nursing education [4]. It provides learners a safe environment to develop, refine, and practice clinical procedures without the risk of harming patients [5].

Traditionally, nurses acquire their skills and knowledge of clinical procedures through repeated practice and eventually perform in actual clinical settings under the supervision of clinical trainers. Although many clinical skills are taught in this method, the retention of knowledge and skills has waned over time [6]. With the current global shortage of nurses [7], the workload for trained nurses has increased, thereby significantly affecting the time spent by clinical trainers for refresher training due to many pressing priorities in dynamic clinical settings. Nurses also experience anxiety as they perform clinical procedures in actual clinical settings, posing an increased risk for errors [8]. However, with staff who are not well trained or have no refresher course for skill competencies, the method will affect patient safety and outcomes. Most adverse effects are preventable [9]. Innovative technologies should be adopted to support the development of novice nurses’ competency in performing clinical procedures. Using IVR can potentially increase the retention of learning [10].

The concept of ‘practice makes perfect’ is not a new notion. However, the practice needs to be deliberate to make it perfect. Deliberate practice involves repetitive performance of a cognitive or psychomotor task together with rigorous skills assessment in a dedicated domain [11]. Using a constructivism approach, the educators help the learners learn through actual experience, build on their existing knowledge, and incorporate new information. Learners engage in the constructs of virtual reality, such as immersion, interaction, imagination and motivation [3]. The learners also gain new knowledge as well as ability to analyze, reflect, and problem solve in a cyclical manner for self-improvement [12,13]. It was found that learners were motivated to learn more in a safe, convenient environment with hands-on experience. They reported satisfaction, self-efficacy and engagement when practicing with immersive technologies [14].

Factors common to effective learning include provision of feedback, deliberate practice, and practice at varying levels of difficulties [15]. Deliberate practice provides learners a task with a well-defined goal, timely feedback, and ample opportunities for repetition and refinement of performance. A scoping review reported that IVR has the potential to increase the confidence of learners [5]. Using IVR for repeated practice has resulted in superior surgical proficiency over standard training [16,17]. In a recent systematic review, VR training was found to be effective in increasing knowledge and was efficacious for deliberate practice in clinical skills [18]. However, further studies with rigorous methodologies were recommended to strengthen evidence on the effectiveness of IVR training [5].

In Singapore, nursing has multiple educational pathways that lead to an entry-level license to practice. One of the pathways offered to become a registered nurse is the Diploma in Nursing Program. The primary goals of nursing education are to prepare nurses to meet diverse patient needs, function as leaders, and advance science that benefits patients and the capacity of health professionals to deliver safe, quality patient care. Registered nurses who continue to pursue academic level may complete the full-time Bachelor of Nursing Program at a local university for an additional 2 years with the advanced placement credit [19]. Although the registered nurses have learnt the core clinical procedures during their diploma in nursing program, most of them do not have plenty of opportunities for hands-on practice and lack retention of key concepts. Therefore, refreshing skills during clinical laboratory classes is essential to ensure competency.

The present work is a follow-up study based on the two prototypes developed in 2021 for clinical procedures: intravenous therapy and subcutaneous injections. For registered nurses who already had prior nursing education, IVR may be a potential pedagogy to their knowledge and skills in procedures. The interactivity of IVR motivates and engages learners in an immersive environment for practice. With experiential learning and built-in timely feedback, learners will be able to work on areas that they need to improve. They may select either the practice or assessment modes depending on their level of confidence through self-analysis and reflection and make meaningful learning to achieve competence. To our best knowledge, VR technologies are increasingly used in trainings with student populations, and their use for teaching registered nurses is uncommon. Therefore, the effectiveness of using VR to improve the knowledge, skills, and confidence in clinical procedures of registered nurses should be explored. This study aims to examine the effectiveness of IVR to refresh knowledge and skills and to explore nurses’ perceptions towards using IVR for refresher training.

Methods

This study adopted a sequential explanatory mixed-method experimental design. Quantitative data was collected from the participants, which informed the qualitative interviews using focus group discussions to provide an in-depth understanding of how nursing learners perceived the use of IVR in their learning journey.

Participants

The participants consisted of nursing graduates of the Diploma in Nursing who were registered nurses pursuing a degree in nursing for upgrading. Hence, the targeted participants were undergraduates who enrolled in the two years full-time Bachelor of Science (nursing) program (n = 108) at a university in Singapore. These undergraduates had prior knowledge of the clinical procedures, but most of them had no actual working experience in clinical settings. Data was excluded for participants who withdrew from the study, did not complete the IVR intervention, or were lost to follow-up. Learners who refused to participate were excluded from the study.

Outcome measures

Participants’ demographic information on age, gender, ethnicity, and past VR experience were collected. A pre-test and post-test 20-items knowledge questionnaires were administered to the participants. The questionnaire was generated from a pool of multiple-choice assessment questions on intravenous therapy (10-items) and subcutaneous injection (10-items). The overall knowledge score ranges from 0 to 20 with higher scores indicating better understanding of the knowledge on the two procedures.

In addition, participants’ opinion about the learning experience with IVR were examined with three scales. The Perception Scales, which were adapted from the study of Roca et al. [20], analyzes the participants’ perception of using IVR as a learning tool for clinical skills. It consists of the following subscales: (i) usefulness, (ii) cognitive absorption, (iii) ease of use, (iv) system quality, (v) confirmation, (vi) satisfaction, and (vii) continuance intention. All items were rated on a five-point scale (1 = strongly disagree to 5 = strongly agree) except for item 1 on the system quality subscale which was scored reversely (1 = 5, 2 = 4, 3 = 3, 4 = 2, 5 = 1). The subscales adapted had Cronbach alphas of 0.81 (usefulness), 0.94 (cognitive absorption), 0.93 (ease of use), 0.68 (system quality), 0.96 (confirmation), 0.96 (satisfaction), and 0.91 (continuance intention). The overall Cronbach’s alpha was 0.97, which signified high internal consistency.

The Online Learning Readiness Scale, which was adapted from the study of Hung et al. [21], focused on the preparedness of the participants in using IVR for learning purposes. It is composed of three subscales: (i) self-directed learning, (ii) learner control, and (iii) motivation/attitude. Items are evaluated on a five-point scale (1 = strongly disagree to 5 = strongly agree). The subscales adapted had Cronbach alphas of 0.91 (self-directed learning), 0.79 (learner control), and 0.95 (motivation/attitude). The scale showed high reliability with an overall Cronbach’s alpha of 0.95. The General Self-Efficacy Scale is a 10-item self-report measure that examined a person’s belief in their problem solving and goal-oriented abilities [22]. Items are considered on a four-point scale (1 = not at all true to 4 = exactly true). Participants had to rate themselves on items like ‘I can usually handle whatever comes my way.’ The scale had a Cronbach alpha of 0.84 in this study.

Procedures

The IVR intervention was a nursing game designed by the university’s in-house information technology team by using Meta Quest 2 VR headsets. It consists of two clinical procedural skills: intravenous therapy and subcutaneous injection. The version made available to the study participants was developed after a series of changes through a user acceptability test with a separate group of learners. The latest version contained prompts and checklists for the participants to familiarize with the VR setting and to execute knowledge that they were exposed to in their tutorial and laboratory sessions. The VR headsets were purchased by the university and loaned to the students.

The study was conducted in a university. A briefing, which included an introduction to using the VR headset, and questionnaires were administered by research assistants. The potential participants were informed prior to the study that they would be compensated for the time and effort upon completion of the questionnaires and focus group discussions. Those who consented to participate completed the 20-minute pre-intervention questionnaire through Qualtrics. Learners who declined to use the VR headsets or participate in the study were excluded. After the pre-intervention questionnaire, participants were oriented to use the VR headset and controllers. Participants were given at least one week of practice to try both procedures. Upon returning of the VR headsets and controllers, the participants completed the post-intervention questionnaire via Qualtrics again.

Quantitative data analysis

Data analysis was performed through SPSS version 27. For categorical data, descriptive statistics were used to describe the demographic data presented as frequency distributions; for continuous data, mean (M) and standard deviation (SD) were used. The scores for the pre-post tests were summarised to have uniform units for comparison. Paired sample t-tests were conducted to evaluate differences in the knowledge, perception, online learning readiness and self-efficacy levels between the pre- and post-tests.

Qualitative data collection and analysis

Participants were invited to share their perception and experience on using IVR technology to refresh their clinical competencies on intravenous therapy and subcutaneous injection through focus group discussion (FGD) sessions. FGD sessions were conducted remotely in English using teleconferencing technology, which lasted between 60 to 70 minutes. The first and third authors conducted the FGD sessions by using a semi-structured interview guide, and a research assistant took field notes to document the contextual information [23]. The FGD sessions were audio recorded and transcribed. All transcriptions are in English. A six-phase thematic analysis was used [24]. The first and third authors checked the transcriptions against the audio recording to familiarize themselves for accuracy prior to data analysis. They independently extracted meaningful quotes, which were cross checked by the second author to enhance trustworthiness and dependability. An audit trail was conducted, which noted the decisions made during the research process, thus providing a guideline for interpretation and synthesis. After that, the reviewers contemplated and agreed on the themes.

Ethical consideration

Potential participants were provided with information sheets, and the research assistant explained the purpose of the study and invited learners’ participation. They were assured that participation in the study was voluntary, and consent was obtained prior to any data collection. Confidentiality was ensured because codes were used for data collection and analyses. The approval from the university’s Institutional Review Board was obtained prior to the commencement of the study.

Results

All 108 nursing learners who enrolled for the module were eligible and participated in the study. One participant withdrew and 19 participants did not complete the study (Figure 1). A total of 88 (81.5%) participated in the study. Data collected via questionnaires between September to December 2021 from the participants were retrieved. The participants comprised 5 males and 83 females aged between 19 and 34 years (M = 21.25, SD = 2.14). Among the participants, 69 (78.4%) were Chinese, 12 (13.6%) were Malay, 3 (3.4%) were Indian, and 4 (4.5%) belonged to other races. Only 2 (2.3%) participants had experience with using VR.

Figure 1.

Flow diagram of data collection process.

A paired sample t-test was used to compare the pre- and post-scores of the measures used (Table 1). For the knowledge quizzes, statistically significant improvements were detected for intravenous therapy (p = 0.008) and subcutaneous injection (p = 0.003). The overall knowledge also significantly improved (p < 0.001). Hence, the IVR nursing game was an effective strategy in refreshing learners’ knowledge on clinical procedures.

Table 1.

Comparison of test scores before and after IVR clinical procedures.

Pre-post-tests	Pre-test Mean (SD)	Post-test Mean (SD)	t	p
Knowledge	12.53 (2.35)	13.38 (2.24)	4.34	<.001*
Intravenous therapy	6.32 (1.56)	6.69 (1.65)	2.71	.008*
Subcutaneous injection	6.22 (1.50)	6.68 (1.22)	3.04	.003*
Perception scales	86.06 (12.55)	87.53 (14.65)	0.92	.359
Usefulness	1.55 (1.84)	1.41 (2.39)	−0.53	.599
Cognitive absorption	17.74 (3.09)	18.75 (3.64)	2.61	.011*
Ease of use	1.33 (1.71)	1.07 (2.23)	−0.94	.351
System quality	16.77 (1.99)	17.49 (2.36)	2.67	.009*
Confirmation	1.30 (1.84)	1.70 (2.26)	1.64	.104
Satisfaction	1.26 (1.96)	1.47 (2.47)	0.73	.466
Continuance intention	1.11 (2.09)	9.65 (2.64)	−1.58	.117
Online learning readiness scale	47.89 (7.06)	49.77 (5.94)	2.77	.007*
Self-directed learning	18.19 (3.06)	18.99 (2.82)	2.51	.014*
Learner control	1.88 (1.70)	11.15 (1.59)	1.47	.147
Motivation/attitude	18.82 (3.14)	19.64 (2.50)	2.52	.014*
General self-efficacy scale	29.82 (2.83)	3.28 (2.68)	1.43	.156

Note. *= p < .05. SD = Standard Deviation. Paired samples t test.

Participants showed significant changes for two of the seven components under the Perception Scales. Statistically significant improvement was noted for cognitive absorption (p = 0.011), and system quality (p = 0.009). No significant changes were detected for participants’ perception of usefulness, ease of use, confirmation, satisfaction, and continuance intention before and after they experienced the IVR intervention.

For readiness on online learning using IVR, the participants reported significant improvements in their self-directed learning (p = 0.014) and motivation or attitude towards IVR (p = 0.014). The overall online learning readiness also significantly improved (p = 0.007). Although the mean score for learner’s control increased, it was not significant. For general self-efficacy, improvements were detected although not significant.

Three FGD sessions were held, with each lasting about 60 minutes (M = 56.94, SD = 0.46). Each FGD session consisted of four to five participants, with a total of 14 participants. Two FGD sessions had five participants, and the other had four participants. Participants were all female with age ranging from 20 to 23 years. The participants generally found the new experience in using IVR for learning as fun and effective to improve their clinical knowledge. They highlighted that they found the IVR experience engaging and refreshing, and they enjoyed the flexibility of using it at their own convenience. The participants valued the realistic virtual environment and felt that the real-time feedback after their practice allowed them to identify areas for improvement. Three themes were identified from the FGD sessions: (i) enjoyable way to refresh clinical knowledge, (ii) learning outside classroom, and (iii) limitations in maneuver.

Enjoyable way to refresh clinical knowledge

The participants reported that they found the novelty of VR to be fun and interesting. They treasured the opportunities to learn from a new approach.

When we take back [brought home] the VR for one week, I just make full use of it. It was quite fun and engaging because it’s not something we do every day. (FGD1, P4)

Several participants said that using IVR helped them to recall the key principles of the procedures and applied them during their practice. This practice would reinforce the key concepts, which they might have forgotten over time.

Maybe we know what are the techniques but we may have forgotten the flow already. Should I do my first check or third check? When should I do it? So, this like gives us very good revision, remind us the key principles such as like how we inject, how we withdraw the insulin, although we already have the experience. (FGD3, P13)

The participants shared that IVR is effective in skills practice because it helped them through visualization. They compared using the IVR intervention with the conventional practice during their Diploma in Nursing program and felt that IVR provided the whole experience. With all the required requisites and guidance from the clinical instructor, they were immersed in the clinical environment and managed to revise their clinical procedures.

… back in polytechnic days, when I want to try to practice for procedures at home, it doesn’t feel right. For example, IV drip I won’t have the bottle of normal saline at home to practice. So, with this VR, I feel that I am really in the school or like in the ward to perform the procedures. (FGD3, P13)

Learning outside classroom

The participants explained that one of the key advantages in using IVR was that they were able to learn without physically attending classes. This flexibility allowed them to strengthen their skills competency after a physical practice class experience.

I think the VR set give us a learning experience outside classroom, like we don’t have to be physically in class [to] learn about these skills. It has to build on the basis that we have already gone through the physical hands-on practice, then the VR can act as a reinforcement tool. (FGD3, P14)

Several participants suggested to integrate IVR into the nursing curriculum. As the device was introduced during the COVID-19 pandemic period, many participants perceived value in IVR technology and indicated its use as part of the curriculum.

I think putting it before our clinical attachment, the activity is relevant, and students might feel more interested in practicing the skills as a revision. In addition, now we only attend lab sessions on alternate weeks due to Covid-19, we can learn the skills for one week, then the following week practice that skill using IVR. (FGD3, P11)

The participants shared that targeted learning strategies enabled them to refine their performance as they were able to correct their mistakes and repeat the practice according to their own learning pace. As such, the technology was beneficial for students with different learning paces.

I think it will, like build up my confidence right because some of the steps I may miss out then it will prompt us to remind us to do the steps. (FGD1, P4)

Most of the participants supported using IVR for revision because their experience and realism of the procedure would boost their confidence in actual practice. They shared that it might not be as effective in increasing the confidence of new students who have no prior clinical experience due to their lack of actual patient experience.

I think for diploma upgraders, it will be useful because we kind of know what’s going on already so like to go through the flow, we will feel more confident. It may not be as useful for new students because if they never tried withdrawing in real life, then they might not feel confident in actual clinical even though they know the process for the procedure. (FGD3, P11)

Limitations in maneuver

Although the participants generally had positive experiences in using IVR, they also reported some limitations. Some of the participants shared that it took time for them to get used to the two hand-held controllers and to maneuver the intended actions because it was not as intuitive in the virtual setting compared to real life.

My main challenge was the delicate movements. For example, like drawing back the syringe and pushing it. It took me 15 minutes to get the hang of it. So that was one of the drawbacks. (FGD1, P5)

Other participants commented that the techniques used by hands in real life was different when compared with technical maneuvers using hand-held controllers. Hence, emphasis was placed on actual hands-on experience to enhance the entire learning experience.

The way when you hold the insulin vial in real life, is different from how you hold in VR. In VR you just press the buttons. In real life you have some techniques to hold and withdraw insulin properly. (FGD1, P3)

Although a few participants reported physical discomfort, such as dizziness, most were able to resolve such sensations when they reduced the use time.

I do have motion sickness like giddiness, but it only happened about 30 minutes after the session, so I had to break it down so in a week I probably tried the VR about three times. (FGD5, P9)

Discussion

This study provided new perspectives in the nursing education design in terms of integrating technology into the traditional refresher course for nursing skills. IVR interventions for intravenous therapy and subcutaneous injections training were developed based on actual clinical scenarios and questionnaires on nursing knowledge. The IVR intervention was promising in improving professional competence as the pre- and post-intervention scores for knowledge showed significant improvement. Considering the global shortage of nurses, increased aging population and disease burden, the results propose a viable alternative to maintain professional competence in clinical procedures by leveraging innovative technologies. Having prior actual hands-on clinical experience may potentially serve as a deterrent to the use of VR for nurses’ training. However, our study findings showed otherwise, that is, the nurses were receptive to the use of technology for training. The participants showed significant improvement in their readiness and motivation for online learning and reported an immersive and enjoyable IVR experience. They shared that learning in this manner was rather fun. This finding corroboratively echoes previous reports on significant improvements in nursing learners’ knowledge level [25,26]. VR yields better results than simulations in terms of post-test knowledge scores, cognitive gain, skill performance scores, and skill success rates [15,16,18,27].

In this study, the participants appreciated the guides and feedback on their performance. For the first practice, the learners were able to select the practice mode to familiarize themselves with the procedures. They could proceed to the assessment mode after gaining some confidence in their skills. This empowered and scaffolded their learning by progressing according to their own pace and self-corrected any mistakes. Other studies also found that IVR provided real-time feedback to allow learners to make corrective actions during the session [4,28]. The present study affirmed the effectiveness of IVR, that is, it enabled learners to learn within a realistic virtual environment and promoted critical thinking within simulated scenarios [25,28]. The participants reported significant increases to their motivation and self-directed learning. One of the reasons could be due to the flexibility of using IVR to refresh their memory in performing procedures at their own time and space. The technology provided opportunities to maintain nurses’ clinical competencies without the added stress of completing the physical refresher course with limited resources in terms of time, manpower, facilities, equipment, and consumables. The technology also improved mental dexterity and gave freedom to the learners to practice in their own time through self-regulated learning [5]. Patterson et al. [29] showed that the adoption of IVR increased learners’ motivation as it provided flexibility and private space for their own learning. The achievement of learning outcomes would depend heavily on the student’s intrinsic drive to participate.

Our study demonstrated an increase in knowledge score after the IVR intervention. However, the amount of change in knowledge score was marginal indicating the need for studies on the effectiveness of IVR. Although the acquisition of this knowledge is a requisite for clinical psychomotor skills [30], the effectiveness of IVR in relation to psychomotor skill acquisition requires further testing. However, several studies have shown that IVR was not effective in improving the psychomotor skills compared to the traditional practical training in nursing education due to the difference between reality and the virtual environment [31,32]. Contrary to the belief that trained nurses may not be attracted to such novel learning strategies, the participants in the present study shared that they found IVR to be effective as they already had prior learning and actual hands-on experience. Hence, they knew what to expect and found the novel approach effective and beneficial. Nurses in this study had prior experience in the procedures, and they were mainly young adults who belonged to the post millennial generation and were tech-savvy [33,34]. The learners’ perception towards technology adoption in their education was a crucial element for the success of using IVR for practice. Despite that most of the learners have no experience in IVR, they reported a positive experience in this study. Although some participants reported challenges like unfamiliarity with the VR hand-held controllers, they explained that these could be addressed by taking the time to get used them.

This study had some limitations. The study was conducted at one university by using a single-group design, hence the results are not generalizable. More studies are required to examine the effectiveness of IVR by comparing intervention and control groups. A larger sample size including more institutions could be employed. The effectiveness of IVR for fresh school leavers with no prior clinical experience should also be explored to determine the feasibility to extend the use of such a novel teaching approach. Future studies should provide more support on device orientation as well as instructional user guide videos on the procedures to better prepare the learners in assimilating to the use of the new device. Lastly, safety measures and maximum usage times should be reinforced to prevent physical discomforts, such as dizziness, for a pleasant experience.

Conclusion

Nurses are receptive to the use of technology for refresher training. IVR training provides opportunities for them to enhance their clinical knowledge and skills by self-regulatory and experiential learning. The findings affirm that IVR enables nurses to actively engage and immerse in the virtual world as well as reflect and conceptualize entire procedures. Further research is needed to explore the effectiveness of IVR on skill acquisition and competence.

Funding Statement

This study was supported by the Ministry of Education, Singapore, Tertiary Education Research Fund 2020 (MOE2020-TRF-053).

Disclosure statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Author contributions

STL and LSY conceptualized and designed the study. STL, CJRS, WLL, KDBR and FPL conducted the data collection. STL, CJRS, WLL and LTS carried out the data management, data analysis and synthesis of data. STL, LSY, CJRS, and WLL wrote the article. All authors reviewed critically, read, and approved the final version of the article.

Data availability statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to restrictions, e.g., privacy or ethical.

Institutional review board statement

This study received ethical approval from the National University of Singapore Institutional Review Board (NUS-IRB-2021-305).