Exploring the application of mixed reality in Nurse education

Abstract

The aim of this study was to explore the contemporary application, inclusive of advantages and challenges, of mixed reality (MR) technology in the education of nursing students and, its contribution to enhanced learning. A descriptive evaluation design was undertaken to explore the learning experience of second year students enrolled in a 3 year Bachelor of Nursing programme. One hundred per cent of the students stated that the experience assisted them in their learning. The key themes of engagement in learning, and developing clinical judgement emerged from students’ responses, and demonstrated ways in which students felt MR enhanced their learning. This emerging technology has the potential to assist in enhancing clinical judgement and developing skills in noticing physical cues in patients. The implementation of MR may also enhance student motivation and engagement with learning.

Introduction

Technology enhanced learning is a key strategy in contemporary healthcare education and can be articulated and operationalised in various forms. The emergence of information technologies has provided educators with a dearth of new resources to facilitate and engage learners in the learning experience. There is however, a lack of clear guidance on their implementation, alignment to pedagogical approaches and impact on teaching effectiveness and learning outcomes.¹ Research into virtual reality (VR) shows clear benefits in medical education in the development of surgical skills^{2 3} and, VR simulation has more recently proved valuable in nurse education to facilitate the application of theoretical knowledge to clinical contexts.^{4 5} This may have important implications for nursing education and may assist with the need to develop nursing students’ ability to be proficient in theoretical knowledge, develop a diverse range of clinical skills and consolidate clinical reasoning.

The advancement in information technologies has seen the evolution of a range of technology. These technologies can be conceptualised as a continuum from VR, where learners are experiencing an environment that is computer generated⁶ to Augmented Reality (AR) in which digital elements are situated within the real world.⁷ Miligram and Kishino⁸ suggest that along the continuum sits a particular subclass of VR related technologies that involve the merging of real and virtual worlds, which are referred to generically as mixed reality (MR). This allows students to be exposed to different ideas and approaches that would otherwise be difficult to visualise and contextualise. Further, it has the potential to promote active engagement and autonomous exploration which may enhance the learners understanding of complex ideas and situational awareness.⁷

This paper focuses on a specific type of MR that allows a holographic image to be placed within the real classroom through a head mounted display (HMD). Here the learner can see the classroom and their colleagues as well as the holographic image. It is important to make this distinction as the literature describing the use of both VR and AR details different subsets of the technology that can vary from the use of fully immersive headsets to mobile telephones, laptops, tablets, and ebooks.⁹ The type of MR described in this study is classified as group 3 within the MR classification⁸ and has not been previously addressed in integrative reviews such as that conducted by Zhu et al.⁹

Background

Simulation is recognised internationally as a safe and effective way to teach nursing students, and has become increasingly important in nursing education as the acuity of patients increase and there is increased pressure on clinical sites to host undergraduate nurses.¹⁰ Further, simulation-based learning provides nursing students with an opportunity to apply knowledge within a safe context while developing clinical judgement approaches to determine patient care needs and goals. Current research into nursing curricula and students preparation for real world employment, suggests that nursing students may not possess the adequate skills and ability to make sound clinical assessments of patients.¹¹ As a result, identifying educational approaches that provides novice nurses with an opportunity to gain experience, learn from mistakes and identify misconceptions within a safe environment, is critical in order for graduates of nursing programmmes to be suitably prepared to care for patients with increasing acuity and complexity.^{12 13}

There is considerable recognition of the impact on learning that simulation provides in nursing education, with research reporting that up to 50% of clinical practice hours within nursing curricula could be potentially replaced with clinical simulation.¹⁴ However, the overuse of some simulation approaches; namely high-fidelity simulation has been criticised as having a detrimental effect on nursing students’ professional development due to their static nature and inability to interact in a humanistic manner.¹⁵ Although it is important that different simulation modalities are chosen for their strengths, Bogossian et al ¹⁶ suggest that nursing programmes face barriers to increasing high fidelity simulation. These limitations have been attributed to staff training, shared resources, best practice and robust evaluation.¹⁶ In the Australian context, simulation is used extensively in nursing curricula to bridge the practice-theory gap and to support the development of critical thinking. Regulatory bodies such as the Australian Nursing and Midwifery Accreditation Council recognise the importance of simulation in the development of nursing graduates, with simulated learning being an essential factor in the accreditation standards for registered nurse programmes.¹⁷

Despite these rapid advancements, how these approaches are adopted and integrated into nursing curricula has not been widely explored. Technology in the form of VR, MR or AR is not the only solution to these barriers, but has the potential to offer an alternative to traditional manikin-based simulation. Interestingly, a study by Howard¹⁸ applying computer-based simulation achieved similar outcomes to high fidelity manikins. The introduction of MR as described in this study presents learners with unique contextual and situated learning experiences, which provides students with the opportunity to safely and comprehensively explore the clinical scenario.

Technology based simulation: VR, MR, AR

AR is an emerging technology that seamlessly bridges the gap between the real world and the virtual world. Lee¹⁹ suggests that the interactive properties of AR within the classroom allows learners to engage with three dimensional computer generated content which has a powerful effect on learning

Studies exploring the educational merit of AR identify improvements in students’ academic performance and motivation, social and collaborative skills along with psychomotor and cognitive skills.^{19 20} However, Martín-Gutiérrez et al ²¹ highlight the absence of pedagogical approaches to optimise the integration of technology into education. They acknowledged the resistance that individuals are likely to have in regard to moving away from more traditional approaches and the opposition to adopt these new approaches, in addition to the cost associated with implementation.²¹ While these aspects are likely to result in some opposition to engaging in new teaching approaches, the potential that this technology has shown as an educational technique in other disciplines should not be ignored.^{6 22}

In a literature review conducted by Lee,¹⁹ disciplines such as engineering, chemistry, history and maths were identified as early adopters of VR and AR. More recently medicine, pharmacy and radiology have adopted the use of VR and AR to teach procedural skills.^{20 23} Zhu et al ⁹ focused a review of AR research in healthcare education and suggested that not only was AR an acceptable learning technology, but it showed potential in developing practioner competence. However, the literature is sparse in relation to the use of MR such as Microsoft Hololens, and little has been done with patient assessment.

Aim

This study sought to explore the contemporary application, inclusive of advantages and challenges of MR technology in the education of nursing students and, its contribution to enhanced learning.

Methods

Study design

A descriptive evaluation design, which surveyed the students’ learning experience with MR was employed for the study.

Sample and setting

Students enrolled in the second year of a 3 year Baccalaureate of Nursing programme, undertaking a core theoretical nursing unit (n=171) were eligible to participate in the 2 hour tutorial. Attendance at the tutorial was not required in order to be eligible to pass the unit of study and the content did not form the basis for an assessment item (figure 1).

Figure 1.

Research recruitment process.

Preliminary information relating to the tutorial was available to the students via a prerecorded lecture and promotional PowerPoint that was uploaded to the online learning management system. On attendance at the tutorial, students were given a brief verbal overview and provided with a participant information sheet to review and provide written consent in order to participate in the study. Students were advised that they could attend and participate in the tutorial and decline to consent to participate in the study, whereby they were not required to complete the questionnaire at the conclusion of the tutorial.

A simulation prebrief was provided to orientate students to the use of the technology and to provide instructions on how to navigate through the MR scenario. Clinical facilitators were present to support the students and troubleshoot any issues that participants encountered.

Hardware

The experience was delivered via Microsoft Hololens; a head mounted wearable holographic headset which permits human-computer interaction within an MR environment. The unit consists of a widescreen stereoscopic head-mounted display containing tinted holographic lenses, which students could wear and adjust to fit their head.

A holographic case study

The application used in this study was Holopatient, developed by Pearson United Kingdom. The application projects a holographic image of a patient, which can be placed in a real world environment. The hologram was developed using volumetric video, where real actors are filmed from multiple angles. In the applied scenario the hologram showed a patient exhibiting signs of anaphylaxis. The holographic object can be viewed from 360° and placed in either a simulated ward environment or simply in a tutorial room (figure 2).

Figure 2.

Depiction of participant engagement with the study.

Participation

Participants were asked to work in pairs and were provided with a brief that the patient had presented to the Emergency Department, and that they were required to undertake a visual assessment to identify key clinical issues and potential clinical assessments. One member of each pair was allocated to wear the Microsoft Hololens and verbalise their objective assessment findings to their colleague, who documented the findings being conveyed. At the conclusion of 30 min time period, the roles within each group were reversed.

Debriefing

At the conclusion of the experience, and in line with contemporary simulation practices, a debriefing session was held. Debriefing is widely recognised as a vital part of the educational experience and the gather, analyse, synthesise method,²⁴ in conjunction with the learning outcomes of the session, were used to structure the debrief. The debrief was conducted by the same facilitator in each session to maintain consistency. The educational experience was concluded by providing the tutorial groups with confirmatory information regarding the patient through a reflective discussion to consolidate the educational experience.

Data collection

At the conclusion of the tutorial participants were asked to complete a 14-item questionnaire (box 1) which included demographic information and open-ended questions which explored the participants learning experience using Microsoft Hololens, including their opinions with regards to the potential for enhancing learning. This questionnaire was developed and peer reviewed by members of both the nursing and education faculties to reduce the risk of bias.

Box 1. 14 Item questionnaire.

• Age

• Gender

• Do you have any experience as a paid worker in healthcare?

• What type of position have you worked in? enrolled nurse, assistant in nursing, health care worker, other

• What are your initial impressions thought and feelings from the simulation?

• What was the experience of participating in the virtual simulation like for you?

• What do you like about experience? What did you not like about the experience?

• Do you think the simulation helped or hindered learning about patient care and assessment?

• How did the application assist you in understanding the concepts associated with the tutorial?

• What recommendations do you have for improvement?

• Do you think the simulation should be offered again?

• Why/why not Do you have any further comments?

Data analysis

Data obtained was analysed via SPSS: IBM V.23 using descriptive statistics pertaining to demographic data. Qualitative data derived from the survey underwent thematic analysis using Braun and Clarke²⁵ phases of analysis (box 2). The coding of the responses was iterative, with responses coded into emerging themes, and then synthesised into small theme matrices. The codes identified were descriptive of the responses in an effort to reduce researcher bias. Participant quotes are displayed below using a number letter schema representative of the tutorial in which they participated (n=1–8) and a letter (A-Z).

Box 2. Braun and Clarke (2006) Phases of Analysis.

1. Familiarizing yourself with your data

2. Generating initial codes

3. Searching for themes

4. Reviewing themes

5. Defining and naming themes

6. Producing the report

Findings

The response rate for the survey was 66% of the total cohort (n=96) (Figure 2). Of the participants who responded, 61.5% of them reported having prior healthcare experience. This question was asked to assess whether having prior experience affected the perception and the relevance of the experience and no difference was noted.

Participants identified a number of unfavourable aspects associated with wearing and interacting with the HMD such as the weight of the headset, slight dizziness and headaches, however, despite these negative aspects, 100% of the students stated that the experience assisted them in their learning. The unfavourable aspects of the HMD described by student’s were in most cases were limited to the time they used the HMD, and were resolved by removing the HMD. Participants were asked about the ways they felt that they had learnt from the simulation. Overarching themes of engagement in learning and developing clinical judgement emerged, with the synthesis of the subthemes of noticing and interpreting, visualising, assessment and reflecting. The common thread in each aspect/theme was that this mode of presenting content engaged learners in a different way (table 1).

Table 1.

Exemplar	Key elements	Subtheme	Themes
‘I think it was a great way to experience a real scenario of examining a patient and really seeing the visual cues applicable to this case study’  (Participant 2F)	Real scenario Experience Visual cues applicable to this case study	Noticing & interpreting cues	Developing clinical judgement
‘visualising a scenario helps creating longer lasting impression in minds of a learner’  (Participant 5C)	Visual Lasting impression/memorable	Visualising in a realistic way	Engagement in learning
‘It really helped to have physical signs of the patient and using knowledge to try and figure out why they are reacting that way, compared to just reading textbooks’ (Participant 8A)	Assessing and interpreting cues Assisted more than a textbook	Assessment	Developing Clinical Judgement
‘Identifying gaps in my assessment technique and the comparison between written and verbal assessments.’	Reflection	Reflecting	Engagement in learning

Engagement in learning

Visualising/Visual learning

Students recognised the importance of being able to see the patient and visualise symptoms. It was also important to the students that the visual cues were represented realistically as these can be sometimes difficult to represent with a manikin   ‘I am a visual learner, I find reading case studies hard… this patient was in front of me to move around and get a clear image’ (Participant 2F).

Students also suggested that this way of presenting content had the potential to be more memorable ‘ visualising a scenario helps creating longer lasting impression in minds of a learner’ (Participant 5C).

Reflecting

Students were able to reflect on their gaps in knowledge, and identify professional language that required development. Students suggested that they were able to ‘Identify gaps in assessment techniques and make a comparison between written and verbal assessments.’ (Participant 1A). Students also highlighted that they learnt from their colleagues ‘I was able to experience and compare how nurses assess.’ (Participant 1C).

Other students suggested that it assisted in developing their confidence as they were able ‘to view a clinical case in a detailed manner without awkward feelings.’ (Participant 3A).

Developing clinical judgement

Noticing and interpreting cues

Students also identified that the experience helped them to develop their interpretation of a situation. They commented that the experience ‘highlighted the importance of further inquiry and not to jump to conclusions’ (Participant 4F) and that it ‘helped me to put the whole picture together and to look closely at the patient’ (Participant 8A). The experience involved assessing the holographic standardised patient and students reported that ‘Not having any background info about the patient really challenged me to think, not only about the condition I was assessing, but how he came to be like that.’ (Participant 6B). Another positive for the students involved was that the experience provided a safe environment in which to learn ‘you can see the patient as a whole and try and make clinical decisions and it doesn’t matter at this stage if they are wrong.’  (Participant 1E).

Students were able to recognise physical cues and observe a patient without the pressure of time or the issues of personal space. As opposed to a real-world clinical experience in which it would become uncomfortable for both the student and patient if a student was observing them so closely for an extended period.

Assessment

Students suggested that the holographic patient assisted them with their assessment skills in the following ways ‘the simulation helped a lot in relation to assessment as it is clear and life like’ (Participant 1C) and it was ‘better than a case study/mannequin’ (Participant 3A). Additionally, students mentioned the assessment of visual cues was enhanced ‘3D gave a more comprehensive picture and the ability to assess visual cues easier than on a mannequin’ (Participant 5F).

When asked the broad question; would you like to see the simulation offered again? One hundred per cent of the participants responded in the affirmative.

Discussion

Simulation in healthcare is widely regarded as both a safe and effective way to expose students to clinical realities and replicate the real world.²⁶ Simulation draws on both active learning^{27 28} and experiential learning theories.^{29 30} Kamphius et al ³¹ suggests that students need a ‘whole’ view of a skill in order to create professional performance in the real world. Components of this whole view were seen in the student’s responses as students were able to learn by exploring and experiencing the holographic image. The experience was authentic, constructive and intentional.

In line with other simulated learning techniques, the use of MR in this study is based in constructivist learning theories.^{27 29 32} However, the emerging educational framework of Paragogy; the practice of peer learning facilitated by technology³³ was also a key consideration in the development of the learning activity, which focused on technology facilitated engagement in learning.

Zhu et al ⁹ suggested that AR can assist in developing spatial relationships, increasing retention of content and performance of skills and tasks, in addition to providing flexibility and authenticity of learning. Along the same technology continuum, MR in this study showed that participants discussed improved learning in these areas. Students suggested that not only was the simulation authentic, but that it improved their spatial awareness and developed their assessment and clinical judgement. They also suggested it was memorable, suggesting perhaps a subjective attractiveness to the way content was delivered. It is however recognised that as an emerging technology this type of MR is still both novel and innovative and that this needs to be taken into consideration. Santos et al ³⁴ however, suggests that the novelty value of the ‘wow’ factor is not the only motivating benefit for learners, and suggest that easing cognitive load and supporting situated experiences were also possible motivating factors.

One of the key aspects of this simulation was the opportunity for the students to develop skills in noticing and visualising physical cues. Noticing is an essential component in development of nursing intuition.³⁵ Tanner¹² suggests that noticing is one of the four key elements to ‘thinking like a nurse’. The opportunity to engage with the MR simulation provided students an opportunity to explore their abilities in clinical assessment, professional communication and deductive reasoning. In addition, the problem-based approach provided scope for students to explore a variety of clinical options, and iterative feedback from staff provided students with insights regarding appropriate dialogue, and the opportunity for students to reflect on their own scope and level of performance. If this is compared with the use of manikins, then one can see the difference in realism and recognising actual symptoms rather than computer generated ones. Dean et al ¹⁵ suggest that while manikins have a place in training student nurses, there is an element missing from manikin-based simulation that may be influencing the perception of caring.

An unexpected benefit of this simulation was the exploration of judgements and personal biases within the cohort. Burgess et al ³⁶ suggests that unconscious biases are ‘habits of mind’, but to prevent these unconscious attitudes and stereotypes from impacting negatively on clinical encounters, strategies and skills need to be developed. It is possible that MR in this format has the potential to visualise, explore and challenge personal biases before they play out in a clinical environment.

The development of simulated experiences using MR provides a contemporary educational approach that prompts the exploration of content in way that is engaging. Additionally, Mather et al ^{37 38} recognised the potential of similar technology in remote learning and teaching. It allows for greater interaction and manipulation of processes for individuals to experiment with, while mitigating the risk of adverse events. This is increasingly important as the nursing profession requires the integration of complex concepts of physiology, adaptive expertise and collaborative practice in order to provide contemporary clinical care. These are challenging concepts to teach and in a Bachelor of Nursing programme due to the limitations in clinical exposure and access to volunteers to participate in clinical workshops and tutorial sessions.

MR in its different forms provides the user a greater capacity to be exposed to ideas that would otherwise be difficult to visualise and contextualise. In turn, it encourages the user to actively engage, foster decision making via autonomous exploration and provides the potential to develop a deeper understanding of complex ideas and situational awareness through the creation of new experiences.

Limitations

This study was conducted using a single cohort of students, in one University and therefore results may not be generalisable. The students were only exposed to the technique once and it is unclear if the positive attitudes would be sustained over time. The questions asked relied on students’ perceptions and not on more objective measures. There were technical considerations for both staff and students in this study. Learning to use the display detracted from the learning in some cases. It is recognised that translational studies will be required to confirm the results. Additionally, the novelty factor associated with the use of the technology needs to be explored and could explain the high positive response to using the simulation technique.

Conclusion

Our study shows the potential of this emerging technology to assist in developing clinical judgement in student nurses. It also suggests that when used in pairs it has the potential to enhance students’ ability to develop the professional dialogue related to describing a patient’s condition to another health professional, and specifically enhances skills in noticing physical cues. The ability of students to be able to visualise patient symptoms and have unlimited exposure to events in the absence of patient risk or discomfort, is a clear advantage of this type of simulated learning experience. This MR experience also allowed unconscious biases and stereotypes to be addressed with students. Making these unconscious biases visible and giving students strategies to deal with them, has a real potential to improve patient care. Further studies are required to understand the full potential of this technology and the optimum pedagogic approach to maximise educational impact.

Figure 3.

Using MR in class.