Immersive virtual reality simulated learning environment versus role‐play for empathic clinical communication training

Abstract

Introduction

The use of immersive virtual reality simulated learning environments (VR SLEs) for improving clinical communication can offer desirable qualities including repetition and determinism in a safe environment. The aim of this study was to establish whether the mode of delivery, VR SLE versus clinical role‐play, could have a measurable effect on clinical empathic communication skills for MRI scenarios.

Methods

A split‐cohort study was performed with trainee practitioners (n = 70) and qualified practitioners (n = 9). Participants were randomly assigned to four groups: clinician VR (CVR), clinician role‐play (CRP), trainee VR (TVR), and trainee RP (TRP). Clinical communication skills were assessed using two methods: firstly, a self‐reported measure – the SE‐12 communication questionnaire and, secondly, a training and assessment tool developed by a panel of experts.

Results

Participants in the VR trainee (TVR) and clinician (CVR) groups reported 11% (P < 0.05) and 7.2% (P < 0.05) improvements in communication confidence post training, whereas trainees assigned to the role‐play (TRP) intervention reported a 4.3% (P < 0.05) improvement. Empirical assessment of communication training scores assessing a participant’s ability to select empathic statements showed the TVR group performed 5% better on average than their role‐play counterparts (P < 0.05).

Conclusion

The accuracy of participant's selection of appropriate empathic responses was shown to differ significantly following the training intervention designed to improve interactions with patients that present for an MRI scan. The results may demonstrate the capacity for immersion into an emotional narrative in a VR environment to increase the user’s susceptibility for recalling and selecting empathic terminology.

The aim of this split cohort study was to establish whether the mode of delivery, immersive virtual reality versus clinical role‐play, could have a measurable effect on clinical empathic communication skills for MRI scenarios. The accuracy of participant's selection of appropriate empathetic responses was shown to differ significantly following the training intervention designed to improve interactions with patients that present for an MRI scan.

Introduction

Effective clinical communication is essential for safe and competent patient care. ¹ It has been shown that even highly qualified practitioners can be uncomfortable with their competency for communicating with patients. ² Sub‐optimal communication by healthcare practitioners has been attributed to poor patient outcomes such as anxiety, dissatisfaction, and non‐adherence to treatment recommendations; ³ furthermore, it is linked to diminished practitioner confidence issues ⁴ and perceived lack of knowledge. ⁵ In addition to the aforementioned practical and perceptual considerations, the emotion theory identifies empathy as the natural competency critical to interpersonal communication. ⁶ Generally empathy can be defined as the ability for one to understand another’s experience or emotions. ⁷ Empathic understanding has been identified as the most important psychosocial characteristic of a physician for effective patient care. ⁸ Empathic practitioner–patient interactions have been shown to lead to increased treatment compliance, higher cancer survival rates, lower infection rates, and a reduction in pain. ⁹ , ¹⁰ However, despite the importance of empathic communication, there is an overall lack of empathy displayed by healthcare practitioners. ¹¹ , ¹²

Empathy has historically been assessed in a variety of ways: situational empathy has been measured using facial gestures, the heart rate, and skin conductance, whereas dispositional empathy has been measured using questionnaires including Hogan’s empathy scale, Mehrabian and Epstein’s emotional scale, and, since the 1980s, Davis’s Interpersonal Reactivity Index. ¹³ However, as the literature suggests, the measurement of training‐derived empathy or all aspects of empathy (e.g. tone, body language, gestures and language) using only self‐reported questionaries can be inaccurate. ¹³ , ¹⁴ It is also worth noting that individuals may have an inherent ability to display empathy; however, clinical empathy requires a practitioner to know how the patient is feeling based on the specific scenario, for example, what it is about an MRI exam that makes a patient feel claustrophobic. Trainee practitioners have limited clinical experience and first need to understand the clinical scenario and what verbal language to use before focusing on nonverbal communication.

Research shows that unlike other clinical skills, the ability for one to communicate effectively does not simply improve with years of experience on the job; in fact it may even degrade. ¹⁵ Attempts to address limitations in interpersonal communication have shown that targeted training can improve both clinician engagement and, most importantly, the clinical experience of the patient. ¹⁶ Previous works ¹⁵ , ¹⁷ , ¹⁸ have reported a variety of techniques for the delivery of communication training, including combinations of didactic and interactive teaching with group tutorials and role‐play with standardised patients. Of the aforementioned techniques, SLEs such as role‐play can provide a framework where repetition and cognitive load reduction can take place, ¹⁹ furthermore, behaviours, interactions, and language can be developed with interchangeable scenarios. ²⁰ Pre‐recorded video can also assist in communication training; however, it has not been shown to offer superior skills transfer to other techniques. ²¹

The advent of virtual reality simulation learning environments (VR SLEs) ²² offers a novel approach to recreate real‐world communication scenarios that might be able to successfully leverage the advantages of software‐based training: diversifiable yet standardisable avatar interactions that can be assessed in a deterministic manner. ²³ Modern immersive VR approaches for the training and development of communication skills offer several advantages over traditional role‐play including, the ability to create large repositories of diverse clinical experiences, the logging of the evolution of communication interactions and the use of natural language processing, and a safe learning environment for repetitive practice. The evidence base for the use of virtual patients (avatars) for clinical communication and empathy training is generally positive, however it is still relatively sparse and has not used modern VR technology (i.e. Oculus Rift) which is fully immersive and interactive. ¹⁷ , ²⁴ , ²⁵ , ²⁶ , ²⁷ , ²⁸ , ²⁹ , ³⁰ , ³¹ Of these available studies, Foster et al., demonstrated that virtual patient interactions combined with real‐time human‐assisted feedback of the virtual scenario improved medical students’ empathic interactions. In a similar approach, Kleinsmith et al., reported that medical students using the Neurological Examination Rehearsal Virtual Environment (NERVE) SLE gave significantly more empathic responses to virtual patients than those made to standardised patients. The results were derived from retrospectively rating the responses and were attributed to the lower‐pressure environment, and the extra time users could have when formulating responses. ¹⁷ Whilst this study provided important evidence for the positive benefit of training with virtual patients, the use of retrospective scoring makes it difficult to build a tool that can be structured to prospectively develop a user’s selection of empathic language.

The study presented herein was designed to leverage the positive outcomes derived from previous VR studies and therefore develop a scenario for prospectively training and then assessing practitioners in the use (selection) of empathic language for a specific task: preparing patients for a magnetic resonance imaging (MRI) scan. The task of better preparing health professionals working in MRI to encounter and coach anxious and claustrophobic patients was selected as the scenario as it was recently reported that approximately two million MRI scans are terminated before completion annually posing a significant cost to healthcare systems. ³² Both traditional role‐play and VR SLEs were used for the delivery of the training and assessment interactions, and both trainees and practitioners (clinicians) with a wide range of experience were recruited for the study. Role‐play was selected as the comparison contingency to the VR SLE (CETSOL) ¹⁹ , ³³ as it allowed for the closest clinical simulated experience when undertaken in a radiography lab/hospital. ³⁴ The aim was to determine whether the mode of delivery of training assigned to the groups (role‐play and VR SLE) gave a measurable difference in the selection of empathic responses selected.

Methods

The study was designed to determine whether the mode of delivery of training assigned to role‐play and VR SLE groups gave a measurable difference in the selection of empathic responses selected. In order to measure and contrast the effectiveness of the role‐play and VR SLEs, a guided, pre‐scored communication scenario was created that focussed on claustrophobic patients undergoing an MRI.

Ethics approval was obtained from both the Peter MacCallum Cancer Centre Human Ethics (LNR/18/PMCC/147) and Monash University Research (MUHREC CF16/661 – 2016000317) Ethics Committees to assign participants in each of the clinician (C) and trainee (T) cohorts into one of two groups: the VR communication scenario or role‐play (RP) tutorial. The aforementioned division of participants resulted in four groups in total: clinician VR (CVR), clinician RP (CRP), trainee VR (TVR) and trainee RP (TRP). A flow diagram showing participant stratification can be seen in Figure 1.

Figure 1.

A flow diagram depicting participant stratification for use of the VR SLE or role‐play (RP) communication training intervention in both the clinician (C) and trainee (T) cohorts. Key: VR, virtual reality.

Reporting of outcomes was conducted in accordance with the CONSORT statement, see the attached checklist in the additional material. Randomisation of the clinician cohort (CVR and CRP groups) was atypical due to their clinical availability; restricted randomisation was performed according to the order in which participants arrived in the tutorial room. The researchers were not aware of the order or communication skill level of the practitioners participating in the study, each practitioner was subsequently allocated to one of the two groups in an alternating pattern. Participants were instructed not to discuss the activity with each other. The clinician cohort comprised four males and five females. Age data were not collected for this cohort.

Randomisation of the trainee cohort (TVR and TRP groups) was performed by the course coordinator at the start of the academic semester (2019) using a random number generator where odd numbers were assigned for group 1 (Monday TVR group) and even numbers for group 2 (Tuesday TRP group).

A summary of trainee and clinician cohort characteristics can be seen in Table 1. The mean is given by the symbol μ, whilst the standard deviation is σ.

Table 1.

A summary of participant characteristics.

Variable	Trainee (N = 70)	Clinician (N = 9)
Male	31	3
Female	39	6
Age (μ ± σ) / [years]	VR = 19.6 ± 2.3; RP=19.8 ± 1.9	> =25 (not recorded directly)
Years of clinical experience	<1	> = 7

All trainees were student radiographers enrolled in Year 1 of the Bachelor of Medical Imaging at Monash University. Year 1 trainees were chosen as they had yet to be exposed to the clinical environment and had not yet engaged with the MRI process or patients. All participants gave informed consent prior to commencement of the allocated training and participation was voluntary. Allocation of participants into groups by the course coordinator and investigators was performed without any prior knowledge of trainees’ or qualified practitioners’ communication skill levels (if any).

Participant self‐perceived communication score (completed pre‐ and post‐training)

The assessment of trainees’ and qualified practitioners’ perceived abilities in the context of the clinical communication skills training were measured using the validated practitioner communication SE‐12 self‐efficacy questionnaire. ³⁵ There was no difference in the method for completing the SE‐12 questionnaire between cohorts. The questionnaire asks users to rate their communication skills regarding their conversation with the patient on a scale from 1 to 10; 1 = very uncertain and 10 = very certain. All participants were asked to complete the online questionnaire both prior‐to and after the completion of their assigned communication training session. A pre‐questionnaire was used to ascertain an accurate baseline measure of skills levels prior to the intervention. The SE‐12 form was administered electronically. The questionnaire data were collated using the Qualtrics survey database.

Communication scenario utilised for training

The communication scenario undertaken by all participants in both cohorts was developed by a panel of three clinical experts using annotated feedback from anxious and claustrophobic patients following previous MRI scans. The scenario was developed in a guided format such that participants using it explained the procedure of an MRI scan to a patient in nine scaffolded steps: five related to the examination and four related to coping mechanisms such as visual and breathing techniques. At each of the nine steps, participants were given a selection of verbal statements to choose from that could be spoken to the patient in the scenario; each statement was assigned a score from one to three according to the quality of the empathic language used and clinical relevance. A detailed overview of the development process scenario and copy of the communication scenario tool developed as part of this study can be found in Appendix S1. Please note the scores associated with responses are not shown as the scenario is used as an assessment tool in current and or future teaching syllabuses. However, the scores can be obtained by contacting the authors.

In order to measure the effectiveness of the VR SLE at delivering the communication scenario, two different delivery formats were developed and compared. The first format was a role‐play tutorial, which consisted of participants discussing and acting out the MRI procedure with a clinical tutor who was experienced in working with anxious and claustrophobic patients undergoing MRI scans. During the role‐play, participants were given scripted and prompting questions/statements according to the guided scenario; this ensured the same patient questions were covered in both delivery formats. For context for the reader, one scripted example the tutor gave was “How would you act out/say if a patient said, ‘What if I can’t breathe?’”. Furthermore, students were allowed to ask the tutor any questions they felt necessary to further their understanding. Participants in TRP and CRP groups were assigned to this form of communication training.

The second delivery format used the same communication scenario; however it was converted to a C# script and embedded into the VR SLE: the Clinical Education Training Solution (CETSOL) VR Clinic software utilising the Oculus Rift CV1 VR headset. Each statement was scripted and animated to have a corresponding patient avatar response; avatar responses were given as audio with corresponding facial animations and also as on‐screen text. Further details regarding the programming of the dialogue scenario including a visual representation (see Supporting Information) can be found in the Figure S1.

The VR software had two communication options: guided and free speech. The guided option took users through the different stages of the communication scenario by showing dialogue statements on screen, participants could use voice recognition to select the desired statement to which the patient avatar responded to the chosen statement. Users could additionally use the free speech option, allowing them to ask the patient avatar alternative clinical questions from other dialogue nodes such as confirming medical history or building general rapport. Participants did not use the free speech option to navigate through the guided MRI scenario; this was to ensure they went through all necessary stages sequentially. Participants’ alternative free speech conversations (which were stored in a separate dialogue tree within the software) did not affect their ability to complete the guided scenario or alter the patient’s responses within the scenario. An example of the VR communication training scenario questionnaire used by the TVR and CVR groups can be seen in Figure 2. Both VR and RP cohorts were allocated 30 minutes to complete their communication scenarios.

Figure 2.

A screenshot of a user interacting with the VR communication scenario, used by participants in Groups CVR and TVR. (A) A patient avatar responding to a voice activated user statement. (B) A user selecting a response to a patient query.

Communication scenario utilised for assessment

Immediately following the training intervention participants in the TRP and CRP groups completed a scored, on‐screen, guided questionnaire which incorporated the same communication questions used in the RP scenario. An example of the digital communication training scenario questionnaire can be found in the Figure S2. Subsequently all participants completed the SE‐12 questionnaire.

A summary of the training interventions for the TRP, CRP, TVR, and CVR groups is described in Figure 3. A detailed outline of the procedural steps undertaken during the training intervention and respective differences between groups can also be found in Appendix S2.

Figure 3.

A flow chart showing the procedure for each of the experiment groups (RP and VR, for both Trainee and Clinician). Equivalent steps are indicated by a double‐head arrow.

Statistical analyses

All statistical analyses were conducted using IBM SPSS Statistics for Windows 24.0. ³⁶ The normalcies of the data distributions were tested using the Shapiro‐Wilk test. The intergroup and intragroup differences were assessed using parametric t‐tests. Statistical significance was reached if P < 0.05. The key variables assessed were group mean scores for the SE‐12 efficacy questionnaire and communication scenario scores.

Results

In this study, four groups of participants completed a communication training scenario which was delivered in two modes RP versus VR. To assess if there was a difference between training interventions, participants completed a pre/post self‐efficacy questionnaire (SE‐12) and an on‐screen guided questionnaire which incorporated the communication scenario questions.

Shapiro‐Wilk test of normality

Pre‐ and post‐significance values were 0.84 and 0.08, respectively; therefore as both significance values were > 0.05, the data were determined to be normally distributed. A two‐sample t‐test was used for intergroup mean analyses, and a paired sample t‐test was selected for analyses of the intragroup means.

Self‐efficacy (perceived) communication questionnaire

Data relating to the self‐efficacy questionnaire was only included in the mean results if participants completed the communication training and both pre‐ /post‐training questionnaires. One trainee participant did not complete the communication training, and five trainee participants did not complete both pre‐post SE12 self‐efficacy questionnaires. These participants were excluded from the final data analyses.

The results of the participant self‐efficacy communication questionnaire are summarised in Table 2 and are discussed below.

Table 2.

A comparison of pre‐ and post–self‐efficacy communication questionnaire scores.

Group	n	Pre‐training µ ± σ	Post‐training µ ± σ	Pre‐ vs. post‐training
CVR (Clinician VR)	5	7.8 ± 0.4	8.4 ± 0.6	P < 0.05 where d = 1.15
TVR (Trainee VR)	32	6.8 ± 1.1	7.6 ± 1.0	P < 0.05 where d = 0.65
CRP (Clinician role‐play)	4	7.3 ± 0.5	7.5 ± 0.3	P = 0.58 where d = 0.305
TRP (Trainee role‐play)	32	6.8 ± 1.0	7.1 ± 1.0	P < 0.05 where d = 0.34
CVR versus CRP	9	P = 0.17 where d = 1.06	P < 0.05 where d = 2.01	NA
TVR versus TRP	64	P = 0.72 where d = 0.89	P < 0.05 where d = 0.4	NA

n, sample size; µ ± σ, mean ± standard deviation; d, Cohens D; NA, not applicable.

Intragroup analysis

Analysis of intragroup (pre‐ vs. post‐) training scores demonstrated that all groups except the CRP group reported a significant improvement in self‐assessed communication scores following training: 11% for TVR (P < 0.05), 4.3% for TRP (P < 0.05), and 7.2% for CVR (P < 0.05). It should be noted that improvement in ratings of “communication self‐efficacy” do not necessarily mean improvements in real world communication.

Intergroup analysis

Assessment of the intergroup (CVR vs. CRP and TVR vs. TRP) pre‐training data showed no significant difference in mean scores. Analysis of post‐training data found a significant difference for the CVR versus CRP groups (12%, P < 0.05). The analogous test for the TVR versus TRP groups also demonstrated a significant difference (7%, P < 0.05).

Empirically assessed communication scenario scores

Table 3 presents the results from the selected verbal responses for each of the four groups for the communication training scenarios were scored according to their consistency with the assessments performed by the expert panel of designers, and then subsequently aggregated (see Table 3).

Table 3.

A comparison of mean communication scenario scores for role‐play versus VR training.

Group	n	Role‐play µ ± σ	n	VR µ ± σ	N	Role‐play versus VR
Trainee	32	23.2 ± 2.4	32	24.3 ± 1.6	64	P < 0.05 where d = 0.51
Clinician	4	19.4 ± 3.8	5	22.4 ± 1.9	9	P = 0.17 where d = 1.03

n, sample size; N, population size; µ ± σ, mean ± standard deviation; d, Cohens D.

The mean scores for the empathic response selections for both trainee groups (TVR and TRP) were found to be significantly higher than those for the clinician groups. However, intergroup statistical significance was not tested due to the small clinician sample size. Both VR training groups (TVR and CVR) performed better on average than their role‐play counterparts (5% and 11%); however, the results were only statistically significant for the trainees: P < 0.05. A post‐hoc analysis indicated that a clinician cohort sample size of 68 would have been required to obtain an effect size of d = 0.7 consistent with the medium to large effect size of approximately 11% seen in our results.

Discussion

The main goal of the communication training and assessment study was to determine whether the mode of delivery of training gave a measurable difference in the selection of empathic responses and whether it increased clinician self‐efficacy. The communication scenario created performed two functions: firstly, it explained in a sequential manner the clinical steps of an MRI procedure; and secondly, it measured a participant’s ability to choose more empathic language to explain the same procedural step. Each participant completed two evaluation methods to assess communication training: the first was a self‐reported measure ‐ the SE‐12 communication questionnaire (reported in Table 3); and the second was a guided communication scenario tool with embedded scoring (see Appendix S1) developed by a panel of experts (reported in Table 3).

To the best of our knowledge, this is the first study to compare the use of a fully immersive head‐mounted VR SLE tool against a role‐play scenario for practitioner communication training. The results of the SE‐12 communication questionnaire showed that the intra‐group pre‐training scores were not significantly different amongst either the trainees or clinicians. However, post‐training, the TVR group’s ability was perceived to be 6.7% greater than that of the TRP group, and the CVR group’s ability was perceived to be 7.2% greater than CRP group’s. This trend was also observed in the results of the empirically assessed communication scenario questionnaire where the TVR group performed 5% better than the TRP group. The CVR group’s performance relative to that of the CRP group showed a similar underlying trend; however the result was not significant. The lack of significance for the clinician group may have been attributed to the small group size.

The observed significant results may be attributable to the individualised experience without the judgment of peers offered by an immersive VR SLE. In addition, the combination of multimedia design principles forming the virtual patient interaction experience has been linked with positive learning outcomes, for example, the combination of voice and text, the learner control principle (allowing users to set delays/actions at their own initiative), the contiguity principle (words/pictures displayed near rather than far) and the segmenting/signalling principles (use of segments and guided cues) ³⁷ .

The VR SLE utilised in this study enabled the implementation of the communication scenario through a generalizable framework, whilst also enabling it to be contextualised as part of the practitioner’s workflow. It should be noted that the process of programming realistic and flexible responses can be difficult and presents challenges that would typically not be encountered in a role‐play scenario. For example, participants in the RP cohort were able to role‐play the patient and practitioner workflow described above without the need for complex dialogue structures. Besides programming challenges, there are the added processes of collecting voice recordings and animating facial expressions to different emotional cues. It is important to bear in mind creating realistic communication scenarios requires a significant time investment.

There have been several previous pilot studies that have used VR SLEs for communication training. Johnsen et al. reported that medical students found immersive VR characters a powerful tool for teaching and training. ²⁵ Likewise, Brown‐Johnson et al. stated that health professionals using a virtual world health game for lung cancer patients demonstrated improved communication knowledge, although no quantitative measures were reported. A more closely related study showed that students using the NERVE software expressed higher median (mdn) levels of empathy to virtual patients (mdn = 4) than standardised patients (mdn = 2) where P < 0.05 and r = 0.51; however, empathic responses were rated using a subjective empathic communication coding system. ¹⁷

It is important to note that some earlier works have suggested that experiences with virtual patients have been perceived as less genuine and more difficult than those using other forms of communication training such as standardised patients. ³⁸ However, these negative findings are thought to be due to the lack of realism and/or means for interaction that can now be overcome due to advances in VR technology and improved graphics processing engines. A recent study by Herrera et al., demonstrated that immersive VR, where the user can interact in the VR environment, resulted in a statistically significant increase in positive attitudes to socially challenging situations, when compared to participants who either just imagined what it would be like to be involved or performed a less immersive perspective‐taking task. A second important conclusion from their work was that narrative‐based and mediated perspective‐taking interventions were more effective at increasing self‐reported empathy. In a different study more aligned with the work reported in this paper, Coulter et al., demonstrated that as the quality and immersiveness of a virtual patient experience increased, so too did student satisfaction and degree of learning. ³⁹ The related increase in the degree of learning may be associated with narrative transportation theory which suggests, that immersion into an emotional narrative may increase the users susceptibility to remember a scenario than when compared to those who are not immersed or transported; additionally, a meta‐analyses of immersive technology relating to a users’ presence found that greater immersion facilitated greater psychological presence. ⁴⁰

Limitations

A key disadvantage of using software‐based communication training is that it takes time to build a robust and realistic communication model. This study did not observe participants using empathy in the natural setting; this was due to the inherent clinical constraints. As mentioned previously this study focused on measuring a single aspect of empathic communication training relating to the ability to choose empathic language. The practical implementation of the study meant that participant selection for the clinical sample was atypical. In addition, the clinician cohort size was under‐powered, and therefore it is unclear whether a true difference would exist if the study were repeated with more participants. If the study were to be repeated, a larger sample size of at least 68 qualified practitioners would be required according to power analyses. An extended summary of the limitations section can be found under the supporting information.

Conclusion

This study measured the accuracy with which empathic language was selected by participants (both trainee and qualified radiographer cohorts) following a training intervention designed to improve interactions with patients that present for an MRI scan. The results demonstrated a significant improvement in the trainee group’s ability to select empathic responses when using the VR mode of delivery when compared to the role‐play mode of delivery. These results may demonstrate the capacity for immersion into an emotional narrative to increase the user’s susceptibility for recalling and selecting the empathic terminology learned from training exercises.

Funding

The cost of this study was approximately AUD $10,000, for which funding was provided by a grant awarded from the Victorian Medical Radiation Practitioners Education Trust (VMRPET). The funding allowed for a dedicated laboratory to be built consisting of five high‐end PC's and five oculus rift virtual headsets and leap motion controllers.

Conflict of interest

The authors declare no conflict of interest.

Supporting information

Figure S1 A screenshot of the dialogue editing canvas used to build dialogue scenarios. Each node represents a node connection showing the users choice and the adjacent respective avatar response. The reader should note that the main dialogue classes’ connections are not “expanded” to demonstrate the dialogue editing canvas in its simplest form.

Figure S2 An example question derived from previous MRI patient experiences used in the multiple‐choice communication scenario. The associated score for each option is indicated by the number in the adjacent green circle. In the example above, the first question is given a score of three points: firstly, it explains the procedural step fully in non‐technical language; secondly, it explains and shows the practitioner is aware of the feeling the patient may experience; and finally, it emphasises the normality of these feelings for other patients in this similar situation. In comparison, the third option which has a score of two explains the feelings and emotions; however the addition of technical aspects such as the bore size can be confusing to a patient. Finally, the second option with a score of one explains the procedure well. However, it doesn’t convey the feelings and emotions that may lead to the patients’ anxiety. Rather, it implies that the patient will be anxious or claustrophobic.

J Med Radiat Sci. 69(2022) 56–65