The implementation of a haptic feedback virtual reality simulation clinic with dynamic patient interaction and communication for medical imaging students

Abstract

Introduction

An immersive virtual reality (VR) simulation clinic with dynamic patient interaction and communication was developed to facilitate the training of medical radiation science students. The software “CETSOL VR Clinic” was integrated into the Medical Imaging programme at Monash University in 2016 in order to benchmark student experiences against existing simulation techniques (Shaderware™).

Methods

An iterative approach to development, based on two cycles of user feedback, was used to develop and refine the simulated clinical environment. This environment uses realistic 3D models, embedded clinical scenarios, dynamic communication, 3D hand gesture interaction, gaze and positional stereoscopic tracking and online user capabilities using the Unity™ game and physics engines. Students’ perceptions of educational enhancement of their positioning skills following the use of the simulation tools were analysed via a 5‐point Likert scale questionnaire.

Results

Student perception scores indicated a significant difference between simulation modalities in favour of the immersive CETSOL VR Clinic, χ² (4, N = 92) = 9.5, P‐value <0.001.

Conclusion

Student perception scores on improvement of their clinical and technical skills were higher for the hand‐positioning tasks performed with the CETSOL VR Clinic™ than with the comparative benchmark simulation that did not provide dynamic patient interaction and communication.

Introduction

Simulation is commonly used for clinical education to develop a range of pre‐clinical and clinical skills. In addition, e‐learning platforms such as virtual reality (VR) environments are now also widely available to deliver these activities, enabling universities to broaden their scope.1, 2 The effectiveness of training software can be limited by its low fidelity.3 In simulation, fidelity describes the degree of realism and accuracy of the system.4 Higher fidelity simulated learning environments (SLEs) such as those used in flight simulators can make up to 30% of a pilot's mandatory flying hours.5 It is anticipated that high fidelity simulation can play a similar role in medical imaging.

The primary goal for institutions that train medical imaging practitioners, as with other disciplines, is to demonstrate the clinical competency of their students at a level that satisfies the accreditation bodies. Traditionally, in the field of x‐ray imaging, universities have used a combination of simulation tools, including anthropomorphic phantoms6 and functional and licensed x‐ray machines. Whilst “mid ‐ range simulation” setups like these enable the development of pre‐clinical skills, there exist significant limitations in functionality, patient presentations and clinical scenarios that can be taught to students.

In recent years, the use of VR SLEs is being increasingly reported for medical imaging (radiography) clinical education, with software such as Shaderware™,7 VERT™,8 Medspace VR9 Medical imaging training immersive environment (MITIE),10 and Secondlife^® 11 Each of these tools and SLE's has been designed to recreate one or several aspects of the real‐world medical imaging/radiation therapy clinical training experience. The advantages of using these tools include human resource, equipment and location cost savings.12 There are, however, some drawbacks evident with existing VR SLE's, including a lack of realism or immersion, ease of use, limited replication of procedural process and limited patient interaction to no patient communication capabilities. However, the major shortcoming as found by Thoirs et al.,3 was the lack of empirical evidence relating to clinical performance. As with real‐world simulation tools, current medical imaging VR SLE's generally do not provide an amalgamation of the complete patient practitioner experience. For instance, the MITIE tool10 can replicate the clinical environment but does not provide the patient communication aspect used in the clinical setting. Additionally, a study assessing non‐immersive VR simulation tools such as Shaderware™, expressed a need for remote access.13 Other studies assessing the radiotherapy simulation tool VERT found students to perceive an improvement in their confidence and understanding of technical skills after using the software; however, students were limited in their ability to replicate certain aspects of the clinical workflow.14

For the transition from student to competent practitioner to occur, trainees ultimately need to consolidate several core skills which include communication, clinical knowledge and procedural processes at one time. Thus, a validated VR SLE that facilitates the knowledge translation of all these competencies in one application could help provide a reduction in the number of hours of clinical placement required for a practitioner to be deemed competent; however, this yet to be validated.

Clinical Education Training Solution (CETSOL) is an immersive radiographic VR SLE that has been under development since 2012. In 2016, it was formally adopted into the Bachelor of Radiography and Medical Imaging (Hons) curriculum at Monash University in order to benchmark its development and performance against other simulation tools and environments such as Shaderware™ evaluated in 2015. Figure 1 shows a visualisation of a student undertaking a hand‐positioning task using the CETSOL VR Clinic in 2016. As part of the iterative development process in designing the SLE, year 1 radiography students’ perception scores from 2015 and 2016 were assessed, in order to gauge experience relative to Shaderware™. The perception score analyses represent the first stage of continuing validation of the efficacy of this software and are presented herein.

Figure 1.

A screenshot of CETSOL SLE showing posterior–anterior (PA) hand x‐ray setup performed by a year 1 radiography student.

Methods

Ethics approval for this project was obtained from Monash University Human Research Ethics Committee (MUHREC). As part of a progressive workup in designing the new VR SLE, we reviewed user feedback from studies utilising existing applications. The process involved a review of the current literature3, 10, 14, 15, 16detailing existing applications up to September 2015. Databases searched included EMBASE, Medline and the Cochrane Database of Systematic reviews. A combination of relevant keywords and subject headings was used. No time or language restrictions were applied. Articles that included medical imaging with the words, VR, virtual reality, simulation, simulated learning, were all reviewed. Based on this review coupled with extensive iterative student and practitioner user feedback, two versions of the software were developed. The two versions were required as it was found that there were significantly different interaction requirements and computer resource availabilities amongst students and practitioners using the software. Beta feedback was acquired from qualified radiographers, demonstrators and small cohort of 2nd year students and 1st year students in September 2014 and 2015. After beta users trialled the software, they were asked to fill out a questionnaire which asked “what do you think could be improved?” and “what did you like the most about the software?”

The software development was undertaken by a registered medical radiation practitioner (first author) with a background in computer programming and design and has been ongoing for 4 years. The platform chosen to build the SLE was Unity™ due to its powerful graphics and physics engine. The computer language chosen was C# due to its flexibility across platforms including web applications and hardware.

Features incorporated into the CETSOL VR Clinic software

Some of the identified features based on user feedback that were included in the CETSOL VR Clinic Software included;

• Ability to complete a full range of clinical examinations including, upper and lower limb appendicular, axial, spinal, chest, abdomen and facial bones x‐ray examinations.

• 3D immersive peripheral integration, enabling the software to be used with wearable hardware including the Oculus Rift^®, HTC Vive^® Leap Motion^®.

• Dynamic patient avatar communication capabilities using application programming interface (API) learning logic with voice recognition and integrated real‐world clinical scenarios. The theoretical basis for the communication script was based on previous dialogue systems such as those described by Beveridge and Fox.17

• Online user progress logging, allowing users and educators remote progress tracking.

In addition to the two versions of the software, multiple modes of functionality and user control interfaces were added, these included: the Oculus Touch and LEAP motion hands free controller.

Detailed specifications of the CETSOL VR Clinic are described in Appendix S1.

2016 study

The first implementation of the CETSOL Clinic was deployed as an additional training tool for all year 1 radiography students in April 2016. The version without the use of VR peripherals was deployed in order to make the subsequent benchmarking task, relative to the Shaderware™ software, comparable. Institutional ethics approval was received. The SLE was introduced as part of 3 h laboratory session where students used the software for a 45 min session. Students were split into five groups where each student had an individual PC workstation. A total of 79 students took part in the study in 2016. Students were instructed to complete an integrated hand‐imaging lesson plan and associated imaging referral for a PA and lateral hand. This involved students communicating with and positioning the virtual patient; furthermore, they had to select the correct imaging cassette, machine parameters including collimation and source to image distance and exposure factors. Students’ perspectives on using simulation were analysed via a 5‐point Likert scale questionnaire at the end of the semester. The 2016 questionnaire results were also compared to the 2015 questionnaire18 for which the previous cohort of 76 year 1 students were asked to evaluate their experience of the same task following the use of Shaderware™. Both sets of assessments were completed during the same teaching period of the academic teaching years. The questionnaire was developed independently from the lead researcher. When evaluating the 2016 cohort, the same questionnaire was adopted to avoid any bias in the comparison study. There were no other changes in course facilitators or curriculum between the 2 years except for the change in SLE software. Completion of the questionnaire was voluntary. Statistical analyses were conducted using a chi‐squared (χ²) test for independence, using an alpha level of 0.05 for statistical significance. Students who had used the software previously were excluded in taking part in the perspective questionnaire. The questionnaire was designed to gauge students’ perceptions of the CETSOL^® SLE. The questionnaire and its results can be seen in Table 1.

Table 1.

5‐point Likert scale questionnaire results assessing year 1 radiography student cohort perceptions using two SLE software programs CETSOL^® Clinic (2016), relative to those using Shaderware™ (2015) for the same hand‐positioning task

First‐year responses to the uses of CETSOL VR Clinic® (2016) n = 49 versus Shaderware™ (2015) n = 43	SA	A		U		D		SD		Total		P‐value	Chi ‐Crit	Chi‐Stat	Accept H0 or H1
Statement	Count	%	Count	%	Count	%	Count	%	Count	%	Count
Shaderware™ develops practical skills in patient positioning	1	2	11	26	9	21	17	40	5	12	43	<0.001	9.5	24.2	H1
VR clinic develops practical skills in patient positioning	2	4	33	67	10	20	2	4	2	4	49
Shaderware™ develops practical skills in x‐ray tube centring	4	9	15	35	11	26	11	26	2	5	43	<0.01	9.5	17.4	H1
VR clinic develops practical skills in x‐ray tube centring	8	17	32	67	6	13	1	2	1	2	48
Shaderware™ develops practical skills in bucky/cassette and tube alignment	5	12	15	32	9	21	10	23	5	12	43	<0.001	9.5	18.3	H1
VR clinic develops practical skills in cassette and tube alignment	7	15	31	66	8	17	1	2	0	0	47
Shaderware™ develops practical skills in cassette selection	6	14	12	28	8	19	12	28	5	12	43	<0.001	9.5	27.2	H1
VR clinic develops practical skills in cassette selection	9	18	33	67	7	14	0	0	0	0	49
Shaderware™ develops practical skills in cassette positioning/orientation	5	12	19	45	4	9.5	9	21	5	12	42	<0.001	9.5	19.8	H1
VR Clinic develops practical skills in cassette positioning/orientation	9	19	28	58	11	23	0	0	0	0	48
Shaderware™ develops practical skills in selecting exposure factors	10	23	18	42	6	14	7	16	2	5	43	0.08	9.5	8.3	H0
VR clinic develops practical skills in selecting exposure factors	8	17	31	65	7	15	2	4	0	0	48
Shaderware™ develops practical skills in collimation	5	12	20	48	7	17	7	17	3	7	42	0.08	9.5	8.3	H0
VR clinic develops practical skills in collimation	9	19	29	60	8	17	2	4	0	0	48
Shaderware™ develops practical skills in side marker placement	7	16	17	40	4	9	13	30	2	5	43	<0.01	9.5	14.9	H1
VR clinic develops practical skills in side marker placement	6	13	31	67	7	15	2	4	0	0	46
Shaderware™ develops practical skills in grid selection	2	5	11	26	13	30	14	33	3	7	43	0.01	9.5	12.6	H1
VR clinic develops practical skills in grid selection	0	0	16	33	27	55	5	10	1	2	49
Shaderware™ develops practical skills in image processing	8	19	20	47	10	23	3	7	2	5	43	0.02	9.5	11.34	H1
VR clinic develops practical skills in image processing	2	4	19	40	16	33	11	23	0	0	48
Shaderware™ develops practical skills in patient dose (and its relationship with exposure factors, SID, collimation)	9	21	17	41	10	24	3	7	3	7	42	0.12	9.5	7.3	H0
CETSOL VR clinic develops practical skills in patient dose (and its relationship with exposure factors, SID, collimation)	4	8	25	52	14	29	5	10	0	0	48
Shaderware™ develops practical skills in understanding the effect of radiographic positioning and parameters on resultant image	1	2	18	41	21	49	1	2	2	5	43	0.09	9.5	7.8	H0
VR clinic develops practical skills in understanding the effect of radiographic positioning and parameters on resultant image	1	2	30	61	14	29	4	8	0	0	49
Shaderware™ develops practical skills in basic projections (AP and lateral)	4	10	15	37	0	29	9	22	1	2	41	<0.001	9.5	18.6	H1
VR clinic develops practical skills in basic projections (AP and lateral)	6	12	35	71	8	16	0	0	0	0	49
Shaderware™ develops understanding of the steps required in preparation and execution of a radiographic examination	3	7	11	26	13	30	13	30	3	7	43	<0.001	9.5	24.9	H1
VR clinic develops understanding of the steps required in preparation and execution of a radiographic examination	3	6	36	75	6	13	3	6	0	0	48
Shaderware™ develops practical skills in using correct terminology	1	2	11	27	12	29	11	27	6	15	41	<0.01	9.5	15.6	H1
VR clinic develops practical skills in using correct terminology	0	0	28	57	15	31	6	12	0	0	49
Mean Shaderware™ clinical positioning and radiography skills perception score	5	11	15	36	9	23	9	22	3	8	43	<0.01	9.5	33.3	H1
Mean VR clinic clinical positioning and radiography skills perception score	5	10	29	61	11	23	3	6	0	1	48

SA, strongly agree; A, agree; U, unsure; D, disagree; SD, strongly disagree, VR, virtual reality; SRP, simulated clinical role play; Chi‐Crit, Chi, chi‐square (χ²) value, H₀, student radiographic skill perception is independent of software used; H₁, student radiographic skill perception is dependent on software used.

Results

Iterative feedback from users

Most of the features incorporated into the CETSOL SLE were based on initial feedback responses from students and practitioners; beta users were asked “what do you like the most about the software” and “what do you think could be improved?”, beta users commented positively on the dynamic nature of the clinic and ability to position patient avatars in almost any clinical position; however, several users expressed a desire for improvement in:

• “Controls, graphics, more interaction with surrounding environment”

• “Graphics”

• “More request forms to complete”

• “Increase the amount of communication with the patient”

• “Identifying marks/signs on the walls or arrows on the ground to prevent from getting lost in the centre”

Early feedback influenced the creation of a dynamic environment. Furthermore, emphasis was placed on patient interaction and communication capabilities to add more clinical dialogue with multiple modes of communication and immersive peripheral interaction.

In total, three ways of communicating with the patient avatars were developed: on‐screen situation‐based responses, keyboard input statements and phrases and direct user voice input. A demonstration of the on‐screen situation‐based communication method can be seen in Figure 2.

Figure 2.

A screenshot of CETSOL SLE showing the operator using their index finger to select one of the several on‐screen situation‐based prompts to elicit a patient response.

5‐point Likert scale questionnaire 2015–2016

Self‐perception scores were assessed in 2015 as part of the preliminary background work in the development of the CETSOL Clinic. Of 76 year 1 radiography students, 43 answered questions on their perception of the educational enhancement of their positioning skills following the use of Shaderware™. On average, 11% of student responses to all questions were in the “strongly agree” category, 36% were in the “agree” category, 23% were in the unsure category and 22% and 8% were in the “disagree” and “strongly disagree” category respectively (see Table 1).

The same questionnaire was given to the 2016 year 1 cohort. A total of 49 of 79 students completed the questionnaire. On average, 10% of student responses to all questions were in the “strongly agree” category, 61% were in the “agree” category, 23% were in the unsure category and 6% and 1% were in the “disagree” and “strongly disagree” category respectively (see Table 1). The results of the chi‐squared test of independence between the mean student radiographic skill perception and SLE demonstrated that 2016 student cohort (CETSOL) perceived the CETSOL^® VR Clinic to improve their clinical and technical skills at a higher level than of 2015 student sample which used Shaderware™ (χ² (4, N = 92) = 9.5, P‐value <0.001). The results of responses to individual questions are summarised in Table 1.

Discussion

Student perception

This project aimed to develop an immersive VR SLE, which could consolidate the acquisition of several core skills into one tool. Student feedback indicates that students found the CETSOL VR SLE to aid their knowledge in patient and equipment positioning for the hand‐imaging tasks they completed.

As with other studies evaluating the use of SLE's in tertiary medical radiation programmes,3, 10, 13, 19 our results show similar perceived outcomes. However, when evaluating the use of SLE's specifically for radiography, we found notably different scores in student perception. For instance, students using the CETSOL SLE showed lower disagreement scores (combined disagree + strongly disagree scores), (6.15%) for perceived improvement in clinical positioning and radiography skills/techniques than the Shaderware™ cohort (25%). Furthermore, these results were also lower than other studies using newer iterations of digital SLE's such as those reported in the MITIE study10 (18% disagreement score). Additionally, agreement scores (combined strongly agree + agree scores) for the CETSOL SLE were also found to be significantly higher, where 70.9% of students found the CETSOL SLE to improve clinical positioning and radiography skills, compared to 47% as seen in the Shaderware™ cohort and to that of 46% of students who reported that the MITIE tool aided development in their “understanding of general radiography techniques10”.

Cost and benefits

Simulation offers several fiscal advantages over real‐world clinical training, one of the main advantages lies in its ability to allow students to practise radiological‐based examinations in a safe manner. Like other recent SLE which allow for large user numbers (over 50) to run the software at any given time, the human resource allocation needed to run an equivocally supervised activity would be a considerable amount more. One of the unique benefits of the CETSOL software is that each student is assigned their own user account. This allows for the use of the software off‐campus or at home, and furthermore, educators can review session progress through an online portal. More importantly with the integration of multiuser interaction in future updates, students will be able to partake in hands on immersive virtual laboratories and tutorials simultaneously and remotely with other students given they have access to an Oculus Rift/Oculus Touch. The implementation of a VR SLE such as the CETSOL VR Clinic could potentially help students learn at different paces, allow for a new form of distance education and allow students to learn using different styles rather than listening in traditional pedagogic teaching practices.

Study Limitations

The comparison against the Shaderware™ software only comprised the general procedural workflow in radiography. The use of peripherals was excluded. The cohorts that assessed the CETSOL and Shaderware™ software were from consecutive year groups. Furthermore, we were only able to assess students perceived improvements in their clinical and technical skills to a limited number of imaging procedures, this may not necessarily translate to clinical improvements.

Future implications

The outcomes of this study have given rationale for the CETSOL VR software to be implemented as part a regular self‐directed module in the practical laboratory session in consecutive years, this will provide a mixed mode real world and SLE learning. Future studies will aim to quantify the perceived outcomes to real‐world performance outcomes.

Conclusion

An immersive radiographic VR SLE with multiple modes of operation was created. The SLE includes the ability to use head‐mounted 3D peripherals such as the Oculus Rift with haptic feedback. The SLE features dynamic patient interaction and communication, voice interface, continuous online student progress tracking, online multiuser capability, extensive patient positioning and x‐ray imaging based on real‐world radiation physics.

Year 1 radiography students from the 2015 and 2016 cohorts completed a hand‐positioning task incorporated as part of the general workflow of performing a radiographic examination. Student perceptions were between the two cohorts were compared. Student perception scores on improvement of their clinical and technical skills were higher for the hand‐positioning tasks performed with the CETSOL VR Clinic™ than with the comparative benchmark simulation that did not provide dynamic patient interaction and communication.

Conflict of Interest

The authors declare no conflict of interest.

Supporting information

Appendix S1. List of features incorporated into the CETSOL VR Clinic software.

Figure A1. A flow diagram describing the steps undertaken for the development and implementation of the CETSOL VR SLE and corresponding research process.

Figure A2. (A) Expert hand x‐ray image for PA projection of the left hand (2016 cohort study), (B) real‐world x‐ray for left hand PA projection (Knipe and Shetty, 2017), reproduced with permission.

J Med Radiat Sci 65 (2018) 218–225