Application of virtual reality technology in clinical medicine

Abstract

The present review discusses the application of virtual reality (VR) technology in clinical medicine, especially in surgical training, pain management and therapeutic treatment of mental illness. We introduce the common types of VR simulators and their operational principles in aforementioned fields. The clinical effects are also discussed. In almost every study that dealt with VR simulators, researchers have arrived at the same conclusion that both doctors and patients could benefit from this novel technology. Moreover, advantages and disadvantages of the utilization of VR technology in each field were discussed, and the future research directions were proposed.

Introduction

By combining a series of different technologies, including head-mounted display (HMD) with head-tracking systems, headphones for sound/music and noise-canceling headphones, as well as manipulation/navigation devices, virtual reality technology provides a multisensory, and three-dimensional (3D) environments that enable users to become fully immersed in a simulated world [1]. The use of the HMD helmet allows users to perceive a 3D stereoscopic images and to determine the spatial position in the visual environment via motion tracking sensors in the helmet. Meanwhile, users can hear sounds from headphones and interact with virtual objects using input devices like joysticks, wands, and data gloves. As a result, users feel as though they can look around and move through the simulated environment [2].

Because of these characteristics, over the past decade the application of VR technology has expanded from entertainment industry to clinical medicine. Researchers and doctors have explored the effects of VR simulation on physical rehabilitation, pain management, surgery training, anatomical education and treatment of psychiatric disorders [1,3-7]. Compared with traditional methods, VR technology is regarded as a cost-effective and efficient tool in the aforementioned areas.

Although VR devices have similar structure, the discrepancies in hardware and software requirements often arise due to different application fields. In addition, different researchers emphasize different models of research utilization. For example, VR simulators for education focus on the 3D environment and network resources [8]. VR simulators for surgical training focus on the interaction with operation devices and virtual anatomical structure [9,10]. VR simulators for psychological disorders focus on the emotion control and feedback. The main objective of this paper is to discuss the application of VR technology in clinical medicine. The three approaches addressed in this paper have been research hotspots for over two decades, and thus far they have been subject of many studies. By summarizing the current research results, we analyze the advantages of VR technology for each field, and make predictions for future utilizations.

VR technology for surgery training

For decades, the acquisition of technical skills in the operating room under the supervision of senior surgeons has been the only way for junior doctors to receive surgical training [11]. As the number of trainees has increased, the opportunities to acquire necessary technical skills have become increasingly limited due to the higher costs, ethical concerns, and decreasing resident work hours. A reduction in training hours has been caused by introduction of the European Working Time Directive (EWTD) across Europe, and the 80 hours per week limit introduced by the Accreditation Council for Graduate Medical Education (ACGME) [12,13]. The real and immediate consequence of these policies was a continuous reduction in ‘hands-on’ experience and independent surgical experience for trainees [14]. Furthermore, as the surgical techniques have advanced and evolved, sole observation was no longer enough for acquiring certain skills and special techniques. In these circumstances, VR training has become an essential prerequisite for junior doctors before they are allowed to actively participate in real operations.

Compared with animal modes, videos and e-learning, VR simulations are more realistic due to very intuitive anatomic structures exhibited in the 3D graphics (Table 1). Trainees can interact with all the anatomical structures, including skin, muscle, bone, nerve, and blood vessel. Changes that occur following each surgical step are very much the same as in reality. Whole performance can be recorded, compared and analyzed, making data permanently available for trainees [11]. From different perspective, senior supervision and patient participation are no longer needed during the period of basic skill training and acquisition, since the VR simulations can provide a controlled virtual environment necessary to satisfy these requirements outside the operating room [6]. Multiple aspects of a trainee’s psychomotor performance skills can be directly measured by the objective performance assessment offered by simulations. The training effect of VR simulations are generally evaluated by standard parameters, including time necessary to complete task, path length, number of collisions, injuries, number of anatomic landmarks identified, number of loose bodies found, satisfaction and so on.

Table 1.

Application of VR simulators for different surgeries

Surgery	VR simulators	Haptics	Visual	Intervention
Laparoscopic	LapSim (Essence and haptic system)	Yes	LCD-based	Basic skills, tasks training and imaging
	Lap Mentor (LAP Mentor III)	Yes	LCD-based	Tasks, skills and suturing simulation
	MIST-VR (VIST® G5)	Yes	LCD-based	Evidence-based validity and high-fidelity simulation
	Simendo (Pro3 laparoscopy)	No	LCD-based	Novice and intermediate skills
Orthopedics	Procedicus KSA VR	Yes	Monitor-based	Laparoscopic surgery skills
	Procedicus virtual arthroscopy simulator	Yes	Monitor-based	Knee laparoscopic surgery skills
	Insight Arthro VR Shoulder Simulator (Arthro MentorTM)	Yes	LCD-based	Fundamentals of arthroscopic surgery training and diagnostic procedures
	TraumaVision	Yes	LCD-based	Hip fracture simulator training
Other surgeries	Visible Ear Simulator	Yes	LCD-based	Basic mastoidectomy tutoring
	Eyesi VR Magic (Eyesi Surgical)	No	Operating microscope-based	Cataract and vitreoretinal surgery training

Laparoscopic

To safely and effectively complete laparoscopic operation, surgeons must face a long learning curve and extensive training. However, it is not possible for surgical trainees to practice basic surgical and laparoscopic skills in the operating room, since this would expose patients to potential risks [5]. Due to these circumstances, VR simulators are widely used in laparoscopic training institutes, since as already stated, acquiring specific surgical skills in the operating room can be inefficient, time consuming, and potentially risky for the patients [6].

LapSim (Surgical Science, Sweden), Lap Mentor (3D System, USA), MIST-VR (minimally invasive surgical trainer-virtual reality, Mentice AB, Sweden), Simendo (Simendo, Holland) are the four common VR simulators in laparoscopic training [5,15-19]. MIST-VR, the earliest VR simulator, was presented by researchers from Manchester Royal Infirmary and Virtual Presence in 1997 (Figures 1A, 2A). The initial system consisted of a 200 MHz Pentium PC with a 32 Mb RAM linked to a jig that contained two standard laparoscopic instruments. Trainees had the possibility to simulate some of the manoeuvers performed during a laparoscopy by “grasping” and “manipulating”. Besides, it was possible to record, compare, and analyze individual task performance [20]. This was the prototype of modern VR simulators. LapSim and Lap Mentor, that appeared around 2002, have shown to be the most frequently used VR simulators (Figures 1B, 1C, 2B, 2C). Both comprise an operating platform, cameras and a monitor, and both of them provide basic skills training and procedural training. Trainees can practice their manipulation and surgical techniques through nonanatomic settings and virtual patients. Competence level of trainees could be assessed by a large number of qualitative and comprehensive performance parameters [21,22]. Simendo, that appeared the latest among VR simulators mentioned above, has smaller application range (Figures 1D, 2D). It was initially designed to simulate the endoscopy and the training mode was similar to LapSim and Lap Mentor [23,24]. Task completion time and surgical skills have been shown to improve following training with all four kinds of VR simulators [5]. Interestingly, none of the four simulators incorporated HMD helmet as graphic output device in any of the available versions, including the latest. This could be because operations in real laparoscopic surgery, like camera manipulation, hand-eye coordination and bimanual maneuvering, are accomplished by observing the monitor. According to the latest research in 2017, combination of LapSim with HMD helmet led to high levels of immersion, but there was no significant difference in the training effect [9].

Figure 1.

Four kinds of VR simulators for laparoscopic training. MIST VR (A), Lap Mentor (B), Lap Sim (C), and Simendo (D).

Figure 2.

Operation interface of four simulators. MIST VR: Realistic image data and simulated vital sign (A), Lap Mentor: Basic knotting training (B), Lap Sim: Clipping and cutting off training (C), and Simendo: Stretching with misorientation training (D).

Orthopedics

Compared with laparoscopic and other surgical disciplines, development and application of the simulators for orthopedic surgery are lagging behind, which is evidenced by a small number of studies that have attempted to create or validate orthopedic surgery simulators over the past two decades [25]. Among a very small number of available papers, research on VR simulators for arthroscopy stands out. Procedicus KSA VR surgical simulator, Procedicus virtual arthroscopy simulator (Mentice AB, Sweden), and Insight Arthro VR Shoulder Simulator (3D Systems, USA) are generally used for arthroscopy training [11].

Procedicus KSA VR simulator and Procedicus virtual arthroscopy (VA) simulator were developed by the same company and have been employed to train novices since 2002 and 2003. Procedicus KSA simulator is a kind of endoscopic simulator with haptic feedback and anatomical graphics. Procedicus VA simulator was the first specific visual-spatial simulator used for knee arthroscopy training (Figure 3A and 3B). Besides platform, cameras, optic device, and probe, it also has haptic feedback. Trainees are instructed to complete tasks on a virtual knee, while the haptic feedback occurs every time an organ is touched. However, there was no evidence to prove that performance would improve following short time training on the simulator [26-29]. Insight Arthro VR Shoulder Simulator uses a life-size plastic shoulder with predefined portals (Figure 3C). The input devices consist of two robotic arms that send haptic feedback to the user. This simulator can create different positions, like beach-chair or lateral decubitus that help young physicians develop necessary skills. In addition, trainees can choose nonanatomic tasks for training of basic visuospatial coordination or anatomic tasks for procedural skills training. This VR simulator has been regarded as an effective tool for both resident experience and shoulder arthroscopic experience [30,31].

Figure 3.

VR simulators for arthroscopy training. Operation interface of Procedicus VA simulator, the localization and manipulation of shoulder arthroscopy in vitro (A) and in vivo (B); and platform of Insight Arthro VR Shoulder and Knee Simulator (C).

Besides these three VR simulators, a group of Japanese researchers has assembled a VR simulator with an electromagnetic motion tracking system, Aurora measurement system (Northern Digital, Canada) and an artificial knee model (Pacific Research Laboratories, WA). A six degrees of freedom sensor was fixed beneath the knee model to detect surgical forces loaded on the joint and to provide the data on set of forces as well as torques. This simulator can make quantitative assessment of arthroscopic skills [32].

VR simulators for open surgery, including total hip replacement (THA), hip trauma and fracture fixation, have also been investigated by researchers [25]. Most VR simulators placed emphasis on preoperative planning and navigation in vivo. A simulator named HipNav was developed for THA training at Carnegie Mellon by Digioia et al. It contained kinematic hip joint models and tools for predicting femoral range of motion, bone motion, and optimal alignment based on implant placement. It could aid surgeons in determining optimal, patient specific acetabular implant preoperatively and in transferring it to surgical navigation in-vivo [33,34]. The TraumaVision fractured femur orthopedic simulator (Swemac Orthopaedics, Sweden) is once-famous VR simulator with haptic feedback that aimed to recreate the sensation of drilling and reaming of cortical and cancellous bones (Figure 4). It is controlled via stylus, which serves as navigator on the computer screen. The haptic feedback provided by Geomagic Touch X (Geomagic, USA) device allows users to feel realistic resistance when in contact with soft tissue and bone (Figure 5). The sensitivity of the haptic device can even generate tactile differentiation between cortical and cancellous bone [35-37]. Jun et el have developed a system to simulate total knee replacement (TKR) surgery based on 3D knee models reconstructed from computed tomography (CT) scans. The system guides surgeons to fully comprehend the patient’s 3D anatomy and to interactively determine custom implant components by extracting knee anatomical parameters from 3D triangular models. A virtual surgery based on a preoperative 3D simulation can be used to obtain surgical parameters for resecting femoral and tibial models. The virtual implants are individually attached to the distal femur and to the proximal tibia following the resection. The axial reference of implant components is aligned with mechanical axis of the femur and tibia bone [38].

Figure 4.

The platform of TraumaVision fractured femur orthopedic simulator (A) and its interface, including the placement of internal fixation for the treatment of femoral neck fractures and the imaging data of the spine (B).

Figure 5.

General view of Geomagic Touch X (A) and its interface, it can simulate drilling operation on bone tissue, and the operator can perform the operation in three-dimensional space (B).

Other surgery

Steven et al have reported an application of VR simulator in mastoidectomy performance assessment [39]. They have chosen Visible Ear Simulator (VES), a fully functional 3D virtual temporal bone simulator with force feedback, to complete this task. Participants have performed a complete mastoidectomy by entering the antrum through a virtual temporal bone, and the results have showed successful acquisition of basic competencies.

EYESI, produced by VRMagic, is currently the only VR surgical training system for ophthalmic surgery available on the market. It is a monitor-based simulator with modular that allows for a breadth of ophthalmic surgical procedures, like vitreoretinal surgery and capsulorhexis [40].

Advantages and limitations

As an upcoming tool in surgery training programs, VR simulators focus on the advanced stages of training. They can provide automated scoring with numerous objective metrics, that are very promising alternative for the laborious and subjective ratings usually performed by experts during live or videotaped procedures [41-43]. Compared with VR simulation, trainees that have received conventional surgical training have been found to perform surgery substantially slower and to be more likely to cause injury, damage tissue or fail to progress with the surgery [44]. The immersive experience created by VR simulation provides enhanced understanding of complex 3D body structures and handling of instruments for trainees in a controlled, risk free environment. Meanwhile, haptic experience in surgical simulation can help trainees to familiarize themselves with operation process and to develop their operative and decision-making skills without potential harm to patients [45,46]. However, the accuracy and efficacy of VR training still need to be improved. Density, palpable properties and convex surfaces are very challenging to simulate in a virtual environment. Moreover, the response rate and intensity feedback of commercial haptic devices are too weak to correctly emulate vibration while drilling [47].

An ideal VR simulator for surgical skills training must satisfy several conditions, including multimodal training plans, artificial platform, accurate haptic feedback, immersive visual and audio technologies, sensitive input devices, as well as appropriate software with real time simulation and assessment criteria. In addition, it should incorporate data from MRI or CT to provide patient-specific simulations. Combination of physical body models and actual surgical tools with VR simulators promotes the interaction between realistic tools, virtual bodily fluids and virtual physical materials. Abnormal and patient-specific cases can be modelled and assessed in VR simulators for trainees to practice and to prepare in vivo. These features and limitations may provide some directions for the future development of VR technology for surgical skills training [25].

VR technology for pain management

Pain, that is acute, intermittent, or chronic in nature, is the most universal medical complaint among hospitalized patients. To enhance pain management, physical and psychosocial treatments must occur simultaneously. However, since pain has different causes, it is difficult to establish standards for when and how to change pain therapy and how to deliver the same [48]. VR technology has been firstly proposed as a tool for pain modulation by Hoffman et al in 1998. This new treatment has shown to be effective in reduction of burn-induced pain and management of pain in other situations [49-52].

VR for acute pain management

Acute pain caused by operation or trauma induces a wide range of pathophysiological responses. Inflammatory, physiological and subsequent behavioral responses follow the activation of nociceptors through tissue injury. Sympathoneural and neuroendocrine activation, combined with uncontrolled pain, can ultimately lead to various detrimental responses [53]. The management of pain in the acute care setting often relies on pharmacological treatments, such as anesthetic and analgesic agents, to attenuate these pathophysiological responses [54].

However, the arrival of VR technology brought another path for the management of acute pain. Debashish et al have investigated the effect of VR based games in management of burn-induced injuries. In their research, the VR equipment used to enable interaction with the virtual environment included a laptop (Dell, USA) with game software, a HMD (IOGlassess HMD with SVGA video resolution of 800x600 16 million colors), and a tracking system (Intersense, USA). This randomized controlled trial study has proved that VR technology provided analgesia with minimal side effects and very little impact on the physical hospital environment [55]. In addition, several other studies have confirmed that standard analgesia coupled with VR based games is effective in reducing acute pain [52,56,57]. The composition of VR equipment was like Debashish’s design, including computer, HMD, game software, headphone and input device.

Another cutting-edge approach for use of VR technology in pain management refers to augmenting hypnosis, also known as virtual reality hypnosis (VRH). According to Patterson, no special hardware, besides VR equipment is necessary. The most interesting point in his study was the application of Vega VR-world-building software (MultiGen-Paradigm, Canada). Participants “glided” through an icy 3D virtual canyon following a predetermined path and shot virtual snowballs at virtual snowmen, igloos, robots, and penguins. Sound effects and animated colors were simultaneously incorporated in VR condition [58]. The patients reported lower levels of pain and anxiety under this condition. In a study published in 2009, based on hypnotherapy theory researchers have developed a VR sequence to replace distraction type programs. The use of VR for relaxation, in addition to use of morphine for pain reduction have shown to be very effective [59].

VR for chronic pain management

Chronic pain is typically defined as pain lasting longer than three months, or beyond the expected period of healing [60]. Chronic pain conditions are difficult to treat because of non-effective traditional pharmacologic treatments and invasive nerve blocks. Recent advantages in functional imaging technology have revealed that chronic pain conditions are caused by neuronal plasticity in the central nervous system (CNS), even though the underlying pathogenesis has not been fully understood [61]. Only a few studies have investigated the use of VR for chronic pain management, therefore very little is known about the use of VR for chronic pain treatment and for long-term pain rehabilitation [1].

Sato et al have developed an advanced VR based mirror visual feedback system and have used it to treat complex regional pain syndrome. The non-immersive computer-based desktop VR system includes a computer, a real-time position and motion tracker (Polhems, USA), a CyberGlove (Immersion, USA) that serves as a hand input device, and a monitor. A virtual environment was developed using Autodesk 3DS Max (San Rafael, USA), where the exercises are target-oriented motor controlled tasks via various movements like reaching out, grasping, transferring, and placing. Their study has demonstrated that VR system might increase analgesic efficacy and provide a promising alternative treatment for complex regional pain [62]. Besides, the effect of tracker-based VR system on chronic neck pain treatment has been investigated. Participants were asked “spray” flies that randomly appeared in the virtual environment provided by the HMD. Researchers have assessed cervical range of motion, which has shown to be significantly related with pathophysiology of chronic neck pain [63].

Advantages and future perspective

The most powerful analgesics cannot control pain in every patient, although opioid analgesics (e.g., morphine and its derivatives) have proven to be quite effective in providing pain relief [64]. VR technology and hypnosis are two potential alternatives to pharmacologic treatments of pain. The use of VR technology that enhances distraction has shown to be relatively effective method for pain management (Figure 6). VR can affect pain perception through immersive virtual environment, by occupying finite attentional resources and by blocking external stimulation associated with real environment and the painful stimuli [65]. Since distraction interventions work by competing for attention otherwise directed towards painful stimuli, pain tolerance and pain threshold have shown to increase under VR conditions [66,67]. Moreover, pain intensity, anxiety and time spent thinking about pain have shown to decrease following VR distraction [67].

Figure 6.

Immersive virtual environment created by the VR equipment [69], an image of the 3D virtual world named SnowWorld. Image by Stephen Dagadakis, U.W., copyright Hunter Hoffman, U.W. (A) and non-immersive virtual environment created by the VR equipment [62], a personal-computer-based desktop virtual reality system for mirror visual feedback therapy. The arm on the affected side (right) and the targets appear in the virtual environment (B).

Immersive VR technology is more likely to generate relief from pain compared to non-immersive VR technology [68]. Pain relief effect has shown to be correlated with technological advancement of instruments used to generate VR; the greater sense of presence in the virtual world created by high-tech equipment that incorporate an active cognitive processing component are more effective in alleviating pain [69,70]. Pain signaling in key pain matrix regions could be inhibited via VR technology. As the price of VR equipment keeps lowering, this approach is becoming increasingly affordable and could potentially turn into self-management tool for pain relief used by patients in and/or out of hospital. For example, a simple Sony PlayStation with a VR HMD could be employed to treat pain [67]. Although research results support the premise that VR devices can reduce pain, the neurobiological mechanisms still need to be determined. The current state of VR as a tool for pain management is still in its early developmental stages and it requires numerous applications for patients with an array of acute and chronic medical conditions. Nonetheless, VR technology will eventually emerge as a promising first-line intervention and complementary therapy for patients with pain [1].

VR technology for psychological diseases therapy

The treatments for many psychological disorders commonly require patients to confront the situations they fear. This kind of approach, also known as exposure therapy, helps patients to accept their anxious emotions and consequently change their beliefs about the likelihood or catastrophic nature of feared consequences. Exposure therapy is very effective but hard to execute, since complex conditions such as social situations, thunder, injury and other phobic stimuli are potentially very challenging to recreate and to expose individuals to the same in a real life setting [71]. Consequently, virtual environment created by VR simulators could be a valuable option for exposure therapy [72].

According to the latest randomized controlled trial for social anxiety, in vitro exposure through VR has shown to be superior compared to in vivo exposure, based on self-reported outcomes registered six months following the treatment [73]. Dana et al have designed virtual environments for phobia treatment. They have searched the methods for creating and combining virtual objects, environments and situations for therapy (Figure 7A). HMD and data glove have been required for immersion and active participation in the virtual world [74]. A group of researchers from Spain have used VR system to enhance emotional skills in children with autism spectrum disorders (Figure 7B and 7C). The system contained L-shaped screens, which allowed the child to observe the environment from the frontal view and to act from below. Mood and pose of the child were determined by a robot with an eye-in-hand system. The avatars in the virtual world and their moods were design via 3DS Max. The child could interact with different avatars and could carry out a series of emotion recognition tasks, thus practicing the emotional responses [75].

Figure 7.

Virtual environment for different symptoms specifically [74], a virtual environment of a plaza created by D. Horváthová and V. Siládi using Blener 3D (A), a virtual environment of a birthday party created by G Lorenzo et al using Vizard program (B), and a child interacted with virtual person [75] (C).

Advantages and directions

Virtual reality exposure therapy and virtual reality cognitive behavior therapy have become effective choices for patients with anxiety disorders and other phobias like fear of flying, claustrophobia, acrophobia or generalized social phobia [76-79]. Instead of being exposed to the real world, patients can experience an anxiety-provoking scenario in the realistic virtual environment generated by VR technology. Patients and therapists can both benefit from this kind of treatment, especially in view of advanced VR technology that is becoming ever more available [80]. Furthermore, VR technology can be used as self-treatment to some extent [81]. VR devices can be used in private practices and even private homes, which is very important since it may additional aid patients to cope with unwanted feelings in “emotionally safe environments” [1].

According to existing publications, the software is the core part affecting the treatment outcomes. Virtual models and environments designed for medical therapies vary according to patient’s specific symptoms. The simultaneous use of panoramic photos, videos and models controlled by the therapist may be helpful, since the number and category of models as well as the interaction methods with virtual objects can be adapted according to therapeutic progress. Moreover, measuring principles are vital software component. Every patient’s symptom is different, each VR platform has diverse working conditions and therapy procedures largely depend on individual therapist’s decisions. An appropriate evaluation system for VR therapy alone, or for VR therapy combined with other therapeutic methods are urgently needed. Treatment of psychological diseases is very time consuming, thus VR devices providing plentiful and vivid contents may be of great help for patients going through difficult periods.

Conclusion

Virtual reality technology is widely applied in the field of medicine. Huge benefits have been reported following its use for rehabilitation, disability management, surgical training, psychological diseases therapy and analgesic modality [3,28,36,62,76,82]. This review presented current virtual reality based simulators for medical application, especially in surgery training, pain management and therapy for psychological diseases. A comparison between operation methods and existing simulator features and relative results was provided to demonstrate the status quo as well as recent developments. In addition, new directions in VR technology research and clinical significance were addressed by evaluating existing literature on the efficacy of VR simulators.

Disclosure of conflict of interest

None.