Virtual reality in surgery

New technologies, in particular virtual reality and robotics, will have a major impact on health care in the next decade. Clinically validated, powerful medical simulators are now available and in use across the world. General surgery leads in the use of simulators, and neurosurgery leads with augmented reality and image guided surgery. Robotics are used in orthopaedics and cardiology. Other virtual reality applications are being used in mental health, anaesthetics, and emergency medicine. Rapid developments in the internet and “e-learning” domains have accelerated the dissemination of simulation techniques, interactive 3D images, and structured courseware. This review describes the application of virtual reality and robotics to surgical training and planning and the execution of procedures in theatre and discusses the near term future of this new technology.

Methods

A successful medical simulator or surgical system based on virtual reality requires the participation of a team of specialists including experts in ergonomics and applied psychology, software engineering and digital 3D design, electromechanical engineering, robotics, and microtechnology. Consequently, no single database adequately covers all the issues involved. This review is based on our experience supplemented with data from searches of Medline and the Ergonomics Information Analysis Centre (University of Birmingham) and of the internet with various web search engines.

Historical perspective

Current virtual reality surgical systems owe their existence to pioneering developments in the early and middle 1980s. Organisations developing robots to replace humans from hostile and hazardous environments—undersea, in nuclear installations, in space, and on the battlefield—turned to an emerging technology that seemed to offer the ideal solution. The developers claimed that, with a special helmet equipped with head tracking devices and 3D displays, it was possible to create an illusion for the wearer that he or she was present in such an environment (“telepresence”) and, with a fibre optic glove, possible for them to use their natural skills to control the robots to perform tasks safely and efficiently.

Anticipated developments

• Introduction of robot assisted and telerobotic surgery into selected disciplines

• Use of computerised simulations to train and assess psychomotor skills needed to perform invasive procedures

• Use of virtual reality technologies to interact with medical images for surgical planning and training

• Use of virtual reality simulators to rationalise surgical training and assessment of fitness to practise

• Use of virtual reality simulators to guide micro-robots undertaking minimally invasive interventions

In the 1990s research teams, notably at the University of North Carolina¹ and in the US Department of Defense,² developed the concept of surgeons of the future equipped with virtual reality headsets and rehearsing real or robotic procedures using advanced computer generated images. At the same time, the growing market for virtual reality technologies encouraged pioneering (if somewhat optimistic) US companies, which were singularly responsible for fuelling the obsession with “making surgical simulation real”.

Recent conferences and exhibitions (such as the US hosted Medicine Meets Virtual Reality 2000³) suggest that a plateau may now have been reached, with some of the front running concepts undergoing consolidation through clinical validation. Established products are becoming available at prices that are affordable to most surgical teaching institutions. In addition, many projects are now receiving academic grant support or national and EU funding (as with the Framework V initiative⁴).

What is virtual reality?

Virtual reality is best described as a collection of technologies that allow people to interact efficiently with 3D computerised databases in real time using their natural senses and skills. This definition avoids any reference to a need for head mounted displays and instrumented clothing such as gloves or suits, as was the bias in the late 1980s and early 1990s. Although this so called immersive technology is still evident today, only 10% of virtual reality applications warrant its use. The key strength of virtual reality, be it in design or training, is that it supports and enhances real time interaction on the part of the user.

The application of this technology to surgical training is evident. Surgical training is expensive, and the pressures from shortened training programmes and reduced working hours for trainees demand that an increasing proportion of the surgical expertise of trainees has to be gained outside the operating theatre. The crucial factor that will determine the uptake of virtual reality technology by surgeons will be the demonstration that virtual reality is capable of delivering reliable and valid training and assessment systems. Recent evidence suggests that this is the case. Not only has the virtual reality community produced experimentally validated systems for the training and assessment of surgical skills, it has done so using established techniques that are now becoming recognised as international standards, such as the International Organisation for Standardization's ISO 13407 “Human centred design for interactive systems” (see box).⁵

Guidelines for ISO 13407 “Human centred design for interactive systems”

• Active involvement of users

• Clear understanding of use and task requirements

• Appropriate allocation of function

• Iteration of design solutions

• Multidisciplinary design

Current applications

There is a substantial difference for surgical trainees between training with artificial or inanimate tissues (such as raw chicken) and supervised procedures on patients in the operating theatre, with all the attendant pressures such as time restraints and clinical governance. A wide range of virtual reality training systems have been developed, but not all are widely available yet. A commercially available simulator for venepuncture has force feedback to simulate the feel of the cannula entering the skin and vein.⁶ This is suitable for training nurses, medical students, phlebotomists, and paramedics. More complex simulators for therapeutic gastroscopy, endoscopic retrograde cholangiopancreatography, and colonoscopic procedures are available for trainees in gastroenterology.⁷ A radiological simulator provides training in cardiac catheterisation and angiography, with real time modelling of physiological parameters and blood flow.⁸ These virtual reality simulators offer repeatable, logged, computerised training, often without the need for supervision.⁹

In addition, some interactive virtual reality simulators that have been developed for procedures such as lumbar puncture and brain ventricular tap are freely available for use over the world wide web.¹⁰

The MIST system

The MIST (minimally invasive surgical trainer) system, a product for training and assessment of surgical laparoscopic psychomotor skills, was originally developed by us in Britain and is now commercially available from Mentice Medical Simulation AB, Gothenburg, Sweden.⁸ In the first step in its development we made an ergonomic evaluation of the psychomotor skills involved in performing laparoscopic surgery in theatre. This led to the identification of a set of simplified minimally invasive part tasks that represented the “toolkit” of skills needed for holding tissue in an accurate and steady manner, adopting different styles of handling or separating tissue and vessels, left hand and right hand instrument control, and appropriate use of electrocautery. We found that each simple part task could be implemented reasonably easily within a proprietary virtual reality software package and each could be associated with equations of human perceptual and motor performance developed from applied psychology studies.¹¹,¹²

The MIST system's training interface, based on modified laparoscopic instruments, is translated into quite simple real time 3D computer graphics that accurately track and represent the movements of the instruments within a virtual operating volume. In this volume, geometric shapes that approximate to those faced during actual operations on organs are generated for display on the computer screen and subsequently manipulated by a surgical trainee (fig 1 ).¹³ Each task can be programmed to deliver varying degrees of difficulty to the surgical trainee, and his or her performance can be recorded and saved for later replay by the supervisor or for statistical analysis. The data can be analysed in several ways, focusing on such aspects as accuracy and errors, time to complete part tasks, right or left hand performance, and even the trainee's economy of movement when handling the virtual instruments.¹³ The advantage of the part task approach to simulation, compared with full anatomical simulation of organs and operations, is that they train generic skills for minimal access surgery common to many surgical disciplines—such as general surgery, gynaecology, thoracic and cardiac surgery, urology, and orthopaedics.¹¹

Figure 1.

Using the MIST system (Mentice Medical Simulation AB, Gothenburg, Sweden) for training and assessment of psychomotor skills for minimally invasive surgery

Objective assessment of surgical skills

Many factors can influence surgical outcome, and psychomotor ability is only a small part of the process of surgery. However, objective assessment of operative skill is fast becoming necessary.¹⁴ The opportunities presented by tools such as MIST go beyond those of a computerised trainer. Randomised controlled studies have shown that MIST can distinguish between experienced surgeons and non-surgeons or inexperienced surgeons (A G Gallagher et al, ninth annual medicine meets virtual reality conference, Newport Beach, CA, 2001).¹⁵,¹⁶ Other factors affecting surgical performance, such as alcohol intake and sleep, can also be evaluated (A Chaudhry et al, medicine meets virtual reality 7, San Francisco, CA, 1999).¹⁷

The objective assessment of the psychomotor aspects of surgical performance is now a practical and affordable reality.¹³ In Europe, Sweden is investing heavily in medical virtual reality technologies: a new part task virtual reality simulator for training laparoscopic surgeons will be marketed in Sweden later this year¹⁸ as well as a full anatomical organ simulator for laparoscopic surgery.⁸ Training on a virtual reality simulator has to be shown to translate into performance in the operating theatre. Hence, the demonstration that the objective assessment of skills to manipulate virtual 3D structures from a 2D monitor display is a significant predictor of laparoscopic surgical performance represents an important advance (A G Gallagher et al, ninth annual medicine meets virtual reality conference, Newport Beach, CA, 2001). Another possible use of virtual reality simulators might be to select medical students or young graduates on their aptitude for surgical skills. It may also be possible to use this type of simulator to check on the psychomotor skills of experienced surgeons to ensure their competence to continue to practise. These systems are attracting the attention of the Royal Colleges of Surgery and are already an integral part of training courses at the European Surgical Institute, Hamburg, Germany, where the MIST system is already a mandatory component of basic and advanced courses covering a wide range of surgical techniques (T Buerger and M Erdtmann, ninth annual medicine meets virtual reality conference, Newport Beach, CA, 2001).

Surgical planning

Virtual reality technologies allow an operation to be practised, and the outcome viewed, before the patient undergoes surgery—such as in breast reconstruction and corrective maxillofacial surgery). Thus, the surgical approaches can be optimised and rehearsed, with obvious advantages for patients and healthcare providers. Virtual reality for surgical planning and training purposes is being investigated in an EU (Framework V) funded project called IERAPSI (integrated environment for rehearsal and planning of surgical interventions) (R J Stone, project IERAPSI: a human-centred definition of surgical procedures. Deliverable D2 (Part 1) for EU contract No IST-1999-12175; May, 2000). The investigators used the same human factors techniques as those used during the early work on MIST (detailed surgical task analyses, elicitation of surgeons' knowledge during video replay and focused interviews) to analyse actual surgical procedures involving mastoidectomy, cochlear implantation, and acoustic neuroma resection. The results of these analyses highlighted the need for a virtual reality system to plan operations and train surgeons. The IERAPSI training system will be based on new technologies, including a stereoscopic virtual reality microscope and a special desktop stylus device called PHANTOM (SensAble Technologies, Cambridge MA, USA). The PHANTOM is being programmed to deliver a wide range of force and touch (haptic) effects to trainees' hands (fig 2), including the sensation of drilling through different densities of bone.

Figure 2.

The PHANTOM haptic feedback device (SensAble Technologies, Cambridge MA, USA). The user holds a small stylus and explores, in this case, a 3D virtual molecular structure. Each time the 3D cursor makes contact with part of the structure, small motors work in tandem to restrict the movement of the stylus, thereby creating a sense of touch (reproduced with permission).

Challenges of introducing virtual reality into routine surgical practice

Little is known about the computer literacy of the medical profession, but computer anxiety and alienation are real problems¹⁹ and healthcare professionals over the age of 30 are regarded as the “lost generation” with regard to information technology.²⁰ The NHS is now making a substantial effort to introduce health informatics to hospital doctors and has been piloting the introduction of the European Computer Driving Licence, which is set to become the baseline qualification for future jobs involving information technology.²¹

The wide acceptance of high tech surgery, such as using videos in laparoscopic procedures, has overcome many surgical prejudices, but virtual reality and robots are still viewed as gimmicks or potentially dangerous by most surgeons. More multidisciplinary teams will be required to develop the use of these new technologies in surgery. As with so many advances in health care, however, these new technologies are likely to increase costs, against which must be weighed the potential for improved surgical competence and reduced medical error, with reduced morbidity and mortality. Evidence for these benefits is likely to take at least 5-10 years to accumulate.

The future

Apart from the development of current applications, virtual reality also probably has a role in such exciting developments as microsurgery and nanosurgery. Just as virtual reality was first developed to control macro-scale robots for use in hazardous environments, future microrobots and even nanobots, such as the DNA “screwdriver,”²² designed for use within the human body will need supervision by skilled operators equipped with advanced virtual reality equipment. Already a German company has produced a “micro-submarine” powered by an induction motor; at 4 mm long and 650 μm in diameter, it is small enough to pass down a hypodermic needle and has the potential for various diagnostic or therapeutic applications.²³ Miniature cameras that can be swallowed and transmit images of the gastrointestinal tract to a viewing station are being used to survey the gut in a variety of diseases.²⁴,²⁵

The fact that European and US developments in micro-lasers and micro-manipulators, such as the da Vinci Surgical System (fig 3), are now being used for master-slave robotic procedures such as minimally invasive coronary artery bypass grafting and laparoscopic surgery²⁶ is proof of the application of virtual reality technology and suggests that a revolution in medical instruments and in the training of those who use them is closer than many people realise. In keeping with the early attempts to introduce technologies such as virtual reality and robotics into other markets, a sea change in medical opinion will be required and a massive learning curve will have to be overcome if the advances already achieved, never mind those to come, are to be translated into realities in health care.

Additional educational resources

Books

• Stanney KM, ed. Virtual environments handbook. Mahwah, NJ: Lawrence Erlbaum Associates (in press). (see http://vehand.engr.ucf.edu/revised2.htm)

• Karkowski W, ed. International encyclopaedia of ergonomics and human factors. London: Taylor and Francis, 2001

• Websites

• Surgery at the Manchester Royal Infirmary. www.biomedical.demon. co.uk/vsurgery.htm (accessed 7 Sep 2001). (Links to laparoscopic surgery, MIST, robotics, and other related surgical technology sites)

• Internet resources of computer aided surgery. http://homepage2.nifty. com/cas/ (accessed 7 Sep 2001). (Includes surgical planning, surgical navigation, image guided surgery, and surgical robotics)

• Emerson T, Prothero J, Weghorst S. Medicine and virtual reality: a guide to the literature (MedVR). Washington DC: Human Interface Technology Laboratory. www.hitl.washington.edu/projects/knowledge_base/medvr/medvr.html (accessed 7 Sep 2001)

• NHS Information Authority. Ways of working with information. www.nhsia.nhs.uk/wowwi/pages/default.asp (accessed 7 Sep 2001). (Education, training, and development programme—includes details on health information, information technology, basic information technology skills, and the European Computer Driving Licence project)

• Mentice Medical Simulation. www.mentice.com (accessed 7 Sep 2001). (Details of MIST system and surgical simulation)

• Surgical Science. Surgical Science—cutting edge in surgical simulation. www.surgical-science.com (accessed 7 Sep 2001). (Details of the LapSim laparoscopic simulator)

• Web-based surgical simulators and medical education tools http://try.at/virtual.surgery (accessed 7 Sep 2001). (Details of lumbar puncture, ventricular tap, etc)

Figure 3.

The da Vinci Surgical System (Intuitive Surgical, California, USA) for performing minimally invasive surgery. The surgeon sits at a control console with 3D visualisation of the surgical field and the robotic surgical instruments (reproduced with permission)