Is the use of augmented reality-assisted surgery beneficial in urological education? A systematic review

Abstract

Background:

Google Glass is an optical head-mounted display that has been used in multiple medical and surgical settings to enhance delivery of education and training. This systematic review focuses solely on the use of this technology in urology operating theaters for the purpose of surgical education.

Materials and methods:

A systematic search strategy was employed using EMBASE (1996–2019), Medline (1946–2019) and PubMed. Search terms included optical head-mounted displays, Google Glass and urological surgical training. Use of this technology in a nonurological setting, nonteaching sessions, case reports, reviews, editorials, abstracts, and articles not in English were rejected. Three studies were identified following the exclusion criteria.

Results:

All 3 studies received positive feedback from trainees regarding this technology in relation to enhanced surgical education. In addition, in all studies the trainees felt the technology had a place for educational training in the future. All studies described disadvantages to the technology as well including battery life, comfort, and cost.

Conclusions:

Studies describe a big potential for Google Glass and similar head-mounted devices for the role of surgical training in urology, however, larger studies looking at more varied operations can help reinforce this viewpoint.

1. Introduction

Technology in surgery is an ever advancing arena, with minimally invasive endoscopic, laparoscopic, and robotic approaches, particularly in the field of urological surgery replacing historic open procedures.^[1] Surgical innovation is essential in moving forwards surgical standards, improving both surgical education for trainees and outcomes for the patients.

The traditional method of one to one teaching by mentor and trainee in the operating room is labor and time intensive especially in the current climate where surgical trainees have increased demands on their time, duty-hour restrictions and changing administrative infrastructures that overall result in limited operating time for trainees. Likewise, the number of surgical trainers is under pressure as demands on their senior experience and expertise grows.

Augmented reality (AR)—defined as a ‘technology that superimposes a computer-generated image on a users’ view of the realise world, thus providing a composite’ is in widespread use in number of domains including aeronautics and the military.^[2]

The advent of AR and the use of optical head-mounted devices (OHMD) such as Google Glass (Google Glass, Mountain View, CA) have brought another technical revolution to surgery (Fig. 1).

Figure 1.

Google Glass Augmented Reality device.

The first prototype of Google Glass was produced in 2011 as a head-mounted computer. Since then many iterations and developments have been made and in 2014, they were made available by Google I/O (Innovation in the Open) at a cost of $1,500.

The feasibility of Google Glass in surgical setting was first described in Germany by Muensterer et al.^[3] in a pediatric surgery 2014 and has since been successfully explored as a teaching instrument in a range of surgical settings.

Google Glass comprises a head-mounted computer with a prism for 720p HD display, processor, touch sensitive controls, and a gyroscope. Users wear Google Glass like a pair of glasses, with the OHMD projecting a display in front of user's eyes. It supports Wifi and Bluetooth allowing communication with either a phone or directly to the internet.^[4]

It comes with the main system on Chip Texas Instruments OMAP4430 Dual 1.2 GHz (ARMv7), 2GB RAM memory, 16GB of storage via Sandisk, an InvenSense MPU-9150 (gyroscope, accelerometer and compass) and a Wolfson WM7231 MEMS microphone and a 5-megapixels camera.^[4]

The Google Glass device is very light weighing 36 g; the same weight category as a regular pair of glasses. Interaction with the Glass is mainly through two primary means: voice activated commands and the Touchpad that runs along the right hand side of the frame. Glass can be activated by head tilt of 30° (can also be altered for preference) or simply tapping the right side of the frame. It turns itself off after a few seconds of inactivity. Google Glass can be controlled with voice activation. Wearers can say an action such as “record a video”. Responses are relayed back to the user using the bone conduction speaker next to the battery that transmits audio through to the inner ear (via bone conduction rather than soundwaves into the user's eardrum)—rendering the sound almost inaudible to other people, effectively providing ears-free audio.^[4]

Effectively the device acts as an interface through which it receives information, gets notified of new content, allows the wearer to interact and share information with others. It serves as the user's home screen allowing control of the graphical of the operational field.

Google also launched an app called MyGlass for both Android and iOS to go with the Glass. The MyGlass app enables users to configure and manage the device.

In the surgical operating room, the primary aim of these technologies is to provide a medium for both training and trainee surgeon to share the same field of view. Furthermore, OHMD technology provides the opportunity for the training surgeon to assist, direct, and advise remotely. OHMD technology has been utilized in multiple surgical fields, including general surgery, orthopedics, neurosurgery, vascular surgery, pediatrics, and urology.^[5] Google Glass with live stream capability has also been used to facilitate tele-proctoring between surgeons in the United States and low- and middle-income countries with significant success.

Within the field of urology, Borgmann et al.^[6] utilized OHMD technology to perform 31 AR-assisted surgeries over 10 different operation types, finding it to be both a safe and useful tool in the operating theater. OHMD technology has also been utilized as a method of displaying vital signs to the surgeon, with Iqbal et al.^[7] finding that both inexperienced and experienced surgeons alike reacted to deranged vital signs with OHMD with no detrimental effects to surgery.

In this systematic review, the authors look to analyze all research pertaining to the use of OHMD devices such as Google Glass in the field of urological surgical education.

2. Material and methods

A systematic search strategy was employed according to PRISMA guidelines using EMBASE (1996–2019), Medline (1946–2019) and PubMed. Key words implemented for search purposes included: “Google Glass,” “head-mounted displays,” “OHMD,” and “urological surgical training.” Reference lists in included papers were also reviewed to identify any additional appropriate articles.

Inclusion and exclusion criteria were predefined. Studies were excluded if they were in a nonurological setting, nonteaching sessions, case reports, reviews, editorials, abstracts, and articles not in English.

The search was performed independently by 2 investigators (S.G. and S.S.) in November 2019, where disagreements occurred with regards to suitability for inclusion, arbitration was done by I.A.A.

2.1. Data extraction

In total, after removing repeats, the search strategy yielded 92 unique results. After reviewing titles, abstracts and full text where necessary, 3 individual articles met the criteria for systematic review. Data was extracted for analysis, authors name, year of study, study population size, objective of study, and outcomes.

Reasons for rejection of the remaining articles were: abstract only (n = 45), use of technology in non-urological setting (n = 20), use during surgery but not for training (n = 11), validation/development of the system but not use in training (n = 4), review articles (n = 7), and editorials (n = 2).

3. Results

In total, the 3 papers included in this systematic review focused on the use of AR with OHMD or Google Glass in 87 medical students, trainees, and consultants in the field of urological surgery. The 3 studies are summarised below in Table 1.

Table 1.

Comparison of studies included in this systematic review.

Title	A novel interactive educational system in the operating room–the IE system	Augmented reality assisted surgery: a urologic training tool	The effectiveness of Google GLASS as a vital signs monitor in surgery: a simulation study
Author	Nakayama et al.[8]	Dickey et al.[9]	Iqbal et al.[7]
Number of participants	20 (all medical students)	30 (10 faculty, 20 trainees)	37 (24 medical students, 8 urological trainees, 5 consultants)
Device	Interactive educational system – Sony OHMD	Google Glass OHMD	Google Glass OHMD
Benefits of using technology	1. Allows communication between surgeon and student.2. Improved visualisation of operative field.3. Most thought it would improve anatomical knowledge.4. Increased motivation in the operating theatre with technology.	1. Allows communication between trainer and trainee.2. Found to be educationally useful for trainees.3. Better understanding of surgical procedure by trainees.4. Better understanding of anatomy by trainees.	1. Quicker response to change in vital signs of simulated patient.2. No increase in bleeding, sweep speed, grams vaporized or laser distance from the tissue compared to no technology.3. Does not reduce surgeons operative performance.
Limitations of study	1. Small sample size.2. Questionnaire data collection system subject to recall bias.3. Personal bias (e.g. by previous experience in surgery).	1. Small sample size.2. Questionnaire data collection system subject to recall bias.	1. Small sample size.2. Questionnaire data collection system subject to recall bias.3. By repeating the activity straight after each other, using the technology second, experience gained in previous session may account for improved results.
Other aspects	1. Most would like to use the technology in the future.2. Most found it easy to use.	1. Seen to be useful for future training.2. High percentage would consider usage in future practice.3. Most found it easy to navigate.4. Questions raised regarding if a trainee relies on the technology would they become deskilled?	1. High percentage would want to use the technology again for a similar procedure.

Nakayama et al.^[8] used the interactive educational system Sony OHMD to improve education in Japanese medical students. The authors quote a shortage of experienced surgical trainers in Japan and proposed that medical students often have unfavourable educational experiences in the operating theater, often feeling intimidated, ignored or unwelcome by surgical staff leading to a decrease in junior doctors electing and progressing to higher surgical training. To provide sufficient and improved surgical training, with the aim of enhancing the visual information and interactive communication between the trainee and operating surgeon the study looked at using OHMD technology during laparoscopic radical or partial nephrectomy or laparoscopic radical prostatectomy. During surgery, senior medical students would wear OHMD and audio transmitters so the lead surgeon could see a magnified 3D view of the operative field whilst being able to communicate two-way with the surgical educator performing the operation.

In total, 20 5th and 6th year medical students from Tokyo Medical and Dental University voluntarily entered the study. Following the surgery, medical students completed a questionnaire based on a 5-point rating scale (1 = lowest, 5 = greatest). Questions were about satisfaction of surgical education with and without the use of OHMD. Students were also asked to evaluate comfort. Overall, students reported they had previously not had favorable experiences in the operating theater, expressing they did not feel welcome (1.6 out of 5 points) and often hesitated to ask questions 2.6). Whilst using OHMD, students felt more motivated (4.5 points), more welcome (3.4 points), and less hesitant to ask questions (3.6 points). Students also reported that they felt the use of the technology improved their knowledge of the anatomy (4.3 points).

The second study by Iqbal et al.^[7] was a prospective, observational, and comparative study that recruited a total of 24 medical students (novices), 8 urology trainees (intermediates), and 5 urology consultants (experts) from various institutions in the United Kingdom during a simulated laser prostatectomy.

Participants carried out the procedure on the previously validated GreenLight Simulator (Boston Scientific, Marlborough, MA) using a standard vital signs monitor that was manipulated to represent various events in surgery such as tachycardia, drop in blood pressure and changes in oxygen saturations. The session was then repeated using the Google Glass device to monitor the vital signs. The study was carried out during a simulated surgical environment with an anesthetist and a scrub nurse present to replicate the surgical scenario. Outcome measures of the study focused on 4 parameters; the time taken to respond to changes in vital signs, the effect of Google Glass on technical performance (provided by the simulator), the effect of Google Glass on the participants’ heart rate and the acceptability and feasibility of Google Glass during the procedure.

They found use of Google Glass allowed for a quicker response time to deteriorating vital signs which was statistically relevant (p = 0.026). They found no statistically significant effect of Google Glass use on technical skills of the participants. The average heart rate of the participants during both sessions was similar (p = 0.77) and subjects did not report anxiety or nervousness and the majority (75.7%) of participants agreed that Google Glass increased their awareness of vital signs during surgery and that they would like to use it again on a different surgical procedure at a later stage. Total 81% agreed the Google Glass was comfortable to use.

The third and final study Dickey et al.^[9] conducted a study on a total of 10 faculty and 20 urology residents on the use of Google Glass OHMD technology to train urology residents in the United States in implantable penile prosthesis (IPP) placement. There were two main components to this. The first component was that trainees were able to use OHMD to view demonstration videos on the process of IPP placement in a penoscrotal approach. This projected the steps of IPP procedure over the patient in real time, allowing the urology resident to visualise the steps of surgery before doing them. The second component was the ability for the OHMD technology software to detect areas of interest throughout the surgery, allowing faculty to interact with the urology resident. This could be suitable to a remote surgeon who would be able to interact with the urology resident highlighting areas of interest with a cursor.

Following the procedure, all surgeons were asked to complete a questionnaire based on a 10-point rating scale (1 = not at all, 10 = always) regarding the effectiveness and interest in the device and software. Survey questions included both quantitative and free text options. Other questions included a binary yes or no response. The surveys were then stratified into trainee and faculty categories and calculated to determine the mean rating for quantitative questions.

Results found the device to be educationally useful (8.6 out of 10 points), easy to use (7.6 points) and likely to want to use again (7.4 points). The technology was not found to be overly distracting (4.9 points).

3.1. Benefits of the device

As technology continues to evolve and permeate every aspect of medicine it is very likely that these devices will evolve to take a larger role within the operating theater.

All 3 studies by Iqbal et al.,^[7] Nakayama et al.,^[8] and Dickey et al.^[9] show significant potential advantages of the use of OHMD in the urological surgical training. They demonstrate the integration of the OHMD in the surgical training of medical students and senior urologists. Iqbal et al.^[7] believed the design of Google Glass to have the design specifications to allow use without obstructing direct and peripheral vision that can prove to be beneficial for use in the operating room such as increasing awareness of patient vital signs, as accepted by candidates of various levels of experience. Dickey et al.^[9] reported the trainees, who are younger often favored the implementation of the devices more than the training counterparts and often found its navigation easier, more educationally useful and less distracting in the operating room compared to their senior colleagues.

3.2. Difficulties encountered

All 3 studies commented on the technical difficulties as well as physical strain associated with using the OHMD. Commonly poor battery life and overheating of the headsets as well as findings of fatigue and eye strain whilst using them were reported. The cost of the device also was a hindering factor in implementing its widespread use.

3.3. Limitations of the studies

Limitations of all 3 studies were the small number of participants. A common drawback of the studies that investigate perceptions and attitudes such as the study by Nakayama et al.^[8] is the role of personal bias. For example, an individual's personal interest in surgery may be biased. In addition the data gathered through a questionnaire includes recall bias. There is potential bias that the data collected was from two different situations including the previous surgical experience and the current study. With regards to the device itself, 10% of users describe discomfort, 15% and 20% reported fatigue in body and eyes, respectively.

The study by Iqbal et al.^[7] had limitations also. As the Google Glass element was tested following the standard prostatectomy procedure it is possible that participants had an improved learning curve and may have eased into the procedure and felt more comfortable during the assessed procedure. In addition, the urology trainees and consultants were only assessed for 10 minutes versus 20 minutes for students due to limited time constraints, hence were unable to complete a full prostatectomy procedure.

They also reported some disadvantages of using the Google Glass in its current form. Participants who already wear glasses reported more discomfort wearing this device over their own spectacles. In addition, they reported poor battery life, requiring recharging every 2–3 hours—making it impractical during prolonged surgeries. In addition, as the optical device is in front of the right eye, users who were left handed reported significant discomfort.

Dickey et al.^[9] report the limitations of their novel study including the lack of validation and small number of participants. In addition, it was focused on the use of Google Glass in only one surgical procedure. The authors report concerns of possibility of surgeons over reliance on the device, creating an intraoperative conflict where the surgeons’ decisions may disagree with the Google's software. They too, also reported limitations of the device itself with regards to battery life, overheating of battery and difficult software integration.

4. Discussion

The field of urological surgery has seen some of the greatest technical advances compared to other specialties with endoscopic and robotic procedures overtaking most traditionally open approaches. To perform these procedures competently surgeons spend years of education and human surgical training to allow them to safely navigate through anatomical planes of the human body. The concept of AR, where a live view of the surgical operation supplemented by OHMD is hoped to enhance the trainee surgeon's perception of reality during a procedure, and to enrich the learning process. A multitude of scientific papers have shown that OHMD is safe and beneficial to the surgeon.^[10–12] What is now important is to explore the potential uses of OHMD in urological education. The need to match the advances in surgical technology with advances in surgical training and education is of utmost necessity—especially in the modern era with increasing time and pressure demands on both trainer and trainee.

OHMD has had multiple educational uses in other surgical fields. Datta et al.^[11] showed how distant mentor surgeons can monitor a trainee surgeon thousands of miles away by teleproctoring. The mentor surgeons are able to observe and comment in real time, leading to the local surgeons to report this as a valuable educative tool. This allows trainee urologists to be able to view surgical education videos in first person, but also to have their performance monitored via mentor surgeons. This is of even greater value in low income countries, where they can access medical education and mentorship of surgical and ward-based skills over long distances.

Nakhla et al.^[12] showed OHMD technology can be used to demonstrate anatomy to surgical trainees, then to monitor and assist the trainee whilst they operate. Evans et al.^[13] showed that the use of OHMD to record first-person video (rather than traditional third-person) helped trainees to retain more knowledge when performing the demonstrated procedure themselves.

Borgmann et al.^[6] assessed the feasibility and safety of AR using smart glass amongst 7 urological surgeons (3 board urologists and 4 urology residents) for 10 different types of operations in a total of 31 urological procedures. They used Glass for taking photographs/recording video for teaching and documentation, hands-free teleconsultation, reviewing patients’ medical records and images and searching the internet for health information. The majority of participants rated overall usefulness as high. They showed that implementing Glass during urological surgery was feasible and safe with no high grade complications in the 31 procedures done.

Within the field of urology education however, there is to date limited research into the use of OHMD technology to educate urological trainees.

The aim of this systematic review was to identify research done in this field to explore whether OHMD has a place. It has identified only a limited number of studies exploring modern AR technology in the field of urological surgical training. In addition, this review also highlights the small numbers and types of operations used in the 3 studies examined. They focused on nephrectomy, prostatectomy and IPP placement. Ideally, for this technology to become in common use in the training environment it would need to be assessed across a range of operations to help aid the urological trainee at different levels of training.

Nonetheless all 3 studies in the systematic review show significant and serious potential for such as device. The most obvious advantage would be interaction with a supervisor in real-time allowing for a shared view of the operative field. It could also be used to watch a recorded video footage of the procedure to allow its replication during the operation. Furthermore, it can be used as a documentation and self-learning device for the trainee who can reflect on their skills. Its implementation so far has shown no hazards to the patient or detriment to the outcome of the procedure. The benefit of voice control is a priority and key advantage in the operating theatre allowing its use without hindering surgeon sterility.

However, despite all the potential advantages these OHMDs provide, there are possible negative impacts that this technology could have on a trainee. Dickey et al.^[9] reports trainees could become over reliant on the device, which may hinder their abilities and learning should the device not always be available, or indeed when the decisions of the trainee conflict those of the technology being used to aid their learning and experience.

All 3 studies also identify the need for the technology to improve usability. The need to optimise battery life and be comfortable and easy to use was reported by most participants in the studies. Numerous published studies report issues with discomfort, overheating, software issues or poor signal strength which impacts on image quality with the currently available technology.^{[3,7,9,12,13]}

In the era of data protection and security, the device needs to compliant with all relevant data security laws. Data streaming would be required to be via secure or encrypted servers, impacting cost. This would create barriers to teleproctoring, especially over long distances or where trainee and mentor are based in different regions with different data laws.

5. Conclusion

Although all studies describe a big potential for OHMD such as Google Glass there are very limited studies which concentrate solely on their use in urology and their training opportunities. All studies use only a small number of participants so drawing significant conclusions from them should be done with caution, however, the feedback in all studies by participants has been positive in regards to educational usefulness as well as their ability to engage users in more training and learning opportunities and their willingness to use the technology again.

Acknowledgments

All of the authors were acknowledged in this article.

Statement of ethics

None.

Conflict of interest statement

The authors declare that they have no financial conflict of interest with regard to the content of this report.

Funding source

None.

Author contributions

Ibraheem Alrishan Alzouebi: Primary and key author;

Sanad Saad, Tom Farmer and Sophie Green: Contributors to the method and results.