Demonstration of the Effectiveness of Augmented Reality Telesurgery in Complex Hand Reconstruction in Gaza

Summary:

Augmented reality (AR) is defined as “a technology that superimposes a computer-generated image on a user’s view of the real world, thus providing a composite view.”¹ This case report describes how emerging AR telesurgery technologies may be used to facilitate international surgeon–surgeon collaboration and training. Here, we illustrate how a remote surgeon in Beirut, Lebanon, was able to offer assistance to a surgeon in Gaza, Palestine, during a complex hand reconstruction case following a bomb-blast injury in an 18-year-old male. We discuss the implications of AR technology on the future of global surgery and how it may be used to reduce structural inequities in access to safe surgical care.

WHAT IS AUGMENTED REALITY?

Augmented reality (AR) provides a live representation of a real-world environment in which computer-generated images are superimposed onto a continuous visual feed to create an enhanced, composite image. Proximie is a secure, cloud-based AR platform that allows real-time collaboration between a local (operating) surgeon and a remote (assisting) surgeon using a birds-eye view of the operative field. Both surgeons communicate by means of a 2-way audio stream, using a number of integrated AR features to further clarify the advice provided. For example, the remote surgeon can proctor their own hands into the virtual surgical field using a webcam or highlight important structures using a range of annotation and drawing tools.

CASE REPORT

Background

In 2016, an 18-year-old male from Gaza severely injured his left hand while attempting to dismantle unexploded ordnance on a building site. Despite having 6 previous operations to save his hand, extensive palmar scar contracture left him unable to work or perform a number of activities of daily living, including showering and dressing. There was a significant loss of range of movement in the hand, particularly in the fingers, and a fixed flexion deformity of the thumb, preventing both pincer and power grip formation.

As an area of conflict with limited resources and expertise, specialist input from a reconstructive hand surgeon was not readily available. Restrictions on free movement and the high costs of transferring personnel in and out of Gaza precluded delivering a team to the patient or delivering the patient to a tertiary center.

These issues were resolved by the use of a novel AR-assisted telesurgery solution in which a remote surgeon from the Conflict Medicine Program at the American University of Beirut, which specializes in providing assistance and training to health professionals in areas of conflict, was able to virtually “scrub in,” guide, mentor and support the local surgeon using a portable tablet device. Although video teleconferencing technology has been successfully used for telementoring in alternative settings previously,² it is currently limited in its application to live surgery. In contrast, AR allows the remote surgeon to outline, in a stepwise fashion, the necessary procedural steps using a combination of hand gestures, annotations, diagrams, and clinical imaging. This is superior to simple unidirectional video or telephone communication: the AR elements allow the remote surgeon to “show” them how to carry out the procedure as opposed to “tell” them (Figs. 1, 2).

Fig. 1.

Remote surgeon in Beirut using a webcam to proctor their hand into the virtual surgical field. Here, they are using a pen to highlight important anatomical structures for the local surgeon in Gaza.

Fig. 2.

Augmented reality-enhanced image visible to the operating surgeon in Gaza. Note how the remote surgeon’s pen is an overlaid onto the birds-eye view of the surgical field.

Setup

A simple camera rig was positioned above the surgical field in Gaza to give the remote surgeon in Beirut a birds-eye view of the operation. All camera types (including smartphones) are compatible with the Proximie software—in this case an action-style camera connected to a computer via USB was used. In previous test cases, fixed cameras were shown to be superior to head mounted devices, as the image remains stable and does not vary with body position or movement. This image, together with AR overlay, was streamed on a tablet visible to the operating surgeon in Gaza. Concurrent audio streaming added a further dimension to the communication between the surgeons, promoting 2-way discussion rather than 1-way instruction.

The platform is designed to operate using Internet and satellite connectivity, although image quality is dependent on adequate connection strength and bandwidth. Proximie is compliant with all Health Insurance Portability and Accountability Act regulations³ (http://www.proximie.com/. Accessed December 29, 2017) with secure end-to-end encryption; however, prudent information governance remains important, especially when using these AR technologies in developing countries (Fig. 3).

Fig. 3.

Local surgeons in Gaza being guided remotely by assisting surgeon in Beirut.

Fig. 4.

Intraoperative image of the patient’s hand following release of palmar contracture and forearm flap insetting.

Procedure

The first stage of the operation involved planning the procedure and marking the hand. After examining the hand and its range of movements over a video link, the contracture was released using a Z-plasty on the median aspect of the palm with a 5-flap Y-V advancement and Z-plasty (“jumping man flap”). Meanwhile, a local flap from the left forearm was raised to cover the palmar defect and exposed median nerve. By the end of the operation, passive range of movement had significantly increased with marked improvements in finger extension and abduction.

DISCUSSION

This case raises issues central to global health and safe access to surgery. These are complex and multifactorial; however, we demonstrate how a simple, cheap, and reproducible AR platform may help to reduce structural inequities in global surgical health.

Centralization of expertise has led to areas with a high density of specialists.^4,5 Although this may benefit certain patient groups, it can leave others with limited access to advanced surgical care,⁶ especially those in areas of conflict or surgically austere environments.⁷ Although it is sometimes possible to transfer either patients or surgeons, this is not always feasible owing to border restrictions, time pressures, or cost.

This proof-of-concept case report demonstrates how innovative AR technologies, such as Proximie, may be utilized to address this vacuum of care by providing a cost-effective and reproducible remote telesurgery solution. Moreover, in allowing local surgeons to perform advanced procedures under expert instruction, AR platforms function as an educational tool, promoting sustainable skill acquisition and a reduction in future dependence on specialist centers. This has implications for the developed world and advanced health care systems: for example, Proximie is already being used in the United Kingdom to deliver specialist, tertiary-level plastics care to regional referring units⁸ (https://www.royalfree.nhs.uk/news-media/news/new-technology-allows-surgeons-to-virtually-scrub-in/. Accessed December 29, 2017). Both the international collaborative model detailed in this case report—and locoregional applications—can be easily reproduced elsewhere.