Digital Video Capture and Synchronous Consultation in Open Surgery

Abstract

Objective:

To achieve real-time or simultaneous surgical consultation and education to students in distant locations, we report the successful integration of robotics, video-teleconferencing, and intranet transmission using currently available hardware and Internet capabilities.

Summary Background Data:

Accurate visualization of the surgical field with high-resolution video imaging cameras such as the closed-coupled device (CCD) of the laparoscope can serve to insure clear visual observation of surgery and share the surgical procedure with trainees and, or consultants in a distant location. Prior work has successfully applied optics and technical advances to achieve precise visualization in laparoscopy.

Methods:

Twenty-five thyroidectomy explorations in 15 patients were monitored and transmitted bidirectionally with audio and video data in real-time. Remotely located surgical trainees (n = 4) and medical students (n = 3) confirmed 7 different anatomic landmarks during each surgical procedure. The study used the Socrates System (Computer Motion, Inc. [CMI], Goleta, CA), an interactive telementoring system inclusive of a telestration whiteboard, in conjunction with the AESOP robotic arm and Hermes voice command system (CMI). A 10-mm flat laparoscopic telescope was used to capture the optical surgical field. As voice, telestrator, or marker confirmed each anatomic landmark the image parameters of resolution, chroma (light position and intensity), and luminance were assessed with survey responses.

Results:

Confirmation of greater than 90% was achieved for the majority of relevant anatomic landmarks, which were viewed by the remote audience.

Conclusion:

The data presented in this study support the feasibility for mentoring and consultation to a remote audience with visual transmission of the surgical field, which is otherwise very difficult to share. Additionally, validation of technical protocols as teaching tools for robotic instrumentation and computer imaging of surgical fields was documented.

With current technology advancements, surgery is now embracing computer technology, robotics, fiber optics, and telecommunications to provide telesurgery for mentoring, proctoring, and consultations. In this study, we report the successful integration of robotics, video teleconferencing, and intranet transmission for real-time distance education and consultation, while validating key elements in telesurgery technology.

High-quality digital video imaging systems, which provide accurate visualization of the surgical field on a standard monitor, have made possible broad clinical acceptance of laparoscopic surgery. Accurate digital visualization of the surgical field has given surgeons confidence in the accuracy and reliability of the video monitor as a true representation of the surgical environment with clarity, true color, and accurate magnification. All members of the surgical team can also view the projected images without limitations to physical constraints. These surgical images can be recorded for asynchronous review and transmitted over a secure Internet link for synchronous discussion with students or consultants.¹ Recent work in our center has validated and adapted high quality optics for application in open surgeries.^1–3 A laparoscopic camera can be brought into close proximity of the open surgical field and can be stabilized to permit data capture, compression, and realistic transmission of the captured image signal at low bandwidth.^4,5 To position a laparoscopic camera using voice commands and not interrupt the surgical procedure when adjusting the video image, the camera was managed robotically and open dialogue was established with an audience at a distant site with supplemented telestrator drawings on the transmitted video image. This permitted feedback to the operating room (OR) from the distant audience for confirmation. Validation of the image sharing was secured by identification of specific anatomic structures and rating the quality of transmitted images by the audience in the remote location.

The patients included in this study underwent thyroidectomy or parathyroidectomy surgical procedures. In these procedures, a small incision and very narrow surgical field severely limits sharing the visual field with others in the OR. Integrated usage of high quality digital video systems, robotics, computing capabilities, and telecommunications were applied to this challenging surgical context. During live video streaming from the OR to a distant location, the audience receiving the transmission evaluated the image quality relative to the lighting, focus (clarity), visibility of the surgical field, and contrast of the images.

MATERIALS AND METHODS

This study was conducted with 2 multimedia stations communicating with each other over the university's local area network (LAN) to validate the feasibility of remote surgical consultation and telementoring of real-time surgical procedures (Fig. 1). The experimental evaluations in this study involved application of currently available technology and the assessment of the technical parameters as well as the final outcomes in providing effective remote surgical collaboration. The study used evaluations of real-time video image streaming during 25 standard thyroidectomies.

FIGURE 1. Schematic illustration of the hardware components comprising the 2 interactive multimedia stations between the OR and the geographically distant site designated as the “remote location.”

Surgical Procedure

One surgeon performed all the operations included in this study. The details regarding the surgical procedure inclusive to this study are as follows. Initially, the neck was opened through a collar incision. Dissection proceeded with elevation of superior and inferior platysma flaps. The sternohyoid muscles were separated in the midline and the sternohyoid was bluntly separated from the sternothyroid. The sternothyroid was reflected from the thyroid surface and the middle thyroid vein was ligated and divided over the carotid sheath. The patient underwent either a thyroidectomy or parathyroidectomy by the surgeon. Both surgeon and distant panel using telemedicine technology identified 7 critical anatomic landmarks. With the use of voice recognition software and hardware (Hermes, Computer Motion, Inc.), the surgeon managed the laparoscopic camera, the robotic arm, and a screen pointer. The laparoscopic camera was secured to the AESOP robotic arm and stabilized by Alpha port (Computer Motion, Inc.; Fig. 2). Earlier studies have validated the use of the Alpha Port in conjunction with the AESOP robot to provide camera stability and flexibility in positioning the camera within the surgical field effortlessly using voice commands.³

FIGURE 2. Schematic representation of the Alpha port, the AESOP robotic arm relative to the surgical table, and multimedia stations used for image capture from the operative field (illustration by A. Rafiq).

Multimedia Computing Station Configuration

Two multimedia stations were setup similarly relative to video teleconferencing with capabilities for high-quality audio and video capture (Table 1). Both multimedia stations were specifically configured for this experimental study with capabilities of providing 2-way interactive digital exchange of audio and multiple video signals between the surgeon in the OR and the consultant as well as trainees in the remote location. Cameras located in the OR and remote site served as the source of image input into the media station for transmission to the reciprocal location over the intranet infrastructure. Connectivity between these 2 stations was established using H.323 over Internet protocols in a host-to-host topology protocol. The link up was established using Internet Protocol addresses and a transmission rate of 384 kilobits per second (Kbps). The 384 Kbps speed of transmission was selected to validate protocols that will allow us to replicate these efforts in transmitting similar video signals to our international collaborating universities. The diagnostic features of the Polycom (Andover, MA) software integrated into the Polycom hardware were used to confirm the transmission rates and the delay times listed in Table 1. The values of transmission speed and delay times were measured within the Polycom unit using Polycom's proprietary software to confirm that the signal transmission was in fact real-time and not experiencing any extensive delay in transmission. Figure 1 illustrates in a schematic representation the interconnected hardware components, which comprise the overall multimedia system used for this study. The technical schematic outlines the pathways through which the video signal traveled between the interconnected hardware components. Arrowheads in the illustration indicate the direction the video signal is traveling between hardware components. Discussion of the details regarding each hardware component within the multimedia stations relative to capacities of networking settings, video image capture, and media presentation hardware is discussed below.

TABLE 1. Technical Details of the 2 Interactive Multimedia Stations

Networks Settings

The multimedia stations were setup with Polycom units (Model VS4000) equipped with 10/100 megabits per second (Mbps) Ethernet connectivity capabilities. Connectivity between local and remote units was achieved by transmitting over the university's LAN system with a preset transmission rate of 384 Kbps.

Video Image Captures

Digital capture of video images from the open surgical field was achieved with a 10-mm fixed laparoscopic telescope, in accordance with an earlier study.⁶ The laparoscopic telescope was stabilized with an Alpha Port at its distal end, and attached proximally to the AESOP robotic arm, which was mounted to the surgical table. The robotic arm provided a stable platform for capturing the laparoscopic optics in close proximity to the open surgical filed and was driven by the surgeon's voice commands spoken into a microphone, which were converted by the Hermes unit into digital signals for AESOP to recognize. With the use of the AESOP robotic arm and a stabilizing Alpha port, camera vibrations were eliminated. The practice of transferring endoscopic image from a CCD camera to a video monitor has the advantage of allowing the surgeon to operate in a less tiring position and improvement in task performance with the adoption of the image display system.⁶ The camera setup using Alpha port has been validated in previous studies to be unobtrusive to the surgeon and required no adjustments in surgical technique.⁷ The CCD video camera was attached to standard laparoscopic video digitizing equipment (Stryker Corporation, Santa Clara, CA), and the video images were displayed locally in the OR on a 2-dimensional imaging system consisted of a Sony 20-inch color video monitor (Monitor A in Fig. 1, and 2); model EVI-D30, (Sony Corporation, Santa Barbara, CA) for immediate verification by the surgeon of the image being transmitted to the remotely located audience. The CCD camera has 900 lines of horizontal resolution, and a light sensitivity of 0.8 lux while the Sony monitor displayed effective pixels of 768 (H) × 492 (V).

Media Presentation Local to OR

The video monitoring hardware of the multimedia station in the OR included 2 20-inch monitors, A and B, as illustrated in Figure 1 and Figure 2. The video signal captured by the laparoscopic camera was displayed as a video stream in Monitor A (Figs. 1 and 2) and was designated as the local signal. The video stream was also transmitted with the use of the university intranet to the remote location and received on a secondary monitor, Monitor B (Fig. 1, and 2) as a feedback signal. The feedback video stream received in the OR from the remote location was displayed in real-time on monitor B. Monitor B also maintained the capacity to display an overlay of telestrator illustrations or highlight markings from the remote site audience.

Media Presentation to Remote Location

At this location 2 monitors were also integrated for viewing the signal transmitted from the OR (Fig. 1). The 2-dimensional imaging systems consisted of a Sony monitor (Monitor C in Fig. 1) and the Socrates monitor (Wacom Co, Japan) (Fig. 1). The Sony monitor, model EVI-D30 has effective pixels of 768 (H) × 492 (V) while the Socrates monitor is 15.1 inch flat panel screen with effective pixels of 1024 × 768. The video images captured in the OR were received in real-time at remote location for synchronized consultation and mentoring on monitor C. The trainees were also capable of viewing the video stream from the OR (OR signal) on an independent monitor (Socrates monitor in Fig. 1) with the additional capacity to highlight the surgical images using overlaid markings with the tools available in the Telestrator (Pointmaker, Boeckler, Japan).

Image Transmission

Once the video image was captured from the operative filed by the laparoscopic camera it was channeled into a standard endoscopic image digitizer and then forwarded into the Polycom station for intranet transmission to the remote location. The remote location was able to switch between 2 video camera inputs for transmission feedback to the OR, a room camera and the telestrator. The 2 video input options provided flexibility for the audience in the remote location to transmit either a video of their location or return the video transmission from the OR with an overlay of the telestrator input.

Image Evaluation

Remotely located surgery trainees identified 7 anatomic landmarks during the course of each surgical procedure while maintaining consultation in real-time with the surgeon in the OR. A total of 6 trainees were present in the remote location for this study, and were classified as being equivalent in expertise since they were all graduates of medical school and were undertaking surgical residency training. The 7 anatomic landmarks, listed in Table 2, encountered during standard operating procedures for both parathyroid and thyroidectomies were discussed between the surgeon in the OR and trainees in remote location. These procedures were applied to 25 unilateral thyroid dissections conducted on 15 separate patients. The trainees, serving as image reviewers, used audio capabilities in the setup to confirm identification of surgical landmarks in the order they routinely appear during a standard thyroidectomy or parathyroidectomy and confirmed the identification of the anatomic landmark in view. Initial evaluation of each anatomic landmark's image quality required all reviewers to complete a Likert scale survey to measure a set of attitudes, from the choices provided, towards each anatomic landmark. Reviewers were requested to select from the following choices on the questionnaire: concur, not concur, or unsure. These parameters were selected to confirm if any problems were encountered during the transmission of the video image from the OR. Additional evaluations of images transmitted in real-time were achieved for image quality and clarity. The image quality evaluations were done relative to light, focus, visibility of the surgical field, and contrast of the images. The above mentioned image quality parameters were selected to reflect the characteristics most significant from a surgeon's point of view when viewing the surgical field in a 2-dimensional image display. The operative field is rich in cues that provide the surgeon with sufficient information to evaluate the field of operation and it is essential to retain a high level of image clarity when transmitting to a distant site. Because layering of fluid on the structures in the surgical field can serve as a reflective surface it is essential to maintain good level of lighting to illuminate the detailed anatomic structures and differentiate the pathology of concern from adjoining anatomic structures. Inadequate lighting can also cause a great deal of shadowing in a 2-dimensional video display since there are over layering anatomic structures that have to be maneuvered around during the course of a surgical procedure. In order for the trainees in a distant location to be certain that finite anatomic structures such as arteries and nerves are clearly visible it is essential that the image transmitted has a great deal of focus. The parameter labeled as “visibility of the surgical field” was selected as a relevant parameter to verify that the CCD camera was in close proximity of the surgical field to display the entire surgical field of significance with enough degree of interrelationships between the adjacent anatomic structures to maintain a true anatomic perspective for the trainees. In a surgical field one can easily gain a sense of contrast between tubular structures to differentiate between arteries, veins and nerves. When transmitting the digital image of the surgical field to a remote audience it is essential to retain the same degree of contrast to more clearly distinguish structures that may appear to have similar color hues. The quality parameters selected for this study were ranked in accordance with a Stapel Scale ranging from 1 to 5 such that a value of 5 corresponds to a degree of excellence while a poor rating was assigned a value of 1.

TABLE 2. Responses From 10 Reviewers of Anatomical Landmarks in 13 Patients

RESULTS

Under open surgical techniques, 25 unilateral thyroid dissections in 15 patients were captured using a laparoscopic flat camera (10 mm), which was stabilized using an Alpha port distally and an AESOP robotic arm proximally as reported in an earlier study.⁸ Using voice command recognition software, HERMES, the surgeon was able to position the camera precisely within the operative field. Interactive teleconsultation and mentoring was emphasized with the superior flap, middle thyroid vein, carotid sheath, 2 parathyroids, inferior thyroidal artery, recurrent laryngeal nerve and superior thyroid pole.

Rating of Anatomic Landmark Confirmation

Surgical trainees located in remote location were able to discuss and identify the anatomic structures in sequence as noted in Table 2. The concur rate for clear visual confirmation of anatomic landmarks transmitted from the surgeries was over 90% for each anatomic landmark with the exception of the middle thyroid vein. The superior flap being the outer most structure was identified in 100% of the transmissions. The greater than 90% concur rate is a reflection of the successful interaction between the observers in the remote location and the surgeon in the OR. Seven trainees synchronously validated visual identification of the 7 anatomic structures listed above in Table 2.

Any discrepancies were clarified with a 2-way verbal consultation between the 2 distant locations. The concurrence rate for the carotid sheath was 98.8%, indicating the ability to identify a clearly visible structure. In 94.2% of the overall transmissions the superior pole was identifiable. Observation for the middle thyroid vein was the weakest concurrence at 89.8% from the overall video-streamed images. The inferior thyroid artery was visually confirmed at 97.4% of the total viewings. Deeper structures, parathyroid gland and the recurrent laryngeal nerve were confirmed at 91.3% and 98.7%, respectively, of the total video transmissions.

Subjective Rating of Image Quality

Image quality was not compromised during the transmission with the methodologies applied in this study. A scale of 1–5 was used to rate the images by the observers in the remote location with poor characteristics rated with a value of 1 and 5 corresponding to excellent qualities. The data represented in Table 3 are mean values from a cumulative ranking of image quality related to video streaming of all 7 anatomic landmarks. Each viewer evaluated the landmarks subjectively during the transmission from the OR.

TABLE 3. Subjective Image Quality Rating of Anatomical Landmarks as Viewed in Real-Time Transmission From OR in Surgical Procedures Involving 13 Patients

The data in Table 3 indicates that the quality of light illuminating the surgical field was scored as very good quality in 84.9% of the interactive viewings. The contrast of the video was also of excellent quality in 87.0% of the viewings. Image focus was ranked as being excellent in 93.6% of the observations. The parameter, field, is indicative of the limitations in the viewing field hampering proper perspective and orientation within the operative field as dictated by the camera. In majority of transmissions the field was of above average perspective (96.8%) and provided the remote viewers clear orientation for the identification of the anatomic landmarks being highlighted.

DISCUSSION

The historical standard for teaching surgical skills is apprenticeship with direct participation in the OR with a mentoring surgeon. Consultations have also only been achieved by physical presence of all concerned individuals in the OR. Physical constraints of equipment and practical visual field limits and dissection make visualizing the operative field difficult for the surgeon, as well as for the surgical assistant. Any additional observers such as medical students and OR personnel typically have minimal or no access to the very confined field of views. This is especially true in procedures such as a thyroidectomy. Applications of multimedia computer aided educational mechanisms have transformed the methods of surgical education into a more versatile and beneficial discipline.⁹ Effective telementoring is now possible with advances in technology allowing interactive mentoring between physically distant locations, and the surgeon in the OR.⁹ The virtual linkage between mentor and trainees increases the versatility of the educational environment for surgical candidates and creates a unique collaborative environment for consultation with other physicians.

This study validates the use of an integrated suite of multimedia tools to capture the video image from close proximity to the open operative field and to transmit to observers in a remote location. From the perspective of the trainees in the distant location no compromise in image quality relative to resolution, light position, or luminance was noted. With a stable camera stand established using both the Alpha port and the AESOP robotic arm the video image captured had neither distortions nor vibrations. Earlier studies confirmed that the AESOP robotic arm and Alpha port provides a stable platform for image capture.^10,11 Additional studies have confirmed that camera stability for video transmission can be achieved effectively at low bandwidths.⁵ In the current study the video signal transmission was evaluated to be smooth with no compromise in image quality when viewed by the trainees in remote location. In contrast, video conferencing software such as Microsoft NetMeeting uses a video transfer method such that complete video transmission is achieved every 15 seconds.¹²

The shared image provided to the trainees in the remote location was displayed in 2 dimensions as is the case in customary laparoscopy. However, these data identify no particular compromise in displaying the video image of anatomic structures in a 2-dimensional monitor screen since anatomic structures were correctly identified in greater than 90% of the instances. Hanna et al have suggested that the coaxial alignment of the light fibers and the optics of the endoscopic video system can be detrimental since the image has a shadowless illumination with consequential distorted depth cues in the video images displayed on a monitor.¹³ In this study, parameters such as uneven shadowing or insufficient field luminance were immediately corrected by the availability of bi-directional audio communication in conjunction with visual video between the surgeon and the remote trainees. The interaction was enhanced beyond voice and video image by inclusion of a screen pointer positioned by the surgeon and telestrator drawings from the audience. There was neither compromise in image quality nor significant delay in time for the feedback operative video image. Earlier studies evaluating psychomotor skills in surgeons have noted that perceptions of 3-dimensional view are not compromised by the display of the surgical image on standard viewing monitors.¹⁴

The digital video imaging system described in this study displayed a very confined operative environment to an audience without interference with the surgical procedure. The camera was not obstructive and did not interfere with surgeon's hands. Standard standing position presented awkwardness since the camera created some distance between surgeon and scrub nurse, which was easily accommodated. The integrated technology described in this study provided mechanisms for synchronous consultation and education of open surgery to a potentially unlimited audience. A recent study by Byrne et al proposed that competency in surgical skills is the culmination of knowledge, manual skills, and judgment.¹⁵ Meanwhile, the Accreditation Council for Graduate Medical Education, through its Residency Review and Institutional Review Committees, has identified 6 general competencies for all physicians in its Program and Institutional Requirements. These competencies are: patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice.¹⁶ Synchronous collaboration allows trainees to gain knowledge, experience, and develop surgical judgment with efficient use of faculty time and broad access to specific surgical procedures. The interaction between the mentor and the observer is enriched by clear visibility of the operative field, which is common to all participants. Making the open surgical field accessible for training and consultation does not compromise patient care since the robotic arm is managed with specific vocal commands allowing the surgeon to operate without impediment. Patient care may actually benefit by reducing the number of observers actually present in the OR attempting to gain access to the very limited surgical field.

ACKNOWLEDGMENTS

The authors would like to acknowledge the efforts expended by the MITAC postdoctoral research fellows, and interning medical students in reviewing the data during all the surgical transmissions.