Abstract
In laparoscopic surgery, senior surgeons spend great efforts to convey knowledge to their trainees. Any misinterpretation of the instructions may lead to inevitable errors that have a great impact on patient safety. To support efficient knowledge sharing, we design a communication support system, Virtual Pointer, to convey trainers’ gestures directly onto the laparoscopic video for trainees to see. We implement the design in an international surgical conference and evaluate the perceived usefulness and acceptance of the system among senior surgeons and surgical trainees. The results showed that the system facilitated the trainees to complete the task, increased their knowledge, and reduced the trainers’ guiding efforts. The study indicates that conveying the gestures directly onto video has the potential to support the intraoperative communication. Barriers in the acceptance of the system highlight the design of novel interaction mechanisms to unobtrusively integrate this technique into the surgical workflow.
INTRODUCTION
The operating room (OR) is the key place for residents to acquire surgical expertise. However, over half of surgical adverse events are attributable to the OR-based surgical training1,2. The main cause for these errors is the communication failure in the high-stakes environment with multiple demands between residents and senior surgeons2. Residents, on one hand, need to efficiently follow the instructions given by their trainers. On the other hand, they master surgical skills through in-situ learning, which requires them to make their own decisions and get corrected and confirmed from the trainers3. Fulfilling these demands rely on the ability of expert surgeons in conveying the knowledge, that is in response to their trainees’ knowledge background, experience levels, and comprehensive abilities.
Previous research has shown that actions, such as pointing or instrument movements, have essential communicative functions that facilitate the conveying and understanding of knowledge4,5. Expert surgeons often use actions to clarify the referred anatomical structure6, manage the trajectory of the operation7,8, and demonstrate the technical skills9. These actions are sometimes transformed into explicit instructions when necessary in open surgery. For instance, an expert surgeon may draw on the skin to indicate the path of the incision and to mark the major landmarks of the anatomy7. The drawing specifies where the residents should attend and ensures that they are on the same page throughout the operation.
However, in laparoscopic surgery, where the indirect view of the operative field limits the ability to directly gesture on the anatomical structures, expert surgeons take great effort in appropriating the view to guide their trainees to see the target structure, make meaning of it, and using it for their actions 3, 9,10. Expert surgeons often guide trainees’ hands to maneuver the laparoscopic camera, accompanied by a series of verbal explanations and gestures over the monitor to reveal the subtle changes in the structure to translate what they are seeing into the abstract anatomy in the textbook3,9. The surgical instruments are used to point at the target or draw imaginary lines to elucidate the structures embedded in the tissues. Trainees, on the other hand, need to parse, envision and make sense of their trainers’ actions to perceive the locations and directions. This often engenders a series of checking, clarifying, aligning of the information that is already presented10. In this process, any misinterpretations of these actions may lead to inevitable technical and judgment errors that have a great impact on patient safety11.
To overcome these problems, we hypothesized that conveying the gestures directly onto the video is beneficial. Previous research has shown that communication becomes more effective when annotation and gesturing tools are provided for the shared view of the workspace12,13. For instance, Fussell et al. found that the pointing and representational gestures added on a shared view of a robot assembly task effectively facilitated task communication and reduced performance time12. Tang et al. projected the hands over the shared view and found that the gestures allowed the collaborators to be more engaged in communication and the tasks13. These studies focused on the communication in completing a task. The communication in OR-based surgical training, yet, entails not only instructions necessary in operating, but also the in-situ knowledge for the trainees to become competent in surgery3,9. It is unknown whether the gesturing tool supports conveying expertise in surgical training.
We designed a Virtual Pointer system, which enables a trainer to point or draw a freehand sketch, i.e., telestrating, over a laparoscopic video for a trainee to see during the operation14. We implement the design in an international surgical conference and evaluate the perceived usefulness and acceptance of the system among expert surgeons and residents. The perceived usefulness is defined as “the degree to which a person believes that using a particular system would enhance his or her job performance.”15 In our evaluation, we mainly focus on knowledge sharing in surgical training. The job performance here refers to trainees’ learning performance, i.e. the knowledge they gained, the skills and abilities they developed, and the effort they expended in training, as well as the trainers’ ability in conveying the knowledge. User acceptance is the degree to which a person is willing to use the system in their work practice15. In our study, the user acceptance refers to the trainers’ willingness to use the system in conveying knowledge, as well as the trainees’ willingness to gain knowledge through the annotation or reference on the video provided by the trainers using the Virtual Pointer. The evaluation presents the surgeons and surgical trainees’ perspectives on how they are going to use a Virtual Pointer-like communication support system, what benefits they will gain from using the system, as well as the concerns and barriers in implementing the system in the surgical environment. Based on the findings, we discuss the implications for designing an intraoperative communication support system.
METHODS
Overview of the Virtual Pointer system
The Virtual Pointer system was designed aiming to facilitate the conveyance of expert knowledge by enabling trainers to point or draw on the laparoscopic video for the trainees to see during an operation. The program works by using a combination of audio key words and hand movements to trigger different functionality. The Microsoft Kinect is used as the gesture and voice control sensor.
Figure 1 shows the interface of the system. The collection of the verbal commands is shown in the upper left corner. The current mode is presented in the center above the laparoscopic view. The lower left corner shows the user’s skeleton to provide timely feedback of the user’s movement. To awaken the Kinect, the first command is verbally saying “Kinect ready”. When this is said the Kinect starts detecting other verbal cues and gestures. There are two verbal cues the Kinect is looking for, either “Kinect draw” or “Kinect point”, to switch between the drawing mode and the pointing mode. In the pointing mode, the user moves the hand to control a small green circle, which acts as a pointer. In the drawing mode, the user closes their hand to draw over the video. The position of the pointer and the drawing responds to the position of the user’s hand. To clear the screen of all annotations, there is the verbal command “Kinect clear”. When the program is finished being used, the voice command “Kinect close” can be used at any time to set the program to sleep and stop the Kinect from detecting.
Figure 1.
The interface of the Virtual Pointer program14. (1: the selected mode; 2: the audio commands; 3: the user’s movement feedback; 4: the laparoscopic view.)
Study Procedure
Virtual Pointer is a novel instructional system for surgery training. To minimize the learning curve, we first demonstrated how the system works and gave a brief introduction of the study. The surgeons or surgical trainees who consented to participate in the study completed a preliminary questionnaire on demographics and surgical experience.
Based on their experience level (attending surgeon, surgical fellows, and residents) the participants were assigned into two groups – the trainer group (attending surgeons) and the trainee group (surgical fellows and residents).
The trainers were asked to use the Virtual Pointer to guide the researcher to perform a simulated laparoscopic task. The researcher gained the fundamental knowledge of the task from the general surgery textbook and case videos. The trainees were asked to perform the task with the guidance from the researcher using the Virtual Pointer. A general guidance transcript was developed to include the features of pointing and drawing – the pointing was for the anatomical identification, and the drawing was for the indication of where to cut. The in-situ guidance was based on the general guidance transcript and adapted based on the needs of the trainees.
Afterward, the participants completed an evaluation questionnaire on their experience, outcomes expectations, attitudes and intentions of using the system. Due to the different demands in using the technology between the trainers and trainees, we administered two sets of questionnaires – one for the trainers on conveying the knowledge, and one for the trainees on acquiring the knowledge. The questionnaire was adapted from previous research on perceived usefulness and acceptance of educational technologies16,17,18, including 21-item 7-point Likert scale and open-ended questions. The study was approved by the University of Maryland, Baltimore County institutional review board. Verbal consent was obtained from all participants before their participation.
The Simulated Task
The simulated task in the experiment was to separate the cystic duct and cystic artery on a paper model. It is an important step in achieving the “critical view of safety” in the laparoscopic cholecystectomy procedure. This task requires the trainees not only to correctly identify the cystic duct and cystic artery that are embedded in the peritoneum, but also envision the imaginary line of safety over the two structures to avoid dissecting into the common bile duct. Thus, this task involves the effort of both referring to task objects or locations and managing the process of dissection.
Study Settings and Participants
The Virtual Pointer system was connected to a Fundamentals of Laparoscopic Surgery (FLS) Box Trainer System (Limbs & Things, Bristol, UK). The Microsoft Kinect sensor (Microsoft, Washington, USA) was oriented towards the trainer. The trainee stood in front of the FLS Box with the dominant hand controlling the scissors and the other hand manipulating the Maryland dissector. The Virtual Pointer provided an overlay of the reference and annotation from the trainer to the laparoscopic video for the trainee to see. As the transfer of the video created a 2-second lag, we presented the telestration on a secondary display (Figure 2).
Figure 2.
Study settings in 2017 Society of American Gastrointestinal Endoscopic Surgeons (SAGES) Annual Meeting. (1: FLS Box Trainer System; 2: Laptop running the Virtual Pointer; 3: Microsoft Kinect sensor; 4: Primary display for the laparoscopic view; 5: Secondary display for the telestration; 6: Virtual Pointer Introduction.)
Recruitment
Attendees of the Telestration-Guided Laparoscopic Visual Skills Practice Station at the Learning Center during the 2017 Society of American Gastrointestinal Endoscopic Surgeons (SAGES) Annual Meeting in Houston, Texas, were invited for participation. A total of 18 trainers (10) and trainees (8) participated in the study, including four female surgeons and 14 males. Experience levels ranged from novice medical students to senior surgeons with more than 20 years in practice. Among the 18 post-task questionnaires collected, 3 (1 trainer and 2 trainees) were identified to contain inconsistent answers for the same assertation asking in different directions and were removed from the analysis.
Data Analysis
We conducted non-parametric tests for the analysis of the questionnaire responses19. For each question, we conducted the one-sample Wilcoxon signed-rank test comparing the sample median with the neutral rate. We used the Mann Whitney U test to compare the responses of the shared questions between the trainers and trainees. Spearmen’s correlations were calculated between questions. The median response of each question was reported, along with the p-values. The analysis was conducted in R version 3.2.2 (R Foundation for Statistical Computing, Vienna, Austria).
RESULTS
Perceived Support in Conveying Knowledge
Overall, 75% of the trainers were satisfied with the support of Virtual Pointer in conveying the knowledge during the task, but their ratings were insignificant (M = 5.5, p = 0.137) (Figure 3). 87.5% trainers agreed that the Virtual Pointer saved their efforts spent on teaching (M = 5.0, p = 0.037). However, only around a half of the trainers were satisfied with the amount of time in conveying knowledge via Virtual Pointer (M = 6.0, p = 0.101). And, 55.6% trainers thought the technology may increase their desire to teach (M = 5.0, p = 0.354). In the open-ended questions, the trainers questioned about the efficiency and reliability of the system. Trainer #5, who was not satisfied with the support of the Virtual Pointer, commented, “it requires I do not use the instrument while telestrating”.
Figure 3.
Virtual Pointer in supporting conveyance of knowledge. (CK: conveying knowledge; The numbers in the circles show the medians of the ratings.)
Perceived Support in Acquiring Knowledge
Overall, both the trainers and trainees perceived the Virtual Pointer’s benefit in increasing trainees’ knowledge (Trainer: M = 5.0, p = 0.058; Trainee: M = 5.5, p = 0.041) (Figure 4). The most commonly perceived aspect of learning that the Virtual Pointer could improve was the procedure knowledge, with the ratings of both the trainers (M = 6.0, p = 0.031) and the trainees (M =5.5, p = 0.038) significantly greater than neutral. Although the trainers significantly agreed that the system could increase trainees’ knowledge in anatomy (M = 5.0, p = 0.026), the trainees found the limited support in their anatomical knowledge building (M = 5.0, p = 0.066). The system’s effect on laparoscopic skills was the least perceived by only 33.3% trainers and 66.7% trainees, with all the ratings prone to neutral. In addition to accumulating knowledge, the Virtual Pointer was perceived to have a positive effect on enabling trainees to apply the possessed knowledge into practices, by 77.8% trainers (M = 5.0, p = 0.046) and 83.3% trainees (M = 6.0, p = 0.039).
Figure 4.
Virtual Pointer in supporting acquiring knowledge. (AK: acquiring knowledge; The numbers in the circles show the medians of the ratings.)
75.0% of the trainers found that the use of the Virtual Pointer may minimize the efforts that trainees spent on learning (M = 5.5, p = 0.036). 66.7% of the trainees agreed with the trainers that the technology reduced their learning efforts. Yet, their agreements were not significant (M = 5.0, p = 0.276). The reduced learning efforts may be due to the effect of the pointing or drawing on clarifying the target structure, as agreed by 83.3% trainees (M = 6.0, p = 0.041). Trainee #3 commented, “it was ease of use and clear”. Besides, 66.7% of the trainees indicated that the system encouraged them to learn more about the procedure (M = 6.0, p = 0.059).
Perceived Moderate Support in Task Performance
We asked the trainers and trainees to evaluate the effect of Virtual Pointer on their task performance, in terms of the easiness, time, effort and distraction (Figure 5). Overall, 66.7% of the trainers and trainees were satisfied with the support of Virtual Pointer in completing the task, however, their ratings were insignificant (Trainer: Median = 5.0, p = 0.167; Trainee: Median = 5.5, p = 0.066). 66.7% of the trainees thought that the Virtual Pointer made the task easier and reduced the amount of time to completion (M = 5.5, p = 0.066).
Figure 5.
Virtual Pointer in supporting task performance (TP: task performance; The numbers in the circle show the medians of the ratings.)
However, the Virtual Pointer was perceived not effective in reducing the performance efforts (M = 5.0, p = 0.144). Trainee #6, who strongly agreed that it minimized the effort on performing the task, contributed this effect to the trainers, justifying that “it saves time for attendings to explain things to medical students and observers”. Interestingly, one-third of the trainees reported distractions and one-third of the trainees reported no distractions resulted from the Virtual Pointer (M = 4.0, p = 0.705). These distractions were caused by the trainees’ concern about the accuracy of the information provided in the pointing or drawing, as justified by Trainee #5 that “it is difficult to draw accurate lines and unreliable to follow voice commands and manual gestures”. Besides, as suggested by Trainee #3, the trainees had to switch their attention between the telestrated screen and the field of surgical view.
User Experience of the Virtual Pointer
User experience is an important factor in user acceptance of technology and implementation of it in the workflow. Thus, we ask the trainers a couple of questions regarding the ease of use. As shown in Figure 6, more than half of the trainers had problems in using the system. Among them, 55.6% of the trainers thought that they would need the support of a technical person to be able to use the system. A main issue of the system, as identified by Trainer #1, was on the sensitivity in capturing the trainers’ hand movements, which interrupting their control of the system. Thus, the trainers relied on the immediate feedback from the system, as commented by Trainer #1, to verify the commands they gave to the system.
Figure 6.
User experience of the virtual pointer in the surgical training. (UE: user experience; The numbers in the circle show the medians of the ratings.)
Positive Attitude on the Virtual Pointer for Training
All trainers (M = 6.0, p = 0.007) and 66.7% trainees (M = 6.0, p = 0.063) agreed that using the Virtual Pointer in training was a good idea (Figure 7). 66.7% trainers (M = 6.0, p = 0.026) and trainees (M = 5.5, p = 0.059) agreed that Virtual Pointer was beneficial to trainees’ learning outcomes. Of all participants, only two trainers felt that using the system in training was unpleasant.
Figure 7.
Attitude towards using the Virtual Pointer in surgical training. (A: attitude; The number in the circles show the medians of the ratings.)
Intention to Use the Virtual Pointer
Although the trainers held a significantly positive attitude on the Virtual Pointer, they were conservative on using the Virtual Pointer in the lab (M = 5.0, p = 0.107), or the operating rooms (M = 5.0, p = 0.230). On the contrary, the trainees, who were moderately positive on the Virtual Pointer, would like to use the system in the lab training (M = 5.5, p = 0.026), but not in the operating rooms (M = 4.5, p = 0.257). Moreover, for using the system in varying difficulty levels, the approximate balance between ‘agree’ and ‘disagree’ indicates that case difficulty is not a decisive point for trainers and trainees to use the Virtual Pointer.
Factors Associated with the Acceptance of Virtual Pointer
The Spearman’s correlation between the questions allows us to investigate the factors associated with the acceptance of the Virtual Pointer in surgical training. Mann Whitney U tests were conducted to assess any significant difference for any common questions shared between the trainers and trainees. No significant difference was found. Thus, we combined these two groups for the responses of the common questions in the correlation analysis. Besides, due to the observed irrelevance of case difficulty for the use of Virtual Pointer, we excluded questions on the case difficulty (IE4, 5) and focused on the factors associated with the use in the lab and operating rooms (IE1, 2).
As shown in Table 1, the use of Virtual Pointer in the simulation lab (IE1) was significantly correlated with the support for task completion (TP1), acquired knowledge (AK1 – 5), and positive attitude on learning outcomes (A3). The use of the Virtual Pointer in the operating rooms was significantly correlated with the support for task completion (TP1), learning efforts (AK6) and acquired knowledge (AK1 – 3, 5). It is noteworthy that learning effort is significantly correlated with the use of the system in the operating rooms, while it is insignificantly correlated with the simulation lab. Task completion is more closely associated with the use of the system in the operating rooms, compared to the lab. On the contrary, acquired knowledge has a smaller effect on the use of the system in the operating rooms, compared to its high correlation with the use in the lab. Moreover, task completion (TP1) and learning efforts (AK6) were significantly correlated with the acquired knowledge, indicating that they may be the primary determinants of the usefulness of the system, as well as the acceptance of the system.
Table 1.
Correlation between questions.
| Correlations | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| N | Mean | Std. Err. | TP1 | AK1 | AK2 | AK3 | AK4 | AK5 | AK6 | A1 | A2 # | A3 | IE1 | IE2 | |
| TP1 Satisfied with VP in task completion | 15 | 5.11 | 0.408 | 1.000 | |||||||||||
| AK1 Increases overall knowledge | 15 | 5.33 | 0.330 | .764** | 1.000 | ||||||||||
| AK2 Increases anatomical knowledge | 15 | 5.25 | 0.300 | .538* | .801** | 1.000 | |||||||||
| AK3 Increases procedural knowledge | 15 | 5.28 | 0.284 | .618* | .810** | .739** | 1.000 | ||||||||
| AK4 Improves laparoscopic skills | 15 | 4.61 | 0.307 | .286 | .456 | .196 | .511 | 1.000 | |||||||
| AK5 Facilitates applying knowledge | 15 | 5.39 | 0.319 | .598* | .870** | .760** | .765** | .315 | 1.000 | ||||||
| AK6 Minimizes learning efforts | 14 | 5.04 | 0.406 | .458 | .704** | .813** | .758** | .313 | .806** | 1.000 | |||||
| A1 Using VP is a good idea | 15 | 5.89 | 0.284 | .483 | .477 | .513 | .457 | .052 | .529* | .387 | 1.000 | ||||
| A2# Using VP is UNPLEASANT | 15 | 4.97 | 0.365 | .015 | -.007 | -.162 | .140 | -.008 | -.012 | -.109 | -.041 | 1.000 | |||
| A3 Beneficial to the learning outcomes | 15 | 5.31 | 0.287 | .125 | .423 | .428 | .384 | .292 | .604* | .530 | .367 | -.160 | 1.000 | ||
| IE1 Use VP in the simulation lab | 14 | 5.35 | 0.339 | .570* | .851** | .700** | .714** | .564* | .719** | .490 | .255 | -.041 | .612* | 1.000 | |
| IE2 Use VP in the operarting rooms | 14 | 4.63 | 0.372 | .808** | .737** | .641* | .788** | .378 | .649* | .638* | .285 | -.005 | .502 | .699** | 1.000 |
**. Correlation is significant at the 0.01 level (2-tailed).
*. Correlation is significant at the 0.05 level (2-tailed).
#. The scale is reversed.
The Spearman’s correlations between unique questions for either the trainers or the trainees were further analyzed. Table 2 shows the correlations between trainer questions. Easy to use was significantly correlated with the use of Virtual Pointer in the lab and in the operating rooms. The satisfactory on the support of the system for conveying knowledge is significantly correlated with the use of Virtual Pointer in the operating rooms, while insignificantly correlated with the use in the lab. The increase of desire to teach is significantly correlated with the use in the lab, but insignificantly correlated with the use in the operating rooms.
Table 2.
Correlation between unique questions for the trainers.
| Correlations | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| N | Mean | Std. Err. | CK1 | CK2 | CK3 | CK4 | UE1 | UE2 | IE1 | IE2 | |
| CK1 Satisfied with the support for conveying knowledge | 8 | 5.13 | 0.639 | 1.000 | |||||||
| CK2 Satisfied with the time in conveying knowledge | 9 | 4.89 | 0.539 | .884** | 1.000 | ||||||
| CK3 Minimizes guiding effort | 8 | 5.25 | 0.412 | 0.455 | 0.199 | 1.000 | |||||
| CK4 Increases desire to teach | 9 | 4.56 | 0.556 | 0.148 | 0.199 | 0.423 | 1.000 | ||||
| UE1 The system is easy to use | 9 | 5.11 | 0.484 | .790* | 0.65 | 0.501 | -0.544 | 1.000 | |||
| UE2 Requires technical support | 9 | 4.44 | 0.648 | 0.057 | 0.357 | 0.511 | -0.415 | 0.115 | 1.000 | ||
| IE1 Use VP in the simulation lab | 8 | 4.88 | 0.479 | 0.689 | 0.199 | 0.637 | .761* | .897** | -0.423 | 1.000 | |
| IE2 Use VP in the operating rooms | 8 | 4.75 | 0.590 | .847** | 0.199 | 0.672 | 0.448 | .951** | -0.228 | .911** | 1.000 |
**. Correlation is significant at the 0.01 level (2-tailed).
*. Correlation is significant at the 0.05 level (2-tailed).
Among trainees’ unique questions, easier identification of target was significantly correlated with perceived task performance (Table 3). However, we have limited power to associate any factors to the acceptance of the system.
Table 3.
Correlations between unique questions for the trainees.
| N | Mean | Std. Err. | Correlations | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| AK7 | AK8 | TP2 | TP3 | TP4 | TP5 | IE1 | IE2 | ||||
| AK7 Assists identifying target | 6 | 5.830 | 0.563 | 1.000 | |||||||
| AK8 Increases desire to learn | 6 | 5.500 | 0.632 | -0.079 | 1.000 | ||||||
| TP2 Makes task easier | 6 | 5.500 | 0.477 | .833* | 0.143 | 1.000 | |||||
| TP3 Increases efficiency | 6 | 5.500 | 0.500 | .833* | 0.143 | 1.000** | 1.000 | ||||
| TP4 Reduces performance effort | 6 | 5.000 | 0.401 | .894* | 0 | .909* | .909* | 1.000 | |||
| TP5 VP causes distraction | 6 | 4.170 | 0.563 | 0.242 | 0.318 | 0.167 | 0.167 | 0.061 | 1.000 | ||
| IE1 Use VP in the simulation lab | 6 | 5.830 | 0.428 | 0.572 | 0.567 | 0.636 | 0.636 | 0.461 | 0.111 | 1.000 | |
| IE2 Use VP in the operating rooms | 6 | 4.500 | 0.833 | 0.485 | 0.111 | 0.788 | 0.788 | 0.485 | 0.045 | 0.652 | 1.000 |
**.Correlation is significant at the 0.01 level (2-tailed).
*.Correlation is significant at the 0.05 level (2-tailed).
DISCUSSION
In this study, we evaluated the perceived usefulness and acceptance of the Virtual Pointer system, which was designed to support efficient knowledge sharing in surgical training, among senior surgeons and surgical trainees. The study suggests that conveying the gestures directly onto the laparoscopic video has the potential to support intraoperative communication and increase the efficiency of the performance. The participants liked the idea of directly pointing and drawing on the video. The Virtual Pointer was perceived to support the trainees efficiently identify the target referred by the trainers on the monitor, enhancing the trainees’ anatomical knowledge and their understanding of the procedure, minimizing the learning efforts, and facilitating the application of the knowledge into practice. The trainers found it easy to convey the knowledge with the system and could foresee the learning benefits their trainees would have in laparoscopic surgery with the improved mentoring strategy.
To date, extensive effort has been contributed in designing systems to support knowledge sharing in the healthcare environment, such as hand-off tools, information retrieving systems, and augmentative information display20-22. However, there have been limited studies examining the opinions and attitudes of users of such systems, or novel concepts in HIT20, even though user perceptions and preferences are essential in the success of system adoption21. Previous research tended to use the technology acceptance model (TAM) examining the users’ perceived usefulness16,18 Our study suggests that although perceived usefulness from the users was an important factor, the root determinant is based on the perceptions of both the direct users and the stakeholders who would be affected by the system. For instance, in our study, the trainees are not the direct users of the system. Yet, their knowledge acquisition process is the main concern in the surgical training and in the trainers’ consideration of using the system in the lab or the OR. Besides, we found that the user experience of the system affects how the system is perceived to be used, when it would be used, and the system efficacy. Responses in our open questions suggest that for a system to be implemented in the surgical environment, the main concern is on the accuracy of the information the system provides and the interruption it may cause into the surgeons’ workflow.
Different users even from similar professional backgrounds use the systems in different ways24. In this study, we captured perceptual nuances of how the system would be used, what it would achieve, and where it could be used. This information could be used not only to validate technology objectives, but also to identify the target cases and environments for deployment, and to envision hindrances for implementation.
The different associated factors between the use in the lab and the use in the operating rooms reflect different system requirements in these two training environments. In the lab training, where trainees can gain proficiency in a safe environment, the system should support the initial learning, where a vast of factual knowledge is recalled and built up, motor skills are developed, and strategies are encouraged to be developed and tested out. In the operating rooms, where patient safety is emphasized, the system should target at improving the quality of performance, making the anatomical variants salient to minimize trainees’ effort in learning to see the field.
It is interesting to find that the trainees intended to use the system in the lab only, although the operating rooms are the key place for the trainees to gain expert vision on the anatomical structure and the decision making to proceed. Our correlation analysis between questions reveals several factors that hinder the acceptance of the system. The user experience is directly related to the success of the system adoption. However, the trainee indicated the concerns that the system may distract them from their task. The distraction may be due to that the illustration was shown on a secondary display, so that the trainees needed to frequently switch their attention between the laparoscopic video, which they were working on, and the Virtual Pointer screen, which they gained information from. This barrier highlights that the lag should be solved and the pointing and telestrating should be integrated into the trainees’ monitor. Besides, the trainers had problems in using the system. The trainers would need the support of a technical person to be able to use the system. They found the gestural and voice control was not very reliable. This barrier could be solved by providing a training session, where the system could be tested out by the users and further configured to recognize the gesture and speech of the individual surgeon. In addition, the system may interrupt the operation as the system required the trainer to pause the procedure and put down the instruments, waving in the air to draw any annotation. To address this issue, new interaction mechanisms that do not depend on arm movements can be designed to control the system. Furthermore, the system lacks precision. The trainees must spend extra effort to identify inaccurate annotations on the monitor, discuss them with the trainer, and make judgments on how to proceed. This suggests the algorithm to be improved to accurately relate the hand position to the pointer.
In this study, we evaluated the system in the Society of American Gastrointestinal Endoscopic Surgeons (SAGES) Annual Meeting, where most attendees are American surgeons. The strength of this setting is that it provided us a diverse pool of American surgeons from different hospitals. However, we acknowledge that surgical training varies among different countries. For example, residents may enter the OR at different levels or they may be trained using different tools. These factors may influence the efficacy of the system in surgical training. Besides, although, we have a rich collection of the trainers, who are the targeted user of the system, we have a limited sample size of the trainees. However, we identified the trend of perceived usefulness in the trainees’ responses and the trend was verified by the trainers’ perceived usefulness.
CONCLUSION
To facilitate the efficient conveyance of expert knowledge in laparoscopic training, we hypothesized to convey the gestures onto the video and designed the Virtual Pointer system that enables trainers to point or draw freehand sketches over a video for a trainee to see. In this study, we evaluated the perceived usefulness and acceptance of the system in surgical training. Our study indicates that conveying the gestures directly onto video is an effective and useful approach for the communication between surgeons. Barriers in the acceptance of the system highlight the design of novel interaction mechanisms to unobtrusively integrate this technique into the surgical workflow.
Acknowledgements
The authors gratefully acknowledge SAGES for equipment support. This work was partially supported by NSF Grants IIS #1422671 and #1552837. The authors thank Hannah McGowan for editing the manuscript, and the SAGES conference committee for the support of surgical instruments, training equipment, and the space.
Figures & Table
Figure 8.
Intention to use the virtual pointer in the surgical training. (IE: intention to use; The number in the circles show the medians of the ratings.)
References
- 1.Leape LL, Brennan TA, Laird N, Lawthers AG, Localio AR, Barnes BA, Hebert L, Newhouse JP, Weiler PC, Hiatt H. The nature of adverse events in hospitalized patients: results of the Harvard Medical Practice Study II. New England journal of medicine. 1991 Feb 7;324(6):377–84. doi: 10.1056/NEJM199102073240605. [DOI] [PubMed] [Google Scholar]
- 2.Gawande AA, Zinner MJ, Studdert DM, Brennan TA. Analysis of errors reported by surgeons at three teaching hospitals. Surgery. 2003 Jun 30;133(6):614–21. doi: 10.1067/msy.2003.169. [DOI] [PubMed] [Google Scholar]
- 3.Mentis HM, Chellali A, Schwaitzberg S. Learning to see the body: supporting instructional practices in laparoscopic surgical procedures. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems. 2014 Apr 26;:pp. 2113–2122. ACM. [Google Scholar]
- 4.Feng Y., Mentis H.M. 2017 Improving Common Ground Development in Surgical Training through Talk and Action. In AMIA Annual Symposium Proceedings (Vol. 2017, p. 696). American Medical Informatics Association. [PMC free article] [PubMed] [Google Scholar]
- 5.Gergle D, Kraut RE, Fussell SR. Action as language in a shared visual space. In Proceedings of the 2004 ACM conference on Computer supported cooperative work. 2004 Nov 6;:pp. 487–496. ACM. [Google Scholar]
- 6.Svensson MS, Luff P, Heath C. Embedding instruction in practice: contingency and collaboration during surgical training. Sociology of Health & illness. 2009 Sep 1;31(6):889–906. doi: 10.1111/j.1467-9566.2009.01195.x. [DOI] [PubMed] [Google Scholar]
- 7.Prentice R. Drilling surgeons: The social lessons of embodied surgical learning. Science, Technology, & Human ValuesM. 2007 Sep;32(5):534–53. [Google Scholar]
- 8.Hirschauer S. The manufacture of bodies in surgery. Social studies of science. 1991 May;21(2):279–319. [Google Scholar]
- 9.Cope AC, Mavroveli S, Bezemer J, Hanna GB, Kneebone R. Making meaning from sensory cues: A qualitative investigation of postgraduate learning in the operating room. Academic Medicine. 2015 Aug 1;90(8):1125–31. doi: 10.1097/ACM.0000000000000740. [DOI] [PubMed] [Google Scholar]
- 10.Zemel A, Koschmann T. ‘Put your fingers right in here’: Learnability and instructed experience. Discourse Studies. 2014 Apr;16(2):163–83. [Google Scholar]
- 11.Lingard L, Espin S, Whyte S, Regehr G, Baker GR, Reznick R, Bohnen J, Orser B, Doran D, Grober E. Communication failures in the operating room: an observational classification of recurrent types and effects. Quality and Safety in Health Care. 2004 Oct 1;13(5):330–4. doi: 10.1136/qshc.2003.008425. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Fussell SR, Setlock LD, Yang J, Ou J, Mauer E, Kramer AD. Gestures over video streams to support remote collaboration on physical tasks. Human-Computer Interaction. 2004 Sep 1;19(3):273–309. [Google Scholar]
- 13.Tang A, Neustaedter C, Greenberg S. Videoarms: embodiments for mixed presence groupware. People and Computers XX—Engage. 2007:85–102. [Google Scholar]
- 14.Feng Y, McGowan H, Semsar A, Zahiri HR, George IM, Turner T, Park A, Kleinsmith A, Mentis HM. A virtual pointer to support the adoption of professional vision in laparoscopic training. International journal of computer assisted radiology and surgery. 2018 Sep 1;13(9):1463–72. doi: 10.1007/s11548-018-1792-9. [DOI] [PubMed] [Google Scholar]
- 15.Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly. 1989 Sep;1:319–40. [Google Scholar]
- 16.Young MR, Klemz BR, Murphy JW. Enhancing learning outcomes: The effects of instructional technology, learning styles, instructional methods, and student behavior. Journal of Marketing Education. 2003 Aug;25(2):130–42. [Google Scholar]
- 17.Young MR. Windowed, Wired, and Webbed—Now What? Journal of Marketing Education. 2001 Apr;23((1)):45–54. [Google Scholar]
- 18.Hu PJ, Chau PY, Sheng OR, Tam KY. Examining the technology acceptance model using physician acceptance of telemedicine technology. Journal of management information systems. 1999 Sep 1;16(2):91–112. [Google Scholar]
- 19.Sullivan GM, Artino Jr AR. Analyzing and interpreting data from Likert-type scales. Journal of graduate medical education. 2013 Dec;5(4):541–2. doi: 10.4300/JGME-5-4-18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Abraham J, Kannampallil T, Patel V. A systematic review of the literature on the evaluation of handoff tools: implications for research and practice. Journal of the American Medical Informatics Association. 2013;21(1):154–162. doi: 10.1136/amiajnl-2012-001351. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Chen E. PalmCIS: A Wireless Handheld Application for Satisfying Clinician Information Needs. Journal of the American Medical Informatics Association. 2003;11(1):19–28. doi: 10.1197/jamia.M1387. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Parush A, Kramer C, Foster-Hunt T, Momtahan K, Hunter A, Sohmer B. Communication and team situation awareness in the OR: Implications for augmentative information display. Journal of biomedical informatics. 2011;44(3):477–85. doi: 10.1016/j.jbi.2010.04.002. [DOI] [PubMed] [Google Scholar]
- 23.Massaro TA. Introducing physician order entry at a major academic medical center: I. Impact on organizational culture and behavior. Academic Medicine. 1993 Jan; doi: 10.1097/00001888-199301000-00003. [DOI] [PubMed] [Google Scholar]
- 24.Zheng K, Padman R, Johnson MP, Diamond HS. Understanding technology adoption in clinical care: clinician adoption behavior of a point-of-care reminder system. International journal of medical informatics. 2005 Aug 1;74(7-8):535–43. doi: 10.1016/j.ijmedinf.2005.03.007. [DOI] [PubMed] [Google Scholar]