Validation of a Novel Laparoscopic Adjustable Gastric Band Simulator

Abstract

Background

Morbid obesity accounts for more than 90,000 deaths per year in the United States. Laparoscopic adjustable gastric banding (LAGB) is the second most common weight loss procedure performed in the US and the most common in Europe and Australia. Simulation in surgical training is a rapidly advancing field that has been adopted by many to prepare surgeons for surgical techniques and procedures.

Study Aim

The aim of our study was to determine face, construct and content validity for a novel virtual reality laparoscopic adjustable gastric band simulator.

Methods

Twenty-eight subjects were categorized into two groups (Expert and Novice), determined by their skill level in laparoscopic surgery. Experts consisted of subjects who had at least four years of laparoscopic training and operative experience. Novices consisted of subjects with medical training, but with less than four years of laparoscopic training. The subjects performed the virtual reality laparoscopic adjustable band surgery simulator. They were automatically scored, according to various tasks. The subjects then completed a questionnaire to evaluate face and content validity.

Results

On a 5-point Likert scale (1 – lowest score, 5 – highest score), the mean score for visual realism was 4.00 ± 0.67 and the mean score for realism of the interface and tool movements was 4.07 ± 0.77 [Face Validity]. There were significant differences in the performance of the two subject groups (Expert and Novice), based on total scores (p<0.001) [Construct Validity]. Mean scores for utility of the simulator, as addressed by the Expert group, was 4.50 ± 0.71 [Content Validity].

Conclusion

We created a virtual reality laparoscopic adjustable gastric band simulator. Our initial results demonstrate excellent face, construct and content validity findings. To our knowledge, this is the first virtual reality simulator with haptic feedback for training residents and surgeons in the laparoscopic adjustable gastric banding procedure.

Introduction

Obesity is a disease that has now reached epidemic proportions. The World Health Organization (WHO) predicts that there will be 2.3 billion obese (BMI >30) individuals worldwide by 2015. Commonly performed weight loss surgical procedures include the Roux-en-Y gastric bypass (RYGBP), laparoscopic adjustable gastric band (LAGB) and gastric sleeve.

Virtual reality (VR) based trainers offer the advantage of unlimited training material, exposure to rare adverse events, a customizable training environment and objective assessment in real time without the need for a proctor. A virtual reality simulator for training in gastric bypass is available from Simbionix (Simbionix, USA Inc.). For the LAGB procedure, however, no virtual reality trainer currently exists.

The objective of our study is to evaluate the face, content and construct validity of a virtual reality LAGB (V-Band) simulator. We have developed a VR-based simulator for the LAGB performed via the pars flaccida approach for band placement. This simulator has custom built interface and simulation software and provides force feedback to the user. In this manuscript we report the face, content and construct validation of this simulator [1–3].

Methods

The simulator consists of a hardware platform with a laparoscopic tool interface that provides haptic feedback to the user and simulation software that enables the replication of the actual procedure on the computer. The hardware platform and simulation software are presented below. The LAGB procedure was divided into a set of tasks by experts (DJ, JA) for the validation studies which are presented next.

Hardware Platform

The hardware platform consists of a model of a fiberglass abdomen that was fabricated to imitate an insufflated abdomen during surgery. The abdomen was mounted on an adjustable aluminum frame attached to the base of a wooden platform. Two laparoscopic surgical instruments – a laparoscopic grasper and a laparoscopic electrocautery device - were inserted through the 5 mm trocars placed in holes made in the abdomen model. The tips of the tools were attached to the stylus of two PHANToM Omni haptic interface devices (Sensable Technologies, Inc., Boston, MA) placed on the platform using a standard RC mini jack. Each PHANToM Omni is a low-cost 3 degrees-of-freedom force feedback device which can sense 6 degrees of freedom displacement. A linear potentiometer attached to the tool handle was programmed to record the opening and closing of the virtual tool tip. A USB foot pedal interface (Savant Elite, Kinesis Corporation, Bothell, WA) with three pedals was part of the interface that provided additional inputs to the simulator. One of the foot pedals was used for electrocautery, another to toggle the camera view between two fixed views and the third to initiate the timer for timing the tasks. Figure 1 demonstrates the hardware setup of the simulator in use.

Figure 1.

The virtual LAGB simulator with the hardware interface

Simulation Software

The simulation software consists of anatomical models of the upper abdomen simulated using our in-house software framework. The anatomical models of liver, stomach, spleen, gastrohepatic ligaments and diaphragm were custom-modeled by Zygote Media, Inc. (Zygote Media Group, American Fork, UT). The ligaments, fat and internal organs were simulated at interactive rates using the NVIDIA PhysX engine [4]. Wet shaders were used in rendering the anatomical scene to provide visual realism. Real-time bleeding (Figure 2a) and smoke (Figure 2b) effects were also rendered using image-based techniques. A grasper and an electrocautery device were made available to the user in the virtual scene. The laparoscopic adjustable gastric band was simulated using a hybrid technique with articulated links for the band coupled with a spring-mass system with bending stiffness for the tube. This simulator reflects the pars flaccida surgical approach, initially described by Fielding et al [5]. The pars flaccida technique has shown to have less post-operative, long-term LAGB complications, specifically with reduction in gastric prolapse, compared to alternative approaches [6]. The steps of the simulated LAGB placement are shown in Figure 3.

Figure 2.

(a) Bleeding while dissecting pars flaccida. (b) Smoke rendering during electrocautery.

Figure 3.

The virtual LAGB simulator tasks: (a) Undissected scene, (b) Dissection of pars flaccida, (c) Dissection of the pertioneal layer medial to right crus, (d) Grasping of the band on top of the fundus through the retrogastric channel, (e) Placement of the band around the stomach, (f) Locking the band in place.

Experiment Design

The virtual LAGB simulator was tested for face, content and construct validity. For this Institutional Review Board approved study, subjects were recruited from Beth Israel Deaconess Medical Center and Harvard Medical School in Boston, MA. Twenty eight subjects volunteered to participate in the study, performed at the Carl Shapiro Minimally Invasive Institute at Beth Israel Deaconess Medical Center. A power analysis was done for estimating the sample size for the two groups in this study using the G*Power software [7]. Assuming a 50% difference between the novice and expert groups, a total of 13 subjects for each group was needed to detect difference at 80% power with α=0.05.

Subjects were categorized into two groups and consisted of experts (PGY 5/fellows/faculty) and novice (4^th year medical students and PGY1–4). There were 13 subjects in the expert group and 15 in the novice group.

In studies testing novel surgical simulators, it is essential to determine face, content and construct validity. Face validity determines the extent that the simulator resembles the actual task in real life. Content validity determines the extent to which all relevant dimensions (i.e., tasks) within a given domain (i.e., surgical operation for placement of LAGB) are measured. Construct validity determines the ability to detect differences in the ability to complete the simulation correctly depending on levels of competence.

Face and content validity were established using a 5-point Likert scale questionnaire [8] The face validity questions asked subjects to assess the realism of the simulated procedure, realism of tool movements and the realism of haptic feedback. For content validity, the experts were asked to complete questions on the usefulness of the simulator for training residents/surgeons and possible certification. Construct validity was assessed by automated computing normalized individual scores for band placement and dissection of ligaments and fat using electrocautery, in addition to a combined total score. Penalty in scores were assessed by the number of times the band was dropped and the size of the perforations made in the stomach or the esophagus.

For electrocautery and band placement tasks, completion time was the objective measurement factor. The completion time was then subtracted from a predetermined cutoff time for these tasks to determine a numerical score that rewards the speed of completion [9]. For both these tasks, penalty was awarded for the size and number of perforations in the stomach and esophagus. For the band placement task, the total number of band drops was recorded and a corresponding penalty was applied to this task score (Table 1).

Table 1.

Virtual LAGB Simulator Tasks and Scoring Parameters

Task Name	Description	Scoring Parameters
		Measure	Penalty
Electrocautery	Dissection of pars flaccida and exposure of the right crus	Time (s)	Perforations in stomach and esophagus (n)
Band Placement	Placement of the band around the stomach and locking it in place	Time (s)	Perforations in stomach and esophagus (n) + number of band drops (n)

At the beginning of the experiment, each subject filled in a demographic sheet for background information and was shown an instructional video that explained the LAGB procedure and the tasks at hand. Each subject was then allowed two practice trials to familiarize themselves with the virtual LAGB simulator interface and the procedure. After the subjects completed the practice trials, they proceeded to complete the simulation. The locking of the band was considered the end of the procedure. The novice and expert subjects were then asked to fill in the face validity questionnaire while only the expert subjects were asked to fill out the content validity questionnaire.

Data Analysis

For statistical analysis, the SPSS 18.0 statistical software package (Chicago, IL) was used. For face validity, descriptive statistics for scores were first computed for both the groups. A two-tailed Mann-Whitney U test was used to test the agreement on the questions between the two groups. For content validity, descriptive statistics was computed and reported. For construct validity, descriptive statistics was computed for both individual and total scores. Because of the smaller sample size and the non-normality of the data, a two-tailed Mann-Whitney U test, the non parametric equivalent of the independent samples t-test, was used to evaluate difference between the two groups.

Results

Face Validity

The scores pertaining to face validity are reported in total (novice and expert groups combined) and in individual groups (novice and expert, respectively) (Table 2). On a 5-point Likert scale, the highest mean rating for an individual question was 4.07 for the question related to the realism of equipment and laparoscopic instruments. The lowest mean rating for an individual question was 3.32 for the realism of the stomach and fat behavior. The second highest rating was 4.0 for the question related to the quality of the visual display of the surgical scene. The question relating the overall realism of the simulator compared to the actual procedure received a combined mean of 3.5, whereas the question related to the subject’s trust in the ability of the simulator to quantify accurate measures of performance received a combined rating of 3.54. Subjects rated the quality of force feedback in the simulator with a score of 3.36. The Mann-Whitney U test between the scores of the two groups showed no significant difference (p>0.2), indicating a general agreement on all questions between the expert and the novice groups.

Table 2.

Face Validity Scores

Questionnaire	Total		Expert		Novice		M-W U Test p Value
	Mean	Std	Mean	Std	Mean	Std
1. Overall Realism of the Simulator (Compared to actual procedure)	3.50	0.839	3.38	0.870	3.60	0.820	0.856
2. Quality of Visual Display of the surgical scene	4.00	0.667	4.08	0.641	3.93	0.704	0.618
3. Quality of the Force Feedback	3.36	0.951	3.00	0.816	3.67	0.976	0.339
4. “Realism” of Equipments and Tools (“realism” of laparoscopic tools, trocar placement and physical Interface)	4.07	0.766	4.00	0.816	4.13	0.743	0.786
5. Realism of the Anatomical Features	3.61	0.737	3.38	0.650	3.80	0.775	0.316
6. Realism of the Stomach and Fat behavior.	3.32	0.819	3.08	0.862	3.53	0.743	0.496
7. Realism of the Lap-band behavior	3.39	0.875	3.23	0.927	3.53	0.834	1.00
8. Your Trust in the Ability of this simulator to Quantify Accurate Measures of Performance	3.54	0.962	3.15	0.987	3.87	0.834	0.217

Construct Validity

The completion times used to compute the individual and overall scores for the two tasks are shown in Figures 4. It can be seen from Figure 4a and b that the overall time for both the tasks combined ranged from 2 minutes for our most expert surgeon to 11 minutes for a novice. The scores pertaining to construct validity are demonstrated in Table 3. Experts received higher scores than novice subjects (total score 80.57 vs. 54.36, respectively; p< 0.0002; Figure 5). For the band placement task, the mean score was higher for the experts compared to the novices (86.62 vs 56.28, respectively; p=0.0003; Figure 6). For the electrocautery task score, the expert group scored higher than the novice group (75.86 vs. 58.10, respectively; p=0.006; Figure 7).

Figure 4.

Simulation time in minutes for: (a) Lap-band placement task, (b) Electrocautery task.

Table 3.

Construct Validity Scores

Scoring Parameter	Experts		Novice		Mann-Whitney U Test
	Mean	Std	Mean	Std
Lap-Band Placement	86.623	10.758	56.282	24.447	0.0003
Electrocautery	75.861	22.585	58.095	16.518	0.0061
Total Score	80.572	14.82	54.36	18.578	0.0002

Figure 5.

Box plot for the total score.

Figure 6.

Box plot for band placement score.

Figure 7.

Box plot for the electrocautery score.

Content Validity

The descriptive results for content validity, completed by the expert group, are demonstrated in Table 4. The mean score for Question 2 was 4.5 (Question 2: Could the VR-LAGB simulator be a useful trainer for residents/surgeons before their operating room experience?). The mean score for both Questions 1 and 3 was 3.4 (Question 1: Is the VR-LAGB a realistic trainer for the LAGB procedure? Question 3: Could virtual LAGB simulator be a useful trainer for certification in this field?). The ratings ranged from a lowest score of 1 by one expert for Question 3 to the highest rating of 5 by expert subjects for Question 2.

Table 4.

Content Validity Scores

Questionnaire	Mean	Std
1. Is the VR-LAGB a realistic trainer for the LAGB procedure?	3.40	0.699
2. Could the VR-LAGB simulator be a useful trainer for residents/surgeons before their operating room experience?	4.50	0.707
3. Could the VR-LAGB simulator be a useful trainer for certification in this field?	3.40	0.699

Discussion

The obesity epidemic in the United States and abroad is a serious public health issue. The demand for surgical treatment of morbid obesity is greater than the availability of trained providers. Laparoscopic adjustable banding is becoming the most commonly performed procedure for this indication, yet many surgeons still do not have this as a part of their surgical training. Models and simulators that can assess proficiency will provide a role in training programs, both for surgeons in formal residency and fellowship programs, as well as surgeons already in practice.

We have designed the first LAGB virtual reality simulator. In our study, subjects with varied laparoscopic experience were able to have exposure to this simulation device. Through our trials, we were able to determine excellent face, content and construct validity. The face validity scores highlight the positive response of the users to the visual and haptic realism of our simulator. The virtual laparoscopic tools were modeled based on real tools to the same scale. Moreover the interface was made as realistic as possible by using a model of the abdomen, real 5 mm trocars and using actual laparoscopic tools provided for interaction. Also, during electrocautery interaction, subjects stated that the rendering of the smoke was very realistic. The construct validity results demonstrate that the current version of the virtual LAGB simulator can clearly distinguish between expert and novice groups. Between the two tasks, the band placement task scores were relatively better than the electrocautery task scores in differentiating between the two groups. Experts scored generally high, while the scores of the novice group ranged from low to high values. Novices with some experience with manipulation of the laparoscopic tools did better than the novice subjects with very little laparoscopic experience. With the electrocautery task, the scores varied widely for both the expert and novice groups. This may be due to the subjective nature of the decision of how much of the pars flaccida and the peritoneal layer require removal during the procedure. The variability was greater in the novice group, as they either removed too little or too much of the ligamentous structures. Also, five of the novices made perforations in either the stomach or the esophagus, compared to just two experts committing the same error. The content validity results showed that attending surgeons with great clinical laparoscopic experience have a high confidence with the virtual LAGB simulator. The experts agreed that the virtual LAGB simulator would be a useful tool to train residents and surgeons in the LAGB procedure before their first operating room experience.

While the current version of the LAGB simulator includes the most critical steps in the LAGB procedure, it excludes Angle of His dissection, GE junction fat pad excision and gastrogastric imbrication. We plan to extend the simulator to include these steps before engaging in a multi-institutional study to determine the transferability of skills from the simulator to the operating room. In this future study, we will randomize trainees to two groups, a.) one group with traditional clinical and operative training and b.) one group with training sessions with the virtual LAGB simulator, in addition to traditional clinical and operative training. We anticipate that training with the LAGB simulator will result in better surgical outcomes in trainees. In conclusion, we believe that models and simulators, such as this LAGB simulator, will assure safe surgical practices and will become more of the cultural ethos of surgical training.