Differences in Mental Workload Between Traditional and Single-Incision Laparoscopic Procedures Measured with a Secondary Task

Abstract

Intro

The mental workload associated with laparoscopic suturing can be assessed with a secondary task that requires the same visual-spatial attentional resources. The purpose of this study was to use a secondary task to measure the incremental workload demands of single-incision laparoscopic surgery (SILS) procedures versus traditional laparoscopic procedures.

Method

12 surgery residents and surgical assistants who had met FLS criteria on an FLS and SILS simulator performed one trial each of peg transfer, cutting, and intracorporeal suturing tasks simultaneously with the secondary task and provided subjective workload ratings using the NASA-TLX.

Results

SILS procedures resulted in lower primary and secondary task scores, p < .001 and higher workload ratings, p < .0001. Suturing resulted in lower primary (p < .003) and secondary task scores (p < .017) and higher workload ratings (p < .017) compared to the other tasks.

Conclusions

SILS procedures were significantly more mentally demanding than traditional laparoscopic procedures corroborated by primary and secondary tasks scores and subjective ratings.

Background

Single-incision laparoscopic surgery (SILS) is performed with a single incision, usually through the umbilicus, as opposed to the multiple incisions used for traditional laparoscopic procedures.^1,2 The single incision offers patients potential cosmetic benefits due to less scar visibility.³ Other potential benefits include less postoperative pain and reduced recovery time, but these have not been firmly established.^4,5 SILS procedures do, however, introduce additional challenges to the surgeon beyond those associated with traditional laparoscopic procedures. The instruments are inserted in closer proximity to one another thereby reducing triangulation and restricting movement. There is greater opportunity for extracorporeal and intracorporeal instrument collisions as well as deflections of the laparoscope.^2,6,7 In fact, some surgeons have resorted to intracorporeal or extracorporeal crossing of the instruments to improve triangulation and maneuverability.⁸ Further, there are reports that surgeons find these procedures or specific port configurations to be more challenging than traditional laparoscopic procedures, with significantly increased operative time.^4,9,10,11

Accordingly, the purpose of the present study was to examine the incremental mental workload associated with SILS procedures over traditional laparoscopic procedures. Mental workload has been described as the attentional demands placed on workers or the demands associated with specific tasks.¹² According to Wickens, there are separate and distinct “pools” of attentional resources dedicated to different aspects of information processing.¹³ The encoding of perceptual and cognitive information in working memory uses different attentional resources than those required for the selection and execution of responses. Further, verbal and linguistic activities use different attentional resources than spatial activities and auditory activities use different resources than visual activities. According to Wickens’ theory, two tasks that use different pools of attentional resources can be timeshared reasonably well. However, when two tasks that are performed simultaneously draw resources from the same attentional pool, performance on one or both tasks is likely to suffer. Thus, a challenging primary task may use up the attentional resources leaving few or no residual resources for the second task. Hence, performance on the secondary task will decline as the difficulty of the primary task increases.

Several researchers have recently shown that when individuals perform a laparoscopic surgical task together with a secondary task requiring visual spatial attentional resources, the secondary task is sensitive to differences in laparoscopic demands. Specifically, increases in task difficulty or lower levels of skill resulted in a poorer secondary task performance.^14–16

Our group has recently developed a secondary task designed specifically for laparoscopic procedures. A visual-spatial task is superimposed directly onto the laparoscopic display resulting in a combined video image in which both tasks are viewed in the same focal field and require the same visual attentional resources. To date, we have shown that this secondary task is sensitive to differences among surgical experience, laparoscopic training tasks, and the transition from FLS to fresh cadavers.^17–19

The primary goal of this study was to establish the incremental mental workload associated with the more challenging SILS procedures using a visual spatial secondary task as well as subjective ratings. It was hypothesized that compared to FLS tasks, the more challenging SILS tasks would result in lower performance scores and higher mental workload as indexed by higher subjective ratings and lower secondary task scores.

Method

Participants

There were 12 participants (6 surgery residents and 6 students, 7 males and 5 females, in the Master of Surgical Assisting program at Eastern Virginia Medical School) in this IRB-approved study. The residents were trained over a period of 8 weeks in 1–2 hour sessions, once per week. The surgical assistants trained for 10 weeks in 1-hour sessions, once per week. The residents had passed FLS and the surgical assistants met the FLS criteria described by Ritter and Scott²⁰ for the tasks examined within 2–3 months of participating in the study.

Tasks

The primary tasks were the peg transfer, cutting, and intracorporeal knot tying from the FLS curriculum. A measure of speed and accuracy for each task was calculated so that higher scores would reflect better performance using the following calculations:

• Peg Task Score = 300 (maximum time limit in sec) − (completion time in sec) − 10*(# of drops)

• Cutting Task Score = 300 (maximum time limit in sec) − (completion time in sec) − 10*(accuracy error - distance from inside/outside of circle in mm, if greater than 2mm)

• Intracorporeal Suture Task Score = 300 (maximum time limit in sec) − (completion time in sec) − 10*(accuracy error - distance (in mm) from the dot on the penrose drain to the suture, with a 1mm allowance). Also, knot security errors or avulsion of the penrose drain resulted in a score of zero.

Participants performed the three primary tasks on a standard FLS simulator and a 3DMed simulator fitted with a Covidien SILS port and three 5 mm trocars, two for the laparoscopic instruments and one for a 5 mm 35° camera. They used standard laparoscopic instruments on both simulators: Ethicon Maryland graspers and scissors for the peg and cutting tasks, and Ethicon needle drivers for the suturing task.

The secondary task was the visual spatial ball-and-tunnel detection task in which four balls are presented in a representation of a 3D tunnel. Depth is conveyed within the tunnel by small dots that decrease in size and relative distance toward the center of the image. In the standard configuration, balls are located at the 12, 3, 6, and 9 o’clock positions. Participants are presented with successive images and asked to respond when they detect that a ball has changed its position. Targets consist of one ball that appears to shift in depth: either closer (the diameter increases from 26 to 53 mm and shifts 53 mm from the center) or farther (the diameter decreases from 26mm to 11mm and shifts 11mm from the center). Only one ball changes position at any given time and no importance is placed on which ball moves or the direction of movement. Images were presented for 300 ms every 2 to 4 s with a mean of 3 s. Participants responded to targets using a Savant Elite USB 3 pedal triple action foot switch. Performance in the ball-and-tunnel task was assessed by the proportion of correct detections, false alarms, and response times (RTs).

The ball-and-tunnel task is superimposed onto the video image from the simulator to create a single combined image showing both tasks to the user. Thus, the secondary task appears in the same viewing area as the primary task, but is shown at 50% transparency so that it does not fully obstruct the view of the primary task.

Subjective workload

Subjective ratings of mental workload were assessed using the NASA–Task Load Index (NASA-TLX).²¹ This instrument provides an overall index of mental workload based on the relative contributions of six subscales: mental, physical, and temporal task demands; effort; frustration; and perceived performance. Total workload scores were calculated by summing the ratings across the six subscales (range = 0 to 120). The psychometric characteristics of the NASA-TLX are well documented and Yurko, Scerbo, Prabhu, Acker, and Stefanidis reported that NASA-TLX scores reflected differences in task proficiency due to training and were also positively related to the errors in OR procedures.^22,23

Procedure

The participants first performed the ball-and-tunnel by itself to establish baseline performance. Next, they performed one trial each of the peg, cutting and suture tasks (in that order) simultaneously with the ball-and-tunnel task. There were given a maximum of 300 seconds to perform each task and then completed the NASA-TLX workload scale immediately afterward. They were then transferred to the SILS simulator and performed the same three tasks in the same order. Prior to the experimental trials they were given 5 minutes of practice to acclimate to the SILS environment.

Statistical Analysis

An a priori power analysis for repeated measures with a power level .80, correlation value of .5, to detect an effect size of .5 at the .05 level, required 12 participants for 2-level factors and 10 participants for 3-level factors. All data were analyzed with repeated-measures ANOVAs. Comparisons among the three tasks that did not meet repeated-measures variance requirements were evaluated with Greenhouse-Geisser conservative degrees of freedom and were followed up with Bonferroni post hoc corrected alpha level of .017. Comparisons with unequal variances were analyzed with a Wilcoxon Signed Rank Test.

Results

Primary Task

The results for the three tasks performed on the FLS and SILS simulators are shown in Table 1. The Wilcoxon Signed Rank Test showed that the SILS scores were lower than the FLS scores for all three tasks (p < .002).

Table 1.

Mean Primary Task Scores (standard errors)

	Peg Task	Cutting	Suturing
FLS*	213.4 (8.3)	191.2 (11.1)	150.4 (19.7)
SILS	10.4 (5.9)	61.1 (17.4)	2 (1.9)

Higher primary task scores represent better performance

^*All FLS higher than SILS, p < .002, Wilcoxon Signed Rank Test

Secondary Task

The percent correct detections and response times for the secondary task are shown in Table 2 separated by primary task and simulator along with the results from the baseline condition. Participants made very few false alarms (< 5%) so those data were not analyzed. The initial analysis consisted of a 2 (FLS, SILS) × 3 (peg, cutting, suture) repeated measures ANOVA. The difference between FLS and SILS tasks was significant, F(1, 11) = 53.931, p < .0001, η² = .831. The mean percent correct detections for SILS tasks (Mean = 38.3, SE = 5.7) was about 40% lower than for the FLS tasks (Mean = 65.3, SE = 4.7).

Table 2.

Mean Secondary Task Scores for All FLS and SILS Tasks (standard errors)

	Baseline	Peg FLS	Peg SILS	Cutting FLS	Cutting SILS	Suture* FLS	Suture SILS
Percent Correct	95.8 (1.9)	81.6 (3.7)	52.2 (7.1)	69.3 (6.9)	43.8 (7.4)	49.6 (7.0)	19.9 (6.1)
Response Time (sec.)	.752 (.030)	.836 (.042)	.961 (.034)	.895 (.043)	1.011 (.052)	.920 (.027)	1.121 (.088)

Response Times for FLS and SILS did not differ for Cutting and Suture tasks.

^*All Suture Percent Correct scores differ, p < .003, Wilcoxon Signed Rank Test

The difference among tasks was also significant, F(2, 22) = 44.51, p < .0001, η² = .802. The data for the FLS and SILS simulators were compared with the baseline for each primary task with separate one-way repeated measures ANOVAs. For the peg task, the difference among the means was statistically significant, F(1.3, 11.7) = 24.242, p < .001, η² = .729. Participants made fewer correct detections during the SILS task than the FLS task and correct detections lower for both simulators compared to baseline (p < .015 for all comparisons). A significant difference was also found among the response times, F(2, 18) = 34.353, p < .001, η² = .782. Participants were slower to respond during the SILS than the FLS task and slower for both simulators compared to baseline (p < .03 for all comparisons).

For the cutting task correct detections, the difference among the means was statistically significant, F(2, 18) = 30.038, p < .001, η² = .769. Again, correct detections were lower for the SILS task than the FLS task and lower for both simulators compared to baseline (p < .015 for all comparisons). A significant difference was also found among the response times, F(2, 18) = 12.273, p < .001, η² = .577. Participants were slower for both simulators compared to baseline (p < .003), but the difference between the simulators was not significant.

Regarding the suturing task, the Wilcoxon Signed Rank Test showed that detections were lower with SILS than FLS (p < .003), and both simulators were lower than baseline (p < .003). For response times, the difference among the means was statistically significant, F(1.1, 10.1) = 13.906, p < .003, η2 = .607. Participants were slower for both simulators compared to baseline (p < .002), but the difference between the simulators was not significant.

The correct detections were lower for the suturing task (Mean = 33.6, SE = 5.5) compared to the peg (Mean = 66.2, SE = 4.7) and cutting tasks (Mean = 55.5, SE = 5.7), which did not differ. This pattern was the same for both FLS and SILS procedures.

Workload

The workload ratings are shown in Table 3 separated by primary task and simulator along with the results from the baseline condition. The initial analysis consisted of a 2 (FLS, SILS) × 3 (peg, cutting, suture) repeated measures ANOVA. The difference between FLS and SILS was significant, F(1, 11) = 166.44, p < .0001, η² = .938. The mean workload rating for SILS tasks (Mean = 105.22, SE = 3.98) was about 60% higher than for the FLS tasks (Mean = 65.14, SE = 5.15).

Table 3.

Mean Workload Ratings for All FLS and SILS Tasks (standard errors)

Baseline	Peg FLS	Peg SILS	Cutting FLS	Cutting SILS	Suture FLS	Suture SILS
39.5 (5.4)	55.9 (5.2)	102.3 (4.6)	63.6 (4.0)	99.1 (6.6)	75.9 (7.3)	114.3 (2.0)

Higher NASA-TLX ratings reflect higher metal workload.

The difference among tasks was also significant, F(2, 22) = 19.37, p < .001, η² = .638. The ratings for the FLS and SILS simulators were compared with the baseline for each primary task with one-way repeated measures ANOVAs. The results showed that workload ratings were significantly higher for SILS over FLS on all three primary tasks: Peg, F(2, 20) = 90.626, p < .0001, η² = .901; Cutting, F(2, 20) = 61.385, p < .0001, η² = .860; Suture, F(2, 20) = 88.223, p < .0001, η² = .898; and further ratings were higher on all simulator tasks compared to baseline (p < .03 for all comparisons).

The differences among tasks were also compared. The workload ratings were higher for the suturing task (Mean = 95.08, SE = 4.26) compared to the peg (Mean = 79.12, SE = 4.58) and cutting tasks (Mean = 81.33, SE = 4.98), which did not differ.

Discussion

The present study compared the task demands between traditional laparoscopic and SILS procedures on simulators. The results showed that the SILS procedures were significantly more challenging than traditional laparoscopic procedures. The primary task results showed a significant drop in performance scores for all tasks performed on the SILS simulator. In fact, the mean time and accuracy-based score for suturing on the SILS simulators was nearly zero.

Collectively, these results provide additional evidence for the real challenge posed by single-incision procedures.^9,10 Our participants demonstrated fair to good performance across all three tasks on the FLS simulator, but struggled with each one on the SILS simulator. In fact, only one participant was able to complete the SILS suturing task within the 300-second time limit. The participants clearly recognized the increased difficulty with the SILS tasks, rating the workload 60% higher than the FLS tasks, with total workload scores near the high end of the NASA-TLX scale. The higher workload ratings for the SILS procedures were observed across all three tasks.

More important, the incremental difficulty of the SILS tasks was mirrored in the secondary task scores. The percentage of correct detections declined by more than 40% when participants transitioned from the FLS to the SILS simulator and the deficit was observed on all three tasks. This pattern of results shows that the increased mental workload associated with performing the SILS procedures demanded most of the participants’ attention, leaving few resources available to respond to targets appearing the same visual field.

It should be noted that our results differ somewhat from those of others who did not find dramatic differences between conventional laparoscopic and single-incision procedures on a simulator⁹ or in subjective ratings.¹⁰ In both of these experiments, the investigators used novice participants with no prior laparoscopic experience. Second, these investigators studied only the peg transfer task. By contrast, we examined several different tasks with participants who had all met FLS criteria. Thus, it is possible the differences observed in our study emerge when individuals are beyond the initial portion of the learning curve. Further, the substantial differences seen on the suturing task in this study suggest that the incremental workload imposed by SILS procedures may be exacerbated by more demanding laparoscopic tasks.

Another important finding from this study was that the ball-and-tunnel task showed sensitivity to differences among the laparoscopic tasks. The percentage of correct detections was lower for suturing than the peg and cutting tasks. Compared to the baseline condition, secondary task scores were lower for all laparoscopic task conditions. Thus, the ball-and-tunnel secondary task was sensitive to differences between the FLS and SILS formats as well as the more difficult suturing task compared to the peg and cutting tasks. In addition, the workload ratings for the suturing task were higher than for the peg and cutting tasks. Thus, for this experiment, the subjective workload scores reflected differences among laparoscopic tasks and complemented the secondary task scores.

One limitation to this study concerns the experience levels of the participants. Although all participants had met the FLS criteria described by Ritter and Scott,²⁰ they were also new to the SILS environment. Thus, the workload differences observed between FLS and SILS tasks in this study are likely greater than what might be obtained with more experienced SILS surgeons. However, the unique challenges associated with SILS described above have been shown to increase workload and surgical operating times over traditional laparoscopic tasks.^4,9,11 Thus, the incremental attentional demands posed by SILS for surgeons with experience in this form of surgery need to be verified with the secondary task method.

Second, the SILS port and laparoscopic instruments used in this study placed severe limits on instrument movement and triangulation and likely contributed to the high workload findings. Also, the simulators were from different manufacturers. Thus, it is possible that differences in the physical characteristics of the simulators could have played some role in results. Thus, other port systems and articulated instrument designs offer greater freedom of movement and better triangulation and may result in lower workload. This possibility should be examined in the future.

The goal of this study was to measure the incremental workload demands of SILS over traditional laparoscopic procedures using the secondary task technique and subjective ratings. The results showed that the SILS tasks were indeed more difficult. Performance scores for the peg transfer, cutting, and suturing tasks were all substantially lower for the SILS tasks. The lower SILS performance scores were accompanied by significantly lower secondary scores, indicating that participants had a decreased ability to attend to little more than the primary task. Further, the lower performance and secondary task scores were corroborated by increased subjective workload ratings. Collectively, this pattern of results underscores the cognitive difficulty faced by surgeons when initially transitioning from the traditional laparoscopic to the SILS environment.

The results from this study also show that the ball-and-tunnel secondary task can provide a complementary measure of mental workload, corroborated by traditional measures of primary task performance and subjective ratings. Further, the present study supports the results of others who have shown that the ball-and-tunnel task is sensitive to differences in laparoscopic difficulty.^17–19 Collectively, these results suggest that the ball-and-tunnel task offers a standard, common index of mental workload that allows investigators to compare relative differences among a variety of laparoscopic tasks. This secondary task may enhance opportunities to assess subtle differences in laparoscopic performance that indicate when skills have reached the criteria that maximize their transfer to the clinical environment.²⁴ Ideally, the benefits of such a secondary task would be expected to improve patient safety by helping optimize competency of practicing surgeons. Ultimately, we hope that the real benefit of using the secondary task method will be seen in improved patient safety achieved by helping optimize the competency of practicing surgeons.