Cognitive Load in Electromagnetic Navigational and Robotic Bronchoscopy for Pulmonary Nodules

Abstract

Background: Electromagnetic navigational bronchoscopy (ENB) and robotic-assisted bronchoscopy (RB) require a high degree of decision-making and psychomotor skill. Cognitive load theory is the overall effort expended by individuals in response to a task and is closely related to the usability of devices for medical procedures. High cognitive workload leads to poor surgical outcomes and represents a bottleneck for learning that affects performance.

Objective: To analyze the cognitive load associated with ENB and RB in experienced ENB practitioners learning RB.

Methods: Six experienced ENB bronchoscopists performed ENB and RB on a human cadaver model of peripheral pulmonary nodules. To assess cognitive load, we used the Surgery Task Load Index (SURG-TLX) and biometric changes. The SURG-TLX questionnaire was given to the provider after every peripheral pulmonary nodule biopsy with ENB and RB. Pupillary dilation and screen changes were continuously measured throughout the procedure for each biopsy attempt to collect biometric measures of cognitive load. Procedural time and biopsy outcome were also recorded.

Results: Forty procedures (ENB and RB) were analyzed. Task complexity (23%) and mental demand (21.4%) were the highest contributors to cognitive load in ENB and RB. The cumulative SURG-TLX was significantly lower for the RB (69.25 vs. 101.25; P < 0.01). Total procedure time was greater for ENB (6.7 min; SD 1.5) compared with RB (4.4 min; SD 1.5; P = 0.01). Pupillary diameter was similar across the modalities (RB vs. ENB), but the diameter was higher during the biopsy portion (4.25 mm) than the navigation portion (4.01 mm).

Conclusion: The intrinsic cognitive load of RB was highly manageable by existing ENB practitioners, and in this study, RB appeared to be less mentally demanding. Future development and training should focus on task complexity and mental demand for RB. The biopsy portion, regardless of bronchoscopic modality, should be a focus for education and training.

Cognitive load (CL) theory is the level of overall effort expended by individuals in response to a task and is closely related to the usability of devices developed to fulfill the task (1). Procedures such as electromagnetic navigational bronchoscopy (ENB) and robotic-assisted bronchoscopy (RB) use complex devices to biopsy pulmonary nodules. Each of these procedures has a CL associated with it, and by understanding operators’ subjective and objective perceptions of CL in ENB and RB, we may be able to identify learning bottlenecks and improve outcomes. Although no studies exist in bronchoscopy, high amounts of CL have been associated with worse surgical outcomes (1).

RB is a novel platform to biopsy pulmonary nodules, particularly in comparison with ENB (2–4). In a prospective, randomized, single-blinded trial in a cadaveric model, data showed operators using RB had an improved ability to navigate to and accurately biopsy peripheral pulmonary nodules (5). However, the addition of novel technology requires understanding the bottlenecks that slow the development of competency and expertise.

Although CL has never been evaluated in the bronchoscopy setting, it has been studied in the minimally invasive surgical setting. High amounts of CL have been associated with worse surgical outcomes, and understanding CL can help to develop expertise (1, 6). Although bronchoscopy and surgery are not interchangeable, the similarities between laparoscopic surgery and ENB (reliance solely on the operator to position the camera and necessary tools for diagnostics and therapeutics) as well as RB and robotic surgery (similar robotic platforms) afford us the ability to use surgical literature to motivate our study. In the surgical literature, the robotic approach took longer to perform than the laparoscopic technique for simple tasks, but there was a trend toward less subjective stress in robotics for complex tasks (7). In addition, prior experience with conventional surgery improved ease of obtaining robotic surgery skills, but focus in certain areas for improvement was still required (8).

Cognitive workload has not been evaluated in ENB or RB. In this study, we aim to compare the CL of ENB with that of RB in a population of experienced ENB, but novice RB, operators.

Methods

Six experienced ENB bronchoscopists (more than 100 ENB cases in the past 24 mo) with no prior experience with RB were recruited into the study. All practitioners had an initial systems orientation to the Ion Robotic Bronchoscopy System (Intuitive), and all procedures were performed in the same endoscopy suite reviewing controller functions and needle operation. Demographic and self-assessment information was collected on each participant’s prior bronchoscopy experience (Table E1 in the data supplement).

Practitioners performed ENB (SuperDimension version 7.1; Medtronic) and RB (Ion Robotic Bronchoscopy System) on four different nodules located in different locations in a cadaveric model. Both systems used a transbronchial needle aspiration needle for a biopsy instrument together with their associated device; a 90° extended working channel (SDK3900-FT; Medtronic), a 21-G (AKI00101–01; Medtronic) peripheral needle, and a 21-G (Flexision needle; Intuitive Surgical) peripheral needle. The order of technology used was randomized throughout the study.

Metrics

The Surgery Task Load Index

Immediately upon completion of the first attempted biopsy of each pulmonary nodule, the physicians completed the Surgery Task Load Index (SURG-TLX) questionnaire, a validated instrument for measuring the following six dimensions of CL: mental demands, physical demands, temporal demands, task complexity, situational stress, and distractions (9). The end result is a total workload score that encompasses all six dimensions of CL. Each dimension is scored from 0 to 100 by multiplying two subscores that the operator determines together. The first score ranks the six components of CL on the basis of the procedure just performed from 5 (largest portion of the workload) to 0 (lowest portion of the workload). Then, for each of the six components of CL, the operator is given a 20-point Likert scale that ranges from 0 to 20. These two scores are then multiplied together for each of the six components of CL, and an aggregate total workload is obtained by adding the six components (9).

The SURG-TLX is further subdivided into intrinsic dimensions (mental demands, physical demands, temporal demands, and task complexity) and extrinsic dimensions (situational stress and distractions). Because we were using a cadaveric model and the same endoscopy suite for all procedures, extrinsic dimensions were standardized across all providers.

Biometrics (pupillary dilation and gaze deviation)

While performing the ENB and RB procedures, physicians used eye-tracking glasses at 50 Hz (Tobii Technologies) to measure pupillary dilation (task-evoked pupillary response) and track gaze deviation between different screens used during the procedures. Baseline pupillary diameter were measured in the procedure room before the start of any procedure. Changes in pupil size correlate with changes in cognitive processing demands, with maximum dilation representing the highest CL (intrinsic and/or extrinsic) in various parts of a procedure (navigation and biopsy). We averaged subsequent pupil diameter measurements to determine a mean pupillary size reading per second. These data points then were averaged for the navigation or biopsy portion of the procedure on ENB and RB to give a composite mean percentage pupil diameter change from baseline. Gaze deviation was tracked via counting the number of times the pupil focused on a different screen throughout the procedure. To account for differences in time across the different platforms, we standardized gaze deviation per minute (average number of gaze deviations at a given lesion/average total procedural time at a lesion in minutes). Notably, one of the practitioners was excluded from gaze deviation given his need for corrective lenses, precluding the wearing of the eye-tracking glasses.

Procedural time

Procedural time was collected and divided into a navigation phase, a biopsy phase, and total procedural time. The navigation phase is defined as the time from when the scope enters the endotracheal tube until the nodule target is reached. The biopsy phase is defined as the time from when the nodule target is reached until the end of the first biopsy.

Statistics

SURG-TLX scores were analyzed using either paired student’s t tests for normative data and Wilcoxon matched-pairs signed-rank tests for nonparametric data. Categorical data were compared using the χ² test and were converted to proportions before analysis when applicable. Logistic regression analyses were used to investigate correlation with outcomes.

Results

Forty procedures (ENB and RB) were analyzed. Figure 1 shows the percentage distribution of the six components of CL broken down by RB and ENB. Task complexity (23%) followed by mental demand (21.4%) were the two highest contributors to SURG-TLX CL in ENB and RB. The aggregate total workload scored by SURG-TLX was significantly lower for the robotic bronchoscopy (mean, 69.25 vs. 101.25; P < 0.01; median, 73 vs. 103), as depicted in Figure 2. The SURG-TLX subscore analysis (Table 1) revealed that task complexity was significantly higher for ENB compared with RB (mean, 4.14 vs. 3.71; P = 0.03; median, 4 vs. 3). Mental demand was also significantly lower for RB versus ENB (mean, 3.4 vs. 3.6; P = 0.03; median, 3 vs. 4). The other components of intrinsic CL (physical demands and temporal demands) showed no significant difference between the two groups.

Figure 1.

Categorization of cognitive load (CL) distribution. The percentage breakdown of each of the six components of CL is shown for (a) RB and (b) ENB. ENB = electromagnetic navigational bronchoscopy; RB = robotic-assisted bronchoscopy.

Figure 2.

SURG-TLX score for total cognitive workload in electromagnetic navigational bronchoscopy (ENB) versus robotic-assisted bronchoscopy (RB). The average cognitive workload per procedure is shown. ENB is shown in blue, whereas RB is shown in red. *P < 0.01. Whiskers indicate the highest and lowest data points within 1.5 interquartile range of the upper and lower quartiles, respectively. SURG-TLX = Surgery Task Load Index.

Table 1.

SURG-TLX average values for the intrinsic components of cognitive load

	Mean of SURG-TLX Score for ENB	Mean of SURG-TLX Score for RB	P Value
Mental demands	3.6	3.4	0.03
Physical demands	2.0	1.6	0.55
Temporal demands	2.5	2.7	0.64
Task complexity	4.1	3.7	0.03

Definition of abbreviations: ENB = electromagnetic navigational bronchoscopy; RB = robotic-assisted bronchoscopy; SURG-TLX = Surgery Task Load Index.

Average values for SURG-TLX for both ENB and RB. P values to detect a difference between ENB and RB are shown. Note that extrinsic SURG-TLX categories (situational stress and distractions) were removed because they were not the focus of the article.

Total procedure time (Figure 3A), measured from scope insertion to biopsy completion, was greater for ENB than for RB (mean, 6.7 ± 1.5 min vs. 4.4 ± 1.5 min; P = 0.01; median, 7 vs. 4). This difference was primarily due to longer navigation time (Figure 3B) to the target nodule for ENB compared with RB (mean, 5.9 ± 2.7 min vs. 3.1 ± 2.2 min; P < 0.01; median 6 vs. 2) biopsy time.

Figure 3.

(A) The total time to complete procedure was significantly lower in robotic-assisted bronchoscopy (RB) than electromagnetic navigational bronchoscopy (ENB) (*P = 0.01). (B) Time to successfully navigate to the target nodule was significantly lower in RB than ENB (*P < 0.01). Whiskers indicate the highest and lowest data points within 1.5 interquartile range of the upper and lower quartiles, respectively.

Total numbers of gaze deviations (RB vs. ENB) were as follows: mean 149.6 versus 172.0 (median, 150 vs. 172 for lesion 1), mean 198.3 versus 75.0 (median, 200 vs. 75) for lesion 2, mean 183.0 versus 129.81 (median, 82 vs. 88) for lesion 3, and mean 125.0 versus 107.75 (median, 126 vs. 105) for lesion 4. Lesion 1 RB had a slightly lower number of total screen changes (mean, 149.6 vs. 172.0; median 150 vs. 170). Adjusting for procedural time, the numbers are as follows (RB vs. ENB): 24.1 versus 18.1 for lesion 1, 27.3 versus 9.7 for lesion 2, 31.7 versus 17.4 for lesion 3, and 18.9 versus 9.0 for lesion 4, and this is depicted in Figure 4.

Figure 4.

The time-adjusted gaze deviations for each of the two modalities. ENB is shown in blue; RB is shown in green. ENB = electromagnetic navigational bronchoscopy; RB = robotic-assisted bronchoscopy.

Pupillary dilation (Figure 5) was measured in millimeters during the navigation to the lesion and the biopsy of the lesion. Pupillary dilation (right eye, left eye) during the navigation phase is as follows: RB (4.06, 3.96) and ENB (4.08, 3.95). Pupillary dilation during the biopsy phase is as follows: RB (4.30, 4.23) and ENB (4.25, 4.20). There was no significant difference in pupillary dilation.

Figure 5.

Pupillary dilation during different portions of the procedure. The x-axis shows the part of the procedure (localization of the lesion vs. biopsy of the lesion) and whether it was an RB or ENB. These are the mean values for the 17 procedures performed across 5 subjects. ENB = electromagnetic navigational bronchoscopy; RB = robotic-assisted bronchoscopy.

Discussion

The detection rates of peripheral pulmonary nodules (PPNs) have improved significantly over the past several years, resulting in an increased need for safe and efficacious diagnostic approaches (3, 4). Bronchoscopic techniques for biopsy have an excellent safety profile with low amounts of complications, but the diagnostic yield for PPNs ranges from 50% to 70% (10, 11), affording the opportunity for innovation and improved technology (5, 12). RB has shown promise in navigating to the periphery of the lung, but further study is required to determine its role in the diagnosis of PPN (2, 13). If early returns are substantiated, implementation of this technology will rely on novice and experienced bronchoscopists to learn RB and to discern whether existing skills are transferrable between ENB and RB.

The aim of this study was to provide insight into the challenges associated with learning the new technique of RB by comparing the CL associated with performance of both RB and ENB modalities in expert bronchoscopists with no previous RB experience. Through evaluating the associated CL, we may be able to better understand how to design improved curriculum to educate the next generation of RB operators. This has been demonstrated with various surgical procedures in which high amounts of CL have been associated with poor outcomes, and it is useful as a tool for expertise development (1, 6).

CL theory often divides itself into intrinsic and extrinsic components (1). Our interest was specifically intrinsic CL, which represents the working memory of the task (i.e., bronchoscopy). As tasks become overly complex, CL increases for a given individual’s ability level, causing worse performance and decreased learning (6). High levels of CL prevent conversion of working memory into long-term memory and can delay development of competence and expertise (6). Understanding the CL associated with components of bronchoscopy may help instructors personalize instructional design to the skill level of their trainee.

In our study, CL was measured using a validated tool (SURG-TLX) and biometric markers (pupillary dilation and gaze deviation) (9). The SURG-TLX measures both intrinsic and extrinsic CL. By nature of using the same cadaveric model, extrinsic CL was controlled across all operators and was therefore not the focus of this study. This controlled environment afforded us the ability to focus our analysis on factors innately associated with the learning process for advanced bronchoscopic techniques.

Using the SURG-TLX, task complexity was rated the largest component of CL by the operators, followed by mental demands, physical demands, and temporal demands. When evaluating each of the four components of intrinsic CL and comparing them across the modalities, the average SURG-TLX scores for task complexity and mental demands were significantly lower for RB than for ENB. Although task complexity and mental demands made up a larger proportion of the CL in RB than ENB, the total weighted CL SURG-TLX score was significantly lower for RB (69.25) than for ENB (101.25), suggesting that operators believed the robotic procedure was less mentally taxing than the navigational procedure. The lower task complexity associated with RB is consistent with previous surgical literature (7) and may be attributable to practitioners not having to constantly engage their own muscles to account for the location of the tip of the bronchoscope, instead relying on the robotic arm to maintain the position. There also could be an outcomes bias to this as well, as this educational study was nested within the PRECISION-1 (A Prospective Randomized Comparative Study of Three Guided Bronchoscopic Approaches for Investigating Pulmonary Nodules) study, in which RB had a 100% success rate of navigating to the peripheral lesion, whereas ENB was at 85% (5). We further acknowledge that the SURG-TLX is a subjective score that is potentially prone to an initial elevation bias that is then attenuated with repeated performance of procedures (14). To mitigate this bias, we used biometrics to obtain a more objective measure of CL.

Pupillary dilation has gained traction as a surrogate marker for CL (15–17). Pupils immediately contract or dilate in response to emotional and mental stimuli, with dilation occurring during times of heightened stress or during more complex tasks (18). The data in our cohort study show similar pupillary dilation across both modalities of bronchoscopy during the individual components (i.e., navigation) of the bronchoscopy. However, when comparing the navigation and biopsy phases, the pupillary dilation is higher during the biopsy phase (Figure 5) in both RB and ENB. The larger pupillary dilation during biopsy performance is likely attributable to this being the critical moment for diagnosis and having the potential to cause the most harm. Given the likely higher amount of cognitive stress, procedural training should recognize obtaining competency in biopsy technique. This also represents a time point at which working memory may overwhelm early learners who may not be able to process additional information (6, 19). Instructional design educators may consider breaking up the biopsy component to create bite-size learning appropriate to their trainees.

Another component of competency and expertise is gaze training. Randomized studies evaluating training novices to focus their gaze showed improved time to completion of tasks and less gaze deviation, suggesting that this is an indirect marker of competence and possible mastery of a procedure (20, 21). Although not statistically significant in our study, there was a trend toward increased screen changes per minute in RB when compared with ENB and a longer procedural biopsy time. Although this would seem to contradict the subjective SURG-TLX CL scores, the most likely explanation is the novel technology and lack of provider familiarity on where to focus his or her gaze. Further study into the gaze deviation of providers as they gain competency and expertise with RB is needed.

One of the barriers to the implementation of a new technique is often accounting for the time that will be lost while operators hone their skills. Although not bronchoscopy related, other studies have shown that robotic procedures tend to take longer when they are implemented (7, 22). This can lead to patients being exposed to longer periods of anesthesia, longer recovery times, less throughput in the endoscopy suite, and subsequently higher costs. This study deviates from that trend, given that the new technology (RB) had a significantly lower procedure time than the technology the providers were familiar with (ENB). The navigation portion of the bronchoscopy is when the discrepancy occurred, with RB taking significantly less time than ENB. This may be attributable to having a robotic arm maintain scope position and navigate as opposed to using one’s own muscles and may contribute to the lower associated subjective CL. Although this is a small study, navigation times to lesions were essentially halved in the RB portion, suggesting that the reduced physical demand on the operator allowed for quicker arrival to the lesion.

We do recognize the limitations of this study. First, using a cadaveric model removes significant extrinsic stress that is present in live patient procedures and may have impacted the subjective SURG-TLX scores. However, the scope of this study was to remove extrinsic components of CL and focus solely on procedural learning. In addition, because it was controlled across all operators, it should not have significantly altered overall CL scoring. We recognize that operators were highly experienced with advanced bronchoscopic techniques and that some of their existing skill set was likely transferred to RB despite this being their first use, as has been described in surgical literature (4). The results we describe may not be applicable to novice or intermediate learners, and future study into the CL of RB should include both expert and novice operators. We also recognize that although there are accepted metrics for measuring CL in the surgical realm, none of these have been validated in bronchoscopy. We do believe that there is some overlap between bronchoscopic and surgical procedural education, but a validation of these metrics in bronchoscopy would be important for further scientific exploration into interventional pulmonary education.

Intrinsic CL was evaluated in expert ENB proceduralists while they performed typical ENB and were exposed to a novel technology, RB. We were able to demonstrate that subjective intrinsic CL was reduced in RB despite the operators being unfamiliar with the technology. The objective data deviated from the subjective, with no change in pupillary dilation and higher average gaze deviation with RB. This is helpful to understand, as procedural training should target developing biopsy technique and gaze training. Furthermore, gaze deviation may be useful as a surrogate marker for procedural competency with bronchoscopy. We hope this study will motivate further exploration into evaluation of CL and the role it plays in bronchoscopic education.

Conclusions

Intrinsic CL of shape-sensing RB was highly manageable by existing ENB practitioners. Future development and training should focus on task complexity and mental demand for RB. Bronchoscopic education, regardless of modality, should focus on improving biopsy technique and potentially using gaze deviation as a surrogate marker for competence.