Improving Performance During Image-Guided Procedures

Abstract

Objective

Image-guided procedures have become a mainstay of modern health care. This article reviews how human operators process imaging data and use it to plan procedures and make intraprocedural decisions.

Methods

A series of models from human factors research, communication theory, and organizational learning were applied to the human-machine interface that occupies the center stage during image-guided procedures.

Results

Together, these models suggest several opportunities for improving performance as follows:

1. Performance will depend not only on the operator’s skill but also on the knowledge embedded in the imaging technology, available tools, and existing protocols.

2. Voluntary movements consist of planning and execution phases. Performance subscores should be developed that assess quality and efficiency during each phase. For procedures involving ionizing radiation (fluoroscopy and computed tomography), radiation metrics can be used to assess performance.

3. At a basic level, these procedures consist of advancing a tool to a specific location within a patient and using the tool. Paradigms from mapping and navigation should be applied to image-guided procedures.

4. Recording the content of the imaging system allows one to reconstruct the stimulus/response cycles that occur during image-guided procedures.

Conclusions

When compared with traditional “open” procedures, the technology used during image-guided procedures places an imaging system and long thin tools between the operator and the patient. Taking a step back and reexamining how information flows through an imaging system and how actions are conveyed through human-machine interfaces suggest that much can be learned from studying system failures. In the same way that flight data recorders revolutionized accident investigations in aviation, much could be learned from recording video data during image-guided procedures.

The continued growth of image-guided procedures warrants considering how they might be improved. These procedures have replaced a large number of open surgical procedures because they leverage more direct routes to the site of interest. Because these routes are chosen to minimize damage to the surrounding tissues, these procedures tend to cause lower perioperative and postoperative morbidity. The disadvantage of these procedures is that the human hand is replaced by long thin and often flexible tools that tend to provide less tactile and proprioceptive feedback. In addition, direct binocular visualization is replaced by imaging equipment that conveys data using a video monitor. As a result, health care workers performing these procedures rely on a more constrained data set to assess whether the procedure is proceeding as planned. Although technology will likely soon provide us with instruments that provide better tactile feedback and imaging equipment that offers 3-dimensional (3D) viewing, the increasing complexity of these procedures and their long learning curves will continue confounding improvement efforts.

Knowledge is the key to improved performance. According to Argote,¹ organizations store knowledge in people, processes, and technology (Eq. 1). Knowledge does not guarantee that the procedure produces the desired result; rather, item response theory contends that knowledge only increases the probability of that outcome (Eq. 2).^2,3 A highly skilled team working with state-of-the-art technology and following well-established protocols can still fail because of either chance events or causal factors such as patient-specific variables. The probability that any individual step in a procedure might fail is finite, and because these probabilities accumulate in a multistep procedure,⁴ we propose viewing image-guided procedures as dynamic systems in which preprocedure plans are frequently changed as new information becomes available.

As a result, outcomes will depend on the team’s ability not only to plan and execute a set procedure but also to effectively respond to information collected during the procedure. As this new information becomes available, the team must continually decide whether to retain, revise, or reject the baseline plan. Performance in such situations can be modeled as a dynamic, feedback-driven system,⁵ and we regard image-guided procedures with particular interest because the images generated during the procedure constitute a feedback channel. Further, those images can be collected, and an external observer can review those images not only to reconstruct the procedure but also to begin assessing organizational knowledge.

For example, consider a case in which a patient hemorrhages after ultrasound-guided central venous access. The human operator performs a series of voluntary movements during the procedure, and motor control theory suggests dividing each action into planning and execution phases.⁶ Planning is a cognitive process and includes evaluating the ultrasound image and determining whether the image provides the information needed to safely access the internal jugular vein (Fig. 1).

FIGURE 1.

Path planning process. The process begins with interpretation of the map image. The target is identified on the basis of predetermined criteria. The next obstacles are identified, and the result is an image that is segmented into target, obstacle, near obstacle (unsafe zone), and safe zones. A series of potential paths are generated. Finally, rules are used to select the optimal path.

Although these cognitive processes are not visible, we can begin to assess the quality and the efficiency of that cognitive work by observing the resulting motor actions. We will likely conclude that the operator is skilled if we observe evidence of careful planning and proficient execution.⁷ For ultrasound-guided central venous access, evidence of skill would include quickly moving the transducer to determine the relative positions of the target and surrounding obstacles, selecting a path that optimizes the procedure’s risk-benefit ratio, and positioning the transducer to effectively monitor the needle's course and distance to target during needle advancement. Further evidence of skill would include using ultrasound to repeatedly reassess the positions of the needle, target, and obstacles after needle insertion and efficiently advancing the needle to the target (Fig. 2).

FIGURE 2.

Control loop for ultrasound-guided procedures. Observations of simulated and patient procedures find that considerable time is spent searching for ultrasound images that provide the information needed to plan the course from the current needle tip position to the target. This model also assumes that the needle is advanced only when information regarding the needle's course and distance to target is available. US indicates ultrasound.

Poor planning (Table 1) and poor execution (Table 2) are manifested in various forms. Although both planning and execution are vital to successful outcomes, Reason^8,9described how poor planning can lead to more serious complications than poor execution (Table 3). He observed that good planning predicts that execution errors will occur. Good plans use sensors and error detection algorithms to trigger contingency plans. In contrast, planning errors are frequently unrecognized until catastrophe occurs.

TABLE 1.

Observable Manifestations of Poor Planning

TABLE 2.

Observable Manifestations of Poor Execution

TABLE 3.

Probability of Desired Outcome

Imaging technology plays a key role because it allows one to map the relative positions of targets and obstacles. Imaging also furnishes a feedback channel to monitor progress during the procedure. Previously, central venous access depended on palpating the carotid artery and guessing that the jugular vein was slightly anterior and lateral to the artery. The only feedback occurred when blood was aspirated. In this system, an arterial puncture rate of 1% to 2% was considered acceptable. Effective use of ultrasound has lowered that rate,¹⁰ but given the numerous steps required to extract effective feedback information from ultrasound images (Figs. 1, 2), it should be no surprise that performance in ultrasound-guided procedures obeys a learning curve.¹¹ To flatten this learning curve, systems that combine electromagnetic tracking with ultrasound imaging have been devised.¹²

Such systems highlight technology’s role in augmenting organizational knowledge. First, technology should be designed to provide the information needed to plan the tool’s path from the starting point to the target. Second, imaging technology should provide feedback information as the tool is advanced toward the target. Third, the information provided by the imaging technology should be conveyed in a manner that is clear to both novices and experts. Such technology will thereby minimize the performance difference between novice and expert operators. Achieving these goals requires understanding the human-machine interface and using that knowledge to anticipate not only what information the operator will need but also how it can best be communicated.

Models of Human Information Processing and Voluntary Movement

Guiding a tool to the desired location within a patient during an image-guided procedure follows the general paradigm shown in Figure 3. The human operator does not directly see either the current or the target position of the tool. Rather, those locations are inferred from interpretation of the data provided by the imaging equipment.¹² The imaging equipment provides data that are processed by the operator’s visual cortex, and the resulting information is delivered to working memory. In this model, planning is depicted as an interaction between working and long-term memory. The resulting decision about what is needed to move the tool to its target position is conveyed to the motor control cortex, which, in turn, generates neural signals that produce the desired hand movement.

FIGURE 3.

Information processing during an image-guided procedure. The operator views the current situation (stimulus) via an imaging system and manipulates it indirectly by applying forces to a tool. Adapted from Beta et al.¹³

Figure 3 illustrates how technology extends the capabilities of the human operators. Imaging technology allows us to “see” inside the patient. Tool technology provides instruments that are extensions of our hands and are inserted through openings that are much smaller than our fingers. Both types of technology improve over time because they are continually redesigned to address the shortcomings identified during prior work. Improvements in imaging and tool technology now make it easy for a novice to quickly achieve results that were once the exclusive domain of experts.

Imaging Data as an Informational Resource

Imaging data constitutes an informational resource that is used to plan and execute procedures. The equipment, electricity, and consumables needed to acquire the images and the operator time spent analyzing the images are all resources that are consumed during procedure planning and execution. Because computed tomography (CT) scanning and fluoroscopy are common imaging studies, the procedure’s balance sheet must include the long-term costs of exposing patients to ionizing radiation.

Balancing the Benefits Versus the Risks of Imaging Studies That Use Ionizing Radiation

Computed tomography, fluoroscopy, and other forms of medical imaging have revolutionized health care and led to a marked expansion of image-guided procedures. This growth has also markedly increased radiation exposure.^14,15 Mettler et al¹⁶ estimated that the growth of medical imaging since 1970 has more than doubled the per capita exposure. Exposing patients to increasing amounts of ionizing radiation has led to concerns about inducing future cancers.^17–22 Recent data demonstrate that, for susceptible populations such as children, CT scans are linked to a small but measurable increase in cancer risk.^21,23

These data emphasize the importance of continually improving the balance between the risks and benefits of image-guided procedures. Optimization strategies include increasing the benefits, lowering the risks, or a combination of the two. Fluoroscopic procedures are particularly interesting because radiation metrics are readily captured and can be used to estimate the radiation risk of the procedure. In addition, the operators have a particular interest in optimizing radiation use because they are typically standing alongside patients during the procedures.

Optimization means that the desire to minimize patient and occupational exposure must be balanced against the risks of reducing the dose to a point where the image fails to provide a reasonable probability of the desired result. For fluoroscopic procedures, the easiest method of reducing exposure is to disconnect the fluoroscope. However, the resulting lack of information would jeopardize our understanding of the target’s location and lead to blindly advancing the tool toward a poorly informed guess of the target’s location.

The notion of an uninformative image prompts considering how humans transform imaging data into information. Although data and information are commonly viewed as synonyms, they are distinctly different from an informatics perspective.¹⁶ For image-guided procedures, a single image is best considered a data array and a series of images are a data stream. The information needed to guide the procedure is encoded within spatial patterns of the data array and the temporal patterns of the data stream. The operator uses his/her knowledge of patient anatomy, physiology, and experience interpreting prior images/image sequences to decipher those patterns and transform the imaging data into the information needed to complete the task at hand. Stated another way, letters in a bowl of alphabet soup are data, and their arrangement into words and meaningful phrases conveys information.¹⁶ In the same way that meaning in written language is sensitive to the preceding and following sequences of letters (i.e., context) and the prior knowledge of the reader about the encoding principles (i.e., familiarity with the English or Japanese alphabets), information communicated through imaging technology depends on the sequence of images and the operator’s knowledge of context.

This distinction between data and information becomes important when trying to improve the efficiency of the imaging systems. Both increases in spatial or temporal resolution provide more data, but they do not necessarily convey more information. High frame rates are not needed to understand a static process, nor do high spatial resolutions improve our interpretation of a featureless object. Although high spatial and temporal resolution can clearly improve our ability to understand complex and highly dynamic phenomena, we can usually make informed decisions with far fewer data (Fig. 4).

FIGURE 4.

The spatial resolution of an iconic image was modified to illustrate how content and prior experience influence image interpretation. Almost anyone will recognize all 4 images as a human face. A child who has not encountered this image would likely need panel C to describe an elderly man sticking out his tongue. Anyone who has seen this iconic image before will likely need only panel B before responding “Albert Einstein sticking out his tongue.” Although panel C requires 4-fold fewer bits to convey, it is virtually indistinguishable from panel D. Regrettably, default settings on fluoroscopes typically provide images analogous to panel D because emphasizing image detail has long been a marketing strategy for imaging equipment.

Optimizing Performance When Planning Any/Every Step

Planning an image-guided procedure means that the operator must choose what data sources will best provide the information needed to successfully complete the procedure. Direct visualization of internal surfaces by endoscopy and laparoscopy is a common guidance strategy, as is tissue transillumination by fluoroscopy. Cross-sectional techniques such as ultrasound, CT, and magnetic resonance imaging are commonly used for biopsies and other procedures because they provide a detailed 3D map of a body region that depicts the target and obstacles. The cost/benefit of these cross-sectional techniques is being shifted by the introduction of tools equipped with sensors that convey their location in 3D electromagnetic fields.¹² By combining the data sets, one can incorporate navigational algorithms into guidance system. The goal of such systems is to combine the data residing within previously obtained imaging studies with a smaller current data set. The combination provides the operator with sufficient data to plan and execute the procedure. For example, ultrasound, CT, and positron emission tomography imaging can be combined to allow one to see the most metabolically active area of a tumor on a CT image, select the best path toward this lesion, and then use ultrasound to provide feedback while advancing the neede.¹⁷ The vascular anatomy depicted on an MR or CT angiography could potentially be combined with the fluoroscopic image to obviate the need for the nonselective angiograms. Combining prior imaging information with images obtained during the procedure is becoming more commonplace. It is wasteful to continually reacquire large comprehensive data sets when our decisions are governed by local changes over time.

Planning for Problems That Might Occur When Executing the Planned Action

Accurate navigation requires maps that possess sufficient spatial resolution and signal-noise ratios so that the operator can reliably distinguish target from obstacle. In addition, the map must also accurately convey information that allows the operator to correctly categorize the potential obstacles along the planned path. Endoluminal paths may contain obstructions and stenoses. Needle paths that traverse tissue planes must be examined for structures that would lead to potential complications. This perspective suggests that mapping inaccuracies because of poor resolution or insufficient signal-noise ratios will degrade performance. Further, because maps represent previously acquired data, temporal changes create additional problems.

Procedure planning must also allow for problems with execution. A plan that requires “threading the needle” is far more likely to fail than a plan that tolerates navigation errors. When planning a procedure, one must consider the frequency and the severity of deviations from the planned path (Fig. 5). Some obstacles constitute less of a penalty. Thus, the operator must possess knowledge of the penalties associated with traversing each type of obstacle and incorporate this knowledge into the planning process (Fig. 6).

FIGURE 5.

Red circles denote obstacles. The green circle denotes the target. Accounting for the operator's experience is required for optimal planning. For a novice (top), the greater uncertainty in execution results in only 1 favored path. Experienced operators (bottom) are expected to demonstrate less variation and thus can choose between 2 acceptable paths.

FIGURE 6.

Red circles denote severe obstacle. Yellow circle denotes moderate obstacle. Green circle denotes target. Including the severity of encountering an obstacle in the planning process leads to a more realistic outcome.

Measuring Improvement in Image-Guided Procedures

Improving system performance requires objective assessment of current performance. Subscores can also be calculated for planning and execution phases. Improving performance during each phase can be achieved by improving quality, increasing efficiency, or optimizing the balance between quality and efficiency. Efficiency is relatively easy to measure because one can readily sum the resources consumed during planning (e.g., time spent or radiation used to create or review images both before and during the procedure) or execution (e.g., time spent maneuvering tools, number and costs of the tools used).¹⁸ Quality is more difficult to define, but other industries define quality in terms of variation.^19–21 Accordingly, we suggest measuring quality of planning and execution by assessing deviation from prediction (Table 4).

TABLE 4.

Assessing Performance During Planning and Execution

Some may argue that defining execution quality as conformance to plan will penalize the highly skilled operator who frequently adjusts his/her plan to accommodate patient-specific factors. Such arguments are true only if we restrict our analysis to linear plans. Compared with branching systems, linear systems are far less reliable.⁴ As discussed before, good plans are ready to accommodate variation in patient-specific factors and include the necessary branch points. These branch points fit well into our scheme because observable criteria and criteria-specific plans are combined to yield a list of possible plans. Such branching schemes provide flexibility, but they also provide new opportunities for error.

Improving Performance During Image-Guided Procedures (Summary and Future Directions)

We have described how a successful image-guided procedure depends on multiple factors. Included in these successes are careful preprocedural planning, accurate analysis of image information, rigorous adherence to the planned execution protocol, and flexible response to intraprocedural contingencies. Success will require combining visual information acquired via various imaging technologies with tactile feedback and other sensory cues. Optimal allocation of resources was considered, and quality was defined as deviation from prediction. Improving quality will require analyzing the differences between observed and predicted to determine what factors were the most likely causes for any failed prediction. Such investigations require data, and we suggest that the same series of images that were used to plan and execute the procedure should be recorded to facilitate after-action reviews. We agree with Denham et al that health care still has much to learn from high-reliability organizations such as aviation and nuclear power generation.²⁴ Digital video and audio recorders can serve as “black boxes” that routinely record performance data during image-guided procedures.²⁵ In the same way that black box data are being used to improve the quality of routine flight operations,²⁶ algorithms based on statistical process control can be applied to image-guided procedures.^27–29 Routine analysis of “successful” procedures will provide insights that allow incremental improvement.³⁰ Innovative leaps will require detailed analysis of focused data sets and should be modeled along the processes used by the National Transportation Safety Board to investigate events and recommend systematic improvements.²⁴

One concern about recording procedures is the cost of installing and using the recording systems. An ongoing pilot project has demonstrated marked and sustained improvement in team performance during the preprocedure time out.³¹ Secondary benefits of recording procedures included establishing a culture of safety and optimizing radiation exposure.^32,33 The general need to improve time-out performance arises from their frequency and the requirement that all hospitals develop a procedure for routinely auditing time-out performance. Wrong site/wrong patient procedures continue and generate substantial costs for hospitals. The cost-effectiveness of recording procedures will depend on error frequency and severity as well as the efficiency and effectiveness of corrective action and feedback. Given the rapidly decreasing costs of recording and the known utility in other complex human endeavors such as aviation and athletics, we expect that recording of medical procedures will become increasingly commonplace.^34,35 Not only will recording, analysis, and feedback drive improvements during routine procedures, we believe that those recordings will improve investigations of untoward events.

Consider again the case in which a patient hemorrhages after ultrasound-guided central venous access. At present, the ensuing investigation relies on human memory, a few still images, and written notes in the medical record to reconstruct the event. As a result, we have difficulty discerning whether the bleeding resulted from the patient’s underlying physiology or operator errors such as image misinterpretation, poor target selection, or execution errors when advancing the needle tip to the target point. A recording system with multiple inputs could devote the first input to capturing the ultrasound images, a second external camera to record the operator hand movements, and, potentially, a third to capture the location of specialized tools (such as electromagnetic tracked tools, if used).^12,17,22 Such a data set would allow investigators to reconstruct the procedure and pinpoint what portion of the feedback-controlled loop broke down. Such a detailed analysis would help determine the failure modes for image-guided procedures and provide targets for improved technology, planning, and execution.

We suggest that detailed analysis of performance during image-guided procedures offers a unique opportunity to study the linkages between diagnostic decisions and treatment execution. We believe that these procedures can also provide objective data on human-machine interactions and the crew resource management issues confronting health care.