Systems Approaches to Surgical Quality and Safety

Abstract

Objective:

This approach provides the basis of our research program, which aims to expand operative assessment beyond patient factors and the technical skills of the surgeon; to extend assessment of surgical skills beyond bench models to the operating theater; to provide a basis for assessing interventions; and to provide a deeper understanding of surgical outcomes.

Summary Background Data:

Research into surgical outcomes has primarily focused on the role of patient pathophysiological risk factors and on the skills of the individual surgeon. However, this approach neglects a wide range of factors that have been found to be of important in achieving safe, high-quality performance in other high-risk environments. The outcome of surgery is also dependent on the quality of care received throughout the patient's stay in hospital and the performance of a considerable number of health professionals, all of whom are influenced by the environment in which they work.

Methods:

Drawing on the wider literature on safety and quality in healthcare, and recent papers on surgery, this article argues for a much wider assessment of factors that may be relevant to surgical outcome. In particular, we suggest the development of an “operation profile” to capture all the salient features of a surgical operation, including such factors as equipment design and use, communication, team coordination, factors affecting individual performance, and the working environment. Methods of assessing such factors are outlined, and ethical issues and other potential concerns are discussed.

Research into surgical outcomes has primarily focused on the role of patient risk factors and the skills of the individual surgeon. This article argues for a much wider assessment of factors that may be relevant to surgical outcome, such as equipment design and use, communication, team coordination, factors affecting individual performance, and the working environment.

Research into surgical outcomes has primarily focused on the role of patient pathophysiological risk factors, and on the skills of the individual surgeon. The outcome of surgery is, however, also dependent on the quality of care received throughout the patient's stay in hospital and the performance of a considerable number of health professionals, all of whom are influenced by the environment in which they work. Drawing on the wider literature on safety and quality in healthcare, and recent papers on surgery, this article argues for a much wider assessment of factors that may be relevant to surgical outcome. In particular we suggest the development of an “operation profile” to capture all the salient features of a surgical operation. The aims of this initiative are: to expand operative assessment beyond patient factors and the technical skills of the surgeon; to extend assessment of surgical skills beyond bench models to the operating theater; to provide a basis for assessing interventions and to provide a deeper understanding of surgical outcomes

COMPLICATIONS AND ADVERSE EVENTS IN SURGERY

There is certainly a need for a better understanding of the factors that influence surgical outcome and outcomes in healthcare generally. A number of studies around the world suggest that approximately 10% of patients admitted to the hospital suffer some kind of harm, about half of which is preventable with current standards of treatment.^1,2 Although the majority of these adverse events are minor, some lead to serious injury or death. A significant percentage of these adverse events are associated with a surgical procedure. For instance, in the Utah Colorado Medical Practice Study,¹ the annual incidence rate of adverse events among hospitalized patients who received an operation was 3.0%, of which half were preventable. Technique-related complications, wound infections, and postoperative bleeding produced nearly half of all surgical adverse events.

In the United Kingdom, complication rates for some of the major operations are 20–25% with an acceptable mortality of 5–10%.³ However at least 30–50% of major complications occurring in patients undergoing general surgical procedures are thought to be avoidable.⁴ The wide variation in surgical complication rates between different centers and different surgeons would support this view. Many adverse events classified as operative are, on closer examination, found to be due to problems in ward management rather than intraoperative care. For instance, Neale et al⁵ identified preventable pressure sores, chest infections, falls, poor care of urethral catheters in their study of adverse events, together with a variety of problems with the administration of drugs and intravenous fluids. The failure to remove swabs, needles, and instruments from a surgical site has for many years been a major problem in all fields of surgery. A swab or any other foreign body left in the body will result in considerable morbidity and even mortality.⁶ The costs of retreatment, additional surgical time, recovery, hospital stay, and subsequent litigation are considerable. Although there are strict protocols in the use of swabs and instruments in surgery, the process is exclusively human-led and involves a manual count of swabs, needles, and instruments at various stages of the procedure and a final count at the end. Despite this, diligence there is considerable opportunity for error.

THE EXPLANATION OF SURGICAL OUTCOMES

The primary determinants of surgical outcomes are generally held to be the patient's condition and the skills and performance of the individual surgeon. Some patient risk factors for anesthesia and surgery are generic, applying to any operation. Factors such as increased body mass index, advanced age, and the presence of comorbidity are associated with poorer outcomes and a higher risk for a range of complications.⁷ Risk factors for specific operations are also extensively studied, although it can be surprisingly difficult to identify a set of factors that consistently appears in different case series from different centers.⁸

The skills of the surgeon, and indeed of all members of the operative team, are an obvious prerequisite for a good outcome.⁹ Although it is clear that a certain level of skill is essential, it is more difficult to relate degrees of technical skill to outcomes. The introduction of laparoscopic cholecystectomy in the early nineties was a good example of the impact of technical skills on patient outcome. Although the overall complication rates were reduced, the incidence of major complication, such as bile duct injuries, significantly increased. Technical skills are very rarely assessed during actual operations whether for training or research purposes.¹⁰The primacy of technical skills is nevertheless the underlying assumption and is implicit in the creation of league tables or rankings of surgical performance.¹¹ Once outcomes (usually mortality) have been correctly adjusted for patient risk factors, the remaining variance is presumed to be explained by individual surgical skill (Fig. 1)

FIGURE 1. Explaining surgical outcomes.

However, this view of the primacy of patient factors and surgical skill neglects a wide range of factors that have been found to be of important in achieving safe, high-quality performance in other high-risk environments. These factors include ergonomic factors, such as the quality of interface design,¹² team coordination and leadership,¹³ organizational culture,¹⁴ and the quality of decision making.¹⁵ In safety-critical industries, these are the focus of the discipline of human factors, although we may anticipate a larger role for a number of basic sciences, particularly psychology, as the study of safety develops.

HUMAN FACTORS APPROACHES IN MEDICINE

Human factors is a discipline that spans engineering, cognitive psychology, and ergonomics and emerged specifically in response to the safety concerns of other high-risk industries. Although theoretically based, it has a resolutely practical emphasis, always aiming to bridge the gap between theory and application.¹⁶ Human factors thinking is now applied to healthcare in a variety of ways. Anesthetists have made important advances in safety through systematic incident monitoring and analysis, attention to design and ergonomic aspects of equipment, implementation of safety devices (such as the pulse oximeter), and attention to fatigue and cognitive overload.^17–19 Joice, Hanna and Cuschieri²⁰ have used human reliability analysis within endoscopic procedures. Mistakes and their precursors have been investigated in high hazard cardiac surgery²¹ and pharmacy.²² Human factors and ergonomics are also applied to the design of medical equipment to ensure it is efficient, effective and fail-safe.²³

Reason's organizational accident model, which integrated much of the available human error theory and human factors knowledge,^24,25 has been an influential general framework in healthcare. The Reason model has been adapted specifically for use in healthcare and a protocol produced to guide the investigation and analysis of clinical incidents.^26,27 An important lesson of these analyses is that serious adverse events and complications are often preceded by a chain of individually unimportant errors and problems, in turn influenced by a wide variety of contributory factors. This finding points to the importance of direct monitoring of these minor events in attempting to understand and prevent serious adverse events.

A particularly influential and important model of error in the medical environment has been developed by Helmreich and colleagues at the University of Texas Human Factors Project.²⁸ While influenced by the broad organizational approach of Reason and others, Helmreich's model has brought a new depth of approach through combining conceptual sophistication with hard-edged observational measures that have been applied in both aviation and medicine. Helmreich's model aids the identification of errors committed, incorporates error management strategies, and considers the impact of both immediate and systemic threats to patient safety. Immediate threats include such factors as fatigue, communication or patient related factors, such as a difficult intubation, while systemic threats concern organizational matters such as shift patterns and staffing.

A SYSTEMS APPROACH TO SURGICAL SAFETY

An operating theater is an extraordinarily complex system. The complexity is manifested not only in the patient and their condition but also in the sophistication of instrumentation, the high volume of information that must be processed, the nature of communication and team co-ordination, and the urgency and occasional uncertainty with which decisions and interventions must be made. This complexity, combined with heavy workloads, fatigue, and production pressures, makes surgical care particularly vulnerable to adverse events. Despite this vulnerability, most cases are performed proficiently and safely, highlighting the resilience of individuals and surgical teams to the potential adversity of the setting. This suggests that in addition to studying errors, it is crucial to also study the achievements of teams and how threats to safety are successfully managed.

In a recent paper Calland et al²⁹ have argued for a “systems approach to surgical safety,” pointing out that many other high-risk environments (such as aviation) have made effective use of systems approaches and studies of error in complex environments. Such approaches suggest that it is necessary to study all aspects of the system that comprises a surgical operation, including such issues as equipment design and use, communication, team coordination, factors affecting individual performance, and the working environment. A number of important studies on error and adverse outcomes have been conducted from a surgical perspective^20,30–32 but few have systematically addressed the full range of potentially important factors. Surgical adverse events may be due to poor communication, bad operative technique, malfunctioning or improperly used equipment, cognitive errors due to stress or inattention, all compounded by resource and organizational problems. However, these factors have been poorly studied in the field of surgery. Communication in the operating suite is often poor and may be implicated as a contributor to adverse outcomes.^13,33,34 The effects of fatigue and stress on performance have been well documented but not taken seriously in healthcare^35,36. Taffinder et al³⁷ found that sleep deprivation similar to that of being on a night on call for surgical trainees increased the error rate by 20% and increased the time taken by 14% in a simulated surgical task. Noise in theater, which can reach 85 Db,³⁸ can lead to deterioration in the ability to communicate, increase stress levels and affect complex motor skills.

Inexperience or lack of training also contributes to poor surgical outcome. A Canadian study³⁹ looked at variations in outcome between patients with rectal cancer treated by specialist versus nonspecialist colorectal surgeons, and independent of that, between results of surgeons with high versus low volume work. The analysis showed that the risk of local recurrence was increased and disease specific survival was lower in patients treated both by surgeons not trained in colorectal surgery and by surgeons performing less than 21 procedures during the study. Thus the best result was obtained from the trained surgeon performing a higher volume of work (10.4% recurrence, 67.3% survival) and the worst by the untrained surgeon with low numbers (44.6% recurrence, 39.3% survival). Birkmeyer et al⁴⁰ analyzed 14 types of procedures and demonstrated mortality decreased as volume increased with Begg et al⁴¹ demonstrating similar findings for morbidity rates in radical prostatectomy.

Prospective observational studies have been used to successfully understand the incidence and scope of adverse events. Observers attended a variety of case conferences, where adverse events were discussed and checklists were used to categorize error causation.⁴² Detailed observational and retrospective analyses of surgical failures associated with the neonatal arterial switch operation, has been performed by de Leval.^32,43 These studies show that analytical observational techniques can be successfully used to identify and analyze surgical performance successes, deviations and failures.

THE OPERATION PROFILE

The implication of the preceding line of argument is that if we want to fully understand and improve surgical performance and outcomes, we need to examine a much wider range of factors than hitherto. The operation profile is the term we have adopted to characterize the full range of factors implicated in the surgical outcome in the perioperative period.

The outcome of care is, broadly speaking, determined by the structural aspects of the system (personnel, environment, infrastructure) and the process (the procedures carried out the quality of care provided).^44,45 In the case of surgery, much of the infrastructure is relatively standardized, and will not be a major focus. The Operation Profile focuses on describing the process of perioperative care while acknowledging that events both before and after the operation may also play a major part. We first attempt to characterize the major operative events and audit the procedures carried out. In addition, following Reason^25,46 and Vincent,^26,27 we examine the major contributory factors to the quality and safety of care (Table 1). The higher-level influences (institutional context and organization and management) cannot be directly assessed, but the remaining 5 levels are all of considerable importance: patient factors, task factors, individual factors, team functioning and working environment. All of these are assessed to some degree in the Operation Profile (Table 2).

TABLE 1. Framework of Factors Influencing Clinical Practice

TABLE 2. Principal Features of the Operation Profile

The patient factors are the standard clinical information and risk factors. Basic operative procedures and processes are audited to detect salient events and omissions, with key operative events (such as blood loss) also recorded to supplement clinical information and also to give an indication of whether the conduct of the operation was routine. Finer grained analyses are made of specific intraoperative communications, and of the technical elements of the operation. Technical skills are assessed both by global ratings and assessment of specific errors. In both surgery and other high-risk domains, it is as yet unclear whether behavior of any kind is best assessed on scales or as specific ‘behavioral markers’²⁸ so both approaches are included. At present we are primarily focusing on surgical skills, though anesthetic and nursing assessments should shortly be included. The assessment of team performance in surgery is embryonic and a major task for the future is to develop team performance measures, which are both grounded in more general understanding of team dynamics, yet cognizant of the particular features of the operative environment. Finally, we assess a number of aspects of the immediate working environment, which are known to affect task performance in other contexts and so, potentially, in surgery.

In the present state of knowledge, we do not know which factors are most critical to outcome, beyond patient risk factors, basic levels of individual skill and the organization of care. The factors that lead to very high quality performance may differ from those that are required to achieve an average level of performance. For instance, surgeons who routinely achieve good outcomes in even adverse circumstances may be distinguished more by their preoperative and intraoperative clinical judgment than their technical skills. We suggest that, given this uncertainly, the initial measurement instruments should be broad in their focus. In time, in the course of research, we should be able to narrow our attention to a smaller number of key variables.

DATA CAPTURE METHODS

In the current practice of surgery the only data that are routinely collected for the record of an operation are the dictated, transcribed, subjective, post hoc report and pathology specimen.²⁹ However, it is now entirely feasible, through use of video and audio, to record the actions and movements of the surgical team. Ideally, the full operative sequence should be captured: preoperative care and consent, anesthesia, intraoperative care, handover, and ward care. Data collection for the operation profile will require a range of methods will be used, including video and audio recording, the use of participant observers, and brief postoperative questionnaires and interviews. Although operations obviously vary in duration, personnel, and technical aspects, it should be possible to apply a broadly similar methodology across different types of operations. This will enable potentially interesting comparisons of different case series. For instance, do communications patterns differ between routine, low risk operations and major surgery?

Different techniques will be applicable at different stages. During the operation, video and audio recording are the primary methods, providing DVD recordings for later review and analysis. However, this will need to be supplemented with the observations of research staff within theater, as a check on the validity of review of video recordings and to ascertain whether there are important features of the environment (eg, interruptions from other staff) that are not picked up by recordings of the operation. Handovers and other key points in the sequence can be observed and recorded using checklists of information. The role of postoperative interviews and reports from members of the surgical team remains to be explored. Clearly, verbal reports must form a major component of any investigation of decision-making or reaction to operative events and crises. However, de Leval and colleagues were surprised to discover that a questionnaire assessing aspects of team performance and the operative environment appeared not to be useful when completed postoperatively by members of the surgical team. Reports by a human factors researcher proved a much more important resource and a better predictor of surgical outcomes.³² It may be that absorption in the task of a difficult operation precludes a subsequent assessment of team performance. A surgeon may experience good or bad team performance, but may later find if difficult to remember and rate specific events with any precision.

Technological Innovations

A small number of surgical centers around the world now have the facility to routinely record operations, both during routine clinical procedures and during training and research exercises in simulators. The introduction of video-endoscopic surgery now allows video documentation of the operative procedure mainly for educational and training purposes. More recently technologies such as head mounted cameras have been used to record the operative field, eye movements can be tracked and hand motion analysis during operations may soon be possible. At Imperial College we have developed the Clinical Data Recorder, which uses 8 camera and 4 microphones. This provides:

• Visual and Audio recording of (1) the complete operative procedure and environment and (2) movements and communications within the theater

• Linked, real-time recording of physiological outputs, therapeutic equipment settings and performance. Personnel identification and motion tracking

The Clinical Data Recorder is paralleled by a simulated operating theater with exactly the same range of visual and auditory recording capability. A variety of high fidelity imaging technologies has been developed for training purposes. The full operating theater simulation allows the entire surgical team to train in specific skills and crisis management without risk to patients.

We therefore have the ability to record the complete operation sequence, providing objective data in the assessment of the performance of surgical teams and the monitoring of communication. In addition analysis of such information is growing in sophistication with the development of bespoke software by the University of Virginia. Guerlain et al's RASE system (Remote Analysis of the Surgical Environment)²⁹ allows real time coding of specific communications and interactions, and will in time evolve to code and capture a wider range of operative events and communications.

ETHICAL ISSUES AND PERSONAL CONCERNS

Although the observation of surgical teams and the associated technological innovations offer enormous potential for the understanding and improvement of the quality and safety of surgery, a number of ethical questions must be addressed.

Blame and Disciplinary Action

Fears will naturally be expressed by members of surgical teams that cameras and observation may be used for “surveillance,” checking up and possibly as a basis for disciplinary action. This is an important issue and there is a balance to be struck between the benefits of observation and unwarranted interference or personal intrusion. We have found it helpful to first point out that many parts of a hospital, for instance Accident and Emergency, are already observed by security cameras, as indeed are banks, supermarkets, and city streets. Many operating theaters already have a video link for teaching purposes. We have also stressed that cameras will only be used during operations and it will be clear when they are in use. Most importantly, it is necessary to emphasize that the purpose of such observations is not to study individuals, but processes, procedures and team performance generally. The aim is to observe common patterns over a series of operations, not to examine individual deficiencies.

Fear of Promoting Litigation

Fears are also sometimes expressed that cameras will encourage litigation. This is sometimes due to a misunderstanding, as some staff members believe that a patient can bring an action without the need of expert surgical opinion to back up their claims. However recordings, like all other medical records, are potentially open to scrutiny, even though we ensure that recordings that are made are anonymous, which makes them inaccessible within a short time. However it is perhaps more important to consider the use to which a recording might be put. Many legal actions are prolonged because of lack of good information and a recording might well be of great help in determining what actually did occur. Current risk management practice and patient safety initiatives are now encouraging openness and resolution of genuine grievances⁴⁷ and in these circumstances a recording would assist such rapid resolution. Conversely, if there has been no negligence, a recording will act to protect staff against unwarranted accusations and help to resolve misunderstandings. In the longer-term research aimed at enhancing safety and reducing adverse events should of course lead to a reduction in claims and complaints.

EVALUATION OF INTERVENTIONS AND LINKS TO OUTCOME

The focus of this work has been to delineate the nature and range of the many factors potentially relevant to surgical outcome and to outline an approach to measurement of these various factors. Reliable and valid measurement, while essential, is not an end in itself but a foundation for a number of longer-term goals.

Expanding Our Understanding of the Determinants of Surgical Outcome

Descriptive data on adherence to procedures and protocols, assessment of technical skills and team performance will be immediately informative. Obvious problems and weaknesses in systems can be addressed, whether they are equipment malfunctions, a tightening of procedures or rehearsal of particular skills. In the longer term, the operation profile should provide a more sophisticated understanding of the determinants of outcome than that currently available. Clearly patient factors are the most important determinant, and the fundamental component of any regression analyses that seek to determine the influence of different components. Understanding these more subtle influences of, say, team performance is likely to firstly require a subtler outcome measure than mortality. In high-risk cardiovascular surgery, de Leval et al³² made effective use of a defined category of “near miss” to broaden assessment of outcome beyond mortality. In more routine operations, where complications are rare, factors such as team performance may have to be linked to process measures such as duration of operation or conceivably physiological indices reflecting short term outcome for the patient, and acting as “surrogate” markers of longer-term outcome.

Evaluation of Interventions to Enhance Safety and Quality

The operation profile will provide an immense amount of information, but for some purposes this will need to be aggregated to produce broader indices of performance. An analogy might be made between the study of a surgical team and assessment of personality. In modern personality theory, the many facets of a person are reflected in 5 core dimensions of personality.⁴⁸We might similarly hope to produce dimensions on which operations could be assessed–technical skills, communication, team performance and so on. A great advantage of such a system, assuming it was based on a reliable and valid scoring system, is that interventions could be assessed for their immediate effect without necessarily being directly linked to outcome measures. An illustrative example is given in Figure 2, with pre- and postinterventions measures. In the first panel improvements in the operative environment are reflected in an increase in technical performance, and a reduction in technical errors, but no effect elsewhere. In the second panel, a preoperative briefing brings better communication and team performance without having any direct effect on technical skills.

FIGURE 2. Evaluating interventions.

Education and Training

A measurement technology developed in the operative environment would have clear face validity when used in training or research in a simulated environment. Simulator based training could in addition be targeted directly at known problems observed during operations, which would give it added power and validity. Crew resource management training now in its sixth generation of successive refinement has, after some loss of focus in the earlier years, acquired a more powerful and sharper focus on the management of specific threats and errors in the cockpit environment.⁴⁹ Similarly simulator training, in both anesthetics and surgery, is likely to achieve greater acceptance and be better focused if it is grounded in the observational data from actual operations rather than a more general attempt to enhance team performance. Some of these technologies will also add objectivity to the quality of training acquired and possible assessment of trainees through their surgical training ladder.

CONCLUSION

The direct observation and assessment of surgical teams will, we believe, greatly enhance our understanding of the key determinants of high performance in surgery and good clinical outcomes. Patient risk factors and surgical, anesthetic and nursing skill will always be fundamental, but more subtle factors may be of particular importance when aiming for very high levels of performance. To put it crudely good surgical skills coupled with basic team performance and basic equipment may enable a surgeon to achieve a 90% success rate in a high-risk operation. However refinements in surgical skill may be a relatively small element in the drive to reduce mortality from 10% to 1%. Optimizing the surgical environment, attention to ergonomics and equipment design, understanding the subtleties of decision making in a dynamic environment, enhancing communication and team performance may be more important than skill when is reaching for truly high performance. Poor team performance and inadequate equipment can cause anger and have a detrimental effect on performance. Conversely, an excellent team and a supportive environment will enable the surgeon to “raise his or her game” with a considerable benefit to the patient. Looking to the future, and the wider issues of patient safety, we believe that an open and comprehensive examination of the performance of surgical teams will be a powerful stimulus to safety culture and improvements in the wider healthcare system.