Quality of Complication Reporting in the Surgical Literature

Abstract

Objective

To identify 10 critical elements of accurate and comprehensive reports of surgical complications.

Summary Background Data

Despite a venerable tradition of weekly morbidity and mortality conferences, inconsistent complication reporting is common in the surgical literature.

Methods

An analysis of articles reporting short-term outcomes after pancreatectomy, esophagectomy, and hepatectomy was performed. Randomized clinical trials (RCTs) published from 1975 to 2001 and retrospective series of more than 100 patients published from 1990 to 2001 were reviewed.

Results

A total of 119 articles reporting outcomes in 22,530 patients were analyzed. This included 42 RCTs and 77 retrospective series. Of the 10 criteria developed, no articles met all criteria; 2% met 9 criteria, 38% 7 or 8, 34% 5 or 6, 40% 3 or 4, and 12% 1 or 2. Outpatient information (22% of articles), definitions of complications provided (34% of articles), severity grade used (20% of articles), and risk factors included in analysis (29% of articles) were the most commonly unmet quality reporting criteria. Type of study (RCT vs. retrospective), site of institution (U.S. vs. non-U.S.) and journal (U.S. vs. non-U.S.) did not influence the quality of complication reporting.

Conclusions

Short-term surgical outcomes are routinely included in the data reported in the surgical literature. This is often used to show improvements over time or to assess the impact of therapeutic changes on patient outcome. The inconsistency of reporting and the lack of accepted principles of accrual, display, and analysis of complication data argue strongly for the creation and generalized use of standards for reporting this information.

Although an array of short-term postoperative outcomes such as operative time, estimated blood loss, blood transfusion, length of hospital stay, time to return to work, or hospital charges have been reported, death and complication rates remain the most frequently measured and reported endpoints. With their preeminence as surgical outcome measures, they are often the sole data provided as a means of comparing surgical techniques or perioperative management decisions. Complication reports within both hospitals and the medical literature thus deserve consistency and clarity in reporting.

There have been more than 11 million articles annotated in the Index Medicus since 1966, and better than 8,000 new publications are added each week. With this explosion of publications in the medical literature, variability in the quality of the reports inevitably occurs. The ability of the physician to discern these differences in quality requires considerable time, expertise, and insight. This places an increasing burden of responsibility on the journal editors and reviewers of submitted articles to make consistent recommendations to potential authors that will ensure that improved reports meeting minimal standards will be published. This is particularly important for those endpoints, such as surgical complications, that are so frequently portrayed as the prime reason for making changes in patient management.

This study was designed to critically evaluate the quality of the surgical literature as it relates to the reporting of complications. Selected articles are assessed for compliance with 10 criteria that judge this quality.

METHODS

A convenience sample of the literature pertaining to three major surgical procedures (pancreatectomy, hepatectomy, and esophagectomy) was reviewed. All prospective randomized clinical trials (PRCTs) published from 1975 to 2001 and all retrospective articles with study populations of 100 or more patients published from 1990 to 2001 were included in this analysis. To identify these reports, Medline, Embase (Excerpta Medica), and PreMedline databases were searched incorporating MeSH headings for subject, disease, and operative terms as reported in the English literature.

Articles were reviewed and judged on 10 criteria relating to completeness of reporting surgical complications (Table 1). Specific complication definitions, number of unique complications, methods of risk stratification, causes of death, and indications for return to operating room were also recorded for comparison across studies.

Table 1. CRITERIA FOR ARTICLE EVALUATION

RESULTS

One hundred nineteen articles reporting outcomes in 22,530 patients were analyzed. These included 40 studies of pancreatectomy, ^1–40 35 esophagectomy articles, ^41–75 and 44 pertaining to hepatectomy. ^76–119 A total of 42 PRCTs and 77 retrospective series were analyzed. Thirty-three reports were conducted in the United States, but 86 were published in American journals.

No article met all reporting criteria, although two studies scored 9 out of the available 10. The median score was 5 and the mean score 6, with the distribution shown in Table 2. The number of criteria met was comparable across all three surgical procedures reviewed. PRCTs were no more thorough in reporting complication data but had higher-quality reports when compared with retrospective studies (26% vs. 31% scoring 7–9). Country of origin similarly did not influence the probability of higher-quality reporting, with scores of 7 to 9 seen in 34% of U.S. studies compared with 26% in reports from outside the U.S. American journals published articles of similar quality, with 31% of reports scoring 7 to 9 compared with 30% of the non-U.S. journal publications.

Table 2. TOTAL COMPLICATION CRITERIA MET DISTRIBUTED BY OPERATION

Specific reporting criteria are shown and compliance rates are reported in Table 3 for each of the operations and as a total. The only criterion that was universally met was the method of data accrual, which distinguished PRCTs from retrospective series. The duration of follow-up, reported in 67% of studies, was most commonly a 30-day minimum or duration of initial hospitalization, whichever was greater, and was used in 33 studies (28%). The duration of review of postoperative morbidity and mortality was the initial hospitalization period in 23 (19%) and 30 postoperative days in 15 (13%). Other study periods used included 60, 90, and 120 days. Outpatient data were acquired in 26 studies (22%), primarily esophageal stricture, which was encountered and commonly managed with dilation in the outpatient setting.

Table 3. SPECIFIC COMPLICATION REPORTING CRITERIA MET DISTRIBUTED BY OPERATION

If at least one definition of a complication was included in the report, the criterion for providing definitions was met. This occurred in 41 studies (34%) and was more common for those reporting complications after pancreatectomy (75%) than esophagectomy (11%) or hepatectomy (16%). In reports of pancreatectomy, nearly half provided definitions for pancreatic leak or fistula and delayed gastric emptying. Variability among definitions, though, was also most pronounced with these two diagnoses: 12 versions of pancreatic leak or fistula were offered, along with seven different descriptions of delayed gastric emptying (Tables 4 and 5). Definitions of anastomotic leak, pneumonia, or respiratory failure after esophagectomy or liver failure after hepatectomy were distinctly uncommon.

Table 4. EXAMPLES OF COMPLICATION DEFINITIONS BY OPERATION

Table 5. EXAMPLES OF COMPLICATION DEFINITIONS BY PROCEDURE

In these studies, 969 deaths were recorded, with 98% of the articles reporting death rates. These were the most consistent data related to complications that were reported in these 119 articles. The specific complication leading to death was reported in 70% of articles; those 520 causes are listed in Table 6. The most common cause of death after pancreatectomy was pancreatic leak (27%), pneumonia (37%) after esophagectomy, and liver failure (30%) after hepatectomy. Cardiac-related complications were a common cause of reported death across all three operations, with hemorrhage being more common after pancreatectomy (13%) and hepatectomy (18%) than after esophagectomy (2%).

Table 6. OVERALL MORTALITY REPORTED BY OPERATION

“Other” denotes all the other causes of perioperative deaths that were less than 2% of the overall mortality. n/a = not applicable to disease type, and thus not reported.

There were 7,342 patients with at least one complication among the 22,530 included in this report (33% complication rate). These patients sustained 9,157 complications, or 1.2 per patient. All pancreatectomy and esophagectomy studies reported patients with complications and total complications, but hepatectomy reviews were complete for this reporting criteria in only 30 of 44 articles (68%) (Table 7). These complications are shown in Table 8, which highlights both the median occurrence rate per complication and the considerable range among reports. Studies listed complication type 87% of the time, with a median of eight types reported (range 1–33).

Table 7. MEDIAN AND RANGE OF MORBIDITY AND MORTALITY RATES REPORTED, DISTRIBUTED BY OPERATION

Table 8. SPECIFIC COMPLICATION RATES REPORTED AS MEDIAN AND RANGE OF RATES REPORTED

n/a = not applicable to operation and thus not reported.

Procedure-specific complications after pancreatectomy included pancreatic or biliary fistula or leak and delayed gastric emptying. These were specifically recorded in 29 studies (72%). Esophagectomy reports included pneumonia, anastomotic leak, and atrial arrhythmia in 18 studies (51%), and hepatectomy reviews covered the procedure-specific complications of liver failure, intraabdominal abscess, and bile leak in 21 (66%). Return to operating room did not represent a reporting quality criterion in this analysis but was included in the reports of 33% of the articles, characteristically with specific indications for reoperation, principally bleeding, infection, and wound complications.

Severity grading of complications was reported in 24 of the 119 studies (20%). This was most commonly limited to undefined major or minor categorizations, although segregation of data into surgical and nonsurgical, surgical and general, or early and late categories was used by one researcher for each of these versions of severity grading. A numeric grading system was not used in any study.

Half of the studies (59 studies) reported length-of-stay information, and 20 of these included both median and range data as a description of postoperative course. The inclusion of cost data in these reports, another surrogate for complexity of hospitalization, was not a portion of the quality criteria scoring system but was also not included in any of the studies reviewed.

Risk factors were used to describe the severity of comorbid conditions in 29% of articles. These were principally ASA class (10 reports) or a variety of unvalidated general descriptions of age and comorbid diseases. Hepatectomy articles used Child criteria or indocyanine green clearance to describe hepatic function as a risk stratification method in 10 of 44 reports reviewed.

DISCUSSION

Despite a venerable tradition of weekly morbidity and mortality conferences, inconsistent complication reporting is common in hospitals ^120–122 and in the surgical literature. ¹²³ Incomplete records, multiple sites of postoperative care, medicolegal concerns regarding the documentation of patient safety issues, and worry over public disclosure of data often hinder the accurate portrayal of the postoperative course and consequent tabulation of data. Rates of postoperative death and reoperation were the only parameters consistently collected across 21 centers surveyed in the United Kingdom conducted to provide a comparison of hospital outcomes for patients undergoing pancreatic resection. ¹²⁴ Such comparisons are further frustrated by inadequate risk factor adjustments, highlighted in a series of reports from the Veterans Administration’s National Surgical Quality Improvement Study. ^125,126

Medication-related toxicity and its completeness of safety reporting in randomized trials have been critically analyzed. ^127,128 In this survey of 192 randomized drug trials with sample sizes of at least 100 patients, severity of clinical adverse effects was adequately reported in only 39% of trial reports. Only 46% of trials stated the frequency of specific reasons for discontinuation of study treatment due to toxicity. The allocation of article space to safety reporting was also studied and revealed a median of 0.3 pages of safety data per report.

The revised CONSORT statement for reporting randomized trials provides 22 checklist items with descriptors to assist in the accurate description of these critical data in publications. ¹²⁹ Use of these guidelines allows the readers to fully understand the design, conduct, and analysis of the trial as they judge its value before adopting recommendations for clinical care of patients. Guidelines for reporting results of each primary and secondary outcome are included but lack specific recommendations for clarity of reporting safety outcomes.

To gauge the breadth and depth of these limitations in the reporting of morbidity and mortality data in the surgical literature, 10 criteria were established to judge the quality of complication reporting. The variability in publications is the most obvious feature that emerged in this analysis. No study provided ideal reporting of complication data, and only one third met at least 7 of the 10 reporting criteria. Each of these higher-scoring reports, however, lacked clarity and ease of comparison with other studies because we lack an accepted vocabulary and standard for such reports.

The prospective or retrospective methods used to accrue the data are nearly always evident in the “methods” section of the article. A standard for duration of follow-up of postoperative patients has not been established, although a 30-day limit for death is most frequently a component of the time period of interest. Although convenient to have a set number of days as an endpoint, this choice may not fully or accurately depict the outcome, principally as it relates to prolonged intensive care unit care or death beyond 30 days. The additional inclusion of complications occurring during the initial hospital stay addresses this concern but was used in only 28% of reports. A review of 103 gastrectomy articles published from 1970 to 1990 identified a similar inconsistency of postoperative death: 48 included 30-day deaths only, 14 included all hospital deaths, 31 did not define the time period of interest, and 10 provided six other definitions of duration of the study. ¹³⁰

Particularly with the reduction of length of hospital stay, complications may now be initially encountered in the outpatient setting. Those that require readmission during the initial 30 days may be captured in the inpatient records and included in reports. Readmission to a different hospital or care of a complication exclusively in an outpatient environment will likely go unreported. Wound infection is often prone to this phenomenon, with as much as a 53% increase in infection rates reported by complete 30-day inpatient and outpatient reporting. ¹³¹

The greatest variability observed in this assessment can be traced to the lack of consistent or standardized definitions for specific complications. Although many definitions would be held to be common knowledge not requiring codification, all types of complications would benefit from this clarification to permit consistent abstracting of data by researchers across the array of technical training reviewing hospital charts for data accrual. The National Cancer Institute has produced and promulgated the Common Toxicity Criteria, providing definitions and grading criteria for 310 types of complications within 24 categories. This is currently undergoing a third update and is widely used by research assistants and principal investigators to accurately record medication toxicity encountered in randomized clinical trials. The RTOG/EORTC late toxicity criteria similarly provide clear standardization of greater-than-90-day outcomes after radiation therapy.

A recent and detailed review of anastomotic leak after gastrointestinal surgery studied 97 reports from 1993 to 1999 that provided definitions of upper gastrointestinal, hepatobiliary, or lower gastrointestinal anastomotic leaks. ¹²³ The 56 definitions analyzed in this review highlight the variability in severity classification, inclusion of radiologic or clinical criteria, and the timing of any routine radiologic investigations. The authors concluded that these variations preclude accurate comparison of rates between studies and institutions.

Death rates were readily identified in the articles studied in this review, with 70% clearly defining both the number of patients dying and the causes of death. Ideally, inclusion of specific causes of death provides the reader with a precise understanding of the procedure-related events. When multiple organ failure was the immediate cause of death, the triggering or critical event, often procedure-related, was relayed in the most thorough reports.

Complication rates were similarly easily derived from the data reported. Total complications were also evident in 88% of studies, although separating the number of patients from the number of complications was often difficult to calculate. The easy identification of these data invites comparison between studies but fails to address other concerns noted in this quality analysis. The critical and procedure-specific complications are less commonly reported (57%), although each operation certainly has unique complications that deserve consistent inclusion in all reports. Pancreatic fistula or leak, delayed gastric emptying, and bile leak are examples of such complications that are encountered after pancreatectomy. With 72% of reports of this procedure including these important complications, further procedure-specific standards of reporting would aid in these desirable comparisons. The considerable range of reported occurrence rates for specific complications might suggest that true differences exist among institutions as they relate to a surgical technique, patient selection, or postoperative care. The level of confidence with this conclusion must be challenged in view of the variations in the quality of these reports.

Severity grading was used in less than one third of cases, although in all instances these were simply divided into minor or major categories, with definitions for inclusion in each of these similarly infrequently provided. A five-level grading system is available for chemotherapy- and radiation-related toxicity, with grade 5 representing death. The use of a four-level severity grading and classification system in surgery has been provided in a report by Clavien et al. ¹³² In this system, grade 1 complications are minor and are likely to resolve spontaneously. Grade 2 complications are potentially life-threatening and usually require some form of intervention. Subsets of this include 2a, requiring medications for treatment; 2b implies the need for reoperation or an invasive procedure. Grade 3 complications have lasting disability or require an organ resection. All complications in this classification are categorized as cardiac, respiratory, gastrointestinal, urinary, local, or “other.” No specific complication definitions are provided. This system of reporting was modified and served as a means of judging the completeness of morbidity and mortality reporting in conference. ¹²¹ Although death (grade 4) complications were reported 88% of the time at conference, grade 2 or 3 complications were included only 33% of the time.

A surrogate short-term outcome endpoint is length of stay. This was included in 50% of the articles and only occasionally was causally related to the severity of complications. It should not serve as a substitute for the clarity provided by complete disclosure of complication information. There was similarly uncommon use of risk stratification in these reports to place the complications in context with patient comorbidity. When used (29% of reports), they were characteristically ASA status, although articles describing hepatectomy used Child criteria or indocyanine green clearance to describe preoperative liver function.

Appropriate criticism of the reporting of randomized clinical trials has led to the development of the CONSORT statement, which provides a framework for such reports. Surgeons must participate in the definition and quality control of surgical procedures in randomized trials where surgical performance is a potential factor in outcome. With the emergence of surgically initiated clinical trials, it will become mandatory that standardized complication reporting criteria be established for specific procedures. Based on this report and others, it is entirely appropriate that comparable guidelines be developed for the standardization of the reporting of surgical morbidity and mortality data as we provide solutions to the identified problem. This data quality imperative is even more important as we inevitably migrate from elective publication in journals to mandatory reporting to oversight agencies.

Discussion

Dr. R. Scott Jones (Charlottesville, VA): First I want to compliment Drs. Jaques, Martin, and Brennan for this very excellent revelation. I had really never thought of this particular issue in the way they have presented it. Their presentation and manuscript are important contributions. I would like to make a couple of comments and ask a question.

My first comment is probably obvious, and everyone is thinking this: what gets into the Journal is generated by surgeons and surgical investigators. The first message here is that we need to improve our discipline and quantification of the outcomes of our work. And as we have already talked about, the opportunities for doing that are available. So I think the first step in correcting what goes into the Journal will be to examine with great rigor the clinical data that we collect. The second comment, which is also obvious, is that we need to work together. And in terms of talking about editorial things, I will have to say that we have met the enemy and it is us. As I look about the room, I see a lot of people who are on editorial boards and who are responsible for editing articles, refereeing articles, and, to some extent, responsible for editorial policy. And I think the authors have given the editorial group a wake-up call. And I will have to say that I would certainly welcome the introduction of their ten-point protocol into the editorial process. Protocols are used for other things such as experimental design or trial design. We should think a bit about guidelines for reporting morbidity and mortality.

This study period spanned a number of years. And I think the awareness of all of us has been increasing progressively, particularly in the last decade, about reporting morbidity and mortality figures. And my question is, in your examination of this data, have you noticed improvement as a variable with time? In other words, if you had looked at the percentage of data included, say, 20 years ago, was it a certain level? And in the last 5 years are we improving? It would be interesting to see if time and discussions that we have had have improved this.

And lastly, I will just close by saying again I think this is very interesting and important work. And I think just by doing the study, you have helped all of us take a look at what we do.

Dr. Layton F. Rikkers (Madison, WI): This excellent and well-presented study by Dr. Brennan and his colleagues is a timely and important contribution to the surgical literature. The Institute of Medicine’s recent report gained headlines in our nation’s newspapers and mobilized the public’s concerns regarding medical errors and complications of medical and surgical therapies. At the same time, our certifying boards, including the American Board of Surgery, are looking for ways to better measure competence of individual physicians and surgeons. The bottom-line finding of this particular study is that we appear to be a long way from being able to do so. Even when analyzed within the framework of a clinical investigation, complications of surgery are incompletely and inconsistently reported. Quite surprisingly, we apparently do not do any better when data are collected prospectively rather than retrospectively. From my perspective as a journal editor who reviews a large number of manuscripts, I fully agree with the author’s findings and conclusions. Complications are often poorly defined, and they are rarely graded. This confounds the comparison of various reports and attempts of doing meta-analyses. The criteria developed by the authors for analyzing clinical trials with respect to the reporting of mortality and morbidity data should be useful. Each year the editors of several surgical journals meet at the American College of Surgeons meeting. This group has become quite active and recently developed a statement regarding duplicate publication, which was published in several surgical journals in June 2001. I think the template put forth as criteria for reporting surgical morbidity in this investigation, or some modification of it, could be considered by that group for adoption as guidelines for both reviewers of manuscripts and for authors. Probably the more important issue, however, is how mechanisms can be put in place in all of our hospitals to prospectively accumulate such data for the purpose of improving quality of care. This data would be available for analysis in either retrospective or prospective trials. Just as complications are avoided by meticulous surgical technique, the consistent and accurate reporting of them when they do occur will require carefully constructed databases run by professionals. This will be quite expensive. Who should pay for such databases, especially when they are directed towards not just publication but also towards quality of care? Who will be responsible for developing the definitions and grading of complications for each surgical specialty? Is this a function for the surgical specialty societies, or for consensus conferences? Finally, how can physician resistance to the careful recording of the complications be overcome? Who should be able to access the information once it has been organized into a database?

Dr. Hiram C. Polk, jr. (Louisville, KY): I rise to speak to the issue of the difficulty in getting primary reporting of accurate complication rates in surgery. Quality Surgical Solutions is a network of 70 surgical specialists working in part of Kentucky and southern Indiana that consists of eight different specialties working under 40 different protocols in eight hospitals in six different cities. As of last Tuesday we had access to 12,062 patients. These patients are interesting in that they are easier than most of what you are used to listening to because they are all elective operations, with the exception of acute appendicitis and acute cholecystitis. The outcome of our ongoing study speaks to several issues that have come up this morning. First of all, the cost of data acquisition is very real. We insist that every report is completed by a surgeon and reviewed by a surgeon. That brings the cost for every case reported to nearly $50 per case. The rest of that is “Alice in Wonderland” dreaming if you think you are going to acquire data and analyze it for much less than that. If you want a medical student to do it, if you want some nurse to do it, that is all well and good. But if you want a trained surgeon to do it, the cost is going to be higher. The focus of my comments is that a big part of QSS is our systematic verification or reverification of accuracy of data. We now have gone back through thousands of data points on these 12,000 patients, and guess what? The verification results are perfect—except in the reported rate of complications. Our reported rate of complications is 1.2%. Reverification of these, good surgeons doing elective procedures, showed that only half of the complications were reported accurately and well. Half were missed for one reason or another. It is particularly interesting in that the way QSS works would be to the doctor’s slight financial advantage to report a complication, and yet still half the complications were not reported. This is primary data from surgeons and not the data in the literature. It suggests that Dr. Brennan, Dr. Jaques, and Dr. Martin have a lot of work in front of them to improve the accuracy of reporting of complications.

Dr. Alan S. Livingston (Miami, FL): Once again the Southern Surgical is assuming a leadership role in identifying a particular problem that many surgeons have not been willing to look at. And I think that the two papers from Drs. Jones and Brennan will serve as seminal contributions. And they are an important first step, but only a first step, in identifying how to approach the problem that actually segues into another area. Dr. Jones commented that as professionals we haven’t looked at this issue as carefully as we should. Another group of professionals, however, has extensive experience, and that is the lawyers, and they are looking at this in tremendous detail. And it segues into what you were saying yesterday, sir, in your presidential address, that one of the major areas of problems is medical, legal, and malpractice issues. This is actually one of the issues that is threatening the financial viability of academic medical centers, as we are all aware, and, of course, the private practitioners. In the state of Florida we are facing a tripling of malpractice premiums this year, and surgeons and neurosurgeons are facing premiums of $192,000. That literally is making it impossible for some of us to practice. At the University of Miami, our current premiums, we are self-insured, and we pay $12 million a year. And what we have done is by looking at data such as this, we are looking at not risk management but risk prevention. And until you have the information that is being pointed out today, you cannot possibly address the issue of managing this. We are self-assessing ourselves and we will increase the premium by about $4 million this year, which is a tremendous hardship. The thing that we have had a lot of trouble with is getting physicians, as Hiram has said, to look at the true incidence of complications and then to look at it from the perspective of learning from it. And what we are trying to do is instead of having a punitive M&M conference, we are trying to have an educational M&M conference to try to prevent the future complications. And our payouts last year, which were some $27 million in malpractice, one third of those were secondary to errors in judgment, and hopefully preventable with education. This is not only a necessary thing to do, it is the right thing to do.

Dr. Basil A. Pruitt, jr. (San Antonio, TX): I rise to ask the authors what role editors and editorial review play in this in contrast to the problem of definitions. I believe that you indicated that for one complication there were 12 different definitions. Similarly, there are several injury severity scales, each of which has its advocates. In light of that, how do we arrive at a consensus? I can foresee having a checklist that the author has to send to me and indicate that he has defined the accrual data method, indicated the duration of follow-up, and provided definitions of complications. Unfortunately, he may have different complication definitions than the next author. Consequently, there is a need for consensus definitions, and perhaps those could be developed by professional societies focused on particular diseases.

Dr. Robert R. Nesbit, jr. (Augusta, GA): I would just like to point out that the vascular surgical societies actually have made some progress in this area. They have developed fairly elaborate reporting standards and have subsequently refined them. I think that they have made a difference in the vascular literature, and I would like to suggest that perhaps other specialty societies might look at doing something similar.

Dr. J. Bradley Aust (San Antonio, TX): I would like to draw your attention to the fact that the VA National Surgical Quality Improvement Program has 500,000 cases that are completely analyzed as far as the definition of complications and the number of complications, and is available for scrutiny. It is now being applied to a number of civilian hospitals. These databases will become available as time goes by, and I believe this is the start of what needs to be done.

Dr. David P. Jaques (New York, NY): I am thankful that so many of you are as excited about this topic as we actually were. We started this truly to go looking for those definitions that we suspected were out there, fearing that as we established ours we might miss the opportunity to have identified standards that do exist. That was really the impetus for this. And yet we come to find out, as portrayed here now with these criteria, that in fact they are not out there, that they didn’t improve over time. That is the simple answer to the first question from Dr. Jones. The retrospectives were all done in this past decade, and they are no different from those prospective trials that have been reported over the past 25 years. We are not getting better. But none in the audience were surprised at that. Dr. Rikkers, your question cuts to the quick. It says, what are the solutions? In my own career I have been trained to not present a problem without a solution. And certainly in my current employment it is an unacceptable approach to not have the solution. In fact, we have put a few of those steps in place. We now have some 186 defined and fully graded complications. We are working hard with the NCI to incorporate portions of these alongside the common toxicity criteria that exist in medical oncology reporting. They have 310 complications and 24 categories. You can record a grade 4 hiccup if you receive a medication for cancer treatment, yet we have none of this in surgery. Our problem with that group is that the medical oncologists got there first, and so all surgical interventions essentially are grade 4. Now, we understand in our discipline that there are variations in operations, really how severe they are, so we were fighting a bit of an uphill battle last year as we tried to incorporate these into this version of the CTC. But from the solution point of view, I think it is there. I think severity grading opportunities can be promulgated and need to be alongside consistent reports in the surgical literature. The grading system that we have chosen—you asked that question—is centered on a modification of one put forth a number of years ago that to the best of my knowledge never really caught hold, as evidenced by this report. And it is centered on the concept that you wanted to look at the degree of effort it took to correct the complication. We all know that there are what we consider grade 1 pancreatic fistulas. It is a patient who indeed has increased drainage from the drain site but really no impact on that patient. We somehow need to focus on those patients who required greater intervention for the correction of the same complication—a grade 3 complication then would require angiography, endoscopy, intubation, operation, or interventional radiology. From Dr. Polk, a 1.2% morbidity rate is a little low. And I suspect two times that is a little less low, but still low. Dr. Rob Martin looked back at our own records, despite what we considered to be an excellent audit and editing process for morbidity. After we had fellows and attendings review our own morbidity events, he tried to identify any evidence of further complications that we could detect by looking at both nursing notes and physician notes. And in fact the phenomenon that occurs is, once the surgeon confesses to a high-grade complication, we lose track of the lower-grade ones. Perhaps that is okay. Maybe the lesson in this cost-containment era is that we should focus on the complications that matter a great deal and be less concerned with the grade 1 or 2 complications. So in our world the grade 3 and 4 and 5 complications are where we ought to be putting the majority of our energy. I don’t know that for many operations it is as essential to invest so greatly in capturing some of the urinary retentions that seem to proliferate so freely in our M&M conferences, perhaps because it is viewed as one of those acts of God that none of us has any personal control over. Other questions centered on the issue of educating and not punishing. And we totally agree with that. Drs. Pruitt, Nesbit, and Aust have each spoken to various means of getting those definitions for complications, and we totally concur. Consensus will be necessary for the buy-in to occur. Where it exists, I think they have made strides. In the VA program referred to, much of the risk stratification, I fear, has centered on the presence or absence of complications. I think we can refine that database a bit by focusing on those complications that really matter and risk-stratify for those more specific endpoints.

Again, from the standpoint of being able to control costs, it may be that we want to focus our attention on several high-risk procedures and their specific complications as we put our energies towards capturing those for comparison across institutions. As this consistent reporting of that limited data is established, we can expand the information to a more costly and comprehensive review, such as those that we have instituted at Memorial Sloan-Kettering Cancer Center.