Evidence-Based Medicine in Surgical Education

Abstract

Evidence-based medicine (EBM) is increasingly important for clinical surgery and for promotion of best practices into surgical decision making. Although barriers exist in the current surgical literature, for certain surgical scenarios, formal efforts to promote evidence-based surgery (EBS) into surgical education are helping to equip future surgeons with these important tools for optimizing patient care. As our evidence-base grows and standards of care evolve, incorporating EBM into everyday practice for trainees and staff surgeons remains an ongoing challenge.

Objectives: After completing this article, the reader should (1) know the definition of evidence-based medicine, (2) understand the challenges and limitations encountered when applying evidence-based medicine to the practice of surgery, and (3) be aware of the Evidence-Based Review in Surgery program.

What is evidence-based medicine (EBM)? EBM is best described as “the judicious use of the best current evidence in making decisions about the care of the individual patient.”¹ The primacy of using clinical research to guide decisions about individualized patient care originates in internal medicine and several medical subspecialties, particularly cardiology and oncology. Because of its rigor in design and control for different types of bias, randomized double-blind placebo controlled trials have had the largest impact in leading clinicians toward the concept of EBM. Other methods, such as prospective cohort studies, can also be used for evidence-based decision making, but only randomized trials can generate level I evidence.² Competing models of clinical decision making include extrapolations and inferences based on pathophysiology, judgment based on individual clinical experiences, and reliance on the opinions and traditions put forth by leaders in the field through textbooks, professional meetings, teaching settings, or review articles. These are good models of care for some clinical scenarios, but should not supersede recommendations generated by high-quality evidence in the literature. As challenging as it is to design and perform a randomized controlled trial or other types of rigorous studies, understanding how to promote incorporation of the results of such studies into daily clinical practice remains as much a formidable lynchpin for integrating EBM into reality.

EBM has been cross-applied to surgery and surgical subspecialties over the past several decades. Increasing numbers of randomized controlled trials (RCTs) are performed in surgery. Meta-analyses on surgical topics addressed by RCTs are also available through the Cochrane Collaboration and through the peer-reviewed literature. There are, however, unique problems with EBM in surgery, also known as evidence-based surgery (EBS), many of which pertain to the limited ability to define and then test different surgical techniques used in the operating room. Even if techniques can be compared in a randomized fashion, it is challenging to ensure that the techniques can be precisely defined and then performed in a standard manner in each operation, both by the researcher and by the practicing surgeon trying to apply the evidence. Surgical techniques are also increasingly driven by technological innovations in equipment and devices. These industry-driven changes can evolve faster than, and usually separately from, the slow-moving academic research model. Another major barrier to EBS is that surgical training is highly focused on an apprenticeship model, which guides training both in and out of the operating room and tends to lead a young surgeon to gravitate around the “tips and tricks” of his or her mentor(s). Despite barriers described, no studies have discussed any harmful outcomes that have come out of evidence-based practice.

Colorectal Surgery and Evidence-Based Surgical Practice

Some aspects of colorectal surgery practice lend themselves well to evidence-based practice. Cancer care is well informed by clinical trial data, and evidence-based practice in rectal cancer management, for example, has been relatively easy to promote. Perioperative care processes, particularly those targeted by quality measures, such as preoperative prophylactic antibiotics, bowel preparation, and deep vein thrombosis prophylaxis, are yet another domain where evidence-based best practices have been defined and then protocoled through the literature, professional societies, and even payers (e.g., Centers for Medicare & Medicaid Services [CMS] and the Surgical Care Improvement Project [SCIP] in the United States).

On the other hand, many other aspects of colorectal surgery are not easily practiced with an evidence-based approach because high-quality evidence currently does not exist. Management of emergent or urgent colorectal conditions, though amply described through retrospective review, is not commonly studied in a prospective fashion because of the difficulty of enrolling these types of patients into controlled trials. As an example, though common and life-threatening, acute diverticulitis has only recently been examined in one contemporary randomized trial addressing a relatively narrow question: laparoscopic versus open sigmoid resection.³ The management of relatively uncommon diseases, such as colonic volvulus or ischemic colitis, depends more upon traditional approaches described in textbooks because treatment decisions, such as when to operate, have not been studied in a rigorous fashion. Judgments can be based on generic principles of assessing how ill the patient is through patient history, physical exam, and laboratory evaluation. Inferences can be made from the literature on the natural history of disease.

Another area of problematic decision making is defining and treating subgroups of patients who fall outside established evidence-based practices because clinical trials either exclude these patients or do not have sufficient statistical power to stratify results by these subgroups. In these situations, the clinicians have to rely upon inferences concerning the pathophysiology of the disease in different types of patient populations, experience, and surrogate evidence to piece together the best treatment choice.

Additionally, many important technical aspects of surgery—questions such as whether a drain should be left after pelvic surgery, whether ureteral stents should be placed prior to a procedure, whether the splenic flexure should be routinely mobilized in a sigmoid resection—are currently inadequately studied with some notable exceptions: for example, whether an anastomosis should be stapled or hand-sewn⁴ has been studied in a RCT; and whether laparoscopic colectomy for colon cancer is safe has been studied in several well-designed RCTs.⁵ Because many technical questions can be defined, however, the future holds promise for quality evidence to emerge for some of these questions. Complex, but critical technical issues that involve multiple steps or a combination of actions and judgments, however, such as assessing tension on an anastomosis or the quality of a total mesorectal excision (TME) for rectal cancer are not as easy to study in a controlled fashion. Instead, high-quality nonrandomized data needs to be rigorously generated and critically evaluated. For example, TME has been established as the superior technical approach to rectal cancer surgery through careful retrospective examination gross assessment of pathologic specimens.⁶ A randomized trial comparing TME to non-TME surgery is, and probably never will be, feasible to perform because of ethical issues associated with this type of study.

Evidence-Based Surgery and Clinical Appraisal Skills

Incorporating EBS into clinical decision making requires critical appraisal skills. Critical appraisal skills of the literature and clinical situations allow the surgeon to identify and frame pertinent decisions, and in doing so, he or she can gather, clarify, and interpret clinical data required to make these decisions. An evidence-based approach can be overlooked by surgeons, or clinicians in general, for a variety of reasons, ranging from surgeon bias to lack of education to structural problems involving the timing and physical setting in which clinical decisions are made, which impact access to information.

Surgical decision making on a case-by-case basis is usually complex. The patient may be complex. For example, patients can have multiple medical problems and a complex surgical history, but may have a well-researched and well-understood disease, such as stage II colon cancer. Or the disease may be a poorly studied variation of a common disease, such as asymptomatic colon cancer with an isolated ovarian metastasis. In counterdistinction to this, the patient may have a relatively well-studied disease with minimal to no evidence-based guidelines, such as anal intraepithelial neoplasia.

Not uncommonly, the patient's diagnosis may not be clear, and the surgeon may have to decide whether an operation could help diagnose and treat the patient. Even in a straightforward clinical scenario, there are usually multiple decisions involved. Jacklin et al from the Imperial College in the United Kingdom used interview research methods with surgeons to dissect apart the decisions made in cholecystectomy cases during preoperative, intraoperative, and postoperative phases of care.⁷ On average, 15 decisions were made for this relatively straightforward procedure. With increasing complexity of the patient and disease, and with increased attention to phases of care, for example, preoperative “cooling” off in cases of diverticulitis, many decisions in a typical patient can be considered for questioning. To assess which of these decisions can be understood with evidence-based surgery, critical appraisal skills are required to find the evidence and apply it to the patient at hand. Disseminating critical appraisal skills to practicing surgeons and to surgical trainees is, as alluded to before, an ever-remaining challenge.

Multiple studies have sought to understand why evidence-based surgical practice has been slow to establish. Several interview and survey studies on surgeon attitudes and opinions toward EBS and clinical practice guidelines show that a major barrier is that many surgeons trust their experience and judgment, and incorporating published evidence may not fit into this model. Kitto et al surveyed 25 surgeons in Australia and found that most surgeons were using primary research articles, systematic reviews, and clinical practice guidelines (CPGs; 81.7%, 79.2%, and 84.7%, respectively), but on a Likert scale of 1 through 5 (1 being the strongest agreement), individual judgment was rated the best source of “evidence” (1.9), compared with the Cochrane Library (2.0), textbooks and journals (2.1), contact with colleagues (2.4), and CPGs (3.2).⁸ However, surgeons admit that EBM-generated knowledge was useful in daily decision making. Through an interview study of 22 surgeons, Kitto et al further clarified that while some surgeons saw the value of EBS, half of these surgeons interviewed still resorted to the same traditional practices they had learned through experience. These studies found that the objections to EBS are not strong, but interviews clarified that further training of practicing surgeons is required to better integrate evidence-based practice with experience-based surgery, and such training could likely be optimally provided by someone who understands surgery and can bring the focus onto day-to-day case examples.⁹

Barriers from a Trainee's Point of View

Surgical residents from McMaster University in Ontario were interviewed in a focus group setting and the results were described by Bhandari et al.¹⁰ The findings of this study focused on barriers to implementing EBS into practice. Twenty-eight surgical residents were interviewed. Most residents embraced an evidence-based approach and strongly desired to incorporate it into their training. However, they perceived that staff surgeons did not encourage using this approach, and they recommended that EBS-oriented faculty be sought out for hire. The critiques of faculty from the residents revealed the following barriers: “ego,” “older age,” “rigidity,” and “insecurity.” In this study, concerns were also raised about lack of time and inefficient processes to look up the evidence during direct patient care. In a survey study by Mittal et al, surgical trainees from India completed questionnaires. Although the majority of respondents believed that EBM improves patient care, with 68% indicating that they used guidelines and protocols developed through EBM, only 33% used the primary literature to practice EBM. The barriers listed in this study were related to education (poor understanding of statistics 61%), access (articles not available 51%, and lack of centralized sources of information 56%), and attitude/cultural barriers (administration would not allow implementation 45%, individuals did not have the authority to change practice 45%).¹¹

As argued by Norman, clinical reasoning is a complex phenomenon.¹² Reasoning skills are closely integrated with knowledge. Without a strong knowledge base, good reasoning skills cannot effectively lead to good decisions. Norman cites research from the 1970s that sought to discover the components of a “reasoning process” that was used by experts, to devise an approach for teaching students these skills. In these studies, the outcome measured, for example, “diagnostic accuracy” was strongly related to “content knowledge” rather than the “process” of clinical reasoning. This research suggests that a strong factual knowledge is a critical component of clinical reasoning, and generic approaches to teach the process of clinical reasoning are unlikely to be successful if taught in a vacuum. Although not directly related, and not contemporary with concepts of EBM, this research supports the notion that EBS is more appropriately taught using a specific and relevant clinical scenario as the center of the discussion, rather than focusing educational efforts on the process of interpreting the evidence. A case-based learning approach seems more likely to be successful.

Programs to Teach Evidence-Based Surgery

There have been several small studies that describe approaches to teaching EBS to trainees. Ubbink et al from the Netherlands describe an annual course for new staff members and surgical trainees teaching literature searches and formulation of answerable clinical questions using the PICO approach,¹³ which identifies the patient's problem, the intervention under study, the comparative standard intervention, and the outcome variables(s) of interest.¹⁴ Thereafter, trainees present a weekly session on critically appraised topics.

Haines and Nicholas describe a comprehensive EBM approach for a U.S. neurosurgery training program that follows up on clinically important questions arising in case conferences.¹⁵ Primary literature was queried and synthesized by residents, then later rediscussed at another session. Available resources were textbooks and online resources, without an expert or EBM mentor available, yet the program was described as successful because of the commitment and motivation level demonstrated in this group of residents.

The Evidence-Based Reviews in Surgery (EBRS) Program

The Canadian Association of General Surgeons first took the process of disseminating critical appraisal skills to a national level, which eventually evolved in part toward the Evidence-Based Reviews in Surgery (EBRS) and Evidence-Based Reviews in Colorectal Surgery (EBRCRS) programs. In 2004, MacRae et al found that an Internet-based journal club was better than self-directed reading among general surgeons throughout Canada, based on results of an examination developed to test critical appraisal skills.¹⁶ The intervention was tested in a randomized-controlled trial of 81 Canadian surgeons with a mean number of 23 years since graduation. The intervention was a series of eight programs sent every month that included an article, a discussion question packet designed to guide critical appraisal, and a subsequent listserv discussion about the methodology of the article moderated by a general surgeon.

The same investigators tested the EBRS program in U.S. general surgery programs, comparing an Internet-based journal club as described in the previous study with a face-to-face onsite moderated journal club.¹⁷ Twelve general surgery programs and 441 residents were studied. In one of two examinations based on one of two test articles, the moderated group had a higher score than the Internet group (53.0% vs 45.9%; p = 0.05), but the total test scores were not different between groups. Far more work was done by the moderated group than the Internet group, however, with a 5.3 of eight packets completed by the moderated group, and 0.66 of eight packages completed by the Internet group. Both of these studies used rigorous methods and had a clearly defined intervention that showed modest results. However, the outcome of improved or even altered patient care was not tested. These studies offer modest support for the EBRS and EBCRS program. EBRS is available through the Canadian Association of General Surgeons (CAGS) or the American College of Surgeons (ACS). The program runs from October to May and consists of eight monthly packages oriented around a clinical article and a methodological article that assists the learner in a critique of the clinical article. Reviews are provided by experts in the field.¹⁸

Teaching EBS in training programs seems to do best with a micro-level or local approach based on the above RCT and also based on the survey data. As has been found in education research with technical skills or core knowledge/core curriculum, face-to-face conferences led by a surgeon-mentor is most likely to result in teaching critical appraisal skills and to result in adequate resident trainee preparation. EBRS offers a highly structured curriculum. The literature suggests that these conferences should focus on clinically apropos topics, and perhaps highest-yield topics are those that generate discussion and clash-points such as primary anastomosis versus Hartmann procedure for perforated diverticulitis.¹⁸

Where Does Evidence-Based Surgery Fit into Surgical Training?

Some of the challenges for incorporation of EBS into general surgery and colorectal surgery training programs are a variety of competing educational interests. Basic knowledge must be taught in didactic sessions, and increasing emphasis is placed not only on textbook learning, but also use of the United States Surgical Council on Resident Education (SCORE) curriculum.¹⁹ Standardized tests query factual knowledge more frequently than evidence-generated recommendations or critical appraisal skills. The advent of standardized training in surgical skills requires time spent in a skills laboratory or simulation center. However, trainees in the United States also have work-hour restrictions and are better able than ever to pursue new areas of training such as EBS. EBS-type training conferences should focus on integration with relevant cases associated with patients the trainees have cared for; otherwise, it becomes another special interest in a silo, not directly juxtaposed with the practice of day-to-day care.

EBS is inherently difficult to incorporate into practice. Many authors have highlighted these problems. Commonly cited problems are poor quality of studies in surgery²⁰ and the lack of studies available for many surgical decisions.²¹ RCT data can also be difficult to generalize because frequently the trial will study a homogeneous population and exclude common types of patients such as the elderly and patients with complex medical problems.²²

In the EBS, perhaps to date there has been insufficient emphasis on how to use nonrandomized or poorer-quality evidence, such as cohort studies or even retrospective series.^22,23 A couple of examples serve to highlight this deficit. A young patient with cystic fibrosis is presented at a case conference. He is described as having been admitted with a bowel obstruction, which failed to resolve in the hospital. He had no prior surgical history and a laparotomy was performed with ileocolic resection where the transition point was identified. Final pathology showed normal bowel. In this example, a literature search may have not reviewed high-quality evidence or even framed choices in a PICO model, but the diagnosis of DIOS (distal intestinal obstructive syndrome), which occurs in patients with cystic fibrosis, may have been uncovered, and level V evidence,²⁴ together with inferential knowledge and judgment skills, may have allowed the team to consider a longer course of watchful waiting with therapeutic acetylcysteine and diatrizoate meglumine (Gastrografin). In another example, a patient develops colonic pseudoobstruction (Ogilvie syndrome) after orthopedic surgery. Textbook maneuvers are performed with a Gastrografin enema to rule out a mechanical obstruction. The team deliberates on decompressive colonoscopy versus neostigmine therapy. A retrospective series,²⁵ though subject to biases, offers specific information on the risks and expected outcome of neostigmine therapy that can guide counseling and planning for this therapeutic intervention. In these two examples, proficiency with a literature search, and the ability to incorporate this practice into the daily workflow of the surgeon is important to arriving at the best decision for the patient.

Perhaps as great of a challenge is providing education to the practicing surgeon, who is less able to participate in face-to-face conferences on a regular basis. To address the needs of practicing surgeons, the burden of education needs to be taken on by institutions such as hospitals and surgical societies.^26,27 Ultimately, if professional societies or national regulatory bodies, for example, the National Quality Forum or the Joint Commission for Accreditation of Hospitals (JCAHO) can promote EBS as a means of improving patient care and possibly increasing cost-effective care, incentives can be designed.²⁸

Future Directions

The rationale for including evidence-based surgery more routinely in surgical practice is well accepted and opportunities exist to design strategies for the surgeon-clinician faced with information overload as they sift through available publications. Health care organizations have the obligation to provide their clinicians with resources to access high-quality evidence. Professional societies and other networks of experts should routinely appraise and summarize evidence through guidelines and systematic reviews. Teaching programs need to provide trainees with the fundamentals in identification and interpretation of high-quality studies in the primary literature. This can be achieved with moderated journal clubs, or case discussions with a follow-up search for pertinent studies related to the cases at hand. Optimal approaches, not described in the surgical education literature, involve integrating a search for evidence during inpatient ward rounds,²⁹ in the clinic, or in the operating room using a point-of-care approach. As the field of information technology accelerates, creative ways to efficiently incorporate evidence into practice are likely to emerge in the future and the key will be the dissemination of best techniques.

Conclusion

Although surgical literature and CPGs continue to have some limitations and practice patterns are often still based in surgical dogma, EBS is increasingly important and accessible to surgery trainees and practicing surgeons. Formal programs, such as EBRS, as well as other formalized programs to incorporate these principles show promise for better dissemination of these skills. Further evolution of these programs is needed to help surgeons better apply clinical evidence into practice, particularly where poor-quality evidence exists or when patients are complex or do not meet standard criteria to support many of the difficult clinical encounters faced by surgeons in the real world.