Curriculum Providing Cognitive Knowledge and Problem-Solving Skills for Anesthesia Systems-Based Practice

Abstract

Background

Residency programs accredited by the ACGME are required to teach core competencies, including systems-based practice (SBP). Projects are important for satisfying this competency, but the level of knowledge and problem-solving skills required presupposes a basic understanding of the field. The responsibilities of anesthesiologists include the coordination of patient flow in the surgical suite. Familiarity with this topic is crucial for many improvement projects.

Intervention

A course in operations research for surgical services was originally developed for hospital administration students. It satisfies 2 of the Institute of Medicine's core competencies for health professionals: evidence-based practice and work in interdisciplinary teams. The course lasts 3.5 days (eg, 2 weekends) and consists of 45 cognitive objectives taught using 7 published articles, 10 lectures, and 156 computer-assisted problem-solving exercises based on 17 case studies. We tested the hypothesis that the cognitive objectives of the curriculum provide the knowledge and problem-solving skills necessary to perform projects that satisfy the SBP competency. Standardized terminology was used to define each component of the SBP competency for the minimum level of knowledge needed. The 8 components of the competency were examined independently.

Findings

Most cognitive objectives contributed to at least 4 of the 8 core components of the SBP competency. Each component of SBP is addressed at the minimum requirement level of exemplify by at least 6 objectives. There is at least 1 cognitive objective at the level of summarize for each SBP component.

Conclusions

A curriculum in operating room management can provide the knowledge and problem-solving skills anesthesiologists need for participation in projects that satisfy the SBP competency.

Editor's Note: The online version of this article contains 3 supplemental tables providing additional details about the course material, as well as an appendix of the 156 exercises in 17 cases that the students perform.

Background

The Accreditation Council for Graduate Medical Education (ACGME) mandates that residency programs in all specialties teach and assess 6 general competencies, including systems-based practice (SBP).^1,2 The components of the SBP competency (table 1) and functional definitions encompass content related to “cost effective practice,” including “tools and techniques for controlling costs and allocating resources, understanding of financing and insurance structures, mock practice experiences/mock financials, and evaluating risk and benefit of costly prescribing.”³ Anesthesiologists engage in operating room (OR) management, essentially directing the surgical suite and ensuring smooth patient flow. Practitioners must be able to understand coordination of surgical suites and explain the principles to OR nurses, nursing directors, surgeons, surgical technologists, anesthesia technicians, postanesthesia care unit nurses, intensive care unit nurses, and administrators.^4,5 Among the 61 English language articles identified by searching “operating room management” in Web of Science, 74% were published in anesthesiology journals.

Table 1.

Functional Definitions of the Accreditation Council for Graduate Medical Education (ACGME) Competencies in Systems-Based Practice^a

Three characteristics of hospital surgical suites cause coordination of surgical care to differ from SBP in other specialties. First, surgery is non-preemptive, meaning that once a surgical procedure is started, it must be completed. The patient cannot be moved out of the OR and replaced by another patient, with the surgery to be finished at a later time. All decisions concerning the scheduling of a case are therefore made before the case is started. Second, surgical cases with anesthesia have a large mean absolute error in predicted duration.⁶ Third, variability in the daily numbers of scheduled cases affects the efficiency of use of OR time. Coordination of cases is therefore an important issue.

The mathematics of forecasting case durations is nonintuitive. For example, the time remaining in a case that is running late is the time until the end of surgery, plus a relative constant for irrigation, inspection, incision closure, awakening from anesthesia, and transport to recovery. If a case scheduled for 3 hours is still ongoing after 2.5 hours, the median expected time remaining is not 0.5 hour but longer, for example, 1.0 hour.⁷ If still ongoing at 3.5 hours, the best estimate of the time remaining, and when the next case can begin, is still 1.0 hour.⁷ These nonintuitive aspects of OR scheduling should be understood by anesthesiologists coordinating the perioperative team, as well as by anesthesiology residents embarking on an SPB project in OR management. The knowledge and problem-solving skills needed for OR management are generally not readily learned on the job but through formal study.⁸ Innate psychological biases⁹ and misconceptions¹⁰ must be overcome.

In a previous study of teaching related to the SBP competency in anesthesiology, Delphin and Davidson¹¹ showed that resident team projects are effective. Although indispensable to satisfying the competency, the level of knowledge and problem-solving skills needed to select and conduct projects presupposes a basic level of understanding, such as exemplifying and summarizing principles according to higher levels of Bloom's taxonomy (table 2).^12,13 Residents may not yet possess this knowledge. Delphin and Davidson's team-oriented approach is best applied once residents have acquired these basic skills.

Table 2.

Definitions of Standardized Vocabulary for the Knowledge and Skill Levels Attained for Each Competency and Cognitive Objective^a

Curriculum Design

To offer residents a basic understanding of the scientific principles of OR management as they relate to SBP, we developed a course in OR management originally designed for master's-level hospital administration students in the University of Iowa Department of Health Management & Policy. The 35-hour course is given over 3.5 days during 1 weekend or 2 successive weekends. The course teaches residents when statistical forecasting is necessary, ways to apply literature searches to choose among forecasting methods, and how to explain to analysts the needs of an anesthesia group.

Although the curriculum was designed for hospital administration students, participants to date have been anesthesiologists and a few perioperative analysts and OR nursing directors. Other hospitals send representatives to the course to learn principles of OR management for application at their own institutions. One hospital sent a contingent of anesthesiologists, OR nursing directors, and financial analysts to the course to lay a common foundation on which they could base their decisions. Thus, the course has been attended by a wide variety of personnel with varying backgrounds. As part of our needs assessment to apply for continuing medical education credit, we observed that the curriculum's cognitive objectives provide the knowledge and problem-solving skills necessary to satisfy the ACGME recommendations for teaching “an awareness of and responsiveness to the larger context and system of health care” and teaching cost-effective practice (supplemental online tables 4 ^{(48.5KB, doc)} through 6 ^{(98KB, doc)} ).

The curriculum includes 3 educational methods (table 3 and supplemental online table 5 ^{(95KB, doc)} ): 10 lectures and 156 exercises using 17 case studies (online appendix ^{(134KB, doc)} ). Our website (www.FranklinDexter.net/education.htm) contains all slides used in the lectures, 7 targeted articles relevant to SBP for anesthesiology residents,^14–21 and a link to “statistics for anesthesia,” which is a review of required background material involving basic statistics up to the level of Student t test.²² The lectures, slides, and review of statistics are all publicly available. Learners complete the 156 exercises using an Excel workbook tool that provides immediate, adaptive, elaborated feedback to numeric and short answer questions. The adaptive feedback is programmed without macros by the use of more than 4250 Excel spreadsheet formulae containing more than 8300 Excel functions. Variations of the curriculum have been given to groups of 7 to 50 students on 14 occasions.

Table 3.

The Teaching of Cognitive Objectives and the Contribution of Each Objective to Accreditation Council for Graduate Medical Education Components

table 2 lists 5 hierarchical levels of knowledge from the review by Anderson and colleagues¹³ of the revision of Bloom's taxonomy. table 1 shows the ACGME's definitions of the components of SPB. Remembering a new concept is the minimum level of knowledge, followed by an ability to exemplify, summarize, explain, and apply that concept. We used this standardized vocabulary to formulate the 45 objectives of the curriculum and evaluate the extent to which each objective satisfies the core components of the SBP competency.

Using the ACGME language in table 1, we created functional definitions that would provide anesthesia residents with the knowledge and problem-solving skills needed to satisfy the component. Each definition is based only on exemplifying the material (ie, illustrating by example).

Implementation of the Curriculum

We assigned ratings (levels of knowledge using standardized verbs) to calculate the level at which each of the 45 objectives satisfied each of the core components of the competency (supplemental online table 4) and the ACGME recommendations for implementation (supplemental online table 6). Ratings were based on the depth of the lecture material presented to the students and the nature of the 156 exercises in the Excel workbook. The exercises are included as supplemental online appendix. We use Objective 6 as an example to show how the educational materials and exercises were used to determine without ambiguity the appropriate verb from the standardized vocabulary. The number of core components satisfied by each objective is summed in table 3 column 5. The number of cognitive objectives that satisfy each core component at each hierarchical level of knowledge is summed in table 3. The number of cases, readings, and lectures associated with each component is also summed in table 3.

For example, Objective 6 is to “Apply multiple methods of case duration prediction, including use of upper and lower prediction bounds, for making staff assignment and case scheduling decisions, with and without computer support.” This objective refers to the frequent decision whether or not to insert a pending add-on case into a gap in the schedule. Assume the first case of the day was scheduled to last until noon but ended at 9 am because the pathology was more extensive than anticipated. The next case was not scheduled to begin until 12:30 pm. Another surgeon had a 3-hour add-on case in a different OR and was hoping to start sooner than the scheduled noon start time. The OR manager should consider the 90% upper prediction bound for the duration of the add-on case when deciding whether it would fit into the schedule without disrupting later cases and extending the workday. In addition to using prediction bounds, the case requires students to calculate the economic savings by moving a case, revise case scheduling to reduce the probability of resource conflict over a surgical microscope, estimate the cost of time spent waiting for patients, determine optimal hours of staffing based on the standard deviation for the length of the workday, and examine the effects of nonsequential case scheduling on personnel needs. The case thus requires students to “apply” their knowledge.

The curriculum is taught using 17 cases, 10 lectures, and 7 readings (shown in supplemental online table 5 last 3 columns). Each objective contributes to at least 1 of the 8 ACGME SBP core components, and most contribute to at least 4 of them (table 3 last 3 columns). The objectives contributing to just 1 component enable understanding of objectives taught later and thus cannot be deleted.

Each component of SBP is addressed at the minimum requirement level of exemplify by at least 6 of the 45 specific objectives (table 5 column 8). There is at least 1 cognitive objective at the level of summarize (level 3) for each component (table 5 column 5).

Discussion and Conclusions

We designed a 35-hour curriculum to teach the basic principles of operations research for use in OR management. The curriculum consists of 10 lectures, 7 readings, and 17 case studies. We have shown that 45 cognitive objectives provide the knowledge and problem-solving skills needed for anesthesiology residents to perform projects that would satisfy the ACGME competency in SBP. The unique curriculum reflects anesthesiologists' responsibilities for coordinating OR care and the fundamental differences of hospital OR management from that of other medical specialties.

Our curriculum in operations research for surgical services provides the knowledge and problem-solving skills needed to satisfy all elements of the ACGME core competency in SBP. Examples illustrate the reliability and face validity of the objectives of the curriculum, the assignment of levels of knowledge using a standardized vocabulary, and the assessment of the extent to which each objective meets the 8 ACGME core components of SBP and the 4 elements for implementation.

Anesthesiologists differ from physicians in other specialties in that their professional work involves the smooth flow of patients through the surgical suite. The literature on SBP projects for other disciplines focuses on quality improvement or reducing medical error, while our curriculum addresses the coordination of patient flow. Our curriculum is therefore useful for practicing anesthesiologists, nursing directors, analysts, and other OR staff. In addition, the curriculum addresses 2 of the 5 Institute of Medicine core competencies for health professionals—evidence-based practice and work in interdisciplinary teams.²¹ Evidence-based practice is addressed by almost all of the cases and lectures. Participation of nursing directors, certified registered nurse anesthetists, operations managers, financial officers, and others enhances understanding on the part of anesthesia personnel of the viewpoint of others, satisfying the Institute of Medicine's competency of “work in interdisciplinary teams.”²³

Anesthesiologists, OR nursing directors, industrial engineers, analysts, and administrators report that the curriculum is useful. All 17 students enrolled in the most recent course presentation submitted evaluation forms. They were asked to agree or disagree with various statements on a 5-point scale, with 4 being “agree” and 5 being “strongly agree.” All students scored the course a 4 or 5 with respect to “class increased my trust in applying evidence-based statistical methods and analytic reports in health care management decisions,” “class enhanced my ability to analyze managerial questions critically and solve the problems in a logical, evidence-based manner,” “course was well planned and content was organized logically,” and “cases and readings aided in learning the material.” However, we have no such formative assessments from anesthesiology residents. Future work is needed to evaluate the curriculum to determine the extent to which it aids anesthesiology residents in developing projects for SBP.