Implementation of a Novel Conference Series on Clinical Practice Variations Provides an Opportunity for Constructive Discussion of Faculty Practice Patterns: Do as We Say? … or Do as We Do?

Abstract

Background

Although evidence‐based medicine (EBM) is routinely used to guide management in the emergency department, there is still considerable variation in clinical practice. Trainees may not fully appreciate the reasons for these clinical practice variations (CPVs) and may find it frustrating when they encounter them. We used areas of CPV among our faculty as the basis for resident educational sessions and assessed the perceived utility of these sessions.

Methods

Topics were identified when residents noticed significant variability among the faculty in their management of particular clinical presentations. Sessions were conducted by facilitator‐led reviews of EBM guidelines, by faculty panel discussions of their management rationale where EBM guidelines are not available, or by pro–con debates. Residents were surveyed after the initial sessions to assess the utility of this series and changes in their understanding of CPV.

Results

There was a 72% response rate. The percentage of residents who were frustrated with CPV decreased from 64% to 35%; the percentage who felt that the presence of CPV enhanced their learning increased from 19% to 48%. Sixty‐five percent felt that the educational series contributed to decreased frustration, 77% felt that the sessions helped them understand why CPV occurs, and 93% felt that they helped their overall learning.

Conclusion

Explicit discussion and exploration of CPV in an educational setting can provide multiple benefits. Trainees may gain a better understanding of why CPV occurs and of the rationale behind practice variations. Faculty may benefit from analyzing CPV to determine whether these truly represent the “art of medicine.”

Bedside teaching plays a significant role in graduate medical education (GME) and is often quite different from the teaching that occurs in the classroom. Although much of the clinical medicine that is taught today in GME is “evidence‐based,” there is considerable individual variation in clinical practice, even among physicians practicing in the same hospital or training program.1 Variations in clinical practice are frequently cited in the medical literature, but almost always as a problem to be corrected in order to optimize care across geographic and demographic boundaries.2, 3 However, clinical practice variations (CPVs) seen in emergency medicine (EM) practice can be exploited for their educational value for trainees.

Individual physicians may practice in different ways, depending on the time of day; resources available; volume of patients in the emergency department (ED); and personal differences in temperament, risk tolerance, and prior experiences.1 There are also many circumstances in which clear evidence‐based guidelines do not exist. Each physician may have his or her own preferred “default” management strategy for a clinical problem, with some added variations that depend upon the individual case.1 These “default” strategies are often based on the individual physician's own values, knowledge, and experience, although factual misconceptions or outdated practice habits may at times play a role as well.

Trainees, especially novices, who are often rule‐driven and in search of definitive answers from which to create rules, can feel frustrated with variations in clinical practice and may not understand all the reasons they may exist.4 We sought to take advantage of existing variations in practice among our faculty members by developing an educational series in which faculty and EM and EM/internal medicine (IM) residents could address practice variations that had been noted to occur in our programs at the University of Pittsburgh Medical Center and Hennepin County Medical Center (HCMC). We describe the successful implementation of this educational innovation at our institutions.

Methods

Our EM residencies were 3‐year programs with 48 and 36 trainees, respectively, based at large urban academic medical centers; the EM/IM residency at HCMC was a 5‐year program with 10 trainees. We identified a series of clinical issues based on feedback from residents. We periodically surveyed residents by e‐mail to identify areas in which they had encountered significant variability in the faculty's management of particular clinical presentations or situations (Table 1). Those that lent themselves most to discussions of evidence‐based guidelines, or the lack thereof, were preferentially selected. Specific clinical questions within each topic were chosen for discussion. Some examples were: 1) cervical spine injury—how to manage patients with persistent pain following a negative cervical spine CT; 2) asymptomatic hypertension—the utility or necessity of checking “screening studies” such as creatinine and ECG; 3) dental pain—the utility of prescribing antibiotics in patients with tenderness to dental percussion without definitive evidence of an abscess; 4) hyperkalemia—the utility of bicarbonate as a treatment modality; and 5) excited delirium—how to decide if patients who receive ~5 mg/kg intramuscularly of prehospital ketamine require endotracheal intubation.

Table 1.

Examples of Conference Topics

Dental pain	Chronic pain
Chest pain	Pharyngitis
Ankle sprain	Musculoskeletal pain
Alcohol intoxication & withdrawal	Otitis media
Closed head injury	Epistaxis
Seizure	Sexually transmitted diseases
Headache	Cutaneous abscesses
Asymptomatic hypertension	Renal colic
Medical clearance of psychiatric patients	Gastrointestinal bleeding
Patients leaving against medical advice	Indications for pelvic and rectal examination
Musculoskeletal pain	Online medical control
Pediatric fever	Cervical spine injury
Agitation/excited delirium	Hyperkalemia

An e‐mail survey or questionnaire was sent to faculty members prior to each session. Most questions were based on hypothetical cases related to the topic to be discussed that month. The de‐identified responses were compiled and then presented at the educational session to stimulate questions and discussion.

The sessions were planned and conducted by residency faculty leadership, and the specific topics selected were those most frequently identified by residents on periodic surveys. Sessions generally took place on a monthly basis (10–12 per year) at each institution as part of required residency conference time and were typically 1 hour in length. Typical attendance at these conferences was about 30 residents and six to eight faculty.

The sessions were conducted in one of three ways: 1) the faculty facilitator reviewed evidence‐based literature and current published recommendations on diagnosis and management (examples of questions and answers on a typical topic are presented in Table 2); 2) sessions on topics where no evidence‐based medicine (EBM) guidelines were available included a panel of three to four faculty who served as “experts” in a case discussion and described their approach and management rationale in specific clinical situations; and 3) additional sessions were conducted at bimonthly faculty meetings in a debate format, with one “pro” and one “con” debater on each side of a clinical question, with subsequent group discussion. Background material on evidence‐based guidelines, when applicable, was supplied to participants in advance. Following each educational session, a brief summary of the clinical question, applicable EBM guidelines, and experts’ consensus, along with reasonable alternative approaches (when applicable), was distributed to all residents.

Table 2.

Sample Questions and Answers for Conference

Sample Questions	Sample Answers
Do you always order an ethanol level on patients that have been drinking alcohol?	Yes if patient is obtunded. Yes, but more out of habit than need.
If not, how do you decide which patients to order it on?	Not unless there is a change in mental status. Not unless the diagnosis is in question.
Do you order vitamins for your alcoholic patients that present intoxicated?	I only give vitamins to alcoholic patients who might be deficient, and give IV only if they can't take the folate/thiamine/MVI by mouth (difference is pennies vs. hundreds of dollars for banana bag). I don't give to younger patients who were just out at a party.
If so, via which route do you give the vitamins?	I'll administer IV vitamins if I suspect chronic alcoholism and nutritional issues, with delirium or stigmata. Yes, to homeless, chronically nutritionally depleted patients.
	Not IV. No evidence for benefit. If I am concerned about their vitamins I give a PO multivitamin, thiamine, and folate when they are awake. I give IV if patient systemically ill, emaciated, or being admitted for reason other than EtOH. I don't give vitamins IV. If it is an alcoholic, I order PO MVI, thiamine, and some food.

We assessed the perceived utility of these sessions by means of a questionnaire assessing residents’ perceptions of a CPV conference before the inauguration of these sessions and following the first year of their inclusion in our educational programs (see Data Supplement S1, available as supporting information in the online version of this paper, which is available at http://onlinelibrary.wiley.com/doi/10.1002/aet2.10074/full). Questionnaires were developed by the authors and were reviewed by four emergency medicine educators and researchers. A representative sample of residents was interviewed to identify ways to improve the survey items. At the end of the first year of these sessions, the survey was revised and distributed to the residents at HCMC.

Due to the small sample size, we elected to collapse the two survey items indicating frustration with CPV and the two items indicating enhanced learning from CPV for purposes of data analysis. A chi‐square test was used.

Results

Thirty‐one of 43 (72%) residents at HCMC, approximately 10 per level of training, responded to the questionnaire. Sixty‐eight percent of responding residents indicated that at baseline, when working clinically, they generally felt frustrated by CPV, but less than a quarter routinely asked faculty about these variations during clinical shifts (Table 3).

Table 3.

Percent of Time CPV Is Discussed With Faculty on Shift

How Often Do You Discuss CPV With Faculty When It Occurs During Shifts?	% of Resident Responses
100% of the time	0%
75% of the time	23%
50% of the time	55%
25% of the time	23%
0% of the time	0%

Answers to the survey question, “When you notice practice variation (CPV) occurring on clinical shifts, approximately how often do you have a discussion with the faculty about this practice variation?”

Percentages are rounded to the nearest unit.

CPV = clinical practice variation

Before the initiation of the conference series, 64% of residents reported significant or moderate frustration with practice variations, and only 19% felt that exposure to CPV enhanced their understanding of emergency practice. After the first year of the conference series, only 35% of residents reported significant or moderate frustration, and 48% felt that exposure to CPV enhanced their learning. This difference was statistically significant (p = 0.039; Tables 4 and 5).

Table 4.

Frustration With Faculty CPV Before and After Implementation of CPV Conference Series

Are You Frustrated With CPV Between Faculty Members?	Before CPV Series	After CPV Series
Yes, significantly frustrated	19%	3%
Yes, somewhat frustrated	45%	32%
Neutral	16%	16%
No, I feel CPV somewhat enhances my learning	19%	35%
No, I feel CPV significantly enhances my learning	0%	13%

Answers to the survey question, “Are you frustrated with CPV between faculty members?”

Percentages are rounded to the nearest unit.

CPV = clinical practice variation.

Table 5.

Comparison of Frustration With Faculty CPV Before and After Implementation of CPV Conference Series

Are You Frustrated With CPV Between Faculty Members?	Before CPV Series	After CPV Series
Yes, frustrated	64%	35%
Neutral	16%	16%
No, CPV enhances my learning	19%	48%

Collapsed answers to the survey question, “Are you frustrated with CPV between faculty members?” with the two survey options indicating frustration with CPV and the two survey items indicating enhanced learning with CPV collapsed for purposes of data analysis. The chi‐square test indicates statistical significance (p = 0.039).

Percentages are rounded to the nearest unit.

CPV = clinical practice variation.

Sixty‐five percent of residents felt that the conference series directly contributed to decreased frustration with CPV among faculty, 77% felt that it helped them to understand why practice variation occurs, and 93% felt it helped their overall learning on the clinical cases that were discussed. Examples of how resident behavior or approaches changed as a result of the CPV sessions are listed in Table 6.

Table 6.

Excerpts of Residents’ Answers to Survey Question, “How Has the Clinical Practice Variations Series Affected Your Behavior and/or Approach to Patient Presentations in the Clinical Environment?”

1. I have started treating tactile pediatric fevers at home as real fevers per the CPV session.

2. I think it gives you comfort with knowing there isn't always a right answer to things.

3. In general, I always understood that while CPV can be frustrating it is certainly a valuable component of our education. I think where it was most frustrating was when I felt uncomfortable with certain CPVs or faculty did not have a strong explanation behind their CPV. I think with all of the discussions that I was able to view it provided a formal setting to discuss evidence, reasoning, etc., behind certain practice variables. Overall, I felt that it was a very beneficial addition to our conference time and I hope that it continues next year.

4. I know the range of CPV for a particular topic and gauge my own comfort level for future practice. I also know which faculty member falls where on the CPV spectrum and try to anticipate what they would want to avoid conflict.

5. Appreciate that in some cases literature is sparse and the art of medicine shines through.

6. The peds fever is very, very interesting. I am less inclined to order x‐ray and other things in kids with viral illnesses. The c‐spine was also kind of interesting.

Discussion

Variations in clinical practice continue to be common in many real‐world clinical settings,2, 3, 5, 6 and the practical limitations of EBM have been widely noted, even by some of its most prominent proponents.7, 8, 9 Our residents reported that they found the educational sessions devoted to CPV helpful in understanding why variations in practice may occur. Residents also gained insight into the reasons for individual physicians to vary in their practice patterns and were prompted to think about how they themselves might proceed when next presented with similar clinical situations.

Teaching physicians face many different pressures when working in a busy ED. They are constantly being pulled in many directions at once, with frequent interruptions and immediate demands related to patient care. While the physician's primary obligation is to care for and communicate with individual patients and their families, the academic physician has also committed to a career in teaching, much of the time engaging with learners at many different levels—residents, fellows, medical students, nurses, and paramedics. Physicians are also exposed to external pressures, such as patient satisfaction surveys, productivity measures, incentive plans, potential medicolegal liability, and interactions with admitting and consulting staff in the hospital. In response to these factors, individual physicians may manage patients in ways that deviate from the “evidence‐based” approach that trainees are taught in the classroom or may demonstrate varied approaches to patient care when EBM guidelines are not available.10, 11

Our conference series was intended to increase our residents’ awareness of the factors that affect clinical decision making in actual ED practice and to provide an opportunity to examine their own developing practice patterns. A conference dedicated to CPV can help to address the frustration and confusion residents may feel when learning and working in the ED with a large clinical faculty. It can provide a forum in which residents can gain an understanding of the various approaches practicing emergency physicians may use in response to similar presentations or situations and helps residents more fully appreciate what independent practice may be like in their future careers. It also allows residents to develop a better understanding of both EBM and of the “art of medicine.”12

Finally, these CPV sessions can also provide a springboard for discussion among faculty members themselves. The ideas generated can spur the creation of clinical practice guidelines and the development of quality improvement initiatives. With these results in mind, we plan to continue to expand the range of clinical issues discussed at these sessions, adding new topics and continuously updating the material when appropriate in order to reflect current guidelines and practice. It is hoped that similar educational offerings can be fruitfully initiated in other training programs.

Limitations

Our report is limited to experiences at two institutions, and the formats we have developed may not be readily reproducible in other settings. In addition, we present data from only a limited time period at the initiation of this educational series.

Conclusions

Explicit discussion and exploration of clinical practice variations among academic emergency medicine faculty in an educational setting such as a residency conference offers multiple benefits. It allows trainees to better understand existing EBM guidelines, the reasons practice variations occur, the rationale behind various approaches to similar presentations or situations, and the fact that there is not always one “right” answer. Furthermore, faculty may have an opportunity to identify areas in which their practice may vary from others’ and consider whether these variations reflect the art of medicine or are practices that should be modified by available evidence.

Supporting information

Data Supplement S1. CPV Series Survey.

AEM Education and Training 2018;2:15–19.