Update in Medical Education 2010–2011

Introduction

This review brings important medical education articles of 2010 to the attention of academic general internists. These studies offer methods to improve trainee supervision, continuity of care and the physical examination. The summaries also serve as a reminder of the environmental challenges facing faculty and learners, including burnout and a sense of entitlement among learners, and offer some insight in addressing both issues.

Methodology

Articles selected were identified through a hand search of 14 medical education and clinical medical journals and through a search of PubMed using a comprehensive search strategy with terms on medical education, learners and outcomes/evaluation. The 14 journals searched by hand were:

Academic Medicine	Journal of Hospital Medicine
Journal of Continuing Education in the Health Professions	Annals of Internal Medicine
Journal of Graduate Medical Education	Lancet
Journal of General Internal Medicine	BMJ
Medical Education	Journal of the American Geriatrics Society
Medical Teacher	JAMA
Teaching and Learning in Medicine	New England Journal of Medicine

Selected articles were reviewed for relevance to a general internal medicine audience, potential to impact practice, study design, educational innovation and overall study quality. Over 300 articles were considered, with the final articles chosen by author consensus.

Peets AD, Cooke L, Wright B, Coderre S, McLaughlin K. A prospective randomized trial of content expertise versus process expertise in small group teaching. BMC Med Educ. 2010;10:70.

This study compared the effect of preceptors with either "process" or "content" expertise on medical students’ learning outcomes at one Canadian medical school. One hundred fifty-one first year students were randomly assigned to one of 11 small groups for the cardiovascular/respiratory course. Each group was randomly allocated to be facilitated by content experts, process experts or both. Content experts were cardiology or pulmonary medicine subspecialists, and process experts were generalist physicians trained in an 80-h certificate program in teaching. All faculty received a preceptor guide for the content of each session. Students completed a multiple choice question exam at the end of the course and evaluated faculty teaching after all 21 sessions using a modified Stanford University Faculty Development Program (SUFDP) tool. Process experts were rated higher overall and on each domain of the SUFDP tool, and the mean score on the exam was not significantly different between groups.

The generalizability of these findings is limited since this was a small, single-center study. The preceptors' content guide may have reduced the importance of content expertise as a factor. Process experts also received significant training in teaching pedagogy. This study found no differences in learning outcomes between groups facilitated by content or process experts. These data can inform faculty recruitment efforts when there is a shortage of content experts or where resource-intensive curricula create an increased demand for small group preceptors.

Chretien KC, Goldman EF, Craven KE, Faselis CJ. A qualitative study of the meaning of physical examination teaching for patients. J Gen Intern Med. 2010. 25(8):786–91.

This study sought to understand the meaning of physical examination teaching for patients. Semi-structured interviews of 12 hospitalized adults who participated in bedside teaching for third year medical students at the Washington, DC, VA were conducted 1 day to 1 week following contact with students. Questions assessed the experience of being examined and perceived benefits. Data were collected until thematic saturation was achieved. Interviews were audio-taped, transcribed verbatim and analyzed by two investigators using a phenomenological approach. Disagreements were resolved through discussion. Multiple coders, peer code checking and member checks were used to increase trustworthiness.

Patient experience themes included positive impressions of students, a belief that participation was part of the program at a teaching hospital, expectations that learning was a student’s job and having positive interactions with students. Physical exam teaching had four meanings: tolerance (no perceived benefit from the experience but willing to endure it), helping (willingness to help students learn), learning (experience contributed to their own learning about their medical conditions) and social (experience provided needed social interaction). Most experiences fell into categories of tolerance or helping. Suggestions for improving physical exam teaching included avoiding meal time, giving patients lead time, checking for patient comfort and student introductions.

The generalizability of these results is limited as participants were predominantly older men who had already agreed to participate in bedside teaching. Patients hold positive impressions of students, and bedside teaching has distinct meanings for them. Educators should employ strategies to increase comfort and enhance value for patients who participate in this important effort.

Oxentenko AS, West CP, Popkave C, Weinberger SE, Kolars JC. Time spent on clinical documentation: A survey of internal medicine residents and program directors. Arch Int Med 2010;170:377–80.

In light of decreased duty hours, this study measured the balance between clerical and direct patient care time and the potential educational value of documentation. A total of 21,371 residents taking the 2006 Internal Medicine In-Training Examination (IM-ITE) were asked about hours spent on documentation versus face-to-face direct patient contact during their most recent inpatient rotation, and the frequency and importance of feedback on documentation. Program directors (PDs) were asked about feedback on documentation and whether time spent on documentation detracted from other learning opportunities. A total of 15,417 residents (81%) and 263 (69%) PDs from 381 residency programs completed the survey. Sixty-eight percent of residents spent >4 h/day on documentation, but only 39% spent >4 h/day in direct patient contact. The majority of residents and PDs perceived that there was feedback on documentation <50% of the time. Seventy-three percent of PDs vs. 59% of residents perceived that feedback on documentation was important. Most (57%) PDs believed that time spent on clerical documentation detracts from other learning opportunities.

Times reported were resident estimates and likely imprecise; results might also have been different if focused on resident outpatient rather than inpatient work experience. That said, nationally Internal Medicine residents appear to spend more time on documentation than direct patient contact; their PDs believe this is detracting from resident learning. Given that available work hours are diminishing, future efforts should focus on interventions to displace, eliminate or shorten clerical tasks. Research into the value of feedback about documentation could reinforce the need to provide feedback more frequently.

Gordon JA, Alexander EK, Lockley SW, Flynn-Evans E, Venkatan SK, Landrigan CP et al. Does simulator-based clinical performance correlate with actual hospital behavior? The effect of extended work hours on patient care provided by medical interns. Acad Med 2010;85:1583–88.

Prior work from this institution showed more errors in a 24–30-h than a 16-h ICU shift. This study examined whether simulator performance is similarly affected by work hours. Seventeen PGY1 residents were tested during a 40-h/week ambulatory rotation and then retested after a 24–30-h ICU shift. A subset of eight of these residents (cohort 2) did a second study in which performance was retested during an ambulatory rotation and after a 16-h shift that began at 9:00 p.m. There were four potential cases; cohort 2 was tested on cases that they hadn’t done initially. Performance was measured with a previously validated tool assessing eight domains that were then averaged. Maximal score was 8; a score of 4 or less was “unsatisfactory.” Scoring was done in real time with a separate blinded videotape review with high inter-rater reliability between scores. In cohort 1, performance declined from 6.0 rested to 5.0 post 24–30-h shift (p < 0.001). In cohort 2, baseline rested scores of 6.6 declined to 5.8 after a 16-h overnight shift (p = 0.036). In cohort 2, performance after a 16-h night shift (5.8) was significantly better than performance after a 24–30-h shift (4.3 with p < 0.001). A higher proportion of interns had unsatisfactory scores after the 24–30-h shift (6 of 8 or 75%) than after the 16-h shift (3 of 8 or 38%).

This study used a small sample from a single institution. The selection of the previously tested cohort 2, who had higher scores when part of cohort 1 and more experience in simulation, may have biased the results towards better performance when tested after the 16-h shift. That said, this study does suggest better performance after a shift that coincides with new duty hour restrictions for PGY1 residents.

Schwartz A, Weiner SJ, Harris IB, Binns-Calvey A. An educational intervention for contextualizing patient care and medical students’ abilities to probe for contextual issues in simulated patients. JAMA 2010;304:1191–97.

Clinical decision making requires that a physician correctly identify the diagnosis and best management for a patient’s condition, but also that they understand individual patient circumstances and modify the plan of care if necessary (contextualization). A prior study of attending internists demonstrated more contextual than biomedical errors with unannounced standardized patients. This study aimed to increase fourth year medical students' skills in identifying patient context and decrease contextual errors. Intervention students had four weekly 1-h case-based sessions during subinternship on contextualizing patient care including domains of patient context, red flags, assumptions and errors. Control students were assessed with standardized patients (SPs), 3–10 days after the final session. With probing, standardized patients could reveal one of: no atypical features (baseline), either biomedical or contextual variants, or both variants. Plans were coded by an investigator blinded to student assignment and whether or not the student had probed the red flag.

One hundred twenty-four students completed 494 encounters. Both student groups probed biomedical red flags in 77% of encounters. Control students probed contextual red flags in 61% of encounters versus 86% for the intervention group. Intervention students were more likely to write an appropriate treatment plan in the contextual variant than control students (67% vs 24%). There were no significant differences between groups in management plans for the baseline, biomedical complexity or mixed biomedical/contextual complexity cases. In contextual cases, appropriate plans were written 4% of the time when the contextual red flag was not probed, 57% when control students probed and 71% when intervention students probed.

Specific training in contextualizing care can improve students’ ability to elicit specifics of that context and to modify treatment plans appropriately, though the durability of this intervention is unknown. The real world common combination of biomedical and contextual variation appears to be more complicated to teach.

Dyrbye LN, Massie FS Jr, Eacker A, Harper W, Power D, Durning SJ, Thomas MR et al. Relationship between burnout and professional conduct and attitudes among US medical students. JAMA. 2010 Sep 15; 304(11): 1173–80.

In a survey of 2,682 medical students at 7 US medical schools in 2009, the authors explored the association of burnout and personal distress with unprofessional behaviors and attitudes, hypothesizing that burnout is a more powerful influence on professional conduct and attitudes than personal distress. The survey contained the Maslach Burnout Inventory (MBI—the criterion standard for assessment of burnout), the PRIME-MD depression screening scale, and the SF-8 scales for mental and physical quality of life. The survey assessed professional behaviors and attitudes with questions about academic and clinical dishonesty; with a standardized instrument for attitudes toward providing care for underserved patients; and with questions derived from AMA Ethical Guidelines for interactions with industry. Sixty-one percent of students responded to the survey; 1.5% admitted academic cheating, while up to 44% admitted clinical dishonesty (e.g., falsifying omitted physical exam findings as normal). Students generally disagreed with AMA Guidelines but endorsed altruistic attitudes toward underserved patients. Fifty percent met study criteria for burnout. Students with burnout were significantly more likely to report one or more unprofessional behaviors than those without burnout (35.0% vs 21.9%; OR, 1.89; 95% CI, 1.59–2.24) and to have less altruistic views toward underserved patients. This effect was not seen with personal distress.

This study is limited by possible self-selection bias and use of self-reported rather than observed professional behaviors. Nonetheless, it does demonstrate a clear and specific relationship between burnout and reported unprofessional behavior and attitudes. Educators promoting students’ professionalism should take burnout into account when planning interventions and identifying students at risk.

Sah S, Loewenstein G. Effect of reminders of personal sacrifice and suggested rationalizations on residents' self-reported willingness to accept gifts: a randomized trial. JAMA 2010 Sep 15;304(11).

This experimental study examined the causal relationship between either (1) providing reminders of personal sacrifices in medical training or (2) suggesting rationalizations for accepting industry gifts, on the likelihood that residents would agree that accepting gifts from industry representatives was reasonable. The experimental manipulation consisted of a survey with questions in varying order. A total of 323 residents in pediatrics, obtained from one hospital and family medicine, obtained with the assistance of 450 program directors, were randomized into one of three groups: a “sacrifice reminders” group, in which they were asked about lack of sleep and income prior to being asked about attitudes toward gifts from industry; a “suggested rationalization” group, in which they were asked whether physicians are entitled to gifts because of financial sacrifices in training, prior to their own attitudes being ascertained; and a control group, in which their attitudes toward gifts were measured before the other questions were asked. The questions were embedded in a larger survey about resident quality of life; the residents were blinded to the experimental manipulation. Overall, 37.5% of residents agreed that accepting gifts from industry was reasonable. The rate of agreement was greatly increased in the sacrifice reminders group and in the suggested rationalization group, an effect independent of residents’ baseline views of their working conditions.

This study is limited by differences in recruitment strategy between family medicine and pediatrics residents. Family medicine residents were offered a chance for a media player for participation in the study. In addition, levels of resident debt were not assessed and may have confounded the outcome. While it is well known that question order influences survey responses, this study shows how potent this effect is for the specific issue of residents’ views of industry gifts. Educators should be aware of these messages as they encourage more appropriate views in residents.

Dyrbye LN, Power DV, Massie FS, Eacker A, Harper W, Thomas MR, Szydlo DW, et al. Factors associated with resilience to and recovery from burnout: a prospective, multi-institutional study of US medical students. Med Educ 2010 Oct; 44(10):1016–26.

The authors undertook paired surveys in 2006 and 2007 for all students in five US medical schools to assess the association between learning environment, social support and stress, on resiliency to burnout, and the tendency to recover from it. Both surveys contained the MBI, PRIME-MD and SF-8 scales; the second survey also included scales for sleepiness and stress. Survey items also examined demographic factors, personal life events and aspects of the learning environment. Data were only used when students responded to both surveys. Students were classified as “resilient” if not burned out on either survey, “vulnerable” if burned out on at least 1 survey, “chronically burned out” if burned out on both surveys, and “recovered” if burned out on the first but not the second survey. Response rates of 55% and 65% over the two surveys resulted in 792 students with paired data. Thirty-seven percent of students were resilient, and 63% were vulnerable to burnout. Fifty-four percent of vulnerable students were chronically burned out. Resiliency was reduced by employment outside medical school and with each major life event reported. Resiliency was improved with family support, faculty support and positive learning environment. Recovery from burnout was reduced by fatigue, stress and sleep deprivation, and improved by faculty support and positive learning environment.

This study is potentially limited by self-selection bias, as it is unclear whether students with burn out would be more or less likely to respond to the survey. It does provide significant data to assist educators in planning supports and interventions in medical school to assist students at risk of burnout. For example, improving levels of faculty support, developing a positive learning environment and discouraging student employment outside of medical school may prevent burnout and improve the chance of recovery for those in whom it develops.

Farnan JM, Johnson JK, Meltzer DO, Harris I, Humphrey HJ, Schwartz A, Arora VM. Strategies for effective on-call supervision for internal medicine residents: the superb/safety model. J Grad Med Educ 2010; 46–52.

This single institution study used qualitative methodology to develop a model to guide resident supervision. The authors used critical incident technique and appreciative inquiry in interviews with Internal Medicine residents and attendings to discover shared themes around the challenges of effective clinical supervision. Authors performed qualitative analysis of interview transcripts from 44 (88%) attending physicians and 46 (92%) residents at the conclusion of the general medicine rotation. Analysis of shared themes between physicians and residents led to an optimal supervision model mnemonic of SUPERB (for attendings) and SAFETY (for residents) (see Table 1).

Table 1.

SUPERB and SAFETY Mnemonics; Numbers Connote Number of Times Mentioned in Interviews

SUPERB: model for attending physicians providing supervision	SAFETY: model for resident physicians seeking supervision
Set expectations for when to be notified (41)	Seek attending physician input early (28)
Uncertainty is a time to contact (68)	Active clinical decisions (39)
Planned communication (36)	Feel uncertain about clinical decisions (40)
Easily available (48)	End-of-life care family/legal discussions (46)
Reassure resident not to be afraid to call (50)	Transitions of care (43)
Balance supervision and autonomy for resident (46)	Help with the sYstem/ hierarchy (22)

Further research to demonstrate the value of this model in improving resident and attending satisfaction with supervision, as well as improving outcomes of patients cared for by resident/attending teams should be performed. In the meantime, programs may adopt this model for supervision if one is not already in place. Adopting a jointly agreed upon set of expectations in the format of an easy mnemonic may enhance communication between attendings and residents, and reduce hesitation by residents in requesting supervision that ensures patient safety and adequate training.

Farnan JM, Paro JA, Rodriguez RM, Reddy ST, Horwitz LI, Johnson JK, Arora VM. Hand-off education and evaluation: piloting the observed simulated hand-off experience (OSHE). J Gen Intern Med. 2010 Feb;25(2):129–34.

This study describes the creation of an Observed Simulated Hand-off Experience (OSHE) to evaluate medical student hand-off skills using a real-time assessment tool, the hand-off CEX. Thirty-two fourth year medical students enrolled in a residency preparation elective at the University of Chicago participated in a 90-min interactive session focused on provision of high-quality sign-outs. One week after the workshop, students participated in a 2-h standardized hand-off experience. They reviewed a mock history and exam write-up of a patient admitted with pneumonia and viewed a 5-min video of interval events. Students subsequently developed a written sign-out and delivered an oral sign-out to standardized resident receivers. Residents used a hand-off CEX to evaluate the students. A single faculty reviewer assessed the written sign-out on a 20-point scale.

Standardized resident receivers rated overall student performance with a mean score of 6.75 (range 4–9, maximum 9). Faculty review of the written sign-out revealed a mean score of 16.2 (range 0–20, maximum 20). The most frequent omission was lack of anticipatory guidance to the covering physician. Prior to intervention 27% of students self-assessed as prepared for actual patient sign-out and post-intervention 67% of residents reported feeling prepared (p < 0.009).

This study has limitations, including no control group, lack of pre-post assessment of skill and lack of validated hand-off CEX. However, it describes a novel method, the OSHE, for teaching effective hand-off skills to medical students. Further efforts should focus on establishing validity and reliability of the hand-off CEX and the inter-reliability of receivers. The assessment of the OSHE in a multi-institutional study with use of a randomized control design needs to be undertaken. The OSHE and hand-off CEX are intriguing tools for training purposes and in-hospital hand-offs in need of further study.

Gakhar B, Spencer AL. Using direct observation, formal evaluation, and an interactive curriculum to improve the sign-out practices of internal medicine interns. Acad Med. 2010 Jul; 85(7):1182–8.

This study describes a baseline assessment of sign-out at a single institution and subsequent use of findings to develop, implement and then re-assess a curriculum and system change. The curriculum consisted of a 30-min didactic and a 30-min interactive component. The system change involved the implementation of a web-based program for creating sign-out sheets. Assessment included direct observation of oral sign-out and review of written sign-out for completeness and accuracy. The pre- and post-assessment included direct observation of oral sign-out prior to the curriculum and then 8 weeks after curriculum by two trained residents using a checklist. The written sign-out was evaluated for completeness and accuracy. Completeness was determined using an eight-item checklist, and accuracy was assessed through comparison of the sign-out sheet with the patient record within 30 min of the sign-out.

Following the curriculum, interns orally reported all seven items in the SIGN-OUT * mnemonic more frequently (p < 0.001 for 6 items; p < 0.02 for 1 item). The percentage of complete written sign-out sheets rose from 16% to 70% (p < 0.001). The accuracy of written sign-out also improved in three of four areas (identification data 64% vs 89%; code status 82% vs 100%; and medication list 4% vs 79%; all p < 0.001) with no change in allergy list (96% vs 82%; p = 0.82).

This study suggests that a needs-based focused curriculum along with implementation of a web-based written sign-out system improves the quality of both oral and written sign-out. With the increase in number of hand-offs due to duty hour regulations, institutions may benefit from using a similar process to improve accuracy and completeness of communication among team members.

*Yale University SIGN-OUT mnemonic
S: Sick or “Do not resuscitate” status
I: Identification data
G: General hospital course
N: New events of the day
O: Overall Health
U: Upcoming possibilities
T: Tasks to do

*Horowitz L. Moin T. Green M. Development and implementation of an oral sign-out skills curriculum. J Gen Intern Med 2007; 22:1470–1474.