Abstract
Background
The completeness of hospital discharge summaries may reflect the overall quality of inter-professional communications. The effect of reducing resident workload on clinical performance is largely unknown.
Objective
We examined the impact of reducing housestaff workload on the quality of discharge summaries, an indicator of clinical performance, as compared to discharge summaries created by trainees on a team with a more typical workload.
Design
Patients admitted to a medicine service at a community teaching hospital affiliated with an academic medical center were alternately allocated between a control and intervention care team of residents. First-year residents on the intervention team carried an average census of 3.5 patients compared with 6.6 patients for peers on the control team. A random selection of 142 discharge summaries from a 3-month period were blindly evaluated using a structured tool based on professional documentation standards.
Participants
61 internal medicine residents.
Main Results
Inter-rater reliability of discharge summary quality was 0.9. Discharge summaries produced by residents on the intervention team with a reduced workload had significantly more of the required elements (74 vs 65%, p < 0.001). Intervention team summaries were more frequently complete and contained significantly more of the required elements describing the patient history (65.7% vs 36.1%, p = 0.0005), the inpatient narrative (47.1% vs 22.2%, p = 0.003), discharge planning (20.0% vs 5.5%, p = 0.012), and continuity of care (24.3% vs 6.9%, p = 0.005). Fewer than a quarter of the summaries reviewed included discharge instructions, information on follow-up care, or a discharge medication list.
Conclusions
Reducing resident workload can significantly improve discharge summary quality, a measure of resident performance.
Electronic supplementary material
The online version of this article (doi:10.1007/s11606-010-1465-z) contains supplementary material, which is available to authorized users.
KEY WORDS: discharge summary, residency design, work hours
BACKGROUND
Residency programs are under pressure to control the work hours and workload of house staff; however, the effects of workload reductions on resident performance outcomes are largely unknown. Methods to evaluate clinically meaningful intermediary outcomes in resident performance are needed. Our study speculated that discharge summaries, which are an independently valuable work product, may provide insight into resident performance.
Almost half of hospitalized patients will experience at least one medical error related to medication continuity or follow-up care1. Twenty percent of patients experience an adverse event of some kind soon after discharge; half of these events are preventable2. Improved communication between inpatient and primary care providers can prevent these errors and events3, and discharge summaries are key components of these information transfers. Deficiencies in the content, template or delivery of discharge summaries contribute to poor quality of care, poor follow-up, adverse events and medical error4–7. Interventions to improve the quality of discharge summaries have focused on the modernization of the production of the summaries8, standardization of content through didactic interventions9, templating or providing more explicit guidelines for use by residents10. Each of these can be at least partially successful3.
The burden of responsibility for discharge summaries in academic medical centers generally falls on less experienced residents, who are key participants on which to focus improvement interventions8. Residents are particularly vulnerable to lapses that may result from time demands, workload and exhaustion11. Pressure from work hour restrictions imposed by ACGME has required a reconsideration of time distribution and resident activities12. Additional focus on quality of care, as initiated by the Institute of Medicine’s report, demands closer attention to documentation, medical records and quality in transitions of care, including discharge summaries13. This study sought to investigate whether workload reductions could result in improved discharge summary quality and thereby estimate whether workload was a root cause of poor-quality discharge summaries.
METHODS
Development of the Instrument
The Joint Commission standards require discharge summaries to contain, at a minimum, the reason for hospitalization, significant examination and/or test findings, procedures performed, treatments provided and responses to treatments, services provided, patient condition upon discharge, and instructions to the patient and family14. Additionally, primary care physicians (PCPs) value summaries that are brief and focused15. Combining these elements, and after evaluating components used in prior studies10,16, the authors developed an assessment tool that would measure the presence of specific discharge elements and whether the elements were clear, thorough, and contained the necessary details for other clinicians to provide informed and high-quality follow-up care. Our tool placed the greatest weight on those elements required by the Joint Commission and heavily favored by PCPs in prior studies10,16 (see Online Appendix A). Two primary care physicians confirmed the face and content validity of the instrument.
Summaries were scored for the presence and adequacy of these specific elements. Prior to scoring, a rubric was created in collaboration by the reviewers dictating the minimal standards for an item to be awarded the highest score, describing model characteristics for each element (see Online Appendix B). Most components (e.g., principal diagnosis, allergy information) were assessed as present (score = 1) or omitted (score = 0); others were assessed for “adequacy” as indicated by the relevance and thoughtfulness of the content. Laboratory results that were listed without interpretation would be scored as 1, but those that were tied to the patient history or clinical decision-making were scored as 2. If the history of present illness was included but failed to account for the entirety of the illness up to the point of hospitalization, omitting the timeline, actions taken by providers prior to hospitalization, or other information provided to the inpatient physician upon admission, scores were similarly affected. The score assigned by the reviewer was then weighted by the factor assigned to the element based on Joint Commission standards and value to primary care physicians and aggregated for analysis. Aggregate scores were then calculated in percentage of possible points earned. This scoring system is necessarily limiting to the evaluation of the written content; however, although it accounts for the presence and quality of information, there was no way of ascertaining the accuracy of the content.
Study Design
We applied a new model of inpatient care to half of the medical service at a community teaching hospital affiliated with an academic medical center internal medicine residency program. A description of the intervention has been published17. Though both models were in compliance with ACGME requirements, characteristics of the redesigned model of care for the two intervention teams include capped admissions (maximum of five patients per resident) and a reduced call schedule (1:6). The two control teams of the medical service operated with a traditional cap as defined by the ACGME (up to 12 patients) and a typical 1:4 call schedule18. Work hours were monitored by resident self-report using a standardized survey completed weekly. Five housestaff (3 interns and 2 upper-level residents) on the intervention team cared for the same number of patients as three housestaff on the control team.
Patients not assigned to the Physician Assistant service based on low acuity were alternately allocated between the intervention and control teams for the duration of the study; housestaff were randomly assigned between services. Discharge summaries were selected for review using a random number generator. The protocol was approved by the hospital’s Institutional Review Board.
No additional training on discharge communication was provided to members of the intervention or control groups, and the hospital’s discharge summary creation process remained unchanged during the 3-month study. The summary was dictated by the resident and transcribed by an outsourced service, before being made available to the resident for editing and then sent to the attending. For expediency, the transcription was mailed to the primary care physician listed in the dictation before being signed by the attending physician. Upon signature by the attending of record, the summary was entered into the medical record. Upon signature by the attending of record, the summary was entered into the medical record. Resident and faculty participants were unaware of the tracking of this outcome measure during the study period.
Data Analysis
After testing of the evaluation tool with a sample of discharge summaries, a sample size of 64 records per group was estimated to be necessary to detect a 5-point difference in mean summary score assuming a 10-point standard deviation. Discharge summaries representing 10.3% of all discharged patients (n = 142) produced by 61 different residents over 3 months of rotations were randomly selected for evaluation by two blinded reviewers. Patients who expired while hospitalized were excluded from the study.
Inter-rater reliability for the assessment tool was calculated after 16 discharge summaries were reviewed in tandem by two independent reviewers (MC, GM).
Elements were clustered by practical use and cohesive themes including the patient histories (incorporating present illness as well as family, social and medical histories); medication management (including elements on medication reconciliation and documentation); discharge planning (including information on discharge medications, follow-up appointments, follow-ups, “to-do” lists, and instructions for patients); the inpatient narrative (incorporating descriptions of the care received while at the hospital, especially the hospital course); and continuity of care (including documentation of consults, follow-up appointments, and interactions with other caretakers and providers). The percentage of discharge summaries with adequate content in these general categories produced by the intervention group was compared with the percentage produced by the control group using Fisher’s exact test. Differences in the scores between the intervention and control groups were measured using the Wilcoxan rank sum test. Statistical significance was implied at p < 0.05. Analyses were performed using Stata version 9 (College Station, TX).
RESULTS
The evaluation tool used performed well; it yielded consistently high inter-rater reliability with a kappa statistic of 0.90.
Overall, the reviewed discharge summaries frequently missed essential components of a high-quality discharge summary: 94 of 142 (66%) summaries reviewed clearly stated the patient’s principal diagnosis, and 27 of 142 (19.0%) of summaries reviewed included any mention of the instructions or information given to patients upon discharge. Of those summaries that indicated a consult had been obtained, 29.1% (25 of 86) included the name of the consulting physician. Discharge activity elements were most frequently missing. Of the summaries reviewed, 32 of the 142 (23%) contained no information on follow-up care. Similarly, 92 summaries (65%) contained no or insufficient explanation for changes made to the medication list. Twenty-three summaries (16%) had no discharge medication list.
There was no difference in case mix between patients randomized to each service (data not shown). There was no significant difference in the number of hours worked per week by control and intervention residents during the study (68.7 vs 66.5, p = 0.66) according to resident self-reporting. The residents on control teams, however, discharged a median of 11 patients per week, whereas those on intervention teams discharged a median of 6 patients in the same time span. There was no statistically significant difference in the average length of discharge summaries by word count (1,148.3 vs 1,124.6, p = 0.736).
Reduced resident workload within the new care model was associated with a significantly higher score on the discharge summary evaluation tool. Discharge summaries produced by house staff within the intervention teams (n = 70) had an average score of 74.3% of all possible points, compared with 65.1% for summaries from control teams (n = 72) (p < 0.0001).
Scores for subsections of the discharge summary focusing on admission information (80.7% vs 71.3%, p = 0.0001) and discharge information (60.5% vs 51.0%, p = 0.003) were significantly higher for patients discharged from the residents with reduced workload. The average score for the section on the patient’s hospital course was also numerically higher for patients in the new team model (87.1% vs 81.3%, p = 0.42), but the difference was not statistically significant (Table 1).
Table 1.
Discharge Summary Score Median and Interquartile Range by % of Total Possible Points
| Intervention (N = 70) | Control (N = 72) | P value* | |
|---|---|---|---|
| Average length of stay (in days) | 4.1 | 4.5 | |
| Median length of discharge summary (in words) | 1,124 | 1,148 | 0.736 |
| Overall score | 74.3% (14.0%) | 65.1% (15.2%) | <0.0001 |
| Section scores | |||
| Admission | 80.7 (16.7) | 71.3 (20.8) | <0.0001 |
| Hospital course | 87.1 (16.7) | 81.3 (16.7) | 0.424 |
| Discharge planning | 60.5 (25.0) | 51.0 (26.8) | 0.003 |
*P values calculated using Wilcoxon rank sum test
The traditional and intervention teams’ discharge summaries differed when the information was clustered by thematic group. We subsequently re-categorized the information gathered using cohesive themes of discharge functions rather than the chronological course of admission, hospital course and discharge as the scoring rubric. The fraction of discharge summaries that contained the necessary information was higher for clusters incorporating the patient histories (65.7% vs 36.1%, p = 0.0005), discharge planning (20.0% vs 5.5%, p = 0.012), the inpatient narrative (47.1% vs 22.2%, p = 0.003), continuity of care (24.3% vs 6.9%, p = 0.005), and medication management (25.7% vs 15.2%, p = 0.147). (Table 2)
Table 2.
Fraction of Discharge Summaries that Contained All of the Required Elements within each Theme
| Intervention (N = 70) | Control (N = 72) | P value* | |
|---|---|---|---|
| Patient histories | |||
| History of present illness, past medical history, social history | 46 (65.7%) | 26 (36.1%) | 0.0005 |
| Medication-related data | |||
| Pre-admission medications, allergy identification, discharge medications, explanation for changes in medications | 18 (25.7) | 11 (15.2) | 0.147 |
| Discharge planning | |||
| Discharge condition, discharge medications, pending results, follow-up plan | 14 (20.0) | 4 (5.5) | 0.012 |
| Inpatient narrative | |||
| Principal diagnosis, results of physical examination, ancillary test results, treatments and treatment responses | 33 (47.1) | 16 (22.2) | 0.003 |
| Continuity of care | |||
| History of present illness, consultant type and name, pending test results, follow-up plans | 17 (24.3) | 5 (6.9) | 0.005 |
*P values calculated using Fisher’s exact test
There were two incomplete summaries among in the sampled summaries from the intervention group and eight incomplete summaries in the control group sample (p = 0.10).
DISCUSSION
The completeness of hospital discharge summaries may reflect the overall quality of inter-professional communications. This study demonstrates that a programmatic change that reduces workload can improve processes of care, as measured by discharge summary quality.
Residents consistently report that high-quality summaries are time consuming and report a conflict between quality and exigency12,19. Few studies have directly addressed the discharge summary and communication with PCPs in the context of the typical residents’ workload. A survey of senior residents in Britain revealed a lack of priority placed on discharge summary quality, attributable to reduced prioritization from attendings15. The discharge summaries evaluated for this study were of low overall quality, demonstrating that efforts to improve these documents are warranted. While the causes of the poor quality are unknown, this study indicates that reduced workload and improved supervision may lead to improved communication quality.
There is increased research interest in how the ACGME-mandated limitations on time may affect the quality of patient care and resident education20–22. Reducing duty hours without workload reductions would be predicted to result in poorer communication. This study was conducted to test the impact of reducing workload within the constraints of duty-hour restrictions. The intervention and control groups worked a similar number of hours per week despite a lower patient census, leaving only the distribution of this time as a variable. Since discharge summaries generated by house staff within the revised rotation were of higher quality than those produced by teams following the traditional schedule and workload, our findings suggest that given a constrained number of hours to conduct clinical care and education, reducing resident workload may contribute to an overall improvement in task performance.
There are several factors that may explain higher quality discharge summaries on the intervention team. The decreased patient load may have allowed for more protected time to dictate these documents, in addition to allowing for more reflection and thorough preparation. Increased opportunity for reflection regarding daily patient care may have improved the residents’ attention to relevant details: the greatest difference in aggregate mean between the control and intervention teams was found in the section on discharge planning. The difference in completion rate may be attributable to interruptions during dictation, supporting the hypothesis that the house staff on the intervention teams produced higher quality discharge summaries because of reduced time constraints. Through the combination of additional reflective time and greater time for faculty interaction, residents on the intervention team may have gained a fuller appreciation of the natural history of common medical conditions and therefore were cognizant of the specific requirements of follow-up management.
The differences are unlikely to be explained by variability in resident competency since the discharge summaries evaluated were produced by 61 different residents over three 4-week rotations; there was no difference in case mix between patients randomized to each service. Interventions to improve discharge summaries most frequently focus explicitly on discharge summary production and delivery process. While this approach does allow for better control of confounding variables in research design, this study broadens the intervention opportunities and quantifies the effect that could be expected from programmatic change.
Our study was limited by several factors. To maintain reviewer blinding, it was necessary to assume that whatever information was included in the discharge summary was accurate. Additionally, the results are limited by their specificity to our single center and evaluation tool, as well as the constrained time period. The discharge summaries evaluated were dictated rather than produced using the templated, computer-based systems that are increasingly common. Consequently, our findings should be replicated at other centers and for other types of programs to enhance its validity.
Though the practical purpose of discharge summaries is readily apparent, they may have a broader function in graduate medical education. The process of creating a discharge summary may, for example, encourage reflection and analysis on the part of the resident, leading to the synthesis of new knowledge. Discharge summaries necessitate that the writer reflect on the next steps for the patient and may augment the attention given to continuity of care and the importance of inter-professional communication. Further research into the value of this documentation in training physicians is warranted.
The results indicate that attention to resident workload may have important implications for overall quality of discharge summaries, resident performance, and potentially for patient care. The findings also reveal significant opportunities for improved education in effectively managing communication during transitions in care for all trainees.
ELECTRONIC SUPPLEMENTARY MATERIAL
Below is the link to the electronic supplementary material.
(PDF 33 kb)
Acknowledgments
We thank Mary Thorndike, MD; Bruce Levy, MD; Joseph Loscalzo, MD, PhD; Maia Laing; members of the Residency Redesign Committee, the medical residents, and the medical, nursing and health information staff at the Faulkner Hospital. The study was supported by internal funding from the Brigham and Women’s Hospital Department of Medicine.
Conflict of Interest None disclosed.
References
- 1.Moore C, Wisnivesky J, Williams S, McGinn T. Medical errors related to discontinuity of care from an inpatient to an outpatient setting. J Gen Intern Med. 2003;18(8):646–651. doi: 10.1046/j.1525-1497.2003.20722.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. The incidence and severity of adverse events affecting patients after discharge from the hospital. Ann Intern Med. 2003;138(3):161–167. doi: 10.7326/0003-4819-138-3-200302040-00007. [DOI] [PubMed] [Google Scholar]
- 3.Kripalani S, Jackson AT, Schnipper JL, Coleman EA. Promoting effective transitions of care at hospital discharge: a review of key issues for hospitalists. J Hosp Med. 2007;2(5):314–323. doi: 10.1002/jhm.228. [DOI] [PubMed] [Google Scholar]
- 4.O'Leary KJ, Liebovitz DM, Feinglass J, Liss DT, Baker DW. Outpatient physicians' satisfaction with discharge summaries and perceived need for an electronic discharge summary. J Hosp Med. 2006;1(5):317–320. doi: 10.1002/jhm.118. [DOI] [PubMed] [Google Scholar]
- 5.Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW. Deficits in communication and information transfer between hospital-based and primary care physicians: implications for patient safety and continuity of care. JAMA. Feb 28 2007;297(8):831-841. [DOI] [PubMed]
- 6.Balla JI, Jamieson WE. Improving the continuity of care between general practitioners and public hospitals. Med J Aust. Dec 5-19 1994;161(11-12):656-659. [PubMed]
- 7.Wilson S, Ruscoe W, Chapman M, Miller R. General practitioner-hospital communications: a review of discharge summaries. J Qual Clin Pract. 2001;21(4):104–108. doi: 10.1046/j.1440-1762.2001.00430.x. [DOI] [PubMed] [Google Scholar]
- 8.van Walraven C, Weinberg AL. Quality assessment of a discharge summary system. CMAJ. May 1 1995;152(9):1437-1442. [PMC free article] [PubMed]
- 9.Myers JS, Jaipaul CK, Kogan JR, Krekun S, Bellini LM, Shea JA. Are discharge summaries teachable? The effects of a discharge summary curriculum on the quality of discharge summaries in an internal medicine residency program. Acad Med. 2006;81(10 Suppl):S5–8. doi: 10.1097/01.ACM.0000236516.63055.8b. [DOI] [PubMed] [Google Scholar]
- 10.Rao P, Andrei A, Fried A, Gonzalez D, Shine D. Assessing quality and efficiency of discharge summaries. Am J Med Qual. 2005;20(6):337–343. doi: 10.1177/1062860605281078. [DOI] [PubMed] [Google Scholar]
- 11.Haney EM, Nicolaidis C, Hunter A, Chan BK, Cooney TG, Bowen JL. Relationship between resident workload and self-perceived learning on inpatient medicine wards: a longitudinal study. BMC Med Educ. 2006;6:35. doi: 10.1186/1472-6920-6-35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Accreditation Council for Graduate Medical Education. Policies and Procedures. http://www.acgme.org/acWebsite/about/ab_ACGMEPoliciesProcedures.pdf. Accessed July 15th 2010.
- 13.Committee on Optimizing Graduate Medical Trainee (Resident) Hours and Work Schedules to Improve Patient Safety. Resident Duty Hours: Enhancing Sleep, Supervision, and Safety. Washington, DC: Institute of Medicine; 2008.
- 14.Standard IM 6.10: Hospital Accreditation Standards. Oakbrook Terrace, IL: Joint Commission on Accreditation of Healthcare Organizations; 2006:338-340.
- 15.Walraven C, Rokosh E. What is necessary for high-quality discharge summaries? Am J Med Qual. Jul-Aug. 1999;14(4):160–169. doi: 10.1177/106286069901400403. [DOI] [PubMed] [Google Scholar]
- 16.Pantilat SZ, Lindenauer PK, Katz PP, Wachter RM. Primary care physician attitudes regarding communication with hospitalists. Am J Med. Dec 21 2001;111(9B):15S-20S. [DOI] [PubMed]
- 17.McMahon GT, Katz JT, Thorndike ME, Levy BD, Loscalzo J. Evaluation of a redesign initiative in an internal-medicine residency. N Engl J Med. Apr 8;362(14):1304-1311. [DOI] [PubMed]
- 18.Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Residency Education in Internal Medicine 2007.
- 19.Frain JP, Frain AE, Carr PH. Experience of medical senior house officers in preparing discharge summaries. BMJ. Feb 10 1996;312(7027):350. [DOI] [PMC free article] [PubMed]
- 20.Horwitz LI, Kosiborod M, Lin Z, Krumholz HM. Changes in outcomes for internal medicine inpatients after work-hour regulations. Ann Intern Med. Jul 17 2007;147(2):97-103. [DOI] [PubMed]
- 21.Ogden PE, Sibbitt S, Howell M, et al. Complying with ACGME resident duty hours restrictions: restructuring the 80-h workweek to enhance education and patient safety at Texas A&M/Scott & White Memorial Hospital. Acad Med. 2006;81(12):1026–1031. doi: 10.1097/01.ACM.0000246688.93566.28. [DOI] [PubMed] [Google Scholar]
- 22.Fletcher KE, Davis SQ, Underwood W, Mangrulkar RS, McMahon LF, Jr., Saint S. Systematic review: effects of resident work hours on patient safety. Ann Intern Med. Dec 7 2004;141(11):851-857. [DOI] [PubMed]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Below is the link to the electronic supplementary material.
(PDF 33 kb)