Audit and feedback: an intervention to improve discharge summary completion

Abstract

Discharge summaries (DS) communicate important clinical information from inpatient to outpatient settings. Previous studies noted increased adverse events and rehospitalization due to poor DS quality. We postulated that an audit and feedback intervention of DS completed by geriatric medicine fellows would improve the completeness of their summaries. We conducted a preintervention post intervention study. In phase 1 (AUDIT #1 and FEEDBACK) we scored all DS (n ¼ 89) completed by first year fellows between July 2006 to December 2006 using a 21-item checklist. Individual performance scores were reviewed with each fellow in 30-minute feedback sessions. In phase 2 (AUDIT #2) we scored all DS (n ¼ 79) completed after the first phase between February 2007 to July 2007 using the same checklist. Data were analyzed using generalized estimating equations. Fellows were more likely to complete all required DS data after feedback when compared with prior to feedback (91% vs. 71%, P < 0.001). Feedback was also associated with improved admission (93% vs. 70%, P < 0.001), duration of hospitalization (93% vs 78%, P < 0.001), discharge planning (93% vs. 18%, P < 0.02) and postdischarge care (83% vs. 57%., P < 0.001) section-specific information. In conclusion, audit and feedback sessions were associated with better DS completeness in areas of particular importance to geriatric care.

Introduction

Discharge summaries correlate with rates of re-hospitalization (1, 2) and adverse events after discharge (3). The Joint Commission on the Accreditation of Healthcare Organizations acknowledges their importance and mandates that certain elements be included (4). Thus far, however, discharge summaries are not standardized across institutions and there is no expectation that they be available at post discharge visits. There have been numerous attempts to improve the quality of discharge summaries by using more structured formats or computer generated summaries with positive results in term of comprehensiveness, clarity and practitioner satisfaction (5, 6, 7, 8) but with persistence of serious errors and omissions (9).

Post-graduate training is often the first opportunity for physicians to learn information transfer management skills. Unfortunately, discharge summaries are created by house-staff who have minimal training in this area (11) and feel like they have to learn “by osmosis” (12), resulting in poor quality discharge summaries and lack of availability at the point of care (13, 14, 15).

Previous research suggested that individualized feedback sessions for Internal Medicine residents improved the quality of certain aspects of their completed discharge summaries (10).We postulated that an “audit and feedback” educational intervention on discharge summaries for first year geriatric medicine fellows would also improve their quality. This technique involves chart or case review of clinical practice behaviors for a specific task followed by recommendation of new behaviors when applicable (16). “Audit and feedback” incorporates adult learning theory (17, 18, 19), an essential part of continuous quality improvement that fits within the ACGME competency of practice based learning and improvement (20), as an educational activity.

Methods

Setting

We conducted a pre-post intervention study at the Brookdale Department of Geriatrics and Palliative Medicine at Mount Sinai Medical Center (MSMC) in New York City between July 1, 2006 and June 30, 2007. The study received an exemption from the MSMC Institutional Review Board. First year geriatric medicine fellows at MSMC were required to complete two months of inpatient service; the first during the first six months of the academic year and the second during the last six months of the year. Fellows dictated all discharge summaries, which were transcribed and routed for signature to the attending of record. Prior to our study, a discharge summary template consisting of 21 items was developed for clinical use. Template items, agreed upon by an expert internal panel of geriatricians and interprofessional faculty, were selected for their importance in assuring a safe transition of older adults from the inpatient to the outpatient setting. .

Participants

All five first year fellows at the Brookdale Department of Geriatrics and Palliative Medicine at MSMC were invited to participate in the study.

Intervention

Audit #1

All available discharge summaries for each fellow’s first month of inpatient service were audited for completeness of the 21 item discharge summary template by one author (AD) The 21 items were focused on 4 distinct periods of the hospitalization: admission, hospital course, discharge planning, and post-discharge care (Figure). Content under each of the 21 items was classified as complete, partially complete or absent. An item was considered complete if most information was present and appropriate medical terms were used, partially complete if information was unclear and absent if no information was present for that area of the discharge summaries. To ensure investigator reliability, a random sample of 25% of each fellow’s discharge summaries was scored by two additional investigators (RK and HF) and all disagreements were reviewed and resolved by consensus.

Feedback

Between December 2006 and January 2007, one-on-one formative feedback sessions were scheduled. The sessions were approximately 30 minutes long, confidential, performed by one of the authors (AD) and followed a written format. During these sessions, each fellow received the results of their discharge summary audit, each partially complete or absent item was discussed, and the importance of discharge summaries was emphasized.

Audit #2

All available discharge summaries for each fellow’s second month of inpatient service were audited for completeness, using the same 21 item assessment tool and the same scoring system.

Statistical Analysis

To determine the impact of our audit and feedback intervention, we compared scores before and after formative feedback sessions, both overall and for the composite discharge summary scores for each of the four domains of care: admission, hospital course, discharge-planning, and post-discharge care. Scores were dichotomized as being ‘complete’ or ‘partially complete or absent’. We used generalized estimating equations to account for the clustering of discharge summaries within fellows. Analyses were performed using SAS 9.1 (SAS Institute, Inc., Cary, NC). All statistical tests were 2-tailed and used a type I error rate of 0.01 to account for multiple comparisons.

Results

Five fellows participated, four of whom were women; two were in post-graduate year four, three in year five. A total of 158 discharge summaries were audited, 89 pre-feedback and 79 post-feedback. Each fellow dictated an average of 17 discharge summaries during each inpatient month.

During Audit #1, the 21 item discharge summaries were complete among 71%, incomplete among 18%, absent among 11%. Admission items, hospital course items and discharge planning items were complete among 70%, 78% and 77% of discharge summaries respectively, but post-discharge items were complete among only 57%. Examining individual items, the lowest completion rates were found for test result follow-up (42%), caregiver information (10%) and home services (64%), as well for assessment at admission and discharge of cognitive and mental status (56% and 53% respectively) and functional status (57% and 40%). Of note, all these items are of particular importance to geriatric care.

After receiving the audit and feedback intervention, fellows were more likely to complete all required discharge summary data when compared to prior-to-feedback (91% vs.71%, P<0.001). Discharge summary completeness improved for all composite outcomes examining the four domains of care: admission (93% vs. 70%, P<0.001), hospital course (93% vs. 78%, P<0.001), discharge planning (93% vs. 77%, P<0.02), and post-discharge care (83% vs. 57%., P<0.001). (Table)

TABLE.

Mean proportion of discharge summaries pre- and post- audit and feedback intervention that were complete or absent.

CRITERIA	PRE-INTERVENTION		POST-INTERVENTION
	COMPLETE, % (range)	ABSENT, % (range)	COMPLETE, % (range)	ABSENT, % (range)	P value*
ADMISSION COMPOSITE (5 ITEMS)	70 (35–85)	30 (15–65)	93 (79–100)	7 (0–21)	<0.001
HPI	79 (38–100)	21 (15–63)	100	0	<0.001
PMH	94 (75–100)	5 (0–25)	99 (93–100)	1 (0–7)	<0.001
COGNITIVE/MENTAL STATUS	56 (19–79)	44 (21–82)	99 (93–100)	1 (0–7)	<0.001
FUNCTIONAL STATUS	57 (25–88)	43 (13–75)	97 (89–100)	2 (0–10)	0.001
PHYSICAL EXAM	63 (19–100)	37 (0–82)	72 (0–100)	28 (5–100)	0.27
HOSPITAL COURSE COMPOSITE (3 ITEMS)	78 (25–93)	22 (7–75)	93 (76–100)	7 (0–23)	<0.001
HOSPITAL COURSE	84 (25–100)	15 (0–76)	99 (93–100)	1 (0–7)	<0.001
PROCEDURES AND TESTS	70 (6–90)	30 (10–94)	90 (57–100)	10 (0–43)	<0.001
COMPLICATIONS	80 (44–90)	20 (5–56)	90 (77–100)	10 (0–23)	0.07
DISCHARGE PLANNING COMPOSITE (8 ITEMS)	77 (49–89)	22 (11–51)	93 (64–100)	7 (0–36)	0.02
PRIMARY DIAGNOSIS	93 (75–100)	6 (0–26)	100	0	0.03
SECONDARY DIAGNOSIS	82 (56–100)	18 (0–44)	100	0	0.002
OVERALL CONDITION	81 (38–100)	19 (0–62)	86 (21–100)	14 (0–79)	0.47
COGNITIVE/MENTAL STATUS	53 (13–80)	57 (20–88)	97 (93–100)	3 (0–7)	<0.001
FUNCTIONAL STATUS	40 (13–81)	50 (19–88)	99 (93–100)	1 (0–7)	<0.001
DIET	89 (63–100)	12 (5–38)	81 (0–100)	19 (0–100)	0.25
ACTIVITY	89 (69–100)	11 (0–32)	82 (0–100)	18 (0–100)	0.49
MEDICATIONS	83 (50–100)	17 (0–50)	100	0	0.002
POST-DISCHARGE CARE COMPOSITE (5 ITEMS)	57 (41–83)	43 (17–59)	83 (69–98)	18 (2–31)	<0.001
CRITERIA	PRE-INTERVENTION		POST-INTERVENTION
	COMPLETE	ABSENT	COMPLETE	ABSENT	*
POST-DISCHARGE CARE COMPOSITE (5 ITEMS)	57 (41–83)	43 (17–59)	83 (69–98)	18 (2–31)	<0.001
F/U RESULTS	42 (11–90)	58 (10–89)	81 (50–100)	20 (0–50)	0.02
DISCHARGE LOCATION	92 (88–100)	8 (0–12)	100	0	0.02
CAREGIVER INFO	10 (0–25)	89 (75–100)	48 (7–95)	52 (5–84)	<0.001
HOME SERVICES	64 (32–100)	35 (0–68)	87 (71–95)	12 (0–29)	<0.001
F/U APPOINTMENTS	78 (33–100)	23 (0–67)	96 (86–100)	4 (0–14)	<0.001
OVERALL COMPOSITE (21 ITEMS)	71 (42–87)	29 (13–58)	91 (73–99)	9 (2–27)	<0.001

Note: Proportion may not sum to 100 because of rounding.

^*P value for the comparison of the proportion of discharge summaries that were complete in each period.

Discussion

Our study found that audit and feedback sessions significantly improved the completeness of discharge summaries dictated by geriatric medicine fellows at one academic medical center. Before feedback, completeness was high in most traditional areas of the discharge summaries including admission data, hospital course and discharge planning, but was low in other areas critical for safe transitions of older adults such as post discharge care, test follow-up, caregiver information, and cognitive and functional status changes. These findings were surprising, as using a template should render a completion rate close to 100%. Notably, during feedback sessions, fellows suggested low completion rates were due to lack of awareness regarding the importance of completing all 21 items of the template and missing documentation in patient medical records.

Feedback sessions dramatically improved overall completeness of subsequent discharge summaries and in most of areas of specific importance for geriatric care, although we remain uncertain why all areas did not show improvement (for example, caregiver information completion remained low). One possible explanation is the lack of accurate documentation for all necessary items in the hospital medical record. Moroever, we did not observe completion improvement for other items, i.e. diet and activity. Overall, we believe that drawing attention to areas of particular importance to geriatric care transitions and providing learners with individual reports on their performance increased their awareness and motivated changes to their practice, improving discharge summary completion.

Our study has limitations. This study was a pilot intervention without a control group, because of time and budgetary constraints. Also we were unable to assess for sustainability because the fellows studied for this project graduated after the second audit. Third, we studied discharge summary completion, further research should focus on accuracy of discharge summary content. Finally, while we did not use any advanced technologies or materials, faculty time required to conduct the audit and feedback in this study was estimated at 45 hours. In our opinion this estimate would classify our audit and feedback intervention as a low external cost and moderately-high human cost intervention, which may represent a potential barrier to generalizability. On the other hand, we believe that even an audit of a small sample of discharge summaries done by a physician could provide valuable data for feedback and would involve less faculty time. Our finding that audit and feedback sessions improved the completeness of discharge summaries among house-staff is important for two reasons. First, we were able to demonstrate that focused feedback targeted to areas of particular importance to the transition of older adults changed subsequent behavior and resulted in improved documentation of these areas. Second, our study provides evidence of a programmatic approach to address the ACGME competency of practice based learning and improvement. We believe that our intervention can be reproduced by training programs across the country and are hopeful that such interventions will result in improved patient outcomes during critical care transitions such as hospital discharge.