Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2009 Sep 4.
Published in final edited form as: Ann Intern Med. 2009 Feb 3;150(3):178–187. doi: 10.7326/0003-4819-150-3-200902030-00007

A Reengineered Hospital Discharge Program to Decrease Rehospitalization

A Randomized Trial

Brian W Jack 1, Veerappa K Chetty 1, David Anthony 1, Jeffrey L Greenwald 1, Gail M Sanchez 1, Anna E Johnson 1, Shaula R Forsythe 1, Julie K O'Donnell 1, Michael K Paasche-Orlow 1, Christopher Manasseh 1, Stephen Martin 1, Larry Culpepper 1
PMCID: PMC2738592  NIHMSID: NIHMS123019  PMID: 19189907

Abstract

Background:

Emergency department visits and rehospitalization are common after hospital discharge.

Objective:

To test the effects of an intervention designed to minimize hospital utilization after discharge.

Design:

Randomized trial using block randomization of 6 and 8. Randomly arranged index cards were placed in opaque envelopes labeled consecutively with study numbers, and participants were assigned a study group by revealing the index card.

Setting:

General medical service at an urban, academic, safety-net hospital.

Patients:

749 English-speaking hospitalized adults (mean age, 49.9 years).

Intervention:

A nurse discharge advocate worked with patients during their hospital stay to arrange follow-up appointments, confirm medication reconciliation, and conduct patient education with an individualized instruction booklet that was sent to their primary care provider. A clinical pharmacist called patients 2 to 4 days after discharge to reinforce the discharge plan and review medications. Participants and providers were not blinded to treatment assignment.

Measurements:

Primary outcomes were emergency department visits and hospitalizations within 30 days of discharge. Secondary outcomes were self-reported preparedness for discharge and frequency of primary care providers′ follow-up within 30 days of discharge. Research staff doing follow-up were blinded to study group assignment.

Results:

Participants in the intervention group (n = 370) had a lower rate of hospital utilization than those receiving usual care (n = 368) (0.314 vs. 0.451 visit per person per month; incidence rate ratio, 0.695 [95% CI, 0.515 to 0.937]; P = 0.009). The intervention was most effective among participants with hospital utilization in the 6 months before index admission (P = 0.014). Adverse events were not assessed; these data were collected but are still being analyzed.

Limitation:

This was a single-center study in which not all potentially eligible patients could be enrolled, and outcome assessment sometimes relied on participant report.

Conclusion:

A package of discharge services reduced hospital utilization within 30 days of discharge.


One in 5 hospitalizations is complicated by postdischarge adverse events (1, 2), some of which may lead to preventable emergency department visits or readmissions. Despite this finding, hospital discharge procedures have not been standardized (3). In addition, the declining presence of primary care providers (PCPs) in hospitals has not been adequately accompanied by systems to ensure that patient data are transferred to subsequent caregivers (4, 5). For example, discharge summaries frequently lack critical data and are not sent to the PCP in a timely fashion (6, 7), resulting in outpatient clinicians being unaware of test results that were pending at discharge (8) and evaluations that were scheduled to be done after discharge not being completed (9). Similarly, patients are often left unprepared at discharge; many do not understand their discharge medications and cannot recall their chief diagnoses (10). With more than 32 million adult discharges in the United States each year (11), these deficiencies in the transition of care increase illness, unnecessary hospital utilization, and cost.

Some peridischarge interventions have shown a reduction in hospital readmission rates and cost (12-14), emergency department visits (15), and postdischarge adverse events (16), whereas some have shown little or no effect (17-20). Peridischarge interventions have also shown improved PCP follow-up and outpatient work-ups (21) and higher patient satisfaction (15). Most of these studies have focused on specific diagnoses (14, 22, 23) or highly selected populations, such as geriatric adults (12, 13, 19, 24). Some have focused on specific aspects of the discharge, such as increasing access to primary care follow-up (25), connecting with transitional nursing services (26), or improving patients′ ability to advocate for themselves after discharge (12). To date, no study has evaluated a standardized discharge intervention that includes patient education, comprehensive discharge planning, and postdischarge telephone reinforcement in a general medical population.

Context

Emergency department visits and rehospitalizations are common after hospital discharge.

Contribution

This trial demonstrated that a nurse discharge advocate and clinical pharmacist working together to coordinate hospital discharge, educate patients, and reconcile medications led to fewer follow-up emergency visits and rehospitalizations than usual care alone.

Caution

The trial was conducted at a single center, and not all eligible patients were enrolled.

Implication

A systematic approach to hospital discharges can reduce unnecessary health service use.

The Editors

In 2004, we began an in-depth examination of hospital discharge, for which we designed a package of services to minimize discharge failures—a process called reengineered discharge (RED) (Table 1) (3, 27). We did a randomized, controlled trial to evaluate the clinical effect of implementing RED among patients admitted to a general medical service.

Table 1.

Components of Reengineered Hospital Discharge

In-hospital component (discharge advocate)
1. Educate patient about relevant diagnoses throughout hospital stay.
2. Make appointments for clinician follow-up and postdischarge testing.
     Solicit input from patient about convenient date(s) and time(s) for appointments.
     Coordinate appointments with physicians, testing, and other services.
     Discuss reason for and importance of physician appointments.
     Confirm that patient knows location and transportation plan and review barriers to keeping appointments.
3. Discuss with patient any pending in-hospital tests or studies completed and who will follow-up with results.
4. Organize postdischarge services.
     Be sure patient understands the importance of such services.
     Make appointments at times convenient for patient.
     Discuss the details about how to receive each service.
5. Confirm medication plan.
     Reconcile the discharge medication regimen.
     Explain what medications to take, emphasizing any changes in the regimen.
     Review each medication's purpose, how to take it correctly, and important side effects.
     Be sure the patient has a realistic plan about how to obtain medications.
6. Reconcile the discharge plan with national guidelines and critical pathways.
7. Review appropriate steps for what to do if a problem arises.
     Instruct how to contact the primary care provider (or coverage) by providing contact numbers for evenings and weekends.
     Instruct on what constitutes an emergency and what to do in the case of an emergency.
8. Transmit discharge summary to physicians and services accepting responsibility of patient's care that contains the following:
     Reason for hospitalization with specific principal diagnosis.
     Important findings.
     Procedures done and care, treatment, and services provided to patient.
     Patient's condition at discharge.
     Complete and reconciled medication list (including allergies).
     List of acute medical issues, tests, and studies for which confirmed results are pending at the time of discharge and require follow-up.
     Information about input from consultative services, including rehabilitation therapy.
     When creating this document, the original source documents— laboratory, radiology, operative reports, and medication administration records—should be in the transcriber's immediate possession and be visible when it is necessary to transcribe information from 1 document to another.
9. Assess the degree of understanding by asking the patient to explain in his or her own words the details of the plan.
     May require contacting family members who will share in the caregiving responsibilities.
After-hospital care plan
10. Give the patient a written discharge plan at the time of discharge that contains the following:
     Reason for hospitalization (discharge diagnosis and significant comorbid conditions).
     Discharge medication list (how and when to take each medication and how to obtain medication).
     Contact information and picture of primary care provider and discharge advocate.
     Information for follow-up primary care, specialty care, and outpatient test appointments.
     Calendar, labeled with scheduled appointments and tests.
     Information for tests and studies for which confirmed results are not available at the time of discharge.
Pharmacist postdischarge telephone component
11. Call the patient to reinforce discharge plan, review medications, and solve problems.

METHODS

Setting and Participants

We conducted a 2-group, randomized, controlled trial of English-speaking patients 18 years of age or older who were admitted to the medical teaching service of Boston Medical Center, Boston, Massachusetts—a large, urban, safety-net hospital with an ethnically diverse patient population. Patients had to have a telephone, be able to comprehend study details and the consent process in English, and have plans to be discharged to a U.S. community. We did not enroll patients if they were admitted from a skilled nursing facility or other hospital, transferred to a different hospital service before enrollment, admitted for a planned hospitalization, were on hospital precautions or suicide watch, or were deaf or blind. Boston University's institutional review board approved all study activities.

Randomization

Each morning, a list of admitted patients was reviewed for initial eligibility (hospital location, age, date and time of admission, and previous enrollment). Last names of potential participants were ranked by using a random-number sequence to determine the order in which to approach patients for enrollment. A trained research assistant then approached each patient and further determined eligibility according to inclusion and exclusion criteria (Figure 1).

Figure 1. Study flow diagram.

Figure 1

* Patients did not meet inclusion criteria if they were admitted from or planned discharge to an institutional setting (n = 74), planned hospitalization (n = 3) or discharge to a non-U.S. community (n = 5), were transferred to different hospital service (n = 8), did not speak English (n = 371) or have a telephone (n = 71), were on hospital precautions (n = 274) or suicide watch with a sitter (n = 10), were unable to consent (n = 181), had sickle cell disease as the admitting diagnosis (n = 38), had privacy status (n = 8), were deaf or blind (n = 2), or other (n = 4).

† Usual care participants did not meet eligibility criteria if they were discharged to a nursing facility (n = 28), were transferred to another hospital service (n = 1), were previously enrolled (n = 1), died during index admission (n = 2), requested to be removed (n = 5), or other (n = 3).

‡ Intervention participants did not meet eligibility criteria if they were discharged to a nursing facility (n = 21), were transferred to another hospital service (n = 6), died during index admission (n = 1), requested to be removed (n = 2), or other (n = 8).

§ 107 intervention participants did not receive a reinforcement call because they could not be reached by telephone (n = 93), they were readmitted the same or next day (n = 2), there was no staffing coverage (n = 8), or other (n = 4).

By using block randomization (28) with varying block sizes of 6 and 8, we randomly arranged index cards indicating either the usual care or intervention group. We placed the cards in opaque envelopes labeled consecutively with study numbers. We assigned eligible participants who consented to enrollment to a study group by revealing the concealed index card. This process continued until 2 participants were enrolled each day of the week (or 3 participants if the first 2 participants were randomly assigned to the usual care group). This protocol ensured that research assistants could not selectively choose potential participants for enrollment or predict assignment. Participants randomly assigned to usual care received no further intervention. There were 40 participants in the usual care group and 38 in the intervention group who were enrolled but no longer met inclusion criteria at discharge (most commonly because they were discharged to a nursing facility). Because the primary analysis was by intention to treat, we included these participants in the analysis, with the exception of those who died before index discharge, requested to be removed, or were previously enrolled (Figure 1).

Interventions

Nurse discharge advocates (DAs) carried out all aspects of the in-hospital intervention. We hired 6 part-time DAs to work with intervention participants to ensure coverage by 1 DA 7 days a week, 5 hours a day. We trained all DAs to deliver the RED intervention by using a manual containing detailed scripts, observation of relevant clinical interactions, and simulated practice sessions. The primary goals of the DA were to coordinate the discharge plan with the hospital team and educate and prepare the participant for discharge. At admission, the DA completed the RED intervention components outlined in Table 1. Additional information about the DA training manual is published elsewhere (3) and can be found on our Web site (www.bu.edu/fammed/projectred/index.html).

With information collected from the hospital team and the participant, the DA created the after-hospital care plan (AHCP), which contained medical provider contact information, dates for appointments and tests, an appointment calendar, a color-coded medication schedule, a list of tests with pending results at discharge, an illustrated description of the discharge diagnosis, and information about what to do if a problem arises. Information for the AHCP was manually entered into a Microsoft Word (Microsoft, Redmond, Washington) template, printed, and spiralbound to produce an individualized, color booklet designed to be accessible to individuals with limited health literacy. By using scripts from the training manual, the DA used a teach-back methodology (29) to review the contents of the AHCP with the participant. On the day of discharge, the AHCP and discharge summary were faxed to the PCP.

A clinical pharmacist telephoned the participants 2 to 4 days after the index discharge to reinforce the discharge plan by using a scripted interview. The pharmacist had access to the AHCP and hospital discharge summary and, over several days, made at least 3 attempts to reach each participant. The pharmacist asked participants to bring their medications to the telephone to review them and address medication-related problems; the pharmacist communicated these issues to the PCP or DA.

Outcomes Measures and Follow-up

At the time of recruitment, research assistants collected baseline data, including sociodemographic characteristics; the Short Form-12 Health Survey, Version 2 (30); the depression subscale from the Patient Health Questionnaire-9 (31); and the Rapid Estimate of Adult Literacy in Medicine (32). We calculated the Charlson Comorbidity Index score by using primary and secondary diagnoses recorded on the index admission discharge summary (33). We determined the number of hospital admissions and emergency department visits in the 6 months before index admission through medical record review (Boston Medical Center hospital utilization) and participant report (all other hospital utilization).

The primary end point was the rate of hospital utilization—the total number of emergency department visits and readmissions per participant within 30 days of the index discharge. Any emergency department visit in which a participant was subsequently hospitalized was counted as a readmission. Secondary end points were self-reported preparedness for discharge, rate of primary care follow-up visits, and knowledge of discharge diagnosis. We collected outcome data by review of the hospital's electronic medical records (EMRs) and by contacting participants by telephone 30 days after discharge. We obtained dates of subsequent emergency department visits and readmissions at Boston Medical Center from the EMRs and collected those at other hospitals through participant report. For participants who could not be reached within 60 days after discharge, we assumed that they were alive and relied on hospital EMRs for primary outcomes. Research staff doing follow-up telephone calls and reviewing hospital records were blinded to study group assignment. Discharge advocates and pharmacists recorded time spent working with each participant.

Statistical Analyses

On the basis of unpublished pilot data from the general medical service at Boston Medical Center from July 2003 to June 2004, we estimated that with a readmission incidence rate of 0.197 visit per person per month and an emergency department visit incidence rate of 0.17 visit per person per month (combined hospital utilization rate of 0.367 visit per person per month), we needed to enroll 750 participants to detect an incidence rate reduction of 0.25 visit per person per month in the primary outcome and achieve 80% power, with a 2-sided α level of 0.05.

For outcome data, we followed each participant for 30 days after index discharge. We measured person-time in months, making total person-months equal to the number of participants in each study group. We used the Poisson test and proportions test to test for significance of primary outcomes and secondary outcomes, respectively. We conducted a sensitivity analysis and excluded outliers with high subsequent hospital utilization.

We generated cumulative hazard curves for time to multiple events (emergency department visits and readmissions) and compared them by using a log-rank test. We measured the time-to-event from the index discharge date. This method corresponds to the Wei, Lin, and Weissfeld (34) marginal data model for ordered multiple events, which allows each event to have a separate underlying hazard (35).

We did subgroup analysis with Poisson regression by using total hospital utilization number per participant as the dependent variable. We determined subgroups a priori and included depression diagnosis (36), previous hospital utilization (37), health literacy level (38), sex, and age. To evaluate potential interactions between these variables and the intervention, we included interaction terms in the Poisson regression. We used 2-sided significance tests. We considered P values less than 0.05 to be statistically significant. All data were analyzed with S-Plus, version 8.0 (Insightful, Seattle, Washington), and Intercooled Stata, version 10 (StataCorp, College Station, Texas).

Role of the Funding Source

The Agency for Healthcare Research and Quality and the National Heart, Lung, and Blood Institute, National Institutes of Health, funded this work. The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, review, and approval of the manuscript.

RESULTS

Study Sample

During the study period from 3 January 2006 to 18 October 2007, we admitted 5489 patients and assessed 3873 for eligibility (Figure 1). Because of a lack of available research staff, we did not assess 1616 patients. Of those assessed for eligibility, 1049 did not meet eligibility criteria, 120 were previously enrolled, 527 declined to participate, 474 were unavailable in their hospital room at the time of enrollment, and 954 were not approached because the maximum number of enrolled participants was reached that day. We enrolled and randomly assigned 749 participants: 376 in the usual care group and 373 in the intervention group (Figure 1). For primary and secondary outcome analyses, we excluded 11 participants on the basis of participant request (n = 7), death before index discharge (n = 3), and previous enrollment (n = 1), which left 368 in the usual care group and 370 in the intervention group. Baseline demographic and clinical characteristics were similar across study groups (Table 2).

Table 2.

Baseline Participant Characteristics*

Characteristic Usual Care Group
(n = 376)
Intervention Group
(n = 373)
Men, n (%) 176 (47) 195 (52)
Mean age, (SD), y 49.6 (15.3) 50.1 (15.1)
Race, n (%)
     White non-Hispanic 103 (27) 106 (28)
     Black non-Hispanic 197 (52) 191 (51)
     Hispanic 38 (10) 38 (10)
     Other race or mixed race 38 (10) 38 (10)
Annual personal income, n (%)
     <$10 000 119 (32) 118 (32)
     $10 000–$19 999 61 (16) 73 (20)
     $20 000–$49 999 74 (20) 58 (16)
     ≥$50 000 24 (6.4) 19 (5.1)
Health insurance, n (%)
     Private 64 (17) 58 (16)
     Medicaid 184 (49) 174 (47)
     Medicare 49 (13) 51 (14)
     Free Care 72 (19) 86 (23)
Education level, n (%)
     Less than high school 33 (8.8) 22 (5.9)
     Some high school 69 (18) 66 (18)
     High school graduate or GED 131 (35) 151 (40)
     Some college 94 (25) 84 (23)
     4-year college graduate or higher 45 (12) 48 (13)
Health literacy level, n (%)
     Grade 3 or below 56 (15) 58 (16)
     Grade 4 to 6 37 (9.8) 39 (10)
     Grade 7 to 8 119 (32) 110 (29)
     Grade 9 or above 154 (41) 153 (41)
Current employment status, n (%)
     Full-time 96 (26) 83 (22)
     Part-time 40 (11) 48 (13)
     Retired 65 (17) 69 (18)
     Disabled 88 (23) 78 (21)
     Unemployed 68 (18) 75 (20)
     Other 16 (4.3) 16 (4.2)
Homeless in past 3 mo, n (%) 40 (11) 35 (9.4)
Mean previous hospital admissions (SD), n§ 0.71 (1.4) 0.64 (1.1)
Mean previous emergency department visits (SD), n§ 1.0 (1.8) 0.86 (1.6)
Mean length of stay (SD), d 2.6 (3.0) 2.8 (3.4)
PCP at enrollment, n (%) 303 (81) 299 (80)
Mean Charlson Comorbidity Index score (SD) 1.2 (2.0) 1.2 (1.8)
Mean Physical Component Summary score (SD) 40.7 (7.4) 40.1 (7.3)
Mean Mental Component Summary score (SD) 46.3 (9.8) 46.7 (9.3)
Major depressive disorder, n (%)** 52 (14) 69 (18)
Minor depressive disorder, n (%)** 60 (16) 58 (16)

PCP = primary care provider; REALM = Rapid Estimate for Adult Literacy in Medicine.

*

Not all column percentages sum to 100% because of missing values.

Free Care is a Massachusetts state program for uninsured patients.

Health literacy categories correspond to total REALM scores (32) of grade 3 or below (REALM score, 0–18), grade 4 to 6 (REALM score, 19–44), grade 7 to 8 (REALM score, 45–60), and grade 9 or above (REALM score, 61–66).

§

Previous hospital admissions and emergency department visits include those that occurred within 6 mo before index admission.

The Charlson Comorbidity Index (33) score reflects the cumulative increased likelihood of 1-year mortality. The higher the score, the more severe the comorbid condition. A 35% increase in risk for death is reflected in a 1-point increase in weights. The minimum score is zero; there is no maximum score.

From the Short Form-12 Health Survey (30). The Physical Component Summary score range is 0–100. Mean score for U.S. population is 50 (SD, 10). Higher scores suggest greater physical functional status. The Mental Component Summary score range is 0–100. Mean score for U.S. population is 50 (SD, 10). Higher scores suggest greater mental functional status.

**

Determined by using the Patient Health Questionnaire-9, a 9-item, 4-point Likert scale, standard scoring algorithm to screen for major and minor depression (31).

Process Measures

In the intervention group, we discharged 346 of 370 (94%) participants with a primary care appointment, 306 (83%) left with an AHCP, 197 (53%) had their medications reconciled with the ambulatory EMR and had their updated medication list included in their AHCP, and 336 (91%) had their discharge information sent to their PCP within 24 hours after discharge. The pharmacist reached 228 (62%) of the intervention participants a median of 4 days (interquartile range [IQR], 3 to 6 days) after discharge and completed medication review with 195 (53%) intervention participants. The pharmacist found that 126 of 195 (65%) intervention participants who completed medication review had at least 1 medication problem and 103 (53%) needed corrective action by the pharmacist, such as contacting the participant's PCP.

In the usual care group, we discharged 127 of 368 (35%) participants with a primary care appointment; data on medication reconciliation and discharge summary transfer to the PCP were unavailable.

Outcome Follow-up

We obtained participant-reported outcome data by telephone for 615 of 738 (83%) participants a median of 32 days (IQR, 30 to 36 days) after discharge. We reached similar proportions of intervention (307 [83%]) and usual care (307 [83%]) group participants (P = 0.87). Likewise, similar proportions of intervention (12 [3%]) and usual care (7 [2%]) group participants reported hospital utilization at hospitals other than Boston Medical Center (P = 0.36).

Hospital Utilization

In the intervention group, 56 (15.1%) participants had 1 hospital utilization and 24 (6.5%) had more than 1 hospital utilization. These 80 (21.6%) participants had 116 hospital utilizations (61 emergency department visits and 55 readmissions) during 370 person-months of follow-up (0.314 visit per person per month). In the usual care group, 69 (18.8%) participants had 1 hospital utilization and 30 (8.1%) had more than 1 hospital utilization. These 99 (26.9%) participants had 166 visits (90 emergency department visits and 76 readmissions) during 368 person-months of follow-up (0.451 visit per person per month) (Table 3). Intervention participants had a lower rate of hospital utilization than usual care participants (incidence rate ratio, 0.695 [95% CI, 0.515 to 0.937]); P = 0.009). After we repeated the analysis excluding 1 usual care participant with more than 8 hospital utilizations, hospital utilization between study groups remained statistically significant (P = 0.028). Approximately 30% of participants in each study group with any subsequent hospital utilization had more than 1 subsequent hospital utilization.

Table 3.

Primary and Secondary Outcomes

Variable Usual Care Group Intervention Group P Value
Primary outcomes ≤30 d after index hospitalization
     Patients, n 368 370
     Hospital utilizations, n (visits/patient/mo)* 166 (0.451) 116 (0.314) 0.009
          IRR (95% CI) 1.0 0.695 (0.515–0.937)
     Emergency department visits, n (visits/patient/mo) 90 (0.245) 61 (0.165) 0.014
          IRR (95% CI) 1.0 0.674 (0.476–0.955)
     Readmissions, n (visits/patient/mo) 76 (0.207) 55 (0.149) 0.090
          IRR (95% CI) 1.0 0.720 (0.445–1.164)
Secondary outcomes
     Patients, n 308 307
     Able to identify discharge diagnosis, n (%) 217 (70) 242 (79) 0.017
     Able to identify PCP name, n (%) 275 (89) 292 (95) 0.007
     Visited PCP, n (%) 135 (44) 190 (62) <0.001
          How well were your questions answered before you left the hospital? 108 (62) 129 (77) 0.002
          How well did you understand your appointments after you left the hospital? 219 (79) 254 (86) 0.025
          How well did you understand how to take your medications after leaving the hospital? 233 (83) 264 (89) 0.049
          How well did you understand your main problem or diagnosis when you left the hospital? 167 (57) 198 (66) 0.014
          How prepared were you to leave the hospital? 163 (55) 197 (65) 0.013

IRR = incidence rate ratio; PCP = primary care provider.

*

Defined as the sum of emergency department visits plus rehospitalizations. An emergency department visit that leads to a rehospitalization is counted only as a rehospitalization.

Denominators were participants who were reached at the 30-day follow-up phone call and those who answered questions.

Questions were answered on a 5-point Likert scale. The percentage reflects participants who responded with either of the top 2 categories on the scale (“very prepared” or “prepared”).

Figure 2 shows the cumulative hazard curves comparing hospital utilization in the 2 groups over the 30 days after discharge (P = 0.004).

Figure 2. Cumulative hazard rate of hospital utilization for 30 days after index hospital discharge.

Figure 2

* The denominators for the events were 433 for usual care and 397 for intervention. This represents the number of discharges for each group, which includes index discharges and discharges from all subsequent admissions. At each discharge, the participant is returned to the risk pool. The denominator is thus constant during the entire 30 days.

† Two events for the usual care group and 6 events for the intervention group were removed from this analysis because the date of admission was missing.

Subgroup analyses revealed that the intervention was more effective at reducing hospital utilization for participants with greater hospital utilization in the previous 6 months (P for interaction = 0.014).

Secondary Outcomes

Participants receiving the intervention could identify their index discharge diagnosis (242 [79%] vs. 217 [70%] participants; P = 0.017) and PCP name (292 [95%] vs. 275 [89%] participants; P = 0.007) more often than usual care participants. Intervention participants also reported a higher PCP follow-up rate than usual care participants (190 [62%]) vs. 135 [44%]; P < 0.001). Intervention group participants reported being more prepared for discharge at 30 days (Table 3). Each component of the AHCP tool was highly rated by intervention participants (Appendix Table, available at www.annals.org).

Time Spent Providing Intervention

The DA spent a median of 42.5 minutes (IQR, 30 to 60 minutes) speaking directly with each participant, both collecting participant information and teaching the AHCP booklet. The DA made a median of 3 attempts (IQR, 2 to 5 attempts) per participant to call or page interns. An additional estimated 45 minutes was spent reviewing the participant's EMR, communicating with the medical team, and preparing the AHCP. Therefore, total DA time was estimated to be 87.5 minutes per participant. Estimated weekly DA time (following 14 participants per week) was 20.4 hours or approximately 0.5 full-time equivalent.

The pharmacist postdischarge telephone calls took a median of 14 minutes (IQR, 10 to 19 minutes), with 10 (6 to 18) additional minutes spent on call preparation, missed calls, and resolving problems identified during calls. It took the pharmacist a median of 2 attempts (IQR, 1 to 3 attempts) to reach participants by telephone. Median total pharmacist time was approximately 26 minutes (IQR, 18 to 36 minutes) per participant. Estimated weekly pharmacist time (following 14 participants per week) was 6.1 hours or approximately 0.15 full-time equivalent.

Outcome Cost Analysis

The actual cost of emergency department visits totaled $21 389 for the usual care group and $11 285 for the intervention group. The actual cost of hospital visits totaled $412 544 for the usual care group and $268 942 for the intervention group. Follow-up PCP appointments were given an estimated cost of $55, on the basis of costs from an average hospital follow-up visit at Boston Medical Center. The estimated cost of primary care outpatient visits within 30 days after discharge totaled $8906 for 44% of 368 usual care participants and $12 617 for 62% of 370 intervention participants. The difference between study groups in total cost (combining actual hospital utilization cost and estimated outpatient cost) for 738 participants was $149 995—an average of $412 per person who received the intervention. This represents a 33.9% lower observed cost for the intervention group.

Discussion

The RED intervention decreased hospital utilization (combined emergency department visits and readmissions) within 30 days of discharge by about 30% among patients on a general medical service of an urban, academic medical center. More intervention group participants reported seeing their PCP for follow-up within 30 days and reported higher levels of preparedness for discharge. In addition, the intervention was successful in reducing hospital utilization among participants who frequently used hospital services. These data support implementation of a comprehensive program for hospital discharge among similar hospitals.

Our intervention includes patient-centered education, comprehensive discharge planning, and postdischarge reinforcement and is practical and easily applied to general medical patients. The RED intervention has 3 core elements: the DA, the AHCP, and the follow-up telephone call by those of the pharmacist. Because these elements were bundled, we could not clearly determine the degree that each part contributed to the effects demonstrated. No previous studies have evaluated this trio of interventions together, although the roles of the DA and the pharmacist build on previous literature (12, 15, 16, 19). For example, peridischarge nursing support services have been shown to improve discharge for patients with heart failure (14, 23, 39, 40). Coleman and colleagues (12) used a nurse “transition coach” to demonstrate reduced readmissions at 30 and 90 days among elderly patients. Naylor and coworkers (13, 19) found that nurse specialists involved during and after discharge also effectively reduced acute readmissions.

Several studies have analyzed pharmacist interventions. Dudas and colleagues (15) randomly assigned patients to receive a telephone call by a pharmacist after discharge and demonstrated fewer emergency department visits. Schnipper and coworkers (16) used pharmacist counseling before and after discharge and showed reductions in preventable adverse drug events and medication-related readmissions and emergency department visits. Al-Rashed and colleagues (41) found that predischarge pharmacist-based counseling for elderly patients followed by a postdischarge home visit resulted in fewer unplanned primary care visits and fewer readmissions.

The techniques used to teach the AHCP, its content, and its format (for example, pictures, color, and large font) were informed by the literature on limited health literacy (42, 43). Overall, the intervention improved patient comprehension of key elements of self-care: 30 days after discharge, intervention participants were better able to identify their primary diagnosis and reported better understanding of their diagnosis, medications, and appointments. The content, format, and teaching of discharge preparation tools deserve further attention because few studies have assessed the effect of patient education on subsequent hospital utilization.

Because intervention group participants were more likely to report seeing their PCPs after discharge and we transmitted discharge information to PCPs promptly after discharge, the intervention optimized the chance that PCPs could identify and address outstanding issues. In addition, the pharmacist follow-up telephone call identified any problems that a patient was having after discharge and relayed those issues to the PCP. Previous studies have suggested that improved access to community-based follow-up alone may not be enough to reduce hospital readmissions (18, 25). We provide evidence that when combined with other elements of RED, improving PCP follow-up may help reduce hospital utilization.

Implementing this discharge intervention required about 1.5 hours of nursing time and 30 minutes of pharmacist time per participant. Because some of the DA activities were redundant with those of existing hospital personnel, implementation of the RED intervention using existing hospital staff would require less time per patient. Also, because information was manually entered to create each AHCP, hospital information technology solutions could be developed to make this process more efficient. Despite this, we demonstrated hospital utilization cost savings averaging $412 per discharge. These figures do not include the cost of the intervention, which involved 0.5 full-time equivalent for a nurse and 0.15 full-time equivalent for a clinical pharmacist. If adopted broadly, this intervention could produce substantial effects on health care financing (44). However, an important challenge for programs like RED is that health providers, who are best situated to implement such a program, may have no financial incentive to do so. Hospitals serving capitation-based patient populations may benefit financially from reducing unneeded rehospitalization. Under the fee-for-service scheme, the payer will benefit even after paying the full cost of the intervention. Hospitals will also benefit from decreasing the rehospitalization rate as an important quality-improvement target, and investment in strategies proven to work will be attractive to payers. The National Quality Forum is reviewing new metrics of quality care surrounding readmission rates (45), and programs like RED may be used to improve health care organizations' quality ratings.

Our study has limitations. Because of staffing limitations, we were only able to enroll 2 to 3 participants per day, and we could not enroll participants on some weekends and holidays. Because of the nature of our urban, underserved patient population and exclusion of patients coming from nursing homes, the study sample was younger and had fewer comorbid conditions than those in other studies; thus, our results may not be generalizable to all patient groups. Also, we relied on participant self-report for outcomes that we could not gather from EMRs, notably data on PCP follow-up and visits at hospitals other than Boston Medical Center. Previous studies have suggested that patient reports of emergency department and hospital use correlate well with electronic records from 6 months to 1 year (46, 47). Ritter and colleagues (48) demonstrated that patients tended to underreport outpatient visits over 6 months compared with electronic charts and found no demographic or health-related predictors of underreporting. In our case, recall bias should be expected to be nondifferential because our study was randomized, we reached both study groups equally, and outcome assessors were blinded to study assignment. We assumed that study participants not reached by telephone for an outcome assessment were alive for 30 days after the index discharge, and we relied on hospital EMRs to gather primary outcomes. Therefore, we did not capture deaths or hospital utilizations at institutions other than Boston Medical Center for this limited number of participants. For the cost analysis, we could not determine a generalizable cost for the intervention because costs vary widely by institution and location. Similarly, we could not estimate the downstream cost implications of avoided emergency department visits and readmissions. Still, we present the actual costs for 3 important types of directly related medical utilization. The cost of hospital utilization and outpatient visits also cannot be easily generalized. Our goal is to provide the direct comparison that can be made for these key costs between study groups, and we observed a 33.9% reduction in these costs.

In summary, the RED program successfully reduced hospital utilization, improved patient self-perceived preparation for discharge, and increased PCP follow-up. In 2007, the National Quality Forum Consensus Standards Maintenance committee identified hospital discharge as a critical area for improvement. The resulting National Quality Forum “Safe Practice” was based largely on the principles of the RED program (49). Our study provides data supporting the implementation of the discharge standards promoted by the National Quality Forum.

Acknowledgment

The authors thank Caroline Hesko, MPH, for data collection and Lynn Schipelliti, RN, and Mary Goodwin, RN, for implementing the intervention. The RED study employed Ms. Hesko, Ms. Schipelliti, and Ms. Goodwin.

Grant Support: By Agency for Healthcare Research and Quality grants 1UC1HS014289-01 and 1U18HS015905-01 (Dr. Jack) and National Heart, Lung, and Blood Institute, National Institutes of Health, grant 1 R01 HL081307-01 (Dr. Jack).

Funding: Agency for Healthcare Research and Quality and National Heart, Lung, and Blood Institute, National Institutes of Health.

Appendix

Table.

Evaluation of the AHCP by Intervention Participants 30 Days After Discharge

Question Participant
Response, n (%)*
In the past 4 weeks, how often did you refer to your AHCP?
     Daily 31 (12)
     Frequently 39 (14)
     Occasionally 75 (28)
     Once or twice 78 (29)
     Never 21 (7.8)
How useful was the AHCP booklet?
     Extremely useful 46 (17)
     Very useful 92 (34)
     Moderately useful 50 (19)
     A little bit useful 40 (15)
     Not at all useful 10 (3.7)
What was the most helpful part of the AHCP?
     RED medication schedule 51 (19)
     Appointment page 41 (15)
     Medical provider contact information 26 (9.7)
     Appointment calendar 24 (8.9)
     Diagnosis information 29 (11)
     Other 29 (11)
How helpful was the RED medication calendar?
     Extremely helpful 26 (17)
     Very helpful 46 (30)
     Moderately helpful 15 (9.7)
     A little bit helpful 10 (6.5)
     Not at all helpful 4 (2.6)

AHCP = after-hospital care plan; RED = reengineered discharge.

*

Not all percentages sum to 100% because of missing values (participants did not answer the question—they either declined or ended the call early).

The denominator was intervention participants who were reached for the 30-day follow-up telephone call and received an AHCP (n = 269).

The denominator was intervention participants who were reached for the 30-day follow-up telephone call and received an RED medication calendar in their AHCP (n = 155).

Footnotes

ClinicalTrials.gov registration number: NCT00252057.

Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies.

Potential Financial Conflicts of Interest: Grants received: B.W. Jack (Agency for Healthcare Research and Quality; National Heart, Lung, and Blood Institute, National Institutes of Health).

Reproducible Research Statement: Study protocol and statistical code: Available from Dr. Jack (e-mail, brian.jack@bmc.org). Data set: Available through written agreement with Dr. Jack (e-mail, brian.jack@bmc.org).

References

  • 1.Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. The incidence and severity of adverse events affecting patients after discharge from the hospital. Ann Intern Med. 2003;138:161–7. doi: 10.7326/0003-4819-138-3-200302040-00007. [PMID: 12558354] [DOI] [PubMed] [Google Scholar]
  • 2.Forster AJ, Clark HD, Menard A, Dupuis N, Chernish R, Chandok N, et al. Adverse events among medical patients after discharge from hospital. CMAJ. 2004;170:345–9. [PMID: 14757670] [PMC free article] [PubMed] [Google Scholar]
  • 3.Greenwald JL, Denham CR, Jack BW. The hospital discharge: a review of a high risk care transition with highlights of a reengineered discharge process. J Patient Saf. 2007;3:97–106. [Google Scholar]
  • 4.Wachter RM. Hospitalists in the United States—mission accomplished or work in progress? N Engl J Med. 2004;350:1935–6. doi: 10.1056/NEJMp038201. [PMID: 15128892] [DOI] [PubMed] [Google Scholar]
  • 5.Moore C, Wisnivesky J, Williams S, McGinn T. Medical errors related to discontinuity of care from an inpatient to an outpatient setting. J Gen Intern Med. 2003;18:646–51. doi: 10.1046/j.1525-1497.2003.20722.x. [PMID: 12911647] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW. Deficits in communication and information transfer between hospital-based and primary care physicians: implications for patient safety and continuity of care. JAMA. 2007;297:831–41. doi: 10.1001/jama.297.8.831. [PMID: 17327525] [DOI] [PubMed] [Google Scholar]
  • 7.van Walraven C, Seth R, Austin PC, Laupacis A. Effect of discharge summary availability during post-discharge visits on hospital readmission. J Gen Intern Med. 2002;17:186–92. doi: 10.1046/j.1525-1497.2002.10741.x. [PMID: 11929504] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Roy CL, Poon EG, Karson AS, Ladak-Merchant Z, Johnson RE, Maviglia SM, et al. Patient safety concerns arising from test results that return after hospital discharge. Ann Intern Med. 2005;143:121–8. doi: 10.7326/0003-4819-143-2-200507190-00011. [PMID: 16027454] [DOI] [PubMed] [Google Scholar]
  • 9.Moore C, McGinn T, Halm E. Tying up loose ends: discharging patients with unresolved medical issues. Arch Intern Med. 2007;167:1305–11. doi: 10.1001/archinte.167.12.1305. [PMID: 17592105] [DOI] [PubMed] [Google Scholar]
  • 10.Makaryus AN, Friedman EA. Patients' understanding of their treatment plans and diagnosis at discharge. Mayo Clin Proc. 2005;80:991–4. doi: 10.4065/80.8.991. [PMID: 16092576] [DOI] [PubMed] [Google Scholar]
  • 11.Levit K, Ryan K, Elixhauser A, Stranges E, Kassed C, Coffey R. HCUP facts and figures: Statistics on hospital-based care in the United States in 2005. Accessed at www.hcup-us.ahrq.gov/reports.jsp on 26 May 2008.
  • 12.Coleman EA, Parry C, Chalmers S, Min SJ. The care transitions intervention: results of a randomized controlled trial. Arch Intern Med. 2006;166:1822–8. doi: 10.1001/archinte.166.17.1822. [PMID: 17000937] [DOI] [PubMed] [Google Scholar]
  • 13.Naylor MD, Brooten D, Campbell R, Jacobsen BS, Mezey MD, Pauly MV, et al. Comprehensive discharge planning and home follow-up of hospitalized elders: a randomized clinical trial. JAMA. 1999;281:613–20. doi: 10.1001/jama.281.7.613. [PMID: 10029122] [DOI] [PubMed] [Google Scholar]
  • 14.Anderson C, Deepak BV, Amoateng-Adjepong Y, Zarich S. Benefits of comprehensive inpatient education and discharge planning combined with outpatient support in elderly patients with congestive heart failure. Congest Heart Fail. 2005;11:315–21. doi: 10.1111/j.1527-5299.2005.04458.x. [PMID: 16330907] [DOI] [PubMed] [Google Scholar]
  • 15.Dudas V, Bookwalter T, Kerr KM, Pantilat SZ. The impact of follow-up telephone calls to patients after hospitalization. Am J Med. 2001;111:26S–30S. doi: 10.1016/s0002-9343(01)00966-4. [PMID: 11790365] [DOI] [PubMed] [Google Scholar]
  • 16.Schnipper JL, Kirwin JL, Cotugno MC, Wahlstrom SA, Brown BA, Tarvin E, et al. Role of pharmacist counseling in preventing adverse drug events after hospitalization. Arch Intern Med. 2006;166:565–71. doi: 10.1001/archinte.166.5.565. [PMID: 16534045] [DOI] [PubMed] [Google Scholar]
  • 17.Siu AL, Kravitz RL, Keeler E, Hemmerling K, Kington R, Davis JW, et al. Postdischarge geriatric assessment of hospitalized frail elderly patients. Arch Intern Med. 1996;156:76–81. [PMID: 8526700] [PubMed] [Google Scholar]
  • 18.Holland R, Lenaghan E, Harvey I, Smith R, Shepstone L, Lipp A, et al. Does home based medication review keep older people out of hospital? The HOMER randomised controlled trial. BMJ. 2005;330:293. doi: 10.1136/bmj.38338.674583.AE. [PMID: 15665005] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Naylor M, Brooten D, Jones R, Lavizzo-Mourey R, Mezey M, Pauly M. Comprehensive discharge planning for the hospitalized elderly. A randomized clinical trial. Ann Intern Med. 1994;120:999–1006. doi: 10.7326/0003-4819-120-12-199406150-00005. [PMID: 8185149] [DOI] [PubMed] [Google Scholar]
  • 20.Shepperd S, Parkes J, McClaren J, Phillips C. Discharge planning from hospital to home. Cochrane Database Syst Rev. 2004:CD000313. doi: 10.1002/14651858.CD000313.pub2. [PMID: 14973952] [DOI] [PubMed] [Google Scholar]
  • 21.Balaban RB, Weissman JS, Samuel PA, Woolhandler S. Redefining and redesigning hospital discharge to enhance patient care: a randomized controlled study. J Gen Intern Med. 2008;23:1228–33. doi: 10.1007/s11606-008-0618-9. [PMID: 18452048] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Andersen HE, Schultz-Larsen K, Kreiner S, Forchhammer BH, Eriksen K, Brown A. Can readmission after stroke be prevented? Results of a randomized clinical study: a postdischarge follow-up service for stroke survivors. Stroke. 2000;31:1038–45. doi: 10.1161/01.str.31.5.1038. [PMID: 10797163] [DOI] [PubMed] [Google Scholar]
  • 23.Phillips CO, Wright SM, Kern DE, Singa RM, Shepperd S, Rubin HR. Comprehensive discharge planning with postdischarge support for older patients with congestive heart failure: a meta-analysis. JAMA. 2004;291:1358–67. doi: 10.1001/jama.291.11.1358. [PMID: 15026403] [DOI] [PubMed] [Google Scholar]
  • 24.Evans RL, Hendricks RD. Evaluating hospital discharge planning: a randomized clinical trial. Med Care. 1993;31:358–70. doi: 10.1097/00005650-199304000-00007. [PMID: 8464252] [DOI] [PubMed] [Google Scholar]
  • 25.Weinberger M, Oddone EZ, Henderson WG. Does increased access to primary care reduce hospital readmissions? Veterans Affairs Cooperative Study Group on Primary Care and Hospital Readmission. N Engl J Med. 1996;334::1441–7. doi: 10.1056/NEJM199605303342206. [PMID: 8618584] [DOI] [PubMed] [Google Scholar]
  • 26.Naylor MD, Brooten D. The roles and functions of clinical nurse specialists. Image J Nurs Sch. 1993;25:73–8. doi: 10.1111/j.1547-5069.1993.tb00757.x. [PMID: 8449535] [DOI] [PubMed] [Google Scholar]
  • 27.Anthony D, Chetty VK, Kartha A, McKenna K, DePaoli RM, Jack B. Patient safety at time of discharge—an example of a multifaceted process evaluation. In: Henriksen K, Battles JB, Marks ESL, Lewin DI, editors. Advances in Patient Safety: From research to Implementation. vol. 2. Concepts and Methodology. Agency for Healthcare Research and Quality; Rockville, MD: 2005. [Google Scholar]
  • 28.Lachin JM, Matts JP, Wei LJ. Randomization in clinical trials: conclusions and recommendations. Control Clin Trials. 1988;9:365–74. doi: 10.1016/0197-2456(88)90049-9. [PMID: 3203526] [DOI] [PubMed] [Google Scholar]
  • 29.Paasche-Orlow MK, Riekert KA, Bilderback A, Chanmugam A, Hill P, Rand CS, et al. Tailored education may reduce health literacy disparities in asthma self-management. Am J Respir Crit Care Med. 2005;172:980–6. doi: 10.1164/rccm.200409-1291OC. [PMID: 16081544] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Ware J, Jr, Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34:220–33. doi: 10.1097/00005650-199603000-00003. [PMID: 8628042] [DOI] [PubMed] [Google Scholar]
  • 31.Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16:606–13. doi: 10.1046/j.1525-1497.2001.016009606.x. [PMID: 11556941] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Davis TC, Long SW, Jackson RH, Mayeaux EJ, George RB, Murphy PW, et al. Rapid estimate of adult literacy in medicine: a shortened screening instrument. Fam Med. 1993;25:391–5. [PMID: 8349060] [PubMed] [Google Scholar]
  • 33.Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373–83. doi: 10.1016/0021-9681(87)90171-8. [PMID: 3558716] [DOI] [PubMed] [Google Scholar]
  • 34.Wei LJ, Lin DY, Weissfeld L. Regression analysis of multivariate incomplete failure time data by modeling marginal distributions. J Am Stat Assoc. 1989;84:1065–73. [Google Scholar]
  • 35.Therneau TM, Grambsch PM. Modeling Survival Data: Extending the Cox Model. Springer Science and Business Media; New York: 2000. [Google Scholar]
  • 36.Büla CJ, Wietlisbach V, Burnand B, Yersin B. Depressive symptoms as a predictor of 6-month outcomes and services utilization in elderly medical inpatients. Arch Intern Med. 2001;161:2609–15. doi: 10.1001/archinte.161.21.2609. [PMID: 11718593] [DOI] [PubMed] [Google Scholar]
  • 37.van Walraven C, Mamdani M, Fang J, Austin PC. Continuity of care and patient outcomes after hospital discharge. J Gen Intern Med. 2004;19:624–31. doi: 10.1111/j.1525-1497.2004.30082.x. [PMID: 15209600] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Baker DW, Gazmararian JA, Williams MV, Scott T, Parker RM, Green D, et al. Functional health literacy and the risk of hospital admission among Medicare managed care enrollees. Am J Public Health. 2002;92:1278–83. doi: 10.2105/ajph.92.8.1278. [PMID: 12144984] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Blue L, Lang E, McMurray JJ, Davie AP, McDonagh TA, Murdoch DR, et al. Randomised controlled trial of specialist nurse intervention in heart failure. BMJ. 2001;323:715–8. doi: 10.1136/bmj.323.7315.715. [PMID: 11576977] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Tsilimingras D, Bates DW. Addressing postdischarge adverse events: a neglected area. Jt Comm J Qual Patient Saf. 2008;34:85–97. doi: 10.1016/s1553-7250(08)34011-2. [PMID: 18351193] [DOI] [PubMed] [Google Scholar]
  • 41.Al-Rashed SA, Wright DJ, Roebuck N, Sunter W, Chrystyn H. The value of inpatient pharmaceutical counselling to elderly patients prior to discharge. Br J Clin Pharmacol. 2002;54:657–64. doi: 10.1046/j.1365-2125.2002.01707.x. [PMID: 12492615] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Paasche-Orlow MK, Schillinger D, Greene SM, Wagner EH. How health care systems can begin to address the challenge of limited literacy. J Gen Intern Med. 2006;21:884–7. doi: 10.1111/j.1525-1497.2006.00544.x. [PMID: 16881952] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Paasche-Orlow MK, Parker RM. Improving the effectiveness of patient education: a focus on limited health literacy. In: King TE, Wheeler MB, editors. Medical Management of Vulnerable and Underserved Patients: Principles, Practice, and Populations. McGraw Hill; New York: 2007. pp. 101–9. [Google Scholar]
  • 44.Friedman B, Basu J. The rate and cost of hospital readmissions for preventable conditions. Med Care Res Rev. 2004;61:225–40. doi: 10.1177/1077558704263799. [PMID: 15155053] [DOI] [PubMed] [Google Scholar]
  • 45.National Quality Forum National quality forum endorses consensus standards for quality hospital care: Patient safety in hospitals focus of 48 NQF-endorsed measures. Accessed at www.qualityforum.org/news/releases/051508-endorsed-measures.asp on 21 May 2008.
  • 46.Marquis KH, Marquis MS, Newhouse JP. The measurement of expenditures for outpatient physician and dental services: methodological findings from the health insurance study. Med Care. 1976;14:913–31. doi: 10.1097/00005650-197611000-00002. [PMID: 824510] [DOI] [PubMed] [Google Scholar]
  • 47.Roberts RO, Bergstralh EJ, Schmidt L, Jacobsen SJ. Comparison of self-reported and medical record health care utilization measures. J Clin Epidemiol. 1996;49:989–95. doi: 10.1016/0895-4356(96)00143-6. [PMID: 8780606] [DOI] [PubMed] [Google Scholar]
  • 48.Ritter PL, Stewart AL, Kaymaz H, Sobel DS, Block DA, Lorig KR. Self-reports of health care utilization compared to provider records. J Clin Epidemiol. 2001;54:136–41. doi: 10.1016/s0895-4356(00)00261-4. [PMID: 11166528] [DOI] [PubMed] [Google Scholar]
  • 49.National Quality Forum . Safe practices for better healthcare 2006 update. A consensus report. National Quality Forum; Washington, DC: 2007. [Google Scholar]

RESOURCES