Abstract
Objective
Recommended as a ‘universal precaution’ for improving provider–patient communication, teach-back has a limited evidence base. Discharge from the emergency department (ED) to home is an important high-risk transition of care with potential for miscommunication of critical information. We examined whether teach-back improves: comprehension and perceived comprehension of discharge instructions and satisfaction among patients with limited health literacy (LHL) in the ED.
Methods
We performed a randomized, controlled study among adult patients with LHL, to teach-back or standard discharge instructions. Patients completed an audio-recorded structured interview evaluating comprehension and perceived comprehension of (1) diagnosis, (2) ED course, (3) post-ED care, and (4) reasons to return and satisfaction using four Consumer Assessment of Healthcare Providers and Systems questions. Concordance with the medical record was rated using a five-level scale. We analyzed differences between groups using multivariable ordinal logistic regression.
Results
Patients randomized to receive teach-back had higher comprehension of post-ED care areas: post-ED medication (P < 0.02), self-care (P < 0.03), and follow-up instructions (P < 0.0001), but no change in patient satisfaction or perceived comprehension.
Conclusion
Teach-back appears to improve comprehension of post-ED care instructions but not satisfaction or perceived comprehension. Our data from a randomized, controlled study support the effectiveness of teach-back in a busy clinical setting. Further research is needed to test the utility and feasibility of teach-back for routine use including its impacts on distal outcomes.
Keywords: Emergency department, Health literacy, Intervention studies, Physician–patient relations, Teach-back communication
Introduction
Health literacy is ‘the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions.’1 Limited health literacy (LHL) is widely recognized as a major determinant of health outcomes, affecting nearly half of American adults and estimated by the Institute of Medicine (IOM) to cost $73 billion annually.2 Poor health literacy is said to be ‘a stronger predictor of a person’s health than age, income, employment status, education level, and race’.3–6 LHL is associated with lower health status, less knowledge about chronic disease self-management, lower rates of medication adherence, and higher rates of acute health care utilization in patients with chronic diseases.1,4,7–9 The mechanisms by which LHL affects health are complicated and multifactorial10; communication characteristics of the health care system can contribute to poor health outcomes.
The IOM recently identified high-risk situations, such as transitions of care, as contexts in which to improve provider–patient communication and assure that patients fully understand information.11 Discharge from the emergency department (ED) to home is recognized as a high-risk transition of care with potential for miscommunication of important information.11–18 This is of growing importance as the ED plays an increasing role as a critical access point into the health care system.19,20 ED visits increased by 30% in the past decade, and EDs now provide nearly one third of all acute care visits in the USA, and more acute care for the uninsured than all other settings combined, particularly for minority groups.21,22 ED-based studies demonstrate that patients have a limited recall of discharge information.23,24 Though compliance with instructions is associated with comprehension,25 this is infrequently assessed at discharge.26 Further, patients may not recognize when they have limited understanding.27,28–30 The prevalence of LHL may be higher in the ED than in the general population, with estimates ranging from 10.5 to 88%, depending on the type and location of the ED and the screening instrument used.28–31
Many major health care organizations have made health literacy a priority in policy and research agendas.32–35 Despite increasing attention,5,36–38 few studies have described successful communication interventions to improve comprehension and/or patient outcomes, with none we are aware of in the ED setting.9 Though providers generally believe in the effectiveness of commonly recommended techniques to improve communication with patients, many report not using them.39,40 Effective and feasible interventions to improve provider–patient communication at high-risk transitions of care are greatly needed.11,41
The teach-back technique, whereby a patient is prompted to ‘teach-back’ to a provider the information conveyed and receive clarifying feedback, is an often-recommended intervention to improve and confirm comprehension.42–44 The teach-back approach has a basis in cognitive psychology experiments showing that repeating short sequences of information helps to improve recall of information.45–47 However, though this technique is recommended as a ‘universal precaution’ by the Agency for Healthcare Research and Quality (AHRQ) and the National Quality Forum,42,48 there are few studies evaluating the effectiveness or feasibility of teach-back,49,50 and prior to this study there have been no randomized controlled studies assessing teach-back effects. Furthermore, we are not aware of any studies examining teach-back in the ED setting. An examination of teach-back in the ED setting is therefore important because a successful intervention might have significant benefits for a sizeable population of ED patients to improve comprehension, which could translate into improved outcomes. The primary objective of this research was to evaluate the efficacy of teach-back in improving comprehension at the time of discharge among LHL patients in the ED setting.
Methods
Study design
We conducted a randomized controlled study of teach-back vs. standard discharge instructions and report on this using the CONSORT guidelines.51 This study was approved by the Washington University institutional review board (IRB).
Study setting and participants
We performed this study in an urban academic ED and level 1 trauma center with over 95,000 annual visits. The hospital is located in the city of St. Louis, MO, USA, which was designated by a 2003 report as a ‘hot spot’ for low health literacy.52 All orders and documentation were entered in the ED electronic medical record (EMR; Allscripts HealthMatics), which includes a computerized ED tracking board and physician order entry.
All patients aged 18 and older being discharged from the ED were eligible for participation. We approached patients for enrollment in the ED between 27 June 2012 and 15 August 2012 between the hours of 6 a.m. and 12 a.m. using convenience sampling. Based on our prior study evaluating feasibility and diagnostic accuracy of various instruments in the ED, we chose the Rapid Estimate of Adult Literacy in Medicine-Revised (REALM-R), a validated and widely used measure of health literacy, to identify patients for this study.28,36 Consenting patients scoring six or less on the REALM-R (consistent with LHL) were eligible for enrollment. Exclusion criteria also included aphasia, non-English speaking, mental handicap, psychiatric chief complaint, too high illness acuity as determined by the treating physician, insurmountable communication barrier, evaluations for sexual assault, and clinical intoxication.
Study protocol
Medical student research assistants (RAs) completed training on health literacy, on approaching and enrolling patients, administering the REALM-R, the teach-back technique, audio-recording interviews, and data collection, with observed mock and initial interviews. Nursing staff also underwent basic training in teach-back, including a group education session on the concepts of teach-back with provision of reference materials, and demonstration sessions between the nurses and a Clinical Nurse Specialist (JW).
After screening with the REALM-R, consenting and eligible participants were asked for demographic information. Patients were randomized based upon an odd or even last digit in their medical record number to either the teach-back or the standard discharge instructions group. Checking a box in the EMR launched an icon on the ED tracking board informing the nurse that the RA would attend the discharge. When the treating physician began documenting discharge instructions, this launched a page to the RA, providing approximately 10 minutes of advanced notice prior to discharge. The RA informed the nurse whether the patient would be receiving teach-back or standard discharge instructions. For teach-back patients, the nurse would provide discharge instructions with the RA present, then the patient was asked by the RA to repeat back their understanding in their own words, related to the specific domains of interest, with the nurse correcting and misunderstandings. Discharge instructions and post-discharge interviews were audio-recorded for teach-back patients.
Following discharge, patients participated in structured interviews, which consisted of four questions related to patient satisfaction with aspects of their ED care and instructions and sets of questions assessing comprehension and perceived comprehension of four domains: diagnosis, ED care, post-ED care, and return instructions. Perceived comprehension questions asked patients to indicate their comprehension (poor, fair, good, very good, excellent) and attempted to evaluate the degree of difficulty in comprehending (not at all, a little, moderately, quite a bit, extremely). All data were entered into laptop computers and then uploaded to a secure server. Audio-recordings were captured by a microphone on the laptop computers and were loaded into a database. The RAs read aloud all questions, which were also displayed on the laptop screen for the patient to view. RAs performed all data entry other than for responses to satisfaction questions. Satisfaction questions were printed on a laminated card so they could be read aloud with the computer touchscreen rotated to face the patient to allow patients to make their selections with RAs blinded to their responses. The order in which the outcome measures were assessed was rotated over the course of the study period so as to limit any order bias.
Outcome measures
The study’s primary outcome measures were comprehension and perceived comprehension of discharge instructions. Evaluations of comprehension followed the methodology of Engel et al.27 Perceived comprehension was evaluated using questions directed at understanding of and difficulty with each domain. To determine comprehension, two senior emergency medicine residents (MG and YK) reviewed audio-recorded responses and information available in the medical record to assess the level of concordance between these two sources. Concordance was ranked on a five-level scale: no concordance, minimal concordance, partial concordance, near concordance, and complete concordance. We performed dual review of 25 cases with inter-rater reliability ranging from k = 0.44 to 0.79 across the domains. The study’s secondary outcome was patient satisfaction, which was evaluated using four items derived from the AHRQ’s validated Consumer Assessment of Healthcare Providers and Systems questionnaires.
Questions included (1) whether the medical team explained things in a way that was easy to understand, (2) whether the medical team spent enough time with the patient, (3) satisfaction with the quality of the discharge instructions provided, and (4) whether the patient would recommend this ED to friends and family. All questions had three-level responses.
Data Analysis
Analyses were conducted using SAS version 9.4 (SAS, Inc., Cary, NC, USA). We estimated our target enrollment of 250 based upon detecting a difference between groups in the proportion with complete vs. other concordance level (Figure 1). Using a value for the control group based on the Engel paper,27 (62% concordant), and with 80% power and an alpha-value of 0.05 to detect a 10% difference in concordance between groups, we needed 100 patients in each group. We included an additional 25 per arm to account for those not completing the protocol, having inaudible recordings, etc.
Figure 1.
Study enrollment and randomization diagram.
We examined differences in demographics between randomized groups using chi-squared tests. Bivariate associations between study group (teach-back vs. standard discharge) and study outcomes (i.e. comprehension, perceived comprehension, and satisfaction) were assessed using Mantel-Hanzel chi-squared tests. Multivariable ordinal logistic regression models were then built to examine the effects of study group on each outcome variable, adjusting for race. Data were analyzed with patients grouped as randomized consistent with an intention to treat analysis. Inter-rater reliability was assessed using the kappa-statistic. Statistical significance was assessed as P < 0.05.
Results
We approached 835 patients, of whom 408 were eligible to participate and consented (Fig. 1), with 212 (51.9%) randomized to teach-back and 196 to standard discharge instructions. Two-hundred and fifty-four (127 in each group) completed the protocol, comprising the analytic sample and completing our enrollment target. Randomization was largely successful with no differences observed in age, gender, or educational attainment. However there was a difference in the allocation of race between groups in the analytic sample (Table 1) and we therefore controlled for race in multivariable analyses.
Table 1.
Characteristics of patients by randomized study group in the analytic sample (N=254)
| Overall | Standard discharge (n=127) |
Teach-back (n=127) |
Χ2 | P value |
|
|---|---|---|---|---|---|
| N (%) | N (%) | N (%) | |||
| Race | 6.21 | 0.01 | |||
| White/Other | 31 (12.2) | 9 (7.1) | 22 (17.3) | ||
| Black | 223 (87.8) | 118 (92.9) | 105 (82.7) | ||
| Gender | 0.02 | 0.90 | |||
| Male | 103 (40.6) | 52 (40.9) | 51 (40.2) | ||
| Female | 151 (59.5) | 75 (59.1) | 76 (59.8) | ||
| Education | 1.98 | 0.37 | |||
| Less than high school | 94 (37.0) | 50 (39.4) | 44 (34.7) | ||
| High school diploma | 111 (43.7) | 50 (39.4) | 61 (48.0) | ||
| Some college or higher | 49 (19.3) | 27 (21.3) | 22 (17.3) | ||
| Age | Mean (SD) | Mean (SD) | Mean (SD) | t | |
| 35.4 (13.0) | 34.7 (12.8) | 36.0 (13.2) | −0.82 | 0.42 |
In bivariate analysis, we observed a significant difference by study group in comprehension of two areas within the post-ED care domain, post-ED self-care (P < 0.02), and post-ED follow-up (P < 0.0001), and a marginal difference for post-ED medications (P = 0.054), with higher comprehension in the teach-back group (Table 2). We found no differences in perceived comprehension between the teach-back group vs. those receiving standard discharge instructions (Table 3) and no differences in patient satisfaction between these groups (Table 4).
Table 2.
Bivariate associations between study group and comprehension
| Outcome | Overall N (%) |
Standard discharge N (%) |
Teach-back N (%) |
Mantel- Hanzel χ2 |
P- value |
|---|---|---|---|---|---|
| Diagnosis, n=220 | 1.62 | 0.20 | |||
| No | 4 (1.8) | 2 (1.8) | 2 (1.9) | ||
| Minimal | 6 (2.7) | 4 (3.6) | 2 (1.9) | ||
| Partial | 37 (16.8) | 22 (19.6) | 15 (13.9) | ||
| Near | 61 (27.7) | 31 (27.7) | 30 (27.8) | ||
| Complete | 112 (50.9) | 53 (47.3) | 59 (54.6) | ||
| Testing in ED, n=219 | 0.009 | 0.93 | |||
| No | 5 (2.3) | 2 (1.8) | 3 (2.8) | ||
| Minimal | 8 (3.7) | 6 (5.4) | 2 (1.9) | ||
| Partial | 26 (11.9) | 9 (8.0) | 17 (15.9) | ||
| Near | 50 (22.8) | 30 (26.8) | 20 (18.7) | ||
| Complete | 130 (59.4) | 65 (58.0) | 65 (60.8) | ||
| Treatment in ED, n=217 | 0.90 | 0.34 | |||
| No | 6 (2.8) | 2 (1.8) | 4 (3.7) | ||
| Minimal | 12 (5.5) | 7 (6.4) | 5 (4.7) | ||
| Partial | 28 (12.9) | 17 (15.5) | 11 (10.3) | ||
| Near | 47 (21.7) | 27 (24.6) | 20 (18.7) | ||
| Complete | 124 (57.1) | 57 (51.8) | 67 (62.6) | ||
| Post-ED medications, n=219 | 3.71 | 0.054 | |||
| No | 6 (2.7) | 3 (2.7) | 3 (2.8) | ||
| Minimal | 7 (3.2) | 4 (3.6) | 3 (2.8) | ||
| Partial | 29 (13.2) | 19 (17.0) | 10 (9.4) | ||
| Near | 53 (24.2) | 32 (28.6) | 21 (19.6) | ||
| Complete | 124 (56.6) | 54 (48.2) | 70 (65.4) | ||
| Post-ED self-care, n=206 | 5.34 | 0.02 | |||
| No | 6 (2.9) | 5 (4.7) | 1 (1.0) | ||
| Minimal | 10 (4.9) | 5 (4.7) | 5 (5.0) | ||
| Partial | 33 (16.0) | 22 (20.8) | 11 (11.0) | ||
| Near | 44 (21.4) | 23 (21.7) | 21 (21.0) | ||
| Complete | 113 (54.9) | 51 (48.1) | 62 (62.0) | ||
| Post-ED follow up, n=221 | 16.75 | <.0001 | |||
| No | 12 (5.4) | 8 (7.1) | 4 (3.7) | ||
| Minimal | 10 (4.5) | 7 (6.2) | 3 (2.8) | ||
| Partial | 33 (14.9) | 26 (23.0) | 7 (6.5) | ||
| Near | 45 (20.4) | 27 (23.9) | 18 (16.7) | ||
| Complete | 121 (54.8) | 45 (39.8) | 76 (70.4) | ||
| Return instructions, n=213 | 0.62 | 0.43 | |||
| No | 13 (6.1) | 9 (8.2) | 4 (3.9) | ||
| Minimal | 13 (6.1) | 2 (1.8) | 11 (10.7) | ||
| Partial | 36 (16.9) | 24 (21.8) | 12 (11.7) | ||
| Near | 76 (35.7) | 40 (36.4) | 36 (35.0) | ||
| Complete | 75 (35.2) | 35 (31.8) | 40 (38.8) |
Table 3.
Bivariate associations between study group and perceived comprehension
| Outcome | Overall N (%) |
Standard discharge N (%) |
Teach- back N (%) |
Mantel- Hanzel χ2 |
P-value |
|---|---|---|---|---|---|
| Understanding of diagnosis, N=245 | 0.15 | 0.70 | |||
| Poor | 1 (0.4) | 1 (0.8) | 0 (0.0) | ||
| Fair | 16 (6.5) | 6 (4.8) | 10 (8.3) | ||
| Good | 46 (18.8) | 24 (19.4) | 22 (18.2) | ||
| Very good | 77 (31.4) | 39 (31.5) | 38 (31.4) | ||
| Excellent | 105 (42.9) | 54 (43.6) | 51 (42.2) | ||
| Difficulty with diagnosis, N=245 | 1.07 | 0.30 | |||
| Extremely | 4 (1.6) | 0 (0.0) | 4 (3.3) | ||
| Quite a bit | 14 (5.7) | 9 (7.3) | 5 (4.1) | ||
| Moderately | 18 (7.4) | 10 (8.1) | 8 (6.6) | ||
| Little | 40 (16.3) | 13 (10.5) | 27 (22.3) | ||
| Not at all | 169 (69.0) | 92 (74.2) | 77 (63.6) | ||
| Understanding of care by ED, N=244 | 0.03 | 0.87 | |||
| Poor | 4 (1.6) | 0 (0.0) | 4 (3.3) | ||
| Fair | 13 (5.3) | 7 (5.7) | 6 (5.0) | ||
| Good | 43 (17.6) | 23 (18.6) | 20 (16.7) | ||
| Very good | 76 (31.2) | 42 (33.9) | 34 (28.3) | ||
| Excellent | 108 (44.3) | 52 (41.9) | 56 (46.7) | ||
| Difficulty with care by ED, N=243 | 0.27 | 0.61 | |||
| Extremely | 3 (1.2) | 0 (0.0) | 3 (2.5) | ||
| Quite a bit | 10 (4.1) | 7 (5.7) | 3 (2.5) | ||
| Moderately | 10 (4.1) | 5 (4.0) | 5 (4.2) | ||
| Little | 25 (10.3) | 10 (8.1) | 15 (12.6) | ||
| Not at all | 195 (80.3) | 102 (82.3) | 93 (78.2) | ||
| Understanding of self care, N=243 | 1.74 | 0.19 | |||
| Poor | 1 (0.4) | 0 (0.0) | 1 (0.8) | ||
| Fair | 9 (3.7) | 6 (4.8) | 3 (2.5) | ||
| Good | 29 (11.9) | 19 (15.3) | 10 (8.4) | ||
| Very good | 76 (31.3) | 37 (29.8) | 39 (32.8) | ||
| Excellent | 128 (52.7) | 62 (50.0) | 66 (55.5) | ||
| Difficulty with self care, N=243 | 0.26 | 0.61 | |||
| Extremely | 1 (0.4) | 1 (0.8) | 0 (0.0) | ||
| Quite a bit | 9 (3.7) | 4 (3.2) | 5 (4.2) | ||
| Moderately | 8 (3.3) | 6 (4.8) | 2 (1.7) | ||
| Little | 14 (5.8) | 6 (4.8) | 8 (6.7) | ||
| Not at all | 211 (86.8) | 107 (86.3) | 104 (87.4) | ||
| Understanding of return instructions, N=243 | 0.09 | 0.77 | |||
| Poor | 14 (5.8) | 6 (4.8) | 8 (6.7) | ||
| Fair | 11 (4.5) | 7 (5.7) | 4 (3.4) | ||
| Good | 40 (16.5) | 21 (16.9) | 19 (16.0) | ||
| Very good | 59 (24.3) | 32 (25.8) | 27 (22.7) | ||
| Excellent | 119 (49.0) | 58 (46.8) | 61 (51.3) | ||
| Difficulty with return instructions, N=242 | 0.008 | 0.93 | |||
| Extremely | 5 (2.1) | 3 (2.4) | 2 (1.7) | ||
| Quite a bit | 11 (4.6) | 5 (4.0) | 6 (5.1) | ||
| Moderately | 10 (4.1) | 5 (4.0) | 5 (4.2) | ||
| Little | 18 (7.4) | 9 (7.3) | 9 (7.6) | ||
| Not at all | 198 (81.8) | 102 (82.3) | 96 (81.4) |
Table 4.
Bivariate associations between study group and patient satisfaction
| Outcome | Overall N (%) |
Standard discharge N (%) |
Teach-back N (%) |
Mantel- Hanzel χ2 |
P-value |
|---|---|---|---|---|---|
| Easy to understand instructions, N=241 | 1.31 | 0.25 | |||
| No | 3 (1.2) | 0 (0.0) | 3 (2.5) | ||
| Yes, somewhat | 41 (17.0) | 20 (16.4) | 21 (17.7) | ||
| Yes, definitely | 197 (81.7) | 102 (83.6) | 95 (79.8) | ||
| Recommend ED, N=238 | 0.03 | 0.87 | |||
| Definitely no | 12 (5.0) | 6 (5.0) | 6 (5.1) | ||
| Probably no | 15 (6.3) | 7 (5.8) | 8 (6.8) | ||
| Probably yes | 65 (27.3) | 33 (27.5) | 32 (27.1) | ||
| Definitely yes | 146 (61.3) | 74 (61.7) | 72 (61.0) | ||
| Enough time with patient, N=219 | 1.33 | 0.25 | |||
| No | 19 (8.7) | 9 (8.0) | 10 (9.4) | ||
| Yes, somewhat | 51 (23.3) | 22 (19.6) | 29 (27.1) | ||
| Yes, definitely | 149 (68.0) | 81 (72.3) | 68 (63.6) | ||
| Satisfied with quality of instructions, N=217 | 0.04 | 0.85 | |||
| Very dissatisfied | 1 (0.5) | 0 (0.0) | 1 (0.9) | ||
| Somewhat dissatisfied | 4 (1.8) | 3 (2.7) | 1 (0.9) | ||
| Neither satisfied nor dissatisfied | 1 (0.5) | 1 (0.9) | 0 (0.0) | ||
| Somewhat satisfied | 27 (12.4) | 11 (10.0) | 16 (15.0) | ||
| Very satisfied | 184 (84.8) | 95 (86.4) | 89 (83.2) |
In the multivariable ordinal logistic regression models, we found no significant differences between teach-back and standard discharge for perceived comprehension or patient satisfaction (Table 5). However, controlling for race, patients who received teach-back had significantly higher comprehension in three items comprising the post-ED care domain compared with those patients who received standard discharge: post-ED medications (OR: 1.84; 95% CI: 1.09–3.12), post-ED self-care (OR: 1.83; 95% CI: 1.07–3.13), and post-ED follow-up instructions (OR: 3.61; 2.09–6.22).
Table 5.
Effect of teach-back compared with standard discharge on study outcomes in multivariable ordinal logistic regression models.
| Outcomes | n | Odds ratio* (95% CI) | P-value |
|---|---|---|---|
| Comprehension | |||
| Diagnosis | 220 | 1.35 (0.81–2.25) | 0.25 |
| Testing in ED | 219 | 1.01 (0.60–1.72) | 0.96 |
| Treatments in ED | 217 | 1.60 (0.94–2.73) | 0.08 |
| Post-ED medications | 219 | 1.84 (1.09–3.12) | 0.02 |
| Post-ED self-care | 206 | 1.83 (1.07–3.13) | 0.03 |
| Post-ED follow-up | 221 | 3.61 (2.09–6.22) | < 0.0001 |
| Return instructions | 213 | 1.26 (0.76–2.07) | 0.37 |
| Perceived comprehension | |||
| Understanding of diagnosis | 245 | 0.99 (0.62–1.58) | 0.97 |
| Difficulty with diagnosis | 245 | 0.68 (0.40–1.17) | 0.17 |
| Understanding of ED care | 244 | 1.13 (0.71–1.81) | 0.61 |
| Difficulty with ED care | 243 | 0.78 (0.41–1.47) | 0.44 |
| Understanding of self-care | 243 | 1.43 (0.88–2.33) | 0.15 |
| Difficulty with self-care | 243 | 1.18 (0.56–2.50) | 0.67 |
| Understanding of return instructions | 243 | 1.15 (0.71–1.84) | 0.57 |
| Difficulty with return instructions | 242 | 0.91 (0.47–1.74) | 0.76 |
| Patient satisfaction | |||
| Satisfied with instructions | 241 | 0.73 (0.38–1.42) | 0.35 |
| Recommend ED | 238 | 0.94 (0.56–1.57) | 0.81 |
| Satisfied with ED time | 219 | 0.68 (0.39–1.21) | 0.19 |
| Satisfied with discharge instructions | 217 | 0.78 (0.37–1.65) | 0.51 |
Controlling for race.
Discussion
Over the past two decades there has been increasing recognition of the role of health literacy in patient outcomes. In the ED, the main context in which this has been long recognized is in the transition of care from the ED to home for patients being discharged.54,55 Discharge from the
ED is but one example of the high-risk context of transitions of care in medicine, which are a focus of the recent IOM report regarding attributes of health literate organizations.11 Despite this focus and the importance of transitions of care, however, few interventions have been shown to improve patient comprehension and downstream outcomes following ED discharge.
The search for interventions to improve outcomes among patients with LHL has been challenging.9 Though teach-back is a provider-level intervention with face validity for improving communication in health care, the technique has been advanced for this purpose based on limited data to support its effectiveness. In a direct observation study of 38 physicians and 74 diabetic patients, patients for whom teach-back was used were more likely to have good glycemic control than other patients.44 Teach-back has been used to assess comprehension of informed consent and privacy information,56 and in a descriptive study, improved comprehension of a plain language consent form.4 However, in a randomized study, adding a teach-back component did not improve knowledge retention at 2 weeks compared with a multimedia diabetes education program only.46
We are not aware of other studies that have explored the use of teach-back in the ED. Specifically, no prior randomized controlled studies have examined the efficacy of teach-back in improving comprehension at discharge among patients with LHL. Therefore this study provides essential data that teach-back may improve some aspects of comprehension in the ED setting related to post-ED care, such as medications, self-care, and follow-up. Interestingly, these were the same areas with the highest deficiencies identified in the study by Engel et al. Improvements in these areas of comprehension may be important for downstream patient outcomes; adherence to follow-up and medication instructions is critical for reducing patient morbidity and mortality, return ED visits, and use of other health services.57–59 Further research is needed to examine whether improvements in comprehension of post-ED care information leads to improvements in these more distal outcomes.
Consistent with findings in the same Engel study, in which patients demonstrating deficient comprehension perceived difficulty with comprehension only 20% of the time, relatively few patients in our study perceived difficulty understanding the discharge instructions in both groups, but only about half had complete concordance with the information in their medical record.27 These findings suggest that patients may not know whether or not they understood discharge information until they later need to act on the information. Patients may also be reluctant to state that they did not understand information to a provider,43 causing discordance between comprehension and self-reported perceived comprehension as assessed here. Such discrepancies between comprehension and perceived comprehension suggest that patients might not initiate questions and that provider interventions like teach-back may be critical to improving provider–patient communication.
We were somewhat surprised to find no differences in our measures of patient satisfaction between groups, if for no other reason than that being randomized to teach-back would likely result in increased nurse time spent in the room at the time of discharge discussing the ED course and answering remaining questions. This raises an important issue for the recommendation of teach back, which relates to the time this may take relative to ‘standard instructions.’ We recently evaluated the time required to perform various health literacy measures, presenting this information relative to the diagnostic accuracy of the measures in the ED.28 Only one study has evaluated the time burden of teach-back discharge methods, but none has done so in the ED setting.44 This is obviously important, as additional minutes can translate to potential impacts on bed turnover and throughput in busy settings, and may affect patient satisfaction. In addition to efficacy research, therefore, investigation of implementation outcomes such as feasibility, fidelity, and acceptability is needed.60 Other key questions for future research include whether there are other interventions that should be tested and compared to teach-back, and whether teach-back or other effective interventions, if found, are better advanced as a ‘universal precaution’ or in a more a targeted intervention for patients with LHL.
Limitations
This study has a number of limitations that bear mention. This was a single-center study that employed convenience sampling, as has been the case with nearly all ED-based health literacy studies. Patients with LHL who were anticipated to be discharged to home were our target group for enrollment. However, the timing of a discharge can be hard to predict. Discharges occurring after the end of recruitment hours (usually at midnight) and discharges taking place concurrently (when the RA was engaged with the discharge of another enrolled patient) were the main sources of loss of capture of consenting patients. Since this loss was equally distributed between groups and was unrelated to patient ID number, which was the basis for the randomization scheme, this should not have biased our results. In prior health literacy studies in our facility using convenience sampling, we did not observe differences in basic demographic information (e.g. age, race, and gender) for enrolled patients compared with those declining participation or to the general ED population. We have a very small non-English speaking population (< 2%) and did not attempt to include Spanish-speaking patients. Therefore, generalizability of findings to patients speaking other languages would need to be evaluated.
Our randomization scheme and method did not conceal our intervention from patients or nurses, introducing the possibility of co-intervention, ascertainment, or recall bias. While an alternative (or sham) intervention arm would have reduced these biases, the lack of data on the efficacy of teach-back suggested the importance of comparison with standard practice. We decided to randomize at the patient level, which has the potential for cross-contamination between groups in which experience with teach-back might impact the way that nurses provide discharge instructions for control patients, but which if present would bias findings toward the null hypothesis. Randomization at the nurse level would present significant challenges to implementation in this practice setting, as there are many different nurses on different schedules with sometimes dynamic shifting of locations of responsibility in the ED. Future studies are needed to examine the efficacy and implementation of teach-back with randomization at the level of the practice site. We based our study on the methodology of Engel et al.27 in which patients’ comprehension was assessed at the end of their ED visit after receiving discharge instructions. A delayed assessment of comprehension would allow the examination of retention of discharge information.
Audio-recordings included the discharge instructions provided to teach-back patients but not for patients receiving routine instructions. Our IRB felt we should not audio-record the control group discharge instructions without enrolling all the nurses as study staff, requiring them all to undergo Collaborative Institutional Training Initiative training and including them in our IRB submission. This was not feasible to do. Audio-recorded responses to comprehension questions were compared with the written medical record including the written discharge materials in order to determine concordance. To the extent that reviewers could have been influenced by hearing verbal discharge instructions provided to teach-back patients, this could conceivably have biased reviews toward improved concordance for the teach-back group. However, reviewers were instructed to compare responses to the written record, and since for both groups, nurses usually read patients their discharge instructions from the written materials provided, we did not feel this did presented a significant concern for bias. We are not aware of any harms or unintended consequences of the teach-back technique that we could identify from the data collected, but these issues could be examined further in implementation studies.
Conclusions
It is known that in the ED setting, physicians rarely confirm comprehension of instructions26 and that patients’ perceived comprehension may not accurately reflect their comprehension.27 In our randomized, controlled study, teach-back resulted in no improvement in patient satisfaction or perceived comprehension. However, teach-back appears to improve comprehension for post-ED care (i.e. medications, self-care, and follow-up instructions) among patients with limited health literacy in the ED setting. These findings provide support for the use of teach-back in a busy clinical setting, adding to the limited evidence base on the efficacy of the teach-back approach. To test the recommendation of using teach-back as a universal precaution, further research is needed comparing the effects of teach-back across health literacy strata. Research is also needed to evaluate the feasibility and acceptability of teach-back for routine use in busy clinical settings and the impacts this approach may have on distal outcome measures.
Acknowledgements
The authors would like to acknowledge Ralph O’Neil for assistance with technology procurement, programing, database management, and data retrieval and William Macmillan for assistance in preparation for data analysis.
Funding None.
Biographies
Richard T. Griffey, MD, MPH, is Associate Chief and Director of Patient Safety and Quality in the Division of Emergency Medicine at Washington University School of Medicine and is a Coursemaster and Scholar in the Washington University Institute for Public Health. His work focuses on patient safety, quality measurement and quality improvement, health literacy and communication and evidence-based imaging.
Nicole Shin is a fourth year medical student at the Washington University School of Medicine.
Solita Jones is a fourth year medical student at Meharry Medical College in Nashville, TN. Her interests are in health disparities and barriers to accessing care, preventative healthcare and public and global health.
Nnenna V. Aginam is a fourth year Medical Student at Meharry Medical College in Nashville, TN and a Magna Cum Laude graduate of Towson University in Baltimore, Maryland. She plans to pursue residency training in Emergency Medicine.
Maureen Gross, MD, is in her fourth year at Washington University Emergency Medicine Residency at Barnes-Jewish Hospital where she will be commencing a fellowship in sports medicine.
Yonitte Kinsella, MD, PhD, is in her fourth year at Washington University Emergency Medicine Residency at Barnes-Jewish Hospital.
Jennifer A. Williams, PhD, RN, ACNS-BC, is the Clinical Nurse Specialist in the Department of Emergency Services at Barnes-Jewish Hospital. Dr. William’s areas of focus are sepsis, emergency nursing competency development and public health.
Chris Carpenter, MD, MSc, is an Associate Professor and Director of Evidence Based Medicine in the Division of Emergency Medicine at Washington University in St. Louis, a Fellow of the American College of Emergency Physicians the American Geriatrics Society, and on the Editorial Boards of Academic Emergency Medicine, ACP Journal Club, and the Journal of the American Geriatrics Society. His research interests include diagnostics, dementia, falls prevention, and implementation science.
Melody Goodman, PhD, is an Assistant Professor in the Division of Public Health Sciences, Department of Surgery at Washington University School of Medicine. Dr. Goodman’s areas of focus are biostatistics, health disparities, community engaged research and minority and medically underserved populations.
Kimberly A. Kaphingst, ScD, is a Professor of Communication and Investigator at the Huntsman Cancer Institute at the University of Utah. Dr. Kaphingst’s areas of research focus are health literacy and risk communication.
Footnotes
Disclaimer statements
Contributors RTG and KAK were responsible for the study concept, design, and overall supervision of the project. CRC assisted with design and methodology. MG, WDM, and KAK performed the statistical analysis. JW was responsible for nurse training. NS, JS, and NA were responsible for recruitment and enrollment. MG and YK performed primary assessments of concordance. RTG drafted the manuscript. All authors participated in the review and revision of the manuscript.
Conflicts of interest None.
Ethics approval None.
References
- 1.Nielsen-Bohlman L, Panzer AM, Kindig DA, editors. Health literacy: a prescription to end confusion. Washington DC: The National Academies Press; 2004. [PubMed] [Google Scholar]
- 2.Health literacy: report of the Council on Scientific Affairs. Ad Hoc Committee on Health Literacy for the Council on Scientific Affairs, American Medical Association. J Am Med Assoc. 1999;281:552–557. [PubMed] [Google Scholar]
- 3.Baker DW, Parker RM, Williams MV, Clark WS, Nurss JR. The relationship of patient reading ability to self-reported health and use of health services. Am J Public Health. 1997;87:1027–1030. doi: 10.2105/ajph.87.6.1027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Sudore RL, Yaffe K, Satterfield S, Harris TB. Mehta in the elderly: the health, aging, and body composition study. J Gen Intern Med. 2006;21:806–812. doi: 10.1111/j.1525-1497.2006.00539.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Weiss BD, Hart G, McGee DL, D’Estelle S. Health status of illiterate adults: relation between literacy and health status among persons with low literacy skills. J Am Board Fam Pract. 1992;5:257–264. [PubMed] [Google Scholar]
- 6.Weiss BD. Health literacy and patient safety: help patients understand. In: The American Medical Association Foundation and the American Medical Association, editor. Manual for clinicians. 2nd ed. Chicago, IL: 2007. [Google Scholar]
- 7.Wolf MS, Gazmararian JA, Baker DW. Health literacy and functional status among older adults. Arch Intern Med. 2005;165:1946–1952. doi: 10.1001/archinte.165.17.1946. [DOI] [PubMed] [Google Scholar]
- 8.Baker DW, Wolf MS, Feinglass J, Thompson JA, Gazmararian JA, Huang J. Health literacy and mortality among elderly persons. Arch Intern Med. 2007;167:1503–1509. doi: 10.1001/archinte.167.14.1503. [DOI] [PubMed] [Google Scholar]
- 9.Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med. 2011;155:97–107. doi: 10.7326/0003-4819-155-2-201107190-00005. [DOI] [PubMed] [Google Scholar]
- 10.Paasche-Orlow MK, Wolf MS. Evidence does not support clinical screening of literacy. J Gen Intern Med. 2008;23:100–102. doi: 10.1007/s11606-007-0447-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Brach C, Keller D, Hernandez LM, Baur C, Parker R, Dreyer B. IOM Roundtable on Health Literacy; 2012; Institute of Medicine of the National Academy of Sciences. Washington DC: The National Academies Press; Ten attributes of health literate health care organizations; pp. 1–26. [Google Scholar]
- 12.Fermazin M, Tahan H, Lett JE, Richards J. Improving transitions of care: hospital to home. Washington DC: National Transitions of Care Coalition; 2009. pp. 1–75. [Google Scholar]
- 13.Hastings SN, Whitson HE, Purser JL, Sloane RJ, Johnson KS. Emergency department discharge diagnosis and adverse health outcomes in older adults. J Am Geriatr Soc. 2009;57(10):1856–1861. doi: 10.1111/j.1532-5415.2009.02434.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Hastings SN, Schmader KE, Sloane RJ, Weinberger M, Goldberg KC, Oddone EZ. Adverse health outcomes after discharge from the emergency department – incidence and risk factors in a veteran population. J Gen Intern Med. 2007;22(11):1527–1531. doi: 10.1007/s11606-007-0343-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Hastings SN, Oddone EZ, Fillenbaum G, Sloane RJ, Schmader KE. Frequency and predictors of adverse health outcomes in older Medicare beneficiaries discharged from the emergency department. Med Care. 2008;46(8):771–777. doi: 10.1097/MLR.0b013e3181791a2d. [DOI] [PubMed] [Google Scholar]
- 16.Hastings SN, Purser JL, Johnson KS, Sloane RJ, Whitson HE. Frailty predicts some but not all adverse outcomes in older adults discharged from the emergency department. J Am Geriatr Soc. 2008;56(9):1651–1657. doi: 10.1111/j.1532-5415.2008.01840.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Hastings SN, Barrett A, Weinberger M, et al. Older patients’ understanding of emergency department discharge information and its relationship with adverse outcomes. J Patient Safety. 2011;7(1):19–25. doi: 10.1097/PTS.0b013e31820c7678. et al. [DOI] [PubMed] [Google Scholar]
- 18.Forster AJ, Rose NG, van Walraven C, Stiell I. Adverse events following an emergency department visit. Qual Safety Health Care. 2007;16(1):17–22. doi: 10.1136/qshc.2005.017384. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Morganti KG, Bauhoff S, Blanchard JC, Abir M, Iyer N. The evolving role of emergency departments in the United States. Santa Monica, CA: RAND Corporation; 2013. [PMC free article] [PubMed] [Google Scholar]
- 20.Schuur JD, Venkatesh AK. The growing role of emergency departments in hospital admissions. N Engl J Med. 2012;367:391–393. doi: 10.1056/NEJMp1204431. [DOI] [PubMed] [Google Scholar]
- 21.Pitts SR, Carrier ER, Rich EC, Kellermann AL. Where Americans get acute care: increasingly, it’s not at their doctor’s office. Health Aff (Millwood) 2010;29:1620–1629. doi: 10.1377/hlthaff.2009.1026. [DOI] [PubMed] [Google Scholar]
- 22.Niska R, Bhuiya F, Xu J. National Hospital Ambulatory Medical Care Survey: 2007 emergency department summary. Natl Health Stat Rep. 2010;(26):1–31. [PubMed] [Google Scholar]
- 23.Zavala S, Shaffer C. Do patients understand discharge instructions? J Emerg Nurs. 2011;37(2):138–140. doi: 10.1016/j.jen.2009.11.008. [DOI] [PubMed] [Google Scholar]
- 24.Samuels-Kalow ME, Stack AM, Porter SC. Effective discharge communication in the emergency department. Ann Emerg Med. 2012;60(2):152–159. doi: 10.1016/j.annemergmed.2011.10.023. [DOI] [PubMed] [Google Scholar]
- 25.Clarke C, Friedman SM, Shi K, Arenovich T, Monzon J, Culligan C. Emergency department discharge instructions comprehension and compliance study. Can J Emerg Med Care. 2005;7(1):5–11. doi: 10.1017/s1481803500012860. [DOI] [PubMed] [Google Scholar]
- 26.Vashi A, Rhodes KV. “Sign right here and you’re good to go”: a content analysis of audiotaped emergency department discharge instructions. Ann Emerg Med. 2010;57:315–322. doi: 10.1016/j.annemergmed.2010.08.024. [DOI] [PubMed] [Google Scholar]
- 27.Engel KG, Heisler M, Smith DM, Robinson CH, Forman JH, Ubel PA. Patient comprehension of emergency department care and instructions: are patients aware of when they do not understand? Ann Emerg Med. 2009;53:454–461. doi: 10.1016/j.annemergmed.2008.05.016. [DOI] [PubMed] [Google Scholar]
- 28.Carpenter CR, Kaphingst KA, Goodman MS, Lin MJ, Melson AT, Griffey RT. Feasibility and diagnostic accuracy of brief health literacy and numeracy screening instruments in an urban emergency department. Acad Emerg Med. 2014;21(2):137–146. doi: 10.1111/acem.12315. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Griffey RT, Melson AT, Lin MJ, Carpenter CR, Goodman MS, Kaphingst KA. Does numeracy correlate with measures of health literacy in the emergency department? Acad Emerg Med. 2014;21(2):147–153. doi: 10.1111/acem.12310. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.McNaughton C, Wallston KA, Rothman RL, Marcovitz DE, Storrow AB. Short, subjective measures of numeracy and general health literacy in an adult emergency department. Acad Emerg Med. 2011;18:1148–1155. doi: 10.1111/j.1553-2712.2011.01210.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Herndon JB, Chaney M, Carden D. Health literacy and emergency department outcomes: a systematic review. Ann Emerg Med. 2011;57:334–345. doi: 10.1016/j.annemergmed.2010.08.035. [DOI] [PubMed] [Google Scholar]
- 32.Health Literacy: National Network of Libraries of Medicine [updated 2014 Aug 1] Available from: http://nnlm.gov/outreach/consumer/hlthlit.html.
- 33.Ask Me 3: National Patient Safety Foundation [updated 2014 Aug 1] Available from: http://www.npsf.org/for-healthcare-professionals/programs/ask-me-3/.
- 34.AMA health literacy program [updated 2014 Aug 1] Available from: http://www.ama-assn.org/ama/pub/about-ama/ama-foundation/our-programs/public-health/health-literacy-program.page.
- 35.Facts about patient-centered communications: The Joint Commission [updated 2014 Aug 1] Available from: http://www.jointcommission.org/facts_about_patient-centered_communications/.
- 36.Bass PF, Wilson JF, Griffith CH. A shortened instrument for literacy screening. J Gen Intern Med. 2003;18:1036–1038. doi: 10.1111/j.1525-1497.2003.10651.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Morris NS, MacLean CD, Chew LD, Littenberg B. The Single Item Literacy Screener: evaluation of a brief instrument to identify limited reading ability. BMC Fam Pract. 2006;7:21. doi: 10.1186/1471-2296-7-21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Parker RM, Baker DW, Williams WV, Nurss JR. The test of functional health literacy in adults: a new instrument for measuring patients’ literacy skills. J Gen Intern Med. 1995;10:537–541. doi: 10.1007/BF02640361. [DOI] [PubMed] [Google Scholar]
- 39.Schwartzberg JG, Cowett A, VanGeest J, Wolf MS. Communication techniques for patients with low health literacy: a survey of physicians, nurses, and pharmacists. Am J Health Behav. 2007;31(Suppl 1):S96–S104. doi: 10.5555/ajhb.2007.31.supp.S96. [DOI] [PubMed] [Google Scholar]
- 40.McCarthy DM, Cameron KA, Courtney DM, Vozenilek JA. Self-reported use of communication techniques in the emergency department. J Emerg Med. 2012;43(5):E355–E361. doi: 10.1016/j.jemermed.2012.02.033. [DOI] [PubMed] [Google Scholar]
- 41.Koh HK, Brach C, Harris LM, Parchman ML. A proposed ‘health literate care model’ would constitute a systems approach to improving patients’ engagement in care. Health Aff (Millwood) 2013;32(2):357–367. doi: 10.1377/hlthaff.2012.1205. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.DeWalt DA, Callahan LF, Hawk VH, Broucksou KA, Hink A, Rudd R. Health literacy universal precautions toolkit. Rockville, MD: Agency for Healthcare Research and Quality, Department of Health and Human Services; 2010. [Google Scholar]
- 43.Doak CC, Doak LG, Root JH. Teaching patients with low literacy skills. 2nd ed. xii. Philadelphia: J.B. Lippincott; 1996. p. 212. [Google Scholar]
- 44.Schillinger D, Piette J, Grumbach K, Wang F, Wilson C, Daher C. Closing the loop: physician communication with diabetic patients who have low health literacy. Archives of internal medicine. 2003;163:83–90. doi: 10.1001/archinte.163.1.83. [DOI] [PubMed] [Google Scholar]
- 45.Gupta P. Examining the relationship between word learning, non-word repetition, and immediate serial recall in adults. Q J Exp Psychol A. 2003;56(7):1213–1236. doi: 10.1080/02724980343000071. [DOI] [PubMed] [Google Scholar]
- 46.Kandula NR, Malli T, Zei CP, Larsen E, Baker DW. Literacy and retention of information after a multimedia diabetes education program and teach-back. J Health Commun. 2011;16:89–102. doi: 10.1080/10810730.2011.604382. [DOI] [PubMed] [Google Scholar]
- 47.Page MPA, Cumming N, Norris D, Hitch GJ, McNeil AM. Repetition learning in the immediate serial recall of visual and auditory materials. J Exp Psychol Learn. 2006;32(4):716–733. doi: 10.1037/0278-7393.32.4.716. [DOI] [PubMed] [Google Scholar]
- 48.DeWalt DA, Broucksou KA, Hawk V, Brach C, Hink A, Rudd R. Developing and testing the health literacy universal precautions toolkit. Nurs Outlook. 2011;59(2):85–94. doi: 10.1016/j.outlook.2010.12.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Baker DW, DeWalt DA, Schillinger D, Hawk V, Ruo B, Bibbins-Domingo K. “Teach to goal”: theory and design principles of an intervention to improve heart failure self-management skills of patients with low health literacy. J Health Commun. 2011;16(Suppl 3):73–88. doi: 10.1080/10810730.2011.604379. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Schenker Y, Fernandez A, Sudore R, Schillinger D. Interventions to improve patient comprehension in informed consent for medical and surgical procedures: a systematic review. Med Decis Making. 2011;31(1):151–173. doi: 10.1177/0272989X10364247. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.CONSORT Statement. 2010 Available from: http://www.consort-statement.org.
- 52.Lurie N, Martin LT, Ruder T, Escarce JJ, Ghosh-Dastidar B, Sherman D. Estimating and mapping health literacy in the state of Missouri. Santa Monica, CA: RAND Health for the Missouri Foundation for Health; 2009. Contract No.: WR-735-MFH. [Google Scholar]
- 53.Consumer Assessment of Healthcare Providers and Systems Agency for Healthcare Research and Quality. Available from: https://cahps.ahrq.gov.
- 54.Jolly BT, Scott JL, Feied CF, Sanford SM. Functional illiteracy among emergency department patients: a preliminary study. Ann Emerg Med. 1993;22:573–578. doi: 10.1016/s0196-0644(05)81944-4. [DOI] [PubMed] [Google Scholar]
- 55.Spandorfer JM, Karras DJ, Hughes LA, Caputo C. Comprehension of discharge instructions by patients in an urban emergency department. Ann Emerg Med. 1995;25:71–74. doi: 10.1016/s0196-0644(95)70358-6. [DOI] [PubMed] [Google Scholar]
- 56.Han JH, Bryce SN, Ely EW, Kripalani S, Morandi A, Shintani A. The effect of cognitive impairment on the accuracy of the presenting complaint and discharge instruction comprehension in older emergency department patients. Ann Emerg Med. 2011;57:662–671. doi: 10.1016/j.annemergmed.2010.12.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Simpson SH, Eurich DT, Majumdar SR, Padwal RS, Tsuyuki RT, Varney J. A meta-analysis of the association between adherence to drug therapy and mortality. Br Med J. 2006;333(7557):15–18. doi: 10.1136/bmj.38875.675486.55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Olshaker JS, Barish RA, Naradzay JF, Jerrard DA, Safir E, Campbell L. Prescription noncompliance: contribution to emergency department visits and cost. J Emerg Med. 1999;17(5):909–912. doi: 10.1016/s0736-4679(99)00109-2. [DOI] [PubMed] [Google Scholar]
- 59.DiMatteo MR, Giordani PJ, Lepper HS, Croghan TW. Patient adherence and medical treatment outcomes: a meta-analysis. Med Care. 2002;40(9):794–811. doi: 10.1097/00005650-200209000-00009. [DOI] [PubMed] [Google Scholar]
- 60.Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffey RT, Bunger AC. A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev. 2012;69:123–157. doi: 10.1177/1077558711430690. [DOI] [PMC free article] [PubMed] [Google Scholar]