Abstract
Reminder systems can improve compliance with care standards, yet the reminder delivery parameters and associations with other success factors have not been fully understood. In this study, we assessed patient preferences for reminder delivery in a psychiatry ambulatory service, using both quantitative and qualitative analyses. Results from a survey showed that most patients had a positive attitude to reminders for both scheduled (76%) and missed (89%) visits. Phone call (61%) delivered two days before an appointment (47%) was the most preferred type and time of reminder delivery. Logistic regressions on survey data showed that preferences of reminder delivery parameters were associated with service types and patient populations, which was cross-validated by the follow-up interviews with the staff at two ambulatory sites. A single-mode reminder delivering method cannot satisfy different types of patients. Intervention designs that involve building a system with a variety of methods customized to patient needs and balanced with administrative simplicity need to be further studied.
INTRODUCTION
Previous studies have shown that reminders targeting to clinicians and/or patients can effectively improve compliance with healthcare standards1–2. To ensure quality and efficiency of care, information technology is used to screen patient databases, to identify eligible patients and their providers, and to deliver reminders to target persons through specific channels3–6. Nevertheless, the success of a reminder system depends on many factors, such as the clinical problems addressed, patient populations, clinical settings, insurance coverage, and logistics, to name a few7–9. In addition, although the previous research have shown the effectiveness of reminders, the specific parameters of reminder delivery, such as targets (patients, clinicians, etc.), modes (phone calls, emails, regular mails, etc.), and intensities (frequency, time, etc.), are not thoroughly investigated10. Development of a reminder system thus requires careful planning to select the optimal combination of the reminder delivery parameters and to appropriately address their interactions with the other success factors.
In psychiatry ambulatory settings, patient “no-shows” for initial and follow-up appointments can interfere with effective clinic operations and have a negative impact on patient outcomes11. Development of a reminder system targeting to patients is a potential solution. Although reminder systems have been widely reported in the literature, few have been developed for use with patients in psychiatry ambulatory services12. In this setting, the targets of reminders are special (often vulnerable) populations, and therefore, assessment of patient preferences for reminder delivery is critical to the success of such a system.
In this paper, we report the assessment and analyses of patient preferences for reminder delivery in a psychiatry ambulatory service. This study is the first phase of a project to integrate informatics approaches to addressing patient “no-shows” at the University of Rochester Medical Center (URMC) Psychiatry Ambulatory Services. At this stage, our focus is to assess the preferred reminder delivery parameters and to analyze their optimal combinations in different service types and patient populations. The results of this phase will direct the intervention design for the next stage, which will focus on improving patients’ compliance with scheduled appointments.
METHODS
The study was performed at the URMC Department of Psychiatry Ambulatory Services where a broad spectrum of outpatient treatment across several settings is provided. The patient population consists of children, adolescents, adults, older adults, and families with a range of psychiatric and substance use disorders. The URMC Psychiatry Ambulatory Services receives approximately 150,000 visits annually.
To assess patient preferences for reminder delivery, we designed a simple survey instrument with four questions. These questions addressed the following parameters of reminder delivery: (1) patients’ acceptance of reminders, (2) preferred types of reminders (phone call, regular mail, email), (3) time and frequency of reminder delivery (one week before the appointment, two days before the appointment, the morning of the appointment day, combinations), and (4) needs of follow-up contacts for missed visits. These four questions were integrated into a regular anonymous patient satisfaction survey administered across the URMC Psychiatry Ambulatory Services during the months of May, June, July, and August of 2006. In addition to questions about service satisfaction, the regular survey also captured a patient’s age, gender, race/ethnicity, cultural identity, and religion, as well as the service program the patient was in.
We performed both quantitative and qualitative analyses on the survey data. For quantitative analyses, we classified the fourteen psychiatry ambulatory service programs at the UMRC into four groups based on similarity of service types and patient populations, as shown in Table 1. Specifically, group A includes three programs of general patient groups: the Adult General program provides a variety of services for adults with mental disorder of less severity or need either short-term intensive or long-term maintenance treatment; the Child/Adolescent program provides the same services for patients of younger age; and the Deaf Wellness Center provides a range of mental health treatment for deaf patients. Group B includes four programs that focus on a specific form of treatment: the Family Therapy Service provides family or relationship-based interventions; the Infectious Disease program treats patients with HIV/AIDS; the Medicine in Psychiatry program complements mental health treatment with medical care; and the Older Adult clinical specializes in treating an elderly population. Group C includes five programs, all provided within the Strong Ties clinic for patients who have severe, chronic mental illness, generally involving psychosis: the Lazos Fuertes program treats a Hispanic population; the Assertive Community Treatment (ACT) program is provided by case managers in the community; the Continuing Day Treatment (CDT) program provides long-term therapy for those need slower-paced recovery; the Clinic offers brief monthly medication visits for patients who cannot tolerate CDT programs; and the Young Adult program provides treatment to young adults to prevent chronicity. Group D includes two programs that focus specifically on chemical abuse and dependence: the Chemical Dependency program provides therapy focused on relapse prevention and recovery; and the Methadone Maintenance program provides methadone and therapy for patients recovering from heroin addiction.
Table 1.
Study Groups and Service Programs
| Study Group | Component Service Programs |
|---|---|
| Group A | Adult General
Child/Adolescent Deaf Wellness Center |
| Group B | Family Therapy Service
Infectious Disease Medicine in Psychiatry Older Adult |
| Group C | Lazos Fuertes
Strong Ties ACT Strong Ties CDT Strong Ties Clinic Strong Ties Young Adult |
| Group D | Strong Recovery: CD
Strong Recovery: Methadone |
As part of the assessment, we compared patient demographics and preferences for reminder delivery across the study groups. We used the chi-square test to measure the difference. Based on these analyses, we performed logistic regressions to model the predicting factors for reminder delivery preferences.
To validate the results from the quantitative analyses, we conducted semi-structured group interviews with the staff at two psychiatry ambulatory sites, Strong Recovery (group D) and Strong Ties (group C). These two sites were selected because they had higher incidence of “no-shows” and the patients from these two sites had unique patterns of preferences for reminder delivery (see the Result section). During the interviews, we presented the initial analyses of the survey and asked the staff for feedback on interpretation of the results as well as guidance on how to best approach the intervention design. Because of special patient population and resource limitation, we did not directly interview the patients but assumed that the providers’ feedback would provide a similar level of confidence for validation of the quantitative results.
RESULTS
Quantitative Analyses
In total there were 670 patients who participated in the survey and returned valid answers, with 267 in group A, 121 in group B, 214 in group C, and 68 in group D. There were significant differences in patient demographics across the study groups. The demographics data are shown in Table 2.
Table 2.
Patient Demographics
| Study Group | A | B | C | D | Total | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| Race/Ethnicity** | ||||||||||
| Caucasian | 191 | (72%) | 71 | (59%) | 87 | (41%) | 34 | (50%) | 383 | (57%) |
| African American | 38 | (14%) | 20 | (17%) | 47 | (22%) | 18 | (26%) | 123 | (18%) |
| Latino | 12 | (4%) | 15 | (12%) | 66 | (31%) | 7 | (10%) | 100 | (15%) |
| Other | 14 | (5%) | 12 | (10%) | 7 | (3%) | 7 | (10%) | 40 | (6%) |
| Unknown |
12
|
(4%)
|
3
|
(2%)
|
7
|
(3%)
|
2
|
(3%)
|
24
|
(4%)
|
| Gender** | ||||||||||
| Male | 79 | (30%) | 39 | (32%) | 115 | (54%) | 30 | (44%) | 263 | (39%) |
| Female | 168 | (63%) | 73 | (60%) | 81 | (38%) | 34 | (50%) | 356 | (53%) |
| Unknown |
20
|
(7%)
|
9
|
(7%)
|
18
|
(8%)
|
4
|
(6%)
|
51
|
(8%)
|
| Age* | ||||||||||
| <= 30 | 65 | (24%) | 29 | (24%) | 29 | (14%) | 17 | (25%) | 140 | (21%) |
| 31 – 65 | 186 | (70%) | 88 | (73%) | 171 | (80%) | 48 | (71%) | 493 | (74%) |
| >= 66 | 5 | (2%) | 1 | (1%) | 6 | (1%) | ||||
| Unknown |
11
|
(4%)
|
3
|
(2%)
|
14
|
(7%)
|
3
|
(4%)
|
31
|
(5%)
|
| Total | 267 | - | 121 | - | 214 | - | 68 | - | 670 | - |
p<0.001
p<0.05
Most (76%) patients believed that reminders would be helpful and would like to receive reminders about their upcoming visits. Phone call was the most preferred type of reminder in all groups except for group D, where regular mail was the first choice. One-time reminder sent two days before the appointment was the most popular choice for time and frequency in all groups except for group D, where more patients preferred the reminder to be sent one week before the appointment. A large majority of the patients (89%) would like to be contacted for another appointment if they missed a scheduled visit. There were significant differences in patients’ preferences for reminder delivery across the study groups. The patient preference data are shown in Table 3.
Table 3.
Patient Preferences for Reminder Delivery
| Study Group | A | B | C | D | Total | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| Would like reminders?** | ||||||||||
| Yes, definitely | 163 | (63%) | 42 | (35%) | 123 | (59%) | 32 | (48%) | 360 | (55%) |
| Yes, maybe | 41 | (16%) | 28 | (24%) | 47 | (23%) | 19 | (29%) | 135 | (21%) |
| No, I don’t think so | 39 | (15%) | 39 | (33%) | 20 | (10%) | 8 | (12%) | 106 | (16%) |
| No, definitely not | 15 | (6%) | 10 | (8%) | 17 | (8%) | 7 | (11%) | 49 | (8%) |
| Total valid answers
|
258
|
-
|
119
|
-
|
207
|
-
|
66
|
-
|
650
|
-
|
| Preferred reminders** | ||||||||||
| Phone call | 153 | (65%) | 43 | (53%) | 126 | (66%) | 22 | (36%) | 344 | (61%) |
| Regular mail | 24 | (10%) | 28 | (35%) | 43 | (23%) | 30 | (49%) | 125 | (22%) |
| 34 | (14%) | 2 | (2%) | 5 | (3%) | 1 | (2%) | 42 | (7%) | |
| Other | 24 | (10%) | 8 | (10%) | 16 | (8%) | 8 | (13%) | 56 | (10%) |
| Total valid answers
|
235
|
-
|
81
|
-
|
190
|
-
|
61
|
-
|
567
|
-
|
| Time and frequency** | ||||||||||
| 1 week before | 49 | (21%) | 28 | (34%) | 57 | (29%) | 25 | (40%) | 159 | (28%) |
| 2 days before | 135 | (57%) | 39 | (47%) | 76 | (39%) | 20 | (32%) | 270 | (47%) |
| Morning of visit day | 11 | (5%) | 3 | (4%) | 13 | (7%) | 4 | (6%) | 31 | (5%) |
| 1 week before + morning of visit day | 13 | (6%) | 12 | (6%) | 8 | (13%) | 33 | (6%) | ||
| 2 days before + morning of visit day | 13 | (6%) | 10 | (12%) | 17 | (9%) | 3 | (5%) | 43 | (7%) |
| Other | 15 | (6%) | 3 | (4%) | 19 | (10%) | 2 | (3%) | 39 | (7%) |
| Total valid answers
|
236
|
-
|
83
|
-
|
194
|
-
|
62
|
-
|
575
|
-
|
| Contact for missed visit?* | ||||||||||
| Yes | 221 | (91%) | 90 | (80%) | 180 | (91%) | 56 | (90%) | 547 | (89%) |
| No | 22 | (9%) | 22 | (20%) | 18 | (9%) | 6 | (10%) | 68 | (11%) |
| Total valid answers | 243 | - | 112 | - | 198 | - | 62 | - | 615 | - |
p<0.001,
p<0.05
To examine the relationships among study groups, demographics, and reminder preferences, we performed logistic regression analyses. Specifically, we included four variables, i.e., study group, gender, age, and race/ethnicity, into a model to predict the positive attitude to reminders. We then included the same set of variables into another model to predict the preferred reminder type of phone call. In addition, we added the preferred type of reminder as the fifth variable, together with the original four, into a third model to predict the choice of two days before the appointment as the preferred reminder time. Finally, we added the acceptance of reminders and the preferred time and frequency as the sixth and seventh variables into a fourth model to predict the needs of follow-up contact for missed visits. The results showed: (1) compared to those in group B, patients in group D were more likely to have a positive attitude to reminders; (2) compared to Caucasians, African Americans and Latinos were more likely to think that reminders would be helpful; (3) compared to those in group D, patients in group A and B were more likely to prefer reminders to be sent two days before an appointment; (4) compared to African Americans, Caucasians were more likely to want a two-days-before reminder; and (5) compared to those with a negative attitude to reminders, patients who had a positive attitude at the beginning were more likely to want a follow-up contact in case they missed a scheduled visit. The data of logistic regression analyses are shown in Table 4.
Table 4.
Logistic Regression Analyses
| Predicting Variables | Positive attitude to reminders
|
Preferring reminder type of phone call
|
Preferring reminders sent 2 days before visit
|
Need follow-up contact for missed visits
|
||||
|---|---|---|---|---|---|---|---|---|
| OR | 95% CI | OR | 95% CI | OR | 95% CI | OR | 95% CI | |
| Study group (reference: group D) | ||||||||
| Group A | 0.90 | 0.42 - 1.94 | 1.76 | 0.73 - 4.27 | 3.11 | 1.60 - 6.04 | 1.12 | 0.31 - 4.08 |
| Group B | 0.25 | 0.11 - 0.55 | 1.30 | 0.46 - 3.67 | 2.28 | 1.06 - 4.93 | 1.35 | 0.29 - 6.41 |
| Group C |
0.85
|
0.38 - 1.90
|
0.73
|
0.28 - 1.92
|
1.86
|
0.93 - 3.69
|
0.84
|
0.23 - 3.07
|
| Gender of male
|
0.73
|
0.47 - 1.11
|
1.57
|
0.94 - 2.63
|
0.94
|
0.64 - 1.37
|
0.94
|
0.44 - 2.00
|
| Age (reference: >= 66) | ||||||||
| <= 30 | 0.31 | 0.03 - 3.39 | - | - | 5.84 | 0.45 - 75.69 | - | - |
| 31 – 65 |
0.37
|
0.03 - 3.97
|
-
|
-
|
4.15
|
0.33 - 52.52
|
-
|
-
|
| Race/ethnicity (reference: Caucasian) | ||||||||
| African American | 2.53 | 1.39 - 4.61 | 1.13 | 0.61 - 2.08 | 0.54 | 0.34 - 0.88 | 0.95 | 0.37 - 2.42 |
| Latino |
5.95 |
2.44 - 14.51 |
0.69
|
0.30 - 1.60
|
0.79
|
0.46 - 1.34
|
1.75
|
0.46 - 6.66
|
| Positive attitude to reminders
|
-
|
-
|
-
|
-
|
-
|
-
|
3.50 |
1.50 - 8.14 |
| Preferring reminder type of phone call
|
-
|
-
|
-
|
-
|
1.39
|
0.84 - 2.32
|
0.58
|
0.24 - 1.39
|
| Preferring reminders sent 2 days before visit | - | - | - | - | - | - | 2.02 | 0.94 - 4.35 |
OR: odds ratio; CI: confidence interval; table cells in bold fonts are results with statistical significance
Qualitative Analyses
The interviews with the Strong Recovery (group D) staff cross-validated the findings from the survey. The sensitive issue of chemical dependency with the patients in this group explained the lower percentage of their preferences for phone call and higher percentage of preferences for regular mail. The staff commented that sometimes even the patients’ family members may not be aware of their problem and thus sending phone call reminders about chemical dependency service appointments might breach confidentiality. Based on this observation, the staff strongly underscored the need to have privacy safeguards built into all reminder delivery methods.
The interviews with the Strong Ties (group C) staff found that many of the patients in this group were in supervised settings or relied on case managers to help them keep their appointments. Therefore, they suggested that part of the reminder strategy might include group home staff or case managers as the reminder delivery targets. They also cautioned that such a focus needs to be as efficient as possible given the typically large workloads of the case managers.
DISCUSSION
Our assessment showed that 76% of the patients had a positive attitude to reminders. Since there is a strong association between patients’ beliefs and their health behaviors13, the high percentage of reminder acceptance indicates that it is promising to implement a reminder system in the study setting. Of special interest is that we found African Americans (odds ratio: 2.53, 95% confidence interval: 1.39–4.61) and Latinos (odds ratio: 5.95, 95% confidence interval: 2.44–14.51) were more likely to have a positive attitude to reminders when compared to Caucasians, although these patient populations typically had a higher “no-show” rate (group C and D). A potential explanation of this observation is that the “no-shows” are simply because they forget11. This implies that a reminder system is likely to be helpful to address the “no-show” problem when targeting to these patients. The lower reminder acceptance rate in group B may be attributed to the engagement conditions of these services. Patients in this group are more likely to be higher functioning and more likely to be able to handle their own schedules with less need for intervention (Family Therapy and Older Adults). They may also have interactions with concomitant treatment providers and therefore fall under a different scheduling system (Infectious Disease and Medicine in Psychiatry).
The results showed that there were significant differences in patients’ preferences for reminder delivery parameters across the study groups. Phone call (61%) and regular mail (22%) were the two most popular choices of reminder type; one week before visit (28%) and two days before visit (47%) were the two most popular choices of reminder time. Preferences for reminder time were strongly associated with study groups and race/ethnicity. These data indicate that single-mode reminder delivery may not match the various preferences of the patients. Providing multiple modes of reminder delivery that are tailored to specific groups of patients’ needs is a more promising approach. For example, a regular mail reminder delivered one week before an appointment is probably better for patients in group D, while a phone call reminder delivered two days before an appointment looks more attractive for patients in group C. Ideally, the reminder delivery mode should be customized to individual patient’s choice. In real world implementation, however, the strategies taken need to be simple and easy to manage, as reflected in the interview comments from Strong Ties staff. Thus, we need to further investigate where the balance point is.
Finally, we found that the original positive attitude to reminders was a predictor of the positive response to follow-up contact for missed visit (odds ratio: 3.50, 95% confidence interval: 1.50–8.14). This result indicates that reminders could be also useful as a strategy to re-schedule the missed appointments.
There are a few limitations in this study. First, we collected the study data from a survey. Although the follow-up interviews cross-validated the survey data, the findings drawn from this study need to be further examined in prospective studies when reminders are actually delivered. Second, our data only included patient demographics, study groups (service programs), and reminder preferences. Other factors, such as insurance coverage and logistic issues, may also have significant impacts on patients’ attitudes to reminders and delivery preferences. Further investigations are required to better understand how these variables interact with the reminder delivery parameters. Finally, understanding the relationships among the parameters of reminder delivery is only the first step. Our ultimate goal is to use reminders as behavioral interventions to reduce “no-show” rates and to improve clinical outcomes. These constitute our work for the next stage.
CONCLUSION
Patients in psychiatry ambulatory services are quite positive about a reminder scheme, but have a variety of preferences that depend on service programs, demographics, and combinations of reminder delivery parameters. A single-mode reminder delivering method cannot satisfy the needs of different types of patients. Intervention designs that involve building a system with a variety of reminder delivering methods customized to patient needs and balanced with administrative simplicity need to be further studied.
Acknowledgments
The authors are indebted to Joan Brisbane for collating and entering the data for this study and grateful to Patrick Seche, Joyce Koziol, David Szulgit, John Wegman, and the other staff at the Strong Recovery and Strong Ties for their help on site interviews. Dongwen Wang is partially supported by NIH grants 1 UL1 RR024160-1. John Crilly is partially supported by the New York State Residency Training Program through the URMC Department of Psychiatry.
References
- 1.Szilagyi PG, Bordley C, Vann JC, et al. Effect of patient reminder/recall interventions on immunization rate: a review. JAMA. 2000;284(14):1820–7. doi: 10.1001/jama.284.14.1820. [DOI] [PubMed] [Google Scholar]
- 2.Walsh JME, McDonald KM, Shojania KG, et al. Quality improvement strategies for hypertension management: a systemic review. Med Care. 2006;44(7):646–57. doi: 10.1097/01.mlr.0000220260.30768.32. [DOI] [PubMed] [Google Scholar]
- 3.Johnson KB, Feldman MJ. Medical informatics and pediatrics: decision-support systems. Arch Pediatr Adolesc Med. 1996;150(8):882–3. doi: 10.1001/archpedi.1995.02170250077014. [DOI] [PubMed] [Google Scholar]
- 4.Friedman RH. Automated telephone conversations to assess health behavior and deliver behavioral interventions. J Med Sys. 1998;22(2):95–102. doi: 10.1023/a:1022695119046. [DOI] [PubMed] [Google Scholar]
- 5.Shiffman RN, Liaw Y, Brandt CA, Corb GJ. Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. J Am Med Inform Assoc. 1999;6(2):104–14. doi: 10.1136/jamia.1999.0060104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Garg AX, Adhikari NK, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223–38. doi: 10.1001/jama.293.10.1223. [DOI] [PubMed] [Google Scholar]
- 7.Killaspy H, Baneriee S, King M, Lloyd M. Prospective controlled study of psychiatric outpatient non-attendance: characteristics and outcome. Brit J Psych. 2000;176(2):160–5. doi: 10.1192/bjp.176.2.160. [DOI] [PubMed] [Google Scholar]
- 8.Compton MT, Rudisch BE, Craw J, Thompson T, Owens DA. Predictors of missed first appointments at community mental health centers after psychiatric hospitalization. Psych Services. 2006;57(4):531–7. doi: 10.1176/ps.2006.57.4.531. [DOI] [PubMed] [Google Scholar]
- 9.Irigoyen M, Findley S, Wang D, et al. Challenges of immunization registry reminders at inner city practices. Ambul Pediatr. 2006;6(2):100–4. doi: 10.1016/j.ambp.2005.10.006. [DOI] [PubMed] [Google Scholar]
- 10.Wang D, Jenders RA. Model-based immunization information routing. Proc AMIA Symp. 2000:878–82. [PMC free article] [PubMed] [Google Scholar]
- 11.Bulloch AG, Adair CE, Patten SB. Forgetfulness: a role in non-compliance with antidepressant treatment. Can J Psych. 2006;51(11):719–22. doi: 10.1177/070674370605101110. [DOI] [PubMed] [Google Scholar]
- 12.Cannon DS, Allen SN. A comparison of the effects of computer and manual reminders on compliance with a mental health clinical practice guideline. J Am Med Inform Assoc. 2000;7(2):196–203. doi: 10.1136/jamia.2000.0070196. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Elder JP, Ayala GX, Harris S. Theories and intervention approaches to health-behavior change in primary care. Am J Prev Med. 1999;17(4):275–84. doi: 10.1016/s0749-3797(99)00094-x. [DOI] [PubMed] [Google Scholar]