A Randomized Trial of the Effect of Centralized Reminder/Recall on Immunizations and Preventive Care Visits for Adolescents

Abstract

Objective

To assess the impact of a managed care-based patient reminder/recall system on immunization rates and preventive care visits among low-income adolescents.

Methods

We conducted a randomized controlled trial between December 2009 and December 2010 that assigned adolescents aged 11–17 years to one of three groups: mailed letter, telephone reminders, or control. Publicly insured youths (n = 4, 115) were identified in 37 participating primary care practices. The main outcome measures were immunization rates for routine vaccines (meningococcus, pertussis, HPV) and preventive visit rates at study end.

Results

Intervention and control groups were similar at baseline for demographics, immunization rates, and preventive visits. Among adolescents who were behind at the start, immunization rates at study end increased by 21% for mailed (P < .01 vs control), 17% for telephone (P < .05), and 13% for control groups. The proportion of adolescents with a preventive visit (within 12 months) was: mailed (65%; P <.01), telephone (63%; P <.05), and controls (59%). The number needed to treat for an additional fully vaccinated adolescent was 14 for mailed and 25 for telephone reminders; for an additional preventive visit, it was 17 and 29. The intervention cost $18.78 (mailed) or $16.68 (phone) per adolescent per year to deliver. The cost per additional adolescent fully vaccinated was $463.99 for mailed and $714.98 for telephone; the cost per additional adolescent receiving a preventive visit was $324.75 and $487.03.

Conclusions

Managed care-based mail or telephone reminder/recall improved adolescent immunizations and preventive visits, with modest costs and modest impact.

Introduction

Between 2005 and 2006, 3 new vaccines were added to the recommended immunization schedule for adolescents: pertussis vaccine (Tdap), meningococcal vaccine (MCV4), and human papillomavirus vaccine (HPV).^1,2 Unfortunately, adolescent immunization rates are suboptimal. In 2010, national coverage rates for adolescents 13 to 17 years was 69% for Tdap, 63% for MCV4 vaccine, and, among girls, 49% for ≥1 dose of HPV vaccine and 32% for ≥3 doses of HPV vaccine.³ Furthermore, some disparities exist, with minority adolescents having lower completion rates of 3 doses of HPV vaccine.⁴

Experts,^5–8 including the Task Force on Community Preventive Services,^9–11 recommend that primary care practices use mailed or telephone reminder/recall to encourage immunizations for patients of all ages; reminder messages target upcoming immunizations whereas recall messages target overdue immunizations. These recommendations are primarily based on studies of young children, published before the routine recommendations for adolescent immunization with Tdap, MCV4, and HPV vaccines; these studies noted improved childhood immunization rates due to practice-based reminder/recall.^6,7,10 Because adolescents present infrequently for primary health care visits,^5,12 reminder/recall for adolescent immunizations is attractive in concept.^13,14

Nevertheless, little evidence exists for the effectiveness of practice-based reminder/recall for adolescent immunizations,^15,16 although in one recent study in 4 practices researchers demonstrated increased immunization rates.¹⁷ Moreover, few practices actually use reminder/recall for patients of any age^18–20; barriers include the added workload, costs, and complexities of conducting such a program.^21,22 Furthermore, some recent studies targeting low-income populations found little or no benefit to practice-based reminder/recall for young children because of difficulties in reaching families that often change addresses or telephone numbers.^18,23–26 Although tiered interventions that combine reminder/recall with outreach have improved immunization rates among low-income populations,^16,27–30 they are somewhat costly. In sum, practice-based reminder/recall, although recommended, is underused and has not yet achieved its promise.

A model that has not been well studied is centralized reminder/recall, ie, having messages generated by health systems such as managed care organizations that are associated with multiple primary care practices. Most children and adolescents are now enrolled in managed care plans³¹ and health reform will result in increased enrollment in centralized organizations.³² Centralized systems can take advantage of economies of scale and new technology such as autodialer systems that can broadcast hundreds of telephone reminders in a short time. Managed care organizations are interested in immunization and preventive care visit rates, which are Healthcare Effectiveness Data and Information Set (HEDIS) quality measures.^33–37 However, managed care organizations may not have accurate family contact information and families may not respond to reminders emanating from managed care plans.

We conducted a randomized controlled trial, based in a large managed care organization, to evaluate the effectiveness of a centralized reminder/recall system on improving rates of immunization and preventive care visits among low-income adolescents. We also compared the effectiveness of mailed versus telephone reminders. We hypothesized that centralized reminder/recall would improve immunization rates, with telephone calls being most effective because they appear more personal.

Methods

Setting

The study was based at the Monroe Plan for Medical Care, a not-for-profit managed care organization in upstate New York serving 72,404 publicly insured children and adolescents covered by Medicaid or the NY State Child Health Insurance Program.

Study Design

We conducted a 1-year randomized, controlled comparative effectiveness trial, comparing mailed reminder versus telephone reminder versus standard of care control (December 11, 2009, through December 12, 2010). We randomized adolescents within each practice to one of the 3 groups, allocating siblings to the same group. The Research Subjects Review Board of the University of Rochester approved the study. Parent informed consent was not required, but parents and practices could opt out at any time.

Participants

Primary Care Practices

From the Monroe Plan’s dataset of practices in 15 upstate counties, we selected all primary care practices that served ≥30 adolescents covered by the Monroe Plan in December 2009 (range of 35 to 1308). One of 38 eligible practices dropped out of the study, leaving 37 practices (22 pediatric, 13 family medicine, 2 internal medicine practices), which served 9369 adolescents. Nineteen practices had 1 doctor, 3 had 2 doctors, 2 had 3 doctors, and 13 had ≥4 physicians. We surveyed the practices about their baseline use of reminders. Of the 24 practices that responded, 12 (50%) used telephone or mailed reminders for adolescents who had scheduled preventive care visits, and 6 (25%) used telephone or mailed reminders for patients behind in at least some vaccines. Because adolescents were randomized within each practice, any impact of these practice-level reminders would be identical across study groups.

Subjects

The target population was all adolescents ages 10.5 through 17 years enrolled in Monroe Plan on December 31, 2009, with a primary care provider in a participating practice. We used 10.5 years as the lower age to provide time to generate reminders and for parents to make appointments for immunizations or preventive visits after age 11. Prior to randomization, we excluded (a) adolescents enrolled in the Monroe Plan for <6 months (due to insufficient data on prior healthcare or immunizations) and (b) adolescents with a contraindication (eg, anaphylaxis caused by vaccination) listed on claims files (n = 1). Adolescents who disenrolled from the managed care plan stopped receiving reminders if assigned to a study group but remained in the analytic sample for assessment of outcomes (90% were continuously enrolled throughout the intervention period).

Because the Monroe Plan enrollment files did not include family-level data, we identified adolescents who appeared to reside in the same household using address standardization and geo-coding software (Pitney Bowes Business Insight MapMarker USA, Version 22).

Randomization

We randomly selected a referent adolescent and randomly assigned them (and age-eligible siblings) to 1 of the 2 intervention groups or the control group (by AB, using Stata 9.2) stratifying on practice, age in years, and gender (Fig. 1). Health care providers were unaware of group assignment. Thus, by design the 3 study groups were balanced in numbers of adolescents and age/gender of the index adolescent across all practices.

Figure 1.

Flow diagram for study. As a secondary analysis, we included in an analysis all adolescents who were randomized, that is, including those with no phone or a bad address: controls (n = 2406), mailed reminder (n = 2494), and telephone reminder (n = 2504).

After randomization, we realized that the managed care database often lacked a household telephone number (41%) (neither a land line nor a cell phone was noted) or a geocodable address (3.6%). Because we wished to target the intervention to families who could possibly respond to either a telephone or mailed reminder, we excluded from the main analysis any adolescent for whom the managed care plan’s database lacked either a household telephone number or geocodable household address. This also allowed us to assess the potential “reach”³⁸ of the intervention— ie, the proportion of enrollees who could in the real world be targeted for the intervention. As a secondary analysis, we reanalyzed the results for all randomized adolescents (even if no telephone or address).

Study Interventions

Identifying Adolescents Eligible for Immunizations or Preventive Visits

Adolescents were considered eligible for a preventive care visit if they had none recorded for ≥14 months. We defined adolescents as eligible for Tdap, MCV4, or HPV vaccine based on 2010 ACIP guidelines³: Tdap if no previous Tdap or Td vaccine within 2 years (most practices used this timeframe between Tdap and Td vaccines); MCV4 if no previous vaccine; and HPV vaccine for girls (HPV₁ if none previous, HPV₂ if >60 days from HPV₁, and HPV₃ if >24 weeks from HPV₁ and >12 weeks from HPV₂). We did not include varicella vaccine because of the complexity of determining previous disease or vaccine eligibility,³⁹ or influenza vaccine because of its special seasonality.

The managed care organization developed an automatic algorithm that reviewed vaccination and preventive care visit measures every 5 weeks and triggered a reminder (starting at 10.8 years) if eligible. Mail and telephone reminders informed parents that they could opt out of future reminder/recall messages or could call the managed care plan with concerns.

Mailed Reminders

The managed care organization sent reminder letters advising parents to call their adolescent’s primary care practice to schedule an appointment. The letters provided the practice’s telephone number. Letters were sent at 10-week intervals for Tdap, MCV4, and preventive care visits (maximum of 5 reminders over 12 months). For HPV, the plan sent the letter for the first vaccine at 10-week intervals and reminders for HPV₂ or HPV₃ at 5-week intervals (maximum 8 reminders per vaccine dose). Letters were written in English and Spanish (2-sided), and tested (Flesch-Kinkaid software) as <7^th-grade reading level. Letters stated they were sent on behalf of the child’s insurance company and primary care practice, and they specified the practice name, child age, and specific services recommended (specific vaccines or preventive visit). No patient names were included in order to match the content to the telephone reminders.

Telephone Reminders

Telephone reminders were sent at the same frequency as letters by an autodialer service in which a recorded human voice in English or Spanish was used, with a message that mirrored the information in the letter reminders. No patient names were included due to HIPAA requirements.

Controls

These adolescents received standard of care from each respective practice.

Data Sources

Patient Information

The managed care organization’s enrollment files identified adolescent names, addresses, telephone numbers, birthdates, gender, type of insurance (Medicaid managed care, NY State Child Health Insurance Program), and primary care practice. Race/ethnicity was unavailable. Of note, the managed care organization obtained the telephone numbers and addresses for Medicaid managed care enrollees from the New York State Medicaid enrollment files; in 41% of cases this download resulted in missing telephone numbers.

Vaccinations and Preventive Visits

The managed care organization’s claims files identified vaccinations received (Current Procedural Terminology codes) and preventive care visits (International Classification of Diseases 9 codes). To obtain accurate vaccination information, the plan merged data from their claims records with data from the NYS Immunization Information System (immunization registry) and based vaccination reminders upon this merged database.

Measures

Primary Measures

Main outcomes were receipt of (1) each vaccination (Tdap, MCV4, and HPV_1,2,3 for girls) on or after age 11 years, (2) all vaccinations combined, and (3) a preventive visit during the 12-month period. Preventive visits were defined as comprehensive visits focused on routine preventive care; immunization-only visits were not counted. We also measured the time to vaccination since January 1, 2009 (1 year before the intervention started). The primary independent variable was group assignment (mailed reminder, telephone reminder, control).

Secondary Outcome Measures

We assessed process measures for mailed reminders (returned letters) and telephone reminders (line busy, answered by person, answering machine, no answer) and recorded the number of parents who opted out. We also assessed missed opportunities (primary care visits during which a vaccine was due but not administered), for any type of primary care visit since the start of the intervention. We calculated the number needed to treat (ie, to remind) to be fully vaccinated for all 3 vaccines and also to receive a preventive visit during the year. Finally, we measured costs of the intervention by summing the total personnel and non-personnel costs and calculating costs per year, adjusting costs to 2011 US dollar values. Wages for project personnel were based on the national mean (hourly wage) value for the appropriate job categories using data from the 2011 US Bureau of Labor Statistics Occupational Employment and Wage Estimates.⁴⁰

Statistical Analyses

We determined the hazard ratio of receipt of vaccinations or a preventive care visit at the end of the study for mailed versus control, telephone versus control, and mailed versus telephone groups. We used a clustered stratified Cox proportional hazard regression model with Efron’s method to handle tied events and the Huber/White variance estimator clustering on households and stratifying on sample stratification variables (age, gender, practice). This model accounts for the 22% of subjects who were not enrolled for the entire year before and the entire year during the intervention. For analyses where the outcome was time to vaccination, we measured time from January 1, 2009, and included study group (mailed, telephone, control), an indicator for intervention time period, and an interaction between the 2 independent variables in the models. For the preventive care visit analysis, we ran separate models for the baseline and intervention time periods, and included an independent variable indicating if the child had a preventive visit within 12 months. The beginning time period for both models was January 1st of each year. (StataMP, Version 12.0).⁴¹

We performed a prespecified subgroup analysis on age, gender, insurance, and practice subgroups for the composite immunization and preventive care outcomes. We performed a second analysis that included all adolescents regardless of lack of telephone numbers or addresses to assess the intervention impact at the managed care organization level. To control for multiple testing, we adjusted confidence intervals for the 16 tests for the composite immunization outcome and 18 tests for preventive visits using Šidák’s method.⁴² The study had >90% power for a 5% improvement in immunization rates at study end assuming 50% for controls (two-sided alpha = 0.05), using survival analysis and an intention-to-treat analysis.

We used descriptive statistics to summarize process measures and costs per adolescent per year. We calculated the cost-effectiveness for becoming vaccinated or receiving a preventive visit as the total cost of the intervention divided by: ([no. subjects] × [difference in the%of the outcome between study and control groups]).⁴³

Results

Randomization

In total, 7404 adolescents from 5559 families were randomized into 3 groups (Fig. 1); 3289 (44%) subjects lacked a telephone number or geocodable address, leaving 4115 adolescents (1296 control, 1396 mailed, and 1423 telephone reminder). Of these households, 73% had one adolescent, 22% had 2 and 5% had 3 or more.

Baseline Characteristics

The control and intervention groups had similar demographics (Table 1) and baseline immunization and preventive visit rates. The mean age at the start of the study was 14.4 years (SD 2.0). Baseline immunization rates closely mirrored national rates.⁴ Of note, baseline immunization rates were not significantly different for the adolescents with or without telephone numbers or geocodable addresses.

Table 1.

Baseline Characteristics, Immunization Rates and Preventive Care Visit Rates by Randomization Group

Characteristics	Control (n = 1296)	Mailed Reminders (n = 1396)	Telephone Reminders (n = 1423)
Age, mean (SD), years, no. (%)	14.4 (2.0)	14.5 (2.0)	14.4 (2.0)
11	178 (14)	210 (15)	222 (16)
12	195 (15)	181 (13)	224 (16)
13	190 (15)	194 (14)	196 (14)
14	196 (15)	199 (14)	183 (13)
15	186 (14)	230 (16)	206 (14)
16	192 (15)	218 (16)	226 (16)
17	159 (12)	164 (12)	166 (12)
Male, no. (%)	640 (49)	714 (51)	724 (51)
Insurance, no. (%)
Medicaid managed care	602 (46)	686 (49)	628 (44)
SCHIP	694 (54)	710 (51)	795 (56)
Practice specialty, no. (%)
Pediatric	973 (75)	1053 (75)	1045 (73)
Family medicine	296 (23)	318 (23)	338 (24)
Internal medicine	27 (2)	25 (2)	40 (3)
Residence, no. (%)
Urban/suburban	970 (81)	1017 (80)	1026 (79)
Rural	228 (19)	250 (20)	271 (21)
Baseline immunization, no. (%)
MCV4	838 (65)	917 (66)	904 (64)
Tdap	998 (78)	1073 (77)	1083 (77)
HPV 1	416 (63)	463 (68)	480 (69)
HPV 2	335 (51)	370 (54)	373 (53)
HPV 3	229 (35)	260 (38)	255 (36)
All vaccinations (girls-includes HPV)	551 (43)	625 (45)	605 (43)
Preventive care visit rates in previous 12 mo, no. (%)
All ages	815 (63%)	922 (66%)	917 (64%)
11	136 (76)	155 (74)	178 (80)
12	133 (68)	125 (69)	146 (65)
13	107 (56)	125 (64)	123 (63)
14	121 (62)	134 (67)	107 (58)
15	121 (65)	154 (67)	122 (59)
16	114 (59)	136 (62)	141 (62)
17	83 (52)	93 (57)	100 (60)

SCHIP = NY State Child Health Insurance Program; MCV4 = meningococcal vaccine; Tdap = pertussis vaccine; HPV = human papillomavirus vaccine.

Immunization Rates

Table 2 shows immunization rates at the end of the study for adolescents who were not up-to-date for the given vaccine at the study outset. Immunization rates for individual vaccines and for all vaccines combined were 4–9 percentage points greater for the mailed and telephone reminder groups compared with the control group, with hazard ratios ranging from 1.1 to 1.6 (P < .05 for most comparisons). Among adolescents missing any vaccination at study outset, 21% of mailed reminder, 17% of telephone reminder, and 13% of control group adolescents received all vaccinations by the end of the study period. Results for the mailed versus telephone reminders were not significantly different. Irrespective of the type of intervention, relatively few adolescents who had been behind at the start were vaccinated by the end of the study. The number needed to treat (ie, to remind) for an additional adolescent to be fully vaccinated was 14 for mailed reminders and 25 for telephone reminders.

Table 2.

Immunization Rates at the End of the Study Period Among Subjects Not Vaccinated at Study Outset

Vaccine	N*	Control Number (%†) (n = 1296)	Mailed Reminders (n = 1396)		Telephone Reminders (n = 1423)		HR Mail vs Telephone (95% CI)
			Number (%†)	HR (95% CI)‡ vs Control	Number (%†)	HR (95% CI) vs Control
MCV4	1456	102 (22)	150 (31)	1.5 (1.2–2.0)§	135 (26)	1.2 (0.9–1.6)	1.2 (1.0–1.6)
Tdap	952	51 (17)	73 (23)	1.5 (1.0–2.2)‖	73 (22)	1.2 (0.8–1.8)	1.2 (0.9–1.7)
HPV-1¶	678	51 (21)	59 (27)	1.3 (0.9–1.9)	59 (27)	1.1 (0.8–1.7)	1.1 (0.8–1.6)
HPV-2¶	959	59 (18)	81 (26)	1.5 (1.0–2.1)‖	84 (26)	1.6 (1.1–2.2)§	0.9 (0.7–1.3)
HPV-3¶	1293	58 (14)	78 (18)	1.4 (1.0–2.0)	84 (19)	1.5 (1.0–2.1)‖	1.0 (0.7–1.3)
All vaccinations#	2334	100 (13)	160 (21)	1.6 (1.3–2.1)§	143 (17)	1.3 (1.0–1.7)‖	1.2 (1.0–1.6)

^*Only children who were not up-to-date for the given vaccine were included in the analysis for that vaccine.

^†The denominator of a vaccination rate is the number of unvaccinated adolescents and hence varies across vaccines. For instance, in the bottom row (All vaccinations), the denominator of Control, Mail Reminders, and Telephone Reminders are 745 (=1296–551), 771 (=1396–625) and 818 (=1423–605), respectively (based on “the Baseline Immunization Rates (n)” presented in Table 1).

^‡HR indicates hazard ratio, from a clustered stratified Cox model, stratified on practice, age, and gender and clustered on family; CI, confidence interval; MCV4, meningococcal vaccine; Tdap, pertussis vaccine; HPV, human papillomavirus vaccine.

^§P < .01.

^‖P < .05.

^¶For HPV–girls only.

^#Includes HPV for girls.

Table 3 shows immunization rates at the end of the study, for all ages and all vaccines combined, for all adolescents, regardless of whether they were up-to-date at the beginning of the study. Among adolescents with any telephone number or geocodable address, overall immunization rates at the end of the study were 56% for mailed reminder group, 53% for the telephone reminder group, and 50% for controls (P <.05 for mailed or telephone versus control).

Table 3.

Immunization Rates at the End of the Study Period^*

	Up-to-Date Immunization Rates for All Immunizations at End of Study (MCV4†, Tdap, and, for Girls, HPV-3)
	Control	Mailed Reminders		Telephone Reminders
All Ages (11–17 Years)	N (%)	Number (%)	Hazard Ratio (95% CI) vs Control	Number (%)	Hazard Ratio (95% CI) vs Control
(a) Adolescents with any telephone number or geocodable address, ‡ n = 4114	651 (50)	785 (56)	1.6 (1.3–2.1)§	748 (53)	1.3 (1.0–1.7)‖
(b) All Adolescents (regardless of any telephone or address), n = 7404	1260 (52)	1420 (57)	1.5 (1.3–1.8)§	1345 (54)	1.1 (0.9–1.4)

^*Number (N) and percent (%) of adolescents who were up-to-date for all immunizations at the end of the study period (regardless of being up-to-date at the beginning), for all ages combined, for (a) adolescents with any telephone or geocodable address listed in the managed care plan’s database, and (b) all adolescents (including the 45% without any telephone numbers listed or any geocodable addresses). Hazard ratio, from a clustered stratified Cox model, stratified by practice, age, and gender and clustered on family with study arm, time period and an interaction between the 2 as covariates.

^†MCV4 indicates meningococcal vaccine; Tdap, pertussis vaccine; HPV, human papillomavirus vaccine.

^‡Presence of any telephone number of geocodable address in the managed care plan’s database, whether or not telephone number was accurate or working or the address was correct.

^§P < .01.

^‖P < .05.

Preventive Care Visits by Study Group

For all ages combined, 65% of mailed, 63% of telephone, and 59% of control group adolescents had a preventive care visit (P <.05 for mailed or telephone vs control), with a hazard ratio of 1.2 for the mailed group and 1.1 for the telephone group (Table 4). The mailed and telephone groups did not differ significantly. Of note, preventive visits declined in the control group from 63% (Table 1) to 59% (Table 3) as adolescents aged; this age-related secular trend was not noted in the mailed or telephone groups. The number needed to treat for an additional preventive care visit was 17 for mailed reminders and 29 for telephone reminders.

Table 4.

Preventive Care Visit Rates at the End of the Study Period^*

	Had a Preventive Care Visit During the 12 Months
	Control	Mailed Reminders		Telephone Reminders
All Ages (11–17 Years)	N (%)	Number (%)	Hazard Ratio vs Control	Number (%)	Hazard Ratio vs Control
(a) Adolescents with any telephone number or geocodable address,† n = 4114	768 (59)	908 (65)	1.2 (1.1–1.3)‡	892 (63)	1.1 (1.0–1.3)§
(b) All adolescents (regardless of any telephone or address), n = 7404	1384 (58)	1601 (64)	1.2 (1.1–1.3)‡	1,514 (60)	1.1 (1.0–1.2)§

^*Number (N) and percent (%) of adolescents who had a preventive care visit during the previous 12 months, for all ages combined, for (a) adolescents with any telephone or geocodable address listed in the managed care plan’s database, and (b) all adolescents (including the 45% without any telephone numbers listed or any geocodable addresses). Hazard ratio, from a clustered stratified Cox model, stratified by practice, age and gender and clustered on family with preventive care visit status at beginning of study as a covariate.

^†Presence of any telephone number of geocodable address in the managed care plan’s database, whether the telephone number was accurate or working or the address was correct.

^‡P < .01.

^§P < .05.

Subgroup Analyses

We performed subgroup analyses to assess whether the reminder/recall interventions had greater effects for certain ages, gender, insurance types, practice types, or suburban, urban, or rural residence. We adjusted subgroup confidence intervals for the multiple comparisons within each outcome. We did not find substantial differences for the effect of either mailed or telephone reminder on subgroups (data not shown), ie, the magnitude of the effects was similar across subgroups to the effects for the entire groups.

Analysis for Entire Plan Population

We reanalyzed findings for the entire sample of 7404 adolescents (ie, including the 40.9% or adolescents with missing telephone numbers and the 3.6% of adolescents with no geocodable addresses) (Tables 3, 4). The impact of the mailed reminders was nearly identical but the impact of telephone reminders was blunted.

Process Measures

Only 56 (6%) of the 1431 mailed reminder letters were returned. For 388 (27%) households in the telephone reminder group, the reminder call was unanswered or was picked up by voice mail.

Missed Opportunities

Table 5 compares missed opportunities across study groups. Both mailed and telephone groups had slightly fewer missed opportunities than controls for any vaccinations.

Table 5.

Missed Opportunities^* by Vaccine for Well Child Visits and all Visits

	Controls	Mailed Reminders			Telephone Reminders
	No. MOs/No. Visits	No. MOs/No. Visits	aRR (95%CI)†	P Value†	No. MOs/No. Visits	aRR (95% CI)†	P Value†
MCV4	894/999 (89%)	902/1051 (86%)	0.96 (0.92–0.99)	.03	1095/1232 (89%)	0.98 (0.94–1.02)	.27
Tdap	540/596 (91%)	634/711 (89%)	0.97 (0.92–1.01)	.17	694/771 (90%)	0.98 (0.94–1.03)	.42
HPV‡	845/1006 (84%)	824/1031 (80%)	0.95 (0.91–1.00)	.05	945/1164 (81%)	0.96 (0.91–1.00)	.07
Any missed opportunities§	1445/1707 (85%)	1457/1809 (81%)	0.95 (0.91–0.98)	.001	1702/2063 (83%)	0.96 (0.93–1.00)	.05

^*MO = missed opportunity; aRR = adjusted risk ratio, adjusting for age, gender, and practice; HPV = human papillomavirus vaccine.

^†aRR and P value are for comparing each group versus the control group.

^‡HPV vaccine calculated only for females and includes all doses.

^§Adolescents are included in the numerator if they did not receive all due vaccinations.

Costs

The total cost of the intervention excluding research costs (excluding adolescents without telephone numbers or geocodable addresses) was $26,220 for the mailed component (52% personnel costs) and $23,738 for the telephone component (50% personnel costs). Thus, among all adolescents who received a reminder (n = 1396 mailed group and n = 1423 telephone), the cost averaged $18.78 or $16.68 per adolescent per year for mailed and telephone reminders, respectively. Among the adolescents who actually received a targeted preventive care reminder/recall message, the cost per “additional” adolescent fully vaccinated was $463.99 for mailed and $714.98 for telephone groups, and the cost per “additional” adolescent receiving a preventive care visit was $324.75 for mailed and $487.03 for telephone groups.

Discussion

In this clinical trial of a centralized reminder/recall system based in a large managed care organization for the publicly insured, mailed letter reminders and telephone reminders to parents of adolescents improved rates of immunizations and preventive care visits a modest amount. The interventions had similar impact regardless of patient or practice characteristics. The annual cost for mailed and telephone reminders was also modest.

This study is novel in 3 ways: we evaluated a centralized automated immunization reminder/recall system based in managed care, we focused on adolescent vaccines, and we also included reminders for adolescent preventive care visits. Although recent studies of practice-based reminder/recall had limited effectiveness for low-income children,^23–26 we found a modest impact on both adolescent immunization rates and on preventive care visits. An important question involves the level of impact needed for an intervention that promotes immunizations and preventive care visits to be considered “clinically significant” or worthwhile when applied across a large population. Because of the importance of vaccination and preventive services to the health of the individual and community, we believe an improvement of several percentage points is significant. Of note, the return on investment is difficult to calculate because it involves multiple benefits from added immunizations plus preventive visits, with benefits potentially accruing across many years.

More intensive tiered outreach-based interventions^16,27–30 have demonstrated greater impact on immunization rates than we noted in the current study, but these interventions are also more costly. Thus, although centralized reminder/recall can improve immunizations and preventive visits to some extent, more intensive interventions may be needed to raise rates even further, especially in hard-to-reach populations. This is particularly important for HPV vaccine because 3 vaccinations are required, spaced over time. Also, centralized reminder/recall systems based in managed care plans can impact only patients enrolled in those plans. Centralized systems based in immunization registries might have even greater reach⁴⁴ but need further study.⁴⁵ Finally, because missed opportunities for immunizations occurred frequently, provider-based strategies such as provider prompts may be needed to eliminate missed opportunities during primary care visits.⁵

Although patient reminder/recall is widely recommended, a challenge to both practice-based and centrally-based reminder/recall is the lack of accurate telephone numbers or addresses which limited the potential “reach” of our intervention to about half the eligible population. One lesson is that practices and managed care organizations should make special efforts to update contact information during every contact with families— health care visits or re-enrollment into health insurance plans. Although there is substantial interest in newer methods for sending reminders, such as text messages or emails, there is only scant evidence of their effectiveness^46,47 and practical barriers to their use including continued dependence on accurate contact information and the need to opt in to receive text messages.⁸

Our study unveils a potentially new type of intervention to improve the receipt of preventive care among vulnerable populations—that is, centrally located automated patient reminder/recall. Managed care organizations can take advantage of economies of scale with their ability to generate large numbers of letters using sophisticated mailing systems or telephone calls using autodialers. Managed care organizations typically use HEDIS to measure their quality performance⁴⁸; because HEDIS now includes adolescent immunizations and adolescent preventive care visits, managed care plans have an incentive to improve rates. Furthermore, in New York and some other states and potentially under the Affordable Care Act, Medicaid managed care plans can receive additional quality incentive payments based on achieving certain quality metrics.⁴⁹ A managed-care based centralized, automated reminder/recall system to improve rates of vaccination and preventive visits may have a place in multipronged efforts to optimize preventive care services for adolescents.

Although we hypothesized that telephone reminders would work better than mailed reminders, mailed reminders performed slightly better than telephone reminders. It is possible that remembering verbal rather than printed instructions is challenging. Telephone messages may not necessarily reach the household decision-maker or may be erased if left on an answering machine. At the same time, low literacy among the target population might limit the potential impact of mailed reminders despite low-reading level messages and Spanish versions. The results of this study may temper expectations for the degree to which automated patient reminder/recall can improve preventive care services.

Finally, although the impact of our intervention was modest, so was the cost, which averaged $18.78 for mailed and $16.68 for telephone reminders per adolescent per year. Importantly, the cost per additional adolescent vaccinated was higher than the cost per additional individual vaccinated reported in some studies^50–52 but in the range of other studies^17,53 and less than the costs for stepped reminder/outreach interventions^27,28; also the benefits involved both improved immunizations and greater preventive care visits.

Study strengths include the use of a clinically relevant intervention, a large and diverse sample, a broad spectrum of primary care practices, important outcomes (immunization rates and preventive visits), and a randomized controlled trial design. Our study was unique by focusing on low-income adolescents served by managed care; many studies of patient reminders have targeted families of young children,^7,23,24,26 adult patients,^7,30 privately insured populations,^52,54 or have emanated from primary care practices rather than centralized organizations such as managed care plans. Our analysis of both the group missing telephone numbers and the entire adolescent age group highlights the real-world dilemma of reaching a low-income population with reminders. Our cost-effectiveness analysis of reminders complements the literature on cost-effectiveness of adolescent vaccinations themselves.^55–57

One limitation to generalizability is that baseline immunization rates were lower than in some areas; benefits of the intervention could wane as rates rise. However, baseline preventive visit rates mirrored national rates.^5,58 Second, this study had substantial research support, and we consider it an efficacy trial.⁵⁹ Further implementation and dissemination studies are needed to assess continued effectiveness. Third, the managed care organization lacked telephone numbers for 41% of the population due to an apparent problem with download of data from NYS Medicaid enrollment files. Analysis of baseline data suggested that missing telephones were not the result of some biased enrollment or inclusion in our study. Finally, the intervention took place in a managed care organization that serves publicly insured, low-income families; findings may differ for higher-income populations or in settings with more accurate addresses and telephone numbers.

Conclusion

A centralized patient reminder/recall system using either mailed or telephone reminders, based in a managed care organization serving publicly insured patients, improved rates of immunization and preventive care visits. The impact of the intervention was modest, but costs were relatively low. Clinicians, public health leaders, managed care, or integrated health systems should consider centralized patient reminder/recall systems for low-income adolescents. Further research should assess the effectiveness and cost-effectiveness of these interventions in other settings.

What’s New.

We assessed the impact of a managed care-based patient reminder/recall system on improving adolescent immunizations and preventive care visits. Mailed or telephone reminders improved immunization and preventive care visit rates by 4–9 percentage points, for a relatively modest cost.