Effect of State Immunization Information System Centralized Reminder and Recall on HPV Vaccination Rates

In a 4-arm pragmatic RCT, we implemented IIS-based C-R/R for HPV vaccine in New York and Colorado.

Abstract

BACKGROUND:

Although autodialer centralized reminder and recall (C-R/R) from state immunization information systems (IISs) has been shown to raise childhood vaccination rates, its impact on human papillomavirus (HPV) vaccination rates is unclear.

METHODS:

In a 4-arm pragmatic randomized controlled trial across 2 states, we randomly selected practices representative of the specialty (pediatrics, family medicine, and health center) where children received care. Within each practice, patients 11 to 17.9 years old who had not completed their HPV vaccine series (NY: N = 30 616 in 123 practices; CO: N = 31 502 in 80 practices) were randomly assigned to receive 0, 1, 2, or 3 IIS C-R/R autodialer messages per vaccine dose. We assessed HPV vaccine receipt via the IIS, calculated intervention costs, and compared HPV vaccine series initiation and completion rates across study arms.

RESULTS:

In New York, HPV vaccine initiation rates ranged from 37.0% to 37.4%, and completion rates were between 29.1% and 30.1%, with no significant differences across study arms. In Colorado, HPV vaccine initiation rates ranged from 31.2% to 33.5% and were slightly higher for 1 reminder compared with none, but vaccine completion rates, ranging from 27.0% to 27.8%, were similar. On adjusted analyses in Colorado, vaccine initiation rates were slightly higher for 1 and 3 C-R/R messages (adjusted risk ratios 1.07 and 1.04, respectively); completion rates were slightly higher for 1 and 3 C-R/R messages (adjusted risk ratios 1.02 and 1.03, respectively).

CONCLUSIONS:

IIS-based C-R/R for HPV vaccination did not improve HPV vaccination rates in New York and increased vaccination rates slightly in Colorado.

What’s Known on This Subject:

Although centralized reminder and recall by phone (autodialer) has been shown to improve childhood vaccination rates, little is known about its impact on human papillomavirus (HPV) vaccination rates, particularly when coming from state immunization information systems.

What This Study Adds:

In a pragmatic randomized clinical trial, immunization information systems–based centralized reminder and recall for the HPV vaccine did not raise HPV vaccination rates in New York and raised rates minimally in Colorado. More intensive but perhaps less scalable interventions are warranted.

Infection by human papillomavirus (HPV) causes most cases of cervical cancers; 90% of anal cancers; 40% of vulvar, vaginal, and penile cancers; and >60% of oropharyngeal cancers.^1–4 HPV infection leads to 33 700 new cancer cases in the United States annually,⁵ ∼4000 deaths, and >$4 billion in cancer-related health care costs.^6,7 Although HPV vaccines are highly effective,⁸ and a Healthy People 2020 goal is >80% vaccination coverage among 13- to 17-year-olds, in 2017 (the year this study's fieldwork began) only 53% of US girls and 44% of boys 13 to 17 years of age had completed their HPV vaccine series.^9–11

An evidence-based method to increase vaccination rates is distribution of reminder and recall (R/R) messages to inform patients or parents of upcoming vaccines (reminders) or past-due vaccines (recall).^12,13 A recent Cochrane review revealed that R/R for vaccines, including adolescent vaccines, can increase vaccination rates.¹⁴ The Task Force on Community Preventive Services¹⁵ and immunization experts^12,16 recommend R/R messages for vaccinations. However, less than one-fifth of practices send reminders routinely, and rarely for the HPV vaccine.^17,18 Barriers include limited resources, technological challenges, and competing priorities.^19–21

One potential solution involves centralized reminder and recall (C-R/R), in which health systems, state immunization information systems (IISs), or managed care organizations use centralized electronic databases as the data source for sending R/R messages, thereby capitalizing on economies of scale. Most health providers and parents are in favor of C-R/R.^18,22,23 Previous trials revealed that C-R/R by using IISs for routine childhood vaccines (ie, not adolescent vaccines) was more effective and cost-effective than practice-based R/R.^24–27

HPV vaccination is more challenging than other adolescent vaccines⁸ because of the need for multiple doses²⁸ and frequent parent questions²⁹ or vaccine hesitancy.^30–32 Thus, IIS-based C-R/R may not be effective for the HPV vaccine.

Studies from 2 health maintenance organization settings,^33–35 a state Medicaid system,³⁶ and a combination of practice records plus IIS records³⁷ revealed that C-R/R improved HPV vaccination rates by small to moderate amounts. In none of these studies were data or messages from IISs used alone. IIS-based C-R/R might be scalable because all states have an IIS, and phone numbers, addresses, and vaccination dates are often uploaded from electronic health records.¹⁹ A recent study from New York,³⁸ using C-R/R based on IIS data alone, revealed that mailed letter reminders to parents of adolescents who had not yet initiated the HPV vaccine series increased HPV vaccination rates by 2 percentage points, a significant effect at the population level. However, mailed letter reminders are costly.³⁴ Little is known about the impact of IIS-based autodialer C-R/R on HPV vaccination rates.

In this study, we assessed the effectiveness and cost of implementing IIS C-R/R by using autodialed phone messages. We performed a pragmatic trial, comparing different numbers of IIS-based C-R/R messages to usual care across New York and Colorado. We hypothesized that adolescents whose families were sent IIS-based C-R/R messages would have higher HPV vaccination rates than adolescents receiving usual care and that more reminders would have a dose-response effect.

Methods

We conducted a 4-arm, pragmatic³⁹ (effectiveness) randomized controlled trial (RCT), testing the impact of 1, 2, or 3 autodialer phone reminders per dose of the HPV vaccine needed (versus no C-R/R) sent to parents on behalf of the IIS and primary care practices about HPV vaccination. This study was approved by institutional review boards at University of California, Los Angeles and University of Colorado (with waiver of consent) and by health departments in New York and Colorado. The intervention was conducted between February 2017 and January 2019.

New York State Immunization Information System and Colorado Immunization Information System

The New York State Immunization Information System (NYSIIS) captures vaccination data within the 57 counties outside of New York City. New York requires any immunization provider who vaccinates children <19 years of age to report vaccine and demographic information to the NYSIIS via electronic or manual upload. On the basis of 2017 census data, 97% of 11- to 17-year-olds in New York had ≥2 vaccine records in the NYSIIS.⁴⁰ The NYSIIS conducted the afore-mentioned mailed C-R/R for HPV vaccination.³⁸

In contrast, Colorado does not require providers to report childhood and/or adolescent vaccination data to the Colorado Immunization Information System (CIIS), yet 86% of all known immunizing providers submitted data to the CIIS as of April 2019. Providers can electronically or manually enter data into the CIIS and can set their practice as a patient’s default or primary practice. The Colorado Department of Public Health and Environment has executed large-scale C-R/R studies for childhood but not for HPV vaccines.^24,26,27

Study Populations

We randomly selected practices and adolescents across counties in New York (excluding New York City) and Colorado (Fig 1). Practices were eligible if they had ≥80 adolescents 11 to 19 years of age in New York or ≥50 in Colorado; this was to ensure adequate numbers per study arm. Practices were stratified on the basis of rural, urban, or suburban location in New York and rural or urban location in Colorado. We then used within-practice randomization to allocate adolescents 11 to 17 years (on February 1, 2017) to 1 of 4 study arms; adolescents were excluded if they had completed their HPV series. Patient-level exclusion criteria (Fig 1) were created to identify patients who likely no longer resided in Colorado or New York. We also excluded deceased individuals in both states and adolescents whose parents had opted out of the CIIS (<2%); New York does not allow individuals to opt out of the NYSIIS. We grouped IIS participants into families or households, defined as having the same primary phone number. We randomly selected 1 index adolescent.

FIGURE 1.

Consolidated Standards of Reporting Trials diagram plus practice and patient eligibility criteria. HMO, health maintenance organization.

Selection of Study Sample

Small differences in subject inclusion were based on preferences within each state (Fig 1). In New York, we stratified practices by practice type (pediatrics, family medicine, and community health center) and selected the number of practices from each type to reflect its proportion in the study population. Practices could opt out of the study (23 of 123 opted out [19%]); practice names were included in all autodialer calls. In Colorado, we selected practices proportionally by practice type (pediatrics, family medicine, and community health center) on the basis of where adolescents were seen. Practices could not opt out of the study but had to opt in for practice names or phone numbers to be included in autodialer calls (49 of 80, or 61%, chose this option). Eligible adolescents were selected at random from practices in numbers proportionate to practice size. All analyses were stratified by state.

C-R/R Autodialer Calls

We contracted with a cloud-based telephony company (http://www.teletask.com) to send autodialer calls to the family’s primary phone number in the IIS. Autodialer calls were sent in English and Spanish (respondents needed to “presione dos” for Spanish). The calls contained the practice’s name and phone number, or the name and phone number of the county health department in Colorado when practices did not wish for practice names and phone numbers to be included. Messages were essentially identical in the 2 states. We consulted with parents and providers for feedback on the message content during the message development phase. Messages used the Health Belief Model framework with HPV vaccination framed as cancer prevention for all adolescents.⁴¹ A typical message is shown in Fig 2. We randomly assigned index adolescent subjects to either usual care (no calls) or up to 1, 2, or 3 autodial calls per needed dose of the HPV vaccine, depending on the study arm. Autodial calls reflected monthly IIS data pulls identifying eligible adolescents. Respondents could opt out of the study by calling a toll-free number included in the message or by pressing “9” during the autodial call.

FIGURE 2.

Example of an autodialer reminder script from New York (the script was read by computer-assisted human recording if an individual answered the autodialer call, and the voice sounded like a regular phone call).

We used the updated (October 2016) Advisory Committee on Immunization Practices dosing schedule for the HPV series; adolescents 11 to <15 years required 2 doses and were sent C-R/R messages at time 0 and 12 months after the first dose. Adolescents initiating the series after the 15th birthday, who required 3 doses, were sent calls as follows: dose 1, time 0; dose 2, 2 months after dose 1 was received; and dose 3, 6 months after dose 1 was received and 4 months after dose 2 was received. For adolescents randomly assigned to 2 or 3 calls per dose, we spaced calls by 1 month. Adolescents who had initiated the series at baseline were still eligible for calls for follow-up doses.

Outcome Measures

The primary study outcome was initiation and/or completion of the HPV 2- or 3-dose vaccine series among adolescents aged 11 to <19 years on the basis of IIS data. We compared outcomes for all ages and then by sex and age subgroups. We also calculated the cost of delivering C-R/R and planned cost-effectiveness analyses if C-R/R raised vaccination rates.

Data Analysis

The study was powered to detect an absolute difference of 2 percentage points in vaccine completion when comparing the control arm and any intervention arm. We considered this difference to be the minimum outcome of public health importance.

Primary outcomes were HPV vaccine series initiation and series completion. For New York we used mixed-effects multivariable Poisson regression models with robust SEs and for Colorado we used mixed-effects multivariable log-binomial models to obtain unadjusted and adjusted risk ratios (RRs) along with their 95% confidence intervals (CIs) and P values. The adjusted model contains the prespecified covariates of the child’s age group and sex, the practice type, and the practice’s urban or rural status. We included a random intercept for practice to account for clustering of patients within practices and tested interactions between each predictor and study arm (none were significant, so data are not shown). All major analyses were intention-to-treat analyses. As an exploratory analysis, we also used the Kaplan-Meier estimator to estimate the initiation and completion rate at different ages. In analyses, we used SAS version 9.4 (SAS Institute, Inc, Cary, NC) and R version 3.5.2 (R Foundation for Statistical Computing, Vienna, Austria).

We compared the cost of implementing C-R/R for each intervention arm and for each state (details upon request). Cost domains included the following: (1) consensus building and preliminary work, (2) training, (3) collaboration, and (4) implementation (which included recall and autodialer costs). Costs were stratified by up-front costs (one-time costs to implement the intervention) and intervention costs (associated with activities occurring at each C-R/R round). We calculated total costs for each intervention arm by aggregating costs for each cost domain and reported average costs per randomly assigned child for initiation- and completion-related efforts. We divided the average cost per randomly assigned child by the proportion who achieved initiation (completion) to estimate the within-arm average cost per initiation (or per completion). We then estimated (after adjustment) incremental cost per additional vaccine initiated and completed for all statistically significant study groups. All costs reflected the perspective of the IIS.

Results

Characteristics of Practices and Patients

In New York, most practices were pediatric and urban (Table 1). In Colorado, one-quarter of practices were pediatric, and most were urban. For both states, <2% of patients opted out of phone calls, and <15% had missing phone numbers in IIS databases.

TABLE 1.

Characteristics of Primary Care Practices and Adolescents

Characteristics of Practices and Patients	New York	Colorado
Practices
No. recruited for study	100	80
Practice type, n (%)
Pediatrics	77 (77)	19 (24)
Family medicine	16 (16)	41 (51)
Community health center	7 (7)	20 (25)
Practice location, n (%)
Rural	13 (13)	20 (25)
Urban	87 (87)	60 (75)
Inclusion of practice name on R/R, %	100	61
Patients
Total patients, N	30 616	31 502
11 to <13 y, n (%)	13 412 (44)	14 122 (45)
13 to <15 y, n (%)	9832 (32)	7828 (25)
15 to <18 y, n (%)	7372 (24)	9552 (30)
Sex, n (%)
Female	14 402 (47)	14 824 (47)
Male	16 214 (53)	16 678 (53)
Total patients who opted out of R/R, n (%)a	272 (1)	487 (2)
Missing phone numbers, n (%)a	3859 (12)	2838 (9)

^aThese patients were still included in the intention-to-treat analysis but did not receive autodialer calls.

Impact of IIS-Based C-R/R Messages

HPV vaccine initiation rates for the entire cohort were low as expected because we excluded adolescents who had already completed their HPV vaccine series at the start of the study (Fig 3). In New York (Fig 3A), both postintervention vaccine initiation and completion rates were virtually identical across all study arms, suggesting no impact of the intervention compared with standard care, irrespective of the number of messages sent. In Colorado (Fig 3B), initiation and completion rates were similar across study arms except that the study group sent 1 call per dose had an HPV vaccine initiation rate 2.3 percentage points higher than that of controls (RR = 1.07; 95% CI 1.02–1.13); however, vaccine completion rates were not better in this study arm than in the control arm.

FIGURE 3.

Vaccine series initiation and completion rates in New York (A) and Colorado (B) at the end of the study.

To assess whether the C-R/R may have resulted in earlier receipt of the HPV vaccine, we used Kaplan-Meier estimates to show the initiation and completion rate at different ages. There were no statistically (P = .856 and P = .952 for completion, respectively, in NY; P = .09 and P = .75 in CO) or clinically significant differences between study arms and control arm in the age of initiation or completion of HPV vaccine except for the effect of 1-call on initiation rates in Colorado.

Table 2 shows results for a multivariable mixed model used to compare unadjusted and adjusted RRs of vaccine initiation and completion by study arm. In New York, there were no significant findings on either unadjusted or adjusted analyses for any of the study arms (or for all arms combined). In Colorado regarding vaccine initiation, on unadjusted analyses, the 1-call R/R arm had higher HPV vaccine initiation rates than the control arm, and on adjusted analyses, the 1-call R/R (adjusted RR = 1.07; 95% CI 1.04–1.10) and the 3-call R/R (adjusted RR = 1.04; 95% CI 1.01–1.06) arms had higher initiation rates than the control arm. In Colorado regarding vaccine completion, the adjusted RRs were 1.02 (95% CI 1.00–1.05) for the 1-call R/R arm and 1.03 (95% CI 1.00–1.05) for the 3-call R/R arm.

TABLE 2.

Multivariable Mixed Model Revealing Unadjusted and Adjusted RR for HPV Vaccine Series Initiation and Completion by Study Group (0, 1, 2, or 3 C-R/R Messages)

Study Group	New York			Colorado
	%	Unadjusted RR (95% CI)	Adjusted RR (95% CI)a	%	Unadjusted RR (95% CI)	Adjusted RR (95% CI)a
HPV vaccine initiation
Control	37.4	Reference	Reference	31.2	Reference	Reference
1 call	37.0	0.99 (0.93–1.05)	0.99 (0.93–1.05)	33.5	1.07 (1.02–1.13)*	1.07 (1.04–1.10)*
2 calls	37.0	0.99 (0.94–1.05)	0.99 (0.94–1.05)	31.5	1.01 (0.95–1.07)	1.01 (0.98–1.04)
3 calls	37.0	0.99 (0.94–1.05)	1.00 (0.94–1.05)	32.3	1.03 (0.99–1.08)	1.04 (1.01–1.06)*
HPV vaccine completion
Control	29.2	Reference	Reference	27.0	Reference	Reference
1 call	30.1	1.03 (0.97–1.09)	1.02 (0.97–1.08)	27.8	1.03 (0.96–1.10)	1.02 (1.00–1.05)
2 calls	29.8	1.02 (0.97–1.07)	1.02 (0.97–1.07)	27.5	1.02 (0.97–1.07)	1.02 (0.99–1.04)
3 calls	29.1	1.00 (0.94–1.06)	0.99 (0.94–1.05)	27.7	1.02 (0.98–1.07)	1.03 (1.00–1.05)

^aAdjusted for age, practice type, urban or rural status, and sex.

^*P < .05

Table 3 shows HPV vaccine series initiation and completion rates and adjusted RRs by patient and practice characteristics. Vaccine initiation and completion rates were higher for younger adolescents and for pediatric practices. Rates were generally not associated with sex or rurality.

TABLE 3.

HPV Vaccine Series Initiation and Completion Rates by Patient and Practice Characteristics

Patient and Practice Features	n	Vaccine Initiation		Vaccine Completion
		%	Adjusted RR (95% CI)a	%	Adjusted RR (95% CI)a
New York
Age, y
11.0–12.9	13 412	46.7	Reference	36.0	Reference
13.0–14.9	9832	33.7	0.72 (0.66–0.78)*	29.2	0.83 (0.77–0.89)*
15.0–17.9	7372	24.5	0.55 (0.49–0.61)*	19.7	0.57 (0.52–0.62)*
Sex
Female	14 402	37.2	Reference	30.6	Reference
Male	16 214	37.1	1.00 (0.95–1.06)	28.6	0.94 (0.89–1.00)
Practice type
Family medicine	2828	30.0	Reference	26.1	Reference
Pediatrics	25 353	38.8	1.31 (1.14–1.52)*	30.0	1.27 (0.80–2.02)
CHC or RHC	2435	26.9	0.92 (0.66–1.29)	28.6	1.19 (0.39–3.62)
Rurality
Downstate	14 921	36.6	Reference	27.5	Reference
Upstate rural	2614	37.1	0.96 (0.80–1.14)	28.7	1.14 (0.76–1.71)
Upstate urban	13 081	37.8	0.98 (0.87–1.10)	32.2	1.15 (0.90–1.46)
Colorado
Age, y
11.0–12.9	14 122	39.4	Reference	34.5	Reference
13.0–14.9	7828	31.8	0.81 (0.79–0.83)*	30.1	0.88 (0.86–0.90)*
15.0–17.9	9552	20.2	0.53 (0.52–0.55)*	15.0	0.46 (0.44–0.47)*
Sex
Female	14 824	33.3	Reference	28.6	Reference
Male	16 678	31.1	0.97 (0.95–0.99)*	26.5	0.95 (0.93–0.97)*
Practice type
Family medicine	7177	27.1	Reference	21.5	Reference
Pediatrics	18 107	33.9	1.17 (1.09–1.25)*	29.2	1.32 (1.22–1.42)*
CHC or RHC	6218	33.1	1.20 (1.11–1.30)*	29.2	1.32 (1.21–1.44)*
Rurality
Rural	3661	28.5	0.93 (0.86–1.00)	26.6	0.96 (0.88–1.04)
Urban	27 841	32.6	Reference	27.6	Reference

CHC, community health center; RHC, rural health center.

^aThe adjusted estimates are adjusted for all other covariates in the table plus study arm (0, 1, 2, or 3 C-R/R messages).

^*P < .05

Table 4 shows the average costs per vaccine initiated and the average costs per completion for the 3 active study groups. The costs per randomly assigned child were relatively constant (and low) across study groups, ranging from $0.51 to $0.63 in Colorado and from $0.66 to $0.83 in New York. The average cost per vaccine series initiated ranged from $1.53 to $2.24, whereas the average cost per series completed ranged from $1.85 to $2.85, with higher-contact study groups costing more than the 1-call arm. Compared with controls, the incremental cost to achieve 1 additional vaccine series initiation in Colorado was $24 for 1 call and $51 for 3 calls. Compared with controls, the incremental cost to achieve 1 additional series completion in Colorado was $95 for 1 call and $78 for 3 calls (we only include study arms with adjusted significant findings).

TABLE 4.

Up-front and Intervention Costs Per Randomly Assigned Child and Per Vaccine Initiation and Completion by Study Arm

Study Group	New York				Colorado
	n or %	Up-front Cost, $	Intervention Cost, $	Total Cost, $	n or %	Up-front Cost, $	Intervention Cost, $	Total Cost, $
Cost per randomly assigned child
1 call	7682	0.27	0.39	0.66	7864	0.18	0.33	0.51
2 calls	7631	0.27	0.48	0.75	7870	0.18	0.39	0.57
3 calls	7579	0.28	0.55	0.83	7890	0.18	0.45	0.63
Cost per vaccine initiation
1 call	37.0%	0.73	1.05	1.78	33.5%	0.55	0.98	1.53
2 calls	37.0%	0.73	1.30	2.03	31.5%	0.59	1.23	1.82
3 calls	37.0%	0.76	1.49	2.24	32.3%	0.57	1.38	1.95
Cost per vaccine completion
1 call	30.1%	0.91	1.30	2.20	27.8%	0.67	1.18	1.85
2 calls	29.8%	0.92	1.60	2.52	27.5%	0.67	1.41	2.09
3 calls	29.1%	0.95	1.90	2.85	27.7%	0.67	1.61	2.28

Discussion

This 4-arm pragmatic RCT is the first multistate RCT to test the impact of C-R/R autodialer calls using only IIS data on raising HPV vaccination rates. Unfortunately, we found no significant effect of IIS-based C-R/R autodialer calls on HPV vaccine series initiation or completion rates in New York and only small effects in Colorado for 1 or 3 C-R/R calls on vaccine series initiation and completion. In only 1 case (1 C-R/R call in Colorado for vaccine series initiation) did the intervention reach our predetermined minimally significant finding: ≥2-percentage-point rise in vaccination rates.

These largely negative and surprising findings were contrary to our hypotheses, considering the success of this type of C-R/R for childhood vaccinations²⁵ and the generally positive findings of R/R noted in the 2018 Cochrane report on R/R for vaccinations.

We speculate multiple possible explanations. A few practices might have conducted their own R/R for HPV vaccinations. Some IIS HPV vaccine data might have been inaccurate because of underreporting to the IIS; this would blunt but not eliminate the ability to demonstrate a true impact of IIS C-R/R. Similarly, some patient telephone numbers might have been inaccurate, although IIS data (including phone numbers) are updated with each vaccination, including influenza vaccines. Unfortunately, we had no method to verify accuracy of phone numbers, and few patients opted out of phone calls. Overall, 88% of New York practices and 50% of Colorado practices uploaded to IISs via their electronic health records, but we still cannot calculate the degree to which practices uploaded changes in phone numbers. This might reduce but should not eliminate the ability to detect a true impact of IIS-based C-R/R. Of note, because we excluded patients who had completed their HPV series, we expected overall rates to be much lower than state-reported rates in the National Immunization Survey-Teen.

A third possibility is that autodialer phone calls are simply not as effective today as phone R/R appears to have been in the past. We speculate 2 possible reasons: autodialers may be less impactful than traditional phone calls, and autodialers themselves may have diminished effectiveness recently. There has been a marked rise in the number of autodialer calls⁴² and spam phone calls⁴³ to phones from a variety of sources. Many people screen calls and delete any calls from unknown phone numbers. It is possible that amid the “sensory overload” of autodialer calls and general information overload in today’s society, autodialer R/R may have lost its effectiveness for vaccinations or at least for HPV vaccination.

A fourth possibility is that IIS-based C-R/R by using autodialers does not work for the HPV vaccine specifically. Several previous studies revealed small effects of C-R/R conducted by health systems,^33–35 and 1 New York IIS study found a 2-percentage-point improvement due to mailed C-R/R messages.³⁸ However, mailed reminders are far more expensive and less scalable than autodialer C-R/R; our goal was to develop a highly scalable intervention.

Because studies have revealed that a strong provider recommendation is important in uptake of the HPV vaccine,³⁰ we included the primary care practice’s names on all autodialer calls from New York and on 61% of calls from Colorado. However, because of legal limitations (needing the practice’s permission to use phone numbers) and technical realities (inserting patient-specific practice phone numbers into autodialer calls), we were unable to display the phone numbers of the adolescents’ doctors’ offices as the source of the C-R/R calls; the phone number appeared as a toll-free number. A phone number from the doctor’s office (which likely would have been recognized by many recipients) might have had a greater impact.

Another likely factor is vaccine hesitancy,^30–32 which is experienced by many clinicians. It is possible that the effect of HPV vaccine hesitancy outweighs any possible effect of autodialer reminders. However, we suspect this is not the only answer because we found no or little impact of C-R/R calls on subsequent HPV doses and for older adolescents (groups for which hesitancy is less likely). Finally, we do not have any explanation for why we found no impact in New York but a small impact in Colorado for both vaccine series initiation and completion.

Our study has several strengths, including its design as a large 2-state pragmatic trial with rigorous sampling methods to identify practices and patients and analyses that accounted for clustering within practices and several potential confounders. By randomly assigning patients within practices, we limited confounding at the practice level and improved the study’s power to detect intervention effects. Study limitations include generalizability from these 2-state–level studies, inability to determine the accuracy of phone numbers or the potential level of underreporting of HPV vaccine data in the IISs, and inability to examine in depth the reasons for no or little impact of IIS-based C-R/R.

Conclusions

In these 2 states, IIS-based C-R/R autodialer messages for HPV vaccination were not effective in raising HPV vaccination rates in New York and were only slightly effective for vaccine series initiation and completion in Colorado. Because C-R/R is a low-cost, scalable intervention, further study is needed to determine if IIS-based C-R/R should be part of our nation’s strategy to raise HPV vaccination rates.

Glossary

CI

confidence interval

CIIS

Colorado Immunization Information System

C-R/R

centralized reminder and recall

HPV

human papillomavirus

IIS

immunization information system

NYSIIS

New York State Immunization Information System

RCT

randomized controlled trial

RR

risk ratio

R/R

reminder and recall