Effectiveness and feasibility of three types of parent reminders to increase adolescent human papillomavirus (HPV) vaccination

Abstract

Parent reminders have produced modest improvements in human papillomavirus (HPV) vaccination among adolescents. However, little prior research has compared the effectiveness and feasibility of different HPV reminder types in resource-limited settings. We conducted a quasi-experimental study (2016–2017) to evaluate the effectiveness of three parent reminder types (mailed letters, robocalls, text messages) on next-dose HPV vaccine receipt among 12-year-olds in a large Federally Qualified Health Center in Los Angeles County. Six clinics were matched into three pairs: randomly assigning one clinic within each pair to intervention and control. Intervention clinics were randomly assigned to deliver one of the three parent reminder types. We calculated rates of next-dose vaccine receipt and assessed intervention effects using logistic regression models. We calculated the proportion of each type of reminder successfully delivered as a feasibility measure. The study sample comprised 877 12-year-olds due for an HPV vaccine dose (47% female, >85% Latino). At 4-month follow-up, 23% of intervention patients received an HPV vaccine dose compared to only 12% of control patients. Overall, receipt of any reminder increased rates of the next-needed HPV vaccine compared to usual care (p = 0.046). Significant improvements were observed for text reminders (p = 0.036) and boys (p = 0.006). Robocalls were the least feasible reminder type. Text message reminders are feasible and effective for promoting HPV vaccination. Future research is needed to assess the effectiveness and feasibility of reminders compared to other vaccine promotion strategies.

1. Introduction

Prophylactic human papillomavirus (HPV) vaccines have been widely heralded as a breakthrough in cancer prevention. The HPV vaccine is safe, effective, and protects against HPV types that cause almost all cervical cancers, 90% of anal cancers, and a large proportion of vulvar, vaginal, and penile cancers (Gillison et al., 2008; Senkomago et al., 2019). For these reasons, routine vaccination is recommended for adolescent girls and boys ages 11–12 years in the U.S., prior to HPV exposure when the vaccine has the potential for maximum benefit. Yet, in 2019, over a decade since the vaccine’s introduction, only 67% of U.S. adolescents initiated and 45% completed the vaccine series by age 13 (Elam-Evans et al., 2020).

Parent reminders have been used to improve childhood immunization rates, as they are low-cost, scalable, and widely accepted. Unlike other routine adolescent vaccines, however, the HPV vaccine requires multiple doses and has faced greater parental hesitancy, which poses unique challenges to practice implementation and parental acceptance of vaccination. Understanding the feasibility, effectiveness, and implementation of different HPV vaccine reminder types is particularly relevant in settings such as Federally Qualified Health Centers (FQHCs), where resources to promote preventive health services are limited.

Prior research evaluating parent reminder and recall messages, including letters and phone calls and more recently text and email messages, have shown modest improvements in HPV vaccination among adolescents (Bar-Shain et al., 2015; Cassidy et al., 2014; Chao et al., 2015; Kharbanda et al., 2011; Matheson et al., 2014; Morris et al., 2015; Rand et al., 2015; Suh et al., 2012; Szilagyi et al., 2013; Rand et al., 2017). Many studies have used non-randomized designs (Bar-Shain et al., 2015; Cassidy et al., 2014; Kharbanda et al., 2011; Matheson et al., 2014; Lee et al., 2016) and have been conducted within public health agencies (Morris et al., 2015; Coley et al., 2018; Szilagyi et al., 2020) managed care settings (Chao et al., 2015; Rand et al., 2015; Szilagyi et al., 2013) or private practices (Cassidy et al., 2014; Kharbanda et al., 2011; Suh et al., 2012). Little prior research has focused on low-resource settings or studied the comparative effectiveness and implementation of multiple HPV reminder types.

We report on findings from a study that implemented and evaluated the effectiveness and implementation of three parent reminder types (mailed letters, robocalls, text messages) on receipt of a needed HPV vaccine dose among 12-year-old girls and boys in a large multi-site Federally Qualified Health Center in Los Angeles County. We hypothesized that adolescents of parents who received an HPV vaccine reminder would have a higher rate of needed dose completion compared with those who received usual care and that text messages would be the most feasible reminder type. Given limited prior research, we did not have an a priori hypotheses regarding the relative effectiveness of reminder types.

2. Methods

2.1. Study design and setting

Using a quasi-experimental design, we evaluated the effectiveness and implementation of three different clinic-level reminder systems on HPV vaccine next-dose (dose 1 or 2) completion among adolescents receiving primary care from a large, multi-site FQHC in Los Angeles County (See Fig. 1). Next-dose was defined as either a) first dose among those who had not initiated the vaccine series or b) a second dose among those already initiated. At the time of the study guidelines for 12-year-olds required two doses of the vaccine administered 6 months apart to be considered fully vaccinated. We selected 6 clinics and paired them on baseline HPV vaccination rates (58% overall had received 1 or more HPV vaccine doses), adolescent patient population size, and clinic location, and randomly assigned a clinic within each pair to either usual care or one of the three reminder interventions: mailed letter, automated telephone call (a.k.a. robocall), or text message. Eligible patients were identified through electronic medical record and population health management systems and included caregivers/parents of 12-year-old boys and girls who were due for an HPV vaccine dose (received no doses or only one dose of the HPV vaccine). For each reminder type, we compared rates of HPV vaccine next-dose receipt between intervention and control clinics at 4-month follow-up. The UCLA Institutional Review Board and the FQHC’s Research Committee approved this study, indicating that that the study adhered to all relevant guidelines regarding the protection of human subjects and the safety and privacy of project data.

Fig. 1.

Study Design. Six clinics were paired on baseline HPV vaccination rates, adolescent patient population size, and clinic location, and clinics within each pair were randomly assigned to usual care or an HPV vaccine reminder intervention. For each reminder type, rates of HPV vaccine next-dose (dose 1 or 2) receipt for 12-year-olds were compared between intervention and control clinics at 4-month follow-up.

2.2. Intervention

The reminder intervention was implemented between November and December 2016. The FQHC’s centralized call center generated the mailed letter and robocall reminders, and an FQHC third-party vendor sent the text messages. Reminder content was adapted from existing clinical message scripts. The content of the mailed letter and robocall was as follows:

“Dear [patient name], Our records indicate that you are due for the 1^st dose of the HPV vaccine. The HPV vaccine protects boys and girls against many diseases. The vaccine is given in a series of 2 shots that are 6 months apart. It is important to complete the series. Please call the Appointment Center at XXX-XXX-XXXX to schedule your appointment.”

The text message reminder was briefer in order to comply with the third-party vendor’s character restrictions.

“Hello from [clinic name]. Your child [patient name] is due for the 1^st dose of the HPV vaccination. Call the Appointment Center at XXX-XXX-XXXX to schedule your appt. Text STOP at any time to stop receiving messages.”

Reminder messages were delivered in the patients’ preferred language (either English or Spanish). The same reminder type (either letter, call, or text message) was used on up to two occasions: once at baseline and a second time approximately one month after baseline for parents of adolescents who did not receive a vaccine dose or have scheduled appointment to receive a dose.

2.3. Usual care

At usual care clinics, no initiatives targeting parents to improve HPV vaccine uptake, including reminders, were being implemented. The quality improvement team occasionally disseminated up-to-date child and adolescent vaccine schedules to providers, but no other provider-focused initiatives were being carried out. The first HPV vaccine dose was routinely administered during a provider visit though subsequent doses could be administered during a nurse visit. At the time of the first dose, parents received a CDC Vaccine Information Sheet and information about the timing of subsequent vaccine doses.

2.4. Data measures

Data were derived from electronic medical record and population health management systems. Our primary outcome was receipt of next-dose vaccination (i.e., receipt of the first dose for unvaccinated children or second dose for children who had completed one prior dose, yes/no). Data on patient characteristics, including race and ethnicity and sex were abstracted from the electronic health record system. Related to implementation data, we assumed that reminders were successfully delivered if we did not receive feedback to the contrary (i.e., text message bounce back, returned letter, or non-completed/unanswered robocall). We considered robocalls where a message could be left to be successfully delivered.

2.5. Statistical analysis

All analyses were conducted using Stata/SE 15.1. Analyses were conducted to obtain the overall effectiveness of reminders (any type) and effectiveness stratified by reminder type, sex and dose number (first versus second). Vaccination rates are presented as simple proportions (total vaccinated divided by total sample size). P-values for intervention effects were obtained using logistic regression models with vaccine receipt as the outcome variable. All models except those stratified by reminder type included health center as a covariate. All models except those stratified by dose number included dose number as a covariate.

3. Results

3.1. Sample characteristics

Table 1 reports adolescent characteristics (n = 877). Approximately half of adolescents were female, and the majority was Latino (85%) and insured by Medi-Cal (61%; Medicaid program in California). There were no differences in patient characteristics between intervention and control groups.

Table 1.

Sample characteristics.

	Intervention	Control
	n = 417	n = 460
	n (%)	n (%)
Gender
Female	209 (50%)	204 (44%)
Male	208 (50%)	256 (56%)
Ethnicity
Latino	358 (86%)	390 (85%)
Non-Latino white	36 (7%)	46 (10%)
Non-Latino other	20 (5%)	21 (5%)
Not reported	3 (1%)	3 (1%)
Insurance
Medi-Cal	283 (74%)	255 (52%)
Uninsured	11 (3%)	8 (2%)
Private	1 (<1%)	0 (0%)
Not reported	122 (32%)	197 (40%)
Reminder type (intervention group, n = 417)
Mailed letter	158 (38%)	–
Text message	139 (33%)	–
Robocall	120 (28%)	–
HPV vaccine dose group
Due for 1st vaccine dose	249 (60%)	309 (67%)
Due for 2nd vaccine dose	168 (40%)	151 (33%)
Outcomes (at 4-month follow-up)
Received next-dose vaccine	94 (23%)	56 (12%)
Reminder delivered (intervention group)	352 (84%)	–

3.2. Next-dose HPV vaccine receipt

Overall, 23% of adolescents in any reminder condition received the next HPV vaccine dose compared to only 12% of usual care patients (11 percentage point [pp] advantage over controls, p = 0.046; Table 2). In analyses stratified by reminder type, we only observed a significant effect for text message reminders (11 pp. advantage, p = 0.036). In analyses stratified by child sex, we observed a significant effect of any reminder on receipt of the next needed HPV vaccine dose in boys (15 pp. advantage, p = 0.006), but not in girls (6 pp. advantage, p = 0.829).

Table 2.

Rates of HPV vaccine next-dose at 4-month follow-up.

	Intervention n = 417		Control n = 460		Difference in rates	P-value
	n	%	n	%	%
All reminders	94/417	23%	56/460	12%	11%	0.046
Reminder type
Mailed letter	40/158	25%	18/140	13%	12%	0.085
Text message	32/139	23%	25/210	12%	11%	0.036
Robocall	22/120	18%	13/110	12%	7%	0.115
Child sex
Girls	40/209	19%	27/204	13%	6%	0.829
Boys	54/208	26%	29/256	11%	15%	0.006

3.3. Implementation of reminders

Figure 2 displays the percentage of reminders (n = 417) that were successfully delivered. The percentage of reminders successfully delivered increased for the second reminder compared to the first for both text messages (first reminder 83% [116/139]; second reminder 92% [107/116]) and robocalls (first reminder = 67% [80/120); second reminder 78% [79/101]). This likely occurred because contact information was updated, as possible, when initial reminders were undelivered. Robocalls appeared to be the least feasible reminder type with the lowest rates of successful delivery for the first and second reminders. Ninety-two percent of both text message (107/117) and mailed (118/128) second reminders were successfully delivered. Opt out rates for text messages were low (2%).

Fig. 2.

Percent of reminders successfully delivered out of attempted by reminder type.

Reminders were assumed to be successfully delivered if no data to the contrary was received. We assumed reminders were not successfully delivered in the following scenarios: bounce back received for text messages, unanswered/uncompleted robocalls, mailed letter returned by post office.

4. Discussion

We utilized a quasi-experimental design to evaluate the comparative effectiveness of three types of parent reminders (mailed letters, text messages, robocalls) on receipt of the next needed HPV vaccine dose in adolescents. We found that receipt of any reminder type increased HPV vaccine receipt compared to usual care and that this overall effect was likely driven by significant increases among boys and those who received text message reminders. In addition to effectiveness, we also examined reminder implementation and found robocalls appeared to be the least feasible reminder type, with the lowest rates of successful delivery.

Our results add to the existing literature suggesting the effectiveness of parent reminders on increasing HPV vaccination, particularly text messages. The effect size of our text reminders (11 percentage-points) was within the range of prior interventions conducted among predominantly publicly insured adolescents (3 to 18 percentage-points) (Kharbanda et al., 2011; Rand et al., 2015; Rand et al., 2017). A randomized study targeting parents of unvaccinated adolescents found a modest effect (3 percentage-point difference, 16% intervention versus 13% control) on HPV vaccine initiation (Rand et al., 2015). Stronger effects have been observed in other studies that used a greater number of follow-up messages and incorporated parent reminder preference (Kharbanda et al., 2011; Rand et al., 2017). In a two-arm randomized trial in urban primary care clinics, Rand et al. (2017) offered parents the option between text and robocall reminder methods and found significantly higher rates of HPV vaccine series completion among adolescents in the text message arm compared to controls (49% versus 30%) (Rand et al., 2017). In a non-randomized study, Kharbanda et al. 2011 observed a higher rate of next-dose receipt (for second and third doses) among daughters of parents who actively signed up for weekly text reminders versus those who did not choose to enroll (52% versus 35%) (Kharbanda et al., 2011). Among controlled studies evaluating HPV and other adolescent vaccine reminders, many have utilized individual randomization (Chao et al., 2015; Morris et al., 2015; Rand et al., 2015; Suh et al., 2012; Szilagyi et al., 2013; Rand et al., 2017), which limits the ability of findings to inform implementation of reminders at a system level. We chose to assign clinics rather than individuals to receive reminders in order to understand the effectiveness and feasibility in a context that more closely resembles reminder implementation in the real world. Furthermore, we integrated HPV reminder messages into the existing clinic infrastructure, systems used for appointment and other preventive health reminders, to enhance sustainability.

Consistent with prior studies, we also found that robocalls led to a non-significant improvement in HPV vaccine dose receipt (Rand et al., 2017; Szilagyi et al., 2020), despite prior studies demonstrating their efficacy in improving other childhood immunizations (Szilagyi et al., 2000). Our findings could be due to the relatively low reach of our robocall intervention; one-third of initial reminders were unanswered by a person or machine/voicemail. The vast majority of Americans, including a 97% of low-income adults, own a cellphone or smartphone, which can be easily programmed to block unwanted calls (Pew Research Center, n.d.). Prior qualitative research has also revealed parent preference for text messages over phone or mailed child immunization reminders, as text messages are concise and easily retrievable (Kharbanda et al., 2009).

Our finding that receipt of any reminder significantly improved vaccination among boys, but not girls, adds to the limited literature reporting results by child gender (Morris et al., 2015; Rand et al., 2017). A cue to action alone may have been sufficient to motivate parents with sons, but not those with daughters. Parents with daughters (versus sons) may have been more concerned about vaccine side effects or perceived less need for the vaccine due to their child’s lack of sexual activity, as has been suggested in previous studies (Lindley et al., 2016; Thompson et al., 2017). Additionally, providers may have been less likely to recommend the vaccine to parents of 12-year old boys (versus girls), as shown in prior research (McRee et al., 2014). If so, it is possible that parents of boys in our sample may have been less aware of HPV vaccination prior to their study involvement and thus may have been more influenced by our brief intervention.

Importantly, our intervention was highly feasible, particularly for mailed and text reminders. Rates of successful delivery of text messages, for example, exceeded 80% compared to only 54% in another study conducted within a managed care organization for a publicly insured population (Rand et al., 2015). We recognize that we assessed implementation by a relatively crude measure. Successful delivery rates may be lower than observed if we did not receive notices of unsuccessful delivery (e.g., delays in notification of undelivered mail). Our definition of “successful” reminder delivery does not capture participant reminder receipt (e.g. letters or text reminders read, robocall voicemail heard). Anecdotally, the FQHC perceived mailed reminders to be more challenging to implement than text messages, due to the cost and time-consuming process of generating and sending mailed reminders compared to programming the text message reminders into the vendor-supplied software. Informal feedback from parents suggested they were less receptive to robocalls than other reminder types.

We acknowledge limitations to our study. Our study focused on one FQHC health care system with a centralized system serving a mostly Latino population. We also directed reminder messages to parents with 12-year-old children and thus results may not be generalizable to older adolescents or adults eligible for catch-up vaccinations. Sample size limitations also precluded direct comparisons of reminder types. Nevertheless, our study had several strengths, including use of low-cost reminders that require minimal office staff to sustain: overdue adolescents were identified through population-based management systems, letters were generated through a centralized back office, and text message reminder implementation was tracked through third party vendor data.

5. Conclusion

In conclusion, our study contributes evidence that supports use of parent reminders, particularly text messages, to improve adolescent HPV vaccination. Future research is needed to confirm these findings in a larger sample and to consider the effectiveness, cost, and implementation challenges of parent reminders compared to other clinic-based vaccine promotion strategies.

Abbreviations:

HPV

Human papillomavirus

FQHC

Federally qualified health center

Data availability

The data that has been used is confidential.