Skip to main content
PMC home page
Vaccines logoLink to Vaccines
. 2023 Apr 20;11(4):872. doi: 10.3390/vaccines11040872

Using Electronic Reminders to Improve Human Papillomavirus (HPV) Vaccinations among Primary Care Patients

Kathleen Hanley 1,*, Tong Han Chung 1, Linh K Nguyen 1, Tochi Amadi 1, Sandra Stansberry 1, Robert J Yetman 2, Lewis E Foxhall 3, Rosalind Bello 3,4, Talhatou Diallo 1, Yen-Chi L Le 1
Editors: Jane Richards Montealegre, Lindy U McGee
PMCID: PMC10145812  PMID: 37112784

Abstract

The COVID-19 pandemic led to delays in routine preventative primary care and declines in HPV immunization rates. Providers and healthcare organizations needed to explore new ways to engage individuals to resume preventive care behaviors. Thus, we evaluated the effectiveness of using customized electronic reminders with provider recommendations for HPV vaccination to increase HPV vaccinations among adolescents and young adults, ages 9–25. Using stratified randomization, participants were divided into two groups: usual care (control) (N = 3703) and intervention (N = 3705). The control group received usual care including in-person provider recommendations, visual reminders in exam waiting rooms, bundling of vaccinations, and phone call reminders. The intervention group received usual care and an electronic reminder (SMS, email or patient portal message) at least once, and up to three times (spaced at an interval of 1 reminder per month). The intervention group had a 17% statistically significantly higher odds of uptake of additional HPV vaccinations than the usual care group (Adjusted Odds Ratio: 1.17, 95% CI: 1.01–1.36). This work supports previous findings that electronic reminders are effective at increasing immunizations and potentially decreasing healthcare costs for the treatment of HPV-related cancers.

Keywords: human papillomavirus, vaccine, immunization, patient, electronic reminders, provider recommendation, patient portal, electronic health record system

1. Introduction

Over 90% of HPV-related cancers are preventable; increasing HPV vaccination rates help reduce high-risk HPV infections that lead to pre-cancers, cervical cancer, anogenital cancer, oropharyngeal cancer, and genital warts [1,2,3,4]. The American Cancer Society (ACS) and American Academy of Pediatrics (AAP) state HPV Vaccination is recommended for ages 9 through 26 and can also be recommended for patients 27–45 with shared clinical decision-making [5,6]. Patients starting their HPV vaccine before age 15 are recommended to have two doses, while patients 15 or older are recommended to have three doses [7].

The COVID-19 pandemic led to public health challenges such as delays in preventative care and decreases in immunization rates [8,9,10,11]. With the reduction in adolescents receiving vaccinations due to the COVID-19 pandemic, providers have been called on to promote vaccinations [12]. HPV vaccine administration in the US decreased by a median of 63.6% and 71.3% for ages 9–12 and ages 13–17, respectively, from 2018/2019 to March–May 2020 [11]. While vaccine administration started to increase June–September 2020 nearly to pre-COVID-19 levels in most U.S. regions, it was not enough to reach catch-up coverage for the many months of missed vaccinations. Healthcare providers were called to assess the vaccine status of their patients in order to reach herd immunity and prevent outbreaks [11]. Furthermore, vaccine hesitancy has drastically increased as well as the spread of misinformation on HPV, influenza and COVID-19 vaccines [13]. Unfortunately, exposure to negative information on HPV vaccination and mistrust in the medical system/healthcare providers is associated with parental hesitancy towards HPV vaccination [13]. Adolescents of HPV vaccine-hesitant parents were less likely to receive or initiate their HPV vaccinations [13,14]. Declines in vaccination coverage created the need for innovative interventions to increase vaccine uptake among individuals eligible for HPV vaccination. HPV vaccination coverage goals set by Healthy People 2030 are 80% [15]; however, in Texas, HPV vaccine initiation rates for ages 13–17 were 71.3% and vaccine completion rates were 51.5% in 2021 [16]. Compared to all 50 states and Washington DC, Texas ranked 48th for HPV vaccine completion rate and 44th for initiation rates [17]. Texas has experienced vaccination challenges, with the 2021 vaccine completion rate dropping 3.4% from 2020 to the 2021 rate [16]. In Harris County, for adolescents ages 9–12, initiation rates were 23.7% and completion rates were 8.2% [16]. For adolescents ages 13–17, HPV initiation rates were 67.8% and completion rates were 40.2% [16].

Systematic reviews have identified physician recommendation as the most significant predictor of vaccination uptake [18,19,20,21,22,23]. Furthermore, the Centers for Disease Control (CDC) recognizes physician recommendation as the top reason for vaccination [24]. Physician recommendation usually occurs during in-person well child or preventive care visits. However, with the ongoing pandemic, there was a dramatic decrease in healthcare utilization [8]. Innovative approaches in primary prevention were necessary to prevent further declines in vaccination. Past research identified several evidence-based strategies for improving vaccination rates. A Cochrane review found support for text messages, telephone calls, letters, postcards, and auto-dialer messages in improving immunization rates [25,26]. Based on the Cochrane review, there is strong evidence that patient reminder or recall system interventions improve vaccinations for children and adolescents [25]. At the time of our intervention development, past studies demonstrated text messaging as an effective reminder system for HPV vaccination [27,28,29,30,31,32]; however, studies on patient portal messaging for HPV reminders are limited with mixed results. One recent randomized controlled study using patient portal reminders found patient portal messaging improved influenza vaccination [33]. Whereas another study using Epic MyChart patient portal messaging for late-season Influenza vaccination found no significant differences in vaccination uptake between the intervention and control groups [23]. Another recent study found no effect of tailored reminders (based on demographics with behavior economic messaging) sent by a health system’s patient portal on influenza vaccination rates and a minimal effect (4.5%) among the young adult patient group who opened the pre-commitment portal messages asking if they planned to get vaccinated [34].

Brief educational messages can be extremely effective when structured with a strong physician recommendation highlighting the importance and the prevention of cancer [18]. We created customized electronic reminder messages that pair strong provider recommendations with brief education and emphasis on the importance and cancer prevention, a recommended strategy for strengthening provider communication about HPV vaccination [18]. To our knowledge, this is the first study to include a patient’s specific provider’s name as part of the electronic patient reminder, rather than the organization’s name [34].

The primary goal of this project was to estimate the effectiveness of customized electronic reminders (SMS, email, and patient portal message) with provider recommendations for HPV vaccination on increasing HPV vaccination rates among adolescents and young adult patients. The intervention was expected to increase appointments scheduled, clinic visits, and receipt of an HPV vaccine dose. Study results will provide useful information to healthcare institution decision-makers for utilizing electronic reminders via the electronic health record (EHR) system.

2. Materials and Methods

2.1. Study Design, Setting, and Ethics Approval and Consent

This study was approved as a Quality Improvement (QI) Project by the UTHealth Quality Improvement Project Registry (No. 2021-1135). This was a pragmatic randomized controlled study at 13 multiple primary care clinics of one organization in Houston, TX. Since this is a QI study, patient consent was waived. All patients received usual care whereas the intervention group received customized electronic reminders in addition to usual care.

2.2. Study Population

The study population included 7408 patients age 9–25 who: (1) had at least one office visit at a primary care (family medicine, general medicine, pediatrics) clinic from 1 January 2021 to 31 December 2021, (2) were eligible to receive the HPV vaccine, and (3) had a valid communication method such as mobile phone number, email address, or patient portal activation documented within the EHR system. Our EHR system is Epic; MyChart is a web-based patient portal in Epic that allows patients access to their medical records and to communicate with their providers. Eligibility included patients who had not yet received an HPV vaccine (or had no record of vaccination in their patient record) as well as patients who initiated but did not complete their HPV vaccine series. Vaccine series completion was defined based on the CDC HPV vaccination recommendations [7]. Patients were excluded if they were pregnant, had immunization contraindications in their medical record, or had documented vaccine refusal.

2.3. Randomization

Patients were randomly assigned by stratified randomization into two groups, usual care (n = 3703) or intervention (n = 3705), based on age, vaccine status, sex and clinic location using Stata 14.0 (StataCorp., College Station, TX, USA).

2.4. Study Intervention and Usual Care Group

The intervention group received one type of electronic reminder, following a hierarchical order: (1) SMS if the patient opted in to receive SMS; (2) a patient portal message via MyChart if the patient opted in for MyChart messages but not SMS; or (3) an email reminder if the patient did not opt in to SMS or MyChart messages. The electronic reminder was addressed to the parent/caregiver or patient (based on the patient’s age) and included physician recommendation and brief education (See Table 1). Messages to patients under 18 years of age were addressed to the parent/caregiver of the patient. Messages to patients 18+ years of age were addressed directly to the patient.

Table 1.

Message framework for electronic patient reminders for HPV.

Patient Less Than 18 Years Old Patients 18 Years or Older
Initial Message Dr. Yetman [PCP name] recommends Jane [Patient name] complete their HPV vaccination to prevent cancers caused by HPV. To schedule go to [Scheduling link]. Reply OK to confirm receipt. John [Patient Name]: Dr. Foxhall [PCP name] recommends you complete your HPV vaccination to prevent cancers caused by HPV. To schedule, go to [Scheduling link]. Reply OK to confirm receipt.
Second or Third Message Reminder: Dr. Yetman [PCP name] recommends Jane [patient name] complete their HPV vaccination. To schedule go to [Scheduling link]. Reply OK to confirm receipt. Reminder for John [Patient Name]: Dr. Foxhall [PCP name] recommends you complete your HPV vaccination. To schedule, go to [Scheduling link]. Reply OK to confirm receipt.
Open in a new tab

The control group received usual care which included in-person provider recommendations, visual reminders in exam waiting rooms, bundling of vaccinations, and phone call reminders. Patient appointments, clinic visits, and HPV vaccine status were captured in the patient’s EHR. For ease, we will refer to the control group as the usual care group.

2.5. Intervention Period

We used Epic’s Campaigns application, a mass patient outreach tool within Epic to send messages (SMS, patient portal, email) to the intervention group. Epic Campaigns allows users to send patient portal messages and emails through the EHR platform. However, SMS messages were sent using a data file exported from Epic and uploaded into a third-party SMS system, Tavoca. Messages were sent once a month for up to three months, based on whether a patient took “action” after the message. Patients who scheduled a future appointment or nurse visit or completed their HPV vaccination series were counted as a success and did not receive future messages. However, patients who did not take “action” received monthly reminders for an additional two months. All patients who opted out of electronic reminders were removed from receiving additional messages. We followed patients for 6 months after the intervention start, to allow time for patients to schedule and complete their HPV vaccine (Figure 1).

Figure 1.

Figure 1

Open in a new tab

Consort flow diagram-participant randomization, intervention, and analysis.

2.6. Study Measures and Data collection

Patient demographic and clinical data, including date of birth, gender, vaccine dose, vaccine status, appointment scheduling, and clinic visits were extracted from the EHR system to determine the eligible population, baseline data, and follow-up data.

The study outcomes include HPV vaccination-associated appointments, clinic visits, and vaccination rates. We considered patients making future appointments as a proxy for vaccine intention. Based on the Theory of Planned Behavior (TPB) and past research, vaccination intentions can predict behavior [35,36]. Therefore, we compared rates of appointment scheduling and clinic visit(s) between the two groups. We included appointments and visits for well child, well women exam, nurse visit or physical only and excluded sick visits.

All additional HPV vaccinations during the intervention period were estimated among all patients. The vaccine initiation rate was also calculated as the percentage of eligible patients who had ≥1 dose but had not yet completed the series among eligible patients who had not received the HPV vaccine. The vaccine completion rate was calculated as the percentage of patients who completed the vaccine series among the entirety of eligible patients.

2.7. Analysis

Descriptive analysis was performed with frequency distributions. A chi-square test was performed for categorical variables. We applied multiple logistic regression models for analyses of the intervention effects. Effect sizes were presented as odds ratios with 95% confidence intervals. Socioeconomic factors and the three stratifying variables were adjusted as potential confounders (age, sex, race/ethnicity, insurance and vaccine status). An intent-to-treat analysis was used as a primary analysis. There were differences between the randomized intervention group and the actual intervention group who received the intervention mainly due to the patients’ opt-out status of all the modes of the intervention (SMS, email and patient portal) and patients’ incorrect phone or email information which make them unreachable. Therefore, we also conducted a secondary analysis among the intervention group who eventually received the intervention in comparison with the usual care group in order to assess the practical effectiveness of the intervention. The statistical analysis was conducted using Stata 14.0 (StataCorp., College Station, TX, USA).

3. Results

3.1. Patient Demographics

Table 2 displays patient demographics at baseline. A total of 7408 patients who met eligibility were randomized. The intervention group and usual care group followed nearly identical distributions in age, sex, vaccination status, race/ethnicity and insurance type. Patients ages 9–14 accounted for approximately 40%, ages 15–18 accounted for 19–20% and ages 19–25 accounted for 40% of the population. Approximately 55% of patients were female and 68% were not yet vaccinated. Non-Hispanic White, non-Hispanic Black, Hispanic and other races/ethnicity each accounted for 25% of the population. Privately insured patients accounted for 55%, Medicaid 36%, and uninsured 8–9% of the population for both groups (intervention and usual care).

Table 2.

Baseline Characteristics by randomized group.

Baseline Characteristics Electronic Reminder
(N = 3705)
N (%)		 Usual Care
(N = 3703)
N (%)		 p-Value
Age 0.689
9–14 1511 (40.78) 1535 (41.45)
15–18 721 (19.46) 693 (18.71)
19–25 1473 (39.76) 1475 (39.83)
Sex 0.942
Male 1676 (45.24) 1672 (45.15)
Female 2029 (54.76) 2031 (54.85)
Vaccine Status 0.834
Not initiated 2542 (68.61) 2549 (68.84)
Initiated 1163 (31.39) 1154 (31.16)
Race/Ethnicity 0.176
Non-Hispanic White 865 (23.35) 939 (25.36)
Non-Hispanic Black 955 (25.78) 962 (25.98)
Hispanic 911 (24.59) 874 (23.60)
Other/Unknown 974 (26.29) 928 (25.06)
Insurance 0.134
Medicaid 1317 (35.55) 1337 (36.11)
Private (Managed Care) 2014 (54.36) 2049 (55.33)
Uninsured 338 (9.12) 291 (7.86)
Other 36 (0.97) 26 (0.70)
Open in a new tab

3.2. Appointment Scheduling and Clinic Visit

The intervention group had higher rate of appointment scheduling for HPV vaccination than the usual care group (Adjusted Odds Ratio (AOR): 1.12, 95% CI: 1.00–1.26). The intervention group also had a higher rate of clinic visits than the usual care group (AOR: 1.12, 95% CI: 0.90–1.38). However, the differences in rates were not statistically significant (Table 3).

Table 3.

Multiple logistic regression * analysis results of HPV outcomes.

Electronic Reminder Usual Care
(N = 3705) (N = 3703)
HPV Outcomes N (%) N (%) Adjusted OR
(95% CI *)		 p-Value
Appointment Scheduling 752 (20.3) 700 (18.9) 1.12
(1.00, 1.26)		 0.056
Clinic Visit 423 (11.42) 419 (11.32) 1.07
(0.94, 1.23)		 0.300
All Additional HPV Vaccination(s) 450 (12.15) 402 (10.86) 1.17
(1.01, 1.36)		 0.036
Open in a new tab

* Model controlled for age, sex, race/ethnicity, insurance and vaccine status.

3.3. HPV Vaccination Rates

The intervention group showed a 17% statistically significant increase in the odds of obtaining an additional HPV vaccination than the usual care (AOR: 1.17, 95% CI: 1.01–1.36) (Table 3). Breaking down the analysis by initiation and completion rate, differences in vaccine initiation rate (AOR: 1.22, 95% CI: 0.98–1.51) and completion rate (AOR: 1.12, 95% CI: 0.90–1.38) were noted between the intervention and usual care group but were not statistically significant (Table 4).

Table 4.

Subgroup vaccination outcomes.

Electronic Reminder Usual Care
(N = 2529) (N = 2538)
N (%) N (%) Adjusted OR
(95% CI**)		 p-Value
HPV vaccine initiation rate 206 (8.15) 181 (7.13) 1.22
(0.98, 1.51)		 0.076
Electronic Reminder Usual Care
(N = 3705) (N = 3703)
N (%) N (%) Adjusted OR
(95% CI**) 		p-Value
HPV vaccine completion rate 225 (6.07) 207 (5.59) 1.12
(0.90, 1.38)		 0.302
Open in a new tab

Secondary Analyses

Of the 3705 patients in the intervention group, only 2768 received one or more electronic reminders. The distributions of age, sex, vaccine status, race/ethnicity and insurance between the two groups were slightly different. The usual care group had a higher proportion of patients ages 9–14 (41.5%) compared with the intervention group (33.6%) and a slightly higher proportion of patients under Medicaid (36%) compared with the intervention group (32.4%) (Supplementary Table S1).

Patients who received the intervention were more likely to schedule an appointment (AOR: 1.2, 95% CI: 1.06–1.37) and have an additional HPV vaccination (AOR: 1.35, 95% CI: 1.14–1.59) compared to the usual care group patients. In addition, among the intervention group HPV vaccine initiation rate (AOR: 1.32, 95% CI: 1.04–1.68) and completion rate (AOR: 1.32, 95% CI: 1.12–1.56) were significantly higher than those among the usual care group. (Supplementary Tables S2 and S3).

4. Discussion

Our study assessed how customized electronic reminders (SMS, email, and MyChart) that included a provider recommendation and education on cancer prevention, impacted HPV vaccination rates among adolescents and young adults in a primary care setting. The intervention was essential to impacting the drop in HPV vaccinations due to the COVID-19 pandemic. Our study had several strengths. We utilized a practical population-based approach to increase vaccination rates in an outpatient, primary care setting. We had a relatively large sample size. Our organization also has bidirectional data exchange with two external immunization information systems (ImmTrac2 and CareEverywhere). This allowed our patient immunization records to contain both immunizations administered by our organization and elsewhere. We also leveraged the EHR to implement a technology-based intervention. Although the initial cost of adopting a systems-level patient reminder tool can be significant, maintenance costs are relatively minimal. In addition, adopting automated patient reminder tools may also minimize the human capital resources that are typically needed to manually place individual phone calls and allow for the customization of messages and tailored interventions. To the best of our knowledge, our study is one of the first to use customized patient portal messages for HPV vaccination [37], in addition to using SMS and emails.

The latest Cochrane review of patient reminder and recall systems to improve immunization found a “high certainty of evidence” to improve immunizations through single-method reminders: postcards, text messages, auto-dialer phone calls and moderate evidence for telephone [26] calls and letters. At the time of Jacobson et al., there were no studies utilizing patient portal messages that met the Cochrane review inclusion criteria. Previous research has found internet and mobile technology (such as recall prompts, SMS, interactive videos, phone calls, and email) effective at improving HPV vaccination and completion [37]. In reviews by Francis et al. (2017) and Acampora et al. (2020) [37,38], the EHR was used for provider recall interventions rather than patient reminder messages. As previous researchers have suggested exploring [26], we utilized an EHR patient portal system and its secure messaging in our intervention and found that the intervention group was 17% more likely to receive an additional HPV dose than the usual care group. Our study also provides evidence that customized electronic patient reminders tailored to include an HPV Vaccine recommendation from the patient’s primary care provider may increase HPV vaccination. The literature supports provider recommendations to increase vaccination, and this study shows an electronic practical solution for healthcare clinics.

Another strength is that our study is one of a few studies that assess future preventive care appointments scheduled during the intervention/post-intervention period as one of the study outcomes. Future scheduled appointments may function as a process measure for measuring intervention effectiveness. Scheduled appointments and clinic visits might serve as an intermediate outcome of the study impact prior to the completion of a vaccination, which would be the primary outcome. For example, a patient or caregiver who received the HPV reminder may have scheduled an appointment for 7 months from the time they received the message with the intention of receiving the HPV vaccine in the future. This patient may have the intention to obtain the HPV vaccine but would not be included among those who received an HPV dose during the study period. In a randomized trial by a managed care organization (MCO), an immunization reminder/recall intervention on routine childhood immunizations (Meningococcus, pertussis, HPV) had a modest impact on preventative care visits (mail intervention: 65% visit, telephone 63% visit, and control group: 59% visit) [39]. Similarly, our study did not find statistically significant differences in appointment scheduling and clinic visits between the usual care and intervention groups, rates in scheduling (20.3% vs. 18.9%) and clinic visits (11.42% vs. 11.32%) were higher among the intervention group compared to the usual care group.

Past studies that have small effect sizes between the intervention and usual care group may be attributed to high standards for usual care [26,40]. In our community-based clinics, usual care typically involves one or more of the following evidence-based strategies: in-person provider recommendation, visual reminders in exam waiting rooms, bundling of vaccinations, and phone call reminders for patients. Some studies have found that patient navigator phone calls increased vaccine completion by 10% more than the control group [41]. A combination of patient reminder/recall and provider reminder interventions has been shown to likely improve immunizations based on moderate certainty evidence [26]. Therefore, it is possible that the additional benefit from our intervention showed a marginal impact when compared to our usual care.

Although our vaccination rate was lower than previous HPV vaccination studies [38], small effect sizes for interventions that increase immunizations are considered clinically meaningful due to their almost universal recommendation for populations and the protection of public health [26]. Every single HPV vaccination that may prevent cancer is a win for primary care providers and public health professionals.

One potential study limitation is that our study was implemented when recommendations for HPV vaccinations had just been expanded to begin at age 9. This may have contributed to our study’s overall low initiation and completion rates. In 2018, the AAP and in 2019, the ACS recommended HPV vaccination starting at age 9 [5,6], while the Advisory Committee on Immunization Practices (ACIP) recommends starting at ages 11–12, and as early as age 9 [7,42]. Considering this more recent change there is a slower uptake in vaccination by ages 9–12, with most adolescents completing vaccination as “catch-up” shots [43]. Prior to the start of our study, most providers in our clinics were routinely recommending HPV vaccination at 11 years of age and we were not systematically outreaching or tracking HPV vaccination rates for 9–10 year olds. The ACIP recommends HPV vaccination to all persons up to age 26. Heeding these recommendations, our study included patients in the 18–25 age group. Historically, this age group has lower HPV vaccine completion rates than adolescents 13–17 [44]. Including the 18–25 age group in our target population may have also lowered our study’s overall low rates. Current data dashboards on HPV vaccinations focus on adolescents 13–17 [45]; therefore, it is difficult to estimate national and statewide changes in HPV vaccinations by age group. Technology-based interventions such as ours that rely on automated reminders can provide a wider range of outreach to patients that include the 18–26 year age range with minimal additional costs.

Another potential study limitation may be delays in receipt of immunization records or missing immunization records. This may have also contributed to our study’s lower vaccination rates. Our study took place in Texas which has an opt-in immunization registry. Patients must provide consent to have their immunizations stored in ImmTrac2 and shared with healthcare organizations. We are not able to retrieve the immunization records of patients who have not consented. Even if patients have consented to ImmTrac2, it might take time for patients’ external vaccination information to be transmitted via ImmTrac2. Another source for external immunization data is CareEverywhere. Although our patients are automatically opted in to CareEverywhere access, participation from external vaccine administering organizations is voluntary. It is possible that our study timeframe may not have allowed us adequate time to capture HPV vaccinations that our patients received elsewhere. It is also possible that some patients may have received vaccinations from external organizations that do not participate in ImmTrac2 or CareEverywhere. Lastly, the pragmatic approach used in this quality improvement study may have also contributed to the small effect size.

Another study limitation is that our study population may not be representative of the general population. Our intervention utilized electronic reminders (email, text or patient portal) which excluded patients that did not provide a way for us to contact them electronically as well as patients who may not have access to electronic communications. Future studies may need to address technology inequities if they utilize a web-based or electronic approach for their intervention.

Even with the study limitations and limited generalizability we describe above, our study makes a meaningful contribution to the literature on HPV vaccination interventions. As a department that implements systematic operations within a healthcare institution, we implemented electronic patient reminders as a QI project, randomized patients, and implemented this intervention without notice to patients and providers and with minimal disruption to existing clinic workflows. Considering our practical setting, our study might show more realistic results while preserving as much internal validity as possible compared to a typical randomized controlled study. Although our study provides valuable pragmatic results, the results may not be generalizable to other locations or populations outside of the healthcare system.

HPV-related cancers have caused a considerable economic burden. In 2020, the total annual medical cost of cervical cancer was estimated to be $2.3 billion and the average per-patient costs for medical services were estimated to be around $160,200, which is the sum of the average annualized cost for each phase of care (initial, continuing and end-of-life) [46]. Given that HPV vaccines are extremely effective in preventing HPV-related cancers [47] and very low cost compared to the total per-patient cost for treating HPV-related cancers [48], vaccinating against HPV can be highly cost-saving for an individual. Therefore, even for studies that show a small increase in HPV vaccinations, the potential healthcare cost savings for vaccinated patients are non-negligible.

Many healthcare organizations use the EHR, making use of communication technologies for interventions more feasible with little added cost compared to human capital [37]. With the novelty of using patient portal messages, particularly Epic Campaigns, to increase vaccinations, and the likelihood of more adoption of the EHR for improving patient vaccinations, future studies should conduct an economic evaluation to investigate the positive financial impact on the healthcare system over time.

Furthermore, clinical practices can apply this messaging framework and patient portal delivery method to other immunizations. As suggested by Jacobson et al. (2018) and Francis et al. (2017) [26,37], future researchers may consider using EHR and technology to customize interventions based on risk or culture.

5. Conclusions

Improving HPV vaccination uptake is an important public health goal requiring continued attention due to the effects of the COVID-19 Pandemic, such as decreased healthcare utilization and growing mistrust in vaccines. Our study adds to the literature that customized electronic reminders can improve HPV vaccinations among eligible individuals. Future research should investigate whether our message framework is useful to other patient populations and if this pragmatic approach is applicable to other healthcare settings. Healthcare organizations may consider customized electronic reminders for other immunizations such as flu or pneumonia as well as other preventative care activities such as wellness visits and screenings that declined during the COVID-19 pandemic but are crucial for optimal health.

Acknowledgments

Authors would like to thank Salim Rahimtoola and Lizzie Koptik for their technological assistance with the conceptualization and launch of this project. We would like to thank Isaac Mancillas for his administrative role in the project.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/vaccines11040872/s1, Table S1: Baseline Characteristics of secondary analysis; Table S2: Secondary analysis of HPV outcomes; Table S3: Secondary analysis of subgroup vaccination outcomes.

Click here for additional data file. (82.4KB, zip)

Author Contributions

Conceptualization, Y.-C.L.L., K.H. and T.H.C.; methodology, Y.-C.L.L., K.H. and T.H.C.; software, T.A. and S.S.; formal analysis, T.H.C. and T.D.; investigation, T.H.C., K.H. and Y.-C.L.L.; data curation, T.A., S.S. and T.D.; writing—original draft preparation, Y.-C.L.L., K.H., T.H.C. and L.K.N.; writing—review and editing, K.H., L.K.N., Y.-C.L.L., T.H.C., T.A., R.B. and R.J.Y.; visualization, L.K.N. and T.H.C.; supervision, Y.-C.L.L., R.J.Y., R.B. and L.E.F.; project administration, K.H. and Y.-C.L.L.; funding acquisition, Y.-C.L.L. and L.E.F. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Ethical review and approval were waived for this study due to not meeting the regulatory definition of human subjects research.

Informed Consent Statement

Patient consent was waived due to the quality improvement nature of this project.

Data Availability Statement

The data presented in this study are available on reasonable request from the corresponding author. The data are not publicly available due to containing protected health information.

Conflicts of Interest

The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Funding Statement

This work was made possible by the UTHealth Academic Excellence Endowment and donors to The University of Texas MD Anderson Cancer Center’s Annual Fund.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

References

  • 1.Senkomago V., Henley S.J., Thomas C.C., Mix J.M., Markowitz L.E., Saraiya M. Human Papillomavirus–Attributable Cancers—United States, 2012–2016. MMWR Morb. Mortal. Wkly. Rep. 2019;68:724–728. doi: 10.15585/mmwr.mm6833a3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Drolet M., Bénard É., Pérez N., Brisson M., Ali H., Boily M.-C., Baldo V., Brassard P., Brotherton J.M.L., Callander D., et al. Population-Level Impact and Herd Effects Following the Introduction of Human Papillomavirus Vaccination Programmes: Updated Systematic Review and Meta-Analysis. Lancet. 2019;394:497–509. doi: 10.1016/S0140-6736(19)30298-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.CDC HPV Vaccine Safety and Effectiveness. [(accessed on 17 March 2023)]; Available online: https://www.cdc.gov/vaccines/vpd/hpv/hcp/safety-effectiveness.html.
  • 4.CDC HPV Vaccination: What Everyone Should Know. [(accessed on 17 March 2023)]; Available online: https://www.cdc.gov/vaccines/vpd/hpv/public/index.html.
  • 5.O’Leary S.T. Why the American Academy of Pediatrics Recommends Initiating HPV Vaccine at Age 9. Hum. Vaccines Immunother. 2022;18:2146434. doi: 10.1080/21645515.2022.2146434. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Saslow D., Andrews K.S., Manassaram-Baptiste D., Smith R.A., Fontham E.T.H., The American Cancer Society Guideline Development Group Human Papillomavirus Vaccination 2020 Guideline Update: American Cancer Society Guideline Adaptation. CA Cancer J. Clin. 2020;70:274–280. doi: 10.3322/caac.21616. [DOI] [PubMed] [Google Scholar]
  • 7.CDC HPV Vaccination Recommendations. [(accessed on 5 January 2023)]; Available online: https://www.cdc.gov/vaccines/vpd/hpv/hcp/recommendations.html.
  • 8.Czeisler M.É. Delay or Avoidance of Medical Care Because of COVID-19–Related Concerns—United States, June 2020. MMWR Morb. Mortal. Wkly. Rep. 2020;69:1250–1257. doi: 10.15585/mmwr.mm6936a4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.CMS Issues Urgent Call to Action Following Drastic Decline in Care for Children in Medicaid and Children’s Health Insurance Program Due to COVID-19 Pandemic | CMS. [(accessed on 23 December 2022)]; Available online: https://www.cms.gov/newsroom/press-releases/cms-issues-urgent-call-action-following-drastic-decline-care-children-medicaid-and-childrens-health.
  • 10.Santoli J.M. Effects of the COVID-19 Pandemic on Routine Pediatric Vaccine Ordering and Administration—United States, 2020. MMWR Morb. Mortal. Wkly. Rep. 2020;69:591–593. doi: 10.15585/mmwr.mm6919e2. [DOI] [PubMed] [Google Scholar]
  • 11.Patel Murthy B., Zell E., Kirtland K., Jones-Jack N., Harris L., Sprague C., Schultz J., Le Q., Bramer C.A., Kuramoto S., et al. Impact of the COVID-19 Pandemic on Administration of Selected Routine Childhood and Adolescent Vaccinations—10 U.S. Jurisdictions, March–September 2020. MMWR Morb. Mortal. Wkly. Rep. 2021;70:840–845. doi: 10.15585/mmwr.mm7023a2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Ryan G., Gilbert P.A., Ashida S., Charlton M.E., Scherer A., Askelson N.M. Challenges to Adolescent HPV Vaccination and Implementation of Evidence-Based Interventions to Promote Vaccine Uptake During the COVID-19 Pandemic: “HPV Is Probably Not at the Top of Our List”. Prev. Chronic Dis. 2022;19:210378. doi: 10.5888/pcd19.210378. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Tsui J., Martinez B., Shin M.B., Allee-Munoz A., Rodriguez I., Navarro J., Thomas-Barrios K.R., Kast W.M., Baezconde-Garbanati L. Understanding Medical Mistrust and HPV Vaccine Hesitancy among Multiethnic Parents in Los Angeles. J. Behav. Med. 2023;46:100–115. doi: 10.1007/s10865-022-00283-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Szilagyi P.G., Albertin C.S., Gurfinkel D., Saville A.W., Vangala S., Rice J.D., Helmkamp L., Zimet G.D., Valderrama R., Breck A., et al. Prevalence and Characteristics of HPV Vaccine Hesitancy among Parents of Adolescents across the US. Vaccine. 2020;38:6027–6037. doi: 10.1016/j.vaccine.2020.06.074. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Office of Disease Prevention and Health Promotion, Office of the Assistant Secretary for Health, Office of the Secretary, U.S. Department of Health and Human Services Increase the Proportion of Adolescents Who Get Recommended Doses of the HPV Vaccine—IID-08—Healthy People 2030 | Health.Gov. [(accessed on 5 January 2023)]; Available online: https://health.gov/healthypeople/objectives-and-data/browse-objectives/vaccination/increase-proportion-adolescents-who-get-recommended-doses-hpv-vaccine-iid-08.
  • 16.ACS HPV Vaccination Project Texas HPV Vaccination Landscape Dashboard. [(accessed on 5 January 2023)]. Available online: https://public.tableau.com/app/profile/acs.hpv.vaccination/viz/TexasHPVVaccinationLandscapeDashboard/Dashboard1.
  • 17.ACS HPV Vaccination Project HPV Landscape Dashboard. [(accessed on 5 January 2023)]. Available online: https://public.tableau.com/app/profile/acs.hpv.vaccination/viz/HPVLandscapeDashboard-November2019/Dashboard.
  • 18.Gilkey M.B., McRee A.-L. Provider Communication about HPV Vaccination: A Systematic Review. Hum. Vaccines Immunother. 2016;12:1454–1468. doi: 10.1080/21645515.2015.1129090. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Caldwell A.C., Madden C.A., Thompson D.M., Garbe M.C., Roberts J.R., Jacobson R.M., Darden P.M. The Impact of Provider Recommendation on Human Papillomavirus Vaccine and Other Adolescent Vaccines. Hum. Vaccines Immunother. 2021;17:1059–1067. doi: 10.1080/21645515.2020.1817713. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Rosenthal S.L., Weiss T.W., Zimet G.D., Ma L., Good M.B., Vichnin M.D. Predictors of HPV Vaccine Uptake among Women Aged 19–26: Importance of a Physician’s Recommendation. Vaccine. 2011;29:890–895. doi: 10.1016/j.vaccine.2009.12.063. [DOI] [PubMed] [Google Scholar]
  • 21.Kester L.M., Zimet G.D., Fortenberry J.D., Kahn J.A., Shew M.L. A National Study of HPV Vaccination of Adolescent Girls: Rates, Predictors, and Reasons for Non-Vaccination. Matern. Child Health J. 2013;17:879–885. doi: 10.1007/s10995-012-1066-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Gargano L.M., Herbert N.L., Painter J.E., Sales J.M., Morfaw C., Rask K., Murray D., DiClemente R.J., Hughes J.M. Impact of a Physician Recommendation and Parental Immunization Attitudes on Receipt or Intention to Receive Adolescent Vaccines. Vaccines. 2013;9:2627–2633. doi: 10.4161/hv.25823. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Fisher L., Loiacono M.M., Payne N., Kelley T., Greenberg M., Charpentier M., Leblanc C., Sundaresan D., Bancroft T., Steffens A., et al. A Novel Household-based Patient Outreach Pilot Program to Boost Late-season Influenza Vaccination Rates during the COVID-19 Pandemic. Influenza Respir. Viruses. 2022;16:1141–1150. doi: 10.1111/irv.13041. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.CDC The Top 10 Tips for HPV Vaccination Success. [(accessed on 3 February 2023)]; Available online: https://www.cdc.gov/hpv/downloads/Top10-improving-practice.pdf.
  • 25.Pich J. Patient Reminder and Recall Interventions to Improve Immunization Rates: A Cochrane Review Summary. Int. J. Nurs. Stud. 2019;91:144–145. doi: 10.1016/j.ijnurstu.2018.05.015. [DOI] [PubMed] [Google Scholar]
  • 26.Jacobson Vann J.C., Jacobson R.M., Coyne-Beasley T., Asafu-Adjei J.K., Szilagyi P.G. Patient Reminder and Recall Interventions to Improve Immunization Rates. Cochrane Database Syst. Rev. 2018;2018:CD003941. doi: 10.1002/14651858.CD003941.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Rand C.M., Vincelli P., Goldstein N.P.N., Blumkin A., Szilagyi P.G. Effects of Phone and Text Message Reminders on Completion of the Human Papillomavirus Vaccine Series. J. Adolesc. Health. 2017;60:113–119. doi: 10.1016/j.jadohealth.2016.09.011. [DOI] [PubMed] [Google Scholar]
  • 28.Kharbanda E.O., Stockwell M.S., Fox H.W., Andres R., Lara M., Rickert V.I. Text Message Reminders to Promote Human Papillomavirus Vaccination. Vaccine. 2011;29:2537–2541. doi: 10.1016/j.vaccine.2011.01.065. [DOI] [PubMed] [Google Scholar]
  • 29.Rand C.M., Brill H., Albertin C., Humiston S.G., Schaffer S., Shone L.P., Blumkin A.K., Szilagyi P.G. Effectiveness of Centralized Text Message Reminders on Human Papillomavirus Immunization Coverage for Publicly Insured Adolescents. J. Adolesc. Health. 2015;56:S17–S20. doi: 10.1016/j.jadohealth.2014.10.273. [DOI] [PubMed] [Google Scholar]
  • 30.Tull F., Borg K., Knott C., Beasley M., Halliday J., Faulkner N., Sutton K., Bragge P. Short Message Service Reminders to Parents for Increasing Adolescent Human Papillomavirus Vaccination Rates in a Secondary School Vaccine Program: A Randomized Control Trial. J. Adolesc. Health. 2019;65:116–123. doi: 10.1016/j.jadohealth.2018.12.026. [DOI] [PubMed] [Google Scholar]
  • 31.McGlone M.S., Stephens K.K., Rodriguez S.A., Fernandez M.E. Persuasive Texts for Prompting Action: Agency Assignment in HPV Vaccination Reminders. Vaccine. 2017;35:4295–4297. doi: 10.1016/j.vaccine.2017.06.080. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Matheson E.C., Derouin A., Gagliano M., Thompson J.A., Blood-Siegfried J. Increasing HPV Vaccination Series Completion Rates via Text Message Reminders. J. Pediatr. Health Care. 2014;28:e35–e39. doi: 10.1016/j.pedhc.2013.09.001. [DOI] [PubMed] [Google Scholar]
  • 33.Ueberroth B.E., Labonte H.R., Wallace M.R. Impact of Patient Portal Messaging Reminders with Self-Scheduling Option on Influenza Vaccination Rates: A Prospective, Randomized Trial. J. Gen. Intern. Med. 2022;37:1394–1399. doi: 10.1007/s11606-021-06941-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Szilagyi P.G., Albertin C.S., Casillas A., Valderrama R., Duru O.K., Ong M.K., Vangala S., Tseng C.-H., Humiston S.G., Evans S., et al. Effect of Personalized Messages Sent by a Health System’s Patient Portal on Influenza Vaccination Rates: A Randomized Clinical Trial. J. Gen. Intern. Med. 2022;37:615–623. doi: 10.1007/s11606-021-07023-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Shah S.F.A., Ginossar T., Bentley J.M., Zimet G., McGrail J.P. Using the Theory of Planned Behavior to Identify Correlates of HPV Vaccination Uptake among College Students Attending a Rural University in Alabama. Vaccine. 2021;39:7421–7428. doi: 10.1016/j.vaccine.2021.10.082. [DOI] [PubMed] [Google Scholar]
  • 36.Rimer B.K., Brewer N. Chapter 4: Introduction to Health Behavior Theories That Focus on Individuals. In: Glanz K., Rimer B.K., Viswanath K., editors. Health Behavior: Theory, Research, and Practice. Jossey-Bass & Pfeiffer Imprints, Wiley; San Francisco, CA, USA: 2015. Jossey-Bass Public Health. [Google Scholar]
  • 37.Francis D.B., Cates J.R., Wagner K.P.G., Zola T., Fitter J.E., Coyne-Beasley T. Communication Technologies to Improve HPV Vaccination Initiation and Completion: A Systematic Review. Patient Educ. Couns. 2017;100:1280–1286. doi: 10.1016/j.pec.2017.02.004. [DOI] [PubMed] [Google Scholar]
  • 38.Acampora A., Grossi A., Barbara A., Colamesta V., Causio F.A., Calabrò G.E., Boccia S., de Waure C. Increasing HPV Vaccination Uptake among Adolescents: A Systematic Review. Int. J. Environ. Res. Public Health. 2020;17:7997. doi: 10.3390/ijerph17217997. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Szilagyi P.G., Albertin C., Humiston S.G., Rand C.M., Schaffer S., Brill H., Stankaitis J., Yoo B.-K., Blumkin A., Stokley S. A Randomized Trial of the Effect of Centralized Reminder/Recall on Immunizations and Preventive Care Visits for Adolescents. Acad. Pediatr. 2013;13:204–213. doi: 10.1016/j.acap.2013.01.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Sullivan G.M., Feinn R. Using Effect Size—Or Why the P Value Is Not Enough. J. Grad. Med. Educ. 2012;4:279–282. doi: 10.4300/JGME-D-12-00156.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Szilagyi P.G., Humiston S.G., Gallivan S., Albertin C., Sandler M., Blumkin A. Effectiveness of a Citywide Patient Immunization Navigator Program on Improving Adolescent Immunizations and Preventive Care Visit Rates. Arch. Pediatr. Adolesc. Med. 2011;165:547–553. doi: 10.1001/archpediatrics.2011.73. [DOI] [PubMed] [Google Scholar]
  • 42.Petrosky E., Bocchini J.A., Hariri S., Chesson H., Curtis C.R., Saraiya M., Unger E.R., Markowitz L.E., Centers for Disease Control and Prevention (CDC) Use of 9-Valent Human Papillomavirus (HPV) Vaccine: Updated HPV Vaccination Recommendations of the Advisory Committee on Immunization Practices. MMWR Morb. Mortal. Wkly. Rep. 2015;64:300–304. [PMC free article] [PubMed] [Google Scholar]
  • 43.Chido-Amajuoyi O.G., Talluri R., Wonodi C., Shete S. Trends in HPV Vaccination Initiation and Completion Within Ages 9–12 Years: 2008–2018. Pediatrics. 2021;147:e2020012765. doi: 10.1542/peds.2020-012765. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Boersma P., Black L. Human Papillomavirus Vaccination among Adults, Aged 18−26, 2013−2018. NCHS; Hyattsville, MD, USA: 2020. [PubMed] [Google Scholar]
  • 45.TeenVaxView | Adolescent Vaccine Coverage Interactive Data | NIS | CDC. [(accessed on 9 February 2023)]; Available online: https://www.cdc.gov/vaccines/imz-managers/coverage/teenvaxview/data-reports/index.html.
  • 46.Health and Economic Benefits of Cervical Cancer Interventions | Power of Prevention. [(accessed on 17 March 2023)]; Available online: https://www.cdc.gov/chronicdisease/programs-impact/pop/cervical-cancer.htm.
  • 47.CDC HPV Vaccine Safety. [(accessed on 10 February 2023)]; Available online: https://www.cdc.gov/hpv/hcp/vaccine-safety-data.html.
  • 48.CDC VFC: Current CDC Vaccine Price List. [(accessed on 17 March 2023)]; Available online: https://www.cdc.gov/vaccines/programs/vfc/awardees/vaccine-management/price-list/index.html.

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Click here for additional data file. (82.4KB, zip)

Data Availability Statement

The data presented in this study are available on reasonable request from the corresponding author. The data are not publicly available due to containing protected health information.

Articles from Vaccines are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)

RESOURCES