Abstract
Background and objectives.
US health departments routinely conduct in-person quality improvement (QI) coaching to strengthen primary care clinics’ vaccine delivery systems, but this intervention achieves only small, inconsistent improvements in HPV vaccination. Thus, we sought to evaluate the effectiveness of combining QI coaching with remote provider communication training to improve impact.
Methods.
With health departments in 3 states, we conducted a pragmatic 4-arm cluster randomized clinical trial with 267 primary care clinics (76% pediatrics). We randomized clinics to receive in-person QI coaching, remote provider communication training, both interventions combined, or control. Using data from states’ immunization information systems, we prospectively assessed HPV vaccination among 176,189 patients, ages 11-17, who were unvaccinated at baseline. Our primary outcome was the proportion of those, ages 11-12, who had initiated HPV vaccination at 12-month follow-up.
Results.
HPV vaccine initiation was 1.5% points higher in the QI coaching arm and 3.8% points higher in the combined intervention arm than in the control arm, among patients ages 11-12, at 12-month follow-up (both p<0.001). These improvements persisted at 18-month follow-up. The combined intervention also achieved improvements for other age groups (ages 13-17) and vaccination outcomes (series completion). Remote communication training alone did not outperform the control on any outcome.
Conclusions.
Combining QI coaching with remote provider communication training yielded more consistent improvements in HPV vaccination uptake than QI coaching alone. Health departments and other organizations that seek to support primary care providers in HPV vaccine delivery may benefit from a higher intensity, multi-level intervention approach.
Table of Contents Summary
Results of a multi-state cluster randomized trial indicate that in-person QI coaching combined with remote provider communication training increases HPV vaccination coverage in primary care.
INTRODUCTION
US health departments are key partners in the national campaign to achieve widespread human papillomavirus (HPV) vaccination coverage, which could prevent the vast majority of the >35,000 HPV cancers diagnosed annually.1,2 Most notably, with programmatic support from the Centers for Disease Control and Prevention (CDC)’s Immunization Quality Improvement for Providers (IQIP) program, health departments routinely conduct in-person quality improvement (QI) coaching with primary care clinics to strengthen their vaccine delivery systems.3 This light-touch intervention typically includes vaccine-related education, as well as assessment and feedback on the clinic’s vaccination coverage. The available evidence suggests that this intervention is highly acceptable to providers and inexpensive to deliver,4,5 but may yield only small, inconsistent improvements in HPV vaccination coverage.6
One promising opportunity to enhance QI coaching is provider communication training. Health departments have typically implemented this intervention by hiring external physician consultants to deliver virtual trainings directly to providers, with an emphasis on improving their HPV vaccine recommendations. Although in-person provider communication training improves HPV vaccination coverage,7–9 this intervention has not to our knowledge been tested in the health department context or using remote delivery. Furthermore, its combined impact with QI coaching is unknown. Thus, we conducted a pragmatic cluster randomized clinical trial to evaluate the effectiveness of combining in-person QI coaching with remote provider communication training. We hypothesized that the interventions would increase HPV vaccine uptake among adolescent patients.
METHODS
Overview
We conducted a 4-arm pragmatic cluster randomized clinical trial, randomizing primary care clinics to receive in-person QI coaching, remote provider communication training, both interventions combined, or active control. Following randomization, we recruited clinics to each trial arm in a manner similar to Zelen’s design; the goal of this strategy, which is commonly used in cluster randomized clinical trials evaluating population-based interventions, was to reduce the administrative burden needed to join the trial so as to enroll a diverse sample of clinics that would represent those that health departments seek to serve in their real-world practice.10,11 We used data from states’ immunization information systems to prospectively assess HPV vaccine uptake among unvaccinated patients who were ages 11-17 at baseline. We compared each intervention arm to control, with our primary endpoint being series initiation among the 11- to 12-year-old cohort at 12-month follow-up.
Clinic selection, randomization, and recruitment
We partnered with three health departments to conduct our trial. Partners served a Midwestern state, a Southwestern state, and three large counties in a Northeastern state. We used states’ immunization information system to identify eligible clinics, defined as those that: 1) were pediatric or family medicine clinics; 2) participated in Vaccines for Children (VFC), a federally funded program that provides free vaccines; and 3) had 200-7,000 patients, ages 11-17. To promote balance among trial arms, we excluded clinics that were exceptionally large (i.e., >7,000 patients) or part of an exceptionally large system (>29 clinics). We also excluded clinics with high baseline coverage (≥85%) for HPV vaccine initiation. These criteria yielded 855 eligible clinics (Figure 1).
Figure 1.
Flow diagram
Within each state, we randomized eligible clinics using blocking and yoking. Some clinics were part of healthcare systems. Because providers working in these systems may deliver care at multiple clinics, we randomized clinics in the same system as a block to reduce the potential for contamination. We also yoked systems and clinics by size (i.e., number of patients ages 11-17) to ensure balance among trial arms. The trials’ statistician (MB) used an online random number generator to allocate single clinics and clinic blocks in a 1:1:1:1 ratio to each trial arm (ClinicalTrials.gov identifier: NCT03442062).
We recruited clinics after randomization from April to September 2018 until we reached the target of 90 clinics per arm or the end of the recruitment period. For the QI coaching arm, health departments recruited clinics by phone and email. For the communication training and control arms, the research team’s physician educators and other staff recruited clinics by phone and email. For the combined intervention arm, recruitment occurred sequentially, with QI coaching sessions scheduled first and communication trainings scheduled second. We made up to six attempts to contact clinics.
A total of 267 clinics enrolled in the trial (Figure 1). Among clinics that did not enroll, 281 did not respond, 140 declined, and 167 were not contacted because we met the trial arm quota (116 clinics), or for another reason, including the recruitment period ending (51 clinics). Unenrolled clinics were similar to enrolled clinics on baseline HPV vaccination coverage, but differed somewhat by state and patient load (Supplemental Table 1). The University of North Carolina Institutional Review Board determined that the study did not constitute human subjects research.
Procedures
We delivered interventions from May through September 2018. We have described intervention procedures previously,12 and summarize them briefly here:
QI coaching.
Clinics in the QI coaching arm received an in-person coaching session that health departments’ QI staff delivered to one or more members of the clinic’s healthcare team. We designed sessions to be similar to CDC’s AFIX (Assessment, Feedback, Incentives, and eXchange) program, now known as IQIP, but with a specific focus on HPV vaccination.12 Sessions lasted 66 minutes on average. First, QI staff used slides and talking points to deliver a brief presentation on HPV vaccination. Second, QI staff shared assessment and feedback results of the clinic’s adolescent vaccination coverage via an immunization report card; the report card presented coverage for HPV vaccine initiation, compared to coverage for other adolescent vaccines. Third, QI staff worked with clinics to develop an action plan, which involved setting a QI goal and selecting strategies (e.g., establishing standing orders) to meet the goal. Finally, QI staff sent follow-up report cards at 3- and 6-month follow-up so that clinics could track their progress. We offered participants an incentive of one hour of continuing medical education (CME) credit.
Communication training.
Clinics in the communication training arm received a single virtual session delivered by Zoom. Healthcare teams, with an emphasis on vaccine prescribers who recommend vaccines, were the intended audience. The research team’s trained physician educators delivered an adapted version of an evidence-based intervention called Announcement Approach Training (AAT).4,9 AAT teaches healthcare teams to use “presumptive announcements” to introduce adolescent vaccines in a way that presents vaccination as the default choice and then, if needed, to identify and address parents’ concerns. Trainings begin with a didactic session, followed by a role play activity on responding to common parent concerns. Originally designed as an in-person training, we adapted the AAT for remote delivery by, for example, using virtual breakout rooms to conduct role play activities. Didactic training lasted 45 minutes on average, and we offered participants one hour of CME for the full session.
Combined intervention.
Clinics in the combined intervention arm first received in-person QI coaching, followed by remote provider communication training.
Control.
Clinics in the control arm received didactic instruction about delivering the second dose of meningococcal conjugate vaccine. Research staff delivered the intervention by phone. The purpose of this “active” control was to minimize the selection bias that can occur with passive controls.
Across all four trial arms, staff who delivered interventions were not blinded to clinic allocation, due to the interactive nature of the interventions. Providers and other staff working in participating clinics were aware of the intervention they were receiving, but not other interventions. Data came from each state’s immunization information system. To avoid contamination across trial arms, health departments’ QI staff did not deliver other vaccine-related QI coaching or communication training to participating clinics, beyond what was provided for this trial, for the duration of the 18-month follow-up period.
Measures
Data came from each state’s immunization information system, which we queried on a rolling basis to prospectively assess the vaccination status of the cohort of patients, ages 11-17, who had not initiated HPV vaccination at baseline. We included patients in the analysis if they were attributed to a participating clinic, which typically meant they had received their last vaccine dose from that clinic.
Our primary outcome was the proportion of patients who had initiated the HPV vaccine series (≥1 doses) at 12-month follow-up, among those who were unvaccinated 11- to 12-year-olds at baseline. We prioritized 12-month follow-up to best control for the seasonality of adolescent vaccination.13 We also assessed other HPV vaccine outcomes (series completion), age groups (13-17 years), and time points (6- and 18-month follow-up).
Statistical analysis
Using an intent-to-treat approach, we analyzed patient-level intervention effects using generalized estimating equations for logistic outcomes. For our primary outcome, we modelled HPV vaccine initiation (0=no, 1=yes) among patients ages 11-12, comparing each intervention to the control between baseline and 12-month follow-up. The model accounted for the within-patient association in clinic networks by specifying a working correlation structure and included the population average effects for trial arm. The model also controlled for state, patient sex, intervention month, clinic’s baseline coverage for HPV vaccine initiation among ages 11-17, and the clinic network size (i.e., number of patients). For secondary outcomes, we repeated this analysis for other vaccination outcomes (HPV vaccine completion), age groups (13-17 years), and time points (6- and 18-month follow-up).
We conducted two sensitivity analyses to probe intervention effects. First, to assess geographic variation, we stratified our sample by state and re-ran our primary analysis. Second, we conducted exploratory analyses to probe the impact of the combined intervention because, unlike in the other trial arms, relatively few clinics in the combined intervention arm were adherent to the intervention as allocated. More specifically, although all clinics in the combined intervention arm received QI coaching, only a minority completed the subsequent step of provider communication training (Figure 1). In our sensitivity analysis, we dropped these non-adherent clinics and re-ran our analyses to examine the effect of the combined intervention on the subset of clinics that completed both interventions as allocated. All analyses were two-tailed with a critical alpha of .05 and conducted using SAS Version 9.4 (SAS Institute, Cary, NC).
RESULTS
Clinic and patient characteristics
About three-quarters (76%) of the 267 clinics enrolled in our trial had a pediatric focus (Table 1). Clinics were in the Southwest (38%), Northeast (36%), and Midwest (26%). At baseline, clinics had initiated HPV vaccination with about half (47%) of their 11- to 12-year-old patients and about two-thirds (69%) of their 13- to 17-year-old patients. Our analyses focused on the 98,682 patients, ages 11-12, and 77,507 patients, ages 13-17, who had not initiated HPV vaccination at baseline.
Table 1.
Clinic and patient characteristics
| QI Coaching (90 clinics) |
Communication Training (33 clinics) | Combined Intervention (85 clinics) | Control (59 clinics) |
|||||
|---|---|---|---|---|---|---|---|---|
| n | (%) | n | (%) | n | (%) | n | (%) | |
| Clinic characteristic | ||||||||
| Specialty | ||||||||
| Pediatric | 70 | (77.8) | 25 | (75.8) | 64 | (75.3) | 45 | (76.3) |
| Family practice | 20 | (22.2) | 8 | (24.2) | 21 | (24.7) | 14 | (23.7) |
| State | ||||||||
| Southwestern | 30 | (33.3) | 12 | (36.4) | 30 | (35.3) | 30 | (50.9) |
| Midwestern | 20 | (22.2) | 10 | (30.3) | 26 | (30.6) | 13 | (22.0) |
| Northeastern | 40 | (44.4) | 11 | (33.3) | 29 | (34.1) | 16 | (27.1) |
| Patient load, ages 11-12 | ||||||||
| ≤100 patients | 24 | (26.7) | 12 | (36.4) | 25 | (29.4) | 22 | (37.3) |
| 101-500 patients | 34 | (37.8) | 13 | (39.4) | 44 | (51.8) | 25 | (42.4) |
| >500 patients | 32 | (35.6) | 8 | (24.2) | 16 | (18.8) | 12 | (20.3) |
| Baseline HPV vaccine coverage, ages 11-12 | ||||||||
| Initiation, mean (SD) | 41.7 | (25.2) | 50.7 | (25.4) | 47.2 | (25.7) | 52.5 | (26.5) |
| Completion, mean (SD) | 13.9 | (9.6) | 15.1 | (7.9) | 14.8 | (8.7) | 15.7 | (8.2) |
| Baseline HPV vaccine coverage, ages 13-17 | ||||||||
| Initiation, mean (SD) | 66.0 | (12.9) | 71.1 | (12.1) | 70.2 | (15.0) | 72.1 | (13.4) |
| Completion, mean (SD) | 51.0 | (14.1) | 53.4 | (15.4) | 53.8 | (14.7) | 55.4 | (13.5) |
| Patient characteristics | ||||||||
| Sex | ||||||||
| Male | 38,063 | (53.4) | 11,119 | (52.6) | 26,880 | (53.2) | 17,535 | (52.8) |
| Female | 33,263 | (46.6) | 10,028 | (47.4) | 23,636 | (46.8) | 15,665 | (47.2) |
| Age | ||||||||
| 11-12 years | 40,299 | (56.5) | 11,824 | (55.9) | 27,956 | (55.3) | 18,603 | (56.0) |
| 13-17 years | 31,027 | (43.5) | 9,323 | (44.1) | 22,560 | (44.7) | 14,597 | (44.0) |
Note. HPV: human papillomavirus; SD: standard deviation.
Baseline coverage for HPV vaccine initiation among patients, ages 13-17, differed by trial arm (p=.04). Trial arms did not statistically differ on other clinic characteristics.
12-month outcomes
Ages 11-12.
For the primary trial outcome of HPV vaccine initiation among patients who were ages 11-12 at baseline, coverage changes were higher for the QI coaching and combined intervention arms than for the control arm at 12-month follow-up (1.5 and 3.8% point difference, respectively, both p<0.001, Table 2, Figure 2). The communication training arm did not outperform the control on the primary outcome. For the secondary outcome of HPV vaccine completion, we did not observe an intervention effect for any intervention compared to control.
Table 2.
Intervention effects on HPV vaccination at 12-month follow-up
| Ages 11-12 (n=98,682) |
Ages 13-17 (n=77,507) |
|||||||
|---|---|---|---|---|---|---|---|---|
| Coverage change at 12 months % points | Difference from control % points (95% CI) | p | Coverage change at 12 months % points | Difference from control % points (95% CI) | p | |||
| Series initiation | ||||||||
| Control (59 clinics) | 15.5 | (ref) | -- | 15.3 | (ref) | -- | ||
| QI Coaching (90 clinics) | 16.9 | 1.5 | (0.9, 2.1) | <0.001 | 14.9 | −0.4 | (−1.2, 0.3) | 0.28 |
| Communication Training (33 clinics) | 15.0 | −0.5 | (−1.2, 0.3) | 0.22 | 14.9 | −0.4 | (−1.4, 0.6) | 0.46 |
| Combined Intervention (85 clinics) | 19.3 | 3.8 | (3.1, 4.6) | <0.001 | 16.7 | 1.4 | (0.5, 2.3) | 0.002 |
| Series completion | ||||||||
| Control (59 clinics) | 3.7 | (ref) | -- | 4.0 | (ref) | -- | ||
| QI Coaching (90 clinics) | 3.7 | 0.0 | (−0.3, 0.3) | 0.84 | 4.3 | 0.4 | (−0.1, 0.8) | 0.12 |
| Communication Training (33 clinics) | 3.4 | −0.3 | (−0.7, 0.1) | 0.14 | 4.2 | 0.2 | (−0.4, 0.8) | 0.52 |
| Combined Intervention (85 clinics) | 3.7 | 0.0 | (−0.3, 0.3) | 0.99 | 5.1 | 1.1 | (0.5, 1.7) | <0.001 |
Note. HPV vaccine coverage change and difference from control account for clustering and are adjusted for clinic’s baseline HPV vaccine coverage, state, patients’ sex, month, and cluster size.
CI: confidence interval; QI: quality improvement.
Figure 2.
Coverage changes by trial arm for HPV vaccine initiation among patients ages 11-12. Bars show 95% confidence intervals.
Ages 13-17.
Among patients who were ages 13-17 at baseline, coverage change for the combined intervention was larger than control for HPV vaccine initiation and completion at 12-month follow-up (1.4 and 1.1% point difference, respectively, both p<.01, Table 2). We did not observe intervention effects for QI coaching or communication training for this age group.
Outcomes at other time points
6-month follow-up.
Coverage change for HPV vaccine initiation was higher for the combined intervention than the control among patients ages 11-12 at 6-month follow-up (2.2% point difference, p<0.001, Supplemental Table 2, Figure 2). Neither QI coaching nor communication training alone outperformed the control at 6 months.
18-month follow-up.
Coverage changes for HPV vaccine initiation were higher for the QI coaching and combined intervention arms than for control among patients ages 11-12 at 18-month follow-up (2.6 and 5.0% point difference, both p<.001, Supplemental Table 2, Figure 2). We did not observe an intervention effect for communication training alone.
Sensitivity analyses
State-stratified outcomes.
State-stratified analyses of our primary outcome indicated an intervention effect for QI coaching and the combined intervention in two of three states for HPV vaccine initiation among patients ages 11-12 at 12-month follow-up (Supplemental Table 3). More specifically, QI coaching achieved higher coverage changes in the Southwestern and Northeastern states, but lower coverage change in the Midwestern state, compared to control (all p>0.05). The combined intervention exhibited the same pattern of findings. Communication training did not outperform control in any state.
Combined intervention.
A sensitivity analysis of the subset of 17 clinics in the combined intervention arm that received both interventions indicated the same pattern of intervention effects as intent-to-treat analyses. Coverage change in this subset of clinics was larger than control for HPV vaccine initiation among patients ages 11-12 (8.2% point difference, p<0.001), as well as for HPV vaccine initiation and completion among patients ages 13-17 (2.0 and 1.2% point difference, both p<0.001). Consistent with the primary analysis, we did not observe an intervention effect for HPV vaccine completion, among patients ages 11-12.
DISCUSSION
This pragmatic cluster randomized trial found that combining in-person QI coaching with remote provider communication training yielded more consistent improvements in HPV vaccination uptake than QI coaching alone. In the combined intervention arm, we observed a 3.8% point advantage over control on our primary outcome of HPV vaccine initiation among patients ages 11-12 at 12-month follow-up, and this intervention effect persisted at 18 months. The combined intervention also yielded small intervention effects for other age groups and vaccination outcomes (ages 13-17, series initiation and completion). In contrast, the QI coaching intervention improved HPV vaccine initiation by 1.5% points over control at 12-month follow-up. Although we still observed this intervention effect at 18 months, we did not observe an effect for other outcomes. The remote provider communication training alone did not improve HPV vaccination on any outcome.
The relatively consistent performance of the combined intervention suggests that health departments and other organizations that deliver vaccine-related QI coaching may benefit from a higher intensity, multi-level approach. Combining the systems-level focus of QI coaching with the interpersonal focus of communication training may be especially powerful in the case of HPV vaccination, given that infrequent and low-quality recommendations are more of a barrier for HPV vaccine than other adolescent vaccines.14–17 Perhaps for this reason, several prior multi-level interventions have proven effective for increasing HPV vaccination coverage.8,18–20 At the same time, such interventions require more time, are more expensive, and may be adopted by fewer clinics. Indeed, in the present trial relatively few clinics randomized to the combined intervention arm completed both QI coaching and communication training. Findings from our previously published process evaluation suggest that the combined intervention required substantially more time than either intervention component alone, primarily due to scheduling challenges.12 Such drawbacks constitute a real concern to health departments, given their budget constraints and mandate to serve all VFC-participating clinics.
Compared to the combined intervention, the impact of QI coaching alone was more limited. The intervention’s small improvement among younger, but not older adolescents was consistent with what we have observed previously.6 Given the low delivery cost and high acceptability of QI coaching,4,5 it may still have value for increasing HPV vaccination coverage if advantages continue to accumulate over time, with subsequent coaching sessions, and across other vaccine types. Importantly, recent updates to the CDC’s QI coaching program have included increasing the frequency with which QI staff meet with participating providers, which may increase effectiveness.
We were surprised that, in the absence of QI coaching, remote provider communication training did not improve HPV vaccine uptake. As previously reported, trainings were relatively well attended, attracting a median of 5 participants, including 2 vaccine prescribers, per clinic.12 In contrast, QI coaching typically involved just 2 participants per clinic and rarely attracted vaccine prescribers. Given the effectiveness of in-person provider communication training,7,8 the disappointing performance of remote training may have been due to lower engagement with virtual delivery. The one-time nature of the training may have also contributed to low impact, in contrast to QI coaching which employed two follow-up contacts. Whatever the case, remote communication training deserves further development to overcome barriers and capitalize on its potential to extend training to more clinics, including those in hard-to-reach areas.
Our trial used a randomized design to provide novel data on the effectiveness of two interventions, QI coaching and communication training, commonly used to increase HPV vaccine uptake. It is noteworthy that primary care clinics in our large, multi-state sample were similar to the U.S. as a whole on coverage for HPV vaccine initiation at baseline (69% versus 68% among ages 13-17).21 This correspondence lends support for the generalizability of our findings, although generalizability may be more limited for small clinics or patient age groups (e.g., ages 9-10) not included in our trial. Another limitation is differential recruitment and adoption of interventions by trial arm, which we explored in more depth in our process evaluation; these differences may have introduced selection effects despite our use of a randomized trial design and intent-to-treat analytic approach.12 Finally, our trial relied on states’ immunization information systems for HPV vaccination data. These systems derive vaccination data from provider reports, but are limited in their ability to provide data on patient characteristics, such as race/ethnicity and insurance status, or on healthcare use for office visits that did not involve vaccinations. Future studies using electronic health records to restrict analyses to patients with office visits during the trial period may provide more precise, if less generalizable, estimates of interventions’ impact.
CONCLUSIONS
Health departments are important partners in immunization QI because they offer an existing national workforce to conduct QI coaching and other light touch interventions in primary care. Findings of our trial indicate that supplementing in-person QI coaching with remote provider communication training offers an opportunity for health departments to improve HPV vaccination coverage among adolescents. Our experience suggests that only a subset of clinics will adopt this higher intensity intervention, but those that do may achieve HPV vaccination coverage improvements across multiple vaccination outcomes. These goals are critically important for increasing HPV vaccination coverage and protecting adolescents from HPV cancers.
Supplementary Material
What’s Known on This Subject
US health departments routinely conduct in-person QI coaching to improve primary care clinics’ vaccine delivery systems. Prior research suggests this light touch intervention yields small, inconsistent improvements in HPV vaccination coverage. Higher intensity interventions may be needed.
What This Study Adds
We conducted a cluster randomized trial with 267 primary care clinics to evaluate the impact of supplementing health departments’ QI coaching with remote provider communication training. The combined intervention improved HPV vaccine initiation for multiple time points and age groups.
Funding/Support:
This study was funded by a cooperative agreement with the Centers for Disease Control and Prevention (U01IP001073-02). Dr. Grabert’s time was supported by a training grant from the National Cancer Institute (T32CA057726-28).
Role of Funder/Sponsor (if any):
Dr. Bjork, our CDC co-author, contributed to study design; the collection, analysis, and interpretation of data; the writing of the report; and the decision to submit the manuscript for publication. NCI played no role in study design, collection, analysis and interpretation of the data, in writing the report, or in the decision to submit the article for publication.
Abbreviations:
- AAT
Announcement Approach Training
- AFIX
Assessment, Feedback, Incentives, and eXchange
- CDC
Centers for Disease Control and Prevention
- CI
confidence interval
- CME
continuing medical education
- HPV
human papillomavirus
- QI
quality improvement
- VFC
Vaccines for Children
Footnotes
Conflict of Interest Disclosures (includes financial disclosures): N.T. Brewer has served as a paid advisor for Merck, CDC and WHO. The remaining authors disclosed no conflicts of interest.
Clinical Trial Registration: NCT03442062
https://clinicaltrials.gov/ct2/show/NCT03442062?term=NCT03442062&draw=2&rank=1
Data Sharing Statement: De-identified individual participant data will not be made available. The data belong to the states involved in the trial.
REFERENCES
- 1.U.S. Centers for Disease Control and Prevention. Cancers Associated with Human Papillomavirus, United States—2013–2017. Atlanta, GA; 2020. https://www.cdc.gov/cancer/uscs/about/data-briefs/no18-hpv-assoc-cancers-UnitedStates-2013-2017.htm. Accessed March 31, 2021. [Google Scholar]
- 2.Huh WK, Joura EA, Giuliano AR, et al. Final efficacy, immunogenicity, and safety analyses of a nine-valent human papillomavirus vaccine in women aged 16–26 years: a randomised, double-blind trial. Lancet. 2017;390(10108):2143–2159. doi: 10.1016/S0140-6736(17)31821-4 [DOI] [PubMed] [Google Scholar]
- 3.Centers for Disease Control and Prevention. (IQIP) Immunization Quality Improvement for Providers. https://www.cdc.gov/vaccines/programs/iqip/index.html. Accessed March 31, 2021.
- 4.Gilkey MB, Moss JL, Roberts AJ, Dayton AM, Grimshaw AH, Brewer NT. Comparing in-person and webinar delivery of an immunization quality improvement program: A process evaluation of the adolescent AFIX trial. Implement Sci. 2014;9(21). doi: 10.1186/1748-5908-9-21 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Calo WA, Gilkey MB, Leeman J, et al. Coaching primary care clinics for HPV vaccination quality improvement: Comparing in-person and webinar implementation. Transl Behav Med. 2019;9(1):23–31. doi: 10.1093/tbm/iby008 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Gilkey MB, Dayton AM, Moss JL, et al. Increasing provision of adolescent vaccines in primary care: A randomized controlled trial. Pediatrics. 2014;134(2):e346–353. doi: 10.1542/peds.2013-4257 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Brewer NT, Hall ME, Malo TL, Gilkey MB, Quinn B, Lathren C. Announcements versus conversations to improve HPV vaccination coverage: A randomized trial. Pediatrics. 2017;139(1). doi: 10.1542/peds.2016-1764 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Dempsey AF, Pyrznawoski J, Lockhart S, et al. Effect of a health care professional communication training intervention on adolescent human papillomavirus vaccination a cluster randomized clinical trial. JAMA Pediatr. 2018;172(5):e180016. doi: 10.1001/jamapediatrics.2018.0016 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.National Cancer Institute. Making Effective HPV Vaccine Recommendations | Evidence-Based Cancer Control Programs (EBCCP); 2020. https://ebccp.cancercontrol.cancer.gov/programDetails.do?programId=26926144. Accessed March 31, 2021. [Google Scholar]
- 10.Adamson J, Cockayne S, Puffer S, Torgerson DJ. Review of randomised trials using the post-randomised consent (Zelen’s) design. Contemp Clin Trials. 2006;27(4):305–319. doi: 10.1016/J.CCT.2005.11.003 [DOI] [PubMed] [Google Scholar]
- 11.Ford I, Norrie J. Pragmatic Trials. N Engl J Med. 2016;375(5):454–463. doi: 10.1056/NEJMRA1510059 [DOI] [PubMed] [Google Scholar]
- 12.Grabert BK, Kurtzman R, Heisler-MacKinnon J, et al. Implementation of quality improvement coaching versus physician communication training for improving human papillomavirus vaccination in primary care: A randomized implementation trial. Transl Behav Med. In press. doi: 10.1093/TBM/IBAB071 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Moss JL, Reiter PL, Rimer BK, Ribisl KM, Brewer NT. Summer peaks in uptake of human papillomavirus and other adolescent vaccines in the United States. Cancer Epidemiol Biomarkers Prev. 2016;25(2):274–281. doi: 10.1158/1055-9965.EPI-15-0574 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Gilkey MB, McRee AL. Provider communication about HPV vaccination: A systematic review. Hum Vaccines Immunother. 2016;12(6):1454–1468. doi: 10.1080/21645515.2015.1129090 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Gilkey MB, Moss JL, Coyne-Beasley T, Hall ME, Shah PD, Brewer NT. Physician communication about adolescent vaccination: How is human papillomavirus vaccine different? Prev Med. 2015;77:181–185. doi: 10.1016/j.ypmed.2015.05.024 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Perkins RB, Clark JA, Apte G, et al. Missed opportunities for HPV vaccination in adolescent girls: A qualitative study. Pediatrics. 2014. doi: 10.1542/peds.2014-0442 [DOI] [PubMed] [Google Scholar]
- 17.Finney Rutten LJ, St. Sauver JL, Beebe TJ, et al. Association of both consistency and strength of self-reported clinician recommendation for HPV vaccination and HPV vaccine uptake among 11- to 12-year-old children. Vaccine. 2017;35(45):6122–6128. doi: 10.1016/j.vaccine.2017.09.056 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Perkins RB, Legler A, Jansen E, et al. Improving HPV vaccination rates: A stepped-wedge randomized trial. Pediatrics. 2014;134(3):e666–674. doi: 10.1542/peds.2019-2737 [DOI] [PubMed] [Google Scholar]
- 19.Rodriguez SA, Mullen PD, Lopez DM, Savas LS, Fernández ME. Factors associated with adolescent HPV vaccination in the U.S.: A systematic review of reviews and multilevel framework to inform intervention development. Prev Med. 2020;131:105968. 2020;131. doi: 10.1016/j.ypmed.2019.105968 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Fiks AG, Grundmeier RW, Mayne S, et al. Effectiveness of decision support for families, clinicians, or both on HPV vaccine receipt. Pediatrics. 2013;131(6):1114–1124. doi: 10.1542/peds.2012-3122 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Walker TY, Elam-Evans LD, Yankey D, et al. National, regional, state, and selected local area vaccination coverage among adolescents aged 13-17 years - United States, 2018. Morb Mortal Wkly Rep. 2019;68(33):718–723. doi: 10.15585/MMWR.MM6833A2 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.