Effect of Training Pediatric Clinicians in Human Papillomavirus Communication Strategies on Human Papillomavirus Vaccination Rates

Key Points

Question

Can training pediatric clinicians in human papillomavirus (HPV) communication strategies using online modules reduce missed opportunities for HPV vaccination and increase rates of HPV vaccination?

Findings

In this cluster randomized clinical trial involving 48 pediatric practices across 19 states, online communication training reduced missed opportunities for HPV vaccination, particularly by improving receipt of the first HPV vaccine dose.

Meaning

Online training of pediatric clinicians focused on effective communication may increase rates of HPV vaccination.

Abstract

Importance

Missed opportunities for human papillomavirus (HPV) vaccination during pediatric health care visits are common.

Objectives

To evaluate the effect of online communication training for clinicians on missed opportunities for HPV vaccination rates overall and at well-child care (WCC) visits and visits for acute or chronic illness (hereafter referred to as acute or chronic visits) and on adolescent HPV vaccination rates.

Design, Setting, and Participants

From December 26, 2018, to July 30, 2019, a longitudinal cluster randomized clinical trial allocated practices to communication training vs standard of care in staggered 6-month periods. A total of 48 primary care pediatric practices in 19 states were recruited from the American Academy of Pediatrics Pediatric Research in Office Settings network. Participants were clinicians in intervention practices. Outcomes were evaluated for all 11- to 17-year-old adolescents attending 24 intervention practices (188 clinicians) and 24 control practices (177 clinicians). Analyses were as randomized and performed on an intent-to-treat basis, accounting for clustering by practice.

Interventions

Three sequential online educational modules were developed to help participating clinicians communicate with parents about the HPV vaccine. Weekly text messages were sent to participating clinicians to reinforce learning. Statisticians were blinded to group assignment.

Main Outcomes and Measures

Main outcomes were missed opportunities for HPV vaccination overall and for HPV vaccine initiation and subsequent doses at WCC and acute or chronic visits (visit-level outcome). Secondary outcomes were HPV vaccination rates (person-level outcome). Outcomes were compared during the intervention vs baseline.

Results

Altogether, 122 of 188 clinicians in intervention practices participated; of these, 120, 119, and 116 clinicians completed training modules 1, 2, and 3, respectively. During the intervention period, 29 206 adolescents (14 664 girls [50.2%]; mean [SD] age, 14.2 [2.0] years) made 15 888 WCC and 28 123 acute or chronic visits to intervention practices; 33 914 adolescents (17 069 girls [50.3%]; mean [SD] age, 14.2 [2.0] years) made 17 910 WCC and 35 281 acute or chronic visits to control practices. Intervention practices reduced missed opportunities overall by 2.4 percentage points (−2.4%; 95% CI, −3.5% to −1.2%) more than controls. Intervention practices reduced missed opportunities for vaccine initiation during WCC visits by 6.8 percentage points (−6.8%; 95% CI, −9.7% to −3.9%) more than controls. The intervention had no effect on missed opportunities for subsequent doses of the HPV vaccine or at acute or chronic visits. Adolescents in intervention practices had a 3.4-percentage point (95% CI, 0.6%-6.2%) greater improvement in HPV vaccine initiation compared with adolescents in control practices.

Conclusions and Relevance

This scalable, online communication training increased HPV vaccination, particularly HPV vaccine initiation at WCC visits. Results support dissemination of this intervention.

Trial Registration

ClinicalTrials.gov Identifier: NCT03599557

This cluster randomized clinical trial evaluates the effect of online communication training for clinicians on missed opportunities for HPV vaccination rates overall and at well-child care visits and visits for acute or chronic illness and on adolescent HPV vaccination rates.

Introduction

Each year in the US, human papillomavirus (HPV) causes 33 700 new cases of cervical, oropharyngeal, anal, vulvar, vaginal, and penile cancers; more than 4000 deaths; and more than $4 billion in cancer-related medical costs.^{1,2,3,4,5,6,7} Despite a safe and effective vaccine,^6,8,9 national goals of 80% HPV vaccine coverage for adolescents aged 13 to 17 years,¹⁰ and recommendations to start vaccinating adolescents aged 11 and 12 years¹¹ (or after 9 years of age¹²), in 2019 only 54% of US adolescents aged 13 to 17 years were up-to-date, and only 71% received at least 1 HPV vaccination.¹³

Observational studies have demonstrated that an effective recommendation by health care professionals is associated with higher HPV vaccination rates.^14,15 Clinicians who normalize HPV vaccination by treating it like other adolescent vaccines in discussions with parents and adolescents have adolescents with higher vaccine uptake.^6,16 However, parental hesitancy for the HPV vaccine is associated with vaccination refusal or delay.^{14,16,17,18,19} Experts call for strategies to optimize clinician communication.^6,8,16,20,21

Studies using in-person communication training demonstrated promising findings. Evaluating a communication intervention,^22,23 Brewer et al²⁴ sent a physician educator to local practices to train clinicians to use announcements about HPV vaccinations. Dempsey et al²⁵ used multicomponent communication training interventions that included both a self-guided webinar and 2 in-person sessions at practices. Both studies noted modest increases in HPV vaccination rates.^25,26,27

A challenge for these interventions is widespread scalability because they require in-person training.⁸ In contrast, online asynchronous training is scalable and widely used by clinicians, but it awaits investigation to train clinicians on HPV communication skills.^28,29 A second challenge is tailoring content to the type of clinician (ie, physicians, nurses, and medical assistants). A third challenge is ensuring that busy clinicians complete the training. Concise, salient, content materials and Maintenance of Certification credits can increase fidelity with training.^30,31 A fourth challenge is reinforcement of principles or skills to prevent regression of learned skills.³²

We developed and tested, within pediatric primary care practices across the US, an asynchronous, online communication training intervention for primary care pediatric clinicians to improve their communication skills regarding HPV vaccination. Using a cluster randomized clinical trial (RCT), we evaluated the effect of this communication intervention on (1) missed opportunities for HPV vaccination overall and at well-child care (WCC) visits or visits for acute or chronic illness (hereafter referred to as acute or chronic visits) and (2) adolescent receipt of initial and subsequent HPV vaccinations (person-level outcome) among 11- to 17-year-old adolescents. We hypothesized that the communication intervention would reduce MOs and increase HPV vaccination rates, overall and particularly at WCC visits where most HPV vaccinations are administered,³³ and for vaccine initiation, where clinician communication can be pivotal. Secondary study objectives were to evaluate uptake and completion of the online communication training.

Methods

Study Design and Setting

From December 26, 2018, to July 30, 2019, we conducted a 2-group, longitudinal, parallel cluster RCT of 48 primary care pediatric practices to evaluate HPV communication training vs standard of care. We evaluated missed vaccination opportunities, receipt of initial and subsequent adolescent HPV vaccine doses at WCC visits and acute or chronic visits, and secondary metrics during a 12-month baseline period followed by a 6-month intervention period. This study was approved by institutional review boards at the University of California, Los Angeles and the American Academy of Pediatrics and was determined to be exempt by the institutional review board of the Children’s Hospital of Philadelphia (CHOP) per 45 CFR §46.104(d) 2 because CHOP received limited data sets, no CHOP patients were involved, and CHOP investigators surveyed physicians and not patients (trial protocol in Supplement 1). Parental consent was waived by the University of California, Los Angeles and the American Academy of Pediatrics because the data were deidentified at the patient level and the research involves no more than minimal risk to the participants. We recruited primary care pediatric practices via the American Academy of Pediatrics Pediatric Research in Office Settings network, an established pediatric primary care practice–based research network in the US.³⁴

Selection of Practices and Adolescents

Inclusion Criteria

We recruited practices that used Physician’s Computer Company or Office Practicum as their electronic health record (EHR) vendor (83 practices approached), plus practices in 2 health systems (Austin Regional Clinic in Texas and Children’s Minnesota) that could extract EHR data (30 practices approached). Practice inclusion criteria were (1) EHR in place for 1 year or more and no plan to change, (2) no plan to participate in other HPV studies or quality improvement interventions during the study, (3) less than 20% of the practice’s adolescents were believed to have received HPV vaccinations from other settings, and (4) 50% or more of practice clinicians agreed to participate in communication training (Figure).

Figure. CONSORT Diagram Plus Practice and Adolescent Eligibility Criteria.

Eligibility criteria for clinicians: worked in a participating practice and had at least 10 visits at the practice with an adolescent aged 11.0 to 17.9 years during the study period. Eligibility criteria for adolescents: aged 11.0 to 17.9 years on the day of the office visit. Adolescents were not recruited for the study. All adolescents meeting eligibility criteria at practices were included in analyses. The number of adolescents may be greater than the number of visits because some adolescents may not have had a human papillomavirus vaccine–eligible visit.

Recruitment of Practices

We distributed recruitment materials to practices using the EHR vendors via electronic communications and at vendor-sponsored conferences. Physician contacts at the health systems distributed recruitment materials to practices. Practices identified a lead clinician who communicated with other clinicians.

Recruitment of Clinicians for Communication Training

We offered HPV communication training to all clinicians (physicians, nurse practictioners, and physician assistants) at intervention practices; clinicians who agreed to the training (hereafter referred to as participating clinicians) provided written informed consent and demographic information prior to practice randomization.

This trial was the first in a longitudinal set of RCTs with sequential interventions. Clinicians signed up for Maintenance of Certification for activities in future studies; there was no Maintenance of Certification activity or credit for this RCT. Practices were compensated according to number of participating pediatricians (mean [SD] financial compensation, $200 [$88]; range, $120-$420).

Inclusion of Adolescents and Randomized Allocation of Practices

All adolescents 11 to 17 years of age who made a visit to the practice were included in the primary analysis regardless of whether visits were with participating clinicians. Using restricted randomization, we balanced the 2 study groups of 24 practices per group^35,36 for 3 factors: source of EHR data (2 EHR systems and 2 health systems), HPV vaccination rates the year before the study, and number of adolescent WCC visits during the prior 2 years. We randomized participating practices in 2 waves (23 and 25 practices) with intervention starting times of December 2018 and February 2019. To implement randomized allocation for each wave, we generated and ranked more than 200 000 possible allocations and randomly selected from the top 10 most balanced allocations using a customized program in the R software package, version 3.6.1 (R Group for Statistical Computing).³⁷ Study statisticians remained blinded to practice assignment until all analyses were complete (eAppendices 1, 2, and 3 in Supplement 2).

Intervention

The intervention consisted of 3 online modules viewable using a computer, tablet, or smartphone, plus weekly “Quick Tips” sent to participating clinicians. The online modules were based on prior studies of practice improvement that had used group conference calls plus webinars.³⁸ The 20- to 30-minute modules used Articulate Rise,³⁹ a readily available software program that allows for lesson development without programing expertise. Clinicians accessed modules via password-protected learning management systems⁴⁰ and were aware that the system tracked module completion; they could pause or resume modules if desired. The modules (Table 1) described HPV disease burden, how to give an effective HPV recommendation using a “same way same day approach” to introduce HPV vaccination the same way that clincians typically introduce other vaccines,⁴¹ how to answer common questions by parents or adolescents, strategies to engage office staff in communication about the HPV vaccine, and how to address parental hesitancy. Participating clinicians were reminded up to 11 times to complete all 3 modules. We invited nursing and medical assistant office staff to complete module 1 and asked clinicians to meet with practice staff at least once to discuss practice changes to support effective communication strategies.

Table 1. Content of the HPV Communication Training Modules.

Module	Contenta
Module 1: an effective approach	Burden of HPV cancers; importance of HPV vaccine; giving an effective recommendation (what to do [same way, same day], bundling HPV vaccine with other vaccines, what not to do); answering common questions; summary; and resources for the practice
Module 2: your confident recommendation	Review of module 1; involving the office staff (their messages, involving entire team, how office staff should answer questions); answering more common questions (safety, sexuality, others); summary; and resources for the practice
Module 3: talking with parents who hesitate	4 Common groups of parents (with respect to HPV vaccine and hesitancy); common scenarios of hesitant parents; using motivational interviewing strategies; supporting yourself and your colleagues; summary; and resources for the practice

Abbreviation: HPV, human papillomavirus.

^a Many components of the modules were interactive.

Weekly Quick Tips (eAppendix 1 in Supplement 2), designed to reinforce module content, were programmed into REDCap (Research Electronic Data Capture; Vanderbilt University)⁴² and sent automatically to participating clinicians (not nurses) by text message or, if preferred, by email.

Control practices received standard of care (ie, no intervention from the research team except for 1 brief communication during the intervention period reminding lead practice clinicians that they were in the control group).

Outcome Measures

Missed Opportunities for HPV Vaccination (Primary Outcome)

We defined a missed HPV vaccination opportunity as a visit at which an HPV vaccine was due⁴³ and not administered. We measured MOs and HPV vaccination rates using EHR data. Reasons for MOs were not routinely available from the EHR.

The primary analysis referred to visits by HPV vaccine–eligible adolescents aged 11.0 to 17.9 years to any clinician in the practice (visit-level outcome). We assessed outcomes overall and separately by visit (WCC or acute or chronic) and vaccine dose (initial or subsequent). A secondary analysis was limited to visits by HPV vaccine–eligible adolescents to participating clinicians.

HPV Vaccinations

We assessed dates of initial and subsequent HPV vaccinations (person-level outcome), using 2018 Advisory Committee on Immunization Practices criteria⁴³ to determine vaccine eligibility. We stratified by age (11-12 years and 13-17 years) because the modules emphasized vaccination of 11- to 12-year-olds, Healthcare Effectiveness Data and Information Set quality measures reflect vaccination by age 13 years,⁴⁴ and national data reflect HPV vaccination among 13- to 17-year-olds.⁴⁵

Adolescent Visits to the Practice

We extracted from EHRs the following data for each visit during the 1-year baseline and 6-month intervention periods: adolescent characteristics (birthdate, sex, race, and ethnicity), visit date, Evaluation and Management Current Procedural Terminology codes to identify WCC or acute or chronic visits, and initial or subsequent HPV vaccination. For this clinician-focused study, we excluded nurse-only visits (3.9% [5955 of 152 702] of HPV vaccine–eligible visits).

Survey of Participating Clinicians

At the end of the intervention we surveyed participating clinicians (using the RE-AIM [Reach, Effectiveness, Adoption, Implementation, and Maintenance] framework)^46,47 in the intervention group about their use of communication skills, confidence in communication about the HPV vaccine, and time spent discussing HPV vaccination with parents (instrument available on request).

Statistical Analysis

Missed Opportunities

We performed visit-level analyses among 2 sets of visits defined before randomization: (1) visits to all the clinicians in the study practice, excluding nurse-only visits, regardless of clinician participation (ie, intent to treat at the practice level) and (2) visits to participating clinicians. Analyses of missed opportunities compared the 6-month intervention period with the 12-month baseline period. We fit logistic models for missed opportunities using generalized estimating equations with clustering at the practice level and an independence working correlation to account for correlation of outcomes within a practice. Models included randomization group, period (baseline or intervention), and the 2-way group-period interaction. The clustered RCT resulted in a modest design effect of approximately 2, which is reflected in all reported 95% CIs. The study had more than 90% power to detect an excess improvement of 5 percentage points in the intervention group. The primary hypothesis was that the intervention would reduce missed opportunities for HPV vaccination, particularly at WCC visits for HPV vaccine initiation, when effective clinician communication is especially important to vaccine receipt.

Initial and Subsequent HPV Vaccinations

We conducted person-level analyses by determining initial and subsequent HPV vaccinations by the end of the period among HPV vaccine–eligible adolescents who made a visit with any clinician at a practice during the baseline and intervention periods. To synchronize the seasonality and duration of observation between the baseline and intervention periods, we compared HPV vaccinations in the 6-month intervention period with HPV vaccinations in the same 6 months of the baseline period 1 year earlier. We deliberately made no formal adjustment for multiple comparisons because the dose-based and WCC-based subgroup analyses and contrasts were preplanned.

We used logistic regression models of vaccination status at the end of the period, fit with generalized estimating equations with clustering at the practice level, to estimate differences in changes in receipt of initial or subsequent HPV vaccinations over time by study group. We used the R, version 3.6.1 software package for data management and Stata, version 16 software for primary analyses (StataCorp LLC). Hypothesis tests were 2-sided, and the type I error rate was set to 0.05.

Results

Altogether 48 practices were randomized (Table 2), and 234 of 365 clinicians (64.1%) consented to participate; participating clinicians performed 68.6% of office visits (73 277 of 106 841) in the practice during the baseline period (4 left the practice during the intervention period). Among the 122 of 188 participating clinicians in intervention practices, 120, 119, and 116 clinicians completed modules 1, 2, and 3, respectively (116 of 122 completed all 3 modules). The intervention and control groups were similar with respect to baseline characteristics of adolescents (total number, race, ethnicity, and receipt of initial or subsequent HPV vaccinations, although 20.0% [20 871 of 104 438] were missing race designations) and visits (total number of visits, WCC visits, acute or chronic visits). During the intervention period, 29 206 adolescents (14 664 girls [50.2%] and 14 542 boys [49.8%]; mean [SD] age, 14.2 [2.0] years) made 15 888 WCC visits and 28 123 acute or chronic visits to intervention practices; 33 914 adolescents (17 069 girls [50.3%] and 16 845 boys [49.7%]; mean [SD] age, 14.2 [2.0] years) made 17 910 WCC visits and 35 281 acute or chronic visits to control practices. Across all practices, 43 655 of 106 841 visits by 11- to 12-year-old adolescents (40.9%) and 36 335 of 106 841 visits (34.0%) by 13- to 17-year-old adolescents represented opportunities for HPV vaccine initiation. A total of 15 840 of 16 572 initial HPV vaccinations (95.6%) and 10 562 of 12 442 subsequent HPV vaccinations (84.9%) at baseline occurred at WCC visits.

Table 2. Baseline Characteristics (for 12 Months) of Practices, Clinicians, Adolescents, and Visits^a.

Characteristic	No. (%)
	Overall	Intervention group	Control group
Practices, No.	48	24	24
Clinicians
No.	365	188	177
No. participating	234 (64.1)	122 (64.9)	112 (63.3)
Adolescents aged 11-17 y
No.	104 438	50 569	53 869
Sex
Male	52 578 (50.3)	25 472 (50.4)	27 106 (50.3)
Female	51 855 (49.7)	25 093 (49.6)	26 762 (49.7)
Missing data	5 (0.005)	4 (0.008)	1 (0.002)
Raceb
White	67 112 (64.3)	36 054 (71.3)	31 058 (57.7)
African American	10 042 (9.6)	4962 (9.8)	5080 (9.4)
Asian or Pacific Islander	4635 (4.4)	2734 (5.4)	1901 (3.5)
American Indian/Alaska native	677 (0.6)	229 (0.5)	448 (0.8)
Multiracial group	700 (0.7)	348 (0.7)	352 (0.7)
Otherc	401 (0.4)	178 (0.4)	223 (0.4)
Missing data	20 871 (20.0)	6064 (12.0)	14 807 (27.5)
Ethnicityb
Non-Hispanic	67 724 (64.8)	34 323 (67.9)	33 401 (62.0)
Hispanic	18 214 (17.4)	11 204 (22.2)	7010 (13.0)
Missing data	18 500 (17.7)	5042 (10.0)	13 458 (25.0)
HPV vaccination receipt as of last day of baseline periodd
Initiated series at 11-12 y	17 074 (59.7)	8551 (61.3)	8523 (58.3)
Completed series at 11-12 y	6079 (21.3)	3105 (22.3)	2974 (20.3)
Initiated series at 13-17 y	59 104 (77.9)	28 251 (77.2)	30 853 (78.6)
Completed series at 13-17 y	49 163 (64.8)	23 314 (63.7)	25 849 (65.9)
Visits at which adolescent was eligible for vaccine
No. of visits	106 841	51 339	55 502
Visit type
Well-child care	48 211 (45.1)	24 080 (46.9)	24 131 (43.5)
Acute or chronic	58 630 (54.9)	27 259 (53.1)	31 371 (56.5)
Age at visit
11-12 y	53 640 (50.2)	25 255 (49.2)	28 385 (51.1)
13-17 y	53 201 (49.8)	26 084 (50.8)	27 117 (48.9)
HPV dose eligible
Initial dose at all ages	79 990 (74.9)	38 073 (74.2)	41 917 (75.5)
Subsequent dose at all ages	26 851 (25.1)	13 266 (25.8)	13 585 (24.5)
Initial dose at 11-12 y	43 655 (40.9)	20 350 (39.6)	23 305 (42.0)
Subsequent dose at 11-12 y	9985 (9.3)	4905 (9.6)	5080 (9.2)
Initial dose at 13-17 y	36 335 (34.0)	17 723 (34.5)	18 612 (33.5)
Subsequent dose at 13-17 y	16 866 (15.8)	8361 (16.3)	8505 (15.3)

Abbreviation: HPV, human papillomavirus.

^a Baseline periods were wave 1: October 31, 2017, to October 31, 2018, and wave 2: December 19, 2017, to December 19, 2018.

^b Data on race/ethnicity are missing in many cases, and more often among control practices, owing largely to 5 control practices that were missing race/ethnicity data for at least half their adolescents.

^c Other races are unknown.

^d The denominators include all adolescents who had a visit during the 12-month period, regardless of whether they were eligible for an HPV vaccine during these 12 months. Many adolescents had been vaccinated prior to this 12-month period.

Missed Opportunities for HPV Vaccine Overall (All Visits and Doses Combined)

The rate of missed opportunities decreased from baseline to the intervention period by 2.4 percentage points (95% CI, 1.2%-3.5%) more in intervention practices than in control practices (eAppendix 2 in Supplement 2). In intervention practices, the rate decreased from 73.7 percentage points (37 063 of 51 339) to 70.7 percentage points (14 932 of 21 033; difference, 3.0%), and in control practices, the rate decreased from 72.8 percentage points (40 764 of 55 502) to 72.2 percentage points (18 343 of 24 829; difference, 0.6%), for a difference between study groups of −2.4% (95% CI, −3.5% to −1.2%).

Missed Opportunities for HPV Vaccine Initiation at WCC Visits

The rate of missed opportunities decreased in intervention practices more than in control practices by 6.8 percentage points (95% CI, 3.9%-9.7%) for HPV vaccine initiation at WCC visits (Table 3; eAppendix 2 in Supplement 2). Among the 24 intervention practices, 22 improved in the intervention period, 1 remained unchanged, and 1 regressed; 18 of 24 control practices also improved but, on average, to a much smaller degree. Adjustment for age and sex did not alter the finding (adjusted difference in reduction in missed opportunities, −6.5%; 95% CI, −9.2% to −3.8%).

Table 3. Adolescent Office Visits at Which an MO Occurred by Study Groups by Visit Type and Vaccine Dose (Visit-Level Analysis)^a,b.

Visit type	Office visits with MO for HPV vaccination, %c
	Initial HPV vaccine dose			Subsequent HPV vaccine dose
	Baseline period	Intervention period	Percentage-point change in MOs (95% CI)	Baseline period	Intervention period	Percentage-point change in MOs (95% CI)
Well-child care
Intervention group	57.5	48.7	−8.8	13.8	10.0	−3.8
Control group	55.0	53.0	−2.0	10.3	8.9	−1.3
Intervention vs control groups	NA	NA	−6.8 (−9.7 to −3.9)	NA	NA	−2.5 (−5.5 to 0.5)
Acute or chronic
Intervention group	98.3	97.9	−0.3	86.0	88.7	2.7
Control group	98.4	98.1	−0.3	88.6	90.1	1.6
Intervention vs control groups	NA	NA	0 (−0.7 to 0.6)	NA	NA	1.1 (−1.5 to 3.7)

Abbreviations: HPV, human papillomavirus; MO, missed opportunity; NA, not applicable.

^a The denominator in these analyses is all office visits by adolescents 11 to 17 years of age during the relevant study period (baseline or intervention) who were eligible for an HPV vaccination. The numerator is all office visits by these adolescents who did not receive an HPV vaccination (ie, an MO).

^b A reduction in MOs represents an improvement.

^c Baseline periods were wave 1 (October 31, 2017, to October 31, 2018) and wave 2 (December 19, 2017, to December 19, 2018). Intervention periods were wave 1 (December 26, 2018, to June 11, 2019) and wave 2 (February 13 to July 30, 2019).

The unadjusted rate of missed opportunities decreased in intervention practices more than in control practices by 7.3 percentage points (95% CI, 3.8%-10.7%) for 11- to 12-year-old adolescents and by 5.8 percentage points (95% CI 1.9%-9.6%) for 13- to 17-year-old adolescents. This difference of 1.5 percentage points (95% CI, −3.3% to 6.3%) did not provide evidence of effect modification by age.

Missed Opportunities for Subsequent HPV Vaccination at WCC Visits

The unadjusted rate of missed opportunities decreased in intervention practices more than in control practices by 2.5 percentage points (95% CI, –5.5% to 0.5%) (Table 3; eAppendix 2 in Supplement 2). These results did not attain conventional levels of statistical significance.

Missed Opportunities at Acute or Chronic Visits

There was little effect of the intervention on either initial or subsequent HPV vaccination at acute or chronic visits. The difference in missed opportunities was 0 percentage points (95% CI, –0.7% to 0.6%) for the initial vaccine and 1.1 percentage points (95% CI, –1.5% to 3.7%) for the subsequent vaccination (Table 3; eAppendix 2 in Supplement 2).

Proportion of Adolescents Receiving HPV Vaccinations (Person-Level Analyses)

For adolescents with at least 1 HPV vaccine–eligible office visit during the intervention period, HPV vaccine initiation was higher by 3.4 percentage points (95% CI, 0.6%-6.2%) in intervention vs control practices (Table 4; eAppendix 3 in Supplement 2). HPV vaccine initiation was higher by 3.8 percentage points (95% CI, 0.2%-7.4%) among 11- to 12-year-old adolescents in the intervention group and by 2.4 percentage points (95% CI, −0.5% to 5.3%) among 13- to 17-year-old adolescents in the intervention group; effects did not differ by age (difference, 1.4 percentage points; 95% CI, −2.3% to 5.0%). The intervention did not affect subsequent HPV vaccinations.

Table 4. Adolescents Who Received an Initial or Subsequent HPV Vaccine Dose Among Adolescents Who Were Eligible to Receive an Initial or Subsequent HPV Vaccination (Person-Level Analysis)^a.

Group	Adolescents who received initial or subsequent HPV vaccine doses among those eligible to receive HPV vaccine, %b
	Initial HPV vaccine dose			Subsequent HPV vaccine dose
	Baseline period	Intervention period	Percentage-point change in vaccination (95% CI)	Baseline period	Intervention period	Percentage-point change in vaccination (95% CI)
Adolescents aged 11-17 y (total)
Intervention group	26.4	32.1	5.6	55.5	57.1	1.6
Control group	27.2	29.5	2.3	54.3	56.5	2.2
Intervention group vs control group	NA	NA	3.4 (0.6 to 6.2)	NA	NA	−0.6 (−4.1 to 3.0)
Adolescents aged 11-12 y
Intervention group	34.0	41.3	7.3	58.0	59.9	1.9
Control group	34.4	38.0	3.5	56.8	59.2	2.5
Intervention group vs control group	NA	NA	3.8 (0.2 to 7.4)	NA	NA	−0.6 (−6.1 to 4.9)
Adolescents aged 13-17 y
Intervention group	16.2	18.3	2.1	53.6	54.4	0.8
Control group	16.4	16.1	−0.3	52.3	53.8	1.5
Intervention group vs control group	NA	NA	2.4 (−0.5 to 5.3)	NA	NA	−0.7 (−5.6 to 4.3)

Abbreviations: HPV, human papillomavirus; NA, not applicable.

^a The denominator in these analyses is all adolescents 11.0 to 17.9 years of age who made a visit during the relevant period and were eligible for an initial or a subsequent HPV vaccination. This denominator is different than the denominator in Table 2. The numerator is all adolescents who made a visit during the relevant period and received an initial or a subsequent HPV vaccination. We month-matched baseline and intervention periods, and clustered by practice.

^b The baseline and intervention periods were both 6 months’ duration: study periods were wave 1 (baseline: December 26, 2017, to June 11, 2018; (intervention: December 26, 2018, to June 11, 2019) and wave 2 (baseline: February 13 to July 30, 2018; intervention: February 13 to July 30, 2019).

Subgroup Analysis of Visits for Participating Clinicians

The rate of missed opportunities decreased in intervention practices more than in control practices by 7.3 percentage points (95% CI, 4.0%-10.6%) for the initial HPV dose at WCC visits provided by participating clinicians after adjustment for age and sex of adolescents.

Adjustment for Adolescent Demographic Characteristics

Covariate adjustment by age, sex, and their interaction did not appreciably alter the findings.

Survey of Participating Clinicians

The postintervention survey was completed by 101 of 122 partipating clinicians (82.8%). Most reported practicing communication skills with an office colleague (61 of 94 [64.9%]), sharing training materials with nursing staff (63 of 94 [67.0%]), using the “same way same day approach” to introduce HPV vaccines (80 of 93 [86.0%]), and feeling confident answering questions about the HPV vaccine (80 of 94 [85.1%]) or talking with hesitant parents (68 of 94 [72.3%]). Most stated that the communication training did not increase the time spent discussing the HPV vaccine (53 of 94 [56.4%]) or increased the time by less than 30 seconds (12 of 94 [12.8%]) or less than 1 minute (14 of 94 [14.9%]).

Discussion

In this cluster RCT, an online communication training intervention for HPV vaccination resulted in a greater reduction in missed opportunities for HPV vaccination in intervention vs standard-of-care control practices for HPV vaccination rates overall and for HPV vaccine initiation at WCC visits. These improvements occurred during a relatively short (6-month) time frame and with less than two-thirds of clinicians participating via the training modules. For subsequent HPV vaccine doses, the reductions were smaller and did not attain conventional levels of statistical significance. The intervention had no effect on HPV vaccination rates at acute or chronic visits. Most clinicians completed all 3 training modules and stated that the intervention did not lengthen discussions about the HPV vaccine.

Our intervention was especially effective during WCC visits, consistent with prior studies,^31,38,48 and not surprising given that 96% of initial and 85% of subsequent HPV vaccinations at baseline occurred at WCC visits, with good consistency across practices. Our intervention focused on vaccine communication, which occurs primarily during WCC visits. An increase in vaccination rates at acute or chronic visits likely requires changes in practice operations during those visits.^26,38,49

The intervention improved HPV vaccine initiation but not subsequent doses. Vaccine communication is most important for vaccine initiation, particularly given frequent parental hesitancy about the HPV vaccine.^{17,18,19,20,21} Most adolescents who receive a first dose of the HPV vaccine receive subsequent doses. Other types of interventions, such as clinician prompts,⁵⁰ standing orders, scheduling subsequent doses, and quality improvement interventions^{31,38,48,49,51} focusing on office workflow and procedures, might have a greater effect on missed opportunites for subsequent HPV vaccinations. Furthermore, after the intervention, the missed opportunity rates for HPV vaccine initiation ranged from 23.6% (33 of 140) to 70.7% (531 of 751; median, 44.3% [70 of 158]) among intervention practices, suggesting room for further improvement. Ultimately, a combination of interventions may be needed to optimally increase coverage.⁵²

Researchers have published reports of effective in-person training by communication experts.^24,25,26 In-person interventions are challenging to deliver and scale widely, yet they may have a larger effect than our study, as noted by Brewer et al²⁴ and Dempsey et al.²⁵ We designed our intervention to be readily scalable, knowing that this focus might diminish the effect. We avoided approaches requiring programming, coding expertise, or special equipment, such as simulations with avatars or virtual reality. The intervention modules are straightforward to update. Nearly all participants completed all 3 modules, and outcomes consistently improved across practice sites. Overall, we believe our intervention is feasible and scalable.

Our finding that the improvement in missed opportunities at visits was greater than the improvement in person-level vaccination rates is not surprising given the brief 6-month intervention period. Adolescents make few primary care visits per year. During our 6-month intervention period, 43.6% of adolescents (14 299 of 32 794) with any visit to intervention practices made only WCC visits, 42.2% (13 827 of 32 794) made only acute or chronic care visits, and 14.2% (4668 of 32 794) had both types of visits. Thus, nearly half of adolescents never had a WCC visit, which is when most HPV vaccinations are administered. Our study suggests that additional training of pediatric clinicians in evidence-based communication strategies can help clinicians take advantage of these WCC visits to increase HPV vaccination rates.

Strengths and Limitations

Our study has several strengths. First, we had a large number of practices with a cluster randomized design that improved statistical power substantially compared with a single-period parallel-group design. Second, the intervention was scalable, with online communication modules and weekly text messages to reinforce training modules. Third, we used a rigorous as-randomized analysis that included in the primary outcome all vaccine-eligible adolescent visits and all clinicians in intervention practices regardless of their participation in training.

Our study also has some limitations, including potential selection bias because practices needed to sign up for our study; only 42.5% of potential practices (48 of 113) participated, and we do not know the reasons for nonparticipation. Second, we were missing data on race/ethnicity in many cases and more often among control practices owing largely to 5 control practices that lacked race/ethnicity data for more than 50% of adolescents. Third, the study population had higher-than-average HPV vaccination rates; our intervention might have a greater effect in settings with lower baseline rates. In addition, our findings may not be generalizable to other settings serving adolescents.^33,53

Conclusions

In this cluster RCT, an online communication intervention increased HPV vaccination rates by reducing missed opportunities for HPV vaccine initiation during WCC visits. This intervention may be scalable and, if adapted widely, might add to our nation’s strategy to reduce HPV-related disease.

Supplement 1.

Trial Protocol

Supplement 2.

eAppendix 1. Weekly Text Message (or Email if Requested) “Quick Tips” Designed to Reinforce the Content of the Educational Modules

eAppendix 2. Percentage of Adolescent Office Visits at Which a Missed Opportunity (MO) Occurred by Study Group Over: a) All Eligible Visits and b) by Visit Type (Well-Child Care or Acute/Chronic)

eAppendix 3. Percentage of Adolescents Who Received an Initial Dose or Subsequent HPV Vaccine Dose (Month-Matched by Period, and Clustered by Site) Among Adolescents Who Were Eligible to Receive an Initial or Subsequent HPV Vaccination

Supplement 3.

Data Sharing Statement