Provider Communication, Prompts, and Feedback to Improve HPV Vaccination Rates in Resident Clinics

Abstract

BACKGROUND AND OBJECTIVES:

Human papillomavirus (HPV) vaccination rates lag behind vaccination rates for other adolescent vaccines; a bundled intervention may improve HPV vaccination rates. Our objective is to evaluate the impact of quality improvement (QI) training plus a bundled practice-based intervention (provider prompts plus communication skills training plus performance feedback) on improving HPV vaccinations in pediatric resident continuity clinics.

METHODS:

Staff and providers in 8 resident clinics participated in a 12-month QI study. The intervention included training to strengthen provider communication about the HPV vaccine. Clinics also implemented provider prompts, received monthly performance feedback, and participated in learning collaborative calls. The primary outcome measure was eligible visits with vaccination divided by vaccine-eligible visits (captured HPV vaccination opportunities). Practices performed chart audits that were fed into monthly performance feedback on captured HPV vaccination opportunities. We used conditional logistic regression (conditioning on practice) to assess captured vaccination opportunities, with the time period of the study (before and after the QI intervention) as the independent variable.

RESULTS:

Overall, captured opportunities for HPV vaccination increased by 16.4 percentage points, from 46.9% to 63.3%. Special cause was demonstrated by centerline shift, with 8 consecutive points above the preintervention mean. On adjusted analyses, patients were more likely to receive a vaccine during, versus before, the intervention (odds ratio: 1.87; 95% confidence interval: 1.54–2.28). Captured HPV vaccination rates improved at both well-child and other visits (by 11.7 and 13.0 percentage points, respectively).

CONCLUSIONS:

A bundled intervention of provider prompts and training in communication skills plus performance feedback increased captured opportunities for HPV vaccination.

Most cases of cervical cancer¹ and >60% of oropharyngeal cancers are related to human papillomavirus (HPV) infection.² Despite the vaccine having been recommended for adolescent girls since 2007³ and for boys since 2011,⁴ HPV vaccination rates in the United States lag behind those for other adolescent vaccines. The Centers for Disease Control and Prevention, the President’s Cancer Panel from 2012 to 2013, and conductors of systematic reviews^5–7 have recommended reducing missed opportunities (MOs) to administer HPV vaccines to increase HPV vaccination coverage.^8,9

Several strategies might reduce MOs. Provider prompts is 1 strategy, yet few researchers have evaluated prompts for the HPV vaccine. One randomized clinical trial revealed that prompts combined with practice-based performance feedback reduced MOs and improved HPV vaccination rates.^10,11 However, a second randomized clinical trial¹² revealed that prompts alone reduced MOs slightly in 1 of 2 practice networks only and failed to raise HPV vaccination rates substantially. It is implied in these results that prompts alone may not be sufficient to raise vaccination rates. A second strategy relates to the finding that parents often report that the HPV vaccine was not recommended by their adolescent’s physician.⁹ Thus, an additional potential strategy is to strengthen provider recommendations.⁸ A third possible intervention is performance feedback for providers. Performance feedback can change provider behavior for many health services,^13,14 including vaccination,^11,15 because physicians do not accurately evaluate their own performance.¹⁶ Its effectiveness depends on how the feedback is provided; feedback is more effective when provided more than once, in multiple formats, and when it includes a goal.¹⁴ Therefore, in this quality improvement (QI) study, we combine provider prompts, training in giving a strong recommendation for the HPV vaccine, and performance feedback, with a goal of reducing MOs for HPV vaccination.

We conducted a QI study over a 12-month period in 8 practices from a national network of pediatric resident continuity clinics that are based at academic medical centers. Whereas much of the literature concerns MOs, our outcome was captured opportunities (eligible visits with HPV vaccination divided by HPV vaccine–eligible visits), which are the inverse of MOs (to display improvement as a positive change). Our specific aim in the project was to increase captured opportunities for HPV vaccination by 10%. A secondary goal was to improve documentation (by 10%) of reasons why the HPV vaccine was not given when due, to better identify issues leading to under-vaccination. We also performed a qualitative analysis of monthly practice diaries in which Plan-Do-Study-Act (PDSA) cycles were documented to assess office changes that were associated with greater success in improving captured opportunities.

METHODS

Context

This 12-month QI study was conducted in a national, practice-based research network that is composed of pediatric resident continuity practices (Continuity Research Network [CORNET]).¹⁷ An e-mail newsletter used to encourage participation was distributed to all 107 CORNET practices, with a goal of enrolling 12 to 14 practices overall. Eight CORNET practices volunteered to participate in the prompt intervention (5 other practices participated in a standing order intervention, described elsewhere). Maintenance of certification credit was provided for faculty as an incentive to participate, in addition to $1000 per practice to compensate for time needed for chart reviews. At the time of the baseline period, none of the practices had ways to prompt providers when an HPV vaccination was due for a patient, and vaccination was not generally recommended outside of well visits.

Interventions

Basic QI Training

Initial QI training involved four 15-minute videos¹⁸ used to introduce the following topics: the model for improvement, leading change, initiating a QI project, and QI measurement. All videos focused on practice improvement in primary care.

Strong Provider Recommendation

Participants watched a 30-minute webinar on “Making a Strong Recommendation About HPV Vaccine to Parents in Primary Care,” based on slides created by the Centers for Disease Control and Prevention.¹⁹ The webinar covered communication strategies including: (1) recommending the HPV vaccine in the same way and at the same visit as other recommended adolescent vaccines, (2) providing accurate and succinct answers to frequently asked questions about the vaccine, and (3) training the entire office team to feel comfortable talking about the HPV vaccine to provide consistent messages supporting HPV vaccination.

Provider Prompts

Office staff members implemented prompts from nurses to remind the providers to give the HPV vaccine to adolescents who were due at any type of visit. Because it was already common practice to review vaccinations at well visits, the prompt intervention targeted acute and chronic care visits. Nurses or medical assistants were expected to review the vaccination record before or at every visit for any adolescent and to prompt the provider to order the vaccine. The prompt mechanisms varied by practice (eg, Vaccine Information Statement [VIS] forms placed on the clinician’s desk in the examination room; a laminated, erasable sign that was used to indicate which vaccines were due; a highlighted patient schedule for the day; printed immunization registry data). Some practices also used electronic health record (EHR) decision support in addition to the nurse prompt, but this was not the sole cue at any site. In addition, practices used visual cues (ie, door signs and posters) to raise awareness of the project for staff, clinicians, and parents.

Performance Feedback

Practices performed monthly chart reviews (10 charts per month)²⁰ for adolescents aged 11 to 17 years who were due for an HPV vaccine at the visit. Practices received monthly feedback reports with run charts describing the overall (across 8 sites) and site-specific progress for HPV vaccine captured opportunities; they were asked to share these reports with providers and staff. Each practice also filled out a diary, describing specific changes the practice worked on during each month (using PDSA cycles).

Monthly Learning Collaborative Telephone Calls

We conducted monthly calls via webinar with intervention practices to teach improvement science, drive PDSA cycles, share practice run charts, and discuss barriers and successes with strong provider recommendations as well as prompt implementation. Learning collaborative (LC) faculty provided coaching to address obstacles and encourage those meeting their objectives.

Improved Documentation

Practices also worked on improving documentation if a vaccine was refused; they edited their EHR note templates to encourage documentation. When possible, they added the following choices to document when a patient did not get an HPV vaccine when due: refused, referred to later date, or unable to obtain consent.

Practice Teams

In addition to nurses, medical assistants, and pediatric residents, each practice team included at least 1 QI lead who was a continuity clinic attending physician. At some sites, receptionists were included for specific parts of the intervention (ie, scheduling follow-up appointments). The practice teams met monthly to review data and plan PDSA cycles. They communicated with other providers in the practice via e-mail and educational sessions for staff and residents.

Academic Pediatric Association Core Team

The project was led by individuals with expertise in immunization delivery (C.A., C.M.R., S.G.H., S.J.S., P.G.S.), QI (K.J.M., W.S.), and CORNET directors and staff (P.M.D., J.R.S., N.D.). Two immunization delivery experts (C.M.R., S.J.S.) led the monthly LC calls.

Measures

The target population was all adolescents aged 11 to 17 years who had a visit to a participating practice during the 12-month intervention period (baseline data were for those with visits in the 12 months before intervention). Visits were considered vaccine-eligible if the adolescent was eligible for the HPV vaccine at the visit on the basis of the 2013 Advisory Committee on Immunization Practices guidelines²¹; that is, HPV dose 1 if none previously, HPV dose 2 if >30 days from HPV dose 1, and HPV dose 3 if >24 weeks from HPV dose 1 and >12 weeks from HPV dose 2.

The primary outcome measure was captured HPV vaccination opportunities (the number of eligible visits with vaccination divided by the number of vaccine-eligible visits). Practices gathered retrospective data for the 12 months before the intervention (10 charts per month) and monthly during the intervention period by using randomly selected charts to report on patient characteristics, visit types, captured opportunities, and the reasons patients were not vaccinated if due (process measure). Reasons were coded as refused, deferred to later date, not mentioned and/or unclear (missing data), or other.

Analysis

We assessed (1) captured opportunities for HPV vaccination by practice for all visits and by visit type and (2) reasons patients were not immunized if due. We compared the baseline period to intervention periods by using descriptive statistics and Pearson’s χ² test. We also assessed the odds of HPV vaccination at the index visit using conditional logistic regression and conditioning on practice, with the time period of the study as the independent variable.

The primary outcome measure was plotted on a statistical process control chart (P-chart) by using Microsoft Excel QI Macros. Upper and lower control limits were set at 3 σ. Special cause (evidence of a change in the system) was noted when 8 consecutive data points were above the preintervention mean.

One researcher (C.M.R.) reviewed the monthly practice diaries to examine common themes and better understand the relationship between practice changes and improvement in captured opportunities.

Ethical Considerations

The Research Subjects Review Board of the University of Rochester approved the project; the 8 CORNET sites obtained institution-specific institutional review board approval.

RESULTS

In Table 1, we show patient demographics in the 8 continuity clinics. Most (7 out of 8) practices were hospital-based, 5 were in urban settings, and all participated in the Vaccines for Children program; most patients were publicly insured.

TABLE 1.

Practice Demographics

Demographic	Value
Practice, n (%)	(N = 8 practices)
Hospital-based clinic	7 (88%)
Private- and/or office-based	1 (12%)
Primary patient population, n (%)
Suburban	3 (38%)
Urban	5 (63%)
Use of an EHR, n (%)
Yes	6 (75%)
No	2 (25%)
No. adolescent patients, mean (SD)	2224 (1748)
Patient insurance type, mean (SD)
Public	68.9 (27.8)
Private	23.9 (28.7)
Military	1.6 (1.9)
None	5.5 (7.6)
Other	1.5 (3.5)

As shown in the P-chart of monthly chart reviews (Fig 1), captured opportunities for HPV vaccination increased by 16.4 percentage points (from 46.9% to 63.3%) overall. Special cause was demonstrated by centerline shift after the intervention.

FIGURE 1.

P-chart of captured opportunities for HPV vaccine, monthly chart reviews. CL, control limit; LCL, lower control limit; UCL, upper control limit.

Comparing the intervention period to the baseline period, we found that adolescents were more likely to receive a vaccine at an eligible visit during the intervention than before the intervention (odds ratio: 1.87; 95% confidence interval: 1.54–2.28). HPV captured vaccination rates improved at both well child (from 65% to 77%) and other visits (from 27% to 40%), with P < .001. Improvement in captured opportunities varied by practice, from 2.5 to 30 percentage points (Fig 2). Practices with the lowest initial rates made the least improvement; practices with the middle range at baseline made the most improvement. Documentation of refusals and deferring vaccination increased from 9.3% to 17.1% and from 5% to 12.8%, respectively, whereas no documentation for an MO declined from 80.2% to 65.5%, with P < .001 overall.

FIGURE 2.

Captured opportunities for HPV vaccination by practice.

Monthly diaries were used to describe the changes in the prompt intervention, barriers encountered, parent responses, and lessons learned (Table 2). Barriers included nursing shortages, staff changes, and decreased response to the prompts at busy periods. A minority of parents continued to be resistant to vaccination, and safety was a concern that was raised by several parents. Lessons learned from successful practices included sending frequent communications about the project and meeting frequently about practice change.

TABLE 2.

Qualitative Responses From Monthly Diaries

Improvement in Captured Opportunities (No. Practices)	Changes in Prompt Intervention	Barriers and How They Were Addressed	Parent Resistance	Lessons Learned
25%–30% (3)	Laminated cards, VIS sheet on fluorescent paper, changing wall color sign, door flag Reminders to nurses to review and document Posted HPV process flowchart	Nursing shortages; intervention improved as staffing improved Priority to update new staff and review data When busy, constant reminders needed to get staff back on track	Some parents still prefer to defer to the well-care visit, want to discuss with a spouse, or wait until their child is older Patient sometimes not accompanied by the primary caregiver; try to obtain phone consent or schedule a nurse visit with the primary caregiver as soon as possible Continued resistance from a small minority of parents regarding questions of vaccine safety and necessity	Sending frequent reminders every month helped improve outcomes Improvement occurred when nurse manager helped coordinate the project
6%–11% (3)	Laminated reminder signs used to promote the vaccine in the physician’s workroom, using door flags in addition to visits sheets Editing the note template to improve documentation of refusals and prompt education resources given Stamps on the printout of the immunization registry to identify that the patient was checked for HPV vaccine need (frequent reminders needed to use stamps)	Staff changes; untrained substitutes assigned to the clinic; no vaccine reminders were provided When new interns arrived, vaccines were often missed	Continued resistance from a small minority of parents regarding questions of vaccine safety and necessity	Early, persistent reminders are helpful in long-term change and outcomes
<5% (2)	Nurse attaches VIS sheet and puts “needs HPV” in intake note Incentivized nurses, reinforced importance Nurse puts prompt in intake of EHR LED light on computer for nurse to turn on	When project champion was out, less prompting occurred Staff notes vaccine due but doesn’t discuss unless parent asks Prompting irregular; depends on staff and how busy it is Changing behavior met with some resistance Some clinicians sympathized with the parent’s reluctance to vaccinate at a sick visit	Parents concerned about young age and giving the vaccine to boys, think they can wait; sometimes want to talk to spouse Concerns about vaccine safety	Generic prompt (review vaccines) had no impact Repetition is important QI Team needs to meet more often to initiate changes

Regarding a strong provider recommendation, providers noted that staff recommendations were not as strong as expected at baseline but appeared to improve because of the intervention. Practice comments toward the end of the study were as follows:

The staff continues to get resistance from some parents. Overall seems less over the past 6 months. The 2 areas for this are: (1) my child is not sexually active yet, “I’ll wait.” and (2) The thought that the HPV is only for girls to prevent cervical cancer only; there is a hesitancy to give it to their sons. Some staff have stepped up their education and overall approach to the vaccine. I believe this change in staff has led to overall less resistance from the parents.

[We have] much less hesitancy or resistance than in the earlier part of the year. It now feels like the HPV is automatic, not just an add-on.

Practices with little improvement (<5%) lacked consensus for the QI activity from staff and providers. In addition, more complicated interventions (such as having the nurse place an LED light on the computer) were difficult to sustain. The simplest interventions, such as putting the VIS sheet on fluorescent paper, led to greater improvement (>25%). In addition, successful practices made it a priority to update new staff and to review monthly data with the QI team.

DISCUSSION

In this QI study in pediatric resident continuity clinics, we found that a bundled intervention consisting of a combination of provider prompts from nurses and EHRs, provider and staff training in giving a strong recommendation for the vaccine, and performance feedback to providers increased rates of captured opportunities for HPV vaccination by 16 percentage points. However, there was great variation in improvement by practice, ranging from 2.5 to 30 percentage points. In addition, absence of documentation for MOs declined from 80% to 66%.

The range of improvement for practices was broad and provides lessons for practice improvement. Practices with the greatest increases in captured opportunities used the following approaches: frequent communications for staff and residents, changing EHR note templates to improve documentation, using multimodal prompts (paper, VIS, EHR), and having a nurse champion who reviewed medical records ahead of time. By improving documentation, practices have a better understanding of causes of under-vaccination. A previous study that was focused solely on EHR prompts without other interventions was not effective.¹² This may be because of “prompt fatigue,”^22,23 indicating the necessity to use multimodal cues. However, prompts need to be simple to be sustainable. For example, a fluorescent VIS sheet worked well as a prompt, whereas stamping the immunization registry printout had an intermediate effect. We also believe that the addition of performance feedback to prompts contributed to the large effects. In addition, practice champions can improve adherence to guidelines, including hand hygiene,²⁴ vaccine delivery,²⁵ and infection prevention.²⁶ Finally, the success of our intervention supports other studies in which researchers have found that multicomponent interventions are more effective^27,28 than single-component ones.

Two practices with the lowest baseline rates of captured opportunities experienced the least improvement. In 1 case, the barriers were potentially identifiable a priori, whereas in the other practice they were not. In 1 practice, the project champions had trouble convincing all providers and nurses in the practice to support vaccinating at illness visits. Practice engagement is a critical component of successful QI initiatives.²⁹ It is important to involve the entire office staff in the intervention³⁰ and to engage leadership in practice change for it to be successful.³¹ A second practice experienced major staff turnover; hence, the intervention became a low priority. Such real-life factors can dramatically diminish practices’ abilities to improve the delivery of preventive services.³² Because primary care practices face a multitude of barriers to practice improvement,³³ a potential target for future research is to identify factors that predict the likelihood of improvement from modifiable practice features.

At the other extreme, the highest performing practice at the baseline period also experienced a low rate of improvement in captured opportunities. At the high range of performance, there is minimal room for improvement, and alternative strategies may be needed to reach the remaining few MOs. This is consistent with other studies that have revealed that high-performing practices make fewer advances than midrange performers in QI interventions.^34,35

The highest rate of captured opportunities in any intervention practice was 84%, with an average of 63%. We believe rates of captured opportunities are unlikely to reach 100% for HPV vaccination in most practices because of several factors. First, parent delay and refusal are likely to contribute to ∼20% of HPV under-vaccination.³⁶ Although a strong recommendation can impact vaccine refusals, as shown by a 12% increase in captured opportunities at well visits, other strategies are needed to reach the most vaccine-hesitant parents. Second, a few patients will be too sick to be vaccinated. Lastly, some days are likely to be too busy to allow time for nurses to assess vaccination status or to draw up and administer vaccines during acute visits. At busy times in the practice, such as during influenza season, the staff had particular difficulty reviewing vaccination records from multiple sources, and, consequently, some practices noted a drop in nurse prompts. During times of stress, practices may need to adjust the intervention.²⁹

One strength of this study was the multidisciplinary nature of the practice teams, which included attending physicians, residents, nurses, and staff. Collaborating with staff from different areas in a practice has been shown to enhance teamwork and communication and counters the “silo effect.”³⁷ In particular, training residents in both QI and in providing a strong recommendation is important because residents tend to use the skills learned during residency during their later practice.^38,39 A second strength was having a multisite network of practice-based teams that allowed providers to learn from one another.⁴⁰ Such LC models are now increasingly used for a variety of health topics. A third strength is potential sustainability, which was enhanced by providers earning Maintenance of Certification Part IV credit for participation; this has been shown to motivate physicians to participate in QI for HPV vaccination.⁴¹ Finally, the 8 practices involved in the intervention care for a large number of adolescents, so this intervention has the potential to impact a sizeable population.

There are several characteristics of the setting and patient population that may limit generalizability. The intervention was performed in pediatric resident continuity clinics serving mostly low-income populations; findings may not be generalizable to other settings. Academic practices may be particularly motivated to implement newer guidelines and are expected to teach current recommendations. On the other hand, these large practices with rotating residents are complex environments in which to implement practice change. There may be selection bias in practices volunteering to participate. The practices received a modest incentive for performing the randomly selected chart reviews. Also, the research and LC faculty leads were supported by grant funding. To scale up this intervention, we would need to integrate such core funding into local, state, or national QI efforts.

CONCLUSIONS

In this QI study, we demonstrate that a bundled intervention consisting of multimodal provider prompts combined with training on the delivery of a strong provider recommendation for HPV vaccination and performance feedback to providers can substantially increase captured opportunities for HPV vaccination in pediatric resident continuity clinics. For best results, practices should have an engaged champion, communicate frequently with the practice team, ensure that all providers understand and support the initiative, and incorporate ongoing performance feedback. If scaled up, such a QI intervention might help raise HPV vaccination rates across the United States.

ABBREVIATIONS

CORNET

Continuity Research Network

EHR

electronic health record

HPV

human papillomavirus

LC

learning collaborative

MO

missed opportunity

P-chart

process control chart

PDSA

Plan-Do-Study-Act

QI

quality improvement

VIS

vaccine information statement