Evaluating Interventions to Increase Influenza Vaccination Rates among Pediatric Inpatients

Abstract

Introduction:

Hospitalization provides an ideal opportunity for influenza vaccination, and strategies can enhance existing tools within the electronic medical record (EMR). The objectives of the study were to introduce and evaluate the effectiveness of provider and family-directed interventions to increase influenza vaccination ordering among inpatients.

Methods:

We conducted a quality improvement initiative for children aged older than 6 months on medical inpatient teams at a large pediatric tertiary care hospital from September 2014 to March 2015, comprising 2 intervention groups (provider reminders and family education) and 1 control group for comparison, using EMR prompts alone. The provider reminder interventions comprised weekly e-mails indicating inpatient immunization status; vaccination reports; and visual reminders. The family education group intervention consisted of handouts regarding the benefits and safety of influenza vaccination. We measured vaccine ordering rates for each group among eligible children and overall vaccination rates. Data were analyzed using Statistical Process Control Charts and Chi-square tests.

Results:

Among 2,552 patients aged older than 6 months hospitalized during the study period, 1,657 were unimmunized. During the intervention period, the provider group ordered 213/409 (52%) influenza vaccines, the family education group ordered 138/460 (30%) and the control group ordered 71/279 (25%) (P < 0.0001). The provider group had higher influenza immunization status than the control group (61% versus 53%; P = 0.0017). Exposure to the intervention did not impact the length of stay/discharge time.

Conclusions:

Provider reminders including e-mails, visual reminders, and vaccination reports are effective ways of increasing inpatient influenza vaccination rates and are more effective than family education, or EMR prompts alone.

INTRODUCTION

Influenza is a leading cause of morbidity, mortality, and utilization of health care services in infants and children.^1,2 Annual influenza vaccination is safe, inexpensive, and the most effective method for preventing influenza infection and its complications.³ Despite recommendations from the Advisory Committee on Immunization Practices and the American Academy of Pediatrics for universal influenza vaccination for individuals 6 months of age and older, national immunization rates were 59% among children during the 2014–2015 season,⁴ remaining below national objectives, highlighting a need to provide vaccination at every opportunity.

Children with certain medical conditions are at higher risk of influenza-related complications,⁵ and influenza vaccination during hospitalization provides an excellent opportunity to target these high-risk individuals, potentially avoiding prolonged hospitalizations and mortality. However, studies suggest low influenza immunization rates among pediatric inpatients and gaps in vaccine ordering during hospitalization.^6–8 A study at our institution demonstrated that 61% of children hospitalized due to influenza were partially vaccinated or unvaccinated against influenza.⁹ Another study in 2 community hospitals in California demonstrated that 50.5% of children received influenza vaccination before admission.¹⁰ Although multiple trials have shown that computerized reminders increase the use of preventive care measures in the outpatient setting,^11,12 several studies have failed to demonstrate an increase in the rates of inpatient influenza vaccination through standing orders and alerts in the medical record alone.^13,14 Some studies suggest that a combination of computerized reminders plus staff education leads to increased immunization rates^15,16 and that hospital-based interventions are cost-effective¹⁷; however, there is a paucity of data in pediatric inpatients.

At our institution, to address influenza immunization among both outpatients and inpatients, several automated features were added to the electronic medical record (EMR) to alert medical providers of influenza vaccination status. However, despite these changes, the use of many of these features was inconsistent, and vaccination rates among inpatients did not increase, remaining at less than 50%, similar to state-based estimates.¹⁸ Given that computer-based prompts alone were not sufficient to increase influenza vaccination rates among inpatients, in this project, we aimed to introduce and evaluate the effectiveness of provider and family-directed interventions to increase influenza vaccination rates among medical inpatients during the 2014–2015 influenza season.

METHODS

Context

The improvement efforts were conducted on the general pediatric inpatient unit at Children’s Hospital Colorado Anschutz Medical Campus in Aurora, Colorado. Our institution is a large academic quaternary care center primarily serving the Denver metropolitan area (population of ~2.5 million people), greater Colorado, and the 7 surrounding states.

The main campus in Aurora has 475 beds and admits approximately 14,000 inpatients per year. The main campus has 3 hospitalist-led medical teams, which are staffed by attending faculty physicians, interns, residents, nurse practitioners, and medical students. Residents and attending physicians rotate between the medical teams. One hospitalist team is staffed by physicians and nurse practitioners only when the census is low during the nonrespiratory season. The patients admitted to these teams include medically complex children with subspecialty needs. Patients on each team are similar concerning demographics, payor mix, and medical complexity.

The patient population consisted of all inpatients aged 6 months of age and older admitted to the inpatient hospitalist teams between September 1, 2014, to October 12, 2014 (preintervention period) and October 13, 2014, and March 30, 2015 (intervention period) without contraindications to the vaccine.

Planning the Intervention

To help inform the nature of our interventions, we developed and distributed separate questionnaires during the preintervention period for providers and parents/guardians based on literature review of inpatient interventions and barriers to influenza vaccination.^19–24 We asked parents/guardians questions regarding immunization status and attitudes toward influenza immunization during hospitalization, and asked nursing and medical staff questions regarding the importance of influenza vaccination and potential barriers to influenza vaccination. Our baseline data demonstrated that influenza vaccination is a high priority for physicians, but lack of time, forgetting to order the vaccine and provider belief of caregiver misconceptions about vaccination are important reasons for failure to order the vaccine. Additionally, parents understood the seriousness of influenza infection, the safety, and effectiveness of the influenza vaccine and were open to inpatient influenza vaccination.²⁵ These survey responses were used to tailor targeted interventions at our institution. Based on the baseline data, we focused on provider reminders and providing real-time data regarding vaccination status and parental education in the form of written materials provided during admission. We continued providing the same questionnaires to the parents and providers during the intervention period, as a process measure to gauge the effectiveness of our intervention.

The Intervention

The 2 interventions for the study were (1) provider reminders; and (2) family education. For the provider reminder group, clinical support staff provided a team-specific vaccination status report (at the patient level) to residents each morning before rounds, sent weekly e-mails indicating influenza vaccination status for each medical team to residents and attendings on the medical teams, and posted visual reminders regarding influenza vaccination ordering in workrooms and computer workstations. These interventions used features that exist in the current EMR system, by Epic (Verona, Wis.). The influenza immunization dashboards were used to generate reports, which outline the immunization rates for inpatients based on data from best practice advisory data by treatment team and unit.

For the family education group intervention, the research assistants provided an education handout to parents/guardians (or inpatients if age-appropriate) on admission, regarding the benefits and safety of influenza vaccination. The handout was originally created for the current project but was also made available on the hospital’s website. The handout was available in English and Spanish, the primary languages for most families seen at our institution.

The project was conducted using 2 intervention groups and 1 control group. All physicians, physician assistants/nurse practitioners, medical students, and patients associated with the team were assigned that team’s intervention. One designated medical team received provider reminders, and the families of the second medical team received influenza vaccine education handouts. The third medical team served as the control group.

Methods of Evaluation

The primary outcome was the percentage of patients with an influenza vaccine order during hospitalization, defined as inpatients aged 6 months and older who received an influenza vaccine order during hospitalization (numerator) divided by the number of inpatients aged 6 months and older unvaccinated against influenza for that season during the study period (denominator). The secondary outcome was the percentage of patients who are immunized against influenza. This outcome was defined as inpatients aged 6 months and older who received an influenza vaccine prior or during their hospitalization from October to March, (numerator), divided by all patients aged 6 months and older admitted during the study period (denominator). We excluded patients if they were contraindicated to receive the vaccine or if the vaccine was not available during the patient’s hospitalization. We determined vaccination status by documentation in the admission note, which is a required field at our institution. We defined a fully vaccinated individual as a child who received the appropriate number of vaccines per Centers for Disease Control and Prevention (CDC) recommendations.²⁶ We defined a partially vaccinated individual as a child aged 6 months to younger than 9 years who required 2 influenza vaccinations within the same season but had only received 1 vaccine. We defined an unvaccinated individual as a child who did not receive influenza vaccination or was partially vaccinated for that season.

Process measures included the following: the percentage of admissions whereby vaccination status from the admission note matched vaccination status obtained from questionnaire; and the proportion of caregivers with a more favorable attitude toward vaccination after review of education materials based on survey results. Balancing measures included patient length of stay (to determine whether exposure to the intervention led to increased length of stay due to vaccine adverse effect such as fever) and time of patient discharge (to determine whether receipt of vaccine led to delays in discharge). A clinical data analyst generated weekly reports and included the measures outlined previously. We entered responses from the questionnaires into REDCap (Research Electronic Data Capture) tools hosted at the University Of Colorado.²⁷

Statistical Analysis

The proportion of patients vaccinated against influenza and patients with influenza vaccine orders was plotted at baseline, and weekly from October to March using p-Charts for each of the medical teams. We annotated the p-Charts with the interventions, created mean lines for pre- and intervention periods, and analyzed for special cause indicating a statistically significant change pre/postintervention using the standard guidelines from the Associates for Process Improvement. We used quality improvement (QI) Charts version 2.0.22 (Scoville Associates, Raleigh-Durham, N.C.) to create the Statistical Process Control charts.²⁸ For statistical comparisons of the pre- and postintervention groups and control and intervention groups, we used the Pearson Chi-square test for dichotomous variables using IBM SPSS version 22 (Armonk, N.Y.). We analyzed pre- and postintervention survey data using McNemar’s test for paired data using SAS 9.3 (Cary, N.C.).

Ethical Considerations

We sought approval from the Children’s Hospital Colorado Organizational research risk and QI review panel (QI # 1403–8). As the study qualified as QI, institutional review board approval was not required.

RESULTS

Inpatient Vaccination Data

There were 2,824 inpatients during the entire season, including the pre- and intervention period (October 2014 to March 2015); we excluded 272 inpatients due to age younger than 6 months. There were no patients with contraindications to receiving the vaccine. Among the remaining 2,552 patients aged older than 6 months hospitalized during the study period, 1,657 were unvaccinated before admission; 626 were in the provider reminder group, 697 were in the family education group, and 334 were in the control group (Fig. 1). The patients in each group were similar with respect to sex, insurance status, comorbidities, and length of stay, with lower median age in the control group (Table 1).

Fig. 1.

Flow diagram of participants on the pediatric medical inpatient teams at Children’s Hospital Colorado in the intervention and control groups from October 2014 to March 2015.

Table 1.

Demographic and Clinical Characteristics of Pediatric Medical Inpatients at Children’s Hospital Colorado in the Intervention and Comparison Groups from October 2014 to March 2015

Of the 422 vaccines ordered during the intervention period among patients eligible to receive the vaccine, 213/409 (52%) were placed by the provider reminder group, 138/460 (30%) by the family education group, and 71/279 (25%) by the control group. There was an absolute 27% increase in vaccine orders among patients in the provider reminder group after the intervention took place compared with the control group (52% versus 25%; P < 0.0001; Table 2). The provider intervention group had significantly higher influenza immunization status (combined outpatient and inpatient vaccination) than the control group (61% versus 53%; P = 0.0017).

Table 2.

Influenza Vaccine Orders and Immunization Rates of Pediatric Medical Inpatients at Children’s Hospital Colorado in the Intervention and Comparison Groups from September 2014 to March 2015

Although special cause was seen before the intervention due to vaccine availability, the provider intervention group sustained ordering rates throughout the season (Fig. 2). There was a slight increase in orders among patients in the family education group, with special cause observed initially; however, there was a subsequent decrease in orders during February to below preintervention levels (Fig. 3). The percent of orders placed in the control group remained low, with no sustained special cause observed (Fig. 4).

Fig. 2.

Statistical Process Control Chart Showing Influenza Vaccine Orders for pediatric medical inpatients in the Provider Reminder Intervention Group at Children’s Hospital Colorado from September 1, 2014, to March 23, 2015. Footnote: Numerator in chart is number of vaccines ordered, and denominator is number of patients 6 months of age or older who are eligible to receive influenza vaccine. Rules for special cause: (1) a single point outside the control limits; (2) 8 or more consecutive points above or below the centerline; (3) 6 consecutive points increasing (trend up) or decreasing (trend down); (4) 2 out of 3 consecutive points near a control limit (outer one-third); (5) 15 consecutive points close to the centerline (inner one-third).

Fig. 3.

Statistical Process Control Chart Showing Influenza Vaccine Orders for pediatric inpatients on the Family Education Intervention Group at Children’s Hospital Colorado from September 1, 2014 to March 23, 2015. Footnote: Numerator in chart is number of vaccines ordered, and denominator is number of patients 6 months of age or older who are eligible to receive influenza vaccine. Rules for special cause are as outlined in Fig. 2.

Fig. 4.

Statistical Process Control Chart Showing Influenza Vaccine Orders for pediatric inpatients on the Control Group at Children’s Hospital Colorado from September 1, 2014 to March 23, 2015. Footnote: Numerator in chart is number of vaccines ordered, and denominator is number of patients 6 months of age or older who are eligible to receive influenza vaccine. Rules for special cause are as outlined in Fig. 2.

Questionnaire Data—Family

Among a subset of 158 caregivers in the family education group selected to participate in a pre- and during-intervention survey, 149 completed preintervention surveys, and 97/149 (65%) completed the survey after reviewing the education handout. After the education intervention, family members were more likely to agree that influenza vaccines work well to prevent against influenza (71% versus 47%; P < 0.0001), realize that influenza vaccination was available to their child while inpatient (87% versus 74%; P = 0.0013), and report being asked about influenza vaccination during their admission (92% versus 86%; P = 0.03).

Process and Balancing Measures

Vaccination status in the EMR was compared with the questionnaire data and was identical in 664/767 (87%) inpatient admissions. The median length of stay was similar for each group: 53 hours [interquartile range (IQR), 36–94 hours) for the family intervention group, 54 hours (IQR, 36–95 hours) for the provider intervention group, and 55 hours (IQR, 37–88 hours) for the control group. The median discharge time was similar for all groups: 3.00 pm for provider intervention (IQR, 12.33 pm to 5.12 pm), 2.30 pm for family education (IQR, 12.13 pm to 5.10 pm), and 2.18 pm for the control group (IQR, 11.52 pm to 4.40 pm).

DISCUSSION

Studies evaluating interventions to increase influenza vaccination in the pediatric inpatient setting are rare. Our project demonstrated that provider reminders led to an approximately 30% increase in influenza vaccine orders among pediatric inpatients on the medical unit, which was sustained over the season. Provider reminders in the form of e-mails, data reports, and visual aids were more successful than family education and prompts in the EMR alone. Such reminders are simple, sustainable, enhance existing features in the EMR, and are a useful method of increasing influenza vaccination rates among pediatric inpatients.

One of the strengths of our initiative was that we used results from local survey data to help inform the nature of our interventions to increase influenza vaccination rates in our population. These data demonstrated that influenza vaccination is a high priority for physicians, but lack of time, forgetting to order the vaccine and provider belief of caregiver misconceptions about the vaccine are important reasons for failure to order the vaccine in an inpatient setting. Additionally, parents understood the seriousness of influenza infection and were open to influenza vaccination during their child’s hospitalization.²⁵ We, therefore, directed our interventions to provider reminders and vaccination reports to aid ordering processes, and family education materials regarding the importance of vaccination and the safety of vaccination during hospitalization. We chose a multi-faceted approach, augmenting features already in existence in our EMR, because studies among children and adults demonstrate that interventions incorporating system-wide approaches, such as patient and provider education and reminders were effective in increasing vaccination rates in inpatient and outpatient settings.^29,30

Our project demonstrated that provider reminders were effective in increasing influenza vaccine ordering for inpatients and increasing vaccination rates in our population. The results of this initiative are consistent with other studies in the literature,^{15,16,31–34} with evidence suggesting that provider reminders are a more effective strategy than provider or caregiver education.^35,36 Furthermore, the use of prospective audit and feedback among providers has been shown to be effective in increasing vaccination rates in studies including the inpatient setting.³⁷ There was an increase in ordering before the intervention, which reflected the introduction of the new seasonal influenza vaccine at the hospital. For this reason, to measure the effectiveness of our intervention, we did not just compare pre- and intervention vaccination orders, we also included a usual care control group for comparison. We anticipated even higher vaccine ordering rates, given the nature of our interventions, and explored potential barriers that exist in the inpatient setting identified in our provider surveys.²⁵ We found that reluctance to vaccinate during illness or the perception that vaccination should be a task reserved for the primary medical home were not widely-held beliefs. Therefore, the reasons for the observed rates warrant further exploration.

We used 2 main metrics for our project, inpatient influenza vaccine orders and overall vaccination status for several reasons. First, as the influenza vaccine season progresses, vaccination rates increase, reflecting vaccination in primary care and other settings, which may lead to a decrease in inpatient vaccine orders. Therefore, to determine whether vaccination rates were impacting inpatient orders, we chose to include both measures. Second, overall vaccination status is the more practical measure, and increasing vaccination rates in our population is the overarching goal of the project. Finally, this metric is an established measure recommended for hospitalized patients by the National Quality Forum.³⁸

Family education was not as effective as provider reminders, with a decrease in vaccine orders among this group toward the latter third of the study period. This decrease occurred despite an increased positive attitude toward influenza vaccination, as evidenced by our pre- and intervention survey data. Several reasons may account for decreased rates in the family education group. The Hawthorne effect may have contributed to the initial improved vaccination rates in this group, given that this effect is usually greatest soon after the beginning of an intervention. Next, although our pre- and intervention survey responses demonstrate a more favorable attitude toward influenza vaccination, it is not known how many caregivers reviewed the handout. Finally, unvaccinated children toward the end of the season may include a higher proportion of vaccine-hesitant families, who may be less likely to be impacted by education materials.

We evaluated 2 important process and balancing measures in our project. We relied on influenza vaccination screening documentation obtained during admission, which is subject to recall bias. Our comparison of vaccination status from the admission note and vaccination status based on survey data showed a high degree of correlation, but we did not directly compare it to immunization data contained within the Colorado Immunization Information System database, which provides vaccination data for approximately 85% of the state of Colorado. Our EMR links to the state registry, and many providers review the immunization data when entering vaccination status on admission, so this information is likely to be more accurate than parent recall alone. Our project also demonstrated that there was no difference in length of stay among the groups, and the intervention groups were similar with respect to the time of discharge, which is an important consideration when implementing such an initiative in the inpatient setting.

Several limitations warrant discussion. We conducted our initiatives in the setting of QI. Therefore, its application to other settings may be limited. There was some crossover of medical staff (residents and attending physicians) between medical teams during our study period, which may have positively impacted orders in the caregiver education and control groups, with the potential to diminish the differences observed between the provider and other groups. Finally, restricting the study to 1 medical team in each group narrowed our sample size; however, including other teams (eg, medical subspecialty, rehabilitation, and surgical teams) may have introduced more confounding due to a greater variation in patient comorbidities, diagnoses, and provider practices.

In conclusion, this project demonstrates the feasibility of a multi-faceted intervention to increase and sustain influenza vaccination ordering among pediatric inpatients, which enhances automated features in the EMR. Such interventions can be adapted for use in the primary care, subspecialty or inpatient setting, without significant additional resources. These interventions have the potential to minimize the burden of disease, influenza-related hospitalization rates, and length of stay, and reduce health care costs.

ACKNOWLEDGMENTS

The authors are grateful to the patients, families, nurses, physicians, residents, and nurse practitioners who participated in the study. The authors thank Angela Moss and Sonja Ziniel for consultation regarding data analysis. The authors thank Michael Rannie and Lloyd Provost for input regarding the Statistical Process Control Charts.