Estimating Vaccine Hesitancy in Colorado by Using Immunization Information System Data

Abstract

Objectives:

Vaccine hesitancy is a complex issue that threatens global health. We used data from the Colorado Immunization Information System (CIIS) to quantify vaccine hesitancy.

Methods:

We examined immunization records from CIIS for patients age 2 up to 9 months to estimate vaccine hesitancy by tabulating the number of doses received per visit and comparing it with the number of expected doses based on recommendations of the Advisory Committee on Immunization Practices. We calculated the percentage of patients in each vaccine hesitancy group who were up to date on the 7-antigen series by age 35 months. We examined the distribution of vaccine-hesitant populations among vaccination providers who report to CIIS to estimate the difference in vaccine-hesitant patient populations among vaccination providers in Colorado.

Results:

Of 201 450 patients, 5147 (2.6%) consistently limited the number of shots received at each visit as compared with recommendations from the Advisory Committee on Immunization Practices; 166 927 (82.9%) patients did not limit the number of shots received; 5693 (2.8%) limited the number of shots received at >1 visit but not all visits; and 23 683 (11.8%) limited the number of shots received at only 1 visit. We found differences in vaccine hesitancy distributions among certain Colorado vaccination providers.

Conclusions:

Immunization information system data, although sometimes incomplete, offer an opportunity to investigate state-level vaccine hesitancy. Areas of future research include performing similar analyses over time and determining geographic and socioeconomic factors that contribute to vaccine hesitancy.

Since their inception, vaccines have played a crucial role in preventing diseases. Creating immunity to a disease through vaccination is a powerful and cost-effective method to prevent morbidity and mortality among children and adults. Vaccinating children protects individuals and communities because certain people cannot receive vaccines, including people who cannot receive vaccines for medical reasons and children who are too young to be vaccinated. The vaccination schedule recommended by the Advisory Committee on Immunization Practices (ACIP) outlines immunization recommendations for children, adolescents, and adults. ¹ From age 2 up to 9 months, ACIP recommends vaccinations for hepatitis B, rotavirus, DTaP (diphtheria, tetanus, and acellular pertussis), Haemophilus influenzae type b (Hib), pneumococcal conjugate 13 vaccine, and inactivated poliovirus vaccine (IPV), as well as an inactivated influenza vaccination at age ≥6 months. The Centers for Disease Control and Prevention publishes the ACIP schedule, which is endorsed by the American Academy of Pediatrics and the American Academy of Family Physicians. ² ACIP recommendations are scientifically backed and evidence based.

Although the ACIP recommended schedule is followed by many parents, some choose alternate immunization schedules or choose not to vaccinate their children. Alternative immunization schedules are not developed with the same scientific standards as the ACIP schedule. Reasons given by parents for not following the ACIP schedule include a lack of confidence in the safety or effectiveness of the immunization process and feelings of complacency because vaccination has effectively eradicated some diseases.^{3 -5} In the United States, these attitudes have led to consequences such as a recent decline in coverage for the measles, mumps, rubella (MMR) vaccine and a steep rise in measles cases. ⁶ The World Health Organization categorized vaccine hesitancy as 1 of the top 10 threats to global health in 2019. ⁷

State vaccination policies in Colorado allow parents to exempt their children from receiving vaccinations required to attend school and childcare for medical and nonmedical reasons.^8,9 Based on 2019 National Immunization Survey results, the percentage of children in Colorado who had received ≥1 MMR vaccine by age 19 months was 88.4%, which is below the 92%-94% community threshold needed for protection against measles. ¹⁰

The Colorado Department of Public Health and Environment (CDPHE) maintains the Colorado Immunization Information System (CIIS), a confidential, population-based, secure computerized system that collects and consolidates person-level vaccine and exemption data for residents in Colorado of all ages from various sources. ¹¹ Per an Internal Immunization Information Systems Annual Report submitted to the Centers for Disease Control and Prevention, the number of people in CIIS exceeded Colorado’s state population, according to 2020 US Census data (113%) (unpublished report, CDPHE, 2020).

Despite the relative maturity of CIIS, vaccine hesitancy is a poorly quantified issue in Colorado, in part because CIIS has historically been a voluntary reporting system. ⁹ However, the importance of estimating hesitancy based on these data cannot be understated. ¹² Immunization information system (IIS) data, although in some cases weakened by reporting gaps and missing data, have important benefits when compared with survey data. ¹³ IIS data allow public health to learn important information about vaccination coverage across large areas without limitations related to time or resources. Historically, related research projects have primarily used qualitative methods such as self-reported survey data to quantify the prevalence of alternative vaccination schedule use.^14,15 Few studies that use IIS data to examine undervaccination exist, suggesting that this resource has been used infrequently.^{16 -19} To our knowledge, quantifying vaccine hesitancy is a relatively novel use of IIS data.

The combination of IIS data and the recommended ACIP schedule provides a unique opportunity to evaluate vaccine hesitancy. Studying vaccine hesitancy in Colorado is especially important in the context of recent legislation. Senate Bill 20-163, which was signed in 2020, updated CIIS to a mandatory reporting system and standardized the nonmedical exemption process for school-required vaccines. ⁹ Historically, no official process or standard form has been required to report exemption information to CDPHE.

Even with these improvements, public health agencies’ understanding of vaccine hesitancy based on IIS data may be affected by record incompleteness. Some vaccination records in CIIS are not up to date because the patient has not received recommended vaccines. Others are not up to date because the records are incomplete (ie, missing vaccines that the patient has actually received) or split between ≥2 records because of duplicate records. Even when a vaccination provider regularly submits vaccination data to CIIS, data transfer delays and failures are possible. While the extent of incomplete records in CIIS is poorly understood, it is likely that incomplete records affect CDPHE’s ability to accurately evaluate immunization coverage with CIIS data. Additionally, patient migration is a large source of change in Colorado data, resulting in incomplete, active CIIS records persisting after a patient has moved out of Colorado. These challenges in Colorado and other jurisdictions have contributed to limitations in conducting research about the use of immunization records to estimate vaccine hesitancy.

Although CDPHE is likely missing data on children exempted from vaccines, we are still able to make inferences about vaccine hesitancy by studying vaccination records in CIIS. In one of the first studies to use IIS data to quantify the proportion of children consistently delaying receipt of vaccines, Robison et al analyzed IIS data in Oregon to describe the prevalence of the use of alternative schedules, categorizing patients as non–shot limiters (ie, patients who did not limit the number of shots received), episodic shot limiters (ie, patients who limited the number of shots received during >1 visit but not all visits), or consistent shot limiters (ie, patients who limited the number of shots received at every visit). ²⁰ A subsequent analysis reproducing these hesitancy categorizations with CIIS patients indicated an increase in the number of Colorado counties with ≥10% of their children following an alternative vaccination schedule in 2016 vs 2007, thus confirming vaccine hesitancy as an issue in Colorado requiring further study (unpublished report, CDPHE, 2016).

The quantitative classification of children into hesitancy categories is an important step toward increasing vaccination coverage across Colorado. Increased knowledge about vaccine-hesitant populations in Colorado would allow surveillance, provider outreach efforts, and public health education campaigns to be efficiently directed toward populations at higher-than-average risk of vaccine-preventable diseases. In the pursuit of these initiatives, our analysis used CIIS data to estimate the relative proportions of Colorado children age 2 up to 9 months in various categories of vaccine hesitancy. The primary objective was to build on work by Robison et al to establish a baseline method to evaluate relative levels of vaccine hesitancy in Colorado.

Methods

In June 2020, we selected a study population that included children with records in CIIS who were born from March 3, 2013, through March 1, 2017 (patients aged ≥3 to <7 years on March 1, 2020). We chose this date range because it was before the observed decrease in childhood vaccinations that occurred during the COVID-19 pandemic beginning mid-March 2020.^21,22 We excluded patients who had moved out of Colorado, died, or had residential addresses outside Colorado. The CDPHE Institutional Review Board determined that this project did not meet the definition of human subjects research and, thus, did not require prior review and approval.

We excluded patients without ≥2 non–influenza visits recorded in CIIS before age 9 months (274 days). We chose these criteria to decrease the likelihood that patients with incomplete records whose migration out of Colorado had not been captured in CIIS were included in the study. The American Immunization Registry Association’s Modeling of Immunization Registry Operations Workgroup recommends that IIS-based analyses exclude patients with no contact with their records in ≥10 years, reasoning that these patients have likely migrated out of the IIS jurisdiction. ²³ Excluding patients without ≥2 noninfluenza visits recorded in CIIS before age 9 months builds on workgroup recommendations to be more relevant for a younger patient cohort.

We sourced complete immunization records for the population from age 2 months (61 days) up to but not including age 9 months (273 days) at vaccination from CIIS. We included data on doses of hepatitis B, rotavirus, DTaP, Hib, pneumococcal conjugate 13 vaccine, IPV, and inactivated influenza vaccination. We grouped the cohort into 4 vaccine hesitancy categories based on the number of unique shots that a patient received in a single visit, defined as a unique vaccination date in CIIS. ²⁰ Because a child may be up to date on all recommended vaccines except influenza at a given visit, we excluded data on single-dose visits during which a patient received influenza vaccine.

Consistent shot limiters received ≤2 shots for all visits from age 2 up to 9 months. Episodic shot limiters received ≤2 shots for >1 visit but not all visits. Non–shot limiters received >2 shots for all visits. One-time shot limiters received ≤2 shots for only 1 visit from age 2 up to 9 months. The definitions of consistent shot limiters, episodic shot limiters, and non–shot limiters are based on work by Robison et al. ²⁰

We analyzed the use of single-antigen vaccines compared with combination vaccines in each hesitancy group. This analysis controlled for the administering provider because providers may have only combination or single-antigen vaccines on hand for a given antigen. ²⁴ We excluded data on patients who received vaccines from >1 provider from age 2 up to 9 months. We grouped hesitancy categories into “likely vaccine hesitant” (consistent and episodic shot limiters) and “likely not vaccine hesitant”' (one-time shot limiters and non–shot limiters). We excluded providers from this analysis who had not administered both single-antigen and combination vaccines to both likely vaccine-hesitant and likely non–vaccine-hesitant patients. To estimate the association between vaccine hesitancy group and dose type received, we fit a logistic regression model to a random sample of 25 vaccination providers (118 927 doses), controlling for the administering provider. We calculated odds ratios and 95% CIs from the model coefficient for hesitancy category.

We determined the percentage of patients in each hesitancy group who had received a complete recommended vaccination series by age 35 months to reproduce National Immunization Survey–Child methodology, which specifies a 7-antigen series (4 doses of DTaP, 3 doses of IPV, 1 dose of MMR, 3 doses of Hib, 3 doses of hepatitis B, 1 dose of varicella, and 4 doses of pneumococcal conjugate vaccine). ¹⁰ We used the Pearson χ² test to determine the significance of the relationship between hesitancy category and up-to-date status.

To estimate the likelihood that certain vaccination providers in Colorado have a higher percentage of vaccine-hesitant patients than other vaccination providers in Colorado, we analyzed the distribution of the percentage of patients categorized as consistent shot limiters across 389 providers in CIIS. We excluded data on patients who received vaccines from >1 vaccination provider from age 2 up to 9 months and vaccination providers with <25 total patients. We calculated the proportion of providers whose patient population did not include any consistent shot limiters, the proportion whose patient population comprised >0% to 10% of consistent shot limiters, and the proportion whose patient population comprised >10% of consistent shot limiters.

We conducted all analyses using R version 4.0.3 (R Core Team).

Results

A total of 203 228 children in CIIS met the inclusion criteria. Of those, 201 450 children had received ≥1 immunization of DTaP, IPV, MMR, Hib, hepatitis B, varicella, or pneumococcal conjugate vaccine from age 2 up to 9 months. A total of 1 977 374 vaccines were administered to the cohort when they were age 2 up to 9 months. A mean of 9.8 and median of 11 vaccinations were administered per patient while they were age 2 up to 9 months.

The smallest number of patients (n = 5147, 2.6%) was categorized as consistent shot limiters, 5693 (2.8%) as episodic shot limiters, 23 683 (11.8%) as one-time shot limiters, and 166 927 (82.9%) as non–shot limiters (Table).

Table.

Patients in each vaccine hesitancy category, based on the number of vaccinations received per visit, children aged 2 up to 9 months, Colorado, 2020 ^a

Vaccine hesitancy category b	No. (%)
Total	201 450 (100.0)
Consistent	5147 (2.6)
Episodic	5693 (2.8)
One-time	23 683 (11.8)
Non–shot limiting	166 927 (82.9)

^aIncludes patients with ≥1 immunization of diphtheria, tetanus, and acellular pertussis; inactivated poliovirus; measles, mumps, rubella; Haemophilus influenzae type b; hepatitis B vaccine; varicella; or pneumococcal conjugate vaccine from age 2 up to 9 months. Patient vaccination records were sourced from the Colorado Immunization Information System. ¹¹

^bConsistent shot limiters are patients who limited the number of shots received at every visit as compared with the recommendations of the Advisory Committee on Immunization Practices. ¹ Episodic shot limiters limited the number of shots received at >1 visit but not all visits. One-time shot limiters limited the number of shots received at only 1 visit. Non–shot-limiting patients did not limit the number of shots received at any visit.

We found a significant association between vaccine hesitancy group and receipt of a single-antigen vaccine dose. The odds of receiving a single-antigen vaccine dose were 1.63 times higher among likely vaccine-hesitant patients than among likely non–vaccine-hesitant patients, controlling for the administering provider (odds ratio = 1.63; 95% CI, 1.45-1.85).

In examining the percentage of patients in each vaccine hesitancy category who had received the full 7-antigen series by age 35 months, consistent shot limiters had the lowest percentage (22.9%, n = 1179) and non–shot limiters had the highest percentage (78.7%, n = 131 431); 76.7% (n = 18 175) of one-time shot limiters and 67.3% (n = 3832) of episodic shot limiters were up to date (Figure 1). The relationship between hesitancy category and up-to-date status was significant (χ²₃ = 9015.8; n = 201 450; P < .001).

Figure 1.

Percentage of patients in each vaccine hesitancy category who had received the full 7-antigen series by age 35 months, Colorado, 2020: 4 doses of diphtheria, tetanus, and acellular pertussis; 3 doses of inactivated poliovirus; 1 dose of measles, mumps, and rubella; 3 doses of Haemophilus influenzae type b; 3 doses of hepatitis B vaccine; 1 dose of varicella; and 4 doses of pneumococcal conjugate vaccine. ¹¹ The denominator for each percentage is the total number of patients categorized in each category. Consistent shot limiters are patients who limited the number of shots received at every visit as compared with recommendations from the Advisory Committee on Immunization Practices. ¹ Episodic shot limiters limited the number of shots received at >1 visit but not all visits. One-time limiters limited the number of shots received at only 1 visit. Non–shot-limiting patients did not limit the number of shots received at any visit. Patient vaccination records were sourced from the Colorado Immunization Information System. ¹¹

Of 389 providers, 116 (29.8%) had no patients categorized as consistent shot limiters, 257 (66.1%) had a patient population comprised of >0% to 10% of consistent shot limiters, and 16 (4.1%) had a population comprised of >10% of consistent shot limiters (Figure 2).

Figure 2.

Proportion of 389 vaccination providers whose patient populations were comprised of 0% of consistent shot limiters, >0% to 10% of consistent shot limiters, and >10% of consistent shot limiters, Colorado, 2020. Consistent shot limiters are patients who limited the number of shots received at every visit as compared with recommendations from the Advisory Committee on Immunization Practices. ¹ Patients who received vaccines from >1 vaccination provider from age 2 to 9 months and vaccination providers with <25 total patients were excluded. Patient vaccination records were sourced from the Colorado Immunization Information System. ¹¹

Discussion

This evaluation contributes to our ability to identify vaccine-hesitant populations by using IIS data. The combination of the ACIP schedule and IIS vaccination records provides a unique opportunity to analyze schedule adherence and vaccine hesitancy. Methods that identify populations and providers with higher-than-average levels of vaccine hesitancy allow health care providers to improve the up-to-date status of their patients, therefore decreasing the risk for vaccine-preventable diseases.^{25 -27} This evaluation found that non–shot limiters and one-time shot limiters had the highest percentage of being up to date by age 35 months. This finding suggests that patients who fully or closely follow the recommended ACIP schedule are more likely to be up to date than those who follow alternative schedules.

This evaluation also found differences among providers in the proportion of their patient populations that fall into the consistent shot-limiting category. These findings support research suggesting that some providers may be more flexible than others in allowing parents to refuse recommended vaccinations for their children or follow an alternative immunization schedule.^26,27 A patient population consisting of >10% of consistent shot limiters could be used as a threshold to determine which providers would benefit most from outreach from Colorado public health agencies about communication strategies with vaccine-hesitant parents.

Limitations

There are limitations in interpreting analyses that rely on IIS data. First, it is difficult to confirm that IIS data contain complete immunization records because of patients who change health care providers and move out of state, resulting in unrecorded mobility within a state’s IIS. This unrecorded mobility can contribute to denominator inflation, a situation in which people who no longer reside in a geographic area are still included as active patients in the area. ²⁸ Second, information about visits during which a patient refused all vaccines is not recorded in the system. Our analysis was limited to visits in which patients received ≥1 vaccine; therefore, it may be biased against the detection of shot-limiting patients. Third, while CIIS data are a powerful tool for evaluating immunization rates and improving public health, they may not represent all vaccinations that Colorado residents have received. Finally, because only data from CIIS were evaluated, the findings of this project are not generalizable to other states’ IIS data. Multistate studies would be required to determine if our findings are reproducible.

Conclusion

IIS data are an important and largely untapped resource that can give public health a better understanding of vaccine-hesitant populations in Colorado. The potential to find and provide outreach to vaccine-hesitant populations is especially critical in the context of increases in COVID-19 vaccine availability starting in 2021 and decreases in routine vaccination observed during the COVID-19 pandemic. ²⁹

Our study details how the number of shots per visit during the time when childhood vaccinations are recommended can be used as an effective method for estimating vaccine hesitancy with IIS data. The significant association between likely vaccine-hesitant groups (consistent and episodic shot limiters) and the receipt of single-antigen vaccines could serve as additional validation for this method, because concern about the use of combination vaccines in vaccine-hesitant populations has been reported. ³⁰ We additionally found that some providers in Colorado have higher percentages of shot-limiting patients than other providers. Public health commonly influences vaccine acceptance through strong provider–patient-level relationships; with this knowledge, public health may be able to offer targeted outreach to providers with a tiered approach corresponding to patient levels of hesitancy, potentially resulting in more efficient vaccine-preventable disease control.^31,32

Future work related to this project includes performing similar analyses over time and determining geographic and socioeconomic factors that contribute to vaccine hesitancy. To further public health’s understanding of how vaccine hesitancy has changed in Colorado over time, one aspect of this future work will examine the distribution of vaccine hesitancy categories in patient cohorts during a 10-year period. Additional work is necessary to understand the geographic distribution of vaccine hesitancy in Colorado, including analyzing how factors such as county population level and distance from the nearest vaccinating provider relate to vaccine hesitancy categorization. Future research will also focus on determining how record incompleteness interferes with our ability to estimate vaccine hesitancy from IIS data.