Abstract
Introduction:
National Immunization Survey-Child data are used widely to assess childhood vaccination coverage in the U.S. This study compares National Immunization Survey-Child coverage estimates with estimates using other supplementary data sources.
Methods:
Retrospective analyses in 2021 assessed vaccination coverage of privately insured children for vaccines recommended by the Advisory Committee on Immunization Practices by age 2 years, using the 2015–2018 MarketScan Commercial Claims and Encounters databases and the 2018–2019 Healthcare Effectiveness Data and Information Set. The coverage estimates were compared statistically with those using the 2016–2018 National Immunization Survey-Child.
Results:
Estimated coverage ranged from 69.9% (≥2 doses of influenza vaccine) to 95.0% (≥3 doses of diphtheria, tetanus toxoids, and acellular pertussis vaccine) using the MarketScan Commercial Claims and Encounters data and from 68.0% (≥2 doses of influenza vaccine) to 92.2% (≥1 dose of measles, mumps, and rubella vaccine) using the Healthcare Effectiveness Data and Information Set. The difference between the MarketScan Commercial Claims and Encounters and National Immunization Survey-Child estimates ranged from 0.1 to 4.3 percentage points and was statistically significant for 6 of the 13 assessed vaccines/doses and percentage of children receiving no vaccinations. The difference between the Healthcare Effectiveness Data and Information Set and National Immunization Survey-Child estimates ranged from 0.4 to 7.2 percentage points and was statistically significant for 6 of the 10 assessed vaccines/doses.
Conclusions:
For certain vaccines and populations of interest, the National Immunization Survey-Child, MarketScan Commercial Claims and Encounters, and Healthcare Effectiveness Data and Information Set data might give comparable coverage of privately insured children.
INTRODUCTION
The Advisory Committee on Immunization Practices recommends that children be vaccinated against 14 potentially serious illnesses.1 Routine childhood vaccination coverage is generally high in the U.S.2–8 The National Immunization Survey-Child (NIS-Child), a random-digit-dial telephone survey with provider validation created by the Centers for Disease Control and Prevention, has been used to assess childhood vaccination coverage since 1994.2–4 Vaccination coverage estimates from survey data are usually higher than estimates from administrative data (e.g., insurance claims).9,10 NIS-Child estimates may be biased because of threats such as incomplete sample frame, nonresponse bias, or errors in classification of vaccination status,11 whereas estimates using administrative data may be biased because vaccinations outside the administrative system are not captured.9,10 Survey and administrative data have perceived strengths and limitations; neither represent a clear gold standard.
To the best of the authors’ knowledge, this study is the first to evaluate NIS-Child and other supplementary data sources, including MarketScan Commercial Claims and Encounters (CCAE) and the Healthcare Effectiveness Data and Information Set (HEDIS), for childhood vaccination assessment. Although NIS-Child is used widely, researchers increasingly turn to alternative data for more timely access. Findings help to understand the trade-offs between various data sources and provide additional information about current childhood vaccination coverage.
METHODS
The 2015–2018 MarketScan CCAE and the 2018–2019 HEDIS data were used to assess coverage with Advisory Committee on Immunization Practices–recommended vaccines among children by age 24 months (Appendix Table 1, available online). The estimated coverage was compared with previously published 2016–2018 NIS-Child estimates among privately insured children (n=12,702).2
The MarketScan CCAE is a private health insurance claims database from IBM Watson Health, with 273 million enrollees since 1995 from >120 private-sector employers and 40 health insurance plans.12 Children born in 2015–2016 who were continuously enrolled in the data from birth through age 24 months and met inclusion criteria were included (Appendix A, available online) (n=135,721), representing a segment of children in Washington, District of Columbia and 49 states (Hawaii excluded) (Appendix Table 2, available online). Vaccinations were identified from inpatient or outpatient claims that had Current Procedural Terminology codes or Healthcare Common Procedure Coding System codes indicating vaccine administration and products (Appendix B, available online). Some children (n=18,016; 13.3%) had vaccine administration without information on vaccine products (Appendix Table 3, available online). The missing information was imputed from children in the same state with complete vaccination information by mean imputation.
The HEDIS data are used for quality assessment by the National Committee for Quality Assurance for managed care health plans that cover about 191 million enrollees, including coverage for a series of childhood vaccines by age 24 months, submitted by insurance companies on the basis of data collected from claims, medical records, or surveys of members and providers (Appendix C, available online).13 The observation unit is vaccination coverage among a preselected sample born in 2015–2016 for each private insurance plan. The analytical HEDIS sample included 348 plans for 2018 and 363 plans for 2019, with 442,774 children aged 2 years from Washington, District of Columbia and 50 states (Appendix Table 4, available online). Overall vaccination coverage was calculated as the weighted average coverage, using the total enrollment as the weight.
There were several important differences across coverage measurement or data (Appendix D). For sensitivity analyses, the coverage was re-estimated using MarketScan CCAE that excluded the children with missing information on vaccine products. Moreover, HEDIS included managed care plans only, and NIS-Child had no information on plan types. MarketScan CCAE were used to examine whether the coverage among children in managed care plans was different from the overall coverage.
Differences in coverage estimates between data sets were assessed in percentage points by 2-sample z-tests for proportions. This secondary study did not require IRB approval.
RESULTS
Coverage from MarketScan CCAE ranged from 69.9% (95% CI=69.7, 70.2) for ≥2 doses of influenza vaccine to 95.0% (95% CI=94.9, 95.2) for ≥3 doses of diphtheria, tetanus toxoids, and acellular pertussis vaccine (DTaP). Coverage from HEDIS ranged from 67.3% (95% CI=67.0, 67.6) for ≥2 doses of influenza vaccine to 92.2% (95% CI=92.0, 92.4) for ≥1 dose of measles, mumps, and rubella vaccine (MMR). Coverage from NIS-Child ranged from 68.5% (95% CI=66.6, 70.4) for ≥2 doses of influenza vaccine to 96.9% (95% CI=96.3, 97.5) for ≥3 doses of DTaP (Table 1).
Table 1.
Childhood Vaccination Coverage Among Privately Insured Children by Age 24 Months, % (95% CI)
| Vaccinea | NIS-Child (n=12,702)b | MarketScan CCAE (n=135,721) | Difference (NIS-Child–MarketScan CCAE)c | HEDIS (n=711) | Difference (NIS-Child–HEDIS)c |
|---|---|---|---|---|---|
|
| |||||
| DTaP | |||||
| ≥3 doses | 96.9 (96.3, 97.5) | 95.0 (94.9, 95.2) | 1.9 (1.3, 2.5) | d | |
| ≥4 doses | 87.1 (85.7, 88.5) | 85.9 (85.7, 86.1) | 1.2 (−0.2, 2.6) | 85.7 (85.4, 85.9) | 1.4 (0.0, 2.9) |
| IPV (≥3 doses) | 96.1 (95.4, 96.7) | 91.8 (91.7, 92.0) | 4.3 (3.6, 4.9) | 90.7 (90.5, 90.9) | 5.4 (4.7, 6.1) |
| MMR (≥1 dose) | 93.7 (92.8, 94.5) | 92.5 (92.3, 92.7) | 1.2 (0.3, 2.1) | 92.2 (92.0, 92.4) | 1.5 (0.6, 2.4) |
| Hib | |||||
| Primary seriese | 95.7 (94.5, 96.8) | 94.6 (94.5, 94.8) | 1.1 (−0.1, 2.2) | d | |
| Full seriesf | 85.5 (83.7, 87.1) | 83.4 (83.1, 83.6) | 2.1 (0.4, 3.9) | d | |
| ≥3 doses | 94.8 (93.6, 95.9) | 93.3 (93.1, 93.4) | 1.5 (0.4, 2.7) | 91.5 (91.3, 91.6) | 3.3 (2.2, 4.5) |
| VAR (≥1 dose) | 93.2 (92.3, 94.0) | 92.2 (92.1, 92.4) | 1.0 (0.1, 1.8) | 92.0 (91.8, 92.1) | 1.2 (0.4, 2.1) |
| PCV | |||||
| ≥3 doses | 94.9 (93.5, 96.0) | 94.4 (94.3, 94.6) | 0.5 (−0.8, 1.7) | d | |
| ≥4 doses | 87.3 (85.6, 88.8) | 86.9 (86.7, 87.1) | 0.4 (−1.2, 2.0) | 85.9 (85.7, 86.1) | 1.4 (−0.2, 3.0) |
| HepA (≥1 dose) | 87.5 (85.9, 89.0) | 87.6 (87.4, 87.8) | −0.1 (−1.7, 1.4) | 87.6 (87.4, 87.8) | −0.1 (−1.7, 1.4) |
| HepB (≥3 doses) | 93.0 (91.8, 94.0) | g | 85.8 (85.6, 86.0) | 7.2 (6.1, 8.3) | |
| RV (≥2 or ≥3 doses)h | 83.5 (81.9, 85.0) | 84.7 (84.5, 84.9) | −1.2 (−2.8, 0.4) | 82.5 (82.2, 82.7) | 1.0 (−0.5, 2.6) |
| Influenza (≥2 doses)i | 68.5 (66.6, 70.4) | 69.9 (69.7, 70.2) | −1.4 (−3.4, 0.5) | 67.3 (67.0, 67.6) | 1.2 (−0.7, 3.1) |
| No vaccinations | 0.8 (0.6, 1.0) | 1.7 (1.7, 1.8)j | −0.9 (−1.2, −0.7) | d | |
Note: 2016–2018 NIS-Child data, 2015–2018 MarketScan CCAE data, and 2018–2019 HEDIS data are reported.
Vaccines received by age 24 months, except for RV in NIS-Child and MarketScan CCAE data, which needed to be received by age 8 months.
Coverage was estimated using Kaplan–Meier analysis for children who were aged <24 months on the date vaccination status was assessed.
The 95% CIs were calculated on the basis of 2-sample z-tests for proportions.
HEDIS data were not available for the assessed vaccines/doses.
A total of ≥2 doses of PedvaxHib or COMVAX or ≥3 doses of other Hib vaccines.
A total of ≥3 doses of PedvaxHib or COMVAX or ≥4 doses of other Hib vaccines.
Not assessed using MarketScan CCAE data because the birth dose of HepB was largely missing.
A total of ≥2 doses of Rotarix monovalent rotavirus vaccine or ≥3 doses of RotaTeq pentavalent rotavirus vaccine, received by age 8 months in NIS-Child and MarketScan CCAE data.
Might be received over 2 influenza seasons. In NIS-Child and MarketScan CCAE data, doses must be at least 24 days apart.
Children receiving ≥1 dose of HepB were counted as receiving vaccinations.
DTaP, diphtheria, tetanus toxoids, and acellular pertussis vaccine; HEDIS, Healthcare Effectiveness Data and Information Set; HepA, hepatitis A vaccine; HepB, hepatitis B vaccine; Hib, Haemophilus influenzae type b conjugate vaccine; IPV, inactivated poliovirus vaccine; MarketScan CCAE, MarketScan Commercial Claims and Encounters; MMR, measles, mumps, and rubella vaccine; NIS-Child, National Immunization Survey-Child; PCV, pneumococcal conjugate vaccine; RV, rotavirus vaccine; VAR, varicella vaccine.
Many of the coverage estimates from MarketScan CCAE and HEDIS were within 3 percentage points of the corresponding NIS-Child estimates (Appendix Figure 1, available online). The difference between NIS-Child and MarketScan estimates was statistically significant for 6 vaccines/doses: ≥3 doses of DTaP (1.9, 95% CI=1.3, 2.5), ≥3 doses of inactivated poliovirus vaccine (IPV) (4.3, 95% CI=3.6, 4.9), ≥1 dose of MMR (1.2, 95% CI=0.3, 2.1), ≥3 doses of any Haemophilus influenzae type b conjugate vaccine (Hib) (1.5, 95% CI=0.4, 2.7), full series of Hib (2.1, 95% CI=0.4, 3.9), and ≥1 dose of varicella vaccine (1.0, 95% CI=0.1, 1.8). The difference in the percentage of children without vaccinations was also significant (−0.9, 95% CI= −1.2, −0.7) (Table 1). Coverage of ≥3 doses of hepatitis B vaccine (HepB) was not assessed using MarketScan CCAE.
The difference between NIS-Child and HEDIS estimates was statistically significant for 6 vaccines/doses: ≥4 doses of DTaP (1.4, 95% CI=0.0, 2.9), ≥3 doses of IPV (5.4, 95% CI=4.7, 6.1), ≥1 dose of MMR (1.5, 95% CI=0.6, 2.4), ≥3 doses of any Hib (3.3, 95% CI=2.2, 4.5), ≥1 dose of varicella vaccine (1.2, 95% CI=0.4, 2.1), and ≥3 doses of HepB (7.2, 95% CI=6.1, 8.3) (Table 1). Coverage of ≥3 doses of DTaP, primary and full series of Hib, and ≥3 doses of pneumococcal conjugate vaccine were not assessed using HEDIS.
The MarketScan estimates that excluded children with missing information on vaccine products were similar to the main estimates. The estimates among only children in managed care plans in the MarketScan CCAE sample were also similar to the overall estimates (Appendix Table 5, available online).
DISCUSSION
Vaccination coverage among privately insured children estimated from MarketScan CCAE and HEDIS was similar to the NIS-Child estimates, in line with the findings of recent studies.3,14 The differences between the coverage estimates were small for most vaccines/ doses, with some significant differences noted for ≥3 doses of IPV, ≥3 doses of any Hib, and ≥3 doses of HepB. Possible reasons include lower uptake of certain individual vaccines (IPV and Hib) and missing birth dose (HepB) in claims.
The comparisons indicate that the bias in NIS-Child estimates may be small. Coverage estimates from MarketScan CCAE may be biased because claims do not capture vaccinations not submitted for reimbursement.15 That potential bias may also affect estimates from HEDIS, a proportion of which are based on information collected from claims. Nevertheless, the differences among the 3 data sources are small for most assessed vaccines. Therefore, for vaccines that are unlikely missing in claims or medical records, claims or HEDIS data might be a reliable source to assess coverage among privately insured young children, especially for those covered by plans included in the data (Appendix D, available online).
Limitations
Missing vaccine information in MarketScan CCAE was imputed. However, the sensitivity analysis showed no impact on the coverage estimates. The comparison between NIS-Child and HEDIS estimates might be biased if children on managed care plans in NIS-Child had different vaccination coverage from that of the total sample. The sensitivity analysis indicated that the results were unlikely to be driven by children not covered by managed care plans in NIS-Child. There might be temporal differences between the data; however, they were unlikely to drive the results because only 2 birth cohorts were included. There were no sufficient data to investigate how the comparisons reflected the differences in sample characteristics.
CONCLUSIONS
This assessment shows that national vaccination coverage among privately insured children by age 24 months was generally high. Despite concerns regarding bias in survey and administrative data, this study shows small differences in coverage estimates among NIS-Child, MarketScan CCAE, and HEDIS for most assessed vaccines that are sufficiently captured in claims or medical records—indicating that for those vaccines, these 3 types of data could be used to assess coverage among privately insured young children in the U.S. Future studies could explore other types of data, such as electronic health records.
Supplementary Material
ACKNOWLEDGMENTS
The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
No financial disclosures were reported by the authors of this paper.
Footnotes
CREDIT AUTHOR STATEMENT
Kai Hong: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Writing - original draft, Writing - review and editing. Holly A. Hill: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing - review and editing. Yuping Tsai: Conceptualization, Data curation, Methodology, Validation, Writing - review and editing. Megan C Lindley, Conceptualization, Validation, Writing - review and editing. Fangjun Zhou: Conceptualization, Data curation, Project administration, Validation, Writing - review and editing.
SUPPLEMENTAL MATERIAL
Supplemental materials associated with this article can be found in the online version at https://doi.org/10.1016/j.amepre.2022.01.020.
REFERENCES
- 1.Robinson CL, Bernstein H, Poehling K, Romero JR, Szilagyi P. Advisory Committee on Immunization Practices Recommended Immunization Schedule for Children and Adolescents Aged 18 Years or Younger - United States, 2020. MMWR Morb Mortal Wkly Rep. 2020;69(5):130–132. 10.15585/mmwr.mm6905a3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Hill HA, Singleton JA, Yankey D, Elam-Evans LD, Pingali SC, Kang Y.Vaccination coverage by age 24 months among children born in 2015 and 2016 - National Immunization Survey-Child, United States, 2016–2018. MMWR Morb Mortal Wkly Rep. 2019;68(41):913–918. 10.15585/mmwr.mm6841e2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Hill HA, Yankey D, Elam-Evans LD, Singleton JA, Pingali SC, Santibanez TA. Vaccination coverage by age 24 months among children born in 2016 and 2017 - National Immunization Survey-Child, United States, 2017–2019. MMWR Morb Mortal Wkly Rep. 2020;69(42):1505–1511. 10.15585/mmwr.mm6942a1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Hill HA, Elam-Evans LD, Yankey D, Singleton JA, Kang Y. Vaccination coverage among children aged 19–35 months - United States, 2017. MMWR Morb Mortal Wkly Rep. 2018;67(40):1123–1128. 10.15585/mmwr.mm6740a4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Lin X, Fiebelkorn AP, Pabst LJ. Trends in compliance with two-dose influenza vaccine recommendations in children aged 6 months through 8 years, 2010–2015. Vaccine. 2016;34(46):5623–5628. 10.1016/j.vaccine.2016.09.037. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Murphy TV, Denniston MM, Hill HA, et al. Progress toward eliminating hepatitis A disease in the United States. MMWR Suppl. 2016;65 (1):29–41. 10.15585/mmwr.su6501a6. [DOI] [PubMed] [Google Scholar]
- 7.Seither R, Loretan C, Driver K, Mellerson JL, Knighton CL, Black CL.Vaccination coverage with selected vaccines and exemption rates among children in Kindergarten - United States, 2018–19 school year. MMWR Morb Mortal Wkly Rep. 2019;68(41):905–912. 10.15585/mmwr.mm6841e1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Zhai Y, Santibanez TA, Kahn KE, Srivastav A. Parental-reported full influenza vaccination coverage of children in the U.S. Am J Prev Med. 2017;52(4):e103–e113. 10.1016/j.amepre.2016.10.040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Lochner KA, Wynne MA, Wheatcroft GH, Worrall CM, Kelman JA. Medicare claims versus beneficiary self-report for influenza vaccination surveillance. Am J Prev Med. 2015;48(4):384–391. 10.1016/j.amepre.2014.10.016 [DOI] [PubMed] [Google Scholar]
- 10.Jiménez-García R, Hernandez-Barrera V, Rodríguez-Rieiro C, et al. Comparison of self-report influenza vaccination coverage with data from a population based computerized vaccination registry and factors associated with discordance. Vaccine. 2014;32(35):4386–4392. 10.1016/j.vaccine.2014.06.074. [DOI] [PubMed] [Google Scholar]
- 11.Centers for Disease Control and Prevention, National Center for Immunization and Respiratory Diseases. National immunization survey-Child: a user’s guide for the 2019 public-use data file. Atlanta, GA: Centers for Disease Control and Prevention, National Center for Immunization and Respiratory Diseases. https://www.cdc.gov/vaccines/imz-managers/nis/downloads/NIS-PUF19-DUG.pdf. Published December 2020. Accessed August 8, 2021. [Google Scholar]
- 12.IBM MarketScan Research Databases for Life Sciences Researchers.Somers, NY: IBM Watson Health; 2021. https://www.ibm.com/downloads/cas/OWZWJ0QO; Published July 2021. Accessed March 12, 2022. [Google Scholar]
- 13.HEDIS and performance measurement. NCQA; 2021. https://www.ncqa.org/hedis/. Updated July 2020. Accessed March 15, 2021. [Google Scholar]
- 14.Hill HA, Yankey D, Elam-Evans LD, Singleton JA, Sterrett N. Vaccination coverage by age 24 months among children born in 2017 and 2018 - National Immunization Survey-Child, United States, 2018‒2020. MMWR Morb Mortal Wkly Rep. 2021;70(41):1435–1440. 10.15585/mmwr.mm7041a1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Zhou F, Lindley MC. Variability in influenza vaccination opportunities and coverage among privately insured children. Vaccine. 2020;38 (41):6464–6471. 10.1016/j.vaccine.2020.07.061 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.