Vaccination Coverage Quality Indicators (VCQI): a flexible open-source tool for survey data analysis

ABSTRACT

The World Health Organization (WHO) and Pan American Health Organization (PAHO) make statistical software scripts freely available to conduct standardized analysis of vaccination coverage surveys. The software and accompanying documentation are named the Vaccination Coverage Quality Indicators (VCQI). The software examines data from child vaccination records and caregiver recall to calculate indicators of vaccination access, coverage, continuity, and quality. VCQI produces output tables and figures that may be copied directly into reports and presentations, as well as survey response data sets augmented with child-level outcome indicator variables calculated according to WHO guidance. It can analyze data from routine immunization surveys, post-campaign coverage surveys, and maternal tetanus coverage surveys. VCQI may be used for either the primary analysis of data from new surveys or secondary analysis of earlier surveys. It can analyze data from USAID Demographic and Health Surveys, UNICEF Multiple-Indicator Cluster Surveys, or WHO Expanded Programme on Immunization surveys. Future versions of VCQI may be extended to analyze data from electronic immunization registries.

VCQI has been available as a set of Stata scripts since 2015, and as of 2025, the core routine immunization indicators are also available as a new R package named vcqiR.

To publicize VCQI’s features and capabilities and to celebrate the milestone of R package availability, the authors share this special report to describe the VCQI vision and design, discuss analyses and insights available from it, and map our vision of what the future may hold for flexible standardized analysis of vaccination coverage data.

INTRODUCTION

Audience

The World Health Organization (WHO) Vaccination Coverage Quality Indicators (VCQI – pronounced Vicki) is a collection of statistical computer scripts and associated documentation used to analyze vaccination coverage surveys (1). This report is written for specialists in vaccination program monitoring and evaluation who may commission, conduct, or analyze household surveys that measure vaccination coverage.

Motivation

VCQI has been available as a set of Stata programs since 2015 and has been used to analyze numerous surveys in Africa, Asia, Latin America, and the Caribbean (2–4). To make VCQI available to a larger global community of analysts, the Immunization Unit of the Pan American Health Organization (PAHO) recently commissioned an initiative to port the core routine immunization indicators from Stata to the free, powerful, popular, open-source statistical software named R. This R package, vcqiR, is now available from GitHub (5). VCQI and vcqiR are compatible with WHO vaccination coverage survey guidance (6).

Companion Case-Study Manuscript

In this same issue of the journal, we share a companion case-study manuscript that showcases vcqiR being used for secondary analysis of the 2022 Jamaica multiple-indicator cluster survey (MICS) (7). The MICS report for that survey included a standard set of vaccination coverage tables summarizing crude coverage, defined by WHO as any evidence of receiving a vaccine dose, either from the home-based record or vaccination card, or caregiver recall (6). The 2022 Jamaica MICS showed high coverage of doses due in the first year of life, but doses due at 18 months had much lower coverage during the coronavirus pandemic than were typically reported from administrative data (8). The VCQI analysis corroborates the MICS report coverage estimates and further analyzes vaccination dates for children ages 12–23 months and 24–35 months, identifying actionable insights concerning vaccination timeliness, including premature administration of some doses and important delays in others, as well as missed opportunities for simultaneous vaccination (MOSVs) (7).

Short History of Vaccination Coverage Indicators for Survey Data

Since the late 1970s, the WHO Expanded Programme on Immunization (EPI) has provided reference materials for designing and analyzing household surveys to estimate vaccination coverage (6, 9–13). From the early days of EPI, the standard WHO approach was to do a survey with 30 clusters per stratum selected with probability proportional to size, and then collect data from a quota sample of seven eligible respondents per cluster (14). To keep methods simple and accessible for district-level staff even in low-resource settings, a simplifying assumption was made that every person in the survey had the same probability of selection and that a self-weighted (i.e., unweighted) analysis was appropriate. Simple coverage indicators could be easily calculated by hand or with a spreadsheet. Coverage point estimates were calculated as the simple arithmetic mean of a binary variable coded 1 if the respondent had evidence of receiving the dose and 0 if they did not. Several aspects of this “EPI 30x7 cluster survey design” were thoughtfully critiqued over the years (15–18). Nevertheless, it was employed around the world to monitor and manage the progress of vaccination coverage levels toward EPI goals (19). It was a helpful and practical tool in the days of simple vaccination schedules, data collection with simple paper forms, and tabulation with pencils and calculators due to the limited availability of statistical resources to EPI staff.

Over time, computational algorithms were developed to analyze vaccination dates (20 and its supplement). In the 1980s, WHO sponsored the development of a program named Coverage Survey Analysis System (COSAS) to conduct standardized analyses using personal computers (21). Support for COSAS ended around the time Microsoft Windows was released in the 1990s. Between the final release of COSAS and the first release of VCQI, there was a two-decade period when WHO did not furnish software for standardized coverage survey analyses.

In 2015, WHO issued a draft update to the Vaccination Coverage Cluster Survey Reference Manual, which was finalized in 2018 (6). To complement that manual, WHO compiled a list of indicators by reviewing earlier WHO coverage survey manuals, PAHO manuals, and COSAS documentation (22) and identified a subset of documented indicators to be programmed into the first version of VCQI, which was built using Stata in 2015. Additional indicators and capabilities have been added over time and are freely available. PAHO recently sponsored an effort to port VCQI’s main routine immunization indicators to R and the resulting vcqiR package software was made publicly available in 2025 (5).

VCQI VISION AND DESIGN

Issues Addressed

VCQI was designed to address several challenges in coverage survey analysis:

• Numerous aspects of survey design and reporting vary from project to project. See Table 1.

• Although personnel from the Demographic and Health Surveys (DHS), MICS, and WHO have worked over the years to harmonize the way vaccination coverage surveys collect data, there are no standardized methods for naming or coding variables that hold survey responses or for calculating coverage indicators. There are small differences in how DHS, MICS, and VCQI process missing data, especially when a child has evidence of a later dose but no evidence of an earlier dose in the same series.

• Without a tool for standardized reporting, survey analysts either write computer code anew for each project or, more likely, reuse the code from earlier surveys. Both approaches can be stressful and error-prone, particularly under time constraints.

• Most surveys collect vaccination dates from home-based records (i.e., cards) or facility-based records (registers), but few surveys report outcomes that analyze those dates thoroughly. Incomplete analysis of vaccination date data limits the capacity to generate meaningful insights into the timeliness of childhood vaccination and the completeness of dose administration at every contact with the health system. For countries with high coverage outcomes, a detailed analysis of the dates can reveal areas of programmatic performance that would still benefit from improvement.

• Many surveys do not report confidence intervals alongside coverage estimates, and some report symmetric Wald-based intervals that are not appropriate for coverage estimates near 0% or 100% from modest sample sizes (23).

• DHS and MICS freely share survey microdata and stratum or reporting domain-level outcomes, but they do not share derived indicator variables at the respondent level or the code that was used to calculate them. In some cases, generic code templates are available but not the country- or survey-specific programs that were adapted from templates to construct the report tables.

TABLE 1. Points of variability managed by VCQI in vaccination coverage surveys and analysis.

Items that vary from project to project	Stata	R
Country vaccination schedule	✓	✓
Doses required to be fully vaccinated	✓	✓
Sample design	✓	✓
Vaccination evidence from card/recall/register	✓	✓
Evidence may or may not be sought from BCG scar	✓	✓
SIA evidence may or may not be sought from finger mark	✓	—
Ages of eligible respondents	✓	✓
Geographic strata (nesting up to 3 levels deep)	✓	✓
Demographic reporting domains	✓	✓
Order of strata in tables and figures	✓	✓
Special handling for doses in a series	2- and 3-dose	Up to 9-dose
Booster dose evidence may be shifted to count as primary	✓	✓
Which indicators to calculate and report	✓	✓
Weighted or unweighted analysis	✓	✓
Type of confidence interval calculated	✓	✓
Table titles and footnotes	✓	✓
Language of text in tables and figures	✓	✓
(e.g., English, French, Spanish, etc.)

Source: Original table for this publication.

Note: SIA = supplementary immunization activities.

Vision

Figure 1 illustrates the relationship between VCQI inputs, programs, outputs, and optional subsequent analysis of VCQI output. Drawing on our own experience addressing the aforementioned challenges, we designed VCQI to have the following characteristics:

FIGURE 1. VCQI overview.

Source: Original figure for this publication.

• a)Project agnostic – the central library of code (more than 300 Stata .ado files or 150 .R scripts) is general-purpose and not changed from project to project. Project-specific details can vary tremendously but are isolated within VCQI input data sets and a single control program (Stata .do file or .R script) that calls the functions in the central library.
• b)Documented – VCQI input data set variable names and coding schemes are structured and rigid (24). Every indicator has a technical specification that describes how to calculate it using the input variables (22) and how to interpret its output in plain language (25). Table 2 lists the indicators available in Stata and R in early 2026. Features of the control program are described in the user’s guides (26, 27).
• c)Flexible – Table 1 lists elements of coverage survey design and analysis that vary from project to project; it indicates that, although there are some indicators that have not been ported to R, the Stata and R versions of VCQI both accommodate a wide variety of survey and analysis options.¹
• d)Tested – We have tested VCQI and validated its output carefully using faux input data sets and compared computational output with the correct answers that are already known. We have also used VCQI to reproduce coverage estimates from numerous DHS and MICS surveys from around the world.
• e)Extensible – Most indicators are coded using relevant subsets of seven steps with one code function or program per step (Figure 2). It is straightforward to extend the pattern of steps to design and write specifications and code for additional indicators. Both the Stata and R versions are hosted on GitHub, so collaborators may contribute to the VCQI code base via the common practice of pull requests (28).

TABLE 2. Vaccination Coverage Quality Indicators (VCQI) available in Stata and R.

Indicator	Description	Stata	R
Summarize the Sample and Responses to Individual Survey Questions
DESC_01	Expected vs. observed sample: clusters, households, and respondents	✓	—
DESC_02	Response to multiple-choice question (e.g., education level) - select one	✓	✓
DESC_03	Response to multiple-choice question (e.g., reasons not vaccinated) - select multiple	✓	✓
DESC_04	Descriptive statistics for a continuous variable	Forthcoming	—
Routine Immunization – Coverage Indicators
RI_COVG_01	Crude coverage	✓	✓
RI_COVG_02	Valid coverage	✓	✓
RI_COVG_03	Fully vaccinated	✓	✓
RI_COVG_04	Not vaccinated	✓	✓
RI_COVG_05	Clusters with surprisingly low crude coverage	✓	—
Routine Immunization – Access to Vaccination Indicator
RI_ACC_01	Crude coverage for one dose	✓	—
Routine Immunization – Continuity of Vaccination Indicators
RI_CONT_01	Unweighted dropout between two crude doses	✓	✓
RI_CONT_01B	Weighted dropout between two crude doses	✓	✓
Routine Immunization – Quality of Vaccination Indicators
RI_QUAL_01	Card and register availability	✓	✓
RI_QUAL_02	Ever had a card	✓	✓
RI_QUAL_03	Percent of doses with dates that were invalid	✓	—
RI_QUAL_04	Percent of doses administered before a certain age	✓	—
RI_QUAL_05	Percent of later doses in a series administered before <threshold> days passed	✓	—
RI_QUAL_06	Percent of valid <dose> doses that were administered before the age of 12 months	✓	—
RI_QUAL_07B	Valid coverage if there had been no early doses and no MOSVs	✓	✓
RI_QUAL_08	Percent of visits with one or more MOSVs	✓	✓
RI_QUAL_09	Percent of children who experienced one or more MOSVs	✓	✓
RI_QUAL_12	Percent of later doses in a sequence administered after a particular interval	✓	—
RI_QUAL_13	Percent of children who receive <dose> by a certain age	✓	—
Routine Immunization – Cumulative Coverage Curves
RI_CCC_01	Unweighted cumulative coverage curves	✓	—
RI_CCC_02	Weighted cumulative coverage curves	✓	—
Routine Immunization – Cumulative Interval Curves
RI_CIC_01	Unweighted cumulative interval curves	✓	—
RI_CIC_02	Weighted cumulative interval curves	✓	—
Routine Immunization – Vaccination Coverage and Timeliness Charts
RI_VCTC_01	Vaccination coverage and timeliness stacked bar charts	✓	✓
Routine Immunization – Vaccination Date Data Quality Summaries
vcqi_ri_dose_assertlist	List dates that point to data quality problems	✓	—
date_tick_chk_01 to 05	Summarize concordance of vaccination evidence; reports dates changed to ticks	✓	—
Maternal Tetanus – Protected at Birth
TT_COVG_01	Children born protected from neonatal tetanus	✓	—
Post-Campaign Coverage Survey – Coverage Indicators
SIA_COVG_01	Crude campaign coverage	✓	—
SIA_COVG_02	Crude coverage where campaign dose was the first dose	✓	—
SIA_COVG_03	Lifetime doses of <campaign dose>, by birth cohort	✓	—
SIA_COVG_04	Campaign coverage stratified by prior number of doses received	✓	—
SIA_COVG_05	Clusters with surprisingly low campaign coverage	✓	—
Post-Campaign Coverage Survey – Quality of Vaccination Indicator
SIA_QUAL_01	Received a campaign card	✓	—
Survey-Adjusted Hypothesis Tests for Coverage Difference Indicators
COVG_DIFF_01	Test for coverage differences between strata	✓	—
COVG_DIFF_02	Test for coverage differences between subgroups within a stratum	✓	—

Source: Original table for this publication.

FIGURE 2. VCQI seven-step outline for indicator source code.

Source: Original figure for this publication.

Note: Some indicators use only a subset of these steps.

Use Cases

VCQI may be used for primary analysis of coverage surveys (29) or secondary analysis to validate or complement the primary analysis (2, 7, 20, 30). This set of programs can analyze microdata obtained from the original survey team or from data repositories such as harmonized DHS and MICS data sets available from the Integrated Public Use Microdata Series (20, 31). VCQI may also be adapted for use with electronic immunization registry (EIR) data.

VCQI has indicators for:

• Routine immunization surveys, abbreviated RI in VCQI documents. RI indicators usually focus on children ages 12–23 or 12–35 months, but the indicators may be used for older children or even adults. VCQI RI analysis goes beyond vaccination coverage outcomes typically reported in DHS and MICS reports by analyzing vaccination dates to assess dose timeliness, valid versus crude coverage, and MOSVs.

• Post-campaign coverage surveys (PCCS) that document coverage achieved in supplementary immunization activities (SIA). PCCS are commissioned by vaccination stakeholders and are perhaps more common in recent years than RI coverage surveys. For most campaigns, the coverage survey estimate is lower than the administrative coverage estimate, so the survey is an important independent assessment of campaign success (32). VCQI facilitates standardized analysis of campaign coverage in line with Gavi requirements and indicators used by the Measles & Rubella Partnership (33). Several SIA analyses that use VCQI output but are not yet formally coded into VCQI have been used in recent Gavi-commissioned SIAs in Africa and Asia. Those survey reports are not publicly available, but the analyses are described by the lead author (34). A public repository demonstrating these auxiliary analyses on faux data is currently under development.

• Maternal tetanus vaccination coverage surveys. This indicator estimates tetanus vaccination coverage among women who gave birth in the previous 12 months. VCQI has seldom been used for this purpose, and no development in this area has occurred since the original implementation in 2015.

Analysis Workflow

Data preparation and analysis follow a straightforward sequence:

• a)Make data VCQI compatible – A data analyst renames and recodes survey microdata variable names and response codes per VCQI documentation (24). Input data sets are factored into separate files for: 1) cluster metadata, and survey responses that describe 2) attributes of the household, 3) attributes of household members, 4) vaccination evidence from card and caregiver recall, and 5) vaccination evidence from health facilities. The code to calculate each VCQI indicator combines elements of the input data sets as needed.
• b)Check data quality – For RI surveys, VCQI applies a set of validation checks to identify implausible vaccination dates recorded during data collection. These checks evaluate temporal bounds (e.g., dates occurring before an allowable range; dates prior to the child’s birth or after the interview date) and internal logical consistency (e.g., violations of dose sequencing or duplicate dates). As summarized in Table 3, each validation rule is defined by a specific condition and a corresponding standard handling procedure. Dates that violate any of these conditions are recoded as tick marks to preserve evidence of vaccination while avoiding the use of implausible timing in analysis.
• c)Correct errant dates – If the survey protocol included photos of vaccination cards or registers, there may be a data cleaning step to review and correct errant dates in input data. This step must be completed by members of the survey team with access to the photos.
• d)Customize and run control program – The analyst copies the control program template and fills in parameters to describe the country’s vaccination schedule, eligible respondents, and to control analysis and reporting options. It is common for a modular analysis plan to be implemented using several VCQI runs, each with its own control program.
• e)Review output for noteworthy features – In accordance with VCQI guidance, the analyst notes outcomes that show very good vaccination coverage performance (e.g., high proportion of children fully vaccinated with valid doses by the age of 12 months) and outcomes that warrant additional inquiry (e.g., high rates of missed opportunities for simultaneous vaccination or numerous clusters with a surprisingly low share of children vaccinated).
• f)Paste output into report and slide decks. The analyst pastes VCQI output into report and slide templates, ensuring results are accurately transferred, clearly labeled, and consistently formatted for dissemination.
• g)Import VCQI output data set(s) into tools for secondary analysis – VCQI saves:
  • i.the input data augmented with respondent-level derived variables and indicator variables; this is a one-row-per-respondent data set; it can serve as the input, for example, to logistic regression to assess correlates of vaccination or non-vaccination; and
  • ii.data sets of stratum- and domain-level outcomes along with their confidence intervals, design effects, intracluster correlation coefficients, and other parameter estimates; these outputs may serve as input for dashboards or programs to make maps or other custom tables or visualizations. Because these are row-and-column flat file data sets, there are no restrictions to the type of software tools that can make use of these VCQI outputs. For example, there is a publicly available R Shiny tool that uses the augmented data set to facilitate browsing MOSV outcomes (35).
• h)Archive and share code and data sets – All code and data sets are systematically archived with clear documentation and versioning, and shared through secure, accessible platforms to support transparency, reproducibility, and future use (see, for example, the GitHub repository that accompanies the companion case study paper where VCQI was used for secondary analysis of the 2022 Jamaica MICS) (7).

TABLE 3. VCQI vaccination date validation rule failures and handling procedures.

Rule type	Validation condition	VCQI handling
Temporal bounds	Vaccination date before the earliest possible date of birth for this survey	Replace with a tick
Temporal bounds	Vaccination date before the child’s date of birth	Replace with a tick
Temporal bounds	Vaccination date after the date of the household interview (or health facility visit)	Replace with a tick
Sequence logic	Vaccination dates in a series out of order (e.g., dose 1 after dose 2)	Replace both dates with ticks
Duplication	Vaccination dates in a series on the same day (e.g., dose 1 on same date as dose 2)	Replace both dates with ticks

Source: Original table for this publication.

RESOURCES, CHALLENGES, AND PLANS FOR FUTURE DEVELOPMENT

Resources to Learn VCQI

The VCQI resources website links to many helpful documents, files, and tutorial videos (1). Readers should begin by reading the document named “Getting Started with VCQI.” The website also links to VCQI source code. Prospective VCQI analysts may download faux data sets and customized control programs to be used for learning and demonstrations. Some publicly available reports furnish supplemental folders of VCQI outputs and materials (3). Finally, there is a VCQI user community on the TechNet-21 website, which is free to join.² Note that we have opted to furnish descriptive tables in this manuscript rather than some of the informative figures that VCQI can produce. Interested readers may browse VCQI documentation and analyses to view graphics that are produced by or facilitated by VCQI (1–3, 7, 20, 25, 29, 32, 35).

Challenges

VCQI is extremely flexible. We sometimes describe it as being a “Swiss Army knife” for coverage survey analysis. But it is a big commitment to learn to use VCQI well. Many countries do a coverage survey every 3 to 5 years, so it would be difficult for public health personnel and staff of national statistical offices to learn VCQI well for one survey and then remember its details and nuances through the several years between surveys.

VCQI can help identify insights from nearly any coverage survey, but some stakeholders do not know to ask for VCQI analyses. Whenever there is a new DHS or MICS survey with an immunization module, countries might consider using VCQI to do secondary analysis for additional insight (7).

VCQI produces a lot of tabular and graphic output. It can be challenging to know which output to review first and how to identify the important or interesting outcomes, so we have published documentation to help the reader identify good news and areas for improvement: “Making Good Use of VCQI RI Output” and “Making Good Use of VCQI PCCS Output” (1).

Future Developments

VCQI development is limited primarily by funding for writing, testing, and documenting code and training materials. Planned future enhancements include adding PCCS indicators to vcqiR and adding remaining RI indicators to vcqiR. VCQI currently offers a mix of weighted and unweighted indicators. The forthcoming 2026 WHO vaccination coverage cluster survey reference manual will recommend that all indicators be weighted; therefore, VCQI will need to be updated for consistency. Additional near-term developments will include sharing examples of PCCS postprocessing with tutorial programs that users can run.

EIR analysis with VCQI will likely be a fruitful direction of work. More countries are adopting electronic registries over time, and PAHO has sponsored initial efforts to apply VCQI to those data. Given the structural similarity between registry and coverage survey data, routine immunization analyses can readily be implemented. In the best cases, EIRs hold data for nearly every eligible child, including information on where they are vaccinated. This can facilitate precise performance metrics for individual health districts or even immunization facilities. EIR VCQI analysis could be repeated annually or even more often without the expense of survey data collection.

One important distinction is that a survey analyzes a random sample of respondents while an EIR ideally holds records for a nearly complete census of each immunization cohort. The traditional survey-based standard errors and confidence intervals in VCQI output are aimed at conveying how the sample-based estimate relates to the broader population. That connection is missing with EIR data, so there will need to be some development of materials recommending how to describe the relationship between EIR-based estimates and the unknown coverage in the broader population. That development would focus on a combination of statistical programming, stakeholder conversations, and needs assessment to learn what will be helpful to calculate and to communicate.

Finally, some of our own near-term research will seek to train and share dedicated artificial intelligence tools to (a) help make survey data compatible with VCQI, (b) interpret VCQI output and summarize salient findings, and (c) be able to chat in a query-and-response fashion and assemble tabular and graphic support for detailed findings that are of interest to stakeholders.

CONCLUSION

Over the past 10 years, the Stata version of VCQI has been used to analyze dozens of routine immunization and post-campaign coverage surveys. Now there is an R version available, too, which will expand the global pool of analysts who can conduct standardized analyses of vaccination coverage surveys. VCQI and now vcqiR are freely available resources that may be used for primary analysis to generate output that is consistent with WHO reference materials or for secondary analysis to yield insights that go beyond the primary analysis.

Coverage survey analysts welcome the opportunity to use code that has been thoroughly tested and that produces output consistent with documented WHO specifications. They appreciate that VCQI does much of the clerical work of producing tabular descriptive coverage estimates in formats that are easily copied and pasted into survey reports. This frees up time for analysts to use advanced analytical skills to explore regression models or insightful data visualization with the VCQI output data sets. Data analysts with Stata and R skills are invited to contribute to the open-source VCQI code bases. All vaccination stakeholders are invited to contribute ideas for new indicators that may help to manage vaccination programs and protect the health of children around the world.

Funding Statement

Authors from Biostat Global Consulting were paid under a contract from WHO to develop VCQI in Stata and under contracts from Gavi and the Bill & Melinda Gates Foundation to enhance VCQI and paid under a contract from PAHO to program portions of VCQI in R.

Data availability statement.

VCQI resources, including sample data sets, are available at https://www.biostatglobal.com/VCQI_resources.html. The Stata version of VCQI source code is available at https://github.com/BiostatGlobalConsulting/vcqi-stata-public. The R package vcqiR is available at https://github.com/BiostatGlobalConsulting/vcqiR. The Jamaica case-study programs and output from (7) are available at https://github.com/BiostatGlobalConsulting/vcqi_jamaica_mics_2022_manuscript.

Declaration of Artificial Intelligence (AI) tool use.

AI tools were not used for this work.

Disclaimer.

Authors hold sole responsibility for the views expressed in the manuscript, which may not necessarily reflect the opinion or policy of the RPSP/PAJPH and/or PAHO and/or WHO.