Assessing vaccination coverage quality in Jamaica during COVID-19 using vcqiR: insights from the 2022 MICS

ABSTRACT

Objective.

This analysis aimed to identify the underlying causes of gaps in routine immunization coverage observed during the COVID-19 pandemic recovery phase by applying the open-source Vaccination Coverage Quality Indicators package in R (vcqiR), addressing limitations in Multiple Indicator Cluster Surveys (MICS) that often omit quality metrics based on detailed vaccination date analysis.

Methods.

This secondary analysis examined vaccination doses and dates from the Jamaica 2022 MICS (MICS4), for 570 children ages 12–23 months and 24–35 months. Specific indicators included coverage, timeliness, dropouts, and missed opportunities for simultaneous vaccinations (MOSVs).

Results.

Vaccination coverage for first-year-of-life (1YL) doses exceeded 83% for both age cohorts. Coverage for MMR1 showed substantial catch-up from 12–23 months to 24–35 months, achieving the 95% target. However, MMR2 coverage lagged substantially (37.2% for 12–23 months and 61.9% for 24–35 months), indicating significant attrition in the second year of life. In the 12–23 month cohort, crude coverage of Penta3 was 89%, but only 70% had three valid doses. Most 1YL doses were administered in a timely manner (within 28 days), but premature administration was observed. MOSVs were negligible for 1YL doses but were frequent and mostly uncorrected for 18-month boosters, especially MMR2.

Conclusions.

The findings complement the MICS4 report and confirm robust early infant immunization delivery but identify programmatic gaps in second year-of-life retention plus the need to strengthen adherence to minimum age interval rules. These results provide detailed evidence necessary to inform targeted interventions and guide the design of future coverage surveys.

Vaccination remains one of the most effective public health interventions, having averted an estimated 154 million deaths over the past 50 years (1). However, immunization programs were severely affected during the COVID-19 pandemic, resulting in an interruption to global vaccination coverage (2, 3). The Pan American Health Organization (PAHO) reported that prior to the COVID-19 pandemic, the Latin America and Caribbean (LAC) region achieved the highest average vaccination coverage globally, reaching approximately 77.8% in 2019 but declined significantly in 2020 to 75% and 74.5% in 2021 (4). PAHO further noted that Jamaica was among the few countries to sustain infant-series vaccination coverage above 90% throughout the pandemic period (5).

Jamaica’s birth cohort in 2022 was estimated at ~32 000 births (6), representing the target population for routine immunization services. The Expanded Programme on Immunization (EPI) established in 1977, boasts major milestones such as the elimination of poliomyelitis (1982), measles (1991), and rubella (2000) (7), underpinned by the Immunization Regulations of 1986, which legally mandated childhood vaccinations (8). Despite this strong foundation, the 2022 Jamaica Multiple Indicator Cluster Survey (MICS4) revealed only 55.5% of children aged 24–35 months were fully vaccinated according to the national schedule and only 61.9% had received MMR2 (9). These estimates fall significantly below national targets, highlighting potential socioeconomic and subnational disparities in vaccine access and uptake.

The evaluation of immunization program performance relies on consistent monitoring of vaccination coverage, through both routine administrative data and periodic household surveys such as UNICEF’s Multiple Indicator Cluster Surveys (MICS). This analysis was commissioned by PAHO to move beyond the coverage estimates of the MICS report and inform the design of an upcoming comprehensive vaccination coverage survey for Jamaica.

The primary objectives of this report are as follows:

1. Examine and interpret key quality indicators: valid coverage, dose timeliness, dropout rates, and missed opportunities for simultaneous vaccination (MOSVs).

2. Demonstrate the utility of vcqiR, through secondary analysis of Jamaica’s 2022 MICS4.

3. Provide evidence to guide the next phase of Jamaica’s EPI strategy, particularly targeting identified programmatic weaknesses.

METHODS

Data Source and Study Population

This study was a secondary analysis of the MICS4, a nationally representative, cross-sectional, stratified, two-stage cluster sample design (8). The MICS4 collected data from 7,903 households across 390 primary sampling units. Vaccination coverage was reported for 570 children: 289 children aged 12–23 months and 281 aged 24–35 months, from child health cards or through caregiver recall. The surveyed cohort received doses between April 2019 and June 2022. This spans prepandemic, peak disruption, and early recovery phases. Since this analysis is cross-sectional and not designed to assess temporal trends, references to “recovery” should be interpreted as contextual.

Vaccination Schedule

The Jamaican national immunization schedule applicable during the 2019–2021 period mandated the following doses: Bacille Calmette-Guérin (BCG) at birth; three doses of pentavalent (Penta1–3) and oral polio vaccine (Polio1–3) at 6 weeks, 3 months and 6 months; and measles-mumps-rubella (MMR1) at 12 months, followed by MMR2, DTP booster (DTP4), and polio booster (Polio4) at 18 months (10).

Data Preparation and Analysis

Analysis was performed using the vcqiR package, which ports the core routine immunization indicators developed by WHO from Stata to the R software program (11). Data management programs were written in R to ensure MICS variables were compatible with VCQI following the VCQI Forms and Variables List Document (12). The MICS Data Use Agreement prohibits sharing the recoded MICS data, but readers may download the data from the MICS website (13) and the data management programs from the manuscript’s GitHub repository (14). Results were reported using the similar demographic subgroups as the original MICS4 report (child sex, urban/rural, caregiver education, ethnicity, family religion, family wealth quintile). The MICS4 report did not report outcomes at the subnational level because of small sample sizes.

Definition of Core VCQI Indicators

VCQI indicators were grouped into three core analytical domains:

1. Coverage and Dropout

   • coverage: proportion of children who received a specific vaccine dose by the survey date, based on evidence from either a card or recall.

   • Valid coverage: proportion of children who received a dose based on minimum age and interval rules, requiring evidence from a vaccination card.

   • Full immunization coverage (FIC): proportion of children who received all scheduled first-year-of-life (1YL) doses. The definition of FIC varies by cohort; see the footnotes for Figure 1.

   • Dropout rates: proportion of children who received an early dose but did not receive the later dose.

   • Zero dose/not vaccinated: proportion of child who did not receive any doses from the FIC list. This differs from the WHO definition (no DTP1) and is used here to align with VCQI- indicators.

2. Timeliness

   VCQI analyzes the date of administration to determine if a dose was received prematurely, timely, or late:

   • Premature doses: proportion of children with doses administered before the minimum age recommended by the national EPI schedule.

   • Timely doses: proportion of children with doses administered within 4 weeks of the scheduled age.

   • Late doses: proportion of children who received doses 28–55 days late or >56 days late.

3. Programmatic Efficiency

   Missed opportunities for simultaneous vaccination (MOSV): This indicator measures instances where a child attended a facility for a scheduled vaccine dose but did not receive all other concurrent doses for which they were eligible. VCQI calculates the prevalence of MOSVs and whether they were subsequently corrected by the time of the survey.

FIGURE 1. Crude vaccination coverage and timeliness in Jamaica for children 12–23 months and 24–35 months, 2022 MICS.

Note: For the 12–23 month cohort, the fully vaccinated dose included BCG, Penta1–3, and Polio1–3. Zero dose/not vaccinated means the child did not receive any doses from the fully vaccinated list: 0% (95% confidence interval: 0–1.3%). For the 24–35 month cohort, the fully vaccinated dose included BCG, Penta1–3, Polio1–3, and MMR1. Zero dose/not vaccinated: 1.5% (95% confidence interval: 0.6–3.7%). DEFF = design effect; HBR = home-based records; N = sample size; NEFF = effective sample size.

Ethical Statement

The analysis used anonymized data from the Jamaica 2022 Multiple Indicator Cluster Survey; no additional ethical review was required.

RESULTS

Table 1 shows the survey sample comprised children aged 12–23 months (n = 289) and 24–35 months (n = 281), with an even distribution by sex (approximately 53% male and 47% female in both cohorts). Urban residence accounted for most of both cohorts (59% of the younger cohort and 51% of the older cohort) and about one-third lived in the Greater Kingston Metropolitan Area (GKMA). Caregiver education was high with reported having upper-secondary education (57%), while roughly one-third reported post-secondary training. Most children were of black ethnicity (95%) and household religion was predominantly Christian. Both cohorts showed significant representation in the poorest wealth quintile (25% in younger cohort and 24% in older cohort).

TABLE 1. Study sample characteristics disaggregated by age and demographic variables.

		12–23 months			24–35 months
Variable	Category	(N)	Weighted N	Weighted %	(N)	Weighted N	Weighted %
National	Jamaica	289	289.3	100.0	281	282.5	100.0
Child sex	Male	147	154.2	53.3	144	147.9	52.4
	Female	142	135.1	46.7	137	134.6	47.6
Location	Urban	174	169.7	58.7	156	143.5	50.8
	GKMA	90	93.6	32.3	89	92.1	32.6
	Other urban	84	76.1	26.3	67	51.4	18.2
	Rural	115	119.6	41.3	125	139.0	49.2
Caregiver education	Lower secondary or less	33	34.2	11.8	34	34.6	12.3
	Upper secondary	165	164.1	56.7	166	163.4	57.8
	Post-secondary+	91	91.0	31.5	80	83.6	29.6
Ethnicity	Black	276	273.7	94.6	269	271.7	96.2
	Other ethnicity	13	15.6	5.4	12	10.8	3.8
Household religion	Church of God	95	92.6	32.0	82	80.6	28.5
	Pentecostal	27	27.4	9.5	29	30.4	10.8
	Seventh-Day Adventist	50	54.2	18.7	41	41.4	14.7
	Other religion	78	76.9	26.6	82	80.1	28.3
	No religion	38	37.2	12.9	46	49.1	17.4
Household wealth quintile	Poorest	75	71.7	24.8	63	66.5	23.5
	Second	66	62.9	21.7	75	76.4	27.0
	Middle	62	68.0	23.5	53	54.1	19.1
	Fourth	46	43.0	14.9	57	54.0	19.1
	Richest	40	43.7	15.1	33	31.6	11.2

Note: Weighted values reflect survey sampling weights. Percentages may not sum to 100% due to rounding. GKMA = Greater Kingston Metropolitan Area.

Figure 1 shows the vaccination coverage and timeliness for both age cohorts. Each bar is a dose. The gray portion of the bar represents children for whom timing is unknown (i.e., the survey did not learn the age at which doses were received, mostly because evidence came from caregiver recall and not cards). Doses are ordered and grouped by scheduled age of administration with BCG due at birth at the bottom and those due at age 18 months at the top. The vertical dashed line indicates that cards were available for 87.3% for the younger cohort and 79.1% for the older cohort.

Full immunization coverage by the survey date was calculated at or above 89% for the 12–23 month cohort and at or above 81% for the 24–35 month cohort. MMR1 coverage in the younger cohort was 89% but achieved a 95% threshold in the older cohort. In contrast, MMR2 coverage was substantially lower, approximately 37% compared to 62% for the older cohort.

The saturated color in each bar indicates whether the doses were received too early (before the scheduled age), in a timely manner (within 28 days of the scheduled age), a little late, or very late. The gray portion represents children for whom timing is unknown (i.e., the survey did not learn the age at which doses were received, mostly because evidence came from caregiver recall and not cards.) The majority of 1YL doses were administered in a timely manner. Only 7–10% of doses were administered either too early (prematurely) or very late.

Figure 2 shows the dropout rate for Penta1 to Penta3 at the national and demographic level for both age cohorts. Dropout was specifically defined as the proportion of children who received the first dose of the pentavalent vaccine that missed the third dose. Nationally, dropouts increased from 8% in the younger cohort to 11.3% in older children. There was minimal difference by sex. Across both age cohorts, residents of urban locations and GKMA showed higher dropouts (9.9–15.6%) than rural areas (6.1–10.1%). The largest observed differences were by maternal education, with dropout increasing sharply to 24.8% among caregivers with lower secondary education compared to 9.5% among those with postsecondary education. There was no consistent pattern between religion and moderate socioeconomic disparities.

FIGURE 2. National and demographic level dropout rates between sequential doses of Penta1–Penta3, for children aged 12–23 months and 24–35 months.

Note: Dropout represents the proportion of children who received Penta1 but did not receive Penta3.

Figure 3 shows crude versus valid coverage of Polio3 and Penta3 among children 12–23 months, distinguishing between doses that did not meet validity criteria based on age or interval requirements. For Polio3, overall crude coverage was 95.0%, of which 71.5% of children had three valid doses administered according to age and interval guidelines. The remaining 23.5% of crude coverage was distributed across components that do not meet validity criteria: 8.8% of children relied on caregiver recall, 7.9% had erroneous recorded dates, 3.0% received at least one dose too early, and 3.8% had intervals that were too short between doses. For Penta3, crude coverage was 89.4%, with 70.7% meeting validity criteria, slightly lower than Polio3. The remaining 18.7% consisted of 4.3% recall-based evidence, 7.4% erroneous dates, 2.3% early doses, and 4.8% with interval errors.

FIGURE 3. Crude versus valid coverage of Polio3 and Penta3 among children 12–23 months.

Note: VCQI gives credit for doses with erroneous dates but cannot access their timing. Both Polio3 and Penta3 are administered at 6 months.

Figure 4 shows missed opportunities for simultaneous vaccinations. Across both age groups, the infant primary series (BCG, Penta1–3, Polio1–3) shows consistently high counts of doses received at the first eligible visit, with very small numbers of corrected or uncorrected missed opportunities. Among those 12–23 months, more than 200 children per antigen received the dose on time, with negligible MOSVs. This was similar in the 24–35 month cohort. For 18-month boosters, the 24–35 month cohort shows the greatest number of MOSVs, with numbers highest for MMR2 and Polio4, including a notable proportion of uncorrected opportunities. However, there is a high correction rate for DPT4 when missed.

FIGURE 4. Missed opportunities for simultaneous vaccinations (MOSVs), by dose and cohort.

Note: Lower counts for the 18-month booster doses in the 12–23 month cohort are expected, as many children in this age group may not have reached the recommended age of eligibility to receive these vaccines.

DISCUSSION

Using vcqiR to analyze recorded vaccination dates provided detailed insights not previously captured in the MICS4 report and offers an important opportunity for continuous learning within Jamaica’s EPI. The analysis confirms the overall robustness and continuity of the program during the 1YL, with high infant-series coverage (over 89%) and low dropout rates despite the challenges posed by the COVID-19 pandemic. High caregiver education levels and the urban concentration of the sample likely contributed to the strong performance observed early in life.

Our findings show progress with both MMR1 and MMR2 coverage, demonstrating catch-up in the 24–35 month cohort. MMR1 met the 95% threshold and MMR2 coverage increased to approximately 62%. However, this is still below the global target and the 2021 MMR2 coverage for the Latin American and the Caribbean region of 74.8% (15). Given the ongoing global resurgence of measles (16) and persistent concerns around MMR hesitancy (17), these results are noteworthy. However, this catch-up may indicate that missed doses are not only caused by hesitancy or refusal but also delays in vaccination. It is, therefore, essential to reinforce that immunization is a life-course intervention, with protection strengthened through timely doses. Low MMR2 coverage poses a significant threat to Jamaica’s elimination status and aligns with known follow-up challenges in the second year of life (18). WHO’s Global Immunization Agenda 2030 (IA2030) includes measles elimination as a core indicator of impact of immunization programs, highlighting the importance of rigorous measles surveillance systems to identify (19).

The analysis highlighted a critical distinction between crude coverage and valid coverage (Figure 3). Despite high crude coverage for both Polio3 (95%) and Penta3 (89.4%), 25–30% of recorded doses did not meet validity criteria, primarily due to programmatic factors such as erroneous date documentation, survey limitations (reliance on caregiver recall), or timing deviations. The results demonstrate good service access but issues with timeliness rather than vaccine-specific barriers. This indicator shifts assessment from the quantity of doses received to the quality and effectiveness of the dosing schedule. Quantifying invalid doses is essential for health managers to assess the true effective coverage, rather than just the crude number of doses administered.

Dropout patterns also offer useful insights for program strengthening. Penta1to Penta3 dropout was higher among children aged 24–35 months (11.3%) compared with the younger cohort (8.0%) and slightly above the 10% reference threshold recommended by WHO (22). The increase highlights the need for follow-up systems. Higher dropouts in urban areas may reflect mobility, clinic congestion, or changing care-seeking patterns, while differences by caregiver education highlight opportunities for enhanced health literacy interventions. These differences were not statistically significant and should be cautiously interpreted as noted by the wide and overlapping 95% confidence interval (CI). Differences in sex and wealth were minimal indicating equitable access to vaccinations.

Programmatic efficiency assessed using the MOSVs indicator also revealed areas for quality improvement. MOSVs were rare for 1YL doses, demonstrating high operational efficiency and consistent adherence to co-administration policies during frequent infant visits. However, more frequent uncorrected MOSVs for 18-month boosters, particularly MMR2 and Polio4, suggest that opportunities remain to strengthen both client retention and provider practices. When caregivers attend for DPT4 vaccination but leave without receiving other due boosters, it signals a chance to reinforce eligibility checks. Qualitative inquiry would help clarify the reasons behind these inconsistencies, including whether service flow, task distribution, or competing clinical priorities influence the ability to identify and correct missed doses. Although the COVID-19 pandemic caused global supply chain disruptions (23), this analysis could not confirm whether such disruptions influenced vaccine availability in Jamaica during the study period.

The results point to interventions for consideration in Jamaica’s EPI and global monitoring:

• Implement reminder and recall mechanisms targeting children who have received MMR1 and due for MMR2, to ensure series completion and reinforce vaccination through life course.

• Investigate factors driving dropout rate and MOSVs. The next comprehensive vaccination coverage survey planned for Jamaica should integrate behavioral and social drivers (BeSD) modules (21), to identify underlying nonclinical barriers that prevent families from completing the second-year immunization series.

• Jamaica’s next coverage survey should focus on assessing 18-month booster doses.

• The gap in MMR2 coverage suggests there may be a need to institutionalize MMR2 as a key global monitored indicator, like MMR1 and DPT3, but with the intention to track the second year of life immunization performance.

Strengths and Limitations

The analysis uses nationally representative MICS data. vcqiR provides a standardized, validated, and open-source platform, ensuring that the results are reproducible, transparent, and compatible with other global studies. The ability to generate data-based indicators adds analytic depth beyond that of standard MICS reports. We expect record retention to decline with age, but the high rate of vaccination card availability (87% for 12–23 months and 79% for 24–35 months) significantly strengthens the validity of the study’s key findings (22). Notwithstanding, we recognize several limitations. The surveyed cohort received doses between April 2019–June 2022, overlapping with potential COVID-19 service disruptions and may not fully capture the results of subsequent recovery efforts. The sample size for the overall cohort is small, which limits power for subnational analysis. A small proportion of doses (7–10%) were administered before the minimum recommended age. Given the relatively low magnitude, these early doses may reflect routine variation in service delivery or documentation rather than systematic programmatic practice. The lack of prior integrated data on behavioral and social drivers of vaccination limits the ability to assign definitive social or behavioral causation to the observed retention gaps. Also, vaccination was not the primary focus of the MICS4 survey, and many of its clusters did not yield data on children in the cohorts of primary interest for immunization program assessment. Although the MICS4 collected data in 390 clusters, the vaccination data came from only 182 clusters for ages 12–23 months and 189 clusters for 24–35 months.

Conclusions

The findings confirm high EPI performance during the COVID-19 pandemic and service continuity during the first year of life, characterized by high coverage and minimal dropout. However, programmatic challenges in the second year, specifically high dropout for MMR2 and frequent, uncorrected missed opportunities for simultaneous vaccination pose a public health threat. The application of the vcqiR package successfully enhanced the analytical capacity of the MICS4 data set, yielding quality metrics helpful for strengthening Jamaica’s EPI.

Funding Statement

The study is funded by the Pan American Health Organization, Jamaica Office. PAHO Jamaica did not influence in any way in the design, the data collection, the analysis, the writing, and the decision to publish these results.

Data availability statement.

All vcqiR scripts, templates, and analytic outputs used in this study are available in the public GitHub repository accompanying this manuscript. The original MICS microdata can be accessed via registration on the MICS website. The vcqiR core code and documentation are available as free resources.

Declaration of artificial intelligence (AI) use.

CANVA was used to enhance figures used in the manuscript. All authors agree with the tool output.

Disclaimer.

Authors hold sole responsibility for the views expressed in the manuscript, which may not necessarily reflect the opinion or policy of the RPSP/PAJPH and/or PAHO.