Estimating the effects of interventions on increasing vaccination: systematic review and meta-analysis

Abstract

As global vaccination rates have reached their lowest point in nearly 15 years, effective interventions are being required globally to promote vaccination; however, there is a lack of rigorous evaluation of the effect of various interventions. Through a global synthesis, we analysed data from approximately 6 125 795 participants across 319 studies in 41 countries to reveal the global landscape of four intervention themes and to assess their effectiveness in increasing vaccination rates. We found an overall positive effect of the interventions across four main themes on improving vaccination. Specifically, dialogue-based interventions increased vaccination rates by 43.1% (95% CI: 29.8 to 57.9%, with effect sizes measured as relative risks (RRs)), though they may not always be effective in adolescents or in the sample with a higher percentage of male participants. Incentive-based interventions, whether implemented alone or combined with other intervention themes, failed to demonstrate a significant effect in children. Reminder/recall-based interventions were also effective for promoting vaccination (38.5% increase, 95% CI: 28.9 to 48.9%), particularly for completing vaccine series. Multi-component interventions exhibited excellent effectiveness in vaccination (54.3% increase, 95% CI: 40.5 to 69.6%), with the combination of dialogue, incentive and reminder/recall proving more effective than other multi-component interventions, but showing no significant effects in populations with high initial vaccination rates. However, we found that in most cases combining additional interventions with a single intervention may not significantly improve their effectiveness, especially for incentive-based interventions, but dialogue-based and reminder/recall-based interventions appear to be beneficial in some specific combinations. These findings underscore the importance of governments, public health officials and advocacy groups implementing appropriate vaccine interventions by selecting interventions tailored to specific populations, strategically promoting the completion of vaccine series and effectively combining interventions to promote global vaccination and save more lives.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Various strategies have been reviewed and grouped into four themes by the WHO Strategic Advisory Group of Experts on Immunisation Working Group: dialogue-based, incentive-based, reminder/recall-based and multi-component interventions.

• Existing meta-analyses often focus on specific populations, vaccine types or intervention types, limiting the generalisability of findings.

• Although systematic reviews have shown substantial heterogeneity, the underlying factors remain unclear due to insufficient data.

WHAT THIS STUDY ADDS

• This study synthesises global large-sample data to assess intervention effectiveness on vaccination and identify predictors contributing to effect variation.

• Four intervention themes generally improve vaccination rates, with dialogue and multi-component interventions showing greater increases. However, simply adding additional strategies to a single strategy does not necessarily lead to better vaccination outcomes.

• Interventions should be carefully evaluated and chosen for implementation based on participants’ characteristics and specific vaccine types.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• Due to the variability in the effectiveness of interventions under diverse conditions, ensuring vaccination gains through interventions hinges on consideration and assessment of actual needs and circumstances.

• Considering the limited research and implementation of incentive-based interventions (especially non-financial incentives) and multi-component interventions combining different themes, further studies are needed to explore their effectiveness and inform intervention design.

Introduction

Despite the effectiveness of vaccination in saving lives from infectious diseases,¹,³ global vaccination coverage has experienced its largest sustained decline in 30 years.⁴ Effective interventions to promote vaccination are necessary throughout the world,⁵,⁹ especially considering the rising vaccine hesitancy fuelled by the COVID-19 pandemic.^{4 10 11} Various strategies such as monetary incentives, poster promotion and call reminders have been piloted and evaluated in diverse environments to understand their effects on vaccination.¹²,¹⁵ These interventions have been reviewed and grouped into four themes by the WHO Strategic Advisory Group of Experts on Immunisation (SAGE) Working Group: dialogue-based, incentive-based, reminder/recall-based and multi-component interventions.¹⁶ However, we still lack an understanding of the generalisability of specific results produced by individual studies, which suggests that rigorous exploration and assessment with global synthesis data is yet to be undertaken. This omission is important because quantifying the effects of interventions on vaccination and discussing the most effective mechanisms to achieve them can help reach scientific consensus. This consensus, in turn, can guide governments, public health officials and advocacy groups in implementing appropriate strategies to enhance vaccination and promote disease prevention globally.

Despite the growing number of systematic evaluations conducted to identify, describe and assess interventions aimed at improving vaccine uptake globally, there is a lack of quantitative data synthesis to ascertain the effects of these interventions, and some general conclusions cannot be drawn regarding the effectiveness of various interventions.¹⁶,¹⁹ Some meta-analyses assessing the interventions on vaccination have limited the target population,²⁰,²³ study setting²⁴,²⁶ and vaccine types,^{21 23 27} or have focused on a specific intervention type,^{25 28 29} which hinders our comprehensive understanding of the impact of interventions and may limit the generalisability and applicability of the findings. In addition, although many systematic reviews have shown substantial heterogeneity in their findings,³⁰,³² potential factors that lead to overall heterogeneity were not examined due to insufficient data. These limitations suggest that our current understanding of the complete landscape of interventions to increase vaccination rates, as well as their context and applicability, is restricted. To address this gap, we present a global synthesis of large sample data from around the world to assess the effectiveness of interventions on improving vaccine uptake and further take into account possible predictors, which may explain the variation in effects of the intervention (detailed explanation of predictors can be found in the online supplemental file).

In our study, we aimed to reveal the global landscape of various interventions and clearly synthesise their effects on increasing vaccination rates, which were summarised and categorised by the SAGE Working Group before. Among the various vaccination intervention models like the Behavioural and Social Determinants of immunisation (BeSD) model,⁸ we chose to adopt the SAGE model, which provides a framework focused on the intervention themes currently under extensive research^{12 13 33} and is applicable to our research objectives of meta-analysis that can guide our retrieval, classification and analysis of interventions. We also performed subset meta-analyses to investigate the impact of intervention on vaccination for different participants’ characteristics and vaccines, as well as stratified analyses based on the vaccination coverage of the baseline or control group. Moreover, we assessed between-group heterogeneity to explore whether the effect sizes vary significantly between four main intervention themes and other subsets. We further took into account the influence of various important predictors on interventions’ effectiveness, including population-specific, context-specific, vaccine-specific and variables related to interventions and vaccination outcomes. Finally, we aimed to shed light on the overall quality of the evidence provided in the meta-analysis and to discuss the implications of the evidence for future research and immunisation practice.

Methods

Search strategies

Through a combination of keyword searches, indexing from published syntheses and snowballing,³⁴ we conducted a search to identify as many relevant primary studies as possible (Table S1). The search process is summarised in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow chart (online supplemental figure S1),³⁵ and a full list of primary studies included is provided in online supplemental table S2. Detailed search strategies can be found in the online supplemental file.

Eligibility criteria

For a study to be included, it must satisfy certain criteria. First, the study must examine the effects of at least one of the following four intervention themes: dialogue-based, incentive-based, reminder/recall-based and multi-component interventions (defined below). The WHO SAGE Workgroup summarises and groups these four intervention themes,¹⁶ which dominated the discussion about the methods and outcomes of interventions to address vaccine hesitancy. Second, the study must report measures of vaccination behaviour in humans, although these were not necessarily the primary outcome of the study. Third, sufficient outcome data must be reported for the calculation of effect sizes. Studies were excluded if they were not primary studies or without full-text articles. Finally, the study must include comparison conditions, and we included randomised controlled trials (RCTs), quasi-experiments and cohort studies. For cohort studies, we included prospective and retrospective designs, with prospective cohort studies following populations over time from baseline and retrospective studies using reliable pre- and post-intervention data to estimate effects.³⁶ Observational designs, when carefully structured, can emulate the conditions of a randomised experiment and yield comparable effect estimates to RCTs,^{37 38} which allows our cohort studies to meet comparability criteria and provide valuable insights alongside RCTs and quasi-experimental studies. The reference group either received no intervention or a passive or active control that did not focus on trying to improve vaccination rates or usual care. Comparing different intervention types head-to-head was not the focus of this review, as the main aim was to establish the effect of the independent variable: being in receipt of different types of intervention. Although comparing different intervention types is very meaningful and important, its insufficient number of studies can easily lead to the failure of robust meta-analysis results. Therefore, studies comparing interventions to one another were excluded from the current review. However, we did not impose any restrictions on the language or country where the research was conducted.

Data analysis

The majority of studies reported events and sample size, so relative risk (RR) was chosen as a standard measure to describe the effect sizes and then were converted to ln(RR). LRRs and their standard errors were calculated using the escalc function. We conducted meta-analyses and meta-regressions in a mixed-effects model using the rma.mv function in metafor (version 3.8–1),³⁹ the package that incorporated both fixed (moderators) and random effects and allowed us to account for (and model) potential shared variation and data non-independence. In all meta-analyses and meta-regressions above, effect size measurements from each study were weighted by the inverse of the variance. The use of the random effects model in this analysis also enabled us to assess the degree of heterogeneity of effect sizes. Statistical heterogeneity was assessed with the I² statistic.^{40 41} For all linear mixed-effect models, we visually assessed residual and QQ plots (online supplemental figures S2 and S3). Furthermore, the marginal R² is reported to quantify how much total variance is explained by fixed effects apart from sampling variance.^{42 43} For all analyses, two-tailed tests were employed and the level of significance was set to p<0.05. Detailed data analysis can be found in the online supplemental file.

Patient and public involvement

Because this review did not focus on any specific patient population, no patients were directly involved in setting the research question or the outcome measures or in the design or implementation of the study. No patients were asked to advise on interpretation or writing up of results.

Results

According to our inclusion criteria, our searches (online supplemental tables S1 and S3) yielded 319 studies in 41 countries, representing data from approximately 6 125 795 participants. See online supplemental figure S1 for the PRISMA flow diagram summarising the study selection process. We calculated a log risk ratio (RR) from each data point to represent the effectiveness of the interventions. In total, we identified 1104 RRs for vaccination, including 305 RRs for dialogue-based interventions, 23 RRs for incentive-based interventions, 412 RRs for reminder/recall-based interventions and 364 RRs for multi-component interventions (online supplemental figure S4 and online supplemental table S2).

Overall, we found that the interventions across four main themes increased vaccination rates by 46.0% (95% CI: 38.5% to 53.9%), with high heterogeneity among the interventions. To gain a better understanding of the impact of each intervention theme, we then performed separate analyses on the four intervention themes. The sample data showed that dialogue, incentive, reminder/recall and multi-component interventions have generally proven effective in improving vaccination rates. Specifically, dialogue-based interventions significantly improved vaccination rates (mean: 43.1%; 95% CI: 29.8 to 57.9%; figure 1, upper panel). These patterns were mainly driven by the good performance of dialogue education, which exhibited high effectiveness (mean: 80.6%; 95% CI: 24.8 to 161.4%; figure 1, lower panel). The incentive-based interventions increased vaccination rates by 29.8% (95% CI: 14.9% to 37.2%), and non-financial incentives may have good performance in promoting vaccination (mean: 48.7%; 95% CI: 18.4 to 86.7%). Reminder/recall-based interventions increased vaccination rates by 38.5% (95% CI: 28.9 to 48.9%), with provider reminder prompts showing particularly effective results (mean: 75.2%; 95% CI: 35.2 to 127.0%). Consistent with prevailing understanding, multi-component interventions demonstrated on average 54.3% higher vaccination rates than did reference condition (95% CI: 40.5 to 69.6%), and the multi-component intervention that combined incentive and reminder/recall (mean: 31.1%; 95% CI: 4.1 to 65.1%) showed a disadvantage over other multi-component interventions.

Figure 1. Effects of interventions on vaccination. Effects of four main interventions (upper panel) and specific interventions under four themes (lower panel) on vaccination. A small number of RR values are not displayed because they fell outside the display range, including two highly positive RRs for dialogue-based intervention, one highly positive RR for reminder/recall-based intervention and six highly positive RRs for multi-component intervention.

To further investigate the effects of interventions targeting different populations and vaccines, we conducted subset analyses split by participants’ age and sex, as well as various vaccine types. For adults, various interventions we analysed generally led to significant improvements in vaccination (figure 2, right panel, and online supplemental table S4), with dialogue education appearing to produce higher average effect sizes (mean = 0.89, 95% CI: 0.30 to 1.48). Unlike interventions targeting adults’ vaccination, interventions for children and adolescent vaccinations have always relied on parental decisions.⁴⁴ In terms of childhood vaccination, our results showed that intervention combined with multiple reminders/recalls seemed to be particularly effective in increasing vaccination (mean = 0.42, 95% CI: 0.12 to 0.73; figure 2, left panel). However, interventions containing incentives, regardless of whether implemented individually or combining the other two intervention themes, failed to demonstrate a significant effect in children (mean financial incentive = 0.04, 95% CI: −0.01 to 0.10; mean incentive & reminder/recall = 0.24, 95% CI: −0.01 to 0.50; mean dialogue & incentive & reminder/recall = 0.66, 95% CI: −0.46 to 1.79). Among adolescents, the pattern of the intervention’s impact was not the same as that observed in the other two age groups. Although dialogue-based interventions were generally effective in promoting vaccination in both children and adults, mass media and dialogue education no longer showed significant intervention effects for adolescents (mean mass media = 0.24, 95% CI: −0.12 to 0.61; mean dialogue education = 0.03, 95% CI: −0.02 to 0.08; figure 2, middle panel).

Figure 2. Effects of specific intervention type on vaccination split by participants’ age. Only intervention types with more than seven RRs were displayed in the forest plot. Specific effect sizes and other statistics applying to this forest plot are presented in online supplemental table S4.

In samples with a higher percentage of female participants, each intervention we analysed demonstrated positive performance in vaccination outcomes. Of these effective interventions, the strongest effect was estimated for multi-component interventions combining multiple dialogue strategies (mean = 0.68, 95% CI: 0.31 to 1.05; figure 3, right panel, and online supplemental table S5). In the male-dominated sample, multi-component interventions were always effective in improving vaccination. However, our results showed that for this group, any specific dialogue-based intervention did not have a significant effect on vaccination outcomes (mean mass media = 0.38, 95% CI: −0.16 to 0.91; mean information-based training for HCW = 0.08, 95% CI: −0.07 to 0.22; mean dialogue education = 1.13, 95% CI: −0.01 to 2.28; figure 3, left panel). Given the variability in the sex ratio dichotomy across studies, we investigate specific sex ratio as a predictor in the meta-regressions that follow.

Figure 3. Effects of specific intervention type on vaccination split by participants’ sex. Only intervention types with more than seven RRs were displayed in the forest plot. Specific effect sizes and other statistics applying to this forest plot are presented in online supplemental table S5.

In the subset meta-analysis of vaccine types, we only examined the outcomes of one vaccine, but not the outcomes of multiple vaccines because the combined effects of multiple vaccines could not distinguish between the individual effects of each vaccine. With respect to the vaccine targeted, most available estimates (RRs) were related to influenza vaccine (31%, 257 RRs), followed by HPV vaccine (21%, 176 RRs), and DTP-containing vaccine (17%, 145 RRs). Importantly, we found that the effect of specific interventions on different vaccines was variable, and the effectiveness of interventions did not extend uniformly across all vaccines. Specifically, our analysis revealed that mass media and texts significantly enhanced the uptake of routine vaccines such as influenza, HPV, DTP and measles-mumps-rubella (MMR) vaccines, but did not demonstrate similar effectiveness for other vaccines (online supplemental figure S5, and online supplemental table S6). Similarly, a significant positive effect on vaccination was observed for information-based HCW training targeted at PCV vaccine and telephone calls targeted at influenza vaccine, yet these interventions were not effective for other vaccines. Moreover, our research highlighted that although most interventions significantly improved influenza vaccination, multi-component interventions combining dialogue, incentive and reminders/recall did not yield the expected effect on it (mean = 0.62, 95% CI: −0.19 to 1.42).

Stratified by vaccination coverage in the baseline/control group, we found that the four intervention themes were generally effective, especially under conditions of low coverage (see online supplemental table S7). However, as the initial reference coverage increased, a gradual decrease in the effectiveness of the interventions was observed, a trend consistent across all intervention themes. At high reference group coverage (>80%), incentive-based and multi-component interventions did not demonstrate statistically significant effects on vaccine uptake (mean incentive-based interventions = 0.03, 95% CI: −0.03 to 0.09; mean multi-component interventions = 0.03, 95% CI: −0.01 to 0.06). Meanwhile, reminder/recall-based interventions showed a mean effect size of 0.04 (95% CI: 0.03 to 0.06) at this high coverage level, appearing to have a relatively larger observed effect compared with the other interventions.

For all the above meta-analyses, we found high levels of heterogeneity with I², suggesting large variations in the effectiveness of interventions. We observed asymmetry in the funnel plot (online supplemental figure S6), and the result of the Egger’s regression suggested statistically significant evidence of small-study bias (Egger’s regression p<0.001). Given the concerns of the normal distribution assumption of the data and study bias, a series of sensitivity analyses were implemented, including reanalysing by bootstrapping, removing the studies with a high risk of bias, removing 5% most variable effect sizes of each dataset, removing 5% highest standard errors of each dataset, using random effect structure without the vaccine type and including only RCTs. Our findings were robust to sensitivity analyses, revealing reliability and stability of our conclusions under different conditions (online supplemental table S8).

We then compared the effect of different intervention themes, aiming to assess between-group heterogeneity. This analysis revealed significant variations in effect sizes across four major intervention themes (Q₄=213.2; p<0.0001; online supplemental table S9), with multi-component interventions performing better in vaccination than dialogue-based and reminder/recall-based interventions (p<0.05). Therefore, we then focused on a subset of interventions that included a specific strategy to compare their vaccination outcomes against those of multi-component interventions, which combined this strategy with additional strategies. Importantly, in most cases, combining additional interventions with a single intervention may not improve their effectiveness in enhancing vaccination, especially for incentive-based interventions. We found no significant difference in performance between incentive-based and multi-component interventions which included incentives, although there was a trend for higher effectiveness in multi-component interventions combining dialogue and incentives (mean dialogue & incentive = 0.13, 95% CI: −0.08 to 0.34; mean dialogue & incentive & reminder/recall = 0.25, 95% CI: −0.18 to 0.68; online supplemental table S10, and figure 4, middle panel). For dialogue-based interventions, we found that the effectiveness is maximised when both of the other intervention themes are also applied (mean dialogue & incentive & reminder/recall = 0.68, 95% CI: 0.37 to 0.99; online supplemental table S10, and figure 4, left panel). Additionally, the effect of reminder/recall-based interventions was improved by 5.2% when combining incentive-based interventions with it, and the differences between categories were significant (mean incentive & reminder/recall = 0.05, 95% CI: 0.01 to 0.09; online supplemental table S10, and figure 4, right panel). However, beyond these findings, we did not observe other significant differences in intervention effects when comparing single strategies in dialogue or reminder/recall with multi-component interventions that included them.

Figure 4. Comparison of vaccination outcomes under a single strategy in relation to implementation of different multi-component interventions. Results are shown for subcategories of studies that indicated whether a single intervention was combined with other interventions as a specific multi-component intervention (interventions containing dialogue, incentive and reminder/recall, respectively): DI (dialogue-based intervention), II (incentive-based intervention), RI (reminder/recall-based intervention), DI+DI (multidialogue intervention), RI+RI (multi-reminder/recall intervention), DI+II (multi-component intervention combining dialogue and incentive), DI+RI (multi-component intervention combining dialogue and reminder/recall intervention), II+RI (multi-component intervention combining incentive and reminder/recall) or DI+II+RI (multi-component intervention combining dialogue, incentive). and reminder/recall). 95% CIs are depicted in heavy black lines (may be hidden by the mean data point), prediction intervals in thin black lines. The size of each data point is proportional to the precision of the study (1/vi (variance)). Significant differences between interventions are indicated by *, and comparisons are made with dialogue-based, incentive-based, reminder/recall-based strategies, respectively.

We further explored what factors might underlie the variation in intervention effectiveness on vaccination behaviour. We assessed the relationship between RRs and a set of variables representing intervention, vaccination behaviour, population, context and vaccine influences, while analyses of incentive interventions were dropped because of data paucity. For reminder/recall-based interventions, the most parsimonious models selected through small-sample corrected Akaike Information Criterion (AIC) scores (online supplemental table S11) showed that the effectiveness of such interventions diminished over time (coefficient=−0.0084, p<0.05; figure 5B; online supplemental table S12). Also, our linear regression analysis revealed a significant positive relationship between age and the effectiveness of reminder/recall-based interventions (coefficient=0.014, p<0.05). However, the inclusion of the quadratic term of age indicated a significant negative relationship (coefficient=−0.0002, p<0.05), suggesting that reminder/recall-based interventions may be less effective for younger or older samples (figure 5A). In addition, our results showed that reminder/recall-based interventions performed better on vaccine series completion than vaccine uptake (p<0.05; figure 5C). For multi-component interventions, we found that the interventions combining dialogue, incentive and reminder/recall did show a significant advantage over any other multi-component interventions (p<0.05; figure 5E). Furthermore, multi-component interventions were more effective in regions including East Asia & Pacific, Latin America & Caribbean, the Middle East & North Africa, and South Asia (p<0.05; figure 5E), whereas in Europe & Central Asia and Saharan Africa, their effectiveness did not significantly differ from that in North America. We did not find evidence of other variables explaining variation in RR values for the interventions. See online supplemental figure S7 for the scatterplot displaying the lack of relationship between RR and predictor variables. These findings were also robust to sensitivity analyses related to risk of bias, particularly variable effect sizes, small studies and random effect structure (online supplemental table S12).

Figure 5. Factors explaining the effects of interventions on vaccination. Best models selected based on AICc scores identified the following factors as explaining RRs: (A) age for reminder/recall-based interventions; (B) intervention time for reminder/recall-based interventions, (C) specific vaccination behaviour for reminder/recall-based interventions, (D) region for multi-component interventions and (E) specific intervention type for multi-component interventions. Scattered dots represent RR values from primary studies, with dot size proportional to the weight of each RR in the meta-regressions. Fitted curve (black lines) and 95% confidence band [dark blue band in (A) and (B)] are generated from meta-regression. A small number of RR values are not displayed because they fell outside the display range, including three high negative RRs for North America, and three, four and one high positive RRs for Europe and Central Asia, North America and other regions, respectively.

The overall risk of bias was considered ‘high’ in 6.3% of all RCTs and ‘serious/critical’ for around 10.4% of the quasi-experimental or cohort studies (online supplemental figure S8). Some additional points about the assessed literature stand out. Because most studies did not report a pre-specified analysis plan or protocol, it could not be completely ruled out that the post-treatment outcomes were ‘cherry-picked’. Additionally, due to the design of our interventions, participants and people delivering the interventions were often aware of assigned intervention during the trial. The most critical methodological problems identified in the RoB and ROBINS-I assessment related to bias due to the randomisation process and confounding, respectively.

Discussion

Understanding the potential effects of various interventions to increase vaccination and establishing a knowledge base that explained the variability in the outcomes of globally implemented interventions are crucial for promoting vaccination and improving population health.^{5 45} We found that dialogue, incentive, reminder/recall and multi-component interventions have generally proven effective in improving vaccination rates. Although dialogue and multi-component interventions showed a higher percentage increase in vaccination,^{16 24} our meta-analysis did not provide evidence that these two interventions increased vaccination more than the others.

Our analysis, comparing vaccination outcomes under implementation of a single strategy in relation to implementation of multi-component interventions, showed that in most cases, combining additional interventions with a single intervention as multi-component interventions may not significantly improve the effectiveness in enhancing vaccination. The varying impacts of different interventions on behavioural decisions, as well as potential interactions among interventions,^{46 47} could explain our results. Specifically, the effectiveness of incentive-based interventions could not be improved regardless of how combined with other interventions. The results of several studies support this finding,^{46 48} suggesting that it is difficult to amplify the effects of incentive-based interventions with additional interventions. The effectiveness of the dialogue-based intervention was enhanced only when the other two intervention themes were also implemented, highlighting the effectiveness of an integrated approach in health communication without considering cost-effectiveness. Similarly, the effectiveness of the reminder/recall-based intervention significantly improved only when combined with the incentive-based intervention, a pattern supported by similar studies which found that reminder/recall coupled with incentives often yielded better outcomes than single interventions.^{12 49 50} Therefore, simply adding additional strategies to a single strategy does not necessarily lead to better vaccination outcomes, as our results show the difficulty of amplifying the effects of incentive-based strategies with additional strategies, but dialogue-based interventions and reminder/recall-based interventions appear to be beneficial in some specific combinations. These findings underscore the importance of strategically combining interventions in vaccination campaigns to optimise vaccination outcomes, providing actionable insights into the allocation of resources and the design of vaccine interventions.

When splitting by participants’ characteristics, we observed that incentive-based interventions, whether implemented alone or in combination with the other two intervention themes, did not effectively increase vaccination for children. This finding is consistent with the results of another meta-analysis,⁵¹ which suggests that external intervention via incentives may undermine intrinsic motivation.^{52 53} When parents vaccinate their children for intrinsic reasons (eg, belief in the benefits of vaccination),^{44 54 55} the introduction of extrinsic incentives (eg, monetary rewards) may shift their attention away from these health benefits to the incentives, and this shift may weaken the original motivation for vaccination and lead to decreased trust.^{56 57} In addition, we found that the vaccination outcome was not significantly improved by dialogue-based interventions in adolescents, as well as in samples with less than 51% female participants, which was observed in several studies.⁵⁸,⁶¹ Previous work has found that interactive digital media and traditional presentations had different effects on adolescents’ attitudes towards vaccination, suggesting that dialogue-based interventions may need to be specifically designed to engage adolescents and meet their unique needs and preferences.⁶² Moreover, there could be gender differences in health communication,⁶³ with men showing less willingness to engage in preventive health behaviours, seek, trust and follow health advice,⁶⁴ and this may result in dialogue-based interventions being less effective in samples with a lower percentage of females. Furthermore, our results showed that these specific interventions generally led to significant improvements in adults and in samples with at least 51% female participants, highlighting the importance of tailoring health interventions to specific populations. In our meta-analysis focusing on different vaccine types, we found that specific interventions varied in effectiveness, with some promoting vaccination for certain vaccines but not for others. Therefore, interventions should be carefully evaluated and chosen for implementation based on the specific vaccine type. Stratified by vaccination coverage of the baseline/control group, our analysis revealed a gradual reduction in the estimated effectiveness of interventions as the initial coverage increased, with incentive-based and multi-component interventions not demonstrating statistically significant effects in populations with high initial vaccination rates (>80%). This finding underscores the importance of prioritising and tailoring interventions based on participant-centred goals, as well as their relative impact and acceptability, particularly in settings with high baseline vaccination coverage.⁶⁵

Moreover, the additional consideration of possible predictors generated further insights into the specific circumstances that might determine the effectiveness of interventions. We observed that the effects of the reminder/recall-based interventions exhibited a gradual decline over time, which was consistent with a meta-analysis on influenza vaccination.²⁶ The widespread implementation of reminder/recall-based interventions around the world suggests that additional attention to the performance of such interventions should be maintained, which is important to determine whether the trend of a continued diminishing effect will persist in the future and to enable timely adjustments. Different from the conclusion of some studies on the relationship between age and intervention effect,^{24 26} our results showed that while increases in sample age initially led to an increase in the effectiveness of reminder/recall-based interventions, beyond a certain point, further increase in age was associated with a decrease in intervention effectiveness. This discrepancy may be due to our collection of detailed age data rather than grouping participants into rough age categories and our inclusion of both children and adults in the sample. Our study highlights the importance of targeting both younger and older samples when implementing reminder/recall-based interventions. We also observed that reminder/recall-based interventions were more effective in promoting the completion of series vaccines than vaccine uptake, implying that this advantage of reminder/recall-based interventions can be fully valued and used by project implementers and managers when targeting series vaccines.⁵⁹ Similar to a meta-analysis that observed significantly lower effectiveness of interventions in North America,²⁶ we observed lower effect of multi-component interventions on vaccination in North America than in East Asia & Pacific, Latin America & Caribbean and the Middle East & North Africa. This finding suggests that the implementation of large-scale vaccination-oriented multi-component interventions in North America should be re-examined and that the potential effectiveness of multi-component interventions needs to be emphasised during the intervention. Further, the data demonstrated that although multi-component interventions are recommended,^{66 67} interventions with different combinations of components can lead to differences in effectiveness. Specifically, the combination of dialogue, incentive and reminder/recall interventions turned out to be more effective than the other multi-component interventions, indicating that it is worth evaluating and balancing the costs and benefits of implementing multi-component interventions to fully consider their effectiveness and applicability.

These findings provide evidence that, to some extent, multi-component and dialogue-based interventions may be prioritised if the objective of an intervention is to enhance vaccination, regardless of cost and the implementation context. Furthermore, due to the variability in the effectiveness of interventions under diverse conditions, ensuring vaccination gains through interventions hinges on consideration and assessment of actual needs and circumstances, which makes the acquisition of such information an urgent research priority.

Interpretation of our results and associated recommendations raises additional issues. First, although our analyses were mainly based on intervention assessment, there was a lack of comparisons between specific interventions, and such studies are necessary to increase the strength of the evidence. Second, although assessing the effect of an intervention on vaccination behaviour was our best choice given data availability, this only covered one aspect of the intervention outcome. Comprehensive assessment should also consider other outcomes such as knowledge, vaccination intention, attitudes and intervention costs,⁶⁸,⁷⁰ which were important for understanding the full pattern of intervention effects and deserved thorough investigation. Third, we restricted the analysis of interventions’ effects to four main types of intervention. This was necessary in order to conduct meta-analysis with a sufficient number of studies for each type of intervention, thereby increasing the reliability of the estimates. However, it did not indicate that other interventions (eg, administrative interventions) had a lower effect on vaccination.⁷¹,⁷³ Fourth, we limited the individual characteristic variables in the meta-regressions to gender and age. But we could not exclude potential effects from other characteristics (including socioeconomic status and educational attainment),^{74 75} even though these variables may not be sufficiently represented for a thorough analysis. A further limitation relates to HPV vaccination policy changes, including the expansion of eligibility to males,⁷⁶,⁷⁸ extended age recommendations^{78 79} and country-specific adjustments,^{80 81} which may influence intervention effectiveness. However, the heterogeneity of these changes across countries and the lack of comprehensive pre- and post-policy data limited our ability to assess their impact. Similarly, this study could not account for multiple vaccinations in the same encounter. While receiving multiple vaccines concurrently (eg, COVID-19 and influenza) may affect vaccination attitudes and behaviours,^{82 83} the included studies did not specify whether vaccines were administered at the same time, preventing systematic analysis of this factor. In addition, socio-cultural factors play an important role in vaccination behaviour. While our meta-regression analyses did consider country-level cultural characteristics through Hofstede’s ‘individualism’ score to reflect societal attitudes towards individual freedom and choice,^{84 85} this measure does not fully capture broader societal trends such as ‘individual freedom’.⁸⁶ Our study also fails to directly analyse specific historical events or individual studies, such as Andrew Wakefield’s study,⁸⁷ which falsely linked the MMR vaccine to autism and had a profound and lasting impact on public trust and vaccine uptake.^{88 89}

Promoting vaccination through interventions is increasingly regarded as an important target on a global scale, and our results provide evidence regarding the effect of interventions on vaccination behaviour. Using data from a large sample, we conduct a worldwide evaluation to quantify the effectiveness of four major intervention themes and provide insights into the implementation of appropriate interventions based on actual needs and contexts by exploring and explaining variations in the effectiveness of these interventions. Despite extensive research aimed at evaluating the effect of interventions towards vaccination,^{17 90} the need for more investment in rigorous effect evaluation is pressing because effective interventions are being required globally to enhance vaccinations and overcome vaccine hesitancy. Considering the limited research and implementation of incentive-based interventions (especially non-financial incentives) and multi-component interventions combining different themes, it is necessary for future research to explore in depth the effectiveness of these interventions more deeply and comprehensively, which may help the development and design of interventions. With regard to incentive-based interventions, variations in health systems, such as publicly funded vs fee-for-service models, may influence how incentives operate in different contexts. In publicly funded systems, incentives such as food or medicine vouchers and small monetary rewards are used to enhance vaccination efforts. Fee-for-service systems, like those in the USA, often offer incentives such as gift cards or reduced-cost vaccinations to help offset individuals’ financial burden. It would be valuable to further explore these system-specific factors impacting the effectiveness of incentives on vaccine uptake, offering insights to refine and optimise incentive-based interventions across diverse public health contexts. Also, further investigation is encouraged to explore direct comparisons between different interventions. Given the reality that intervention approaches and outcomes are often constrained by factors such as funding limitations and the implementation process, future research should address unavoidable trade-offs between the intervention effect of promoting vaccination and other intervention goals.^{70 91 92} In addition, the COVID-19 pandemic posed challenges of public trust,⁹³ service disruptions^{94 95} and antivaccination,⁹⁶ which may affect the effectiveness of interventions. Therefore, we underscore the need for future studies to investigate these factors more comprehensively, especially as more pandemic-era data become available, to better understand and address the impact of pandemics on vaccination efforts. Finally, more evidence is needed to identify which specific factors influence the effectiveness of interventions, such as information frame, incentive amounts and frequency of interventions,²⁸⁹⁷,⁹⁹ to enable us to amplify broader implications for advancing scientific evidence in intervention formulation and prioritisation.

Data availability statement

Data are available upon reasonable request.