Abstract
Achieving and maintaining high vaccination coverage requires investments, but the costs and effectiveness of interventions to increase coverage remain poorly characterized. We conducted a systematic review of the literature to identify peer-reviewed studies published in English that reported interventions aimed at increasing immunization coverage and the associated costs and effectiveness of the interventions. We found limited information in the literature, with many studies reporting effectiveness estimates, but not providing cost information. Using the available data, we developed a cost function to support future programmatic decisions about investments in interventions to increase immunization coverage for relatively low and high-income countries. The cost function estimates the non-vaccine cost per dose of interventions to increase absolute immunization coverage by one percent, through either campaigns or routine immunization. The cost per dose per percent increase in absolute coverage increased with higher baseline coverage, demonstrating increasing incremental costs required to reach higher coverage levels. Future studies should evaluate the performance of the cost function and add to the database of available evidence to better characterize heterogeneity in costs and generalizability of the cost function.
Keywords: Vaccination, cost, effectiveness, intervention
Introduction
Vaccines represent some of the most cost-effective and cost-beneficial public health interventions [1, 2]. Immunization prevents an estimated 2 to 3 million cases of vaccine preventable diseases annually [3]. Estimates suggest that between 2001 and 2020, immunization will prevent over 20 million deaths and save an estimated $350 billion in cost-of-illness across 73 low- and middle-income countries [4]. Spurred by initiatives such as the World Health Organization (WHO) Expanded Program on Immunization (EPI) introduced in 1974, Gavi, the vaccine alliance, launched in 2000 [5], and the 2012 Global Vaccine Action Plan [6], global immunization coverage continues to improve [7, 8]. The global proportion of children receiving 3 doses of diphtheria-tetanus-pertussis (DTP3) vaccine increased from 5% in 1974 to 86% in 2015, albeit with intermittent periods of stagnation and decline [3, 9]. Despite the remarkable progress, many countries remain off track with respect to achieving the global goals of 90% immunization coverage nationally and 80% in each district [3, 10, 11]. An estimated 19.4 million children do not receive basic vaccines, and the WHO reports that increased immunization coverage could avert an additional 1.5 million premature global deaths annually [3].
Recent concerted efforts to improve immunization coverage strive to strengthen national routine immunization programs [12]. Increased use of health economic analyses in decision-making motivated recent studies that characterized the costs of immunization programs in order to inform programming and policy decisions [13–15]. Comprehensive multi-year plans (cMYPs) [16, 17] and aggregations of primary data collected from health facilities [18] provide some estimates of the costs of routine immunization programs at the national level and explore some determinants that explain cost variability within and between countries [15]. Such national level costs of immunization programs can support decisions about introducing new vaccines and expanding immunization programs [19, 20]. However, little is known of the costs of interventions to improve immunization coverage, which represents an important consideration for policy makers. Given the importance of improving coverage particularly to reach harder-to-reach subpopulations, we sought to estimate the non-vaccine costs of interventions to increase absolute immunization coverage.
Prior published reviews evaluated many interventions aimed at increasing immunization coverage [12, 21–26]. A review of the grey literature reported costs per dose of mass immunization campaigns in Cameroon, Senegal, and Turkey ranging from $0.91 to $1.24 (in US$1985–1987, i.e., $1.99 – $2.72 in US$2016) and reported the cost per fully vaccinated child ranging from $18.93 to $27.38 (i.e., $41.45 – $59.96 in US$2016) [21]. A systematic review identified 10 studies with immunization costs across low- and middle-income countries and found similar average costs per dose delivered for campaigns (range $1–20 in US$2001, i.e., $1.36 – $27.10 in US$2016) and for routine services (range $0.5–16 in US$2001, $0.68 – $21.68 in US$2016), and reported that the costs per fully vaccinated child ranged widely from $0.9 to $245 (in US$2001, i.e., $1.22 – $332.02 in US$2016) [22]. One review reported increasing costs at higher levels of baseline coverage [26]. Other existing primary literature on the costs of attaining higher immunization coverage relied largely on global level modelling [27] or national data [28]. Although economies of scale may exist in interventions to increase immunization coverage [26, 28], reaching the last remaining pockets of un- and under-vaccinated individuals may require special efforts and relatively higher costs per vaccine-recipient reached [22]. Translating effective interventions into sustained increased immunization coverage, with the associated health and economic benefits [2], requires political and financial commitment [29, 30].
While the existing literature provides some limited insights, no prior studies characterized a cost function of the non-vaccine costs of interventions to increase immunization coverage (i.e., the incremental costs of increasing coverage). Estimates of the non-vaccine cost per dose of interventions could help decision makers understand the financial requirements for reaching immunization coverage goals. In the absence of prior studies, we reviewed and synthesized the available evidence and developed a cost function of the non-vaccine cost per dose of interventions as a function of baseline coverage for relatively low- and high-income countries for campaigns and routine immunization to provide context that may support expanded immunization efforts.
Methods
We searched PubMed on March 25, 2017 to find the titles and abstracts of papers published in English that included a combination of the following terms: “immuniz*” or “vaccine*” and “cost,” “intervention,” “campaign,” “improv*,” “increas*,” “expand*,” and “coverage.” We screened the titles and abstracts to identify studies that describe specific interventions to improve immunization coverage for one or more vaccine preventable diseases that report both the costs and effectiveness of the interventions. We excluded studies that did not include interventions and studies that targeted animal immunizations. We reviewed the full text of articles for which we could not determine relevance based only on title and abstract. We contacted the authors of studies that reported intervention effectiveness only and requested estimates of the associated intervention costs. We excluded any studies lacking estimates of both costs and effectiveness. We also reviewed studies included in systematic reviews identified by the search and relevant studies not captured in our PubMed search that we identified in the references of papers, but this did not lead to the inclusion of any additional studies.
All authors independently reviewed all of the literature assessed for eligibility and reached consensus on studies meeting inclusion criteria. The second author (TTY) conducted the initial extraction of the data from the included studies, while the first author (SO) and the last author (KMT) independently reproduced the extracted data. We resolved discrepancies in the interpretation of the extracted data through discussion. For each included study, we extracted information on the specific intervention, country setting, target population, vaccines targeted, intervention costs (excluding vaccine costs), and baseline and final immunization coverage. We categorized the interventions by as either occurring in campaign settings (i.e., supplementary immunization activities or SIAs) or in routine immunization (RI) settings, the latter of which we further categorized as: introduction of routine immunization (Ir), education (Ed), reminders (Re), screening and referral (SR), or health system strengthening (Hs). We also categorized interventions as demand-side if they sought to increase the utilization of vaccines and/or adherence to immunization schedules, supply-side if they increased immunization supply or addressed health system barriers to immunization uptake, or both. We characterized countries as relatively low (RL) or relatively high (RH) income based on the World Bank Income Level at the time of the study for the countries in which the studies occurred (i.e., assigning low- and lower middle-income countries to RL, and high- and upper middle-income countries to RH) [31]. For studies that reported costs as an average cost per number of participants, we estimated the total intervention cost from the average cost and number of participants. We converted all intervention costs reported in a foreign currency to US$ using an online historical foreign currency exchange tool [32]. We converted all intervention costs to 2016 US dollars (US$2016 and henceforth simply $) using the consumer price index [33]. For studies with unclear information, including ambiguity about the potential inclusion of the costs for the vaccines in reported intervention costs, we contacted the study authors to request clarification or additional information.
For studies that reported final and baseline immunization coverage for an intervention, we calculated coverage change as the percent difference between the final and baseline values. For studies reporting only coverage differences between one or more intervention groups and a control group, we computed the percent difference between intervention coverage and the control coverage. For studies that reported coverage changes across multiple vaccines, we averaged the changes in coverage. For studies with an unvaccinated baseline population, we calculated coverage change as the final proportion of the population vaccinated after the intervention. Finally, for studies with more than one intervention and for which the authors reported intervention costs and coverage information separately, we reported the multiple interventions separately (see technical appendix).
We extracted the number of vaccine doses delivered in each intervention and the cost per dose reported in the studies when reported. For studies that did not report these, we estimated the number of doses and then estimated the cost per dose. We determined the number of doses according to the coverage outcome definition for each study and then used coverage change information to derive the number of doses delivered. For studies targeting multiple vaccines, we used the number of doses for all vaccines and the number of participants attaining full vaccination status (see technical appendix). We also extracted the baseline immunization coverage in the target population and area, above which the intervention sought improvement.
We used statistical models to examine the relationship between intervention cost per dose and changes in coverage as a function of baseline coverage, delivery (i.e., SIA or RI), and relative income level (RL or RH). We explored other factors, such as the inclusion of vaccine delivery activities or reported externalities. We examined outliers in the data to understand the study context and we excluded very poor and dominated interventions (i.e., studies with very high costs or very low or negative coverage improvements that should not represent real options for future investments). Due to the right-skewed nature of cost data, we performed an ordinary least squares (OLS) regression with a logarithmic transformation of the intervention cost per dose per percent absolute change in coverage to predict the cost as a function of baseline coverage, type of intervention (i.e., SIA or RI) and relative income (i.e., RL or RH) [34].
Results
Figure 1 summarizes our literature search process that ultimately led to the inclusion of 42 studies [35–76]. Following the initial screening, we reviewed the full text of 463 studies. We excluded 249 studies that applied economic modeling/analytical approaches to aspects of immunization but did not assess specific interventions to improve immunization coverage (e.g., analyses of immunization programs or proposed vaccine introduction strategies [77–89]). Although we found many studies (N=102) that reported on interventions that successfully increased coverage, most (N=54, 56%) did not report costs. We contacted authors of 58 studies that reported intervention effectiveness data but not costs, which yielded responses from the authors of 26 studies, 16 of whom reported not collecting cost data. We could not reach the authors of 32 studies. Cost information from author responses led to the inclusion of 3 studies [61, 67, 75]. After examining the data, we further excluded three studies for which we could not estimate the number of vaccine doses [90–92] and three dominated interventions [93–95].
Table 1 summarizes the data we extracted for 56 interventions from 42 studies by publication year and country [35–76]. Most interventions occurred in relatively high-income countries (i.e., N=40, 71% in RH, compared to 16 in RL) and most delivered in routine immunization (RI) (N=45, 80%) with only 11 (20%) interventions in campaign settings (SI). Among the RI interventions, reminders represented the most common type, with 27, 11, 8, 5, 4, and 1 studies reporting interventions that involved reminders, supplemental immunization activities, education, health system strengthening, screening and referral, and introduction of routine immunization, respectively. Over half of the studies (N=33) described demand-side interventions, 13 interventions targeted supply aspects of immunization, and 10 interventions targeted both supply and demand.
Table 1.
Country, year [Ref] | Relative Income* | Intervention category** | Vaccine no. & types*** | Intervention description | Cost per dose (US$2016)‡ | Baseline Coverage, Absolute Coverage Change§ |
---|---|---|---|---|---|---|
Cameroon, 1977 [35] | RL | Sup RI: Ir (a,b) | 5 (B, D, M, P, S) | Introduction of Expanded Program on Immunization into Yaounde city for children under 2 years | $ 0.5 | 22.6, 28.32 h,k |
Ecuador, 1989 [36] | RL | Sup SIA (a,b) | 4 (B, D, M, P) | National mass immunization campaign for children under 5 years | $ 1.58 | 43, 21 m |
Indonesia, 1991 [37] | RL | Sup SIA (a,b) | 1 (T) | Mass immunization campaign for women of child bearing age | $ 3.55 e | 27, 30 m |
Canada, 1992 [38] | RH | Dem RI: Re (c) | 1 (T) | Physician reminders for adult patients 20 years and older | $ 1.05 | 5.7, 19.6 n |
Canada, 1992 [38] | RH | Dem RI: Re (c) | 1 (T) | Telephone reminders for adult patients 20 years and older | $ 7.7 | 5.7, 20.8 n |
Canada, 1992 [38] | RH | Dem RI: Re (c) | 1 (T) | Letter reminders for adult patients 20 years and older | $ 8.87 | 5.7, 27.4 n |
Australia, 1998 [39] | RH | Both SIA(b,c) | 4 (D, H, M, P) | Nurse home-visit vaccination of children at-risk and behind based on IIS data | $ 62.66 | 93, 29.5 n |
USA, 1998 [40] | RH | Dem RI: Ed (a) | 1 (E) | Educational outreach campaigns, door to door case management, reminder notifications targeting Asian Pacific Islanders | $ 67.73 | 78, 24 m |
USA, 1998 [41] | RH | Dem RI: Re (c) | 3 (D, H, P) | Case management with home visits and phone follow-up targeting inner city African-American children | $ 503.65 | 85.8, 13.2 n |
USA, 1999 [42] | RH | Dem RI: SR (c) | 5 (D, E, H, M, P) | Screening for vaccination status and on-site nurse referral for women, infants and children at risk for measles in an urban city | $ 6.89 | 39, 36 m |
USA, 1999 [42] | RH | Dem RI: SR (c) | 5 (D, E, H, M, P) | Screening for vaccination status and on-site clinic referral for women, infants and children at risk for measles in an urban city | $ 8.43 | 38, 41 m |
USA, 1999 [42] | RH | Dem RI: SR (c) | 5 (D, E, H, M, P) | Screening for vaccination status and off-site referral for women, infants and children at risk for measles in an urban city | $ 16.77 | 32, 38 m |
Uganda, 1999 [43] | RL | Sup SIA (a,b) | 1 (C) | Mass cholera immunization campaign for South Sudanese refugees in Uganda | $ 0.35 | 0, 75.9 k,p |
USA, 1999 [44] | RH | Sup RI: Re (c,d) | 4 (D, H, M, P) | Immunization status tracking and prompting of health provider to immunize children not up to date at primary care sites serving impoverished and middle-class children | $ 69.82 | 76, 21 m |
USA, 2000 [45] | RH | Dem RI: Re (c) | 2 (D, P) | Computer generated telephone reminder messages for children under 2 years | $ 7.78 | 81.7 r, 9.3 n |
USA, 2000 [46] | RH | Dem RI: Re (c) | 2 (F, Q) | Educational brochures and telephone reminders for seniors living in a senior center | $ 178.97 | 60, 17.35 h,q |
USA, 2001 [47] | RH | Both RI: Ed (a,b) | 1 (E) | Community based education program, enrolling physicians in the vaccines for children program, home visits of children due for a vaccine dose targeting first generation children of Asian and Pacific Island origin | $ 192.71 | 82 r, 25.4 m |
Cambodia, 2006 [48] | RL | Sup RI: Hs (b,c) | 3 (B, D, M) | Budgeted coverage improvement planning with health managers, performance agreements, monitoring of improvement plans in 10 districts | $ 2.35 e | 67, 16 m |
India, 2007 [49] | RL | Dem RI: Ed (c,d) | 5 (B, D, M, P, T)† | Educational intervention for practices and providers based on assessed barriers and missed opportunities for immunization targeting resource-poor rural populations | $ 52.94 | 53, 19 m |
USA, 2007 [50] | RH | Dem RI: Re (a) | 1 (Q) | Telephone outreach and reminders for unvaccinated patients with chronic conditions and patients 65 years and older in managed care | $ 107.88 | 57, 9.5 h,q |
Pakistan, 2009 [51] | RL | Dem RI: Ed (c) | 2 (D, M) | Structured group discussions on vaccination benefits with adults in rural communities with low vaccination uptake | $ 12.14 | 56, 24.25 h,q |
USA, 2009 [52] | RH | Dem RI: Re (c,d) | 7 (D, E, H, M, P, Q, V) | Reminder postcards, reminder telephone calls, intensive case management and home visits for disadvantaged infants born in a urban safety net hospital | $ 337.96 | 75.6 r, 11 n |
India, 2009 [53] | RL | Sup SIA (a,b) | 1 (P) | Identification, documentation and vaccination of all newborns in communities with high-risk for polio transmission | $ 2.86 | 38, 65 k,p |
India, 2010 [54] | RL | Both SIA (b,c) | 4 (B, D, M, P) | Monthly reliable immunization camp with small incentives for villagers in rural India | $ 51.77 | 22, 39 m |
India, 2010 [54] | RL | Both SIA (b,c) | 4 (B, D, M, P) | Monthly reliable immunization camp for villagers in rural India | $ 39.14 | 22, 16 m |
Canada, 2011 [55] | RH | Dem RI: Re (c) | 1 (M) | Telephone reminders and home visits for children behind in MMR immunizations | $ 7.87 | 67.4, 6.6 m |
USA, 2011 [56] | RH | Dem RI: Re (c) | 3 (N, S, U) | Patient tracking, reminders, recalls and home visits for adolescents in a city | $ 40.61 | 46 r, 12.3 n |
USA, 2012 [57] | RH | Sup RI: Re (c) | 1 (F) | Electronic record reminders for health providers targeting pregnant women | $ 0.91 | 42, 19 m |
USA, 2012 [58] | RH | Both RI: Ed (b,c) | 1 (F) | Education, publicity, incentives and mobile vaccination carts for healthcare personnel in a university hospital | $ 19.06 | 33.4, 6 m |
USA, 2012 [58] | RH | Dem RI: Ed (b,c) | 1 (F) | Education, publicity, and incentives for healthcare personnel in a university hospital | $ 13.28 | 32.2, 9 m |
USA, 2012 [58] | RH | Both RI: Ed (b,c) | 1 (F) | Education, publicity, and mobile vaccination carts for healthcare personnel in a university hospital | $ 14.43 | 31.5, 7 m |
USA, 2012 [58] | RH | Dem RI: Ed (b,c) | 1 (F) | Education, publicity for healthcare personnel in a university hospital | $ 4.29 | 34.3, 3 m |
USA, 2012 [59] | RH | Dem RI: Re (c) | 1 (F) | Reminder/recall systems with letters and telephone calls for adolescents missing vaccine doses at practices in a metropolitan area | $ 9.08 | 51, 4.3 n |
USA, 2012 [60] | RH | Dem RI: Re (c) | 3 (N, S, U) | Text message reminders for children in a low income urban area who had not received the influenza vaccine | $ 10.37 | 40.3, 11 n |
Bangladesh, 2012 [61] | RL | Both RI: Hs (a) | 3 (B, D, M) | Modified EPI session schedule, community support groups, provider training and removing geographical boundary barrier policies for children in hard to reach rural areas | $ 2.7 f | 57, 38 m |
Bangladesh, 2012 [61] | RL | Both RI: Hs (a) | 3 (B, D, M) | Screening checklist administered by healthcare workers to mothers at health facility, provider training and removing geographical boundary barrier policies | $ 3.67 f | 57, 28 m |
USA, 2013 [62] | RH | Dem RI: Re (c) | 7 (D, E, H, M, P, Q, V) | Population-based recall using centralized immunization information system for children behind in immunization in urban and rural counties | $ 1.03 | 40, 18.65 k,p |
USA, 2013 [62] | RH | Dem RI: Re (c) | 7 (D, E, H, M, P, Q, V) | Practice-based recall using centralized immunization information system for children behind in immunization in urban and rural counties | $ 3.79 | 38, 23.9 k,p |
Bangladesh, 2013 [63] | RL | Sup SIA (a,b) | 1 (C) | Mass immunization campaign for endemic urban population | $ 0.81 | 0, 72 k,p |
USA, 2014 [64] | RH | Dem RI: Re (c) | 1† | Reminders and recalls using statewide immunization information system for urban children missing doses | $ 3.55 | 73.4 r, 7 n |
Bangladesh, 2014 [65] | RL | Both RI: Hs (a) | 5 (B, D, E, M, P) | Extended clinic hours, vaccinator training, active surveillance, and community participation targeting children living in urban slums | $ 2.69 | 69, 56 m |
USA, 2014 [66] | RH | Dem RI: Re (c) | 1 (F) | Text message reminder for urban, low income pregnant women | $ 11.31 | 47, 2.7 n |
Thailand, 2015 [67] | RH | Sup RI: SR (c) | 4 (B, D, E, P) | Phone-to-phone information sharing application for identification of children needing vaccination among Highland minority and stateless populations along Thailand border | $ 54.45 f,g | 91.7, 2.7 m |
Somalia, 2015 [68] | RL | Sup SIA (a,b,d) | 2 (M, P) | Integrated human and animal vaccination outreach campaign targeting nomadic pastoralist populations | $ 3.35 e | 69, 31 k,p |
USA, 2015 [69] | RH | Dem RI: Re (c) | 7 (D, E, H, M, P, Q, V) | Collaborative centralized reminder/recall notifications for children behind in immunization schedule in urban and rural counties | $ 1.72 | 53.9, 12.8 k,p |
USA, 2015 [69] | RH | Dem RI: Re (c) | 7 (D, E, H, M, P, Q, V) | Practice based reminder/recall notifications | $ 0.22 | 53.9, 9.3 k,p |
USA, 2015 [70] | RH | Dem RI: Re (c) | 4 (N, S, U, V) | Text message reminders for adolescents at private pediatric and safety net practices | $ 2.64 | 46.5 r, 13.32 n |
USA, 2015 [70] | RH | Dem RI: Re (c) | 4 (N, S, U, V) | Postcard reminders for parents of adolescents missing vaccine doses in an urban area | $ 1.94 | 46.5 r, 11.04 n |
USA, 2015 [70] | RH | Dem RI: Re (c) | 4 (N, S, U, V) | Email reminders for parents of adolescents missing vaccine doses in an urban area | $ 1.26 | 46.5 r, 22.34 n |
USA, 2015 [71] | RH | Dem RI: Re (c) | 3 (N, S, U) | Text message reminders for parents of adolescents missing vaccine doses in an urban area | $ 4.33 | 92.9 r, 3.1 n |
USA, 2015 [72] | RH | Sup SIA (b,c) | 1 (F) | School based immunization for elementary school children in 20092010 flu season | $ 51.02 | 58.6, 11.2 n |
USA, 2015 [72] | RH | Sup SIA (b,c) | 1 (F) | School based immunization for elementary school children in 20102011 flu season | $ 50.66 | 58.6, 12 n |
USA, 2016 [73] | RH | Dem RI: Re (c,d) | 7 (D, E, H, M, p, Q, V)† | Text messaging reminders for Patients over 65 years or with chronic conditions in general practices | $ 10.86 | 88 r, 16.3 n |
UK, 2016 [74] | RH | Dem RI: Re (c) | 1 (F) | Cellphone and text messaging follow up of first time mothers by nurse practitioners | $ 2.88 | 49.9, 1.7 n |
Bangladesh, 2016 [75] | RL | Dem RI: Re (a) | 6 (B, D, E, F, H, M) | Mobile phone application to electronically register births and send reminders targeting children living in hard to reach rural areas and urban streets | $ 76.21 f,g | 49.8, 17.65 h,q |
China, 2016 [76] | RH | Both RI: Hs (c) | 5 (B, D, E, M, P) | Advocacy to increase EPI awareness and funding among local leaders, EPI standard setting, training of health staff, targeted communications, immunization supporting equipment, collaboration between sectors and communities | $ 1.76 g | 64.7, 21.1 m |
RH (high-income or upper middle-income); RL (low-income or lower middle-income)
Both: supply and demand; Dem: demand; Sup: supply; SIA: supplemental immunization activities; RI: routine immunization; Subcategories of RI: Ed: education; Hs: health system strengthening; Re: reminder; SR: screening and referral; Ir: introduction of routine immunization; a: targets unreached population; b: involves vaccine delivery; c: targets people covered by routine immunization; d: includes non-immunization components
B: Bacillus Calmette–Guérin (BCG); D: diphtheria tetanus pertussis (DTP); M: measles (includes measles conjugate vaccine (MCV), measles (M), measles rubella (MR), measles mumps rubella (MMR) vaccines); T: tetanus, F: Influenza/flu; H: Haemophilus influenzae type b (Hib); E: hepatitis B; P: polio (OPV/IPV); S: smallpox; V: varicella; Q: pneumococcal (PCV); C: cholera; N: meningococcal (MCV4); S: human papillomavirus (HPV), U: tetanus diphtheria pertussis (Tdap)
Antigens targeted inferred from official immunization schedules
e: costs include or possibly includes cost of vaccines; f: intervention cost estimates provided by study authors; g: implementation costs incurred by other entity not included in intervention cost
h: coverage change calculated as average of changes in coverage across multiple antigens or intervention groups; k: assumes baseline coverage is zero; m: coverage change calculated as final intervention coverage minus baseline intervention coverage; n: coverage change calculated as final intervention coverage minus final control coverage; p: coverage change calculated as final proportion vaccinated from a previously unvaccinated baseline population; q: baseline coverage as average of baseline coverages across multiple antigens; r: baseline immunization coverage in the target population and area extracted from historic state and local immunization registries or other sources other than reported in paper
Eighteen interventions (32%) involved vaccine delivery as part of the intervention activities [35–37, 39, 43, 47, 48, 53, 54, 58, 63, 68, 72]. Four interventions published in three other studies (9%) included objectives beyond increasing immunization coverage, such as vitamin A supplementation and improving maternal health outcomes [49, 52, 73]. Most of the interventions targeted increasing coverage for multiple antigens (N=36, 64%) while the remaining 20 interventions (36%) sought to increase coverage of a single vaccine antigen (e.g., influenza or tetanus). Most interventions (N=30, 53%) targeted multiple vaccines and reported changes in coverage for fully-vaccinated children instead of reporting individual antigen coverage changes.
The intervention costs per dose per percent increase in absolute immunization coverage for the 56 interventions ranged widely from $0.01 to $38.16, with a mean cost per dose per percent coverage change of $3.13 (standard deviation (SD) $7.02). Interventions from low-income countries appeared the least expensive (mean $0.06, SD $0.08) and those in high-income countries emerged as the most expensive (mean $3.66, SD $7.77). We estimated average cost per dose per percent change in coverage of $1.37 (SD $1.74) for campaigns and $3.56 (SD $7.74) for routine immunization interventions overall (i.e., not accounting for the significant effect of income level). Some studies [35, 36, 42, 44, 52, 56, 70] reported intervention cost per fully vaccinated person, which ranged from $8.48 to $4,213.
Table 2 presents the cost function estimating the intervention cost per dose to achieve a one percent absolute immunization coverage change, with baseline coverage, relative income, and delivery type as predictors. We found that the cost per dose per percent coverage change for interventions increased with higher levels of baseline coverage (i.e., lower relative improvements), as Table 3 illustrates, which suggests that it costs more to reach harder-to-reach populations. Overall, we found lower costs for interventions delivered in RI and in RL income settings compared to those delivered in SIAs and RH income settings. For lower levels of baseline coverage, both RI and SIA interventions showed similar low cost per dose per percent coverage change, suggesting economies of scale associated with targeting larger unvaccinated populations. As illustrated in Table 3 and Figure 2, SIA clearly cost more than RI interventions for higher levels of baseline coverage. We did not find other variables as statistically significant predictors of intervention cost per dose per percent change in immunization coverage, although this most likely reflects the relatively small number of studies for countries with lower income levels.
Table 2:
β | exp (β) | S. E. | P-value | |
---|---|---|---|---|
Baseline coverage | 0.0357 | 1.0363 | 0.0107 | *0.001 |
Country income category | ||||
RL** | [REF] | |||
RH | 1.5040 | 4.4999 | 2.7073 | *0.012 |
Delivery | ||||
SIA | [REF] | |||
RI | −0.4014 | 0.6693 | 0.4555 | 0.555 |
Constant | −3.1187 | 0.0442 | 0.0297 | *0.000 |
Root MSE = 1.7576
R-squared = 0.3134
Significant at p < 0.05
RH, relatively high-income (i.e., high-income or upper middle-income); RI, routine immunization; RL, relatively low-income (i.e., low-income or lower middle-income); SIA, campaign or supplementary immunization activity
Table 3:
Routine baseline coverage (%) | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Relative income** | 0 | 10 | 20 | 30 | 40 | 50 | 60 | 70 | 80 | 90 |
RL | $0.04 | $0.05 | $0.08 | $0.11 | $0.15 | $0.21 | $0.30 | $0.43 | $0.61 | $0.87 |
RH | $0.14 | $0.20 | $0.29 | $0.41 | $0.58 | $0.82 | $1.16 | $1.65 | $2.34 | $3.32 |
Relative income, delivery*** | 0 | 10 | 20 | 30 | 40 | 50 | 60 | 70 | 80 | 90 |
RL, RI | $0.03 | $0.04 | $0.06 | $0.09 | $0.12 | $0.18 | $0.25 | $0.36 | $0.52 | $0.74 |
RL, SIA | $0.04 | $0.06 | $0.09 | $0.13 | $0.18 | $0.26 | $0.38 | $0.54 | $0.77 | $1.10 |
RH, RI | $0.13 | $0.19 | $0.27 | $0.39 | $0.56 | $0.79 | $1.14 | $1.62 | $2.32 | $3.32 |
RH, SIA | $0.20 | $0.28 | $0.41 | $0.58 | $0.83 | $1.19 | $1.70 | $2.42 | $3.47 | $4.95 |
RH, relatively high-income (i.e., high-income or upper middle-income); RI, routine immunization; RL, relatively low-income (i.e., low-income or lower middle-income); SIA, campaign or supplementary immunization activity
Cost per dose per unit absolute coverage change = exp (0.0349 * Routine baseline coverage + 1.3432 * (0 if RL, 1 if RH) − 3.2881)
Delivery not statistically significant, but statistical model provided for context: Cost per dose per unit absolute coverage change = exp (0.0357 * Routine baseline coverage + 1.504 * (0 if RL, 1 if RH) − 0.4014 * (0 if SIA, 1 if RI) − 3.118)
Discussion
By providing the first evidence-based estimates of the incremental costs per vaccine dose associated with increasing immunization coverage, this analysis should help support policy makers as they improve vaccine delivery to reach more children with life-saving vaccines. Such efforts can inform national immunization program decision makers and donors as they pursue initiatives such as Gavi’s Innovation for Uptake, Scale and Equity in Immunization (INFUSE), which seeks to scale up innovations in vaccine delivery [5]. The current evidence remains particularly limited in low- and middle-income countries, even though these settings present the largest gaps in immunization coverage and potentially gain the most from future intervention efforts.
The data on the cost of interventions to improve vaccination coverage remains limited by the lack of complete cost information across studies. Notably, many authors we contacted reported not systematically collecting information on intervention costs, which revealed missed opportunities to estimate the financial resources necessary to improve immunization coverage. We also found that the majority of studies that reported costs did not break down the cost components. Limitations in the data did not support consideration or characterization of heterogeneity in costs and heterogeneity in the available evidence limits the generalizability of our results. Given the importance of economic evidence for planning and securing continued investment in immunization, our findings highlight the need to improve reporting of costs of interventions that aim to improve immunization coverage.
Consistent with a prior review of routine immunization and campaigns [26], we found increasing costs per dose for higher levels of baseline coverage. This may reflect the increasing difficulty of reaching relatively harder-to-reach members of populations characterized by higher overall levels of coverage. Further research on interventions would help to characterize the costs of interventions that can effectively increase coverage among hard-to-reach populations at a lower cost.
We found that interventions to improve coverage became more expensive for higher levels of baseline coverage, suggesting the diminishing returns associated with reaching harder-to-reach and smaller target populations. Reminders in relatively high-income countries emerged as the most common interventions and these interventions may become more economically attractive over time in countries of all income levels due to declining telecommunication costs and high levels of mobile phone penetration. Health system strengthening interventions emerged as among the least expensive interventions, suggesting that policy and process changes at the health system level can provide an economically attractive way to improve immunization.
We found higher non-vaccine intervention costs per dose per percent coverage change in relatively high-income country settings, and the lowest costs in relatively low-income countries, although we emphasize the limitations associated with extrapolation from a small number of studies. For instance, challenging situations (e.g., conflict settings or disaster response) may lead to higher costs. The overall higher level of immunization coverage in relatively high-income countries implies that interventions in these settings invariably targeted harder-to-reach segments of the population, but these countries also incur higher unit costs (e.g., higher wage rates). Our analysis shows the need for further studies that report immunization intervention costs in relatively low-income countries to strengthen the evidence base.
We note several limitations of our analysis. First, we focused on the literature published in English and indexed in PubMed, which may mean we missed some studies. Second, the lack of standardized reporting of costs and coverage data across studies limited the quantity and quality of the information extractable from the evidence. The variable reporting led us to rely on assumptions and inferences, which increased the uncertainty in our outputs. Third, the estimates of intervention costs provided by authors that did not appear in the original study remain subject to recall bias. Fourth, we sought to exclude the costs of vaccines from the overall intervention costs; however, for some studies we could not confirm whether the reported costs excluded vaccine costs. Thus, for the three studies in which we could not confirm exclusion of vaccine costs [37, 48, 68], the cost estimates may overestimate the costs of implementing the interventions; although we note that removing these studies does not significantly change the results. Fifth, the costs of interventions remain time- and context-specific, such that conducting a similar intervention in a different country and in different years may affect both the costs and effectiveness of the interventions. Sixth, given the relatively small number of studies overall, we did not assess the methodological quality of the included studies and we did not use a threshold for study quality as an exclusion criterion. Thus, the variable quality of the included studies represents an important limitation of our study.
Finally, the heterogeneity of study designs necessitated using different methods for calculating coverage changes, which may limit the validity and generalizability of our comparisons. Specifically, for some studies, we calculated overall coverage change as an average across multiple vaccine changes. For studies that lacked a comparator group, we calculated coverage change based on differences in pre- and post-intervention coverage that may overstate the changes in coverage. For studies for which we calculated coverage change as a difference between the intervention and comparator coverage, the estimates may not consider the impact of the baseline level of coverage. Furthermore, for studies that did not report the number of doses, we estimated the number of doses using the number of doses based on the implied definition for full vaccination in the paper and the number of people attaining full vaccination in the intervention. This approach, necessitated by the way most (53%) studies reported coverage change, may undercount vaccine doses delivered to people who did not attain full immunization status but who received partial protection from some doses. Future studies will need to address these and other limitations to support improved estimates of the incremental costs associated with increasing coverage.
The cost function presented here should inform country immunization stakeholders, international donors, and national immunization technical advisory groups as they estimate the resources needed to increase immunization coverage. As the world seeks to achieve the mission of the Decade of Vaccines, this analysis should also help global partners (e.g., WHO, UNICEF, Gavi and the Bill and Melinda Gates Foundation) characterize the costs of pursuing the Global Vaccine Action Plan to prevent millions of lives lost from vaccine-preventable diseases.
Supplementary Material
Highlights.
Interventions to increase immunization coverage require incremental investments
Many interventions report effectiveness estimates but not intervention costs
Costs per dose per percent coverage change increase with higher baseline coverage
Incremental costs differ significantly by country income
Evidence is currently limited for lower-income countries
Acknowledgments
We thank the following authors who provided us with further information about their publications: Gambo G. Aliyu, Deborah Bender, Guthrie Birkhead, Ruairi Brugha, Li Chen, Felicity Cutts, Gretchen Domek, Linda Fu, Madhav Goyal, Simon Gregson, John Grundy, Eelko Hak, Sharon Humiston, Jasmine Jacobs, Aiko Kaji, Jaranit Kaewkungwal, Anna Llupià, Saul Morris, Lance Rodewald, Paramita Sengupta, Melissa Stockwell, Md. Jasim Uddin, Patrick Vivier, Rachel L. Winer, and Anita Zaidi.
Funding
The authors acknowledge support for this work under Cooperative Agreement Number 6NU2RGH001913-01-02, funded by the U.S. Centers for Disease Control and Prevention. The contents of this work are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention or the Department of Health and Human Services.
Footnotes
Conflicts of interest
None.
References
- [1].Ozawa S, Mirelman A, Stack ML, Walker DG, Levine OS. Cost-effectiveness and economic benefits of vaccines in low- and middle-income countries: a systematic review. Vaccine. 2012;31:96–108. [DOI] [PubMed] [Google Scholar]
- [2].Ozawa S, Clark S, Portnoy A, Grewal S, Brenzel L, Walker DG. Return On Investment From Childhood Immunization In Low- And Middle-Income Countries, 2011–20. Health affairs (Project Hope). 2016;35:199–207. [DOI] [PubMed] [Google Scholar]
- [3].World Health Organization. Immunization coverage. World Health Organization; 2017. [Google Scholar]
- [4].Ozawa S, Clark S, Portnoy A, Grewal S, Stack ML, Sinha A, et al. Estimated economic impact of vaccinations in 73 low- and middle-income countries, 2001–2020. Bull World Health Organ. 2017;95:629–38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [5].Gavi The Vaccine Alliance. Infuse. Gavi The Vaccine Alliance; 2017. [Google Scholar]
- [6].World Health Organization. Global Vaccine Action Plan 2011–2020. Geneva, Switzerland: World Health Organization; 2013. [Google Scholar]
- [7].Greenwood B The contribution of vaccination to global health: past, present and future. Philosophical Transactions of the Royal Society B: Biological Sciences. 2014;369:20130433. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [8].Duclos P, Okwo-Bele J-M, Gacic-Dobo M, Cherian T. Global immunization: status, progress, challenges and future. BMC International Health and Human Rights. 2009;9:S2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [9].World Health Organization. Global routine vaccination coverage, 2012. Releve epidemiologique hebdomadaire; 2013;88:482–6. [PubMed] [Google Scholar]
- [10].Brown DW, Burton A, Gacic-Dobo M, Karimov RI, Vandelaer J, Okwo-Bele JM. A mid-term assessment of progress towards the immunization coverage goal of the Global Immunization Vision and Strategy (GIVS). BMC Public Health. 2011;11:806. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [11].World Health Organization. Meeting of the Strategic Advisory Group of Experts on immunization, April 2017 - conclusions and recommendations. Releve epidemiologique hebdomadaire. 2017;92:301–20. [PubMed] [Google Scholar]
- [12].Oyo-Ita A, Wiysonge CS, Oringanje C, Nwachukwu CE, Oduwole O, Meremikwu MM. Interventions for improving coverage of childhood immunisation in low- and middle-income countries. The Cochrane database of systematic reviews. 2016;7:Cd008145. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [13].Gauvreau CL, Ungar WJ, Köhler JC, Zlotkin S. The Use of Cost-Effectiveness Analysis for Pediatric Immunization in Developing Countries. The Milbank Quarterly. 2012;90:762–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [14].Black S The role of health economic analyses in vaccine decision making. Vaccine. 2013;31:6046–9. [DOI] [PubMed] [Google Scholar]
- [15].Menzies NA, Suharlim C, Geng F, Ward ZJ, Brenzel L, Resch SC. The cost determinants of routine infant immunization services: a meta-regression analysis of six country studies. BMC medicine. 2017;15:178. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [16].World Health Organization. Comprehensive Multi-Year Planning (cMYP): A Tool and User Guide for cMYP Costing and Financing. Geneva, Switzerland: World Health Organization; 2014. [Google Scholar]
- [17].Brenzel L What have we learned on costs and financing of routine immunization from the comprehensive multi-year plans in GAVI eligible countries? Vaccine. 2015;33 Suppl 1:A93–8. [DOI] [PubMed] [Google Scholar]
- [18].Brenzel L, Young D, Walker DG. Costs and financing of routine immunization: Approach and selected findings of a multi-country study (EPIC). Vaccine. 2015;33:A13–A20. [DOI] [PubMed] [Google Scholar]
- [19].De la Hoz-Restrepo F, Castaneda-Orjuela C, Paternina A, Alvis-Guzman N. Systematic review of incremental non-vaccine cost estimates used in cost-effectiveness analysis on the introduction of rotavirus and pneumococcal vaccines. Vaccine. 2013;31 Suppl 3:C80–7. [DOI] [PubMed] [Google Scholar]
- [20].Levin A, Wang SA, Levin C, Tsu V, Hutubessy R. Costs of Introducing and Delivering HPV Vaccines in Low and Lower Middle Income Countries: Inputs for GAVI Policy on Introduction Grant Support to Countries. PLOS ONE. 2014;9:e101114. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [21].Batt K, Fox-Rushby JA, Castillo-Riquelme M. The costs, effects and cost-effectiveness of strategies to increase coverage of routine immunizations in low- and middle-income countries: systematic review of the grey literature. Bull World Health Organ. 2004;82:689–96. [PMC free article] [PubMed] [Google Scholar]
- [22].Pegurri E, Fox-Rushby JA, Damian W. The effects and costs of expanding the coverage of immunisation services in developing countries: a systematic literature review. Vaccine. 2005;23:1624–35. [DOI] [PubMed] [Google Scholar]
- [23].Johri M, Pérez MC, Arsenault C, Sharma JK, Pai NP, Pahwa S, et al. Strategies to increase the demand for childhood vaccination in low- and middle-income countries: a systematic review and meta-analysis. Bulletin of the World Health Organization. 2015;93:339–46C. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [24].Odone A, Ferrari A, Spagnoli F, Visciarelli S, Shefer A, Pasquarella C, et al. Effectiveness of interventions that apply new media to improve vaccine uptake and vaccine coverage. Human vaccines & immunotherapeutics. 2015;11:72–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [25].Nelson KN, Wallace AS, Sodha SV, Daniels D, Dietz V. Assessing strategies for increasing urban routine immunization coverage of childhood vaccines in low and middle-income countries: A systematic review of peer-reviewed literature. Vaccine. 2016;34:5495–503. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [26].Johns B, Torres TT. Costs of scaling up health interventions: a systematic review. Health policy and planning. 2005;20:1–13. [DOI] [PubMed] [Google Scholar]
- [27].Wolfson LJ, Gasse F, Lee-Martin SP, Lydon P, Magan A, Tibouti A, et al. Estimating the costs of achieving the WHO-UNICEF Global Immunization Vision and Strategy, 2006–2015. Bull World Health Organ. 2008;86:27–39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [28].Bishai D, McQuestion M, Chaudhry R, Wigton A. The costs of scaling up vaccination in the world’s poorest countries. Health affairs (Project Hope). 2006;25:348–56. [DOI] [PubMed] [Google Scholar]
- [29].Chee G, Pielemeier N, Lion A, Connor C. Why differentiating between health system support and health system strengthening is needed. The International journal of health planning and management. 2013;28:85–94. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [30].Balabanova D, Mills A, Conteh L, Akkazieva B, Banteyerga H, Dash U, et al. Good Health at Low Cost 25 years on: lessons for the future of health systems strengthening. The Lancet. 381:2118–33. [DOI] [PubMed] [Google Scholar]
- [31].World Bank. World Bank Country and Lending Groups. World Bank; 2017. [Google Scholar]
- [32].OANDA. Currency Converter. OANDACorporation; 2017. [Google Scholar]
- [33].United States Department of Labor. Consumer Price Index. Bureau of Labor Statistics; 2017. [Google Scholar]
- [34].Malehi AS, Pourmotahari F, Angali KA. Statistical models for the analysis of skewed healthcare cost data: a simulation study. Health Economics Review. 2015;5:11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [35].Guyer B, Atangana S. A programme of multiple-antigen childhood immunization in Yaounde, Cameroon: first-year evaluation, 1975–1976. Bull World Health Organ. 1977;55:633–42. [PMC free article] [PubMed] [Google Scholar]
- [36].Shepard DS, Robertson RL, Cameron CS 3rd, Saturno P, Pollack M, Manceau J, et al. Cost-effectiveness of routine and campaign vaccination strategies in Ecuador. Bull World Health Organ. 1989;67:649–62. [PMC free article] [PubMed] [Google Scholar]
- [37].Berman P, Quinley J, Yusuf B, Anwar S, Mustaini U, Azof A, et al. Maternal tetanus immunization in Aceh Province, Sumatra: the cost-effectiveness of alternative strategies. Social science & medicine (1982). 1991;33:185–92. [DOI] [PubMed] [Google Scholar]
- [38].Rosser WW, Hutchison BG, McDowell I, Newell C. Use of reminders to increase compliance with tetanus booster vaccination. CMAJ : Canadian Medical Association journal = journal de l’Association medicale canadienne. 1992;146:911–7. [PMC free article] [PubMed] [Google Scholar]
- [39].Bond LM, Nolan TM, Lester RA. Home vaccination for children behind in their immunisation schedule: a randomised controlled trial. The Medical journal of Australia. 1998;168:487–90. [DOI] [PubMed] [Google Scholar]
- [40].Watson B Hepatitis B immunization of Asian Pacific Islanders in the United States. The Pediatric infectious disease journal. 1998;17:S38–42. [DOI] [PubMed] [Google Scholar]
- [41].Wood D, Halfon N, Donald-Sherbourne C, Mazel RM, Schuster M, Hamlin JS, et al. Increasing immunization rates among inner-city, African American children. A randomized trial of case management. Jama. 1998;279:29–34. [DOI] [PubMed] [Google Scholar]
- [42].Hutchins SS, Rosenthal J, Eason P, Swint E, Guerrero H, Hadler S. Effectiveness and cost-effectiveness of linking the special supplemental program for women, infants, and children (WIC) and immunization activities. Journal of public health policy. 1999;20:408–26. [PubMed] [Google Scholar]
- [43].Legros D, Paquet C, Perea W, Marty I, Mugisha NK, Royer H, et al. Mass vaccination with a two-dose oral cholera vaccine in a refugee camp. Bulletin of the World Health Organization. 1999;77:837–42. [PMC free article] [PubMed] [Google Scholar]
- [44].Rodewald LE, Szilagyi PG, Humiston SG, Barth R, Kraus R, Raubertas RF. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics. 1999;103:31–8. [DOI] [PubMed] [Google Scholar]
- [45].Dini EF, Linkins RW, Sigafoos J. The impact of computer-generated messages on childhood immunization coverage. American journal of preventive medicine. 2000;18:132–9. [DOI] [PubMed] [Google Scholar]
- [46].Krieger JW, Castorina JS, Walls ML, Weaver MR, Ciske S. Increasing influenza and pneumococcal immunization rates: a randomized controlled study of a senior center-based intervention. American journal of preventive medicine. 2000;18:123–31. [DOI] [PubMed] [Google Scholar]
- [47].Deuson RR, Brodovicz KG, Barker L, Zhou F, Euler GL. Economic analysis of a child vaccination project among Asian Americans in Philadelphia, Pa. Archives of pediatrics & adolescent medicine. 2001;155:909–14. [DOI] [PubMed] [Google Scholar]
- [48].Soeung SC, Grundy BM, Ly CK, Samnang C, Boreland M, Brooks A, et al. Improving immunization coverage through budgeted microplans and sub-national performance agreements: early experience from Cambodia. Asia-Pacific journal of public health. 2006;18:29–38. [DOI] [PubMed] [Google Scholar]
- [49].Pandey P, Sehgal AR, Riboud M, Levine D, Goyal M. Informing resource-poor populations and the delivery of entitled health and social services in rural India: a cluster randomized controlled trial. Jama. 2007;298:1867–75. [DOI] [PubMed] [Google Scholar]
- [50].Winston CA, Mims AD, Leatherwood KA. Increasing pneumococcal vaccination in managed care through telephone outreach. The American journal of managed care. 2007;13:581–8. [PubMed] [Google Scholar]
- [51].Andersson N, Cockcroft A, Ansari NM, Omer K, Baloch M, Ho Foster A, et al. Evidence-based discussion increases childhood vaccination uptake: a randomised cluster controlled trial of knowledge translation in Pakistan. BMC International Health and Human Rights. 2009;9:S8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [52].Hambidge SJ, Phibbs SL, Chandramouli V, Fairclough D, Steiner JF. A stepped intervention increases well-child care and immunization rates in a disadvantaged population. Pediatrics. 2009;124:455–64. [DOI] [PubMed] [Google Scholar]
- [53].Rainey JJ, Bhatnagar P, Estivariz CF, Durrani S, Galway M, Sandhu H, et al. Providing monovalent oral polio vaccine type 1 to newborns: findings from a pilot birth-dose project in Moradabad district, India. Bull World Health Organ. 2009;87:955–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [54].Banerjee AV, Duflo E, Glennerster R, Kothari D. Improving immunisation coverage in rural India: clustered randomised controlled evaluation of immunisation campaigns with and without incentives. BMJ (Clinical research ed). 2010;340:c2220. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [55].Lemstra M, Rajakumar D, Thompson A, Moraros J. The effectiveness of telephone reminders and home visits to improve measles, mumps and rubella immunization coverage rates in children. Paediatrics & Child Health. 2011;16:e1–e5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [56].Szilagyi PG, Humiston SG, Gallivan S, Albertin C, Sandler M, Blumkin A. Effectiveness of a citywide patient immunization navigator program on improving adolescent immunizations and preventive care visit rates. Archives of pediatrics & adolescent medicine. 2011;165:547–53. [DOI] [PubMed] [Google Scholar]
- [57].Klatt TE, Hopp E. Effect of a best-practice alert on the rate of influenza vaccination of pregnant women. Obstetrics and gynecology. 2012;119:301–5. [DOI] [PubMed] [Google Scholar]
- [58].Lin CJ, Nowalk MP, Zimmerman RK. Estimated costs associated with improving influenza vaccination for health care personnel in a multihospital health system. Joint Commission journal on quality and patient safety. 2012;38:67–72. [DOI] [PubMed] [Google Scholar]
- [59].Stockwell MS, Kharbanda EO, Martinez RA, Vargas CY, Vawdrey DK, Camargo S. Effect of a text messaging intervention on influenza vaccination in an urban, low-income pediatric and adolescent population: a randomized controlled trial. Jama. 2012;307:1702–8. [DOI] [PubMed] [Google Scholar]
- [60].Suh CA, Saville A, Daley MF, Glazner JE, Barrow J, Stokley S, et al. Effectiveness and net cost of reminder/recall for adolescent immunizations. Pediatrics. 2012;129:e1437–45. [DOI] [PubMed] [Google Scholar]
- [61].Uddin MJ, Saha NC, Islam Z, Khan IA, Shamsuzzaman, Quaiyum MA, et al. Improving low coverage of child immunization in rural hard-to-reach areas of Bangladesh: Findings from a project using multiple interventions. Vaccine. 2012;30:168–79. [DOI] [PubMed] [Google Scholar]
- [62].Kempe A, Saville A, Dickinson LM, Eisert S, Reynolds J, Herrero D, et al. Population-based versus practice-based recall for childhood immunizations: a randomized controlled comparative effectiveness trial. American journal of public health. 2013;103:1116–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [63].Khan IA, Saha A, Chowdhury F, Khan AI, Uddin MJ, Begum YA, et al. Coverage and cost of a large oral cholera vaccination program in a high-risk cholera endemic urban population in Dhaka, Bangladesh. Vaccine. 2013;31:6058–64. [DOI] [PubMed] [Google Scholar]
- [64].Dombkowski KJ, Costello LE, Harrington LB, Dong S, Kolasa M, Clark SJ. Age-specific strategies for immunization reminders and recalls: a registry-based randomized trial. American journal of preventive medicine. 2014;47:1–8. [DOI] [PubMed] [Google Scholar]
- [65].Hayford K, Uddin MJ, Koehlmoos TP, Bishai DM. Cost and sustainability of a successful package of interventions to improve vaccination coverage for children in urban slums of Bangladesh. Vaccine. 2014;32:2294–9. [DOI] [PubMed] [Google Scholar]
- [66].Stockwell MS, Westhoff C, Kharbanda EO, Vargas CY, Camargo S, Vawdrey DK, et al. Influenza vaccine text message reminders for urban, low-income pregnant women: a randomized controlled trial. American journal of public health. 2014;104 Suppl 1:e7–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [67].Kaewkungwal J, Apidechkul T, Jandee K, Khamsiriwatchara A, Lawpoolsri S, Sawang S, et al. Application of mobile technology for improving expanded program on immunization among highland minority and stateless populations in northern Thailand border. JMIR mHealth and uHealth. 2015;3:e4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [68].Kamadjeu R, Mulugeta A, Gupta D, Abshir Hirsi A, Belayneh A, Clark-Hattingh M, et al. Immunizing nomadic children and livestock--Experience in North East Zone of Somalia. Human vaccines & immunotherapeutics. 2015;11:2637–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [69].Kempe A, Saville AW, Dickinson LM, Beaty B, Eisert S, Gurfinkel D, et al. Collaborative centralized reminder/recall notification to increase immunization rates among young children: a comparative effectiveness trial. JAMA pediatrics. 2015;169:365–73. [DOI] [PubMed] [Google Scholar]
- [70].Morris J, Wang W, Wang L, Peddecord KM, Sawyer MH. Comparison of reminder methods in selected adolescents with records in an immunization registry. The Journal of adolescent health : official publication of the Society for Adolescent Medicine. 2015;56:S27–32. [DOI] [PubMed] [Google Scholar]
- [71].O’Leary ST, Lee M, Lockhart S, Eisert S, Furniss A, Barnard J, et al. Effectiveness and Cost of Bidirectional Text Messaging for Adolescent Vaccines and Well Care. Pediatrics. 2015;136:e1220–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [72].Yoo BK, Humiston SG, Szilagyi PG, Schaffer SJ, Long C, Kolasa M. Cost effectiveness analysis of Year 2 of an elementary school-located influenza vaccination program-Results from a randomized controlled trial. BMC health services research. 2015;15:511. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [73].Hannan J, Brooten D, Page T, Galindo A, Torres M. Low-Income First-Time Mothers: Effects of APN Follow-up Using Mobile Technology on Maternal and Infant Outcomes. Global pediatric health. 2016;3:2333794x16660234. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [74].Herrett E, Williamson E, van Staa T, Ranopa M, Free C, Chadborn T, et al. Text messaging reminders for influenza vaccine in primary care: a cluster randomised controlled trial (TXT4FLUJAB). BMJ Open. 2016;6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [75].Uddin MJ, Shamsuzzaman M, Horng L, Labrique A, Vasudevan L, Zeller K, et al. Use of mobile phones for improving vaccination coverage among children living in rural hard-to-reach areas and urban streets of Bangladesh. Vaccine. 2016;34:276–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [76].Zhou Y, Xing Y, Liang X, Yue C, Zhu X, Hipgrave D. Household survey analysis of the impact of comprehensive strategies to improve the expanded programme on immunisation at the county level in western China, 2006–2010. BMJ Open. 2016;6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [77].Ramaiah TJ. Cost-benefit analysis of the intensified campaign against smallpox in India. NIHAE bulletin. 1976;9:169–203. [PubMed] [Google Scholar]
- [78].Kaucley L, Levy P. Cost-effectiveness analysis of routine immunization and supplementary immunization activity for measles in a health district of Benin. Cost effectiveness and resource allocation : C/E. 2015;13:14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [79].Driessen J, Olson ZD, Jamison DT, Verguet S. Comparing the health and social protection effects of measles vaccination strategies in Ethiopia: An extended cost-effectiveness analysis. Social science & medicine (1982). 2015;139:115–22. [DOI] [PubMed] [Google Scholar]
- [80].Smith JC, Haddix AC, Teutsch SM, Glass RI. Cost-effectiveness analysis of a rotavirus immunization program for the United States. Pediatrics. 1995;96:609–15. [PubMed] [Google Scholar]
- [81].Bae GR, Choe YJ, Go UY, Kim YI, Lee JK. Economic analysis of measles elimination program in the Republic of Korea, 2001: a cost benefit analysis study. Vaccine. 2013;31:2661–6. [DOI] [PubMed] [Google Scholar]
- [82].Hutchinson P, Lance P, Guilkey DK, Shahjahan M, Haque S. Measuring the cost-effectiveness of a national health communication program in rural Bangladesh. Journal of health communication. 2006;11 Suppl 2:91–121. [DOI] [PubMed] [Google Scholar]
- [83].Huse DM, Meissner HC, Lacey MJ, Oster G. Childhood vaccination against chickenpox: an analysis of benefits and costs. The Journal of pediatrics. 1994;124:869–74. [DOI] [PubMed] [Google Scholar]
- [84].Golden M, Shapiro GL. Cost-benefit analysis of alternative programs of vaccination against rubella in Israel. Public health. 1984;98:179–90. [DOI] [PubMed] [Google Scholar]
- [85].Christensen H, Trotter CL. Modelling the cost-effectiveness of catch-up ‘MenB’ (Bexsero) vaccination in England. Vaccine. 2017;35:208–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [86].Mousavi Jarrahi Y, Zahraei SM, Sadigh N, Esmaeelpoor Langeroudy K, Khodadost M, Ranjbaran M, et al. The cost effectiveness of rotavirus vaccination in Iran. Human vaccines & immunotherapeutics. 2016;12:794–800. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [87].Hoshi SL, Kondo M, Okubo I. Economic evaluation of routine infant rotavirus immunisation programme in Japan. Human vaccines & immunotherapeutics. 2016:0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [88].Pitrelli A Introduction of a quadrivalent influenza vaccine in Italy: a budget impact analysis. Journal of preventive medicine and hygiene. 2016;57:E34–40. [PMC free article] [PubMed] [Google Scholar]
- [89].Le Gargasson JB, Nyonator FK, Adibo M, Gessner BD, Colombini A. Costs of routine immunization and the introduction of new and underutilized vaccines in Ghana. Vaccine. 2015;33 Suppl 1:A40–6. [DOI] [PubMed] [Google Scholar]
- [90].LeBaron CW, Starnes D, Dini EF, Chambliss JW, Chaney M. The impact of interventions by a community-based organization on inner-city vaccination coverage: Fulton County, Georgia, 1992–1993. Archives of pediatrics & adolescent medicine. 1998;152:327–32. [DOI] [PubMed] [Google Scholar]
- [91].Mayer JP, Housemann R, Piepenbrok B. Evaluation of a campaign to improve immunization in a rural headstart program. Journal of community health. 1999;24:13–27. [DOI] [PubMed] [Google Scholar]
- [92].Franzini L, Boom J, Nelson C. Cost-effectiveness analysis of a practice-based immunization education intervention. Ambulatory pediatrics : the official journal of the Ambulatory Pediatric Association. 2007;7:167–75. [DOI] [PubMed] [Google Scholar]
- [93].Robertson L, Mushati P, Eaton JW, Dumba L, Mavise G, Makoni J, et al. Effects of unconditional and conditional cash transfers on child health and development in Zimbabwe: a cluster-randomised trial. Lancet (London, England). 2013;381:1283–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [94].van Steenbergen JE. Results of an enhanced-outreach programme of hepatitis B vaccination in the Netherlands (1998–2000) among men who have sex with men, hard drug users, sex workers and heterosexual persons with multiple partners. Journal of Hepatology. 2002;37:507–13. [DOI] [PubMed] [Google Scholar]
- [95].Foley ME, Kory P, Fairbrother G. Evaluation of “Hope for a Million Kids Immunization Event: “process, outcome, and costs. Journal of public health management and practice : JPHMP. 1998;4:97–105. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.