A systematic review of health economic evaluations of vaccines in Brazil

ABSTRACT

Background: In Brazil, since 2005, the Ministry of Health requires Health Economic Evaluation (HEE) of vaccines for introduction into the National Immunization Program.

Objectives: To describe and analyze the full HEE on vaccines conducted in Brazil from 1980 to 2013.

Methods: Systematic review of the literature. We searched multiple databases. Two researchers independently selected the studies and extracted the data. The methodological quality of individual studies was evaluated using CHEERS items.

Results: Twenty studies were reviewed. The most evaluated vaccines were pneumococcal (25%) and HPV (15%). The most used types of HEE were cost-effectiveness analysis (45%) and cost-utility analysis (20%). The research question and compared strategies were stated in all 20 studies and the target population was clear in 95%. Nevertheless, many studies did not inform the perspective of analysis or data sources.

Conclusions: HEE of vaccines in Brazil has increased since 2008. However, the studies still have methodological deficiencies.

Introduction

Since the second half of the 20th century, scientific and technological advances allowed the increasing development of vaccines. In the last 15 years, vaccines preventing diseases of major importance in public health, such as pneumococcal, meningococcal, rotavirus and human papillomavirus (HPV) were marketed and have led to a large reduction in the burden of disease in countries that implemented immunization programs with high vaccine coverage.^1,2 However, the high costs of these new vaccines hamper their introduction into the National Immunization Programs (NIP), particularly in middle- and low-income countries, which have the greatest burden of disease and would benefit most from a vaccination program.

The financial resources available for health care are, and probably will always be, limited when compared with the needs. As many new vaccines become available, the decision making on which one to incorporate into a national program become more complex. Managers / decision makers often face strong pressure from producers, vaccine defenders and the public for the introduction of new technologies. Delaying the introduction of a vaccine that prevents a public health important disease or, on the other hand, adopting an unsustainable vaccination program that would need to be discontinued, may result in deleterious consequences for population health. In public health, the limited financial resources must be allocated effectively and equitably to achieve the best possible result. The decision on implementing a vaccination program should be based on a clear rationality and ample evidence. Health economic evaluation (HEE), when measuring both costs and health effects, is a tool to estimate the relative value of different health intervention strategies, contributing to a potentially more efficient allocation of resources. HEE is becoming an increasingly important factor for managers faced with decisions on introducing new vaccines into the NIP.^3,4

In Brazil, HEE is a relatively new field. Use of HEE in decision-making processes of new technology incorporation into the public health system (Sistema Único de Saúde, SUS) is also recent. In 2005, the Brazilian NIP childhood immunization schedule included BCG, oral polio, diphtheria-tetanus-whole cell pertussis-Haemophylus influenzae b (DTP/Hib), hepatitis B, measles-mumps-rubella (MMR) and yellow fever vaccines. New vaccines had been introduced into childhood routine immunization in developed countries, such as pneumococcal conjugate 7-valent, in the USA, and meningococcal C conjugate, in UK, with great impact on the burden of disease.^1,2 In the SUS, these new vaccines were available only for high-risk persons, but they were available at the private health system, increasing inequities in access to immunization. The Brazilian Pediatrics and Immunization Societies, as well as the public, put pressure on the Ministry of Health (MoH) to incorporate new vaccines into routine immunization. In this setting, the MoH requested HEE of new vaccines introduction into the NIP. A partnership between the MoH and an academic research team to conduct HEE of new vaccines was established in 2005. The MoH chose which vaccines would be evaluated and financially supported the projects, but had no role in designing and conducting the studies or in data analysis and interpretation of results. In 2011, the National Technology Incorporation Committee (Comissão Nacional de Incorporação de Tecnologias no SUS, CONITEC) was set up and Health Technology Assessment (HTA) and HEE became part of the essential criteria for decision on new technologies incorporation in the SUS.⁵

The aim of this study was to describe and critically analyze the full HEE on vaccines performed in Brazil and published from 1980 to 2013. A systematic review of the Brazilian HEE of vaccines may help to improve the methodological quality and use of these studies.

Methods

This systematic review is part of a larger research that systematically reviewed all the Brazilian HEE studies published from 1980 to 2013.⁶ This study followed the guidelines for systematic review of economic evaluation studies published by the Centre for Reviews and Dissemination – CRD.⁷ The methodology of the systematic review and the search strategy have been described in details elsewhere.⁶

The search included multiple databases: MEDLINE (PubMed), EMBASE, Latin American and Caribbean Literature on Health Sciences Database (LILACS), Scientific Electronic Library Online (SciELO), repositories of economic evaluation studies: the Centre for Reviews and Dissemination (CRD): NHS Economic Evaluation Database (NHS EED), Health Technology Assessment Database (HTA), and Biblioteca Virtual em Saúde / Economia da Saúde (BVS ECOS), citation indexes: SCOPUS, Web of Science, and a bibliographic database of Brazilian Health Technology Assessments studies: the Sistema de Informação da Rede Brasileira de Avaliação de Tecnologia em Saúde (SISREBRATS). The search also included hand searching in the Brazilian Journal of Health Economics (BJHE), a journal non-indexed in the previously mentioned databases.

Articles were included if they were full economic evaluations of vaccines, according to the classification suggested by Drummond,⁸ conducted in Brazil, and at least one of the authors was affiliated to a Brazilian institution.

Studies were considered full economic evaluations if they compared costs and consequences of 2 or more health care interventions alternatives, including cost-consequences analysis (CCA), cost-minimization analysis (CMA), cost-effectiveness analysis (CEA), cost-utility analysis (CUA) and cost-benefit analysis (CBA).

The titles and abstracts of identified citations were screened for relevance independently by 2 reviewers (TCD, RL). Full texts of “relevant” and “unsure” articles were retrieved and independently screened by both reviewers.

Two reviewers (TCD, LMR) independently extracted data using a template developed specifically for this study. Data extracted from each study included: year and journal of publication, type of economic evaluation, vaccine type, immunization strategy (universal, group-specific), target population, first author affiliation (academy, government, research institutes, health organization, consulting, pharmaceuticals or equipment industry, international body), region of the first author, authors’ conflict of interest based on the definition used by Valachis,⁹ financial support, estimates of cost-effectiveness and study conclusion (favorable, unfavorable and neutral). Disagreements on the extracted data were resolved through discussion or through consultation with a third reviewer (PCS).

To compare the results of the studies, the incremental cost-effectiveness ratios (ICERs) were converted into 2013 Reals (R$). Whenever the study did not specify the year of cost, it was assumed that the year of cost was the same as that of the study publication. This strategy has been previously adopted in the literature.¹⁰ The monetary values of the results were corrected by the National Index of Consumer Prices Special Extended - IPCA-E, available at the Central Bank of Brazil calculator. We used the cost-effectiveness threshold proposed by the World Health Organization's CHOosing the Intervention that are Cost-Effective (WHO-CHOICE) as an indicator of cost-effectiveness.¹¹ The methodology quality of individual studies was assessed using items of the Consolidated Health Economic Evaluation Reporting Standards (CHEERS).¹² A qualitative narrative synthesis was conducted and the study characteristics were summarized in figures and summary tables.

Results

Of the 535 HEEs related to Brazil (Fig. 1), 20 were full HEEs that dealt with vaccines,^13-32 all them published between 1995 and 2013.

Figure 1.

Flow diagram of the process for the selection of HEEs related to Brazil, 1980–2013.

Figure 2 shows the growth in the number of HEEs on vaccines published in the period. From 2008 there was an increase in the annual number of publications and 2012 was the year with the highest number of published articles (n = 5).

Figure 2.

Health Economic Evaluation (HEE) on vaccines production in Brazil over the years, 1995–2013.

Table 1 shows the main features vaccination strategy of the 20 selected studies. The most common evaluated vaccines were pneumococcal vaccine (25%, 5/20)^21-24,29 and HPV vaccine (15%, 3/20).^26,27,31 Rotavirus^17,18 and varicella^16,19 vaccines were both evaluated in 2 (10%) publications. Two studies evaluated hypothetical vaccines, not yet available on the market at the time of the evaluation (dengue³² and hepatitis C,²⁰ one study each).

Table 1.

Characteristics of Health Economic Evaluation (HEE) on vaccines in Brazil, regarding vaccine type, vaccination strategy, and target population, 1995–2013.

Characteristics	N (%)
Type of vaccine
Pneumococcal	5 (25)
HPV	3 (15)
Rotavirus	2 (10)
Varicella	2 (10)
Meningococcal	1 (5)
Influenza	1 (5)
BCG	1 (5)
Hepatitis A	1 (5)
Hepatitis B	1 (5)
Hepatitis C	1 (5)
Dengue	1 (5)
Measles	1 (5)
Strategy
Universal	17 (85)
Specific groups	3 (15)
Target population
Children	10 (50)
Adults and children	2 (10)
Adolescents and children	2 (10)
Elderly	1 (5)
Women	1 (5)
Other specific population	3 (15)
Not declared	1 (5)

The vast majority of studies (19/20, 95%) evaluated vaccine introduction into the public national immunization program (NIP). There was only one exception – a study that evaluated influenza vaccination of healthy young adults employees of a pharma-chemical company, adopting the employer perspective.¹⁴ Universal vaccination strategy was assumed in 85% of the articles and the most frequent target population were children (50%). Adverse events after immunization (AEFI) were included in only one study, 8 justified the non-inclusion and 11 did not report. Regarding the vaccine administration costs, among the 14 studies that included it, 7 used international studies as data source, and 4 used expert opinion. Only 3 studies reported using local data.^13,14,30 Indirect effects of the vaccination program (herd protection) was considered in 40% (8/20) of the reviewed studies: in 5 studies, which assessed HPV (2 studies), varicella, hepatitis A and dengue vaccination programs,^19,26-28,32 the indirect effects were captured in the base case analysis by a dynamic model; and in 3 studies that assessed childhood pneumococcal conjugate vaccination using statistic decision tree model, the indirect protection was evaluated in a secondary or sensitivity analysis.^21-23 Six studies did not take indirect protection into account ^{14,17,18,24,25,29} and 6 did not inform. ^{13,15,16,20,30,31}

Table 2 and Table 3 present the main methodological characteristics of the selected articles.

Table 2.

Characteristics of Health Economic Evaluation (HEE) on vaccines conducted in Brazil, 1995–2013.

Characteristics	N (%)
Type of study
Cost-effectiveness analysis (CEA)	9 (45)
Cost-utility analysis (CUA)	4 (20)
Cost-effectiveness analysis / Cost-utility analysis	3 (15)
Cost-consequence analysis (CCA)	2 (10)
Cost-benefit analysis (CBA)	1 (5)
Cost-minimization analysis (CMA)	1 (5)
Journal of publication
International	16 (80)
National	4 (20)
Region of the country of first author affiliation
Southeast	17 (85)
North	2 (10)
Northeast	1 (5)
Type of institution of first author affiliation
Academia	13 (65)
Research institute	3 (15)
Consulting	2 (10)
Source of funding
Industry	6 (30)
Government and Research funding agency	6 (30)
Academia	4 (20)
Research institute	3 (15)
Consultancy and Industry	1 (5)
Conflicts of interest
Informed	7 (35)
Declared no conflicts of interest	2 (10)
Declared conflicts of interest	5 (25)
Present, according to Valachis 9	7 (35)
Study conclusion
Favorable	19 (95)
Unfavorable	0
Neutral	1 (5)

Table 3.

Summary of the main methodological characteristics of Health Economic Evaluations (HEE) on vaccines conducted in Brazil, 1995–2013.

Author, Year	Vaccine	Compared strategies	Year of costs	Outcome	Type of Study	Time horizon (years)	Discount	ICER*	ICER interpretation	Study conclusion	Financial support
Souza, 2009	Pneumococcal conjugate 13-valent	Children immunization vs Non vaccination	2009	DA, LYS	CEA	5	C:5% B:5%	R$2,777/LYS	<1 GDP per capita	Favorable	Industry
Souza, 2009	Pneumococcal conjugate 7-valent	Children immunization vs Non vaccination	2008	CA, DA	CEA	5	C:6% B:3%	Economic	Economic	Favorable	Industry
Vespa, 2009	Pneumococcal conjugate 7-valent	Universal immunization of children aged <1 y vs Non vaccination	2006	CA, LYS and DALY	CEA and CUA	10	C:3% B:3%	R$6,288/DALY	<1 GDP per capita	Favorable	Industry
Sartori, 2012	Pneumococcal conjugate 10-valent	Universal immunization of children aged <1 y vs Vaccination of high risk children	2004	DALY and DA	CEA and CUA	5	C:3% B:3%	R$35,397/DALY	1 – 3 GDP per capita	Favorable	Government and Research Funding Agency
Neto, 2011	Pneumococcal Polysaccharide 23-valent	Vaccination of adults aged >60 y vs Non Vaccination	2008	LYS	CEA	5	C:5% B:5%	R$14,399/LYS	<1 GDP per capita	Favorable	Industry
Vanni, 2012	HPV	Vaccination of pre-adolescents girls vs Non vaccination	2008	QALY	CUA	NI	C:5% B:5%	Economic	Economic	Favorable	Academia
Kawai, 2012	HPV	Vaccination of 12-year-old girls vs Vaccination of 12-year-old girls + catchup for 12 to 26 year-old women vs Non vaccination	NI	QALY	CUA	Lifetime	C:3% B:3%	R$374 to R$769/ QALY	<1 GDP per capita	Favorable	Consultancy/ Industry
Fonseca, 2013	HPV	Vaccination of 12-year-old girls (with screening) vs Non vaccination (with screening) vs Vaccination of 12-year-old girls (without screening) vs Non vaccination (without screening)	NI	QALY	CUA	Lifetime	C:5% B:5%	R$1,932/ QALY	<1 GDP per capita	Favorable	Research Institute
Constenla, 2008	Rotavirus	Children vaccination vs Non vaccination	2003	DALY, LYS, HA, and MVA	CEA and CUA	5	C:3% B:3%	R$1,106/ DALY	<1 GDP per capita	Favorable	Industry
De Soárez, 2008	Rotavirus	Universal vaccination of children at 2 and 4 months of age vs Non vaccination	2004	CA, DA, and LYS	CEA	5	C:6% B:6%	R$1,649 /LYS	<1 GDP per capita	Favorable	Government and Research Funding Agency
Santos, 2004	Varicella	History of varicella vs History of varicella + varicella serology, if negative or uncertain history vs History of varicella + Vaccination if negative or uncertain history vs Varicella serology for all + Vaccination of those with negative serology vs History of varicella + + varicella serology, if negative or uncertain history, and vaccination of those with negative serology	NI	SID	CCA	NI	NI	NA	NA	Favorable	Academia
Valentim, 2008	Varicella	Vaccination of 12-month children vs Vaccination of high risk persons	2004	CA, DA, and LYS	CEA	30	C:5% B:3%	R$26,600/LYS	1 – 3 GDP per capita	Favorable	Government and Research Funding Agency
De Soárez, 2011	Meningococcal C conjugate	Universal vaccination of children aged <1 y vs Vaccination of high risk persons	2006	CA, DA, and LYS	CEA	10	C:5% B:5%	R$31,695/LYS	1 – 3 GDP per capita	Favorable	Government and Research Funding Agency
Burckel, 1999	Influenza	Vaccination of adult workers vs Non vaccination	1997	CA	CBA	NI	NI	Net benefit	NA	Favorable	Industry
Pereira, 2012	BCG	Vaccination of school-aged children vs Treatment of disease	1997	CA	CEA	NI	C:5%	NI	Not possible to calculate	Favorable	Government and Research Funding Agency
Sartori, 2012	Hepatitis A	Universal vaccination of children aged <1 y vs Vaccination of high risk children	2008	Ca, DA, LYS, and DALY	CEA	24	C:5% B:5%	Economic	Economic	Favorable	Government and Research Funding Agency
Ferraz, 1995	Hepatitis B	Serological screening of healthcare workers and vaccination of susceptible persons vs Vaccination of all healthcare workers without previous serological screening	NI	NI	CMA	NI	NI	Cost difference	Not possible to calculate	Favorable	Academia
Massad, 2009	Hepatitis C	Hepatitis C vaccination vs Treatment of disease with interferon	NI	CA, DA	CEA	20	NI	NI	Not possible to calculate	Favorable	Academia
Durham, 2013	Dengue	Several vaccination strategies ‡	NI	DALY	CUA	Lifetime	NI	NI	Not possible to calculate	Favorable	Research Institute
Ohno-Machado, 2000	Measles	Compared several measles vaccination campaign strategies in case of epidemic‡‡ ‡‡	NI	Utility	CCA	NI	NI	NA	NA	Neutral	Research Institute

CA: case avoided, DA: death avoided, LYS: life year saved, HA: hospitalization avoided, MVA: medical visit avoided, SID: Susceptible individuals detected, QALY: quality adjusted life year, DALY: disability adjusted life year

CEA: cost-effectiveness analysis, CUA: cost-utility analysis, CCA: cost-consequences analysis, CBA: cost-benefit analysis, CMA: cost-minimization analysis

NI: Not informed, NA: not applicable

C: costs, B: benefits

GDP: gross domestic product

vs: vs.

2013 Brazilian GDP per capita: R$24,065

^*2013 Reals (R$)

^‡Children vaccination with 0–90% of vaccine coverage vs Mass vaccination including age groups 0–5 years, 0–15 years, 0–40 years, or the whole population vs Whole population vaccination with 0–90% of vaccine coverage

^{‡‡ ‡‡}Vaccination of susceptible children aged 9 months to 6 y and all children aged 6 to 14 years, in mobile units and healthcare centers vs Vaccination of susceptible children aged 9 months to 6 y and all children aged 6 to 14 years, in healthcare centers vs Vaccination of susceptible children aged 9 months to 6 y and susceptible children aged 6 to 14 years, in mobile units and healthcare centers vs Vaccination of all children aged 9 months to 6 y and susceptible children aged 6 to 14 years, in mobile units and healthcare centers vs Vaccination of all children aged 9 months to 6 y and all children aged 6 to 14 years, in mobile units and healthcare centers vs

Vaccination of susceptible children aged 9 months to 6 years, in healthcare centers vs Vaccination of susceptible children aged 9 months to 6 years, in mobile units and healthcare centers vs Vaccination of all children aged 9 months to 6 years, in healthcare centers vs

The most used types of HEE, according to the classification suggested by Drummond,⁸ were cost-effectiveness analysis (CEA, 45%),^{17,19-22,24,25,28,30} cost-utility analysis (CUA, 20%) ^26,27,31,32 and both CEA and CUA performed simultaneously (15%).^18,23,29

Sixteen of the 20 articles (80%) were published in international journals; 5 in scientific journals specialized in vaccines, 4 in public health journals, 5 in infectious diseases journals. Only 2 of them were published in health economics specialized journals, and one in medical informatics journal.

Most first authors (85%, 17/20) worked at institutions located in the Southeast of the country and 65% (13/20) of them were affiliated to academia. Six studies (30%) were funded by the pharmaceutical industry,^14,18,21-24 6 (30%) by the government and research funding agencies,^{17,19,25,28-30} and 3 (15%) were financed by research institutes.^15,31,32 Conflict of interest was reported in 35% (7/20) of the studies; 5 articles reported having conflict of interest. When we used the classification of conflict of interest according to Valachis,⁹ 7 studies were considered to have conflict of interest. Among the 6 (30%) studies funded by the pharmaceutical industry, only 2 declared conflict of interest.

Most studies (95%, 19/20) presented favorable conclusions for the vaccine incorporation. Only one study presented neutral conclusion.¹⁵ Thirteen studies (65%) presented the incremental cost-effectiveness ratio (ICER).^{17-19,21-29,31} One study presented the net benefit,¹⁴ and another presented cost difference.¹³ Based on WHO proposed criteria for cost-effectiveness to interpret the studies’ results,¹¹ considering the 13 studies that reported ICERs, the vaccination strategy was cost-saving in 3 studies; the ICER was <1 per capita gross domestic product (GDP) per DALY, considered as very cost-effective, in 7 studies; and the ICER was between 1 and 3 GDP/DALY, considered cost-effective, in 3 studies. The conclusions of all these 13 studies were favorable to the vaccines evaluated.

Overall, pharmaceutical industry sponsored studies were more likely to report more favorable results (very cost-effective, with ICER <1 per capita GDP) than government or academic sponsored ones (71.4% [5 out of 7] versus 15.4% [2 out of 13] p < 0.012). No statistically significant differences were identified for cost-saving results (14.3% [1 out of 7] vs. 15.4% [2 out of 13] p < 0.947).

Table 4 presents the methodological quality assessment of the selected articles. All studies presented the study question clearly. The target population was clearly mentioned in 95% of studies. The perspective of the study was presented in 70% of the publications and the comparative strategy was clear in 90% of cases. The time horizon was reported by 70% of the studies. In most studies (55%), the time horizon was equal to or greater than 1 y. Three publications (15%) applied a lifetime horizon. The discount rate was applied to costs and benefits in 65% of the publications.^{17-19,21-29,31} Only one study used the discount rate only for costs.³⁰

Table 4.

Methodological quality of Health Economic Evaluations (HEE) on vaccines conducted in Brazil, 1995–2013.

Items / Clear presentation	N (%)
Study question	20 (100)
Target population	19 (95)
Study perspective	14 (70)
Public Health System	5 (25)
Public Health System and Society	4 (20)
Health System and Society	2 (10)
Health System	1 (5)
Employer	1 (5)
Public Health System, Health System and Society	1 (5)
Comparators	18 (90)
Time horizon
1 to 5 years	6 (30)
10 to 20 years	3 (15)
21 to 30 years	2 (10)
Lifetime	3 (15)
Not informed	6 (30)
Discount rate
Applied only to costs	1 (5)
Applied to costs and benefits	13 (65)
Not informed	6 (30)
Sources for effectiveness (vaccine effectiveness/efficacy)
Randomized controlled trial	6 (30)
Literature review	5 (25)
Modeling study	2 (10)
Book	1 (5)
Not informed	6 (30)
Sources for epidemiological estimates (baseline disease incidence)
Modeling study	3 (15)
Reliable administrative database	3 (15)
Randomized controlled trial estimates	1 (5)
Descriptive studies	1 (5)
Literature review	1 (5)
Prevalence studies	1 (5)
National population based survey	1 (5)
Recent published results of prospective data collection	1 (5)
Surveillance data	1 (5)
Cohort study	1 (5)
Not informed	6 (30)
Sources for resource use estimates
Reliable administrative database	5 (25)
Expert opinion	2 (10)
Prospective data collection	1 (5)
Expert opinion and Prospective data collection	1 (5)
Published abstract	1 (5)
Medical records	1 (5)
Not informed	9 (45)
Sources for cost estimates
Reliable administrative database	7 (35)
Reliable administrative database and published studies	1 (5)
Reliable administrative database and Public prices database	1 (5)
Prospective data collection	1 (5)
Published studies	1 (5)
Previous published economic evaluation studies	2 (10)
Public and private prices databases	2 (10)
Not informed	5 (25)
Sources for utilities estimates
Direct utility assessment for the specific study	0
Indirect utility assessment	0
Previous published economic evaluation studies	1 (5)
Published studies	2 (10)
International body (WHO)	1 (5)
Not informed	3 (15)
Not applicable	13 (65)
Decision analysis model
Decision Tree	8 (40)
Markov	1 (5)
Dynamic	6 (30)
Not informed	5 (25)
Sensitivity analysis
Conducted	14 (70)
Univariate	5 (35.7)
Probabilistic	1 (5)
Univariate and Multivariate	6 (42,9)
Univariate, Multivariate and Probabilistic	1 (5)
Not informed	1 (5)
Not conducted	6 (30)
Results – ICER presentation
Reported	15 (75)
Not reported	3 (15)
Not applicable*	2 (10)

ICER: incremental cost-effectiveness ratio

^*ICER presentation does not apply to the 2 cost-consequence studies.

The sources of vaccine efficacy / effectiveness estimates were reported by 70% of the studies. The most common sources were randomized clinical trials (30%), followed by literature review (25%) and modeling studies (10%). One of the studies referred a book as a source of vaccine effectiveness estimate. Importantly, 30% of the studies (6/20) did not report the sources for the epidemiological estimates. The most commonly reported sources for incidence/prevalence of disease in these selected studies were modeling studies (15%) and reliable administrative databases (15%). Of the 20 studies, only 11 (55%) reported the source of the health resource utilization estimates. Reliable administrative databases were used by 25% of the studies and expert opinion was used by other 2 (10%). The other sources of resource utilization mentioned were prospective primary data (5%), published abstract (5%), and medical records (5%).

Five publications (25%) did not report the source of cost estimation. Reliable administrative databases were the most frequent source informed (35%).

None of cost-utility analysis studies used direct measurement (rating scale and its variant, the standard gamble, and the time trade-off) or indirect measurement in a specific study using instruments (such as, EQ-5D, SF-6D) to estimate utilities for the health states. The most frequently reported sources were previously published studies (10%).^26,31 In 3 cost-utility analysis studies, the source of utility estimation of health status was not mentioned.^15,27,32

Regarding the decision models developed in the studies, 40% used decision tree,^{15,17,21-25,29} 30% used dynamic model,^{19,20,26-28,32} 5% used Markov ³¹ and 25% of the studies did not report the model.^{13,14,16,18,30}

A sensitivity analysis was performed in 70% of the studies. Only one study used probabilistic analysis,³² the other studies conducted univariate and multivariate analysis (42.9%),^{14,17,23,26,28,29} univariate (35.7%),^{18,19,24,30,31} and univariate, multivariate and probabilistic (7.1%).²⁵ One study did not report the type of sensitivity analysis conducted.¹³

Discussion

The first HEE on vaccine performed in Brazil was published in 1995, but only 4 papers were published by 2007. The Brazilian production in the field increased from 2008 onwards.

This recent increase in the interest on HEE followed the international trend, and reflects the increasing importance of immunization programs in healthcare spending, as high-cost vaccines became available. WHO encourages the assessment of the economic and financial implications of incorporating a vaccine into the NIP.^4,33 The PanAmerican Health Organization (PAHO) launched the ProVac Initiative to support countries to strengthen the decision-making of new and underutilized vaccine introduction, incorporating HEE in this process, by providing HEE tools, methodological guidelines and training.³⁴ The interest in the field led to a recent expansion of the ProVac Initiative to other WHO regions.³⁵ Recent upward trend in the publication of HEE on vaccines was also noted in systematic reviews of studies published in Spain (from 2005), China (from 2010) and Canada (from 2002).^3,36,37 In Brazil, the MoH request for HEE to incorporate new vaccines into the NIP had a great impact on the field development, encouraging and shaping the scientific production, and promoting human resources training in the area. Most (80%, 16/20) of HEE on vaccines included in this review were published from 2008, 3 y after the MoH request for HEE when considering the introduction of new vaccines into the NIP, and 25% (5/20)^{17,19,25,28,29} of them resulted from the partnership between the Brazilian MoH and an academic research team. In 2010, this academic team became a ProVac Center of Excellence and contributed with the development of methodological guidelines and epidemiologic and costs estimates to validate some PAHO decision models.³⁸

Most studies included in this review (13/20, 65%) evaluated vaccines introduced in the Brazilian NIP within the last 10 y (pneumococcal conjugate, HPV, rotavirus, meningococcal C conjugate, and hepatitis A) reflecting the local scenario at the time. Pneumococcal conjugate and HPV vaccines, which are used in multi-dose schedules, had high costs soon after marketing, and triggered more debates in the media and intense social pressure for incorporation,^29,39 were subject of more studies (4 and 3 Brazilian studies, respectively). In the review of Spanish HEEs on vaccines, pneumococcal vaccine was also the most frequently evaluated (10 studies, 21.7%), followed by influenza and hepatitis B vaccines (8 studies, 17.4%, each).³ In China, hepatitis B (8 studies, 34.8%) and pneumococcal conjugate (5 studies, 21.7%) were the most evaluated vaccines.³⁶ In Canada, a country with more tradition in health economic evaluation, 42 studies were included in the review, and more types of vaccines were evaluated, but most studies were for pneumococcal (8), influenza (6), hepatitis B (5), HPV (5), pertussis (5) and meningococcal vaccines (4).³⁷ In Brazil, just one HEE on hepatitis B vaccination for healthcare workers and one HEE on influenza vaccine for adult workers were published, reflecting the incorporation of both vaccines in the NIP in 1999, before the MoH request for HEE of new vaccine introduction – hepatitis B vaccine in routine childhood schedule and influenza vaccine for adults aged >60 y and high risk persons.

AEFI has not been considered in the vast majority (19/20) of Brazilian HEEs on vaccines, following the trend in international studies. The systematic review of Spanish HEE on vaccines found that only 34.8% of studies included costs of treating AEFI.³ WHO guideline for standardization of economic evaluation of vaccination programs recommends that AEFI should be included in both costs and effects in case they are likely to have a substantial impact on the results of the analysis, depending on both their likelihood of occurrence and their severity.⁵ Vaccines considered for introduction into NIP are generally safe, but mild AEFI may need healthcare and consequently incur in costs. Not taking AEFI costs into account in HEE may result in more favorable results for the vaccine introduction.

Data on administrative costs of vaccination (healthcare workers time and training, injection supplies, infrastructure, cold chain for transport and storage, operational costs) in the Brazilian NIP is not available. Consequently, most (11) of the 14 studies that reported administration costs used international data or expert opinion. Among the 3 studies that reported using national data, one assessed influenza vaccination of a pharma-chemical company's workers adopting the employer's perspective, and the administration costs were based on a survey of Brazilian private providers.¹⁴ Another study evaluated hepatitis B vaccination of healthcare workers and estimated the local costs of administering the vaccine (personal and material) not including the cold chain and operational costs of the program.¹³ The third study assessed BCG vaccination of school-aged children and used an estimate of BCG vaccination cost from a study of costs of tuberculosis in Salvador,³⁰ which report a vaccination cost without specifying how it was estimated.⁴⁰

HEE of vaccines has its specificities.⁴¹ Vaccination programs may result in complex effects. The entire population is exposed to the effects of a vaccination program, even if only part of the population is vaccinated. Besides the direct protection of the vaccinees, when a large part of the population is immunized, the pathogen transmission decreases, leading to reduction in the incidence of disease in the entire population and indirect protection of unvaccinated persons (herd protection). This effect may also result in an increase in the average age at infection. Indirect effects may have a significant impact on the results of HEE of vaccines, mostly but not always, improving the cost-effectiveness of a vaccination program.^41,42 Dynamic transmission models can more easily capture indirect effects of a vaccination program, allowing better representation of reality. Indirect protection has been demonstrated to impact the results of HEE of pneumococcal conjugate vaccines.⁴² Eight (40%) of the reviewed Brazilian studies took the indirect effects of vaccination into account in the analysis.^{19,21-23,26-28,32} Among the Spanish HEEs on vaccines, 82% studies did not consider herd protection.³ The review of Chinese HEE noted that all 4 favorable studies on pneumococcal conjugate vaccine included indirect protection of unvaccinated in the model.³⁶ The Canadian review noted increasing complexity of mathematical modeling of studies over time, with the introduction of dynamic models.³⁷ All 4 Brazilian HEE on pneumococcal vaccine used static (decision tree) models but 3 of them incorporated indirect effects of the program in unvaccinated populations in a secondary or sensitivity analysis.^21-23 Dynamic mathematical models of pneumococcal transmission to evaluate both the direct and indirect effects of childhood vaccination have only recently been developed.⁴³

Cost-effectiveness analysis (CEA) was more frequently used in Brazilian HEEs on vaccines (9 studies, 45%), followed by cost-utility analysis (CUA) (4 studies, 20%). Just one cost-benefit analyses has been identified. In Brazil, utility weights are not available, and therefore utilities from other countries needed to be considered. Authors should analyze carefully the data sources and instruments used to collect available utility data from other countries and decide if they are transferable to the Brazilian setting. Pharmacoeconomic guidelines state that utility values obtained from other countries are, in general, not transferable because of cultural differences.⁴⁴ In Brazil, only recently EQ-5D and SF-6D were cross-cultural adapted and validated, as well as societal preferences weights were estimated.^45-48 The difficulty to derive a valid QALY measure is even greater for pediatric populations, because a valid and reliable preference-based measure of utility for child health states does not exist, and the novel approaches are still under methodological investigation.^41,49 HEEs of vaccination usually incorporate only health benefits (reduction in morbidity and mortality) and averted medical spending, undervaluing the vaccines. Even if the societal perspective is adopted, only productivity losses that arise from reduction in productive time due to illness or to care for a sick child are considered.^50,51 A broader approach has been proposed for HEE of public health interventions, to capture impact beyond the health sector, accounting for full benefits of vaccination, such as improved educational attainment, economic growth and reduced health disparities, derived from avoiding physical and cognitive impairment that may be caused by vaccine-preventable diseases.^50-53 None of the Brazilian HEE included in this review adopted this broader perspective.

Uncertainty in how to discount the long-term impact of the vaccination program is another issue. Uniform discounting (the same rates for costs and health effects) and constant rates over time are mostly used in HEEs. This approach may affect the ICERS, since costs and benefits occur at different times. This may be particularly important if the effects of the program begin only long after the intervention, as is the case of HPV vaccination.⁵⁴ Most (12/20, 65%) of the Brazilian studies reviewed applied uniform and constant discount rates for both cost and benefits, including the 3 HEE on HPV vaccine. Just one study on BCG applied discount only to costs,³⁰ and another study on varicella applied different discount rates for costs (6%) and benefits (3%).¹⁹

The authors of most papers (95%) were favorable to the intervention. Since Brazil does not have a locally established cost-effectiveness threshold, we used WHO’ CHOICE criteria,¹¹ as used by others, particularly in low-and middle-income countries,^55,56 to interpret the reviewed studies’ results. In all 13 reviewed studies that reported ICERs, the vaccination program was below the threshold of cost-effectiveness (<3 GDP/DALY). WHO’ CHOICE criteria for cost-effectiveness threshold has been discussed in the literature.^55,56 A criticism is that it “sets such a low bar for cost-effectiveness that very few interventions with evidence of efficacy can be ruled out," ⁵⁵ limiting the utility of the threshold for decision-making in public health. Additionally, it does not capture the affordability or sustainability of funding, nor the relative value of other interventions that compete for funding.^55,56 Recently WHO recommended not using these thresholds, alone, as a decision rule for funding decisions.⁵⁷

This systematic review identified a significant association between the conclusions of vaccines cost-effectiveness and pharmaceutical industry sponsorship (p < 0.012). Pharmaceutical industry sponsored studies were significantly more likely to report more favorable results than government or academic sponsored studies.

These results corroborate with other studies assessing the relationship between the source of funding and cost-effectiveness results. Examples include studies for statins in cardiovascular prevention,⁵⁸ oncologic drugs,^9,59-61 neuropsychiatric drugs,^62,63 and bisphosphonates for osteoporosis treatment.⁶⁴ The influence of industry funding on cost-effectiveness results has been considered a hot topic. Some authors suggest that industry funded studies may reflect the hidden biases of their sponsors, and that funders influence the assumptions made in cost-effectiveness analyses.⁶⁵ With respect to vaccine HEEs, the main concerns would be the manipulation of vaccine efficacy and price estimates, which can have a direct impact on the conclusion drawn.

Regarding the methodological quality, the research question and compared strategies were clearly stated in all 20 studies and the target population was clear in 19 (95%). However, Brazilian HEE on vaccines still have methodological deficiencies: 30% did not clearly inform the perspective of analysis; 20% did not inform time horizon; 30% did not inform sources for vaccine efficacy/effectiveness; 30% did not inform data sources for epidemiological estimates; 45% did not inform data sources for health service utilization; 25% did not inform data sources for costs estimates; and 30% did not report the type of sensitivity analyses performed. Published guidelines and checklist may help increase the quality of future HEEs.^5,12

A limitation of this paper is a possible failure to identify studies, particularly research reports and academic thesis not published as scientific articles or published in gray literature.

The centralized decision-making process of recommending vaccines and vaccination policies is a facilitator for using HEE. Adopting methodologies and presentation formats that decision makers understand, and presenting in a transparent way the choices and assumptions made in the analysis may enhance the use of HEEs in the decision process.

In Brazil, besides consideration on disease burden and the vaccine characteristics, the decision to incorporate a new vaccine into the NIP must be preceded by economic evaluation and subsidized by discussions in the Technical Advisory Committee on Immunizations (CTAI), a permanent forum, established since 1991 to advise the MoH actions on immunizations. Other factors, such as programmatic capacity, sustainability, affordability and budgetary impact should be taken into account. Since 2011, the demand for incorporation must also be analyzed by CONITEC and meet the conditions for the incorporation of new drugs, products and procedures into the SUS. In the last 10 years, 8 new and underutilized vaccines have been incorporated into the Brazilian NIP: rotavirus, in 2006; 10-valent pneumococcal conjugate (PCV10) and meningococcal C conjugate (MenC), in 2010; inactivated polio in 2012; varicella in 2013, and hepatitis A, in 2014, all them in the childhood schedule. HPV vaccine for teenage girls and adult tetanus-diphtheria-acellular pertussis vaccine (Tdap) for pregnant women were both introduced in 2014. HTA/HEE to support vaccine introduction were conducted for all them.^{17,19,25,28,29,39,66-67} It is not easy to demonstrate how decision makers used economic evaluations when incorporating a new technology. We searched the NIP Technical Reports of new vaccines introduction, from 2006 to 2014, for citation of the economic evaluations – 4 (PCV10, MenC, HPV and hepatitis A vaccines reports) of the 8 technical reports made reference to the cost-effectiveness analyses. We also searched CONITEC reports on vaccine incorporation. From 2011 to 2016, CONITEC evaluated (and recommended) the incorporation of 4 new vaccines (hepatitis A, MMRV, HPV and Tdap for pregnant women) into SUS; 3 of these reports (hepatitis A, MMRV and HPV) made reference to the cost-effectiveness studies.⁶⁸ These results indicate the existence of favorable conditions for the development of expertise in HEE in vaccines and that the quality of many of the studies was recognized.

Nowadays, the Brazilian NIP has one of the most complete and complex national vaccination schedule, providing 18 vaccines against 18 diseases. The NIP now faces new challenges, such as achieving / keeping up high vaccination coverage, improving vaccination registries, enhancing coverage and quality of AEFI surveillance, and expanding national production of these new vaccines in the shortest time and at the lowest price.

New vaccines for diseases of public health importance in tropical areas have been approved recently (dengue vaccine) or are in the pipeline (malaria and others). HEEs on these new vaccines will be required. This review showed some deficiencies in Brazilian HEE on vaccines, which must be overcome in future studies. Additionally, methods of HEE on vaccination programs are developing and researchers must face these new challenges.

Disclosure of potential conflicts of interest

Ana MC Sartori received speaker's fees from Sanofi Pasteur. The other authors declared no conflict of interest.

Acknowledgments

Patrícia C de Soárez, Ana MC Sartori and Hillegonda MD Novaes are researchers of the National Institute of Health Technology Assessment of the National Council of Technological and Scientific Development (Instituto de Avaliação de Tecnologia em Saúde – IATS/CNPq).

Funding

This study was supported by Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP, São Paulo Research Foundation), as part of a larger project entitled “Systematic review of health economic evaluations conducted in Brazil, 1980–2013” (FAPESP Research Grant no. 2012/22126–3), and by Conselho Nacional de Desenvolvimento Cientifico e Tecnológico (CNPq, National Counsel of Technological and Scientific Development) - CNPq Research Grant no. 305614/2013–4. The funders had no role in designing and conducting the study, in interpreting the results, in the writing of the manuscript or in the decision of submitting it for publication. This study was also supported with grants from the National Institute of Science and Technology for Health Technology Assessment (IATS) — CNPq/Brazil.

Author contributions

Decimoni T and De Soarez PC designed the research; Decimoni T, Leandro R and De Soarez PC performed the research; Sartori AMC, Rozman LM, Decimoni T, Novaes HMD, and De Soarez PC analyzed the data; Sartori AMC, Rozman LM, Novaes HMD, and De Soarez PC wrote the paper.