The Potential Cost-Effectiveness of HIV Vaccines: A Systematic Review

Abstract

Objective

The aim of this paper was to review and compare HIV vaccine cost-effectiveness analyses and describe the effects of uncertainty in model, methodology, and parameterization.

Methods

We systematically searched MEDLINE (1985 through May 2016), EMBASE, the Tufts Cost-Effectiveness Analysis (CEA) Registry, and the reference lists of articles following Cochrane and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Eligibility criteria included peer-reviewed manuscripts with economic models estimating the cost-effectiveness of preventive HIV vaccines. Two reviewers independently assessed study quality and extracted data on model assumptions, characteristics, input parameters, and outcomes.

Results

The search yielded 71 studies, 11 of which met the inclusion criteria. Populations included low-income (n = 7), middle-income (n = 4), and high-income countries (n = 2). Model structure varied, including decision tree (n = 1), Markov (n = 5), compartmental (n = 4), and microsimulation (n = 1). Most studies measured outcomes in quality-adjusted life-years (QALYs) gained (n = 6), whereas others used unadjusted (n = 3) or disability-adjusted life-years (n = 2). The range of HIV vaccine costs were $US1.54–75 in low-income countries, $US55–100 in middle-income countries, and $US500–1000 in the USA. Base-case incremental cost-effectiveness ratios (ICERs) ranged from dominant (cost offsetting) to $US91,000 per QALY gained.

Conclusion

Most models predicted HIV vaccines would be cost-effective. Model assumptions about vaccine price, HIV treatment costs, epidemic context, and willingness to pay influenced results more consistently than did assumptions on HIV transmission dynamics.

Electronic supplementary material

The online version of this article (doi:10.1007/s41669-016-0009-9) contains supplementary material, which is available to authorized users.

Key Points for Decision Makers

Most economic models predict HIV vaccines will be cost-effective.

Static and dynamic HIV transmission modeling methods found similar results.

Vaccine cost-effectiveness will likely depend on HIV prevalence, durability of protection, and price of regimen and boosts.

Introduction

The search for an HIV vaccine began over 3 decades ago and had a breakthrough in 2009 [1]. A phase III HIV vaccine trial in Thailand (RV144) found HIV vaccine efficacy (VE) of 31.2% over 3 years [2, 3]. Though the durability of protection was low, vaccine boosting 4–5 years later restored the immune response to HIV [4]. An efficacy trial in South Africa (HVTN 702) is ongoing to confirm the canarypox-based vaccine ALVAC-HIV and bivalent gp120 protein subunit boost evidence with funding from the National Institute of Allergy and Infectious Diseases (NIAID) and other members of the Pox-Protein Public–Private Partnership (P5). The vaccine product was modified to match the predominant HIV strain in Africa and includes the potentially more immunogenic adjuvant MF59. With an added vaccine dose at 12 months, totaling five injections, the regimen is expected to increase the magnitude and duration of vaccine-elicited immune responses [5]. Walensky et al. [6] previously defined several characteristics of a ‘good enough’ therapeutic vaccine for HIV-infected individuals to replace antiretroviral therapy (ART) in the USA, but the characteristics of a clinically and economically viable preventative HIV vaccine have not been defined. When licensed, HIV vaccines may complement or compete with other HIV-prevention interventions such as voluntary male circumcision, treatment as prevention, and now pre-exposure prophylaxis (PrEP). Decision makers now balance investment in the continued development of HIV vaccines and confirmatory trials in Southern Africa with other opportunities to incrementally improve combination HIV-prevention effectiveness [7]. Over the long term, many view the development of a safe and effective HIV vaccine as the only hope to completely eradicate AIDS [8, 9]. This article aims to review existing studies of HIV vaccine cost-effectiveness to identify characteristics of HIV vaccines that may be essential to the value and viability of vaccination.

Rationale

Cost-effectiveness research guides efficient spending of limited healthcare resources and also contributes to research and development (R&D) decision making and prioritization of early-phase products through a clinical trial pipeline. Multifaceted decision making and value assessment of vaccines in development often draw upon economic modeling predictions [10]. Many complex mathematical models have simulated the potential impact of HIV vaccines on transmission of the virus, but few have included costs or measured health outcomes for comparison with other healthcare investments. Unlike models of chronic diseases, sexually transmitted infections often require the addition of transmission dynamics to capture the indirect effect, or positive externality, of herd immunity. Brisson and Edmunds [11] previously showed the impacts of modeling, methodological, and parameter uncertainty on economic analyses of varicella vaccination and emphasized how choices in model development can lead to disparate results. Because few cost-effectiveness studies exist in this area, we aimed to assess methodological differences between studies that may influence implications of value from a vaccine. Comparing detailed characteristics of the small number of existing studies allows us to identify key methods or population characteristics that strongly influence results. As vaccine development progresses, it will be of key importance to assess both clinical and economic feasibility of widespread vaccination campaigns. In this review, we aimed to identify key methodological drivers of, and variability in, the potential cost-effectiveness of an HIV vaccine. Our aim for this work is to help facilitate an informed and successful roll out of a future vaccine. While the cost-effectiveness of PrEP for HIV prevention has been systematically reviewed previously [12, 13], this is the first review of HIV vaccine cost-effectiveness.

Objectives

The main objective of this review was to identify and summarize HIV vaccine cost-utility analyses to understand conditions where vaccination has the potential to be cost-effective or cost saving. On the basis of economic models identified through a systematic review, the secondary objective was to evaluate and compare modeling, methodological, and parameter uncertainty based on guidelines and best practices for dynamic transmission modeling. The range in approaches to address uncertainty provided a case study for methodological comparison of economic modeling in infectious diseases. This paper was targeted at clinical trial scientists and funders to guide identification of characteristics of an HIV vaccine that would be most critical to the economic viability of the vaccine.

Methods

We conducted a systematic review following methods from the Cochrane Collaboration and the Agency for Healthcare Research and Quality (AHRQ) guides for systematic reviews [14, 15]. Content aligns with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement for transparent reporting of systematic reviews [16].

Eligibility Criteria

Eligible articles were published in peer-reviewed journal articles from 1985 to May 2016 in the English language and included an analysis that predicted the economic impact of an HIV vaccine. Eligibility was limited to studies estimating incremental costs and health outcomes measured in units comparable across diseases, including quality-adjusted life-years (QALYs) gained, disability-adjusted life-years (DALYs) averted, or life-years (LYs). For ease of comparability, the review was limited to preventive vaccines of uninfected individuals and excluded therapeutic vaccines for existing HIV patients. Budget-impact analyses, HIV vaccine acceptability, and willingness-to-participate studies were excluded, as were editorial commentaries, conference abstracts, and book chapters.

Search Methods and Sources

Two reviewers independently searched databases using a pre-specified protocol. The PubMed and MEDLINE search included ((“cost benefit analysis” [medical subject heading (MeSH) terms] OR “cost-effectiveness” [all fields]) AND (“aids vaccines”[MeSH] OR “hiv vaccine”[all fields])). EMBASE was searched for the keywords ‘cost-effectiveness analysis’ AND (‘aids vaccine’ OR ‘hiv vaccine’). The Tufts Center for the Evaluation of Value and Risk in Health (CVER) Cost-Effectiveness Analysis (CEA) Registry was searched for “HIV vaccine” and “AIDS vaccine.” Authors screened the reference lists of relevant articles for additional studies not identified in the database searches.

Study Selection

Records identified through database and registry searches were merged into the reference manager Mendeley and duplicates removed. Two reviewers (author BA and acknowledged contributor NV) independently searched and screened the titles and abstracts of all identified records, excluded those not meeting the defined criteria, and assessed the full text of all remaining articles for eligibility. Disagreements or uncertainty in eligibility were discussed and resolved with input from a third reviewer (BD).

Data Extraction

Two reviewers (BA and DD) extracted model characteristics, methods, parameter values, and results from identified manuscripts by populating a standardized table. The following data elements were sought from each manuscript and its supplemental materials: population studied (region, demographics, local HIV prevalence and incidence), HIV vaccine characteristics (regimen, efficacy, durability, and price), model features (name, structure, outcome measure, perspective, discount rates, time horizon, years modeled, transmission dynamics, and assumptions), and results (incremental cost-effectiveness ratio [ICER], willingness-to-pay [WTP] or cost-effectiveness threshold, interpretation of cost-effectiveness results, sensitivity-analysis methods, and findings of sensitivity analyses). ICER was defined as the marginal cost per marginal health gained with the following equation:

$$\text{ICER}=\frac{\mathrm{\Delta }\text{Costs}}{\mathrm{\Delta }\text{Effectiveness}}.$$

To compare the magnitude of cost-effectiveness in relation to the corresponding country’s gross domestic product (GDP) and compare results across studies, we present raw study-reported ICERs and ‘standardized’ cost-effectiveness. Standardized cost-effectiveness was defined as the ratio of ICER to the study-defined WTP threshold, where standardized ICER/WTP values <1 are consistent with the reviewed study author’s interpretation as ‘cost-effective,’ values >1 are ‘unlikely cost-effective,’ and negative values are likely cost saving for the case of vaccines.

Critical Appraisal

The quality of model reporting was evaluated using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist [17]. Our comparisons of model uncertainty and results were guided by recommendations from the US Panel on Cost-Effectiveness in Health and Medicine [38, 39], the World Health Organization (WHO) Guide to Cost-Effectiveness [18], and Briggs [19, 42]. Reviewers assessed the frequency best practices were used as recommended by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR)–Society for Medical Decision Making (SMDM) Modeling Good Research Practices Task Force on dynamic transmission modeling [20]. The terms ‘cost-effectiveness’ and ‘cost-utility’ are used interchangeably through this review, aligning with language from the selected articles while recognizing that cost-utility is a sub-type of cost-effectiveness where the outcome is in units such as QALYs or DALYs.

Though types of model structures are not mutually exclusive, for simplicity the studies were categorized into general types. Decision trees include a series of chance nodes with the probability each outcome will occur using a series of branches. Markov and semi-Markov models describe transitions through health states of a cohort of patients over time. Compartmental models use a system of differential equations to describe a fluctuating population in health states over time. Finally, the microsimulation model structure describes individual agents with defined characteristics as part of a whole fluctuating population over time. Technical strengths and limitations of dynamic transmission assumptions were interpreted with infectious disease mathematical modeling texts by Keeling and Rohani [21] and Vynnycky and White [22].

Results

Selection of Studies

Of the 71 unique records identified from searches, the reviewers assessed 24 full-text articles for eligibility and excluded 13 articles during full-text assessment (Fig. 1). Table 1 summarizes the 11 economic modeling studies meeting the eligibility criteria for inclusion in this review [23–33].

Fig. 1.

Flow of information through the different phases of the HIV vaccine cost-effectiveness model systematic review process. CEA cost-effectiveness analysis, DALY disability-adjusted life-year, QALY quality-adjusted life-year

Table 1.

Summary of 11 economic models reviewed [23–33]

The parameters and ICERs represent the base-case or average values in each analysis and do not reflect ranges evaluated in sensitivity analyses. Blue shading: 2011 study is an update to the 2009 model by same author. Pale grey shading: Stover et al. [33] and Harmon et al. [30] studies both used the Goals model with Spectrum software, so parameters are similar. Dark grey shading: two models in 1 calendar year by the same first author with similar parameters

^aThese studies did not explicitly state their economic perspective, and the perspective listed here was deduced by review authors based on context. Two values are listed for several Harmon parameters to reflect the separate low-income country (LIC) and middle-income country (MIC) analyses

Characteristics of Studies

Methods, population studied, vaccine assumptions, and results are presented in Table 2 and in Fig. S1 in the Electronic Supplementary Material (ESM). The regions of interest included low-income (n = 7), middle-income (n = 4), and high-income countries (n = 2). Two studies using the Goals model (SPECTRUM software package, Avenir Health) included 26 countries in Africa, Asia, Latin America, and Eastern Europe [30, 33]. All models reported parameterization using local epidemiological data on HIV. Economic perspectives included the payer (n = 1), government (n = 1), health system (n = 5), and societal or limited societal (n = 4). All studies included a reference group scenario following regional HIV-prevention practices and the standard of care in HIV treatment. Stover et al. [33] also compared vaccines against scale-up of treatment as prevention (TasP) and explored combining vaccines with TasP and PrEP. The number of studies following each ISPOR–SMDM dynamic transmission task force best practice recommendation is provided in Table S1 in the ESM. Though best practice suggests varying the time horizon in dynamic transmission models, none of the studies varied the time horizon in sensitivity analyses.

Table 2.

Target population, modeling methods, vaccine characteristics, and cost-effectiveness results of the 11 studies reviewed

Attribute	N	%
Population and perspective
Regiona
South Africa	7	64
Thailand	4	36
United States	2	18
Other	2	18
Population scope
District	1	9
Single country	7	64
Several countries	3	27
Perspective
Payer	1	9
Government	1	9
Health system	5	45
Limited societal	4	36
Defined willingness to pay	9	82
Discounted costs, 3%	11	100
Discounted outcomes, 3%	8	73
Modeling methods
Model type
Decision tree	1	9
Markov or semi-Markov	5	45
Compartmental, ODE	4	36
Agent-based	1	9
Dynamic HIV transmission	5	45
Outcome measurement
LYs	4	36
DALYs	2	18
QALYs	6	55
Time horizon
10-year horizon	3	27
20- to 43-year horizon	3	27
Lifetime horizon	5	45
HIV vaccine characteristics
Age at vaccination
Infant	1	9
9–15 years	4	36
>15 years	6	55
HIV vaccine efficacy, average
<50%	7	64
>50%	4	36
Vaccine durability
Lifetime protection	3	27
Waning protection	8	73
Vaccine boosting	7	64
Price per seriesa
≤$US5	2	18
$US20	3	27
$US50–100	6	55
>$US500	2	18
Risk compensation	5	45
Results and conclusions
ICERs (per QALY, DALY, or LY)
Dominant, cost saving	2	18
$US3–100	4	36
>$US1000	5	45
Cost-effectiveness interpretation
Cost-effective	8	73
Unlikely cost-effective	2	18
No interpretation	1	9

DALYs disability-adjusted life-years, ICER incremental cost-effectiveness ratio, LYs life-years, ODE ordinary differential equations, QALYs quality-adjusted life-years

^aThe Goals model includes 24 countries and assumed a price of $US20 for low-income countries and $US55 for middle-income countries

The most frequent measure of intervention impact was incremental QALYs gained (n = 6). Two studies estimated DALYs averted, and the remaining three compared differences in total LYs. Bos et al. [23] incorporated the cost of counseling for an infant’s mother with the vaccination series that may parallel the future of vaccine-induced seropositive (VISP) counseling. No studies included a transient or long-term disutility associated with vaccination.

Model Structure

The earliest study in 2001 evaluated HIV vaccination with a simple decision tree. Two Markov models were published 5 years later and then followed by the first dynamic transmission cost-effectiveness model in 2009. Since 2009, four of the eight HIV vaccine economic models used differential equations to simulate HIV transmission (Table 2). The Markov, compartmental, and microsimulation-based models stratified the risk of HIV infection by age, sex, and/or risk group.

In contrast to the closed-cohort populations in the decision tree and Markov models, the compartmental and microsimulation models allowed for fluctuating populations, with susceptible individuals entering the population at their sexual debut. Models by Leelahavarong et al. [29] and Moodley et al. [31, 32] tried to overcome the memory-less nature inherent in Markov models by using tunnel states. Alternatively, a microsimulation [28] explicitly modeled sexual partnerships while accounting for heterogeneity between people. This structure makes it easier to model preferential/targeted vaccination and revaccination and was designed to reflect individual preferences for participation in healthcare and prevention programs.

Studies modeled HIV transmission as static (n = 6) or dynamic (n = 5). In the static models, including the decision tree and Markov models, the probability of HIV transmission at each time step remained constant over time. In the dynamic models, including ordinary differential equation and agent-based microsimulation models, infections depend on the number of infected and uninfected individuals at each time point as well as on the sexual mixing between different groups. As a result, dynamic models captured not only the direct effects of the intervention on the susceptibility of the vaccinated individuals but also the indirect effects on HIV transmission by accounting for the decreasing exposure to HIV over time (herd immunity). The static models did not address sexual mixing. The hypothesis of risk compensation, assuming that individuals exhibit riskier sexual behavior when they perceive themselves as protected from HIV, was explored in four of the models [27, 28, 30, 33]. Hontelez et al. [28] concluded that this potential change in risk following vaccination could nullify the epidemic impact in South Africa.

Vaccine Effectiveness

The average of 3-year HIV VE values across all studies was 51.3% (Fig. 2). Sensitivity analyses included a range of efficacy from 10 to 90%. Though uncertainty in HIV VE decreased following the Thai trial results in 2009, the average of estimates from models before 2009 (VE = 49%; n = 0.4) did not greatly differ from the average of the estimates after 2009 (VE = 52%; n = 7). Eight of the 11 studies assumed that VE protection remained constant over time, whereas three studies modeled efficacy declining over time since immunization. All studies published after 2009 included boosts from re-vaccination to ensure continuity of protection, following the results from the Thai trial [2]. Two studies from a 2011 theme issue of Vaccine (Nagelkerke et al. [34] and Long and Owens [27]) modeled VE decaying over time with a functional form of VE = 0.78 × exp^−0.06t, where t = time since vaccination in months.

Fig. 2.

Assumed HIV vaccine efficacy and price per series (log scale) across the 11 studies reviewed

Costs

All studies discounted total costs by 3% annually, and most described ranging this rate in the sensitivity analysis. Health outcomes were discounted at 3% in eight (73%) analyses [23, 25–29, 31, 32]. In the year of this review, HIV vaccines are not licensed and the market price has not been set. Modelers assumed the average cost of a vaccine series was $US1.54–75 in low-income countries, $US55–100 in middle-income countries, and $US500–1000 in the USA (Fig. 2). There was no observable trend in pricing assumptions by efficacy across studies (Fig. 4c). The cost of HIV care and treatment varied widely between studies according to local healthcare costs and contributed greatly to differences in cost-effectiveness results. No studies included future research and development costs for the HIV vaccine leading to licensure.

Fig. 4.

Vaccine price and efficacy relationship with standardized cost-effectiveness (ICER/WTP) stratified by country income level. The ICER uncertainty from Hontelez et al. [28] is reported in one direction as a result of the threshold analysis method to set the vaccine price, resulting in an ICER equal to the country-specific willingness to pay. Three authors are included twice to reflect different results from multiple publications (Long [26, 27] and Moodley [31, 32]) and analysis of two populations in one publication (Harmon et al. [30]). Amirfar et al. [24] and Stover et al. [33] did not explicitly state cost-effectiveness thresholds. The threshold from Harmon et al. [30] was applied to the Stover et al. [33] study as both model the same 26 countries. Standardized ICER = ICER/WTP. ICER incremental cost-effectiveness ratio. WTP willingness to pay per health unit gained

Cost-Effectiveness

Base-case ICERs ranged from dominant (cost offsets) to $US91,000 per QALY gained (Table 1; Fig. 3a). The Goals model studies [30, 33] found similar ICER estimates, ranging from approximately $US1000 to $US11,000 per QALY across 24 low- and middle-income countries [30, 33]. Nine of the 11 studies pre-specified a local WTP or cost-effectiveness threshold to interpret results. Base-case vaccination scenarios were not cost-effective in three of the 11 models, including one low-, one middle-, and one high-income country (Fig. 3b). Stover et al. [33] did not explicitly interpret the cost-effectiveness of their results, so we inferred the cost-effectiveness threshold from the supporting information from Harmon et al. [30]. In the rest of the studies, three projected that vaccinations in their base-case scenarios were cost-effective, three were highly cost-effective, and two were cost-saving. Several models used sensitivity analyses to identify target populations for cost-saving vaccination. All modelers agreed that targeting groups with the highest HIV incidence improves cost-effectiveness. As expected, WTP-standardized ICERs increased as price increased (Fig. 4a). There was no clear trend in association between vaccine efficacy and cost-effectiveness across studies (Fig. 4b).

Fig. 3.

Cost-effectiveness studies of HIV vaccines. a Incremental cost-effectiveness ratio results from the base case of each study reviewed; error bars represent lower and upper ranges from the sensitivity analysis; b same as a with ICER standardized to willingness-to-pay threshold specified by study (see Table 1). The ICER uncertainty from Hontelez et al. [28] is reported in one direction as a result of the threshold analysis method to set the vaccine price, resulting in an ICER equal to the country-specific willingness to pay. Two authors are included twice to reflect different results from multiple publications (Long [26, 27] and Moodley [31, 32]), while another presented results for two populations within one publication (Harmon et al. [30]). Amirfar et al. [24] and Stover et al. [33] did not explicitly state cost-effectiveness thresholds. The threshold from Harmon et al. [30] was applied to the Stover et al. [33] study as both model the same 26 countries. Standardized ICER = ICER/WTP. ICER incremental cost-effectiveness ratio, WTP willingness to pay per health unit gained

All studies evaluated parameter uncertainty based on one-way sensitivity analyses. Five used scenario analyses to understand uncertainty by changing several parameters at one time to observe the change in cost-effectiveness. Both models by Moodley et al. [31, 32] in 2016 performed multivariate sensitivity analysis using 1000 Monte Carlo simulations based on random draws from each variable distribution to evaluate the combined effect of parameter uncertainty on the study results. Appropriate to the differing structural form, Hontelez et al. [28] correspondingly characterized combined uncertainty with simulation of 1000 individuals and scenarios. Three of the 11 studies reported model validation of epidemic predictions using historical HIV surveillance data. Scenario analyses that focused on men who have sex with men and injection drug users were found to be more cost-effective than vaccination of the general population. Several studies discussed how a targeted immunization strategy for high-risk sub-populations could result in cost savings for a health system. The microsimulation structure captured patient heterogeneity and sexual mixing more specifically and intentionally than the other economic models and reached very similar conclusions.

Discussion

This systematic review aimed to identify HIV vaccine characteristics and vaccination conditions that may be critical for an HIV vaccine to be cost-effective and valuable. The comparison of the published modeling studies suggested HIV vaccines with an average of 50% efficacy waning over 3 years and supplemented by boosting every few years may be a realistic profile for a ‘good enough’ vaccine to make a large impact on the HIV epidemic. As shown in this collection of economic models, a moderately effective vaccine could be cost-effective in developed and developing countries. The potential cost-effectiveness of vaccination was strongly linked to the HIV burden in each population, but a formal meta-analysis was not conducted because of the heterogeneity between studies. Two studies [27, 29] that predicted HIV vaccines were unlikely to be cost-effective had in common a lower-incidence general population as the target for vaccination. Vaccines targeted to individuals at greater risk of HIV infection will improve cost-effectiveness within the health system, and models did not predict that sexual mixing patterns would dilute the effects of targeted vaccination.

The potential cost-effectiveness of HIV vaccines depended on price and average efficacy. As the price of an HIV vaccine series increased in sensitivity analyses, there reached a point where vaccination would no longer be cost-effective. This threshold for vaccine price depended on each country’s GDP per capita. Several studies explored the components contributing towards average efficacy, such as the rate of decay in immunogenicity, the corresponding durability of protection, the proportion of people who respond, and the frequency of boosting. Our qualitative review indicated a moderately effective vaccine profile with poor durability and frequent boosting could have greater impact on the HIV epidemic than a single vaccination series with improved durability. This suggests future studies in non-human primates and humans should carefully evaluate the change in breadth and depth of immunological response following repeated boosting every 2–5 years.

While we did not find any infectious disease modeling method reviewed to be more valid than others, the structure and assumptions should be carefully selected based on the question of interest and data available. As HIV-prevention and treatment guidelines change dramatically over time, future economic models in HIV prevention should clearly define the reference standard of care and consider including PrEP as a component in the reference for comparison. This need for standardized components, methods, and perspectives to enhance comparability among studies is further supported by a new report from the Second Panel on Cost-Effectiveness in Health and Medicine [38, 39].

The 11 studies published between 2001 and 2016 exemplified a diverse range of model structure choices, economic methods, and population-specific parameter values. Almost half (n = 5) of the studies did not clearly state or define an economic perspective. Impact measured in unadjusted LYs gained did not consider quality of life and may have underestimated the total benefit from vaccination [24, 28, 32]. Fundamental to the epidemiology of infectious diseases, one expects the average age at diagnosis to increase as HIV prevalence decreases and exposure to infection is delayed. For example, two infected lives ending at 60 years from death unrelated to HIV have the same value in LYs as if HIV infection occurred at 30 or 55 years of age. Alternatively, studies estimating QALYs or DALYs capture a difference in total health based on the preference for more healthy years lived before HIV infection. A surprising majority of studies measured impact in QALYs when considering a perception that DALYs are used more often than QALYs for developing countries. We assume the interpretation of QALYs gained versus DALYs averted is the same, though the potential for differences has been discussed elsewhere [36, 37].

If most of the health gains from a vaccine were accumulated late in the time horizon, then models discounting costs only [24, 30, 33] would be more likely to produce results that are cost-effective than studies that discount both costs and QALYs. Reports including plots of cumulative health outcomes, changes in HIV incidence over time, and sensitivity analyses with undiscounted costs and outcomes most effectively communicated the timing of initial vaccine investment, accrual of HIV care cost savings, and the time to capture significant population health gains.

Despite professional society best practice recommendations for dynamic transmission modeling [20], no studies presented results using more than one time horizon (supplementary materials (ESM)). Most studies justified their choice of model structure and conducted some sensitivity analyses of structural assumptions. A minimal modeling movement advocates that development of models should be as simplistic as possible to answer one question of interest [40]. The articles reviewed highlight that a benefit of simple models is the ease of interpretation. Evaluating and balancing the importance of clinical and epidemiological assumptions is vital for readers to assess the face validity of economic models. Each modeling choice has trade-offs, and while a behavioral HIV-prevention intervention may require model complexity of heterogeneous sexual networks with concordancy and preferential mixing, researchers of different interventions not affecting these dynamics may value the benefits of simplicity more than potential incremental validity gained.

We identified a diverse variety of modeling structures and assumptions applied to this infectious disease. Decision trees and Markov models are often developed to evaluate the cost-effectiveness of pharmaceuticals, though in this case only the compartmental and microsimulation models captured the indirect effects of vaccination. Fundamental features for HIV vaccines included local HIV incidence data, a clearly defined population, assumption of efficacy and its waning, the local WTP for health gains, and—importantly—the vaccine price. Like the results from Brisson and Edmunds [11], our qualitative review highlights that choices in (1) the model type and structure, (2) economic methods, and (3) parameter values all introduced uncertainty for decision makers.

This qualitative review had several limitations. The uncertainty in future HIV vaccine characteristics and small number of studies available posed a challenge in drawing definitive conclusions based on this review. As the epidemic context and economic conditions varied greatly between study populations, a meta-analysis was not possible. Over the years 2001–2016, when the 11 studies were published, the technology, access, and standards for HIV testing and treatment changed over time and by country. However, all the studies published after 2009 modeled similar regimens of multi-dose vaccination with waning efficacy over time that correspond to the vaccine candidates currently tested in clinical studies. Reference groups for comparison differ and limit the ability to compare across studies. We assumed methodological differences measuring health outcomes in units of QALYs, DALYs, and LYs did not change overall findings or ICER interpretation. There was potential for incomplete retrieval of published research manuscripts, and publication bias may have prevented analyses with inconclusive findings from being submitted to or accepted by journals. Well-crafted economic models of HIV vaccines presented in book chapters, conference presentations, or languages other than English were potentially missed.

Conclusion

The 11 published studies found HIV vaccines to be cost-effective under certain conditions. HIV vaccine cost-effectiveness depended most on efficacy, price, and HIV incidence in the target population. Country-specific cost inputs and WTP thresholds may explain differences in cost-effectiveness. The studies provided evidence that immunization with a modestly effective HIV vaccine is likely an efficient use of resources in the USA, Thailand, and several sub-Saharan African countries, though decision makers must balance the studies’ findings with acknowledgement of great uncertainty. The review suggests regional HIV epidemiology and assumed WTP thresholds were more influential on study findings than differences from a methodological choice of static or dynamic HIV transmission. The broad disciplinary range among authors affirms the need for interdisciplinary collaboration between health economists, epidemiologists, clinicians, infectious disease mathematical modelers, biostatisticians, and clinical trial scientists to develop valid and meaningful results.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Compliance with Ethical Standards

Conflict of interest and sources of funding

BA received support for this review in part from AHRQ fellowship award number T32HS013853, a fellowship from the American Foundation for Pharmaceutical Education (AFPE), and a Trainee Support Grant from the University of Washington Center for AIDS Research (CFAR), a National Institutes of Health-funded program under award number P30AI027757. The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ or the NIH. Blythe Adamson, Dobromir Dimitrov, Beth Devine, and Ruanne Barnabas have no conflicts of interest.

Data availability statement

The studies systematically reviewed are available from National Library of Medicine on PubMed.