Toward economic evaluation of the value of vaccines and other health technologies in addressing AMR

Abstract

We discuss the need to make economic evaluations of vaccines antimicrobial resistance (AMR)-sensitive and ways to do so. Such AMR-sensitive evaluations can play a role in value-for-money comparisons of different vaccines within a national immunization program, or in comparisons of vaccine-centric and non-vaccine-centric technologies within an anti-AMR program. In general terms, incremental cost-effectiveness ratios and rates of return and their associated decision rules are unaltered by consideration of AMR-related value. The decision metrics need to have their various health, cost, and socioeconomic terms disaggregated into resistance-related subcategories, which in turn have to be measured carefully before they are reaggregated. The fundamental scientific challenges lie primarily in quantifying the causal impact of health technologies on resistance-related health outcomes, and secondarily in ascertaining the economic value of those outcomes. We emphasize the importance of evaluating vaccines in the context of other potentially complementary and substitutable nonvaccine technologies. Complementarity implies that optimal spending on each set of interventions is positive, and substitutability implies that the ratio of spending will depend on relative value for money. We exemplify this general point through a qualitative discussion of the complementarities and (especially the) substitutability between pneumococcal conjugate vaccines and antimicrobial stewardship and between research and development (R&D) of a gonorrhea vaccine versus R&D of a gonorrhea antibiotic. We propose a roadmap for future work, which includes quantifying the causal effects of vaccination and other health technologies on short-term and long-term resistance-related outcomes, measuring the health-sector costs and broader socioeconomic consequences of resistance-related mortality and morbidity, and evaluating vaccines in the context of nonvaccine complements and substitutes.

Antimicrobial resistance (AMR) is a significant emerging threat to global health and economic well-being. Despite the growing awareness of vaccines’ contributions to addressing AMR, economic evaluations of vaccines by health and finance ministries, by global donors, and by the research community have so far insufficiently incorporated that AMR-related value (AMR value). Vaccines therefore remain at real risk of undervaluation and underinvestment in the allocation of AMR-earmarked health sector, public sector, and research and development (R&D) budgets.

AMR-sensitive evaluations can play a role in value-for-money comparisons of different vaccines within a national immunization program, or in comparisons of vaccine-centric and non-vaccine-centric technologies within an anti-AMR program. We aim to promote AMR-sensitive economic evaluation of vaccines by discussing (i) the need for such evaluations, (ii) how to undertake them, (iii) the importance of evaluating vaccines in the context of other potentially complementary and substitutable nonvaccine technologies, and (iv) a roadmap for future work.

AMR-Sensitive Economic Evaluation of Vaccines

Relevant Principles of Economic Evaluation.

The first step in an evaluation is to specify the decision context it will inform. We focus on three AMR-related contexts: those of (i) a national or more local health payer (“payer”) allocating a fixed health sector budget across competing health technologies both AMR-related and not, (ii) a national finance minister setting that budget relative to other public sector budgets (e.g., education, infrastructure), and (iii) nonprofit global funders of health technology R&D (e.g., the Bill & Melinda Gates Foundation) and of national immunization (e.g., Gavi, the Vaccine Alliance) or AMR programs. Economic theory characterizes each context as involving a decision maker maximizing some value criterion subject to a budget constraint. Solving this problem requires the decision maker to compute the value for money represented by each spending option and to fund all options whose values for money pass a threshold level. We characterize the various options as technologies, broadly defined to encompass devices, drugs, programs, guidelines, procedures, and modes of organization.

The second step is to specify the evaluation’s perspective or value criterion. The health sector perspective maximizes health (typically denominated in quality- or disability-adjusted life years, or QALYs or DALYs), while the societal perspective maximizes social welfare, which encompasses both health and the socioeconomic aspects of well-being that health can promote, like productivity. The societal perspective uses individuals’ and/or societies’ monetary willingness to pay (WTP) for health to trade off health and socioeconomic well-being in the value criterion, as well as to assess the optimality of the relative health and nonhealth spending from any non-health-specific budget (like the finance minister’s or that of global funders of both health and nonhealth programs). Both perspectives have equity-sensitive versions.

The third step specifies the analytical approach. Cost-effectiveness analysis (CEA) typically treats each unit of health (e.g., a QALY) as equally valuable regardless of to whom it accrues and what socioeconomic benefits it produces. Cost–benefit analysis (CBA) allows health’s value to vary with its socioeconomic benefits according to individuals’ relative preferences for health and those benefits. Many payers adopt CEA from a health sector perspective (CEA-H), thus aiming to maximize health given the payer’s fixed budget. Its value-for-money indicator (“decision criterion”) is the incremental cost-effectiveness ratio (ICER), which has a candidate technology’s incremental health sector costs relative to a baseline technology in the numerator and its incremental QALY benefits in the denominator. Its decision rule is to fund all candidate technologies whose ICERs fall below some threshold value reflecting the opportunity cost of displacing marginal funded technologies.

CEA from a societal perspective (CEA-S) adds net value of socioeconomic gains outside the health sector (which count as negative costs) to the ICER numerator. A payer adopting CEA-S funds all technologies whose augmented ICERs fall below the above threshold value similarly augmented. Should a candidate technology’s ICER exceed this threshold (i.e., fail the test), a finance ministry adopting CEA-S should expand the payer’s budget at the expense of other public sector budgets or higher taxes to accommodate the technology so long as this ICER is below society’s WTP for a QALY.

CBA paradigmatically involves the societal perspective. It values reductions in age-specific mortality and morbidity risk at individuals’ and societies’ WTP to reduce these risks. These WTPs in turn depend on mortality and morbidity’s impact on socioeconomic quantities like consumption, leisure, production, and financial risk. One of CBA’s decision criteria is the rate of return (RoR): the net monetary benefits of a technology per dollar spent from some decision maker’s budget. The health payer’s decision rule is to fund all technologies whose RoRs meet or exceed a threshold value corresponding to the RoR of the marginal funded health technology. If a technology’s RoR falls below this threshold, the finance minister should raise the payer’s budget to accommodate it, at the expense of nonhealth public expenditures, so long as its RoR exceeds that of the marginal public sector-funded nonhealth technology.

A global funder can adopt any of CEA-H, CEA-S, or CBA depending on whether it seeks to maximize only health or broader social welfare, and on whether it allows socioeconomic factors to drive value differences across units of health. A funder adopting CEA-H would measure the gross benefit of any technology in terms of its expected QALY or DALY output, while one adopting CEA-S would also incorporate the impact of such outputs on the wider economy. A funder adopting CBA would value age-specific mortality and morbidity risk reductions at nation-specific or global WTP to reduce these risks.

AMR-Augmented Valuation.

Economic theory.

Making economic evaluations AMR-sensitive involves no fundamental change at the level of economic theory: Decision makers would use the exact same perspectives, analyses, decision criteria, and decision rules. The decision criteria simply need to have their various health, cost, and socioeconomic terms disaggregated into resistance-related subcategories, which in turn have to be measured carefully before they are reaggregated.

Take CEA and a simple binary disaggregation of disease into susceptible and resistant cases. If resistant disease involves larger treatment (or societal, depending on the perspective) costs and more severe QALY burdens than susceptible disease, then health technologies that disproportionately prevent or treat resistant disease should show greater averted treatment costs (and so smaller incremental costs) in the numerator, greater QALY gains in the denominator, and therefore more attractive ICERs than competing technologies. Similarly, for CBA, technologies that disproportionately address resistant disease will cause larger mortality and morbidity risk reductions, but these can be valued at the same WTP per unit of risk reduction used for susceptible cases. Both analyses should also capture the reality that antibiotic effectiveness is a depletable resource, and that full depletion of this resource (or categories of it) could lead to catastrophic consequences such as the inability to perform entire classes of medical procedures.

The challenge is primarily in quantifying the causal impact of health technologies on resistance-related health outcomes, and only secondarily in ascertaining the economic value of those outcomes.

Causal effects of vaccination on health.

Antimicrobial use exerts evolutionary pressure for the creation and transmission of resistant pathogens, raising the incidence of severe resistant disease and reducing antimicrobial effectiveness. Vaccination’s primary AMR-related benefit is that it reduces the incidence of resistant infections (1, 2). It does this directly by triggering immune responses and indirectly by avoiding antimicrobial use. Immune responses prevent colonization by both resistant and susceptible focal bacteria (“focal” refers to the pathogen in which resistance is a concern) from which resistant carriage would otherwise grow by mutation and horizontal gene transfer and result in downstream infection [e.g., pneumococcal conjugate vaccine (PCV)], prevent progression from carriage to infection of resistant focal pathogens, and prevent nonfocal infections that would facilitate coinfection by resistant focal pathogens (e.g., influenza vaccine and secondary bacterial pneumonia).

Vaccination against a focal pathogen reduces incidence of resistant infections secondarily by obviating the need for narrow or broad-spectrum treatment of those pathogens, relieving selection pressures on those pathogens and on other focal cocolonizing commensal and asymptomatic bystanders. Vaccination against nonfocal pathogens like influenza can prevent inappropriate use (e.g., antibiotic treatment of influenza) and appropriate treatment of focal coinfections (e.g., treatment of secondary bacterial infection).

Both primary and secondary effects are amplified by herd effects on unvaccinated persons. Through both direct and herd effects, vaccination can preserve the value of antimicrobials and of medical procedures like surgeries dependent on antimicrobial prophylaxis. It can delay the need for new antimicrobials whose R&D costs may grow as obvious targets are exhausted. However, vaccines like the PCV exert selection pressures against vaccine serotypes and in favor of nonvaccine serotypes. Experience to date is that this selection provides a temporary extra benefit against resistant strains (which were initially those targeted by the vaccine) but that this benefit can wane as resistance rises in nonvaccine types (3).

The fundamental scientific challenge to AMR-sensitive evaluation lies in quantifying each of these causal mechanisms and extrapolating their long-term implications for a broad range of vaccine–pathogen combinations and geographical settings. Quantified outcomes must go beyond intermediate or proxy outcomes like antimicrobial use and even disease incidence to health outcomes suitable for economic evaluation: mortality and morbid states for which utilities, disability weights, or WTP for risk reduction are derivable.

Progress on this challenge is necessary for all decision contexts, perspectives, and analytical approaches. We currently fall far short in this area (2): The modeling literature has ignored some of these mechanisms (like the secondary effects of vaccinating against nonfocal pathogens like influenza); focused on an extremely small number of pathogens (mainly Streptococcus pneumoniae and Staphylococcus aureus), vaccines (mainly PCV), geographical settings (mainly the United States and France), and only the short-term effects of antimicrobial use; and failed to trace through effects on evaluable mortality and morbidity outcomes. In our roadmap we therefore call for a global coordinated research effort to address this fundamental challenge.

Causal links from resistant health outcomes to cost and socioeconomic outcomes.

Given estimated effects of health technologies on reduced resistant health outcomes, the next step is to quantify the economic consequences of those health outcomes. The health sector perspective is a major approach among health technology assessments globally and is at any rate a subset of the societal perspective, so the proper place to start is with health sector impacts: primarily reduced medical treatment costs (from costlier second-line drugs, longer treatments, and more diagnostics) that would appear in ICER numerators.

The next step, as emphasized by the societal perspective, is to quantify the microeconomic (i.e., individual- and household-level) consequences of reduced resistant health outcomes on such variables as out-of-pocket health and care-related expenditures, caregiving time, education, paid and unpaid productive work, consumption of goods and services, leisure, exposure to financial risk, and income and wealth. There is a fairly large and still growing literature on theory and methods for quantifying each of these effects. The most rigorous of these approaches employ health-augmented lifecycle models in which an individual maximizes lifetime utility subject to a lifetime budget constraint; lifetime utility is a function of health, consumption of goods and services and of leisure, and the degree of stability in consumption, and the lifetime budget constraint reflects the effect of health on out-of-pocket expenditures and productivity and of credit and insurance constraints (4).

Externalities, interactions across economic actors, and government activity imply that macroeconomic outcomes are more than just the aggregate of microeconomic outcomes and require specific modeling. National finance ministers are critical to mobilizing country-level public funds and are, for better or worse, disproportionately swayed by evidence involving traditional macroeconomic aggregates like gross domestic product (GDP), fiscal balances (i.e., net impacts of disease on, say, earnings tax losses and disability transfers), poverty and economic inequality, and value-for-money metrics like RoRs rather than ICERs. A third economic step, then, is to deploy models of the macroeconomic and fiscal impact of population health to quantify the RoR to vaccines and other health technologies in terms of their AMR-mediated impacts on GDP, fiscal balances, poverty, and economic inequality. Many such macroeconomic models exist and are fit for purpose, and indeed some of them have been used to measure the labor-supply-mediated impact of AMR on GDP and poverty (5). However, they have not yet been used to compute the RoR to particular technologies. Although finance ministries are relatively comfortable with these types of analyses, health payers are less comfortable assessing macroeconomic outcomes and RoRs and will benefit from education in this area.

Economic evaluations often fail to control for differences between rich and poor (individuals or nations) in ability to pay for health benefits, or to incorporate any ethical preferences for giving priority to the worse off. Remedying these distributional shortcomings of economic evaluations and addressing AMR-related national and global distributional issues within an economic evaluation can be accomplished using several emerging analytical techniques. For CBAs, this can be done by integrating individual utility theory with a class of theoretical objects called social welfare functions that embody empirical and ethical assumptions about the impact of redistributing income on individual and social welfare. Such work has a distinguished pedigree in welfare economics and is growing in application in some fields of policy evaluation but has made little inroads in health economic evaluation. For CEAs, techniques such as extended cost-effectiveness analyses and distributional cost-effectiveness analyses integrate concerns about the distribution of health and medical expenses alongside maximizing total health gains (6). Like many health burdens, those related to AMR may be disproportionately concentrated among poor individuals and nations, which suggests the value equity concerns can bring to economic evaluation in this area.

Using Economic Evaluations to Inform the Relative Priority of Vaccine and Nonvaccine Interventions

Introduction.

The above discussion addresses the evaluation of vaccines in isolation, but a full economic approach to priority setting also requires addressing the relation between vaccine and nonvaccine technologies for addressing AMR.

Any decision maker seeking to minimize the burdens of resistance, or to maximize health or social welfare by minimizing such burdens, should use economic evaluations to guide choices among alternative technologies for achieving such goals. The unforgiving mathematics of a budget constraint requires percentage allocations to sum to 100, so spending a budget dollar on one technology necessarily foregoes spending it on another. To maximize each dollar’s impact, it should be steered in the direction of technologies with relatively more attractive ICERs and RoRs and implicitly away from those with less-attractive ones. Thus, value-for-money comparisons across different technologies are central to economic evaluation. Such comparisons are relevant even when budgets are not fixed since they inform the allocation of any budgetary increases.

One implication of this inherently comparative nature of economic evaluation is that AMR-sensitive evaluations should be carried out for both vaccine and nonvaccine technologies alike. Among potentially relevant nonvaccine technologies are (i) antimicrobial stewardship (AMS) and other “responsible use” programs that aim to promote appropriate use and reduce inappropriate use (7), (ii) nonvaccine infection prevention and control programs like handwashing or sanitation more generally (8), (iii) surveillance and monitoring systems to track levels and trends of, and program effects on, resistance (7), and (iv) R&D of new antimicrobials and diagnostic tests (9).

Many countries, global funders, and other global priority-setting organizations like the WHO have begun generating AMR-related priority lists. Examples of these are national AMR action plans (10) and the WHO list of antibiotic R&D priorities (9). A related implication of the nature of economic evaluation is that such decision makers should be explicitly comprehensive and comparative in their priority setting: not omitting any major class of technology (including vaccines) and relying on either quantitative (if possible) or qualitative (in the short run in the absence of quantitative evidence) judgments of relative value for money when ranking technologies or allocating budgets.

In allocating budgets across multiple technologies, economic theory suggests that the optimal allocation will depend on complementarities and substitutability across the different technologies. Simplifying somewhat, technologies are complementary to the extent they enhance each other and so are fruitfully employed in combination, where the optimal combination is determined by technical aspects of the enhancement (e.g., the way left and right shoes are naturally used in 1:1 combinations). Technologies are substitutable to the extent any given level of output (e.g., one QALY or a given quantity of mortality risk reduction) can be feasibly produced with varying combinations of the two technologies, giving decision makers the flexibility to choose combinations that lean more heavily on whichever technology has relatively more value for money.

In cases in which technologies are partially complementary and partially substitutable, the complementarity may imply that the technologies should be used in some optimal proportion, but the substitutability may imply that the magnitude of that proportion itself will be swayed by the technologies’ relative value for money. Determining the optimal budget allocations implied by the given degrees of complementarity and substitutability is then reduced to an empirical issue.

There are good reasons to believe that vaccine and nonvaccine strategies for addressing AMR are partially complementary. Vaccine uptake and effectiveness are imperfect, so saving lives threatened by resistance will necessarily require effective last-line antimicrobials to treat the infections vaccines fail to prevent. Some antimicrobial-centric strategies like AMS can reduce inappropriate use, but not the appropriate antimicrobial use that infection-control technologies like vaccines can prevent. Targeting technologies to those who would benefit most from them requires diagnostics, and monitoring progress and impact requires surveillance. Thus, significant expenditures will be required across many classes of technology.

However, different AMR-related strategies are also partially substitutes: Many mortality and morbidity risk-reduction targets are potentially achievable either by devoting relatively more resources to vaccine-led prevention or to AMS-guided treatment. The fact that significant spending must be devoted to both prevention and treatment does not pin down the optimal relative magnitude of such expenditure, which theory suggests should strongly depend on their relative ICERs or RoRs.

We strongly encourage efforts to quantify the net impact of complementarities and substitutability on relative spending on vaccine and nonvaccine strategies for addressing AMR. To stimulate such efforts, we now briefly undertake a qualitative discussion of the relative advantages and disadvantages of vaccine- and non-vaccine-centric technologies for addressing AMR. These relativities speak to their substitutability, which we find to be an underappreciated issue in the literature and policy discussions generally, and which we find largely unconsidered by decision makers formulating priority lists and action plans.

To focus this investigation, we limit ourselves to a comparison between vaccine-centric strategies (specifically increasing uptake of existing vaccines and vaccine R&D) and two of the most prominently discussed AMR strategies (both of which are antibiotic-centric), namely antibiotic stewardship (ABS) and antibiotic R&D. We also limit our discussion to interventions in human populations. Many of our points also apply to vaccine and nonvaccine strategies for addressing resistant viruses, parasites, and fungi and to intervening against AMR in animal populations. Where useful, we illuminate these comparisons using S. pneumoniae and Neisseria gonorrhoeae as examples.

Vaccination vs. ABS, with PCV13 as an Example.

Two classes of potentially substitutable technologies payers might consider for reducing AMR burdens are ABS and vaccination with currently existing vaccines like conjugate vaccines against S. pneumoniae. ABS attempts to reduce inappropriate antibiotic use in inpatient, outpatient, community, and animal agricultural settings while ensuring that those who need antibiotics have access. ABS in healthcare settings can have various elements including clinician and patient education, financial and other incentives to reduce antibiotic prescriptions, special permission requirements (e.g., from an infectious disease specialist) for treatment with certain antibiotics, documentation or audit of prescribing practices, use of technology to validate prescribing decisions, and increased use of diagnostic testing. ABS’s effect on antibiotic consumption can in principle reduce the incidence of resistant infection, thereby preserving antibiotic effectiveness.

The qualitative similarity between ABS and some vaccines is that they can both reduce inappropriate antimicrobial use, but there are also differences that count toward their relative ability to reduce mortality and morbidity.

ABS has the relative advantage of addressing a wide variety of conditions that could be treated inappropriately, while vaccines can address only inappropriate use provoked by their target pathogen. For example, ABS can potentially reduce both inappropriate treatment conditions that do not require treatment, such as viral infections or (according to some national policies) certain self-limiting bacterial infections, thereby avoiding bystander selection on S. pneumoniae bacteria. Pneumococcal vaccines can prevent only pneumococcal disease, its sequelae, and the treatment that results from these.

However, vaccination has some advantages relative to ABS. First, vaccination prevents antibiotic-susceptible infections and so reduces the resistance that comes from appropriate antibiotic use. Pneumococcal vaccines, for example, accomplish this by reducing the incidence of susceptible invasive pneumococcal disease (IPD), pneumonia, and otitis media. Antibiotic prescribing for otitis media in the United Kingdom fell with the introduction of PCV7 and its replacement with PCV13 (11). This advantage is considerable since the resistance that comes from appropriate use is vast and since growing prosperity and access to care in low- and middle-income countries (LMICs), in face of the current scarcity of antibiotics, imply that appropriate use will grow enormously in the future (12).

Second, vaccination also directly prevents resistant infections along with their excess mortality and morbidity consequences. PCV13 accomplishes this, for example, by reducing the incidence of resistant IPD and pneumonia as suggested by evidence from the United States, South Africa, and England and Wales (3, 13, 14).

Third, a benefit-related advantage that vaccination has over ABS is that the evidence for vaccination’s impact on resistance-related mortality and morbidity is stronger than that for ABS. We have already mentioned, for example, evidence on the impact of PCV13 on population-level incidence of resistant pneumococcal disease. In contrast, ABS has proven effective mainly in improving prescribing outcomes in inpatient settings in high-income countries (15). Evidence of its impact on prescribing practices in outpatient settings is scarce (16). Equally scarce is evidence of its impact on actual mortality and morbidity outcomes at the level of populations or even facilities (15, 17, 18).

Fourth, a feasibility-related advantage that vaccination has over ABS is that ABS programs often face steeper barriers to effective population-level scale-up and implementation than vaccination programs. Effective ABS often requires investment in diagnostic tests, laboratories, and microbiology expertise, which are expensive and scarce in LMICs and in many outpatient settings (19, 20). Substantial inappropriate use comes from self-medication (21), which is abetted by nonprescription sales including over the internet and in gray and counterfeit markets (22). In many settings, prescriptions are provided by diverse personnel, including nurses, dentists, pharmacists, dispensers, and midwives, many of whom may have inadequate training and incentives adverse to reducing inappropriate use (23). In 2003 and 2007, a quarter of health ministries used revenues from sales of medicine to pay or supplement health worker salaries (24). In some countries, physicians both prescribe and dispense drugs, and legal reform is often needed to separate these tasks (25). There are well-known difficulties in changing doctor prescribing practices and patient expectation. All of these factors may be barriers to the large-scale implementation of effective ABS reforms, particularly in LMICs. In contrast, notwithstanding the logistical and attitudinal barriers to vaccination programs, the historical success of childhood immunization programs suggests that these challenges are perhaps easier to surmount than those to ABS. For example, even in some resource-poor settings pediatric vaccines have been documented to reach over 90% of target populations (26).

One possible downside of some vaccines relative to ABS is that they can be unaffordable, particularly in middle-income countries receiving no Gavi support (27). However, the true extent of this relative disadvantage is hard to ascertain given the scarcity of studies costing relevant comparators like health system-wide ABS programs.

In sum, there is clear evidence of the impact of PCV on population-level resistance burdens and indications that high population uptake of the vaccine is feasible. There is also ambiguous evidence that ABS can consistently reduce even facility-level burdens, and there are apparent barriers to large-scale implementation of ABS, particularly in LMICs. These observations support the hypothesis, at least with respect to pneumococcal disease, that investment in increasing PCV coverage will have a bigger impact on reducing the health and socioeconomic burdens of resistant disease than would similar levels of investment in ABS, particularly in LMICs. However, as we note above, it is important to remain mindful that ABS could, at least in principle, have an effect on inappropriate treatment of multiple pathogens.

R&D of Vaccines vs. R&D of Antibiotics, with Gonorrhea as an Example.

The global proliferation of drug-resistant bacteria like gonorrhea could potentially be addressed by global R&D funders either by developing new antibiotics to restore susceptibility to treatment or by developing a vaccine to prevent new cases of disease. Because vaccine and antibiotic R&D are both expensive and risky, we focus on their relative benefits and costs. The qualitative similarity between a new vaccine and a new antibiotic for the same pathogen is that they both directly reduce the mortality and morbidity burdens of resistant infections, the former by preventing their occurrence and the latter by restoring their susceptibility to treatment.

The fundamental relative advantage of new antibiotic treatment over new vaccine prevention of resistant disease is that the uptake and effectiveness of vaccine prevention are imperfect, so infection risk is almost always present no matter the intensity of preventive effort. The new antibiotic’s value comes from its reducing the mortality and morbidity consequences of inevitable prevention failures. For a given pathogen, this relative advantage is larger the more expensive the vaccine and the lower its effectiveness. If a prospective vaccine for a given condition is anticipated to be sufficiently expensive and its effectiveness sufficiently low, it may be better to use scarce resources to treat cases as they arise rather than try to prevent them. Another traditional advantage of antibiotics is that they can treat a wide range of pathogens, while a new vaccine will likely directly prevent resistant infections due only to its focal pathogen. For example, a new antibiotic that acts against gonorrhea may have additional value in treating other drug-resistant bacteria like methicillin-resistant S. aureus (28, 29).

However, a fundamental advantage vaccines have over antibiotics is that they are vastly more “evolution-proof” (30). Evolution of resistance has eroded the effectiveness of nearly every antibiotic for nearly every bacterial species they are designed to treat, and dissemination of resistance genes can occur rapidly. For example, since the advent of antibiotics, gonorrhea treatment has involved a progression from one antibiotic to another, as resistance to each initially effective antibiotic has developed (31). WHO data show widespread resistance to most antibiotics used to treat gonorrhea (32). Multidrug- and pan-resistant strains of gonorrhea have emerged, and gonorrhea seems to develop resistance especially rapidly (32–34).

By contrast, few vaccines have selected for clinically significant resistance, reflecting key biological differences between vaccines and antibiotics (30). This means that relative to new vaccines, new antibiotics are likely to represent shorter-term solutions (30, 35). The formidable scientific and economic obstacles of R&D must therefore be more frequently hurdled for antibiotics than for vaccines.

Furthermore, a gonorrhea vaccine is likely to have a much more powerful impact on gonorrhea control than a new antibiotic. A recent modeling exercise suggests that even a vaccine of modest efficacy and duration could have a significant impact on disease burden, reducing prevalence by between 40–90% over 20 y (35). In contrast, gonorrhea prevention through treatment typically depends on symptomatic persons’ seeking care, and thereupon receiving appropriate screening and diagnostics. This implies that the prevention value of a new antibiotic for treating gonorrhea will be limited by high rates of asymptomatic infection (36), low and delayed rates of care seeking for sexually transmitted infections (STIs) (37), significant barriers to accessing care, scarcity of diagnostics (36), and the nonspecificity of gonorrheal symptoms (36), particularly in resource-poor settings, like Africa, which have the highest infection rates.

In general, the scientific hurdles to antibiotic R&D seem to be higher than those to vaccine R&D, which makes the vaccine R&D pipeline more promising than that of antibiotics. It is claimed that no chemically novel antibiotics effective against multiple important pathogens have entered clinical use in three decades, while recombinant DNA technology, conjugate and reverse vaccinology, and adjuvants have yielded 22 new vaccines over the same time period (38).

The economic incentives around antibiotic R&D are also more challenging than those around vaccine R&D. Reducing the burdens of resistance typically requires maximizing uptake of vaccines but rationing the use of antibiotics. The revenue-generating incentives of vaccine producers are therefore in greater harmony with policy goals than those of antibiotic producers. The need to ration new antibiotics may be at odds with the need to provide sufficient access to them. These tensions may make it difficult to price, regulate, ration, provide access to, and incentivize R&D of these antibiotics. In contrast, school-based vaccination programs for the human papillomavirus (HPV) vaccine in Gavi-supported low-income African countries have achieved more than 80% coverage (39), suggesting the same platform and coverage may be achievable for a future gonorrhea vaccine even in the poorest settings. The prospect of vaccinating whole cohorts of adolescents globally can support R&D incentives.

There are downsides to a gonorrhea vaccine: Gonorrhea is an STI, making social acceptance of a vaccine an issue. (However, researchers have found that uptake of HPV vaccine, for example, has depended significantly more on patients’ understanding of the vaccine’s benefits and risks and their general attitude toward vaccination—as well as whether their clinicians promote the vaccine’s uptake—than their perception of its connection with sexual activity; see ref. 40.) An STI vaccine may lead to riskier sexual behavior, although similar fears about HPV vaccination have proven to be unfounded (41).

However, overall, several factors support our hypotheses that the resistant mortality and morbidity benefits of a gonorrhea vaccine will exceed those of a new antibiotic for treating it, and that global efforts to raise vaccine R&D funds will have larger social returns than corresponding efforts to raising antibiotic R&D funds. These include worldwide growth in resistant gonorrhea (32), the weaker prevention value of treatment with new antibiotics relative to significant population-prevalence-reduction potential of even a modestly effective gonorrhea vaccine, the likely rapid development of resistance to a new antibiotic and consequent Sisyphean nature of antibiotic R&D, and the lower scientific hurdles and sounder profitability and implementation prospects of a gonorrhea vaccine over a new antibiotic.

This qualitative discussion suggests that both payers designing national AMR action plans and global funders of AMR-related R&D would do well to seriously consider the substitution possibilities between vaccine- and antibiotic-centric AMR strategies.

Roadmap

We end by summarizing and elaborating on our key arguments in the form of a roadmap for future work.

First, AMR-sensitive-sensitive vaccine evaluations should address at least three decision contexts: those of a national health payer (to address the allocation of health-, vaccine-, and AMR-earmarked budgets), a national finance minister (to address affordability, the optimal size of health budgets, and the optimality of earmarking new vaccines from the general budget), and global funders (to address R&D and LIC vaccine priorities). They should consider the appropriate perspective (health sector and/or societal perspectives) and type of analysis (CEA and/or CBA) on the basis of the decision context, relevant guidelines or reference cases, and relevant policy considerations like efficiency, equity, and other normative principles. Tables 1 and 2 clarify how to incorporate AMR-related impacts into both CEA and CBA.

Table 1.

Incorporating AMR-related impacts into CEA of a vaccination program

	(1) Health sector perspective	(2) Health sector perspective	(3) Societal perspective
Explicitly accounts for AMR	No	Yes	Yes
Costs	• Cost of vaccination program net of:	• Cost of vaccination program net of:	• All items in column (2)
	◦ Reduced cost of treating VPD (no distinction made between s-VPD and r-VPD)	◦ Reduced cost of treating VPD (disaggregated by s-VPD and r-VPD) and infections with resistant bystander pathogens*	• Additional items to net from the cost of vaccination program:
	◦ Reduced cost of treating secondary health effects caused by VPD (no distinction made between s-VPD and r-VPD)	◦ Reduced cost of treating secondary health effects caused by VPD (disaggregated by s-VPD and r-VPD) and caused by infections with bystander pathogens	◦ Value of averted productivity loss due to:
		• Net cost to health sector of altered treatment patterns due to inability to perform medical procedures dependent on effective AM prophylaxis	□ Reductions in VPD (disaggregated by s-VPD and r-VPD)
			□ Reductions in infections with resistant bystander pathogens
			□ Reductions in secondary health effects
			□ Medical procedures that are dependent on effective AM prophylaxis
			◦ Reduced R&D costs for new antibiotics, diagnostics, and other technologies to counter AMR
Benefits	• QALYs or DALYs attributable to direct reduction in incidence of VPD (no distinction made between s-VPD and r-VPD)	• QALYs or DALYs attributable to direct reduction in incidence of VPD (disaggregated by s-VPD and r-VPD)	• Same as column (2)
		• QALYs or DALYs attributable to indirect reduction in incidence of r-VPD and infections with resistant bystander pathogens as a result of reduction in AM consumption
		• QALYs or DALYs attributable to medical procedures that are dependent on effective AM prophylaxis
		• QALYs or DALYs attributable to reductions in secondary health effects caused by s-VPD, r-VPD, and infections with resistant bystander pathogens

AM, antimicrobial; r-VPD, resistant VPD; QALY, quality-adjusted life year; s-VPD, susceptible VPD; VPD, vaccine-preventable disease.

^*The within-host, population-level, and health systems-level pathways between vaccines and AMR are too varied and complex to fully summarize here. See refs. 1 and 2 for more detailed explanations.

Table 2.

Incorporating AMR-related impacts into CBA of a vaccination program

	(1) Societal perspective	(2) Societal perspective
Explicitly accounts for AMR	No	Yes
Costs	• Full social cost of vaccination program	• Full social cost of vaccination program
Benefits	• Reduced cost of treating VPD (no distinction made between s-VPD and r-VPD)	• Reduced cost of treating VPD (disaggregated by s-VPD and r-VPD), infections with resistant bystander pathogens, and conditions that require medical procedures that are dependent on effective AM prophylaxis
	• Intrinsic value of reduced morbidity and mortality due to reductions in VPD (no distinction made between s-VPD and r-VPD)	• Intrinsic value of reduced morbidity and mortality due to direct reductions in incidence of VPD (disaggregated by s-VPD and r-VPD)
	• Productivity gains resulting from reduced morbidity and mortality due to reductions in VPD (no distinction made between s-VPD and r-VPD)	• Intrinsic value of reduced morbidity and mortality due to indirect reductions in incidence of r-VPD and infections with resistant bystander pathogens as a result of reduction in AM consumption
	• Productivity gains from reduced time caretaking for persons ill with VPD (no distinction made between s-VPD and r-VPD)	• Intrinsic value of reduced morbidity and mortality due to medical procedures that are dependent on effective AM prophylaxis
	• Education gains from reduced morbidity due to reductions in VPD in children (no distinction made between s-VPD and r-VPD)	• Productivity gains from reduced morbidity and mortality due to reductions in incidence of VPD (disaggregated by s-VPD and r-VPD), reductions in incidence of infections with resistant bystander pathogens, and medical procedures that are dependent on effective AM prophylaxis
	• Reduced secondary health effects and corresponding social and economic benefits	• Productivity gains from reduced time caretaking for persons ill with VPD (disaggregated by s-VPD and r-VPD) or infections with resistant bystander pathogens, or whose condition would benefit from medical procedures that are dependent on effective AM prophylaxis
		• Education gains from reduced morbidity due to reductions in VPD (disaggregated by s-VPD and r-VPD), reductions in infections with resistant bystander pathogens in children, or medical procedures that are dependent on effective AM prophylaxis
		• Education gains from reductions in morbidity resulting from medical procedures that are dependent on effective AM prophylaxis
		• Reduced R&D costs for new antibiotics, diagnostics, and other technologies to counter AMR
		• Reduced secondary health effects (disaggregated by those associated with s-VPD, r-VPD, and infections with bystander pathogens) and corresponding social and economic benefits

AM, antimicrobial; r-VPD, resistant VPD; s-VPD, susceptible VPD; VPD, vaccine-preventable disease.

To facilitate their consideration by national and global decision makers, economic evaluations should be encouraged to include scenario analyses that adhere to reference cases specified by the relevant national HTAs, NITAGs, and ministries of finance and by influential international bodies like the Second Panel on Cost-Effectiveness in Health and Medicine (42) or iDSI (43). There remain no reference cases for CBA specific to health, but researchers can follow draft reference cases being developed by the Harvard T.H. Chan School of Public Health Guidelines for its Benefit-Cost Analysis Project (44).

To economize on research effort, groups evaluating a particular vaccine–country combination from a health sector perspective should consider also doing at least a simplified societal perspective evaluation within the same study, and also providing age-disaggregated mortality and morbidity outcomes that can set the stage for subsequent CBA by other groups. Methods for conducting simple societal perspective and CBA analyses are provided by the Harvard T.H. Chan School of Public Health Guidelines for Benefit-Cost Analysis Project, particularly by project guidelines on valuing mortality and morbidity risks (44).

Second, the general formulas for decision criteria like ICERs and RoRs and their associated decision rules are unaltered by consideration of AMR-related value. The primary adjustment required in calculating decision criteria is the disaggregation of mortality and morbidity risks, health sector costs, and all other socioeconomic costs and benefits into resistance-related subcategories, and the careful measurement or modeling of each of those subcategories, before reaggregating.

Third, the fundamental scientific challenge and highest research priority lies in quantifying the various primary and secondary effects of vaccination and other health technologies on resistant mortality and morbidity.

One way to address this challenge would be through a global coordinated research effort. Such an effort could identify a matrix of causal mechanisms, vaccine–pathogen combinations, and geographical settings and recruit modeling teams from across the world whose efforts would collectively span this matrix. All models supported in this effort could (45):

• •Collectively span the full range of primary and secondary causal mechanisms, countries, and vaccine–pathogen combinations
• •Adopt as endpoints health outcomes suitable for economic evaluation
• •Rely as much as possible on standardized input data
• •Have appropriate levels of complexity
• •Be rigorously fit to epidemiological data
• •Have internal and external validity
• •Consider ecological effects where relevant
• •Quantify the effects of structural and parametric uncertainty (2).

For any given vaccine–pathogen combination, there may be multiple interacting causal mechanisms relevant to an economic evaluation. Also, it may be impractical for a single research team to model all these relevant mechanisms. If so, then the work of multiple research teams should be governed by a single overarching causal model spanning all of the relevant mechanisms and interactions. Each team could focus its efforts on a submodule of the overarching model that is capable of communicating and interacting with other teams’ submodules in a coherent way.

This global effort should facilitate the third-party use of the resulting models in economic evaluations by allowing such parties to interact with the models online, vary relevant country- and vaccine-pathogen-specific parameter values, and generate model outputs reflecting those parameter changes.

Modeling efforts need to be informed and complemented by empirical study designs such as individual randomized controlled trials for estimates of efficacy, cluster trials for estimating ecological effects, and quasi-experimental designs like nonrandomized control trials, controlled before-and-after trials, and comparative and noncomparative interrupted time series for estimates of real-world effectiveness (15). These studies should disaggregate mortality, morbidity, and other outcomes by host-, pathogen-, and resistance-related subcategories and be designed to identify and quantify some subset of the primary and secondary effects of vaccines on resistance.

In the short run, and to pave the way for longer-term studies, we propose the convening of an expert panel that can identify priority causal mechanisms, countries, and vaccine–pathogen combinations for study; identify best existing estimates of critical parameters; and identify approximation formulas for various primary and secondary effects that can be used in interim research.

Fourth, the economic measurement challenges lie in measuring the incremental medical cost of treating resistant disease; microeconomic impacts of resistant mortality and morbidity on lifetime individual and household consumption, leisure, market and nonmarket production, and financial risk protection; and macroeconomic impacts on GDP, fiscal balances, poverty, and equity.

To address health-sector costs, we propose systematic literature reviews and meta-analyses of existing studies of the incremental cost of treating resistant disease. We also propose a global coordinated research effort in which country teams generate country-level estimates of the incremental treatment costs of resistant infection from relevant data sources like surveys and administrative and claims databases. For countries in which such direct measurement is infeasible, meta-analyses can suggest reasonable extrapolations. These costing studies should include scenario analyses that follow reference cases specified by the relevant countries’ HTAs as well as by influential international standards-setting organizations like the WHO or iDSI.

The Guidelines for Benefit-Cost Analysis Project at the Harvard T.H. Chan School of Public Health (44) provides a good summary of existing methods for quantifying the above microeconomic and macroeconomic effects. Primary research into quantifying microeconomic and macroeconomic effects should consider the use of health-augmented lifecycle models (and within such models, greater consideration of the impact of health on productivity and of financial risks), social welfare functions, and fiscal impact and macroeconomic models.

Fifth and finally, AMR-sensitive economic evaluation of vaccines should not be considered in isolation but in the broad context of complementarities and substitutability across vaccine- and non-vaccine-centric anti-AMR technologies. AMR-sensitive evaluation should therefore occur not just for vaccines but for all other AMR-related technologies like stewardship, antimicrobial R&D, surveillance, and diagnostics R&D. National-level AMR-portfolio evaluations should also be encouraged, in which the various technology classes are considered singly and in combination in light of their complementarities and substitutability.

Existing national AMR action plans and global action plans and priority lists should be scrutinized in terms of whether they have sufficient basis in the economic evaluation literature relevant to the decision making context. Are included interventions justified by strong estimates of value for money? Are omitted interventions justified by strong estimates of lack of value for money? Are relative priorities justified on the basis of relative value for money?

In the medium-to-long term, it is valuable to generate priority lists or league tables spanning all relevant technologies that are informed by quantitative research. In the short run, it may be helpful to generate preliminary versions of these on the basis of existing evidence and qualitative and quantitative expert judgments. Specific versions of these lists and tables should be produced for national payers in high-, middle-, and low-income countries and for global funders.

One way to structure such evidence and judgments in the short run is to use multicriteria decision analysis (46). This involves defining criteria important to a decision, assigning weights to each criterion, scoring candidate technologies according to each criterion, and prioritizing technologies according to their overall weighted scores across criteria. Such analyses could specify AMR-related criteria alongside standard criteria for health and socioeconomic benefits, elicit experts’ judgments regarding how well different technologies score according to these criteria, and generate AMR-sensitive priority lists and league tables on the basis of overall scores incorporating such judgments. Some AMR-related criteria might be reduction in antimicrobial use or resistance. The advantage of this approach is that it would allow AMR-related criteria to be given weight in a decision even when we currently do not have adequate scientific and economic knowledge to robustly incorporate the relationship between vaccination and AMR into CEA and CBA outcome measures (such as the ICER or RoR for an intervention).