An Introduction to the Main Types of Economic Evaluations Used for Informing Priority Setting and Resource Allocation in Healthcare: Key Features, Uses, and Limitations

Abstract

Economic evidence is increasingly being used for informing health policies. However, the underlining principles of health economic analyses are not always fully understood by non-health economists, and inappropriate types of analyses, as well as inconsistent methodologies, may be being used for informing health policy decisions. In addition, there is a lack of open access information and methodological guidance targeted to public health professionals, particularly those based in low- and middle-income country (LMIC) settings. The objective of this review is to provide a comprehensive and accessible introduction to economic evaluations for public health professionals with a focus on LMIC settings. We cover the main principles underlining the most common types of full economic evaluations used in healthcare decision making in the context of priority setting (namely cost-effectiveness/cost-utility analyses, cost-benefit analyses), and outline their key features, strengths and weaknesses. It is envisioned that this will help those conducting such analyses, as well as stakeholders that need to interpret their output, gain a greater understanding of these methods and help them select/distinguish between the different approaches. In particular, we highlight the need for greater awareness of the methods used to place a monetary value on the health benefits of interventions, and the potential for such estimates to be misinterpreted. Specifically, the economic benefits reported are typically an approximation, summarising the health benefits experienced by a population monetarily in terms of individual preferences or potential productivity gains, rather than actual realisable or fiscal monetary benefits to payers or society.

Introduction

Healthcare demand is continuing to grow; however, the resources available for healthcare are explicitly limited. Consequently, ensuring the best value for money spent in healthcare has been placed high on the agenda for governments worldwide, with economic considerations gaining an increasingly prominent role when planning, managing and evaluating health systems (1). Health economic analyses can be used to assess a health interventions value for money and can support the optimal allocation of the limited resources available for healthcare (2).

Using health economic analyses to investigate the value for money of different health interventions is particularly appealing to decision-makers when considering the use of a public budget to fund healthcare services. The role of health economic analyses in informing health policy has increased over time (3, 4). However, the underlining principles of the different types of health economic analyses are not always fully understood by non-health economists, and inappropriate types of analyses, as well as inconsistent methodologies, may be being used for informing health policy decisions, particularly in low- and middle-income country (LMIC) settings where this field of research is less well-established (5). This limits the potential role of these tools in informing policy decisions to lead to the greatest health gains. Although information and methodological guidance for these health economic analyses are available, it is predominantly in the form of textbooks and training programs, that are typically available in and focused on high-income country settings, and tend to be behind a paywall. Although many of the concepts are universally relevant, there are issues more specific to LMICs (such as the different types of data available, effectiveness metrics, and decision rules) and capacity in these settings is an ongoing challenge (5). Currently, there is a lack of up-to-date open access accessible literature focusing on a public health professional audience in LMICs who are playing key roles in health resource allocation.

This review provides a comprehensive, transparent, and freely accessible introduction to economic evaluations—intended to provide a resource for public health professionals, especially those in LMICs where such resources are typically unavailable. Specifically, we provide an up-to-date comprehensive introduction to the main principles underlining the most common economic evaluation methods used in the context of informing resource allocation decisions in global health [namely cost-effectiveness/cost-utility analyses and cost-benefit analyses (6)] (Table 1), outlining their key features, differences, advantages and limitations. By doing this we aim to increase the overall understanding of the key concepts underlining the different types of economic evaluations commonly used in the context of priority setting in healthcare, focusing on LMIC settings. Understanding the key concepts in this context is critical as it may influence the appropriate use of each approach as well as how health economic evidence should be interpreted to inform health policy. It is envisioned that this will help those conducting such analyses, as well as stakeholders that need to interpret their output [such as a cost per quality-adjusted life-year (QALY) gained, cost per disability-adjusted life-year (DALY) averted, net monetary benefit, return on investment, and benefit-cost ratio], gain a greater understanding of the approaches and help them choose/distinguish between the approaches. Note that this review focuses on the general application of these methodologies and is not focused on a particular disease area.

Table 1.

The key types of full economic evaluations (6).

Type	Description
Cost-effectiveness analysis	Cost-effectiveness analysis is a form of comparative economic analysis that evaluates two or more policy alternatives in terms of their relative costs and outcomes, where the outcomes are measured in a single natural unit (e.g. life-years gained, disease case averted etc.).
Cost-utility analysis	Cost-utility analysis (a specific type of cost-effectiveness analysis) is a form of comparative economic analysis that evaluates two or more policy alternatives in terms of their relative costs and outcomes, where the outcomes are expressed by a generic measure of health status that considers both the effect on mortality and morbidity (e.g., quality-adjusted life-years (QALYs) and disability-adjusted life-years (DALYs)).
Cost-benefit analysis	Cost-benefit analysis is a form of comparative economic analysis that evaluates two or more policy alternatives in terms of their relative costs and outcomes, where both the costs and outcomes are expressed in monetary terms. In principle, it should value the interventions relevant costs and outcomes based on the preferences of those affected (i.e., the individuals' willingness to pay).
Cost-minimisation analysis	Cost-minimisation analysis is a form of comparative economic analysis that compares the costs of two or more policy alternatives which are all assumed to have equivalent health effects.
Cost-consequence analysis	Cost-consequence analysis is a form of comparative economic analysis that evaluates two or more policy alternatives in terms of their relative costs and outcomes, where the outcomes are not summarised in a single measure, and multiple outcomes of interest are reported.

Differences in the Main Types of Health Economic Analyses Used for Evaluating Resource Allocation Decisions

There are many different approaches used for health economic analyses. Some types of analysis only examine the costs of an intervention or a disease (e.g., cost of illness studies) independently, whereas other types of analysis evaluate both the costs and consequences of an intervention. It is vital to understand the different roles of partial and full economic evaluations.

Full Economic Evaluations vs. Partial Evaluations

Full economic evaluations are a specific type of health economic analysis that explicitly compare the costs (use of resources) and consequences (effects) of the health intervention(s) in question to an alternative course of action, known as the comparator (Figure 1) (7). Full economic evaluations therefore formally evaluate at least two alternative courses of action, even when only looking at a single health intervention/policy. The comparator (also referred to as the counterfactual or baseline scenario) is typically chosen to reflect common practise or standard of care in the setting where the economic evaluation is undertaken. For example, an economic evaluation of introducing the human papillomavirus vaccine in the UK would use the pre-existing cervical cancer screening program as the comparator (8). In some cases, the comparator will be “do-nothing” or a no intervention scenario, but it is still a formal part of the analysis which needs to be clearly acknowledged with the relevant costs and consequences quantified.

Figure 1.

The difference between full economic evaluations and cost-outcome partial evaluations.

In contrast, partial evaluations (also referred to as partial economic evaluations) are studies that either:

1. Only examine the costs or consequences of an intervention independently i.e. they do not look at both or relate the costs to the consequences. For example, a costing study would evaluate only the costs associated with an intervention (but not compare these to its consequences).

2. Examine both the costs and consequences of an intervention but only evaluate a single course of action (a cost-outcome study). By doing this they are either explicitly or implicitly ignoring the comparator, which may overlook relevant costs/consequences and does not allow for comparisons to relevant alternative policy options for that setting. These analyses are therefore not a full economic evaluation—which has to formally compare the costs and consequences of the interventions in question to a comparator scenario (Figure 1).

Although partial evaluations can provide useful information, they cannot alone guide decision-making as only knowing an intervention's cost or the economic burden of a disease does not indicate an intervention's value of money. In the context of informing healthcare decision making surrounding resource allocation, it is vital to evaluate both the costs and consequences of the intervention in question and to compare it to a relevant alternative course of action/policy option (the comparator). The importance of accounting for the comparator is highlighted in Figure 2. In this hypothetical example, a partial evaluation would find that the benefits of a new treatment (the monetary value of its health benefits) outweigh its cost, favouring its use. However, performing a full economic evaluation whereby the new treatment was compared to the current standard practise as the comparator, the analysis would find that the new treatment is less effective and more expensive than the current standard practise—leading to the opposite policy recommendation (Figure 2). This highlights that when relevant alternative policy options (such as the standard of care) are ignored, analyses can generate misleading conclusions. Note that there is variation in the literature regarding if partial evaluations are referred to as an economic evaluation or not (9). Note that at times studies can be mislabelled. For example, Zarnke et al. (9) found that studies labelled as cost-benefit analyses (a type of full economic evaluation) in the healthcare literature were at times only partial evaluations (Figure 1).

Figure 2.

A hypothetical comparison of the difference between a partial evaluation and full economic evaluation.

Different Types of Full Economic Evaluations

There are five main types of full economic evaluations used to inform and evaluate health interventions (outlined in Table 1) (6, 10). In this review, we focus on the most widely used types, i.e. cost-effectiveness/cost-utility analyses, and cost-benefit analyses (11, 12). There are other types of full economic evaluations, namely cost-minimisation analysis and cost-consequence analysis. As these are less commonly performed, and in the case of cost-minimisation analysis only appropriate in rare circumstances (11–13), these are not discussed in more detail within this review.

These different economic evaluations are based on similar principles. However, while they may first appear to be interchangeable, they differ in their fundamental methodology and interpretation. A key difference across the different types of full economic evaluation is how the outcome is expressed (Table 1). Cost-benefit analyses express the benefit or consequence of the health intervention in question in monetary terms (10, 14). In contrast, cost-effectiveness analyses measure the health consequences of the health intervention in a single natural unit (such as life-years gained, cases averted, or cases detected), and cost-utility analyses (a specific type of cost-effectiveness analysis) measure the health consequences using a generic measure of health status that considers the effects on both mortality and morbidity (10, 14), such as DALYs and QALYs (15). There is also variation in how these different analyses consider the efficiency of resource allocation (Table 2).

Table 2.

A summary of how the different types of analyses consider the efficiency of resource allocation and examples of their use.

	Cost-effectiveness analyses	Cost-utility analyses	Cost-benefit analyses
How they consider the efficiency of resource allocation	Cost-effectiveness analyses using disease or programme specific metrics are often only concerned with how to use healthcare resources in a way that maximises their output for the cost. As such they are considering technical efficiency, identifying the option that provides the maximal health care for a given cost, or delivering a certain service at a minimal cost.1	Cost-utility analyses can consider the optimal allocation of healthcare resources (such as the mix of interventions) in a way that results in the maximum health gain for a given level of expenditure. In this way, it considers allocative efficiency within the health sector (i.e., it only deals with quasi-allocative assessments) (16).	Cost-benefit analyses can be used to consider the optimal allocation of resources in its broadest sense because once the benefits have been converted into monetary terms then the net economic benefit of different activities can be compared (including to those outside of the healthcare sector). It can therefore consider allocative efficiency across different sectors/across society.
Examples of when it is useful	Useful when a stakeholder is interested in a particular output metric and/or when you do not need to compare the outcome to other interventions/policy options.For example, comparing a range of different malaria interventions when investigating the cost per case averted.	Useful for informing health policy decisions when directly comparing different health interventions that fall within the same budget or benefit package (such as when considering the optimal allocation of a health care budget).For example, deciding if a new vaccine or treatment should be adopted within a national health benefit package.	Useful for cross-sectoral comparisons, such as looking at if other government spending should be reallocated to the healthcare sector (17, 18). Also useful when evaluating health policy/interventions where the health outcome metrics are not suitable (e.g., prenatal genetic screening for Down's syndrome (19)), and in certain complex intervention contexts (such as interlinked packages of care, where maximising health is not the only objective). For example, for considering non-pharmaceutical interventions for COVID-19 control in terms of both health and non-health outcomes.

Note that cost-effectiveness analyses can be used to assess allocative efficiency within and potentially beyond the health sector for life-saving interventions when the outcomes are measured in terms of deaths averted or life-years gained, etc. (16).

Underlying the differences in how the outcome is expressed for economic evaluations are differences in the economic foundations. Cost-benefit analysis is traditionally based on a welfarist approach foundation, where the health outcomes are judged by the extent of their contribution to overall societal welfare based on the preferences of the individuals (their willingness to pay). In contrast, cost-effectiveness/cost-utility analysis are based on an extra-welfarist approach foundation, where the objective is traditionally to maximise contributions to societal health, measured as the sum of individual health status. An overview of these foundations is provided in Box 1.

Box 1. Summary of the welfarist vs. extra-welfarist approaches to resource allocation.

Under the welfarist approach, the efficiency of a health intervention is based solely on the individuals' perceived value of the welfare that results from it (known as individual utilities^*). In functioning markets, social welfare based on individuals' preferences can be revealed through the market. However, due to market failures in health care, individuals' preferences must be elicited to value the consequences of implementing health care interventions (20, 21). These individual preferences are reflected by what the individuals are willing to pay (or give up) for the outcomes of the healthcare intervention in question; the higher their willingness to pay, the more the individuals prefer the consumption of these healthcare goods/services over alternative goods (22, 23). Under this approach, each individual is considered to be the best judge of their own welfare (23, 24) and social welfare is typically considered to only be a function of these individual preferences. Improvements in social welfare are judged in terms of a “potential Pareto improvement,” wherein a given change will be a potential Pareto improvement if individuals benefiting can compensate those made worse off whilst remaining better off (known as the Kaldor–Hicks criterion) (20, 25). However, this criterion is almost always applied hypothetically (i.e., no compensation actually needs to be paid) and hence some people do inevitably “win” while others “lose” (20). Here, health is often only taken into account insofar as it enables welfare to be derived from the consumption of healthcare goods/services (24) that is, only based on individual preferences. This approach is based on several assumptions regarding individuals' welfare maximising behaviour, including that individuals will make rational choices based on their preferences for the consumption of different goods, as well as that social welfare is only a function of such individual preferences (26).

The field of economic evaluation in health care has increasingly adopted an alternative framework, known as the extra-welfarist approach. The extra-welfarist approach was developed to adapt the classical welfare economic approach to the particular characteristics and context of priority setting within the healthcare sector (27). There are different interpretations of extra-welfarism, but in practise, they almost exclusively focus on the importance of health as the main outcome of health policies (27). Consequently, it has been described as introducing an important class of extra welfare sources that allows the consideration of other factors beyond individual utilities/preferences (22, 23, 28). Because under this approach the most important output of health services is considered to be health outcomes (28), information regarding different health status or health gains can be considered when evaluating interventions (24). Most extra-welfarists would argue that the “need for health care” and not individual demand, should be the priority for the allocation of resources within the health sector (27) and individuals' own judgements about their welfare are not necessarily the most important (24, 28). Extra-welfarism is based on the underlying idea that the assumptions regarding individuals' welfare maximising behaviour which underpin the welfarist approach, do not necessarily apply to health behaviours and healthcare (26, 29), and it rejects the exclusive focus on individual preferences when evaluating health interventions, allowing other factors to be considered (23).

These concepts are discussed further in the following papers(22–24).

^*Note that utility in this context is not the same metrics used with cost-utility analysis [such as quality-adjusted life-years (QALYs) and disability-adjusted life-years (DALYs)].

A limitation of this review is that it only discusses the most commonly used types of full economic evaluations. However, as well as full economic evaluations, there are other methods/frameworks used in priority setting in global health that include health economic analysis. For example, budget impact analysis examines the financial impact of the adoption and diffusion of an intervention within a particular setting, considering its affordability (30). Furthermore, multicriteria decision analysis (MCDA) is also used to assist policymakers in choosing between options where there are two or more relevant criteria (29) and can extend economic evaluation methods to consider other aspects such as acceptability, ethics and equity. Ochalek et al. (20) provide a summary of the methods/tools available for priority setting focusing on LMICs.

Cost-Benefit Analyses

A cost-benefit analysis (also referred to as benefit-cost analysis) is a comparative analysis of the relative costs and outcomes of two or more alternative courses of action, where both the costs of an intervention and its resulting outcomes are expressed in monetary terms (31). In the past, it has been referred to by some as the gold standard economic evaluation approach (31, 32), and it is also used within other public sectors (17, 31). Cost-benefit analyses can be used to consider the optimal allocation of resources in its broadest sense because once the benefits have been converted into monetary terms then the net economic benefit of different activities can be compared (including to those outside of the healthcare sector). It can therefore consider allocative efficiency across different sectors/across society (Table 2).

Cost-benefit analyses are commonly summarised by estimating the net benefit of the intervention (the monetised benefits minus its costs), or by a benefit-cost ratio (the monetised benefits divided by its costs) (33). However, other outcome measures can also be used, such as the internal rate of return.

Conventionally, cost-benefit analysis involves summing the values of the costs and benefits of an intervention based on the preferences of those affected (34). The theoretical foundation of cost-benefit analyses for healthcare is based on the welfarist approach (Box 1). Conceptually, this is based on three elements (18). The first is that each individual is the best judge of their own welfare (or well-being) (18). The second is that measures of welfare are what is important when measuring health, and these are based on the utility (or perceived value) individuals' receive from the consumption of healthcare goods/services (29). The third is that the preferred policy/intervention is that which maximises social welfare, that is, the reallocation of resources for an intervention are justified as long as the net benefits increase (or the gains can hypothetically fully compensate costs or losses) (18). The framework does not typically take into account “who” is benefiting or losing, and no compensation need actually be paid (Box 1). For functioning markets, social welfare based on individuals' preferences can be revealed through the market (21). However, due to market failures in health care, individuals' preferences must often be elicited using other methods (20). In practise, various methods are used to quantify the health consequences in monetary terms, including willingness to pay (Box 2) and valuing productivity gains (Box 3). It is debatable how many analyses fit within these original underpinnings when being applied to healthcare decision making.

Box 2. Willingness to pay.

The willingness to pay technique is based on the premise that the maximum amount an individual is willing to pay (or sacrifice) for a given commodity is an indicator of its “value” to them (35). Using the willingness to pay technique in this context could involve estimating what an individual is willing to pay for certain health benefits, consequently estimating the value of the health benefits of an intervention in monetary terms for that individual. An advantage of this is that it is argued that when an individual is considering their maximum willingness to pay, they will take account of all the attributes of the service of importance to them, not just the health gains (35). Willingness to pay estimates are usually derived using either stated or revealed preference methods. Stated preference methods typically use survey questions to ask respondents about their willingness to pay for an outcome within a hypothetical scenario (33). In contrast, revealed preference methods indirectly infer the value of nonmarket outcomes (such as health benefits) from observed behaviours or prices for related market goods (33). For example, some individuals accept extra income for work that is associated with an increased risk of injury or death or spend more money on items that have enhanced safety features (31). When the extra income received or increased expenditure is compared to the change in the degree of risk associated with a particular activity, it is possible to establish the individuals' personal valuations implicit in the observed behaviour (31). An example of a willingness to pay metric is the “value per statistical life” which captures how much individuals are willing to pay to reduce the risk of death and is used to estimate a monetary value on reductions in mortality. This is often based on trade-offs individuals are willing to make between fatality risk and wealth. It is important to note that the term is often misunderstood or misinterpreted as the value the government or the researcher places on saving a life (36), which is a fundamentally different interpretation/a different type of output.

Willingness to pay methods are associated with several limitations in the context of healthcare:

• Willingness to pay estimates are sensitive to the survey methodology and how the questions are phrased (37, 38). In addition, an individual's willingness to pay is inevitably correlated to their ability to pay and therefore willingness to pay estimates may be affected by the individuals' wealth, income and social status (31, 39–41). Consequently, it has been argued that monetary valuations of health consequences based on willingness to pay methods leads to evaluations intrinsically favouring interventions against diseases of the wealthy over those of the poor (40, 41). In addition, both stated preference and revealed preference methods have their own specific limitations:

• Stated preference methods in this context are based on a number of assumptions, including that individuals' will make rational choices when valuing different health outcomes (42, 43). However, the choices given within questionnaires are hypothetical and there is little need for a respondent to think carefully about what they would choose in a real situation, or to report accurately (33). In addition, when individuals respond to these questionnaires (especially those asking about unfamiliar outcomes, involving small probabilities and long-time horizons), they tend to ‘construct' their preferences on the spot in response to context-specific stimuli (42, 44). Responses are therefore vulnerable to psychological biases (42, 44). Furthermore, unlike when buying private goods, where consumers gather information, compare it to similar goods and actively evaluate their preferences, they may not have all of the information and knowledge necessary to rationally value different health outcomes.

• Revealed preference methods indirectly estimate individuals' willingness to pay by inferring the value of a health outcome from observed behaviours or prices for related market goods. However, individuals may not have perfect knowledge of the risks associated with the observed behaviours in question, and the estimates may not be representative of the risk preferences of the overall population (45), potentially biassing the willingness to pay estimates. Revealed preference methods are often used to estimate the value per statistical life metric. However, far fewer studies are conducted in low- and middle-income countries (LMICs), and the quality varies widely (17). Consequently, value per statistical life estimates often to need to be extrapolated to LMICs settings (39), potentially generating substantial uncertainty in the resulting output.

Box 3. Valuing productivity gains.

The valuing productivity gains (or human capital) approach involves placing a monetary value on the estimated productivity losses associated with a disease that are averted due to a health intervention. However, this is associated with several challenges (note that these also apply when including productivity gains within cost-effectiveness/cost-utility analyses):

• Quantifying productivity losses or gains accurately is difficult and identifying the appropriate unit to measure is challenging. The approaches used for this can give very different outcomes (Figure 5). For example, within the human capital approach, all potential production not performed by a person because of morbidity or early mortality is counted as a production loss (41, 46, 47). In contrast, within the friction cost approach, the calculated production losses are limited to the time needed to replace an ill employee and train a new employee (friction period) i.e. only the productivity lost by an ill employee before they are replaced (41, 46, 47).

• In addition to how the productivity losses are quantified, how they are valued is also variable (48). For example, whether/how to include taxes, employer-paid benefits, how to capture labour market constraints on how much work an individual does and whether to adjust for future increases in wages are issues subject to debate (48). This variation in how the productivity costs are quantified and valued leads to inconsistencies between studies (49).

• Valuing productivity gains can discriminate against those who receive lower wages and those not in the workforce (such as children and the elderly). This is made worse by the fact that the productivity losses/gains related to unpaid work (such as housework and voluntary work) are difficult to quantify and are often ignored (50).

• This approach has been criticised for not being consistent with the theoretical foundations of cost-benefit analyses in welfare economics, as it focuses on changes in productivity rather than measuring overall social welfare and is not based on an individual's personal valuation of the benefits (31, 51, 52).

Within cost-benefit analyses, the decision rule for selecting an optimal intervention is if its net benefit is positive or its benefit-cost ratio is above one i.e. if its monetised benefits outweigh its costs, then the intervention is deemed justified in terms of increasing societal welfare. Importantly, money or cost savings are not the primary focus (33). Instead, it typically indicates the trade-offs individuals are willing to make between spending on the health intervention outcomes (such as improved health) compared to other goods and services (33)—at least when based on willingness to pay estimates. Although formally a cost-benefit analysis should be based on a welfare economic foundation, the term is often simplified to studies with both costs and outcomes expressed in a monetary form.

There are other health economic analyses that express the benefit of a health intervention in monetary terms, such as return of investment analysis and social return of investment analysis. When these formally evaluate the costs and consequences of an intervention compared to a comparator or counterfactual scenario (i.e., look at two or more alternative courses of action), they are a full economic evaluation and are similar to a cost-benefit analysis and have many of the same advantages/limitations (Boxes 2, 3). However, when these only formally evaluate the costs and consequences of a single policy option and do not appropriately account for the comparator or counterfactual scenario, they are only a partial evaluation (a cost-outcome study) (Figure 1). Although these are often referred to as an extension of cost-benefit analysis, these analyses are less well defined.

Cost-Benefit Analyses: Advantages

More Consistent With How Other Public Interventions Are Evaluated and Facilitate Cross-Sectoral Comparisons

The framework used within cost-benefit analyses is more consistent with how other public interventions/policies (not only within the health sector) are evaluated, i.e., based on how much individuals are willing to trade for the benefits (17, 31). Cost-benefit analyses can therefore be used to consider the optimal allocation of resources in its broadest sense because once the benefits have been converted into monetary terms then the net economic benefit of very different activities can be compared, including to those outside of the healthcare sector (53). For example, the “gain” to society from a new health policy can be compared to investing in transportation infrastructure or education (53). This facilitates cross-sectoral comparisons and allows consideration of the allocative efficiency of resources across different sectors/society (Table 2). In contrast, cost-utility analyses typically only consider the allocative efficiency of resources within the healthcare sector (Table 2).

Potential to Value a Wider Range of Benefits

This type of analysis can incorporate benefits and costs that fall outside of the healthcare sector. This can capture potential benefits of interventions missed by cost-effectiveness/cost-utility analyses.

Monetary Output Is Desirable to a Range of Stakeholders

A monetary output tends to be desirable to both healthcare and non-healthcare decision-makers (such as those of the ministry of finance and patient representatives), as well as the public. Also, by placing 'proxy' monetary values on all identified impacts of interest to the stakeholders they can account for multiple stakeholders' views of impact simultaneously. This facilitates the stakeholders' voice in resource allocation decisions.

Cost-Benefit Analyses: Limitations and Areas of Debate

Methodological Difficulties Regarding Placing a Monetary Value on Health Benefits and Variations Between Studies

There is no consistently agreed-upon gold standard method for placing a monetary value on the health benefits of interventions. Consequently, although they have the same monetary outcome, the approaches used can be highly variable—making it difficult to compare different studies (51). The different main approaches and their associated limitations are outlined further in Boxes 2, 3.

A comparative analysis of the cost-benefit analyses of vaccination found that applying different approaches to monetise health benefits in cost-benefit analyses can lead to widely varying outcomes (51). The variation in the approaches taken (and their quality) is important as there is always going to be a need to compare different health economic analyses informing resource allocation. In addition, the variation in approaches could lead to biases in the results of health economic analyses and subsequently, the health policies/decision making they inform. We would argue that the greater the variation between the different studies the less useful they are for practically informing resource allocation within global health. Recently, a reference case for cost-benefit analysis has been developed (33). These guidelines will hopefully bring much-needed consistency in this area.

Difficulties Capturing Non-fatal Health Outcomes

Willingness to pay estimates are less common for non-fatal health outcomes, particularly within LMICs. Hence, many studies in this area use averted cost or averted cost-of-illness estimates as a proxy to quantify the monetary benefits of non-fatal health outcomes of health interventions (33) (i.e., they look at the averted direct costs and productivity costs that are associated with a condition or treatment). However, the quality of such cost/cost of illness studies is variable (54, 55). This makes it harder to apply a cost-benefit analysis for evaluating interventions whose main health benefits are not related to averted excess mortality. In addition, although this is a pragmatic solution to limited data, it could be argued that it does not fit within cost-benefit analyses welfarist approach underpinning.

Another method is to use a willingness to pay threshold to convert DALYs averted or QALYs gained into a monetary value (33). Although this is increasingly being used, it has been argued that these conversions have no theoretical basis and can be very arbitrary (39).

Potential Misinterpretation and Difficulties Regarding the Practical Interpretation

The results of these studies can have the potential to be misinterpreted. Specifically, these types of studies are typically estimating measures of social welfare or potential economic benefits (not estimating realisable financial benefits). This means that the economic benefits being reported are typically an approximation as they are a way of summarising the health benefits experienced by a population in monetary terms (which can be based on the approximated monetary value of estimated productivity gains or willingness to pay methods). For example, the human capital approach would be valuing potential productivity gains and not the productivity gains actually experienced by the population and the willingness to pay technique would be valuing the benefits based on the individuals' preferences and how much they would be willing to pay to obtain the health gains. The risk here is that many stakeholders may not realise how intangible these estimated economic benefits can be and may misinterpret them as the actual monetary benefit to society (i.e., real or realisable monetary benefits). For example, if the estimated benefit-cost ratio of an intervention is 6.0, this can be misinterpreted as the intervention generating six times the amount spent on it (in the same way as a financial investment portfolio might be interpreted). However, not all the estimated economic benefits are realisable, so it may not generate cost savings at all. This is not to say that cost-benefit analysis is not meaningful but greater care is needed in how these studies are reported and interpreted. Importantly, the goal of such analyses is typically to determine if an intervention is justified in terms of increasing societal welfare and if the estimated economic benefits outweigh the cost of the intervention, not if it generates fiscal cost-savings.

Similarly, when a study reports that the economic benefits outweigh the costs of an intervention it can be referred to as having a positive return on investment. However, this may be misleading to policymakers as the reported return on investment of public health interventions is increasingly being misunderstood as a synonym for interventions generating “cost-savings” (39). For example, a systematic review reported that the median return on investment for public health interventions was 14.3 to 1 (56). Many misinterpreted this to mean that for every £1 spent, public health interventions will save the public sector £14.30 in cash; implying a cash return of 1,430% (39). However, this is not the correct interpretation as not all of the economic benefits calculated will be fiscal cost-savings (39).

A further limitation is that there is no threshold (marker or cut off point for when the perceived costs outweigh the perceived benefits) for which an intervention is considered to be of good value in a particular setting. The decision rule of whether to implement the intervention under consideration is based on whether the benefits outweigh the costs (i.e., if the benefit-cost ratio is above one) (20). Typically, cost-benefit analyses do not account for issues surrounding fixed budget constraints for particular sectors (e.g., health) and therefore fail to accurately reflect the opportunity costs of additional costs being imposed on these budgets and what other health interventions may need to be displaced (20).

Equity and Distributional Concerns

Cost-benefit analyses involve placing an economic value on averted mortality and morbidity. However, this can have implicit equity and distributional concerns and many have argued that such evaluations will intrinsically favour health interventions benefiting richer over poorer populations (40, 41, 57). For example, when aiming to quantify the economic benefits experienced by a group of patients, all things being equal, the value of the increased social welfare or productivity gains of averting disease in richer populations will be higher than poorer populations within the same country/setting (58, 59). In addition, the economic benefits of treating women for a condition (who have a higher burden of unpaid work and on average lower salaries) could easily be estimated to be lower than for treating men, implicitly implying that interventions targeting diseases in men have a higher value than for women (58, 59). A way around this issue is to assume the same value of time or life for everyone (regardless of gender, social-economic status, and employment). However, it could be argued that doing this means the estimated economic benefits are more hypothetical and have a less obvious meaning i.e. if you use the same value of time for everyone, what do the estimated economic benefits actually correspond to. Again, the problem is that it is not necessarily always being understood that the economic benefits being presented are often an approximation/theoretical and not necessarily directly realisable to the setting's society/economy.

It should be noted that attempts have been made to adapt cost-benefit analyses to address these equity and distributional issues (33, 34, 60). However, it is yet to been seen if these will become operationalised into policy-oriented cost-benefit analyses within global health.

Cost-Effectiveness and Cost-Utility Analyses

Cost-effectiveness/cost-utility analyses are some of the main types of economic evaluations used for healthcare (11). They are a comparative analysis of the relative costs and outcomes of two or more alternative courses of action. Cost-effectiveness analyses measure the health consequences of an intervention in a single natural unit (such as cases averted, or life-years saved). However, a limitation of this is that it is difficult to compare studies investigating interventions targeting different diseases or different stages of care, since their health consequences will be expressed in different units, limiting its potential use for informing policymakers. To address, this a specific form of cost-effectiveness known as cost-utility analysis was developed. Cost-utility analyses measure the health consequences with a generic measure of health status, which can account for benefits on both reduced morbidity and mortality, such as DALYs and QALYs (61). As these metrics can be used for a wide range of diseases, the cost-effectiveness estimates for different healthcare interventions can be directly compared to each other (such as comparing the cost-effectiveness of a malaria intervention to a tuberculosis intervention). In practise, there has been a blurring of the distinctions between cost-effectiveness and cost-utility analyses; as a result, literature on cost-effectiveness often encompasses both these approaches and cost-utility analyses are often referred to as a cost-effectiveness analysis (16). Cost-effectiveness/cost-utility analyses are typically based on the extra-welfarist approach (Box 1), where the objective is traditionally to maximise contributions to societal health, measured as the sum of individual health status.

Figure 3 provides a visual representation of what can happen when comparing a new health intervention to a comparator within a cost-effectiveness/cost-utility analysis [outlined further in (62)]. When the analysis is using a generic measure of health status (such as DALYs/QALYs within cost-utility analyses), standardised cost-effectiveness thresholds (an explicit cost per unit of outcome below which an intervention is considered cost-effective—such as £20,000 per QALY gained) are frequently used as a decision rule to class whether an intervention is cost-effective or not (Figure 3). Such thresholds can be determined using a variety of methods (63). When this is done, the analysis can consider the allocative efficiency of resources within the healthcare sector (Table 2) (i.e., the allocation of healthcare resources in a way that results in the maximum health gain for a given level of expenditure) (16, 53). As such, when looking at mutually exclusive policy options (such as different treatment options for tuberculosis), the analysis considers the options incrementally and identifies the most effective option that is below the settings cost-effectiveness threshold (which is not necessarily the same as the option that has the lowest cost per unit of effect). In contrast, when a cost-effectiveness analysis uses a disease or programme specific outcome measure (such as cases averted), it is rare to have a standardised cost-effectiveness threshold available to compare the results to and it is difficult to compare the results to other studies. In this case, the goal of the analysis is often to identify the most efficient option in terms of the lowest cost per unit of effect i.e., identify the option with the lowest cost-effectiveness ratio (such as the lowest cost per case averted). In this context, cost-effectiveness analyses using disease or programme specific metrics are often only concerned with how to use healthcare resources in a way that maximises their output for the cost (known as technical efficiency (Table 2) (16, 53).

Figure 3.

A schematic of the cost-effectiveness plane. Panel A indicates the four different quadrants. Panel B illustrates the different decision rules in relation to a cost-effectiveness threshold within the different quadrants.

The outputs of cost-effectiveness/cost-utility analyses are typically expressed in ratios (a cost per unit of effect). It is also possible to express the outputs of cost-utility analyses in monetary terms (such as net monetary benefit) (20, 64). With net monetary benefit, the health outcomes are monetised based on a cost-effectiveness threshold which represents a societal valuation of a health metric, rather than an individuals' willingness to pay for health gains. Hence, although this is similar to a cost-benefit analysis, it does not have the welfarist foundation and uses a different discussion rule (Box 1).

Note that there are other specific forms of cost-effectiveness analysis used for priority setting, such as distributional cost-effectiveness analysis and extended cost-effectiveness analysis. These are described in more detail in (20, 65–67).

Cost-Effectiveness and Cost-Utility Analyses: Advantages

Focus on Societal Valuation of Health

The framework behind cost-effectiveness/cost-utility analyses is the most appropriate if you accept that health is a fundamental good whose value should be determined by society, regardless of how much individuals are willing to exchange for it (i.e., regardless of their willingness to pay). This is more consistent with the theory of health rights that World Health Organisation (WHO) advocates for (68). In addition, it avoids the difficulties in valuing social welfare that occur due to market failure in the health care market.

More Commonly Performed

Cost-effectiveness/cost-utility analyses are a more commonly conducted and published method compared to cost-benefit analyses (11). This makes it easier for policymakers to compare different studies and interventions, facilitating the decision-making process. However, the comparability of cost-effectiveness analyses using disease or intervention specific metrics is difficult between different health programmes/interventions and it cannot be used to compare programmes with different aims/outcomes; which is why cost-utility analysis was developed.

Forgoing the Need to Monetize Health Benefits

The framework does not require assigning a monetary value on health gains. By foregoing this step, the analysis draws attention exclusively to health benefits and avoids the corresponding ethical and equity issues that can arise when monetizing them (69). However, it could be argued that some of these ethical and equity issues still arise when setting the cost-effectiveness threshold.

Guidelines for Reporting and Methodology

There are more established guidelines/reference cases for conducting and reporting the results of cost-effectiveness/cost-utility analyses within the health sector and in some settings, country-specific guidelines have been established (70–73). Although we would argue that the methodology used is more standardised compared to that of cost-benefit analyses (51), it should be noted that methodological variation still occurs between different cost-effectiveness/cost-utility analyses (such as the costs that are included, the perspective, and the assumed discount rate) (74, 75), particularly for settings that do not have standardised guidelines/reference cases.

Cost-Effectiveness and Cost-Utility Analyses: Limitations and Areas of Debate

Availability of Cost-Effectiveness Thresholds

A cost-effectiveness threshold is often used as a decision rule to class whether an intervention is cost-effective or not. Some countries have established their own cost-effectiveness thresholds (89, 90) but this is rarer for low and lower-middle-income countries. The absence of an established threshold can make the results of cost-effectiveness/cost-utility analyses more difficult to interpret. In the past, when no country-specific threshold has been set many used the cost-effectiveness thresholds set by the Commission on Macroeconomics and Health (91); namely a cost per DALY averted <3 and <1 times the country's gross domestic product (GDP) per capita for a intervention being cost-effective and highly cost-effective, respectively. However, these are now considered to be too high and have become widely criticised (63, 92–95). The WHO has outlined that these thresholds were not intended for individual investment decisions but as a broad principle for global/regional level analyses (96). They have argued that these thresholds have therefore not been used in the way they were originally intended (96). However, unfortunately, these 1-3 times GDP per capita thresholds are currently still being used within global health (95). Although guidance exists, determining a country's cost-effectiveness threshold remains a complex area (93, 97, 98).

Depending on how it is calculated, the cost-effectiveness threshold can be a way of capturing how much health is expected to be foregone if the resources in question are used for an intervention (the health opportunity cost). When this is the case, the cost-effectiveness/cost-utility analysis can account for the budget constraints within the healthcare sector (99). However, in practise, the cost-effectiveness thresholds commonly used in LMICs do not account for this and are often only an expression of value by a specific party (such as an international organisation or government), without consideration of health care system constraints (99) (such as the 1 and 3 times GDP per capita thresholds previously mentioned). Revill et al. (99) discuss these concepts further in the context of priority setting in global health.

The Generic Measures of Health Status (DALYs and QALYs) Have Limitations

The use of DALYs or QALYs as outcomes within cost-utility analyses has been a notable improvement compared to only measuring the intervention's effect in terms of disease cases or deaths averted, and these measures permit comparison across different diseases or interventions. However, these generic measures of health status have limitations (Box 4). A general limitation of these generic summary measures is that they could be argued to be overly simplistic and reductionist and may not be capturing all of the health benefits of an intervention (80, 84). In addition, these metrics do not work well in certain situations, such as when health outcome metrics are not the most suitable. For example, DALY/QALY metrics could be argued to fail to account for the complexities of investigating the benefits of prenatal genetic screening and pregnancy termination for Down's syndrome (such as the impact on caregivers) (19).

Box 4. Limitations of DALYs and QALYs.

DALYs: The universal disability weights used within DALY calculations do not account for how the local context may influence the burden of a disease (such as the impact of living in poverty) (76, 77). Within the updated global burden of disease study (GBD) framework (post-GBD 2010), the disability weights are intended to solely be measures of losses of “optimal health” and are not intended to represent losses of well-being/welfare (78, 79). Consequently, the DALY disability weights do not consistently/fully account for the psycho-social implications of a disease or its sequelae (80) and its overall impact on quality of life. Critics have argued that disease burden should be quantified in terms of overall welfare loss and that only measuring burden as “lost health” may lead to biases when estimating the disability weights (78, 81, 82). Additionally, although the disability weights are meant to be standardised for a given disease, there is still variation in what weights are used in practise (27, 83). Furthermore, in comparison to preference-based health-related quality of life measures, DALYs may not fully capture the relative benefits of interventions that reduce the functional burden of a condition without curing it.

QALYs: QALY weights are typically estimated for a particular setting. A variety of tools/questionnaires can be used for this (84) (such as the EQ-5D questionnaire); however, it has been shown that the different utility measurement tools provide variable results (85) and the outcomes are sensitive to the survey design and sample (86). This can lead to variation in the utility weights being used by different studies. The generic instruments used to generate QALY utility weights are also insensitive to some medical conditions (84) and have also been critiqued for having insufficient sensitivity to measure small but clinically meaningful changes in health status (86). There is also ongoing debate regarding who should value the health states (patients with the condition or the general population) (84, 87), and this variation in methodology can lead to inconsistencies in how QALY utility weights are estimated. In addition, the financial barriers for accessing QALY weight estimation tools and databases disproportionately affect low- and middle-income countries (LMICs) (88).

Figure 5.

A hypothetical example showing the potential difference between the human capital approach or the friction cost approach when estimating productivity costs. Costs are being discounted at 3% per year.

Often Do Not Capture Non-health Benefits and Limited Cross-Sector Comparisons

The effectiveness framework of cost-effectiveness/cost-utility analyses focuses on measuring the benefits of public health interventions in terms of health gains (69, 100). When an intervention leads to cost-offsets/health savings (such as averted future medical costs), these cost-savings can be included and are subtracted from the cost of the intervention. However, many health interventions generate important additional benefits to other sectors, such as the environment or education (69, 100). Although all financial and some non-financial broader benefits can, in principle, be monetised and included in cost-effectiveness/cost-utility analyses when using a broad societal perspective, there is variation in when and how they are included (101). Consequently, many cost-effectiveness/cost-utility analyses only consider the benefits to the health sector and the impacts of public health interventions on other sectors are often not included (100). In some cases, this restrictive perspective is linked to the mandate for the healthcare budget, such as for solely improving health. Not capturing these non-health benefits could be undervaluing the broader benefits of many health interventions (39).

This limitation is particularly relevant for the evaluation of complex interventions (such as those involving interlinked packages of care) as the complexity means the intervention may not fit into one of the current appraisal systems, and maximising health may not be the intervention's only objective (102).

Notably, the decision in question may not only involve the health sector and can involve trade-offs between health and other sectors/societal aims. Using non-monetary metrics limits potential cross-sector comparisons, that is, it is hard to directly compare the estimated value of investing in a health intervention (based on the cost per DALY averted or QALY gained) to investing outside of the health sector, such as in education. This makes it difficult to use this type of analysis to justify the reallocation of other government spending to the health sector (such as adjusting tax policies) and more difficult for decision-makers to compare the value of money for a broad range of potential activities, including those from non-health sectors.

Generally, cost-effectiveness/cost-utility analyses have a more limited potential in how they can consider the efficiency and resource allocation compared to cost-benefit analyses (Table 2).

There is increasing recognition that the maximisation of health gains is not the only factor that individuals are concerned about in relation to healthcare decision making, as DALYs/QALYs may not be capturing the full consequences of interest when evaluating different interventions (24). Consequently, the importance of reporting other metrics/outcomes as well as DALYs/QALYs is becoming recognised (24, 29).

Variation in the Inclusion of Productivity Costs and Potential Double-Counting of Effects

There is variation regarding what types of productivity costs are included within cost-effectiveness ratio calculations (Figure 4). While including the productivity costs that are related to the patients' productivity losses associated with accessing an intervention is relatively uncontroversial when using a societal perspective, many have argued that those related to the productivity gains associated with prevented morbidity/mortality should not be included (10, 46, 103–107) (Figure 4). This is because these benefits are arguably being captured within the QALY/DALY effectiveness measure. Therefore, including these productivity gains within the cost part of the equation would potentially lead to the double-counting of the effectiveness of the health intervention (Figure 4) (41). However, this recommendation has been challenged with some arguing that the QALY measure does not capture these productivity gains (10, 103–107). Guidelines regarding the inclusion of these productivity gains vary (72, 74).

Figure 4.

Schematic surrounding the inclusion of productivity costs within cost-effectiveness ratios. Adapted from (49).

As previously highlighted, estimates of productivity costs are highly sensitive to the method used (48) and it is important to be aware of the potential variation in methodology when comparing studies (Box 3). This variation could lead to potential biases in setting health policy and further guidance is needed.

Equity

It should be noted that although cost-effectiveness/cost-utility analyses avoid the need to monetize health benefits, they are still associated with equity and ethical issues (40, 58, 100, 108) (such as those associated with the DALY/QALY metrics and the setting of the cost-effectiveness threshold). For example, cost-effectiveness/cost-utility analyses rarely provide information about who gains and who loses from health interventions (i.e., are equity neutral) and therefore do not prioritise improving health among the poorest even though this may be a societal objective. There have been new efforts to incorporate equity in cost-effectiveness/cost-utility analyses as a quantitative component which may potentially help with partly addressing these limitations, such as the development of distributional cost-effectiveness analysis and extended cost-effectiveness analysis (65–67). However, this is still an area that needs attention regarding practical implementation with regards to informing resource allocation decisions in global health (109, 110).

Conclusions

The choice of which type of health economic analysis to use is a matter for the decision-maker(s) and will depend on the local context, including the values and interests of the stakeholders, the question being addressed, and the resources that are being considered for reallocation (18, 111). However, in the context of resource allocation within the healthcare sector, it is important that full economic evaluations are used (which explicitly compare the costs and consequences of the interventions or health policies in question with a comparator scenario). Analyses that do not do this, even if they are incorrectly referred to as a full economic evaluation, may ignore comparisons to relevant policy alternatives and can generate misleading conclusions to policymakers.

Among the different types of full economic evaluations, cost-effectiveness/cost-utility analyses and cost-benefit analyses are typically used in this area. Although they may appear similar, it is important to recognise when interpreting these studies, that the welfarist foundation of cost-benefit analyses and the extra-welfarist foundation of cost-effectiveness/cost-utility analyses represent two fundamentally different ways of evaluating and valuing the benefits of health interventions (Box 1). There are also differences in how they consider the efficiency of resource allocation (Table 2).

Our key messages are summarised in Box 5 and Table 3. Moving forward there needs to be greater awareness within the public health field of the foundations, advantages and limitations of the different types of economic evaluations used for informing resource allocation decisions.

Box 5. Key messages.

• In the context of resource allocation within the healthcare sector, it is important that full economic evaluations are used (which explicitly compare the costs and consequences of the interventions or health policies in question with a comparator scenario).

• Moving forward there needs to be greater awareness within the global health field of the foundations, advantages and limitations of the different types of economic evaluations used for informing resource allocation decisions (Table 3).

• Cost-effectiveness/cost-utility analyses have key advantages for directly comparing different health interventions when considering resource allocation within the healthcare sector. However, it is important to acknowledge that notable non-health benefits of interventions may not be captured and it is also difficult to compare the results of cost-effectiveness/cost-utility analyses to other sectors, limiting cross-sector comparisons.

• Cost-benefit analyses facilitate cross-sectoral comparisons and can include a wider range of benefits. However, there needs to be greater awareness of the current methods and variations in the approaches being used when placing a monetary value on health benefits (Boxes 2, 3), and the potential for the results to be misinterpreted.

• Importantly, the economic benefits being reported within such studies are typically an approximation - they are a way of summarising the health benefits experienced by a population monetarily in terms of social welfare or potential economic benefits related to productivity gains. These estimated economic benefits should not be misinterpreted as actual realisable or fiscal monetary benefits to payers or to society.

Table 3.

Summary.

	Cost-benefit analysis	Cost-effectiveness/cost-utility analysis
Summary	A comparative analysis of two or more alternatives in terms of their relative costs and consequences. Within a cost-benefit analysis, both the costs and outcomes are expressed in monetary terms. This type of analysis typically involves summing the values of an intervention's costs and benefits based on the preferences of those affected. The analysis foundation is based on the welfarist approach (Box 1).	A comparative analysis of two or more alternatives in terms of their relative costs and consequences. Within a cost-effectiveness analysis, the health consequences of a health intervention are measured in a single natural unit. In contrast, a cost-utility analysis is a specific form of cost-effectiveness analysis, where the health consequences are measured using a generic measure of health status that considers both the effect on mortality and morbidity (such as DALYs/QALYs). The analysis foundation is based on the extra-welfarist approach (Box 1).
Advantages	More consistent with how other public interventions are evaluated and the facilitation of cross-sector comparisons (i.e., can compare the economic benefit to investments outside of the healthcare sector). It can therefore consider allocative efficiency of resource allocation across different sectors/society (Table 2). Potential to value a wider range of benefits. A monetary output is desirable to a range of stakeholders.	More commonly performed-making comparisons of different studies easier.More established guidelines for methodology and reporting.Forgoes the need to monetize health benefits and the corresponding equity and ethical issues.The cost-effectiveness thresholda can be a way of capturing how much health is expected to be foregone if the resources in question are used for an intervention accounting for the budget constraints within the healthcare sector.
Limitations and areas of debate	Methodological and data difficulties regarding placing a monetary value on health benefits – particularly to non-fatal health outcomes. Methods used are more variable and there are less well-established reporting guidelines – making it harder to compare studies. Output can easily be misinterpreted and difficulties regarding practical interpretation. Less commonly performed than cost-effectiveness/cost-utility analysis. Placing a monetary value on health benefits generates notable equity concerns. Typically, do not account for issues surrounding fixed budget constraints and therefore fail to accurately reflect what other health interventions may need to be displaced.	Limited availability of cost-effectiveness thresholds.DALY and QALY generic summary measures have limitations.Non-health benefits may not be fully captured and there is variation regarding if/how they are quantified and included.Difficult to compare the results to investments outside of the healthcare sector (limited potential for cross-sector comparisons). They have a more limited potential in how they can consider the allocative efficiency of resource allocation particularly beyond the healthcare sector (Table 2).Although cost-effectiveness/cost-utility analyses avoid the need to monetize health benefits, they are still associated with equity and ethical issues.
Key message for policymakers	Cost-benefit analyses facilitate cross-sectoral comparisons and can include a wider range of benefits. However, there needs to be greater awareness of the current methods and variations in the approaches being used when placing a monetary value on health benefits (Boxes 2, 3), and the potential for the results to be misinterpreted. Importantly, the economic benefits being reported within such studies are typically an approximation, in terms of social welfare or potential economic benefits related to productivity gains. The risk here is that many may not realise how intangible these estimated economic benefits can be and may misinterpret them as the actual realisable or fiscal monetary benefits.	Cost-effectiveness/cost-utility analyses have key advantages for directly comparing different health interventions when considering resource allocation within the healthcare sector. However, it is important to acknowledge that notable non-health benefits of interventions may not be captured. Although some of these broader benefits can, in principle, be monetised and included in analyses when using a broad societal perspective, there is variation in when and how they are included. It is also more difficult to compare the results of cost-effectiveness/cost-utility analyses to investment in other sectors, limiting cross-sector comparisons.

^aDepending on how it is calculated.

Author Contributions

HT and YT conceived the manuscript. HT wrote the original draft. RA, LD, WI, KC, MJ, and YT contributed to the writing, reviewing, and editing of the draft. All authors read and approved the final draft.

Author Disclaimer

The findings, interpretations, and conclusions expressed in this article do not necessarily reflect the views of the funding agencies.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.