The Value of Health in a Cost-Effectiveness Analysis: Theory Versus Practice

Abstract

A cost-effectiveness analysis has become an important method to inform allocation decisions and reimbursement of new technologies in healthcare. A cost-effectiveness analysis requires a threshold to which the cost effectiveness of a new intervention can be compared. In principle, the threshold ought to reflect opportunity costs of reimbursing a new technology. In this paper, we contrast the practical use of this threshold within a CEA with its theoretical underpinnings. We argue that several assumptions behind the theoretical models underlying this threshold are violated in practice. This implies that a simple application of the decision rules of CEA using a single estimate of the threshold does not necessarily improve population health or societal welfare. Conceptual differences regarding the interpretation of the threshold, widely varying estimates of its value, and an inconsistent use within and outside the healthcare sector are important challenges in informing policy makers on optimal reimbursement decision and setting appropriate healthcare budgets.

Key Points for Decision Makers

A simple application of the decision rules of a cost-effectiveness analysis using a single estimate of the threshold does not necessarily improve population health or societal welfare.

To arrive at more informative estimates and comparisons of thresholds, researchers should focus on more conceptual clarity on what different thresholds aim to measure and how differences in the value of health between sectors should be interpreted.

A better understanding of the way allocation and displacement decisions are made in practice is needed, which includes the incorporation of equity considerations, the influence of dynamic effects, and differences in bargaining power between sectors.

Introduction

Increasing healthcare expenditures, together with the advance of evidence-based medicine and the desire for transparent decision making, have led to growing use of economic evaluations in healthcare [1]. The purpose of a cost-effectiveness analysis (CEA) is to support decision making aimed at maximising (or optimising) health or societal welfare with available resources. By comparing the costs and benefits of new interventions, such as pharmaceuticals or medical devices, to a relevant comparator, one can answer the question whether these interventions offer value for money. In principle, only interventions that are found to be cost effective should be eligible for reimbursement [2]. Several countries use economic evaluations routinely in their decision-making process, mostly for pharmaceuticals [1, 3]. A CEA is integrated in healthcare decision making to assess the eligibility of funding of pharmaceuticals in countries such as Australia (Pharmaceutical Benefits Advisory Committee), Canada (Canadian Agency for Drugs and Technologies in Health), Poland (Agency for Health Technology Assessment and Tariff System), The Netherlands (National Healthcare Institute), Sweden (Dental and Pharmaceutical Benefits Agency) and the UK (National Institute for Health and Care Excellence). These all pertain to decisions in the context of a collectively financed healthcare system made by public institutions. An important example of a private initiative is the Institute for Clinical and Economic Review in the USA [4].

In some countries, like England and Wales, Australia, and Canada, a healthcare perspective is used in economic evaluations, in which, typically, health is the only outcome considered and costs falling on the healthcare budget are the only costs included. In other countries, such as the Netherlands, Norway, and Sweden, taking a societal perspective is prescribed. The Second US Panel on Cost Effectiveness recommends a two-perspective approach, using both the healthcare and societal perspectives [5]. See Zhao et al. [6] for an overview of the perspectives used in different countries. In both cases, the ratio between costs and health benefits of new interventions (the incremental cost-effectiveness ratio [ICER]) must be compared to a threshold value. This threshold represents the maximum costs per additional unit of health (often expressed as one additional quality-adjusted life-year [QALY]) that is still considered to be acceptable.

The application of a CEA to make societally optimal reimbursement decisions implies an (implicit) view on the value of health [7–9]. While the labelling of what the thresholds represent and the empirical approach to estimating them vary with the perspective chosen [7], their values are related. In both perspectives, the threshold should reflect the societal willingness to pay for additional health, either indirectly through setting a budget for healthcare or directly by estimating the monetary value of health. Both perspectives also face similar challenges in consistently estimating and applying a threshold in practice. For a long time, research in applied economic evaluations focused on (complex) modelling techniques to derive appropriate and accurate ICERs and seemed less involved in determining the appropriate value of the threshold. In recent years, the search for the appropriate values of the comparators has become more prominent and this has resulted in several empirical estimates for both the healthcare and societal perspective [10–12]. These estimates do not only differ between but also within the two perspectives, which raises important normative and empirical questions [13, 14].

The aim of this paper is to help practitioners understand the current research on the value of health, how it relates to the perspective adopted and how this informs cost-effectiveness thresholds. We try to consolidate existing theoretical understanding in a cohesive approach that clarifies the consequences of alternative analytical choices. We do so by first synthesising how the value of health is conceptualised within common theoretical frameworks for a CEA from different perspectives [9, 15–17]. Then, using this synthesis, we highlight three important issues in applying the value of health in practice: conceptual differences in the use of the value of health between the healthcare and societal perspective, empirical differences in the estimated value of health (between and within perspectives) and inconsistencies in how (or whether) costs of interventions are evaluated against their value.

The paper is set up as follows. First, we discuss the theory underlying the decision rules of a CEA and its implications for the value of the threshold. Then, we discuss to what extent assumptions underlying the theory apply in practice and whether potential discrepancies pose a problem for the use of a CEA given its aims. Finally, we conclude with recommendations and avenues for future research.

The Value of Health: Theory

The decision rules of CEAs are derived from conventional economic models. There is a large amount of literature on the theoretical underpinnings of CEAs and the differences and similarities between the healthcare and societal perspectives [9, 15–17]. In this section, we present a short synthesis of the typical economic models in the literature and the role the value of health plays therein. The common starting point in such models typically is a rational decision maker who wants to optimise either health using a given healthcare budget or societal welfare through health interventions, considering all available resources in society. Although models are simplifications of reality, they provide guidance as to what costs and benefits to include in CEAs and how to measure and value them. The decision rules eventually derived from these models depend on the assumptions regarding the resource constraints faced by the decision maker, the goals of the decision maker, and which parameters are considered to be determined within the theoretical model and which are exogenous.

Taking a healthcare perspective, the viewpoint of a (delegated) healthcare decision maker is chosen [18]: the government sets the healthcare budget but delegates the decision what to reimburse from this budget to a lower-level decision maker. To this delegated decision maker, the available budget is exogenous, in the sense that the decision maker has no influence on how it is set. It is typically assumed that the decision maker is concerned only with costs falling on the healthcare budget and wishes to maximise health. Under a fixed healthcare budget, new interventions can only be financed by displacing currently reimbursed care. In that case, the relevant threshold is k: the (marginal) cost-effectiveness ratio of existing care. The resulting decision rule to determine whether a new intervention should be reimbursed can be stated as:

$$\mathrm{\Delta }Q-\frac{1}{k}\mathrm{\Delta }{c}_h>0.$$ 1

This rule entails that the health gained with the new intervention ($\mathrm{\Delta }Q$), most often measured in QALYs, should be larger than the health forgone by shifting a part of the fixed budget to this new intervention at the expense of existing care.1 Dividing the healthcare costs related to the new intervention $c_h$ by k indicates how many QALYs are foregone by displacing current care, either at the margin or through targeted displacement. As in any threshold-based approach, the marginal decision rule only ensures optimal allocation of resources if the current budget is allocated efficiently in the first place.

Consider a simple example. Assume that it is estimated that current care at the margin produces one QALY per 10,000 Euros, so that k = 10,000. Imagine a new intervention would cost 1 million Euros. To pay for this from a fixed budget, 1 million Euros worth of existing care needs to be displaced. This is a marginal budget change. If the intervention is reimbursed, 100 QALYs (1/$\left(\frac{10000}{\mathit{QALY}}\right)$ × 1,000,000) will be sacrificed by displacing current care to pay for the new intervention. If the decision maker wants to maximise health with the budget, reimbursing the new intervention is only optimal if the health gains from the intervention exceed 100 QALYs (which implies that it has an ICER smaller than 10,000). Rewriting shows that Equation (1) is identical to:

$$\frac{\mathrm{\Delta }{c}_h}{\mathrm{\Delta }Q}<k.$$ 1'

This rule shows that the ICER of the new intervention ($\frac{\mathrm{\Delta }{c}_h}{\mathrm{\Delta }Q})$ should be lower than the ICER of the current care (k) it replaces.

Two limitations to this approach can be identified. The first one is that the potentially large costs and benefits outside of the healthcare sector are ignored within this framework. Second, a fixed budget constraint does not allow a decision maker to fund new interventions that offer value for money for society, but less than existing care. The latter point obviously relates to optimality of the fixed budget (and therefore of k).

In contrast, a societal perspective assumes a healthcare decision maker who wishes to maximise societal welfare. In this perspective, the healthcare budget is commonly assumed to be flexible. If new interventions become available of which the societal benefits outweigh the costs, the decision maker can (directly or indirectly) increase the healthcare budget, by raising additional taxes, increasing social insurance premiums or otherwise, to finance them.

When taking the societal perspective, the opportunity costs of an intervention are equal to all consumption being sacrificed, within and outside the healthcare sector. Thus, the total costs associated with producing the QALYs through the new intervention, both inside the healthcare sector ($\mathrm{\Delta }{c}_h)$ and outside the healthcare sector ($\mathrm{\Delta }{c}_c)$, should be counted and their sum should not exceed the consumption value $v_Q$ of the gained QALYs. $v_Q$ reflects the willingness to pay for an additional unit of health: the maximum amount of other consumption (expressed in monetary terms) people are willing to sacrifice per unit of health gained. This implies the following decision rule:

$$v_Q\mathrm{\Delta }Q-\left(\mathrm{\Delta }{c}_h+\mathrm{\Delta }{c}_c\right)>0.$$ 2

As an example, suppose the consumption value of a QALY is estimated at €50,000 and that there is a new intervention that would yield 100 QALYs. For this intervention to be cost effective, the total costs within and outside the healthcare sector should not exceed the benefits of €50,000 × 100 = 5 million Euros.

Rewriting Eq. (2) yields

$$\frac{\mathrm{\Delta }{c}_h+\mathrm{\Delta }{c}_c}{\mathrm{\Delta }Q}<v_Q$$ 2

which states that the costs per QALY produced should be less than the consumption value of one QALY2. A downside of this approach is that it assumes optimally set and (ultimately) flexible budgets, which may be strong assumptions. Additionally, as we discuss below, it might be difficult to find a consensus on how to measure v and aggregate values across individuals.

The differences between the two approaches have been discussed extensively [5, 6, 19, 20]. The healthcare perspective, based on an exogenously set budget constraint, can be considered within the welfare maximisation framework. A relatively general decision rule that brings together both perspectives in an economic evaluation under this framework is the following [18]:

$$v_Q\left[\mathrm{\Delta }Q-\frac{1}{k}\mathrm{\Delta }{c}_h\right]-\mathrm{\Delta }{c}_c>0$$ 3

This rule combines the healthcare perspective (between the brackets) with a full societal perspective. Equation (3) states that the value ($v_Q)$ of the net health gains of an intervention should be larger than the broader societal costs (Δ$c_c$) incurred. The net health gains are calculated as $\mathrm{\Delta }Q-\frac{1}{k}\mathrm{\Delta }{c}_h$, which is equal to the net costs and benefits that fall within the healthcare perspective.

In this way, Eq. (3) provides a clear relationship between the two perspectives. First, the equation reveals that if the healthcare budget is set optimally, so that the marginal cost effectiveness of reimbursed care is equal to the societal value of health and k = v, Eq. (3) reduces to Eq. (2). Moreover, the equation provides a rationale for using a two-perspective approach, as advocated before [5, 21]. This is important when taking a societal perspective in case the healthcare budget is not optimally set. It also allows direct detection of tension between recommendations related to the two perspectives. To allow for the fact that the delegated decision maker might not be able to directly set the healthcare budget, and that this may not be set optimally, healthcare costs are weighted by the fraction $\frac{{\nu }_Q}{\kappa }$. This captures the fact that, under a fixed healthcare budget, the opportunity costs of the funding required for a new intervention will not come in the form of other consumption, but necessarily as the value of the health interventions that will be displaced.

We discussed here the most common descriptions of decision making under a societal or healthcare perspective. Arguably, the main distinguishing features of the healthcare and societal perspectives are the choice of the maximand (health vs welfare) and the scope of the evaluation (including only costs falling on some budget or all societal costs). In practice, mixed situations are possible such as a healthcare decision maker with a fixed budget who attempts to optimise social welfare through healthcare interventions. We return to these mixed situations in the stylised example in Sect. 4.

Does Theory Work in Practice?

The theories underlying the optimal decision frameworks, in both a healthcare and a societal perspective, do not always translate straightforwardly into practice. In this section, we highlight three discrepancies between theory and practice related to the interpretation and use of thresholds3.

v Versus k and the Optimal Size of the Healthcare Budget

In an (economically) optimal world, healthcare budgets should be set in such a way that, at the margin, the costs of producing a unit of health should equal their consumption value. This implies that under optimal (use of) budgets, v should equal k. If v > k, this signals that healthcare budgets are set too low, while k > v signals the opposite. If k is not equal to v, societal welfare could, in theory, be improved by reallocating resources from (or to) the healthcare system to (or from) other societal sectors.

If budgets are not set optimally, applying the default decision rules from neither a healthcare perspective nor a societal perspective alone leads to an optimal estimation of welfare changes and therefore optimal decisions. The optimal size of the healthcare budget can only be determined by consistently applying a full societal perspective, with attention to opportunity costs within and outside the healthcare sector. Decisions made in the context of a non-optimal budget require information on both k and v to make optimal decisions given the non-optimal budgets [18]. It seems that this point, which may be clear from Eq. (3), is still often overlooked.

Conceptual differences in opinion on what v and k should represent complicate the seemingly straightforward comparison between the two offered by Eq. (3). Different approaches to what these thresholds should exactly reflect and choices about how, when and in whom to practically estimate these involve important normative choices that should be made transparent and scrutinised.

Starting with v, as a CEA is typically applied in collective decision making, it should reflect a collective valuation of health somehow. Many may see v as the individual valuation of own health gains that needs to be aggregated across individuals. This could be either ex-ante (before being ill, e.g. in the general public based on hypothetical health changes) or ex-post (after becoming ill, e.g. in patient groups actually experiencing particular health states). Others may view v as a more general willingness to pay for health gains in the context of a collectively financed health system based on solidarity. Such valuations may be broader than individual health gains and could, for instance, be related to or reflect equity preferences [22]. Importantly, using such a broader approach beyond individual valuations of health gains alone does not fundamentally change the decision rules regarding cost effectiveness. Incremental cost-effectiveness ratios can therefore still inform the decision-making process [23–25] and v can still inform the optimal size of the budget. In the Netherlands, an attempt to include societal equity preferences is reflected in the fact that the threshold value increases with the severity of the illness being treated [26, 27]. Ensuring that the applied weights reflect actual societal preferences remains challenging.

Opinions on what k reflects or should reflect differ as well4. If the (re-)allocation of resources within the healthcare budget would be optimal, reimbursing a new (more cost-effective) intervention would lead to the displacement of the least cost-effective intervention currently still funded (e.g. see Siverskog and Henriksson [28]). In that case, k would be equal to the cost effectiveness of this least cost-effective intervention still funded. In practice, however, it is commonly unclear what exactly is removed from the system and displacement is likely to be suboptimal (i.e. not the least cost-effective intervention is taken out) [29].

For decision makers to be able to displace the least cost-effective care requires information on the cost effectiveness of all current interventions, which is unlikely to be available. In addition, the decision maker may not be able or willing to displace care to make room for the new intervention (solely) based on cost-effectiveness information. This means that displacement in practice may take different forms. Arguably, the relevant value of k might thus not be the ICER of the least cost-effective care currently reimbursed, but rather the marginal value of the care that will actually be displaced. Using this alternative value ensures a consistent use of CEA results, albeit in a ‘second-best’ decision-making context, as it facilitates a health-improving allocation of scarce resources even if the displacement and current allocation may not be optimal. In practice, therefore, k is often interpreted as the marginal cost effectiveness of current health spending—a (much) more general estimation of opportunity costs—suggesting that actual displacement may also be more general. Still, even this more general measure might deviate from the actual opportunity costs, as there is no guarantee that the cost effectiveness of the care that is actually displaced is equal to the average marginal cost effectives of the care system. In theory, different deviations from optimal displacement within the budget can be considered into account explicitly (see Eckerman and Pekarksy [30]), but this is hard to do in practice.

In addition to the conceptual differences between researchers on what k and v should reflect, policy makers in practice take other aspects, such as societal and political pressure, into account when making decisions. They do this both in determining the size of the overall healthcare budget and in the weight they assign to different sectors. These policy decisions can result in a healthcare budget and associated value of k that differs from v. Even if we consider these differences legitimate, as “the political and institutional processes that led to these represent social values which may not be captured in other ways” [18], appropriate estimates and comparisons between v and k remain important. They are pivotal in ensuring that both reimbursement decisions and the healthcare budget are in line with the goals set by policy makers and society.

Discrepancies Between Empirical Values of v and k

Next to the conceptual issues that complicate a comparison between k and v, actual empirical estimates of both quantities differ substantially. Looking at the available empirical literature estimating v or k, estimates of v are generally larger than estimates of k [10, 31]. Taking these differences at face value, they suggest that healthcare budgets are set too low. However, before arriving at such conclusions at least four issues need to be considered.

First, next to differences between estimates of v and k, a large variation within estimates measuring either k or v is also observed, as demonstrated by the ranges presented in the literature. Estimates of v have been found to range widely: from 1000 to more than 5,000,000 Euros [10, 31]. Similarly, estimates of k, estimated as marginal cost effectiveness of current spending, range between GBP ~ 12,000 and ~ 58,000 Euro. More recent overviews find a similarly large variation in estimates of both v [32] and k [33–35]. This variation highlights the substantial uncertainty surrounding the crucial estimates required to be able to judge the outcomes of economic evaluations or the size of the healthcare budgets. Some of the uncertainty may be inherent and for instance related to the availability of data. It, however, also reflects that, while the literature has placed much emphasis on obtaining precise estimates of ICERs, it has not placed a similar emphasis on estimating the thresholds required to evaluate these ICERs with. In relation to Eqs. (1) and (2), one might say that it has placed more emphasis on the left-hand-side of the equation than on the right-hand side.

Second, individuals and decision makers are not always consistent in their stated and revealed valuations of health within the healthcare sector. For instance, more conventional estimates of v and k in the Netherlands would suggest them to be around (up to) 80,000 Euros and 40,000 Euros, respectively [27]. The implicit valuation of health gains through actual choices within the Dutch healthcare sector, however, is at times inconsistent with these values. As an example, enzyme replacement therapy for Fabry disease is funded in the Netherlands although its ICER (compared to standard care) was estimated to be as high as €3,282,252 per QALY [36]. If this choice is interpreted to reflect the revealed willingness to pay for QALYs (v) in the Dutch healthcare system, this would open the door to many more interventions (and require a much higher budget). If it would serve as an estimate of k, many current not reimbursed interventions could be selected to replace this therapy. That would likely receive little support though, even if it may result in more healthcare from the available budget. At the same time, cost-effective programmes (e.g. smoking cessation programmes with ICERs that range from dominant to 5200 Euros per QALY [37]) are sometimes not funded. Do such decisions reveal societal preferences and valuations of health gains in particular circumstances? Should they not be seen as reflecting (only) the value of health per se but also, for example, the rarity of the disease or the budget impact? Or are these decisions (partly) driven by considerations that are hard to reconcile with the CEA framework, such a political motives and emotive media pressure?

Third, there are differences between how health is valued in the healthcare sector and other sectors. Healthcare is not the only place where investments in health are made or where individuals or decision makers must (implicitly) value health. Investments in the prevention of traffic incidents, environmental policies, setting wages for hazardous occupations and legal rules for the compensation of injuries all require a monetary valuation (or threshold) of health. These valuations can differ widely: Tengs and colleagues discuss more than 500 studies reporting the cost effectiveness of a wide variety of adopted policy interventions in different sectors [38]. The majority of ICERs were between 10,000 and 100,000,000 Euros per QALY following a broad range of methods, sectors and policy interventions. Such figures may suggest a lack of consensus on the value of health across sectors or that health is valued differently in different sectors, contexts and circumstances. These cross-sectoral differences makes the quest for unique and agreed-upon estimates of the value of health even more difficult.

Just as within the healthcare sector, differences in the conceptualisations of the value of health between sectors further complicate finding common ground. For instance, in transportation, the value of a statistical life-year is commonly used to express the value of health, often without taking (differences) in the quality of those life-years into account. Similarly, in the Dutch legal system, health damages are compensated with often relatively low amounts when compared to valuations of health in the healthcare sector [39]. This difference might make sense given the different purpose of the valuation: when compensating for health damages, the used values may not need to reflect a full valuation of health losses, as compensation for certain elements (like reduced income) are already arranged through social insurance.

Fourth, policy makers might take other considerations into account that lead to (legitimate) differences between v and k. A first consideration is that the funding of healthcare through collective mechanisms like taxation or (mandatory) insurance premiums is not costless. Such costs of public funding [40] are not reflected in typical estimates of v while they do affect the budget setting by the policy maker (and thus affect k). A second consideration is whose valuation counts in what context. As noted above, while v typically is estimated in studies in which individuals value their own health gains, collective decisions may go beyond that perspective. This may make the value a decision maker places on health in different contexts different. For instance, the empirical estimates of k differ substantially across different healthcare sectors and disease areas [11]. This might signal that policy makers place more value on health gains in particular contexts (e.g. more severe circumstances) than in others. Decision making may also not be equally well informed and executed consistently across sectors, which might also explain parts of the apparent variation in the valuation of health across sectors.

Lack of a Consistent Comparison of Costs to Value

Whether based on a healthcare or societal perspective, CEAs are not used systematically to compare the (health) value of interventions (or inputs) to their costs, both within the healthcare system and across sectors. First, CEAs (and their underlying assumptions) are best suited to appraise specific interventions, such as a new drug, and are less suited to appraise expenditures that are not explicitly tied to a specific intervention [41]. As a result, for other more general investments in new resources, such as increasing staffing levels, an explicit weighting of the expected health benefits against the associated costs is often lacking. An inconsistent use of CEAs can lead to an over- (or under-) investment in types of interventions or resources for which CEAs are not performed compared to those for which CEAs are performed. Although challenging, broadening the use of economic evaluations to all new healthcare investments is therefore an important goal [42].

Second, economic evaluations are typically not performed for existing care. This can lead to an (implicit) bias towards spending on existing care instead of on new interventions. As it is likely that not all current care is cost effective, there can be overspending on some types of existing care. At the same time, there might also be underuse of cost-effective current care.

Third, there generally is no consistent comparison of the cost effectiveness of interventions within and outside of the healthcare system. There are many policies that may strongly impact health but fall outside of the healthcare budget and are not initiated by the healthcare sector (e.g. education, certain types of prevention, lifestyle-changing interventions). These interventions may not be recognised in terms of their potential to contribute to health and welfare or may be (implicitly) evaluated against other thresholds, as different sectors may value health outputs differently, especially when health is not a primary policy objective. The risk is that this practice leads to a suboptimal allocation of resources for health, such as an overfunding of things that produce health within the healthcare sector and an underfunding of things that produce health elsewhere. Likewise, ignoring the impact of health interventions on other outcomes outside the healthcare sector when using a healthcare perspective leads to a reduced awareness of the important benefits increased health can have in other sectors.

Stylised Example

The framework indicated in Eq. (3) provides an opportunity to illustrate the relevance and impact of key assumptions with respect to the value of health underlying the decision rules. Consider two interventions, A and B, that both yield 1 QALY ($\mathrm{\Delta }Q=1)$. Both interventions result in 50,000 Euros of costs falling on the healthcare budget ($\mathrm{\Delta }{c}_h=50,000)$ and thus have the same cost effectiveness evaluated from a healthcare perspective.

However, from a societal perspective, intervention A is less cost effective than intervention B. This is the case because intervention B is associated with substantial productivity gains of 20,000 Euros ($\mathrm{\Delta }{c}_c=-20,000$) as it allows people treated for depression to return to work more quickly. Intervention A results in more societal costs outside the healthcare system as it is targeted at older retired individuals with heart failure. Intervention A prolongs their lives but because of that also increases non-medical consumption by 10,000 Euros ($\mathrm{\Delta }{c}_c=+10,000$).

Table 1 highlights the decisions that would be made under different scenarios (‘worlds’) on the value of $k$, using the generalised decision from Eq. (3), combining both perspectives. The consumption value of health $(v_Q$, measured using a societal willingness to pay for a QALY) is kept fixed in all scenarios at 50,000 Euros. This means that from a societal perspective, ignoring the budget constraint of the healthcare budget, intervention A is always rejected, and the decision maker is indifferent with regard to reimbursing B. For a decision maker using a societal perspective that does take the budget constraint into account, the reimbursement decision does depend on the value of k relative to $v_Q$.

Table 1.

Stylised examples applying $v_{\text{Q}}\left[\mathrm{\Delta }Q-\frac{1}{k}{\mathrm{\Delta }c}_{\text{h}}\right]-{\mathrm{\Delta }c}_{\text{c}}$ (Eq. 3) in various scenarios (worlds)

	Intervention A: intervention targeted at patients with heart failure	Intervention B: intervention targeted at patients with depression
ICER in the societal perspective	$\mathrm{\Delta }{c}_c=+10,000$ ICER = (50,000 + 10,000)/1 QALY = 60,000 per QALY	$\mathrm{\Delta }{c}_c=-20,000$ ICER = (50,000 − 20,000)/1 QALY = 30,000 per QALY
First best world:k = vQ  $v_Q$ = 50,000 (consumption value of health) k1 = 50,000 (least cost-effective care identified through a league table)	50,000 × [1 – 50,000/50,000] − 10,000 = −10,000 Reject intervention A (costs exceed benefits)	50,000 × [ 1 − 50,000/50,000] + 20,000 = + 20,000 Accept intervention B (benefits exceed costs)
Second best world: k > vQ: $v_Q$ = 50,000 (consumption value of health) k2 = 80,000 (least cost-effective care identified through a league table)	50,000 × [1 − 50,000/80,000] − 10,000 = 50,000 × [0.375] − 10,000 = 18,750 − 10,000 = 8750 Accept intervention A Efficiency gains by displacing less cost-effective care, which outweigh additional costs outside the healthcare sector	50,000 × [1 − 50,000/80,000] + 20,000 = 18,750 + 20,000 = 38,750 Accept intervention B Accepting this intervention increases efficiency of healthcare spending and yields gains outside of the healthcare sector
Second best world: k<vQ $v_Q$ = 50,000 (consumption value of health) k3 = 40,000 (least cost-effective care identified through a league table)	50,000 × [1 − 50,000/40,000] – 10,000 = 50,000 × [−0.25] – 10,000 = −12,500 − 10,000 = −22,500 Reject intervention A You displace more health than you gain and have more societal costs	50,000 × [1 − 50,000/40,000] + 20,000 = −12,500 + 20,000 = 7500 Accept intervention B The value of lost health is compensated by the gains outside the healthcare sector
Third best world: suboptimal displacement $v_Q$= 50,000 (consumption value of health) k4 = 40,000 (based on marginal cost effectiveness)	50,000 × [1 − 50,000/40,000] – 10,000 = −22,500 Reject Intervention A You displace more health than you gain and have more societal costs	50,000 × [1 − 50,000/40,000] + 20,000 = 7500 Accept intervention B The value of lost health is made up for by the gains outside the healthcare sector

The table shows the application of the general decision rule [Eq. (3)] for two interventions with different healthcare and societal costs in three different scenarios. In the first best world, the healthcare budget is set optimally, in the sense that the marginal cost effectiveness implied by the budget is equal to the societal value of care, so that k = v. In the second-best world, the healthcare budget is either set too high or too low compared with the societal value of health. In the third-best world, the healthcare budget deviates from the optimal, and moreover, the decision maker is not able to displace the least cost-effective care

ICER incremental cost-effectiveness ratio, QALY quality-adjusted life-year

First, we consider a ‘first-best’ world, in which the healthcare budget is set optimally and included care is efficient, so that k₁ (measured as the least cost-effective care currently funded) equals the consumption value of health $v_Q$, and both amount to 50,000 Euros per QALY gained. Under these assumptions, intervention A would not be funded, as it has a negative net societal benefit. B would be funded as it has a positive net societal benefit. (Using a healthcare perspective, the decision maker would be indifferent between A and B and between funding or rejecting both, as their ICERs are identical to k; funding these therefore means that just as much health is gained as displaced, so that there is no net health gain or loss.)

Next, we describe two ‘second-best’ worlds in which the decision maker also aims to optimise welfare but faces a healthcare system with a budget constraint, which corresponds to a k that is different from v. In one case, the healthcare budget is overfunded relative to the societal value of health: $k_2>v_Q$. In the other case, the budget is underfunded: $k_3<v_Q$. The decision maker has the mandate and ability to actively disinvest in cost-ineffective interventions that are currently funded from the budget. In the example where $k_2>v_Q$, both interventions A and B should be accepted, as adopting these interventions would displace less cost-effective care, thereby improving the efficiency of health spending and resulting in overall welfare gains. This also holds for intervention A because the additional health value produced by displacing less cost-effective healthcare outweighs the additional societal costs. (Using a healthcare perspective, both interventions would also be accepted as they both produce more health than gets displaced.)

In the second-best world where $k_3<v_Q$, intervention A is rejected as the new intervention is expected to displace more health than it generates, leading to a net loss in health next to the additional societal costs outside the healthcare sector. The k estimate the decision maker must use also suggests that funding intervention B would displace more health than it would gain. However, the societal savings generated by B compensate for that loss, so that the intervention would be funded under the societal perspective. (Using a healthcare perspective, both interventions would be rejected as they both produce less health than gets displaced.)

In a third-best world, the decision maker aims to optimise societal welfare, but has no mandate or information to make explicit disinvestment decisions and therefore must rely on a k estimate based on the average marginal productivity of healthcare. In the example, this k estimate is lower than the societal value of health: $k_4<v_Q$. As in the second-best world with $k_3<v_Q,$ intervention A is rejected and B is funded. However, the implications for optimal resource allocation are different between the second- and third-best world: where in the second-best world, all interventions currently funded have a cost-effectiveness ratio < k, this is not (necessarily) the case in the third-best world. It might be that less cost-effective interventions than intervention A are currently funded, but still A is rejected. (Using a healthcare perspective, both interventions would be rejected.)

This stylised example is only intended to illustrate the importance of the value of health estimates and their conceptual basis as well as the mandate of and information available to a decision maker. These aspects all are important in determining whether decision makers (can) act in accordance with theory.

Discussion

To ensure optimal spending of scarce resources in healthcare, public and private payers increasingly seek evidence of the added value of medical interventions before deciding on reimbursement. A CEA can provide this type of information by comparing the incremental benefits of an intervention to its incremental costs, relative to a relevant comparator, resulting in ICERs. To inform decisions, ICERs need to be evaluated against a relevant threshold. This threshold represents the maximum costs per QALY that would still be acceptable. The use of a CEA has taken different forms in practice, leading to different interpretations and estimates of the threshold. The topic of defining and estimating a relevant threshold so far remains understudied. In this paper, we focused on several conceptual and empirical issues that are crucial in bridging the gap between the use of thresholds in theory and in practice. We illustrated the importance of these issues using a stylised example.

We emphasise several implications for researchers and decision makers. First, using a two-perspective approach in economic evaluations, incorporating both the healthcare and societal perspective may result in better decision making. While studies of both k and v produce widely varying estimates, estimates of k tend to be lower than those of v. Taken at face value, this discrepancy would suggest that healthcare budgets are set too low compared to the societal value of health, although a direct comparison between values of v and k is more complicated in practice. Given the differences between v and k, estimates of both are required to come to optimal decisions. Using one single estimate of the value of health in economic evaluations does not ensure the improvement of population health or societal welfare. Only using a v threshold and taking a societal perspective may lead to misrepresenting the opportunity costs within the healthcare sector. Only using a k threshold ignores potential discrepancies between the societal value of health and the size of the healthcare budget and complicates the inclusion of societal costs and benefits outside the scope of the budget.

Second, if we want to come to more informative estimates and comparisons of v and k, more research is needed. Researchers need to further conceptualise what it exactly is that they wish to estimate, and what definitions of v and k are most relevant for the decision context economic evaluations are trying to inform. For instance, should v reflect individual valuations of individuals’ own health or a more general willingness to contribute to a collectively financed healthcare system including equity considerations? Should k reflect the least cost-effective option still funded or the marginal cost effectiveness of additional spending? And how to deal with differences in estimates of k between healthcare sectors or interventions? Are they signals of inefficiency, or of ‘equity weights in practice’? It would be good practice for guidelines in economic evaluations to make clear statements on the position of the decision maker with respect to these questions.

Third, researchers should aim for a better understanding of the way allocation and displacement decisions are made in practice. Decision makers at the national level influence the discrepancy between v and k directly by setting the healthcare budget, but also implicitly by the way they organise and delegate decision-making procedures. At the local level, delegated decision makers may be unable to make first-best decisions because they cannot displace or invest in care or health interventions outside of their own mandate. A better understanding of the decision-making process is therefore needed. This includes for instance equity considerations, the influence of dynamic effects (e.g. the budget may be fixed per period but not over time) and differences in political/bargaining power of different sectors. Specific attention is needed on the manner in which displacement decisions are made across different levels within the healthcare system. Systematic disinvestment research is not conducted as frequently as the assessment of new technologies. While this may be explained, for example by clear stakeholder incentives, this is a missed opportunity. The assumptions that current care is optimally allocated and that the least cost-effective care is displaced when new technologies are introduced are crucial for the marginal decision framework used in a CEA to lead to optimal outcomes. In practice, there is lack of a critical evaluation of the cost effectiveness of currently reimbursed care, cost-effectiveness analyses are not systematically applied across healthcare inputs, and it is unclear what care is actually displaced. These issues can lead to a failure to adopt welfare-improving technology and displace non-welfare-improving technologies.

There are, of course, many discrepancies between theory and practice related to the use of a threshold in economic evaluations of healthcare interventions that we did not discuss here. One important aspect is the strategic use of CEAs by both producers of medical technologies and policy makers [43]. In practice, a threshold can serve as a signal of an acceptable price and producers can use this strategically to price their products up to this threshold. Guidelines for CEAs provide little help to decision makers in dealing with strategic price setting, as a crucial assumption in the models behind CEAs is that costs are set exogenously [43], and thus no distinction is made between price demanded or set by the producer and the cost price of the product [44]. More attention for ‘pricing below the threshold’, and therefore a fair division of surplus between the producer and society, is urgently required.

We also did not discuss the role of healthcare inputs in delivering cost-effective healthcare. In theory, it is assumed that healthcare inputs (e.g. workforce, medicines, equipment) are used optimally and not constrained in any way. However, in practice, this is not the case and a shortage in healthcare workers has become a problem in many countries [45]. Such a constraint usually is an indicator of a market failure that is difficult to address. Here, similar to the pricing of new technologies, the wages of healthcare workers might not reflect true opportunity costs, which complicates the use of cost effectiveness and may seriously hamper the cost-effective delivery of many healthcare interventions [46].

Conclusions

We conclude that applying CEA decision rules currently does not ensure optimal decision making, given that the key assumptions of theory regarding the value of health underlying the decision do not always apply in practice. The absence of clear definitions and sound estimates of v and k implies a lack of understanding of the true opportunity costs of adopting new healthcare technologies. This suggests a clear research agenda. The same scrutiny that is used to look at outcomes of economic evaluations should also be used in defining, estimating and evaluating relevant thresholds to be used in decision making. Producing ever more precise ICERs has little use if we do not know how to interpret them. At the same time, it is important to bridge the gap between those attempting to estimate the value of health or the opportunity costs of healthcare spending and those using the results of cost-effectiveness studies in practice. Researchers and policy makers both need to be aware of the uncertainties regarding the interpretation of ICERs. Jointly, they should try to step up to the challenges described in this paper and start bridging the gap between theory and practice by improving both.

Declarations

Funding

This study was sponsored by AstraZeneca and Amgen through an unrestricted research grant. The grant agreement guaranteed academic independence and the right to publish research results without interference of the sponsor.

Conflict of interest

Bram Wouterse, Pieter van Baal, Matthijs Versteegh and Werner Brouwer have no conflicts of interest that are directly relevant to the content of this article.

Ethics approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Availability of data and material

Not applicable.

Code availability

Not applicable.

Author contributions

All authors meet the International Committee of Medical Journal Editors (ICMJE) criteria for authorship for this article, take responsibility for the integrity of the work as a whole, and have given their approval for this version to be published. All authors were involved in the design of the paper, the writing and the editing.