Determining Value in Health Technology Assessment: Stay the Course or Tack Away?

Abstract

The economic evaluation of new health technologies to assess whether the value of the expected health benefits warrants the proposed additional costs has become an essential step in making novel interventions available to patients. This assessment of value is problematic because there exists no natural means to measure it. One approach is to assume that society wishes to maximize aggregate health, measured in terms of quality-adjusted life-years (QALYs). Commonly, a single ‘cost-effectiveness’ threshold is used to gauge whether the intervention is sufficiently efficient in doing so. This approach has come under fire for failing to account for societal values that favor treating more severe illness and ensuring equal access to resources, regardless of pre-existing conditions or capacity to benefit. Alternatives involving expansion of the measure of benefit or adjusting the threshold have been proposed and some have advocated tacking away from the cost per QALY entirely to implement therapeutic area-specific efficiency frontiers, multicriteria decision analysis or other approaches that keep the dimensions of benefit distinct and value them separately. In this paper, each of these alternative courses is considered, based on the experiences of the authors, with a view to clarifying their implications.

Key Points for Decision Makers

Valuing new health technologies is difficult because choices must be made regarding which effects to consider and how to reflect their value in monetary terms.

A commonly implemented approach involves putting a price to a quality-adjusted life-year. This may be considered in light of other factors, such as society may give priority to more severe illnesses and ensuring fair access to resources, but the processes for doing so are neither systematic nor transparent.

Approaches to address broader societal values remain rudimentary but deserve significant efforts to operationalize them.

Introduction

In most countries, clinicians and patients do not have unfettered access to all interventions that can help prevent, diagnose, or treat illness. Even if an intervention is effective and reasonably safe, a decision must be made on whether to pay for it. If the outlays required are too high, its use may be restricted or funding withheld [1]. Although these decisions are taken routinely, they are difficult because assessing what is ‘too high’ involves determining what is reasonable to pay for a given benefit, and this requires putting a price on health gains and other intangibles for which there is no clear monetary valuation, as well as making comparisons across individuals. Ultimately, the value of health effects has to do with what one gives up to obtain them, but that ‘opportunity’ cost is rarely directly evident.

Where the trade-off can be gauged (for example, a hospital), it is feasible to approach the problem directly without explicitly valuing the consequences (e.g. to pay for this intervention we must close a bed) [2]. By contrast, determining value at the healthcare system level is extremely challenging because it is difficult to identify what will be displaced. This is even less clear in systems without a fixed budget, or where insurance premiums might be increased.

A common approach involves estimating the outlays and gains, and assigning them value without invoking opportunity cost directly. Since most outlays are measured in monetary units, their valuation is straightforward, but health gains are not naturally assessed this way. This poses a problem for weighing consequences measured in different clinical units. A common approach is to convert them into duration of life, modified by its quality, or quality-adjusted life-years (QALYs) [3], and then apply a price per QALY gained (i.e. ‘cost-effectiveness threshold’). If this gain exceeds the outlay [4], then the intervention is deemed worthwhile [5]. This determination is made without identifying what to displace or assessing the opportunity costs, although there have been recent attempts to do so [6–8].

The objective of our work was to produce a white paper to help further deliberations on determining value in the UK and elsewhere. The first author selected the other six experts to represent a wide range of viewpoints. The seven met in London, UK, in 2016, where each participant was asked to address: (1) What should health technology assessment (HTA) achieve? (2) What should be the approach to value-based pricing? Based on these deliberations, a draft paper was circulated for comment. Further telephone discussions led to additional versions, and four of the authors participated in an Issues Panel at an International Society for Pharmacoeconomics and Outcomes Research (ISPOR) meeting in Glasgow. There was substantial disagreement but no pressure to reach consensus. Instead, in this paper, we examine these challenging aspects to consider possible improvements to value determination in HTA. We begin by contemplating the aims of healthcare and then set out the requirements for value assessment. An examination of the options follows, ranging from staying the course1 with evidence-based reinforcement, through extending the cost per QALY approach to address some limitations, to abandoning it altogether and tacking to an alternative. Based on this examination, we make some recommendations for consideration by healthcare systems when reforming their HTA processes.

The Societal Aims for the Healthcare System

A healthcare system funded by citizens’ contributions must aim to meet the needs that arise due illness. It should strive to restore and maintain health, providing palliation when necessary. As the needs are vast and ways of meeting them ever-expanding, no system can fully achieve these goals, implying choices when deploying resources and the additional aim of ensuring these are consistent with society’s objectives. Various theories have emerged as to what these are.

One theory is that society wishes to maximize the aggregate health of the population [3], with the aggregate defined as the sum of health across all citizens, regardless of who bears what illness or benefit. An alternative view is that fairness matters [9] and that assuming a QALY is a QALY regardless of who receives it is not entirely fair because other aspects, such as severity of illness, are relevant [10]. A key criterion is that everyone should have equal access to necessary care [11], including not prioritizing those with greater capacity to benefit, either because more can be done for them or because they were healthier beforehand [12]. This means that it is acceptable to commit resources to manage an illness that is given higher societal priority. This alternative view is supported by studies indicating that citizens are concerned not only with their own treatment but also that of others [13]. This appeal to communitarian values [14] implies sharing scarce resources even if this reduces aggregate population health gain because more QALYs could have been obtained via some other use and is supported by empirical research [15, 16]. However, regardless of the criteria adopted, some interpersonal comparison is involved, even if implicitly.

If societal healthcare objectives expand beyond health gain, any number of other aspects might be considered, such as well-being, burden of illness, degree of unmet need, improved productivity, impact on caregivers, improved organizational efficiency, public dislike for all-or-nothing decisions that may discriminate (for example, against those with orphan diseases), maintenance of hope, and even the intervention’s innovativeness [15, 17, 18]. Pursuit of such objectives is not necessarily inconsistent with evaluation of cost effectiveness based on thresholds, but they augment or even constrain the cost-effective outcomes to be delivered.

Stay the Course—‘Efficiency First’, as Exemplified by National Institute for Health and Care Excellence (NICE) in the UK

The common approach, benefitting from extensive experience, judges health gains in terms of QALYs against a predetermined cost per QALY threshold, taken to represent the opportunity cost [19] or willingness-to-pay for new interventions [20]. The QALY, proposed as a health status index to aid clinical decisions involving trade-offs between duration and quality of life [21], is not itself a measure of value. Tariffs based on trade-offs among health states have to be applied but these only represent preference valuations if exceptional and, to many, unreasonable assumptions hold [22]. These tariffs consider the gain as having a linear value: zero QALYs equals no value, one QALY equals one unit of value, two QALYs equal double the value, etc. Health gains can only be considered if they can be measured in QALYs and the nature of the gain, whether in duration or quality of life, is not considered. No other factors are accorded value.

Recently, concerted efforts have been made to provide an evidence base for the threshold [6–8]. An average was computed in Australia and Spain using national mortality data and quality-of-life data from surveys. In the UK study, health authority data on cost increases and mortality reductions from one year to the next were used to compute a cost per QALY threshold in each of 23 broadly defined therapeutic areas (in some it could not be calculated), with substantial variation across resulting estimates (Fig. 1). Although the original theory implies that the threshold is the highest cost-effectiveness ratio [23], the researchers instead took the mean across areas, weighting more efficient practice areas higher. A recent systematic review of 120 empirical economic studies, reporting estimates of the value of a statistical life-year using a variety of approaches, found the median to be nearly US $200,000 among European studies, and even higher among American studies [24, 25].

Fig. 1.

Empirical estimates of cost effectiveness relative to total cost, both axes on a logarithmic scale. The number beside each label is the total cost effectiveness, in thousands. Note, three categories—Social Care, Trauma, and Other—plotted on the horizontal axis have unknown cost effectiveness and only total cost. GBP Great British Pound, GI gastrointestinal, GU genitourinary, ID infectious disease, MSK musculoskeletal, UNK unknown, QALY quality-adjusted life-year

Broadening the Scope of the Cost per Quality-Adjusted Life-Year (QALY)

One possibility for broadening the cost per QALY approach is to incorporate other domains, such as the impact on relationships, autonomy and meaning [26], for example in a measure termed a WELBY (see Brazier and Tsuchiya [27]). Another idea is to consider that efficiency relative to the threshold is but one input into determining an intervention’s value [28], and other inputs can be incorporated by deliberation [29]. The methods for these deliberations are not well codified, including how to resolve disputes and how to weigh factors. Although most HTA agencies allow for discussion, it has been recognized that the numerical nature of the cost-effectiveness ratio and its clear juxtaposition to a threshold tends to hold sway over more qualitative aspects [30].

Adjusting the threshold [31] to take severity of illness into account is used with ‘end-of-life’ interventions in the UK [32]. This tactic can be extended to consider the ‘burden of illness’, as is done in Sweden [33], that is the degree to which an illness decreases QALYs [34] and applying different threshold prices accordingly. Indeed, an even higher threshold has been advocated for the treatment of rare diseases [35]. Apart from difficulties forecasting the QALYs lost [36, 37], it is unresolved if burden should be considered in proportional or absolute terms (thus favouring those with longer life expectancy or better quality of life beforehand), and how to take this into account in valuing an intervention.

Tacking Away from the Cost per QALY

The adjusted threshold approach could be further relaxed by eliminating the imperative to measure benefit in terms of QALYs. If separate thresholds are contemplated for different contexts, then this can be extended to each therapeutic area where the clinical measures of benefit are well established, and can include whatever aspects are relevant, even if they do not map well onto either duration or quality of life. The implied willingness to pay for benefits in a therapeutic area can be established using the ‘efficiency frontier,’ as is done in Germany [38]. However, if an intervention is the first in a therapeutic area, there is no established price, requiring use of a frontier from a similar area as a gauge.

Multicriteria decision analysis (MCDA) [39] provides for addressing a range of effects against a criteria set, which can include duration and quality of life, but also extend to whatever other aspects are relevant, scored and aggregated by quantitatively weighting their importance. For a choice among interventions, MCDA provides the necessary ranking without requiring further valuation, but it is unclear how this applies when deciding whether to fund a particular intervention. The MCDA score by itself does not address this, nor does it consider opportunity costs, and it presents its own technical difficulties [40].

Rather than aggregating multiple dimensions into a single score, the components can be kept distinct and each one valued separately [41]. Gradients of value (Fig. 2) are proposed to accord with the premise that value increases with severity, but there is diminishing marginal value as health is restored (also noted empirically in the study by Rowen et al. [41]). Different gradients are applied to each domain and other criteria affecting them can be incorporated. One point is priced and this used as an anchor. The maximum reimbursable price for any intervention is established by summing the monetary values of each type of gain (under the strong assumption that they are independent).

Fig. 2.

Proposed relative value gradients according to quality-of-life improvement and initial severity

(data taken from Nord [48])

Many stated preference studies may have been misspecified because they were restricted to use value [42], which excludes option value (e.g. access in case of need; risk aversion) and altruistic motivations, which can be captured by choosing increments in private or public insurance premiums as the payment vehicle [43, 44]. These, or the Relative Social Willingness to Pay [45], imply a focus on social transfer payments and a greater role for the budgetary impact of healthcare programs [17, 46]. Fairness-based evaluation frameworks, sometimes collectively referred to as (social) ‘cost-value analysis’ [47], have not reached sufficient maturity to be recommended for formal implementation in HTA.

Discussion

This paper provides a summary of alternative approaches to assessing value to inform the selection of societally preferred healthcare options. Ultimately, everyone would like to see that funds expended on healthcare provide sufficient value to offset the opportunity costs. Nonetheless, positions diverge on what the healthcare system’s aims should be and these largely determine whether the preference is to stay the course or tack away. If maximizing aggregate health is the aim, then sticking with cost per QALY gained gauged against a threshold is appealing. Nevertheless, this implies society must be willing to tolerate the many known problems with the QALY [48–50]. However, there is growing dissatisfaction with the QALY’s constrained and somewhat dominating scope [51, 52] and thus with the inability of the cost per QALY approach to sufficiently capture the benefits valued by societies.

Several countries have already decided not to use a cost per QALY approach. It has been rejected in Germany [53] and Spain, and, despite increasing interest, remains largely unused in the United States [54]. In France, it is a minor component of HTA [55], and some Latin American countries have also shown reluctance to implement it [56]. These jurisdictions have opted largely for incorporating other aspects beyond clinical health in a deliberative approach, although the efficiency frontier and MCDA have both played an occasional role. While HTA in these healthcare systems functions, some will remain uncomfortable with such an approach, preferring to adjust the threshold. Indeed, relaxing the constraint on the threshold appears to be the direction of travel of even NICE in the UK [42]. An in-depth comparison of countries applying alternative approaches with respect to equity and efficiency, as well as transparency and legitimacy, should be high on the research agenda.

Although this tacks away from the foundation of maximizing aggregate health, it complicates providing an empirical basis for decision making. By how much should the threshold be adjusted and for which other benefits? Who makes these modifications and on what basis? What are the implications for considering opportunity cost? Tacking away requires answers to these questions.

Regardless of the manner by which the cost per QALY approach is extended—qualitative deliberations, quantitative adjustments, or both—why stick with the QALY if maximizing aggregate health is not the sole aim? One reason is convenience. Many societies have invested considerable efforts in implementing it, and awareness of the concept has already extended somewhat beyond the specialized confines of HTA, and the introduction of completely new concepts will face substantial hurdles. Another justification is that staying with the threshold provides a benchmark for moving towards other criteria for which explicit arguments have to be made. As with staying the course, a society that moves towards an extended QALY approach is still subject to its violations of the fundamental assumptions and the resulting uncertainties in determining a justifiable reimbursement price. If a single threshold is replaced with a sliding scale, the other criteria and justifications for supporting them have to be explicit.

Relying on an efficiency frontier is one way to move away from both the QALY and the single threshold. However, the efficiency frontier allows for ever-higher prices per unit of benefit as the absolute benefit grows (i.e. decreasing efficiency), with no inherent guidance on the minimum efficiency required. The efficiency frontier approach also assumes that established expenditures reflect societal priorities, but there is no reason to believe this is reliably so in the absence of explicit valuations across therapeutic areas.

MCDA is another way to extend HTA beyond the single dimension of the QALY, but this method has not been developed for recurring decisions at a societal level. The score that results has no inherent meaning, although it can be used in an efficiency frontier as an integrated measure of benefit, thus removing one difficulty with this frontier approach [57].

Separating the valuation of life-extension from quality-of-life improvements avoids difficult-to-sustain assumptions involving a single integrating index, but requires the untenable assumption that there are no interactions between the domains. Moreover, establishing the anchor point is challenging, particularly if the domains are not independently evaluable. This also raises the question of whether a universal quality-of-life index applies in disparate illnesses. Can an index such as the EQ-5D [58] be leveraged in this regard [59]? However, there is nothing preventing extension of the method to other dimensions. By scaling the relative value of gains in disparate domains and anchoring these to specific monetary amounts, the approach implies cross-domain relative values.

Establishing an acceptable price to pay for a given set of benefits is a problem that goes beyond the remit of the HTA into political and other spheres because, in the end, it has to do with how much of society’s resources are allocated to the healthcare budget and what priorities are set therein. Perhaps then, HTA should return to the foundation of opportunity cost, and, rather than appraise whether an intervention is ‘worth it’, provide instead an estimate of what would have to be given up (with all its consequences) or how much additional funding would be needed if nothing is displaced. This would provide the trade-off consequences explicitly, without the need for a cost-effectiveness threshold or even a QALY. The decision-making body thus stands to be swayed less by a single threshold-driven decision rule and can accord due weight to all relevant inputs. This deliberative decision making is arguably much more difficult, and, without explicit well-defined processes and decision rules, may be less transparent and consistent.

Conclusion

Given the substantial implications of these decisions to patients, families and society in general, it behoves our field to strive for an approach that addresses as fully as possible the healthcare system’s aims, is methodologically sound, and is feasible to implement within the available timelines and resource constraints. None of the alternatives examined yet achieve this.

Funding

A meeting in London and subsequent deliberations were supported by an unrestricted grant from Janssen Pharmaceutica. Two senior staff from Janssen were present as observers during the panel meeting; they have had access to the draft versions of this paper and have provided very helpful comments, but neither they nor the company they represent had any editorial control. The agreement with Janssen ensured the authors’ independence in study design, data interpretation, writing, and submitting the manuscript for publication. All decisions regarding the content of this paper and the decision to publish were made independently by the authors.

Conflict of interest

J Jaime Caro, John Brazier, Jonathan Karnon, Peter Kolominsky-Rabas, Alistair McGuire, Erik Nord and Michael Schlander declare no financial conflicts of interest. All authors are well known for their views on the subject matter discussed in this paper.

Overall Guarantor

J. Jaime Caro will serve as overall guarantor for this study and manuscript. All authors participated in the discussions that led to this paper and in the revisions of all drafts. All authors approved the final version submitted for publication.