Embracing the science of value in health

KEY POINTS

• Resource stewardship is recognized as an important goal within our health systems.

• Cost-effectiveness analysis is an important tool supporting wise use of resources, but its application has been limited, in part, by the perception that it is unscientific.

• In recent years, methodologic advances in evidence synthesis, use of population-level administrative data, preference measurement, validation and calibration have strengthened the scientific foundation of cost-effectiveness analysis.

• Cost-effectiveness analysis should, as the science of value in health, be more widely adopted in health decision-making — not only in decisions around reimbursement but also in clinical guideline development and public health decision-making.

“Resource stewardship” is recognized as a key social value in Canada’s health system. This is enshrined in both legislation¹ and a variety of reform initiatives.^2,3 Canadians want a health system that meets their needs, is informed by evidence and fair, but also one that uses Canadian resources wisely. Cost-effectiveness analysis is one way to think rationally about resource allocation. Cost-effectiveness analysis is now used widely in both low- and high-income countries to inform decisions about drugs, vaccines, medical devices and health programs. In Canada, the Canadian Agency for Drugs and Technologies in Health (CADTH) Common Drug Review and the Ontario Health Technology Advisory Committee are examples of groups that use this analysis for decisions about coverage and reimbursement.

Most clinical practice guideline and advisory groups, though, do not use cost-effectiveness analysis. Some consider these analyses to be low-quality evidence.⁴ High-impact general medical journals tend to publish few economic analyses. One reason, perhaps, is that cost-effectiveness is not considered to be reliable or scientific by journal editors and guideline developers trained in epidemiologic methods. Randomized controlled trials are often considered to be the highest form of evidence, because the findings of a well-conducted trial are assumed to be reliable and tell us something “true” about the world. Cost-effectiveness analysis, on the other hand, has a lingering reputation of being an unscientific technique for integrating data of varying quality and dubious provenance, having once been described as building “aerial palaces of decision analysis” amidst the “haze of Bayes.”⁵ But is this still justified?

Canada spends more than one-tenth of its national wealth on health care and the United States spends nearly a fifth.⁶ Advances in science and technology, an aging population, third-party payment and increasing medical specialization are driving costs upward. We need a way to make rational decisions about costs in relation to benefits to patients and society. We argue that cost-effectiveness analysis is in fact scientific. It is the science of value in health, a valuable tool for integrating evidence about resource use into health decision-making. We discuss how cost-effectiveness analysis can inform decisions in a world where patients demand access to promising drugs, physicians demand to use the latest technology, the private sector demands support for innovation, and even the public sector hopes to use health spending to drive economic growth.

Cost-effectiveness analysis: scientific origins and purpose

Cost-effectiveness analysis represents the confluence of several streams of inquiry: economic theory, engineering, operations research⁷ and, arguably, epidemiology. Because each of these is either a type of social science (e.g., economics) or applied science (e.g., engineering, operations research), cost-effectiveness may also be thought of as a hybrid applied science, by which we mean “the use of scientific processes and knowledge as the means to achieve a particular practical or useful result.”⁸

Such analyses typically integrate health data of different types: epidemiologic data for natural history of disease, data from trials or observational studies for effectiveness, preference (utility) data for health outcomes and cost data from various sources. They may employ one of several types of decision model, such as decision trees, state-transition models, compartment- or agent-based models, or may use primary data from administrative databases or trials to represent complex biological or systems phenomena. Data are combined in a common analytic framework, and a summary statistic such as the incremental cost-effectiveness ratio, or net health benefit, is generated. These are measures of efficiency that represent the relation between additional resource use and additional health benefit. The statistic can be interpreted as a tool for maximizing health (an extra-welfarist perspective), improving overall societal well-being (a welfarist perspective) or simply providing support to decision-makers (a social decision-making perspective).⁹

Do we need a science of value?

Consider the following example of an organization deliberately avoiding the question of the value of cancer treatments. In 2010, the Cameron government in the United Kingdom established the Cancer Drugs Fund to provide access to cancer drugs that were not available through the National Health Service. In the midst of intense political pressure from patients with cancer, the government proposed to expand access to cancer drugs, most of which had been rejected by the National Institute for Health and Care Excellence (www.nice.org.uk) because these drugs did not meeting cost-effectiveness criteria. The initial budget of 50 million pounds grew rapidly to 340 million pounds. Following a 3-month evaluation period in 2016, the Cancer Drugs Fund’s mandate was changed. It became closely aligned with the National Institute for Health and Care Excellence, and drugs that had previously been rejected by the institute were no longer funded.

A 2017 review found that there was no evidence that the Cancer Drugs Fund delivered meaningful value to patients of the National Health Service.¹⁰ The fund spent 1.2 billion pounds, with estimated potential gains of 3500 quality-adjusted life-years (QALYs) offset by the potential loss of 18 000 QALYs (600–900 lives) associated with potential alternative uses of the money.

What are some common criticisms of cost-effectiveness analysis?

It is a black box

In addition to the construction of a model, the selection of data and analytic method are subject to the analyst’s judgment. The complexity and potential variability of this process has led to concerns that cost-effectiveness modelling is a “black box,” with results that nonexperts are expected to accept on faith.¹¹ This relates to concerns about the potential for bias, particularly when the analysis is conducted by or for a stakeholder with a strong financial incentive.¹²

Garbage in, garbage out

The early days of meta-analysis were characterized by anguished hand-wringing about the legitimacy of combining “apples and oranges.”¹³ By comparison, cost-effectiveness also combines grapes, bananas, kiwis and the odd durian.

The ethical underpinnings are dubious

The QALY, which is the standardized unit of health most commonly used in cost-effectiveness analysis, assigns every year of full health in every person the same weight. Usually, although not invariably, the analysis places an equal weight on health gains in those who are sick, healthy, poor and wealthy.¹⁴ These ethical implications are not universally accepted.

Is cost-effectiveness becoming more robust?

Yes, we think it is.

In 2016, the US Second Panel on Cost Effectiveness in Health and Medicine published its update of the 1996 guidelines for cost-effectiveness analysis.¹⁵ In 2017, CADTH published the 4th version of the Canadian guidelines for the economic evaluation of health technologies.¹⁶ These were signal events in the half-century history of the discipline and reflect the growing scientific consensus around best methods.

Reference case

Perhaps the most important advance has been the idea of the reference case. In the early years, published cost-effectiveness analyses used different perspectives, sources of preferences, discount rates and time horizons. The idea of the reference case is that analyses should be conducted with similar methods to facilitate consistency and allow comparisons. Canadian reference case analyses should be conducted from the perspective of the public payer, use a lifetime time horizon in the presence of mortality benefit, use community-weighted preferences, and discount future costs and health effects at 1.5% per annum. Use of the reference case enhances the value of the analysis through consistency, even if the consensus on methods is not complete.

Better empirical data

Advanced methods for evidence synthesis (e.g., network meta-analysis) are being used increasingly to estimate relative effectiveness in cost-effectiveness analyses. Large-scale, population-derived administrative data are also being used increasingly to estimate patterns of care, adverse effects and longitudinal costs.¹⁷ Because most provinces now have records of every interaction with the health system, it is possible to estimate the actual costs borne by these health systems. In addition, increasing international experience with community-weighted–preference sets has increased the reliability of preference data.¹⁸

A standardized unit of health

Broad consensus around the use of health-adjusted life-years (e.g., the QALY) as measures of health for cost-effectiveness analysis might equal the adoption of the reference case in importance. The value of having standardized units of health cannot be overstated. Being able to say what is large and what is small across every disease, intervention and setting is an extraordinary advance.

Analytical methods

Analytical methods have seen rapid development over the past 30 years. Groups of researchers have been funded for multi-institution, multiyear efforts to refine and validate policy models (or whole-disease models) that can be used for multiple applications within a single disease. The Cancer Intervention and Surveillance Modelling Network collaboration and the Mount Hood Challenge are 2 such efforts.^19,20 Sophisticated methods of handling heterogeneity and uncertainty regarding estimates of epidemiologic, cost and utility have been developed. Models are now routinely validated against external data, and methods for calibration for unknown model parameters continue to be developed.

Empirical estimation of the cost-effectiveness threshold

Finally, researchers in the UK, Europe and Australia have provided empirical estimates of the cost-effectiveness threshold by estimating the relation between changes in health expenditure and changes in health.²¹

Challenges for the future

Improving model sharing, enhancing transparency and other challenges to methods of analysis remain.²² Perhaps the greatest need is in the area of application. Is cost-effectiveness analysis useful for micro (individual patients), meso (classes of patients) or macro (policy level) decision-making?

Our view is that cost-effectiveness analysis is likely to be most useful at meso and macro levels of decision-making and less useful at the individual level. At the meso level, clinical practice guidelines in many jurisdictions do not include considerations of cost-effectiveness at present. The guideline movement is slowly evolving in the direction of including broader classes of evidence, ^23,24 but more progress is needed. At the macro level, a particular need is within public health. Vaccines and other public health interventions in many jurisdictions are often not evaluated for cost-effectiveness.

A second question is: How should cost-effectiveness analysis be integrated with other factors in decision-making? It is tempting for many health economists to think about cost-effectiveness as the normative solution for decision-making in health.²⁵ In our view, cost-effectiveness analysis cannot play this role. Cost-effectiveness does not fully represent the uncertainties inherent in estimates of the benefits and risks of treatment. Equity considerations related to the distribution of health gains, and broader questions of social value, autonomy, dignity, preferences for treatments and processes, legal, religious and cultural considerations, feasibility and institutional priorities are difficult, if not impossible, to incorporate. In Canadian decision-making contexts, these broader questions usually are, and in our view should be, addressed by other methods.^26,27

The scientific (and policy) challenge for the near future will be in broadening the reach of cost-effectiveness analysis across countries and settings, and integrating the science of value into health decisions in a way that respects not only system-level fiscal challenges but also the full range of factors that are relevant to health decisions: clinical evidence, patient preferences, social values and system feasibility.

Conclusion

Cost-effectiveness analysis has matured to the point where it should routinely inform reimbursement, coverage and disinvestment decisions, in both clinical and public health settings. Canadians should expect guidelines for clinical practice and public health policy to be routinely informed by cost-effectiveness analyses.