Augmenting Cost-Effectiveness Analysis for Uncertainty: The Implications for Value Assessment—Rationale and Empirical Support

Garrison et al.’s Viewpoint

The recently published ISPOR Special Task Force (STF) Report on U.S. Value Assessment Frameworks made recommendations on criteria for coverage and reimbursement decisions. U.S. public and private payers should use the cost per quality-adjusted life-year (QALY) metric as a starting point. This measure should, however, be augmented by considering a number of potential novel elements of value, including several related to uncertainty—in particular, insurance value, real option value, the value of knowing, and the value of hope. With regard to this “augmented” cost-effectiveness analysis (ACEA), the report recommended that “more research is needed on how best to measure and include them in decision making.”¹

The ISPOR STF report defined value from a microeconomic perspective, with total value based on what an individual would be willing to pay considering all of its potential effects.² The following discussion takes this ex ante view of value, that is, based on what individuals would be willing to pay to have coverage of a new medicine in their health benefits package.

The STF report noted that health technology assessment (HTA) decision making for coverage and reimbursement needs to address 3 key questions: (1) what economic value elements to include? (2) how to measure, evidence, and value them? and (3) how to aggregate and judge them in order to reach a decision on value? The purpose of this review is to describe the several potentially significant uncertainty-related novel elements that could be included in an ACEA and discuss what has been or could be done to (a) measure them and (b) aggregate them to total value in an HTA.

Identifying Novel Elements Related to Uncertainty

Conventional Metrics and Measures

Conventional CEA focuses on 3 key elements: lifetime intervention cost minus cost-offsets, expected incremental life-years gained, with a morbidity adjustment to survival. The last 2 elements are usually combined into a measure of expected net health gain (QALY), and the 3 are used to generate the incremental cost-effectiveness ratio (ICER) of net cost to net health gain. This metric has been around for more than 40 years and is the gold standard for economic evaluation of heath technology interventions in many countries around the world,³ including Australia and Canada, which were the earliest adopters of the QALY-based approach. It is the primary metric used by the highly visible National Institute for Clinical and Care Excellence in the United Kingdom and the Institute for Clinical and Economic Review in the United States.^4,5 Over 8,500 such CEA studies are in the Tufts CEA registry.⁶

At the launch of a new medicine, there is uncertainty about the estimates of these conventional elements for any individual or, on average, in a population. The projection is, however, typically deterministic. The value assessment is based on the expected incremental health gains and cost offsets. In current analyses, uncertainty is most often considered in the context of whether further evidence should be sought in order to lower the chance of the decision maker making an incorrect decision. But individuals also face these cost and outcome distributions, and for risk-averse individuals, the variance and other shape parameters can affect their valuations.

As depicted in Figure 1, adapted from the ISPOR STF “value flower,” the novel elements can be separated into those that are related to uncertainty and those that are not. The latter notably includes equity and scientific spillovers. Severity of illness and fear of contagion may interact with the other uncertainty-related elements, but we focus on 4 elements—insurance value, value of hope, real option value, and the value of knowing—with some empirical support.^7-9

FIGURE 1.

Potential Elements of Value

Insurance Value: Financial

The most obvious novel element of value is that insurance coverage not only covers some of the direct medical costs of health care interventions, it also provides plan members with “peace of mind” through knowing that they will be protected, to some extent, against large, potentially catastrophic financial outlays. This margin—the risk premium—above the expected cost supports competitive private insurance as a business enterprise. In a competitive market, the number of healthy people willing to pay for insurance coverage for a medical technology will exceed the number of patients who will eventually need it.

Insurance Value: Physical Health

Peace of mind can also come from reduced health risk through the existence of a medical technology that can cure or help in managing a disease that enrollees are at risk of getting.⁹ Insurance value represents the ex ante willingness to pay (WTP) of an at-risk plan beneficiary (i.e., before becoming a patient with a known disease) to decrease the physical risk of illness through medical treatment, as well as the financial risk derived from having to pay for such medical treatment. Physical health risk protection is less recognized and more novel. Lakdawalla et al. (2017) model the notion that a well-informed purchaser will be willing to pay more to include a technology in the health benefits package beyond the financial risk protection because of the uncertainty related to the potential unavoidable loss of QALYs.¹⁰ In the case of Alzheimer disease, for example, we are all worse off because there are no cures—or even a way to substantially reduce its severity. Given enough income, we can buy insurance to cover all of the medical costs for managing the disease. This is “financial” insurance—we will not go bankrupt if we get the disease. But we cannot buy insurance that will provide us with a highly effective treatment for the disease because it does not exist. This is “physical” health insurance, and it is not available at any price. Our desire to avoid the health consequences of the condition (our risk aversion to getting Alzheimer disease when there is no treatment) means we would be willing to pay more to reward an innovative intervention that could lessen the severity of the disease than is represented by the health gain alone. If reflected in a payer’s willingness to pay a higher price for such a drug, this would, in turn, provide greater incentive for its development. In contrast to Lakdawalla et al.,¹⁰ we would separate the measurement of the financial risk premium from the health risk premium on the grounds of potential risk independence between financial risks and risks to physical health.

Real Option Value

Real option value (ROV) in this context has been conceptualized by an analogy to financial options: a payment now to provide for the opportunity to do something in the future (e.g., to buy stock at a particular price). For example, undergoing a life-extending treatment today may open up opportunities for patients to benefit from more effective treatments that will arrive in the future. As a result, a plan member would be willing to pay more for a life-extending intervention in a disease area with a stronger development pipeline and therefore with a brighter future. Although the expectation is that accounting for technology advancement and ROV will lead to greater QALYs gained in the long term, current modeling methods do not incorporate this. Recognition of this channel of influence does appear, however, to affect real-world decisions made by providers and their patients.¹¹

Value of Knowing

Diagnostic tests can create value by providing prognostic and predictive information that is valuable for patients, since their providers can make better recommendations about prevention or treatment. Not only can this information improve clinical decision making, but also, some patients value “knowing for the sake of knowing” even it is not directly clinically actionable. Just by changing understanding on the probability of disease, either towards confirming or ruling out a disease, diagnostic information can reduce uncertainty for the patient, generating some value of knowing. In theory, the effect of diagnostic information on consumption and health risks is ambiguous and depends on the diagnostic accuracy. If a diagnostic can show that the probability of disease is either near zero or near 1, the dispersion of consumption and health decrease—this reduction in variance is valuable for risk-averse individuals. Nonetheless, the consideration of variations in preferences and risk aversion would imply that there is not a simple direct translation from the amount of dispersion to the valuation of risk for a population. Yet, in general, the value of knowing increases with the degree of a patient’s risk aversion by reducing uncertainty. In practice, however, few evaluations of interventions have tried to assess or measure this additional value. Furthermore, reimbursement systems have tended to undervalue this contribution.¹²

Value of Hope

The value of hope is an appealing label, but its meaning is not immediately clear. It has been used in the health economics literature very specifically to convey the idea that in certain situations, a rational individual would switch from the customary risk-averse stance to take gambles with the potential for a beneficial outcome (e.g., such as a cure). Suppose the expected QALY gain were identical for 2 alternative treatments, but pretty dismal—for example, an expected mean survival of 3 months. But suppose with 1 treatment, while 95% die within days of receiving the treatment, 5% can expect a 5-year life span. With the other treatment, everyone has a survival of 3 months. Discrete choice experiments suggest that people would be willing to pay more for the riskier treatment.¹³

Measuring Additional Elements: Overview of the Empirical Evidence

There is clearly interest in assessing the implications for the ICER if different elements of value are measured. But the number of studies is limited. Table 1 summarizes the latest relevant studies and estimates for 4 uncertainty-related novel elements of value.

TABLE 1.

Empirical Studies Relevant to the Measurement of Novel Uncertainty-Related Elements of Value

Element/Study	Context	Method	Monetary Effect Above Conventional ICER
Insurance value: financial risk protection
Verguet et al., 201314	Rotavirus-India (I) and Ethiopia (E)	Dynamic CEA modeling	Financial risk protection (FRP) of $16k (I) and $8K (E) per 1 million households. Largest FRP in lowest income quintile.
Verguet et al., 201515	Tuberculosis in India	Universal public finance model (90% coverage)	Per million people in India, insurance value is $9,000, and 80% would accrue to the bottom 2 quintiles.
Insurance value: financial and physical health risk protection
Shih et al., 201616	Multiple sclerosis in United States	Parameterized utility function	33% of conventional value
Lakdawalla et al., 201710	General U.S. population	Numerical exercise with a parameterized utility function	38%-62%: The physical insurance values greatly exceed the financial insurance value
Real option value
Sanchez et al., 201217	Small molecule medicine for chronic myeloid leukemia in United States	Projection of mortality trends	9% of conventional survival benefit
Thornton Snider et al., 201718	Monoclonal antibody medicine for renal cell carcinoma and lung cancer in United States	Projection of mortality trends	5%-18% of conventional survival benefit
Li et al., 201919	Monoclonal antibody medicine for metastatic melanoma in United States	Projection of mortality trends and new drug approvals and economic modeling	Incremental QALY gained increased by 5%-8% and ICER decreased by 0%-2%
Value of hope
Lakdawalla et al., 201213	Treatments for metastatic melanoma and metastatic breast cancer in United States	Discrete choice/contingent valuation	WTP $35,000 for a 1 standard deviation increase in survival
Shafrin et al., 201723	Treatments for advanced stage melanoma or lung cancer in United States	Patient and physician surveys	Majority of patients prefer higher variance in survival; physicians do not
Shafrin et al., 201824	Treatments for sqamous non-small cell lung cancer in Canada	Economic modeling estimation	0.039 additional QALYs (equivalent to Canadian $5,580)
Value of knowing
Neumann et al., 201225	Predictive testing for diseases with no preventive option in United States	Stated-preference study	$109-$263 per test
Goldman et al., 201326 (Sood et al., 2013, technical analysis)27	Dx testing in personalized medicine: RA patients at risk for CV event on an NSAID in United States	Population economic modeling	Test generates $1,284 per patient

CEA = cost-effectiveness analysis; CV=cardiovascular; Dx = diagnostic; ICER=incremental cost-effectiveness ratio; NSAID = nonsteroidal anti-inflammatory drug; QALY = quality-adjusted life-year; RA =rheumatoid arthritis; WTP = willingness to pay.

Insurance Value

Taking the Lakdawalla et al. model,¹⁰ the conventional value of a medical technology—based on the QALYs gained and cost offsets—can be interpreted as the net monetary benefit (NMB) in optimal conditions, that is, the adopted WTP threshold, reflecting social preferences for the value of a QALY increase. In this case, the addition of insurance value to conventional value represents about 38%-62% of the conventional NMB.¹⁰ This is mainly driven by the reduction in physical health risk, which goes beyond the reduction of financial risk. Health insurance in its usual form provides financial risk reduction, including the reduction of catastrophic household health spending. Further, this financial risk protection has important societal distributional consequences in countries where much care is currently purchased out of pocket. It has been used to assess the distributional consequences of global health programs in India and Ethiopia.^14,15

In a model similar to Lakdawalla et al.,¹⁰ Shih et al. (2016) estimate a 33% increase above the conventional value due to the introduction of different medical treatments for management of multiple sclerosis.¹⁶ This increase is, however, not only measured from an ex ante perspective (value for the healthy), as in the Lakdawalla et al. model, but also adds the (ex post) “value to the sick.” We would argue that when it comes to inclusion in the health benefits package, the appropriate perspective is the ex ante WTP as an incremental insurance amount.

Real Option Value

In the HTA literature, ROV has been measured as an increase in expected survival and QALY gains. Sanchez et al. (2012) projected the long-term survival trend of chronic myelogenous leukemia and estimated the ROV of imatinib to be about 9% of its conventional survival benefit.¹⁷ Using similar methodologies, Thorton Snider et al. (2017) estimated the ROV of nivolumab to be 18%, 5%, and 10% of its conventional survival benefit for renal cell carcinoma and squamous and nonsquamous non-small cell lung cancer, respectively.¹⁸ Li et al. (2019) projected new drug arrival (i.e., approvals) and long-term survival trends for metastatic melanoma and incorporated them into a cost-effectiveness model. In their analysis, the incremental QALY gains of ipilimumab increased by approximately 7% due to ROV.¹⁹

Several other studies used actual arrival dates rather than projected arrivals or survival and estimated the ex post ROV of the monotherapy zidovudine for human immunodeficiency virus/acquired immunodeficiency syndrome, ipilimumab for metastatic melanoma, and tamoxifen for breast cancer, respectively.^20-22 These ex post estimates of ROV tend to be larger than the ex ante estimates because new drug arrivals in these studies were treated as known events rather than uncertain events with some probability of occurring. However, as we previously argued, the ex post estimates are much less relevant for decisions about benefit plan inclusion when new medicines are first included.

Value of Hope

The inclusion of value of hope as a factor that increases NMB requires the previous assumption of a change in risk attitudes at the end of life. This risk-taking behavior by end-of-life patients has been elicited through WTP or by comparing patient and physician valuations. In a survey of cancer patients with melanoma, breast cancer, and other kinds of solid tumors, 77% of respondents preferred a hopeful gamble to a sure bet, even if the 2 provided the same expected survival.¹³ In that study, patients’ average WTP for a hopeful therapy was $54,362, implying $36,305 for a 1-year increase in the standard deviation of survival. In another survey of melanoma and lung cancer patients and physicians, over 60% of patients preferred the hopeful gamble, compared with 29%-41% of physicians.²³ Patients were willing to give up 11-14 months of mean survival in exchange for the hopeful therapy, whereas physicians would prescribe the hopeful therapy to patients only when the fixed survival therapy had 1-8 months shorter survival than the hopeful therapy.

These findings suggest that terminally ill patients are risk loving, while their physicians are risk neutral to risk averse. In a case study of nivolumab, Shafrin et al. (2018) constructed a utility function for squamous non-small cell lung cancer patients using published risk preference estimates and estimated the value of hope as the difference between the expected survival and the certainty equivalent.²⁴ In that analysis, accounting for the value of hope resulted in 0.039 additional QALYs (equivalent to Canadian $5,850) in addition to the 0.66 QALYs gained when not considering patients’ risk preferences. The analysis also generated derived estimates of an insurance value of 0.56 additional QALYs (equivalent to Canadian $84,179), based on Shih et al.,¹⁶ and of a real option value of 0.037 additional QALYs (equivalent to Canadian $5,599). based on Thornton Snider et al.¹⁸

Value of Knowing

To our knowledge, value of knowing has not been included in the calculation of either NMB or ICER in the economic evaluation of a diagnostic test to select a treatment. Value of knowing has only been measured separately by eliciting WTP for undertaking a diagnostic test for 4 different diseases or as the diagnostic value derived from a better stratification of respondents to treatment incorporating health factors such as tolerance to side effects.^25-27 Median WTP for undertaking the diagnostic test for different diseases varied between $109 for the imperfect arthritis test to $263 for the perfect prostate cancer test. That is, the value of knowing for the patient increased with accuracy of the diagnosis. Also, the willingness to take up the test was larger for patients at greater risk of disease and for diseases with available treatment (e.g., larger uptake for arthritis and prostate cancer than for Alzheimer disease). When studying the value of knowing it is important to separate the expected health gain to the patient from the additional value of knowing. However, it is also important to note that many individuals value knowing whether or not they are at high risk for a condition even if there is currently no treatment for it.²⁵

Affordability Issues

In their response to our review, Watkins and Tsiao (2020; see this issue) raise thoughtful and valid concerns about the prices of new branded medicines and are concerned that our review somehow seeks to justify these prices and fails to address the key issue of affordability.²⁸ It is worth noting that we agree with them that the cost-per-QALY metric should be the starting point for any assessment of value. This is a core recommendation of the ISPOR STF report but is not a widely accepted view in the United States. Along with Watkins and Tsaio, we also agree that the QALY “does not capture the total value that may be gained from a medical intervention.”²⁸ So we find ourselves debating with them the details of what else should be considered and about how we should address affordability. Within this context, we categorize their concerns under 4 questions.

Is the Element of Value Relevant to Me as a Payer?

U.S. private payers should aim to be the agent of their enrollees (who are all patients or potential patients) and should arguably consider any factors that might affect their lifetime well-being. This would potentially include the set of uncertainty-related factors that we describe in this review, given their possible effect on the peace of mind of enrollees and patients, and therefore their willingness to pay an additional premium amount for a health insurance benefits plan that delivers them. These factors, such as insurance value, real option value, and the value of knowing, could constitute an important part of the societal value of medical innovations. As Watkins and Tsiao point out, the “value of knowing” for the patient is an element that insurers have traditionally considered to be out of scope for coverage.²⁸ We would argue that this is rather odd, since it would be a shame if the value that patients attach to being informed is not valued by their health insurance plans.

If It Is Relevant, Can It Be Measured?

Watkins and Tsiao also describe productivity effects and ROV as difficult to measure and the value of hope as subject to great uncertainty.²⁸ Thus, all are likely to have a modest effect on the value calculation, so this is an empirical question. We have summarized the measurement work that has been done. At a minimum, this research suggests that these effects are potentially measurable and sometimes significant in magnitude.²⁸ More work is needed, if we agree in principle that these factors might matter.

If It Is Relevant and Measurable, Can I Include It in My Decision-Making Framework?

As Watson and Tsaio indicate, private insurers feel the pressure of being at the cutting edge of making hard choices about access to cost-increasing new technologies that may provide value of uncertain magnitude.²⁸ Among U.S. insurers, Premera was an early adopter of a value-based formulary that explicitly considers cost-effectiveness and thresholds as factors in its decision making. But as part of its deliberative process, Premera does consider other “bioethical issues and societal values” and what the Institute for Clinical and Economic Review calls “other benefits and contextual considerations.”^5,29 So, we come back to our starting point—“what else should be in there?”

Watkins and Tsiao also argue that multiple-criteria decision analysis (MCDA) would be needed but is “likely beyond the expertise of most U.S. payers”²⁸—of course, payers can (and do) hire or buy expertise on many matters if needed. It is an open, practical question as to how these additional considerations are best brought to bear in the formulary decision process. We would argue that the small set of, and perhaps budding, literature that we have summarized suggests that many of these effects can be estimated in monetary (NMB) terms (or in equivalent QALY terms given an assumed threshold willingness to pay for a QALY), supporting their use quantitatively as part of ACEAs or as a weighted criterion in MCDAs, either of which could support the deliberative process of formulary review and price negotiation. The ISPOR STF supports further experimentation with ACEAs and MCDAs.

Doesn’t This Justify Higher Prices and Make My Central Problem—How Do We Afford All of These High Prices for New Drugs—Worse?

Although our main argument is that ACEA is worthy of further exploration, we would also argue that consideration of other factors that affect the value of medicines can help with the affordability issue. By better accounting for the relative value of new medicines, we can better align the reward-for-value equation, providing more consistent signals and thus incentives to invest in more valuable innovations—over the long run, we would spend our total budget more wisely. Regarding affordability, the STF recommended that we should “manage budget constraints and affordability on the basis of cost-effectiveness principles.”¹ In other words, we should have a broad, augmented concept of value by which we compare existing technologies and proposed new inclusions against a threshold, namely, our willingness to pay for health care versus other uses of our scarce societal resources—medical and nonmedical. The STF also set out how short-term priorities should be agreed upon, given a fixed budget. But this is about making decisions among many things of value when we cannot afford to do them all. Ignoring elements of value that matter does not help us get our priorities right!

Conclusions

The ISPOR STF called for additional research to estimate the effect of novel elements of value on conventional CEAs. In the future, we expect that managed care and managed care pharmacy will be approached with these new value arguments, but this will likely take some time. However, it should be clear that these arguments have some validity from an economic perspective and that their application could result in more refined decisions about formulary management and pricing. These critical decision makers need to start thinking about how they are going to incorporate this information.

In this review, we focused on elements related to uncertainty and found only a few studies for each. The largest effects were related to insurance value, in particular to health risk reduction—adding 30% or more to NMB. Estimated effects for ROV, value of hope, and value of knowing were less than 10%, but their effects on NMB are likely to be additive, although work is needed to limit any double counting.

Clearly, there is much more work to be done to estimate these effects, which may vary by disease area and patient group, but they are potentially large enough to warrant that research. Existing studies estimated the value of knowing and value of hope through willingness to pay elicitation; insurance value through calculating and valuing reduction in dispersion of financial and physical risks; and ROV through projecting and valuing mortality reduction in the future. While ex post estimates—reflecting how much value was actually delivered by a specific intervention—can provide important information, we would argue that for valuation and pricing to affect inclusion in a health benefits package, the ex ante perspective is the correct one. More research is needed on methods and estimates, as well as identifying the opportunity cost for health or nonhealth spending of paying more for health and uncertainty-related gains.^30,31