Time to incorporate time in cost-effectiveness analysis

Abstract

Cost-effectiveness analysis as a means to evaluate medical innovations has become well accepted in the UK and several other Western countries. An important assumption underlying this method is that costs and effects are constant over time. In reality, however, and especially in the short run, variations in costs and effects are likely to occur. These variations can lead to considerable deviations from the outcome of a conventional economic evaluation, which in turn may lead to serious implementation problems at a local level. Taking time into account explicitly in economic evaluations in health care may enhance their utility for both societal and local decision making, and may ultimately smooth the adoption of new and basically cost-effective health care technologies.

Introduction

Cost-effectiveness analysis (CEA) as part of the evaluation of medical innovations has become well accepted and is applied widely in several European countries. For example, in the UK, the National Institute of Clinical Excellence (NICE) uses cost-effectiveness outcomes, expressed as cost per quality adjusted life year gained, as a criterion for coverage recommendations to the National Health Service. In the Netherlands, the Dutch Health Insurance Board (CVZ) uses a cost-effectiveness criterion in their advice to the minister on adding new technologies to the benefit package. In line with this development, research into CEA methodology has played a major role in the field of health economics in the past decades, focusing on a wide range of issues, such as quality of life measurement and value of information research. Most of this research has been aimed directly or indirectly at improving the validity of cost-effectiveness outcomes. This large attention on economic evaluation in health care has given us a method—the CEA—that is relatively well researched and provides a generally trusted estimate of the cost-effectiveness of a medical innovation. However, CEA is based on several assumptions. One important assumption is that costs and effects are constant over time (i.e., that the rates at which costs are incurred and health outcomes are obtained will not vary during the lifetimes of the competing technologies). In reality, however, and especially in the short run, variations in costs and effects might occur, for example due to learning effects and suboptimal occupancy rates of fixed factors of production. Here, we show that, in the short run, costs-effectiveness may deviate considerably from the outcome of a conventional economic evaluation, and we will discuss the implications of such deviations. Building on these observations, we argue that taking time into account in economic evaluations (1) will make cost-effectiveness outcomes more realistic, and (2) could mean a step forward in bridging the gap between societal and local decision making, thereby making cost-effectiveness outcomes more useful for all kinds of decision makers.

The role of time in cost-effectiveness analysis

Standard CEA is conducted from a societal perspective and includes a cost and benefit assessment for society in general. In practice, standard CEA starts by collecting all the relevant costs and effects for a health care technology over a relevant time period, and subsequently averages these total cost and effects over this period. Next, differential costs and effects are calculated as the difference in average costs and effects between alternative technologies. These differential costs and effects are combined into an incremental cost-effectiveness ratio (ICER), or an incremental net monetary benefit (INMB). The ICER represents the ratio of differential costs to differential effects, and the INMB is acquired by multiplying the cost-effectiveness threshold by the differential effects and then subtracting the differential costs [1]. In utilizing the ICER or the INMB, differences in costs and effects between alternative medical innovations are considered to be constant over these innovations’ respective lifetimes. This means that, following implementation of an innovation, possible efficiency gains are assumed to accumulate at a constant rate, as found in the preceding trial or modeling study. Thus, average costs and effects found in the trial period are assumed to be representative of the average costs and effects over the technology’s lifetime. Therefore, standard CEA focuses inherently on the long-run consequences of adopting a technology. In the long run, technology’s costs and effects can indeed be expected to reach or approximate a steady state, so from a long-run perspective, the modus operandi of the standard CEA seems very justifiable. However, in reality, a technological switch will usually give rise to a phase where the average costs and effects (and consequently the cost-effectiveness) vary and have not yet reached a steady state. Therefore, although using constant ICERs and INMBs may be convenient, the assumption of constant marginal cost-effectiveness outcomes is an idealization and is not very realistic as it neglects the short run, where the old and new technology often co-exist. However, the question is how much of a problem this is in practice: to what extent does the marginal cost-effectiveness of the technologies that are being compared vary over time, and how well does the presumed steady state situation approximate the actual time-dependent path?

Short-run vs long-run cost-effectiveness

The CEA framework considers costs, effects and hence cost-effectiveness outcomes as unchanging quantities in some steady state. However, this steady state is achieved only when all trace of the previous technology has disappeared, in other words, when (1) the new technology is fully functional, (2) staff have mastered the new technology, (3) a more or less constant occupancy rate for the new technology has been achieved, (4) all costs for the old technology have dissipated, and (5) there is no longer any spillover of effects from the old technology into the new situation. As noted above, in the initial time period after introduction of a new technology, these conditions are not met. During this time period costs may be induced for both technologies, and clinical effectiveness for the new technology is likely to be not yet optimal [2]. This may very well result in a negative deviation from the long-run cost-effectiveness outcome during the short-run. This is illustrated in Fig. 1, which depicts time-dependent paths for the average costs and effects of both health technologies and for the resulting ICER. It can be seen that the ICER is indeed less favorable in the short run as compared to the long-run steady state.

Fig. 1.

Average costs (a) and effects (b) for the new and the old technology over time, and the resulting incremental cost-effectiveness ratio (ICER) (c) as a function of time

A multitude of reasons could contribute to these deviations, such as learning effects, rigid labor contracts, effects of the old technology spilling over into the new technology’s period, fixed production factors for the old technology that may become a deadweight, a temporary coexistence of the two technologies leading to diseconomies of scale (for instance, due to a gradual implementation of the new technology combined with a gradual dismantling of the old technology’s infrastructure), budget restrictions within the relevant planning period, etc. [3–5]. Hence, it seems that short-run deviations from the long-run oriented cost-effectiveness model could indeed be substantial. This will be the case especially if the short run is a substantial part of a technology’s economic lifetime. Economic theory generally states that the short run becomes irrelevant if the long run is sufficiently long. However, given the turbulence with respect to technological change that can be seen in certain health care areas, we expect that the short run might be relevant for a relatively large number of innovations. Of course, the magnitude and importance of these deviations in a particular CEA depend on the specifics of the technologies and settings under consideration. Here, it helps that the nature and impact of these factors might to some extent be predictable and systematic. For example, short-run cost-effectiveness outcomes for surgical procedures that require a lot of time to master will display a large deviation from long-run outcomes, due to the longer duration of surgery and higher number of complications in the short run. On the other hand, a substitution of one drug for another will probably bring about relatively small short-run deviations, as it seems hard to see which of the above-mentioned reasons would contribute substantially in this case, except possibly learning effects due to initial dosing difficulties.

A good example with which to illustrate the occurrence of short-run deviations when introducing a new technology is the conversion from analog screen film mammography (SFM) to digital mammography (FFDM) in the Dutch population-based breast cancer screening program. Here, a preliminary study was performed to assess the feasibility of converting to a filmless digital screening program [6]. In this study, annual savings associated with abandoning the old SFM technology were estimated at €13,270,000, and the annual extra cost of adopting the new FFDM technology were estimated at €12,285,000. Hence, the Dutch government concluded that substituting SFM for FFDM would yield net annual savings of €985,000. Although estimations suggested that the transition phase would last 3 years, the anticipated net annual savings were assumed to be realized immediately. This assumption is applied consistently in standard CEA, even in projects where a significant change in infrastructure necessitates a long phase-in process, implying coexistence of two technologies, potentially leading to diseconomies of scale. As several exogenous factors (e.g. European tender rules) delayed the adoption and stretched the transition process, potentially obsolete SFM equipment did not immediately become a deadweight. However, due to the extension of the transition process, short-run losses associated with the coexistence of SFM and FFDM are likely to have caused serious inefficiencies during the transition phase.

In order to actually develop a time-dependent cost-effectiveness model and quantify short-run deviations from the steady state, the approach outlined here will have to be specified further and quantified through mathematical modeling. In a preliminary paper [2], a three-step model has been developed that operates by first determining the initial efficiency losses inflicted by deadweight fixed production factors for the old technology, then adjusting for refilling and writing off freed capacity over time, and finally calculating the length of the short-run time frame in which the efficiency losses exist. This model was applied to two cases: substituting in-hospital dialysis for continuous ambulatory peritoneal dialysis, and digitizing a radiography department. In the dialysis case, it was shown that the short-run INMB decreased by 8.6% compared to the steady state INMB. The radiography case illustrated that short-run deviations can indeed be very substantial, as incorporating these deviations changed the cost-effectiveness outcome from favorable to unfavorable [2]. The model mentioned above could be extended further by allowing for delayed adoption of the new technology, thereby making it possible to calculate short-run efficiency losses for a technology that is implemented gradually. Such a model would operate by taking into account the rate of implementation of the new technology and depreciation of capital of the old technology in order to integrate the NMB functions for both the old as well as the new technology into one time-dependent INMB function. One could also think of incorporating learning effects into a model, reasoning that learning leads to an overestimation of costs and an underestimation of effectiveness during the trial period and that these deviations from steady state cost-effectiveness outcomes can be modeled as a function of clinical trial length. Although time has not been much of an issue in economic evaluation in health care up to now, we consider it possible and important to develop models that are able to incorporate, and thereby make visible, short-run deviations from the steady state.

Consequences for local decision making

Besides making societal CEA more valid, another reason why incorporating time in CEA could be relevant, is to bridge the gap between societal and local decision making. As mentioned above, CEA is a method used to assist decision making when adding health technologies to benefit packages. In doing so, CEA adheres to a societal perspective. However, the consequences of decision making at the societal level are felt at different levels in the health care system. The societal decision to add a new technology to the benefit package might put pressure on local health care providers, such as medical doctors, hospital managers, prescribing advisors, hospital pharmacists and directors in public health, to supply the new technology instantaneously. However, if short-run consequences on costs and effects are not communicated to such providers, unanticipated losses might lead to second thoughts about implementation of what is, in essence, a cost-effective innovation, even potentially denying patients access to more efficient health care. Failing to acknowledge short-run aspects in CEA may raise doubts about the validity of its outcome, and may also lead to disappointment with economic evaluations in general, as evidence on cost-effectiveness is of less interest to decision makers who are focusing on the short run [7]. Indeed, a prominent argument used by health care decision makers to resist the use of CEA is the disconnect between formal CEAs, which have a societal perspective and long-run orientation, and the short-run perspective on decisions within health care organizations [7–9]. Due to ever increasing budget pressures, local decision makers are forced to focus on short-run results. This is confirmed by studies such as that of Drummond et al. [10–12], which revealed that, in the UK, 30% of local decision makers state that being unable to take a long-run view, due to the importance of their annual budget, is an important obstacle to the use of economic evaluations. Also, a recent small study aimed to determine if financial managers in the Dutch health care system focus on short- or long-run efficiency showed similar results (Radboud University Nijmegen Medical Center, van de Wetering et al., manuscript in preparation). Therefore, it seems that the real problem articulated by local decision makers is that, given their short-run orientation, CEAs are seen as insufficiently valid for their organization. Incorporating short-run deviations in the cost-effectiveness model might therefore improve trust in the validity and, consequently, the usability of the ICER.

Conclusions

Taking into account the short run by explicitly modeling time in CEA could lead to more accurate and realistic estimates of the cost-effectiveness of medical innovations. This would at least entail making a distinction between the short and the long run. When short-run deviations from standard CEA occur, they are often unanticipated and could very well slow down the adoption of new and basically cost-effective health care technologies. To enhance trust in CEA, it is important to gain insight into the major factors potentially contributing to short-run efficiency losses. All in all, having a framework for CEA that incorporates time and quantifies short-run deviations could help prevent unpleasant surprises, and provide an implementation decision process with more valid cost-effectiveness data. In the Netherlands, the importance of potential short-run deviations in CEA has now been recognized and has indeed been added to the new version of the Dutch manual for costing research in health care [13].