Evidence from Cost-Effectiveness Research

Abstract

Summary: Economic evaluations are a set of outcomes and health services research methods to inform the debate about the rising cost of health care and include cost-of-illness studies and cost-effectiveness research. Cost-effectiveness research is the comparative analysis of two or more alternative interventions in terms of their health and economic consequences, whose results are expressed as an incremental cost-effectiveness ratio, the ratio of differences in cost between a pair of medical interventions to the differences in the corresponding health effects. These research methods are particularly important to neurological diseases with debilitating natural histories, long-term courses, and a growing number of exciting, yet costly, treatment options available. The results of economic evaluations of neurological conditions influence resource allocation decisions, help set reimbursement rates, estimate future healthcare expenses, and improve the quality and efficiency of delivering neurological care. For these research methods to achieve their potential, continued methodological advances within the field are needed, as well as a more systematic integration of these methods into mainstream research to address critical questions regarding the health and well-being of patients with neurological illness.

INTRODUCTION

In 2002, the US spent nearly 1.7 trillion dollars providing health care to its citizens, and the fastest growing components are prescription drugs and hospital care.¹ The US prescription drug market is a $140 billion industry² and has been growing at a rate of 15–18% each year.³ Although neurotherapeutic advances have occurred, they have come at an increased cost—to payers, providers, and patients—and some have even questioned the value of some of these advances.⁴ At the same time, patient out-of-pocket costs are increasing and there are growing pressures to contain costs and more efficiently manage societal resources.

Economic evaluations have emerged as a set of research methods to define the economic burden of diseases, as well as to determine the value of technologies, treatments, and policies to manage patients. Here we review the uses of economic information, cost-of-illness studies, cost-effectiveness research, and key concepts for performing and interpreting such studies to improve the efficiency of neurological care.

USES OF ECONOMIC INFORMATION

Economic information about an illness serves many potential audiences. In addition to providers and patients, other audiences include decision makers at various levels who help establish clinical practice and healthcare system and public policies. Specific examples include practice guideline developers, pharmacy formulary committees, managed care companies, and federal and state legislators.

There are many objectives of economic evaluations. Data from economic studies can be used to track current costs and predict future expenses, inform decisions of health care coverage, access to care, and resource allocation, all of which can lead to improvements in quality of care. In addition, economic information could be used to calculate capitation rates so that providers can be paid appropriate rates for caring for groups of people.^5–8 Patient advocacy groups, lobbyists, and researchers often use economics information (i.e., estimates of economic burden of disease) to support or further their cause of obtaining more funding for social programs and research.

Economic evaluations also inform questions of cost-effectiveness. The main objectives of cost-effectiveness research are to compare the value of new or expensive interventions with the standard of care and to improve the efficiency in the delivery of health care services. For example, deep brain stimulation for Parkinson’s disease (PD) or surgery for temporal lobe epilepsy are costly up-front investment in terms of time and money, and many would rightly ask if their investments are worth the long-term improvements in health gained. The methods of cost-effectiveness research provide the analytic framework to make such assessments.⁹

COST-OF-ILLNESS STUDIES

Cross-sectional studies

Cross-sectional cost-of-illness studies, also referred to as prevalence-based studies, take a snapshot of the defined population and ask the question “What is the cost of providing MS or stroke care across the population of patients over a defined period of time?” Cross-sectional studies assess a defined and relatively heterogeneous population of patients over time. The population scope may include a country, a defined geographical area, an insured population, or a clinic population. These types of analyses can provide “burden of disease estimates,” provide insights into utilization and cost relationships with demographic characteristics, access to care, and quality.

Longitudinal studies

Longitudinal cost-of-illness studies, or incidence-based studies, address the question “What is the cost to care for an individual with PD or epilepsy, or a homogenous cohort of individuals over the course of study (e.g., duration of a clinical trial, patient’s lifetime)?” Longitudinal studies assess a defined and relatively homogenous population of patients over time. The population scope may be by stage of disease (e.g., de novo or early-stage patients) or other patient characteristics (e.g., age) pertinent to the analysis. These epidemiologically based models require a precise knowledge of the natural history of disease and the probability of risk factors and complications over time. Therefore, to track the economic stream of events over time, one needs a thorough and detailed knowledge of the clinical events over time.

Cost-effectiveness research

Cost-effectiveness research is the comparative analysis of two or more alternative interventions in terms of both their health effects and cost. Important considerations when designing a cost-effectiveness study include the following: the audience, the perspective, the type of health effects, the time horizon, the study boundaries, the patient population, the setting, and details about the two competing interventions (often termed the “new” intervention and the “baseline” intervention).¹⁰ Cost-effectiveness research questions are not limited to treatment strategies (e.g., pharmaceuticals, surgery), but can also be applied to competing diagnostic, screening, or education/behavior change strategies.

Many research designs can be used to study the cost-effectiveness of neurological intervention, but the two most common designs are clinical trials and decision analytic modeling. Clinical-economic trials collect cost outcomes in addition to the health outcomes, and general reviews are available regarding important considerations when designing, implementing, and analyzing clinical trials that include an economic component.¹¹ Certain clinical trials are not appropriate for addressing questions of cost-effectiveness. For example, there is little role for a clinical-economic component in placebo-controlled trials, because the placebo arm limits generalizability to clinical practice. Situations that lend themselves to excellent cost-effectiveness assessments include highly prevalent diseases that have available new, expensive technologies of questionable or of marginal benefit.

Decision-analytic models structure evidence on clinical and economic outcomes in a form that can help to inform decisions about clinical practice and health care resource allocation, as well as identify the key parameters that affect these outcomes (a process known as sensitivity analysis). In contrast with clinical-economic trials, models synthesize evidence on health consequences and costs from many different sources, including data from clinical trials, observational studies, insurance claim databases, registries, expert opinions, and preference surveys, and link these data to outcomes that are of interest to health care decision makers.¹² The two designs, modeling and clinical trials, can also be used in combination, so called hybrid analysis,¹³ in which one models the long-term health and cost consequences beyond the time horizon of the clinical trial.¹⁴

The final output for a cost-effectiveness research study, regardless of the study design, are incremental cost-effectiveness ratios (ICERs) calculated for each pair of competing interventions: ICER = ΔC/ΔE = (C₁ − C₀)/(E₁ − E₀) < λ, where C₀ is a cost of “old” intervention (control), C₁ is a cost of a new intervention, E₀ are health benefits of the control intervention, E₁ are health affects due to the new intervention, and λ is a cost-effectiveness threshold value.

Figure 1 is a graphical illustration of how to analyze the cost-effectiveness of a new intervention compared with an old intervention. The cost-effectiveness plane consists of a cost dimension (y-axis) and an effects dimension (x-axis). We begin our analysis of the data by placing the cost-outcome pair of the baseline or old intervention at the intersection point of the cost-effects axes. We then evaluate the new intervention in terms of the incremental change in both costs and effects compared with the baseline intervention. This incremental analysis results in the placement of the new interventions cost-outcome pair into one of four of the following cost-effectiveness quadrants: “northwest, ” “southeast,” “southwest,” and “northeast” (FIG. 1A). In health care, because most new technologies are both health- and cost-enhancing, we often find ourselves in the northeast quadrant asking ourselves “Is the additional effects of our new technology compared with the old technology worth the additional costs?” Figure 1B shows how the two-dimensional cost-effectiveness plane can also be dissected into regions of cost-effectiveness depending on the socially acceptable cost per unit of health effect gained (λ). For example, the blue region is that space in which the estimate for a new technology compared with an old technology would cost less than $50,000 per health effect gained. The yellow region is that space in which the estimate is between $50,000 and $100,000 per health effect gained. Finally, the pink region is that region where the cost effectiveness estimate is greater than $100,000 per health effect gained.

FIG. 1.

Cost-effectiveness quadrants and regions. A: The four cost-effectiveness quadrants: northwest, southeast, southwest, and northeast. The southeast quadrant is the cost-effectiveness space in which the new intervention results in greater effects and less costs compared with the old intervention; in this instance, the new intervention is said to “dominate” the old intervention. The northwest quadrant is the cost-effectiveness space where the new intervention results in greater costs and less effects compared with the old intervention; in this instance, the new intervention is said to dominated by the old intervention. The northeast and southwest quadrant provide results that are not as straightforward. The northeast quadrant is the cost-effectiveness space in which the new intervention has greater effects and greater costs compared with the old intervention and the question becomes whether the gain in health is worth the additional cost in dollars. The southwest quadrant is the cost-effectiveness space in which the new intervention generates less effects and less costs compared with the old intervention and the question becomes whether the loss in health is worth the savings in dollars. B: How these cost-effectiveness quadrants can be sectioned into cost-effectiveness regions that can assist in deciding the desirability of the new technology. For example, a definitely adopt region is defined as the following blue subregions: 1) the northeast quadrant that is <$50,000 per QALY, 2) the southeast quadrant, and 3) the southwest quadrant that is >$100,000 saved per QALY lost. The ambiguously adopt region is the yellow northeast and southwest quadrants between $50,000 and 100,00 per QALY gained ($ lost) or lost ($ gained). The definitely do not adopt region is defined as the following pink subregions: 1) the northeast quadrant that is >$100,000 per QALY, 2) the northwest quadrant, and 3) the southwest quadrant that is <$50,000 saved per QALY lost.

KEY (IF NOT UNIQUE) CONCEPTS WHEN PERFORMING AND INTERPRETING ECONOMIC STUDIES

There are several key steps when performing and interpreting data on the economics of disease that are not part of usual patient-oriented research practice. These include 1) defining perspective and time horizon, 2) collecting data on health care utilization, 3) costing health care resources, 4) analyzing data on utilization and cost, 5) defining and measuring health effects, 6) adjusting costs and effects for inflation and discounting, 7) and evaluating uncertainty. Each of these is discussed below.

Perspective and time horizon

Central to any study on cost is the perspective of the study.¹⁵ The perspective of the cost study refers to the perspective in which the costs are being gained or lost. Commonly used perspectives include the “patient,” “health insurance,” “provider,” and “societal.” From the patient perspective, cost is measured in terms of out-of-pocket losses. From the health insurance perspective, the cost of disease and treatments is measured in terms of insurance premiums received minus the claims paid. A study from the societal perspective would include all costs gained or lost. Defining the “time horizon” of the study is equally important in planning an economic evaluation. The full clinical and economic consequences of an intervention often require a significant period of time to fully evolve; in fact, longer than the duration of most clinical trials. For example, recent dopaminergic trials in Parkinson’s disease used the first onset of dopaminergic events as the primary endpoint. However, from an economic perspective it is important to understand the potential long-term economic consequences associated with these adverse events.

Collecting data on health care utilization

Estimating cost of illness or the impact that a treatment has on the cost of an illness can be a painstakingly tedious process. Most cost studies employ a two-step process, first collecting data on utilization (i.e., resource use) and second, costing each resource use (i.e., unit pricing) to arrive at a total estimated cost. Collecting utilization data can occur alongside many study designs including cohort studies, case series, and randomized trials or retrospectively by asking patients to recall their health care utilization over a defined period of time. In addition, the process of collecting utilization data can be done naturalistically as health insurance members submit claims.

A variety of approaches for collection of data on utilization exist, and these include subject interviews, subject surveys, provider surveys, medical record reviews, health care utilization diaries, and insurance claims data.¹⁶ Collecting data on utilization can be viewed as a detailed accounting exercise. Comprehensive lists of resource categories exist to serve as reminders of possible resources to include in the data collection exercise.¹⁷ The objectives, the perspective, the costing detail required, and budgetary constraints often dictate the data collection method chosen for a study. However, much more research is needed to validate the accuracy of these different data collection methods.

The long list of cost categories can be divided into two discrete resource categories: direct costs and productivity costs.¹⁸ Total cost estimates often combine both cost categories, but it is useful to also report them separately. Direct costs reflect the dollar burden of the medical care and nonmedical care expenditures made in response to disease. The cost of pharmaceuticals is one type of direct medical costs. Other types of direct medical costs include cost of hospitalizations, cost of physician visits, cost of tests and procedures, and cost of durable medical equipment. Direct nonmedical costs include cost to caregivers or the valued time in dollar terms in caring for a loved one. Productivity costs reflect the dollar value of the work lost due to death or morbidity induced by disease or its treatment. Therefore, productivity cost is especially important for studies conducted from the societal perspective.

Costing health care resource use

Costing resource units should be viewed as a research exercise in itself, and usually occurs after the collection of medical resources. Cost estimates for resource units are not organized for research purposes and therefore, are contained in many different sources used for commercial purposes. This decentralization of cost information and a lack of a research-based “cost-coding dictionary” can make the costing exercise challenging, tedious, and, if not careful, inaccurate. Unit pricing decisions and sources used should be prespecified within the protocol.

The ideal cost estimates for each resource use would be their opportunity cost, defined as the value of that good or service in its next best use.¹⁹ Opportunity costs are reflected as the price in a perfectly competitive marketplace. No marketplace is “perfect,” however, and the health care marketplace has many distinguishing features (e.g., information asymmetries, market distortions, and cross-subsidies) that make it less perfect than other markets. Therefore, routinely used prices of health care goods and services (e.g., charges and reimbursements) are not true opportunity costs. At their best, health care market prices can be viewed as “proxy” costs, which can be either higher or lower than opportunity costs. Therefore, cost estimates used in economic studies may be far removed from opportunity costs, and there are methods to convert certain available prices to better reflect costs (e.g., hospital cost-to-charge ratios).^20,21

Important considerations when deciding on sources of unit cost will be data availability, perspective of the analysis, country of origin, and data purchasing costs. In the US, there are a variety of methods for obtaining estimates of hospital costs.²¹ The Medicare reimbursement schedule is becoming an increasingly used method for obtaining cost estimates for physician visits and procedures by Current Procedural Terminology (CPT) codes (Centers for Medicare and Medicaid Services; http://www.cms.hhs.gov/providers/pufdownload/rvudown.asp). In addition, the Red Book can provide average wholesale prices for prescription drugs.²² Gender, age, and occupation-specific productivity costs can be obtained from information from the Bureau of Labor statistics (http://www.census.gov/hhes/www/income.html).

Economic studies can be performed in many different countries and methods are available to make adjustments across countries using the purchasing power parity. In general, most cost estimates should be viewed as country-specific given the differences in national health care systems and financing (e.g., different ratio of specialists to generalists, payment mechanisms, and availability of treatments across countries). Exceptions to this rule can be made, however, for multinational clinical trials and if methods are used to account for this variation.²³

Analyzing data on utilization and cost

Utilization and cost data have several characteristics that make them challenging to analyze.²⁴ Utilization data and cost data in any given population are often skewed because a small minority of subjects often contribute to a disproportionately large percentage of the total costs. Therefore, assumptions of normality do not hold and more sophisticated statistical methods are often required (i.e., Box-Cox transformation, two-part model^25,26). In addition, many of the available utilization and cost estimates are from nonsampled data (i.e., deterministic rather than stochastic), and therefore, traditional methods of statistical analysis are not appropriate. Therefore, alternative methods to investigate the implications of uncertainty are needed (e.g., sensitivity analysis). Finally, missing cost data can be of particular concern and imputation or modeling methods may need to be used to evaluate the impact of nonrandom missing cost information.²⁷

Defining and measuring health effects

Many types of health effects have been used in cost-effectiveness research and examples include “life years saved,” “cases detected,” and “stroke prevented.” These health effects have limitations when trying to allocate societal resources, because it is difficult to interpret the cost-effectiveness of interventions that use different outcome measures. When the objective of cost-effectiveness research is to help “society” understand the value of an intervention and to assist in the allocation of resources, the quality-adjusted life year (QALY) should be used; this allows for the comparison of cost-effectiveness across disease and interventions.

In the calculation of a QALY, estimates of health preferences are needed. Health preferences, also called utilities, are numeric ratings of the desirability of health states, and should be distinguished from health status.²⁸ Health status measures classify patients into specific health states; for example, Hoehn and Yahr stage 2 or “advanced PD.” Health preferences measures have individuals value the desirability of health states. The value scale is usually from death, anchored at 0, to the best imaginable health states, anchored at 1. Some rating systems explicitly include health state valuations worse than death.

There are several methods of assigning preference values to health states that could be then used to estimate QALYs. These include scaling methods such as visual analog systems, and choice methods such as the time trade-off and the standard gamble. In addition, quality-of-life values can be obtained from prescored health state classifications systems including the Health Utilities Index, EuroQol-EQ/5D, Quality of Well-Being, and the SF-6D, a new single summary preference-based measure of health derived from the SF-36.^29–32 These classification systems include a series of health status questions that generate a numeric estimate of the desirability of a health state based on quality-of-life weighting obtained from other populations. Caution should be exercised, however, when using different methods of generating preference estimates, because research has shown that the different elicitation techniques yield different estimates of QALYs gained or lost.³³

Adjusting costs and health effects for inflation and discounting

Cost and effects data need to be adjusted for inflation and time preferences. For example, if the source of cost information is obtained from an earlier year and one would like to express the value of these costs in current dollar terms, then one needs to adjust for inflation. Inflation is related to the change in the value of money due to economic development over time. Methods to adjust for inflation should be based on the Consumer Price Index (at http://www.bls.gov/cpi), its health care components, or one of its subcomponents.

If, on the other hand, a study includes costs or effects gained or lost in future years, then these data should be discounted to present value. Discounting reflects how people value goods and services obtained today versus the same goods and services they could buy in the future. Discount rates can vary, but the most commonly recommended discount rate is 3%.³⁴

Evaluating uncertainty

When the cost and effects data are generated from the same population as in a clinical trial, one can obtain 95% confidence estimates around the point estimate.³⁵ Analytically it is not trivial, however, to estimate a standard error or 95% confidence interval around a ratio such as ICER. For example, in Figure 2, the point estimate is indicated by a star and generates a cost-effectiveness ratio of $106,900 per QALY gained. Using the variance and covariance between the cost and effects estimates, one can generate confidence ellipses, and the Figure shows the 50, 80, 90, and 95% confidence ellipses around the point estimate.

FIG. 2.

Incremental cost-effectiveness and confidence ellipses. The point estimate for the incremental cost-effectiveness ratio is highlighted as a magnified star. 50, 80, 90, and 95% confidence ellipses are superimposed on the distribution of 1000 bootstrapped data points. Bootstrapped distributions of the estimates can be used to determine the probability that the true value of the cost-effectiveness ratio lies within each of the regions of cost-effectiveness. The probability that the ratio is in the definitely adopt region is 0.298 (298 of 1000 points); the probability that the ratio falls in the ambiguously adopt region is 0.207 (207 of 1000 points); and the probability that the ratio is in the definitely do not adopt region is 0.495 (495 of 1000 points).

Although graphically appealing, the confidence ellipses do not fully address the question of cost-effectiveness, which requires an estimation of the probability of falling within a region of cost-effectiveness. There are methods to determine the probability within which the true cost-effectiveness ratio lies.³⁶ For example, using bootstrap techniques³⁷ one can regenerate cost-effectiveness estimates N times and plot them as the superimposed scatterplot (FIG. 2). One can then determine the probability of being below some prespecified cost-effectiveness threshold λ (e.g., <$50,000 per QALY, <$100,000 per QALY, or <$200,000 per QALY) by counting the number of scatterplot points within the region of cost-effectiveness and estimating percentage of the total number of points that fall into this region.

Another method is to use the net benefit approach and generate acceptability curves as shown in Figure 3.³⁸ The acceptability curve plots the probability of the new technology being cost-effective based on different valuations of a QALY and in this illustration, ranges from $0 to $600,000. If decision makers could prespecify thresholds of cost-effectiveness, these methods will become increasingly useful, and recent proposals have recommended that the US societal threshold for considering a new technology to be cost-effective should be $200,000 per QALY.³⁹

FIG. 3.

Acceptability curve: probability that a new intervention is welfare-enhancing given different economic values of a QALY, ranging from 0 to $600,000.

The evidence obtained from most models differs from that generated from clinical trials because many of the model inputs come from nonsampled data; therefore, the cost and effects outputs do not have known variances and covariances that allow confidence intervals to be estimated. The ultimate purpose of most models, however, is not to predict future events but to inform policy decisions as they do in other policy arenas, including environmental and defense policy.⁴⁰ In fact, most models are impossible to validate in the strictest sense given the changing nature of technology over time. In order for a model to influence decision, it has to be believed: its structure, its inputs, the methods of debugging, and the methods to evaluate uncertainty (e.g., sensitivity analysis). It is the responsibility of the researcher conducting modeling studies to adhere to best practice standards and to communicate that the results are completely conditional upon any assumptions. This is particularly important given that multiple biases can exist in the model construction, data inputs, data analysis, and interpretation.⁴¹

HOW TO IMPROVE THE USE OF ECONOMIC INFORMATION FOR NEUROLOGICAL DISEASES

Table 1 provides suggestions to improve the use of economic information. First, one should become familiar with study designs and methods to perform and appraise cost-of-illness and cost-effectiveness research.^42,43 Along with this should be the recognition that although the evidence from cost-effectiveness research is different from that of clinical trials, it can be useful to inform decision making. Second, there needs to be more research on the most reliable and accurate ways to collect and measure resource use, and on natural history studies to serve as the clinical event backbone upon which lifetime costs can be estimated. Third, more phase III and IV clinical trials should include an economic component, and information about cost-effectiveness should be incorporated into clinical practice guidelines and practice parameters. Fourth, collaborative partnerships should be strengthened with health economists, quality-of-life researchers, and health policy researchers. Finally, support for general advances within the field, such as periodically updating and revising cost per QALY benchmarks to account for budget increases and inflation, will improve the credibility of the fields and acceptance by many health care decision makers. Only then will economic information stand beside clinical information to help guide decisions to improve the quality and efficiency of neurological care.

TABLE 1.

How to Improve the Use of Information Generated from Cost-Effectiveness Studies in Neurological Disease

• Become familiar with study designs and methods to perform cost-of-illness and cost-effectiveness research

• Become familiar with how to appraise cost-effectiveness research

• Recognize that the evidence from cost-effectiveness research is different from that of clinical trials but can be useful to inform decision making

• Work collaboratively with health economists and quality-of-life researchers

• Incorporate information about cost-effectiveness into clinical practice guidelines and practice parameters

• Incorporate economic evaluations into existing research programs

○ More natural history studies to serve as the clinical backbone of lifetime cost estimates

○ More phase III and IV clinical trials should include an economic component

• Support and participate in methodological advances within the field

○ Research on most reliable and accurate ways to collect resource use and measure costs

○ Improve methods of evaluating and presenting uncertainty in cost-effectiveness estimates

○ Update and periodically revise cost per QALY benchmarks to account for budget increases and inflation