Value of Research Analysis: An Application to Drug Development

H&O

Can you explain value of research analysis?

RC

Value of research analysis (VOR) is a set of economic methods aimed an improving public agency decision making. In the drug development context, they may be used to identify investments in research and development (R&D) aimed at improving public health. They also may be helpful in designing clinical trials that are smaller and more focused on relevant clinical endpoints. My colleagues at the University of Chicago and I view them as a potential complement to existing scientific, statistical, and ethical criteria for evaluating, prioritizing, and potentially designing proposed clinical trials funded by public cooperatives, academia, and government sponsors.

H&O

Why and how is VOR applicable to drug development?

RC

In the face of growing federal budgetary deficits, numerous government agencies and public policy makers have called for increasing the evidence base to identify and prioritize research investments. There are a handful of economic methods that have the potential to provide a systematic foundation for public decision making in future R&D. VOR analysis is the set of methods most well developed and specifically linked to the public’s perspective. We believe the public health viewpoint is critical, given that the public is the major investor of novel therapeutic techniques, through the National Institutes of Health and national cooperative groups, and is responsible for paying for these therapies once they are developed, through public insurance programs such as Medicare and Medicaid.

The methods are applicable to drug development in a number of ways:

• First, the methods allow a public funding agency to identify key sources of clinical uncertainty facing clinicians in current clinical practice. They may then evaluate a proposed trial based on whether it is addressing the relevant clinical question, including whether it is incorporating specific health outcomes that are measured appropriately and powered to identify differences between trial arms.

• Second, the methods provide a set of tools that analysts may use to calculate the incremental health benefit of proposed additional research relative to the current standard of care and other proposed trials. Such calculations may be used by public policy makers to identify and rank proposed phase II, phase III, or phase IV trials within a specific therapeutic area.

• Third, the methods have been demonstrated to be effective in calculating target subject sizes for phase III trials that account for the cost of enrollment. This application may result in smaller, more targeted trials that require less funding than traditional trials.

H&O

What are the challenges associated with using VOR?

RC

There are a couple of challenges associated with this method.

• First, the analyses require valid and reliable measures of patient health benefit. VOR analyses require the same information that any incremental efficacy or effectiveness calculation requires, which for some clinical areas may be difficult to collect given the time frame and focus on specific endpoints relative to the exclusion of others. Synthesis of evidence is particularly a challenge in oncology where surrogate endpoints are employed. Furthermore, as biomarker or pharmacogenomic subtype analyses become increasingly commonplace in oncology, estimates of the expected benefits of conducting such trials become an important measure. My colleagues and I are currently engaged in research to develop the methods for these applications.

• Second, if costs are to be incorporated into the analyses, valid and reliable measures of trial costs are needed. It is the economic costs of conducting trials, not the accounting costs that are important. The economic costs of running trials include an accounting of time performing the trial and opportunity lost from alternative investments. Although there are aggregate measures of trial duration and overall costs of conducting trials available in the literature and through proprietary vendors, currently were do not have good measures of the economic costs of performing specific types of trials. My colleagues at the University of Chicago, Tomas Philipson and Anup Malani, and I are working on estimating the economic expenditure of incremental patient accrual in oncology trials, overall and by specific type of trial, using a unique administrative dataset. We are also examining how patient recruitment and trial duration are impacted by policy changes. We believe the results of these analyses will potentially act as key pieces of information for the conduct of VOR calculations. They will also be of interest to policy makers and more generally, to those engaged in the financing and organization of clinical trials.

• Third, the analyses requires estimates of treatment costs that are relevant to the targeted patient populations that may ultimately benefit from successful trials and account appropriately for different financing arrangements. For example, list prices for drugs after approval from the U.S. Food and Drug Administration (FDA) are typically not what insurers pay due to rebates and discounts. The prices charged are also in part a result of strategic choices on the part of the maker that account for the expected benefit of the drug, compared to alternative therapies. Furthermore, modern treatment for a number of disease areas such as oncology is composed of multimodal therapy—drugs, surgery, and perhaps radiation. The introduction of a new therapy or a novel application of an existing therapy may change the whole bundle of treatments that need to be considered when estimating expected treatment costs. Such changes may be difficult for an analyst to anticipate and account for when calculating expected treatment costs of the potential introduction of a new therapy based on trial results.

• Fourth, it is very important to understand that cost of potential treatments does not have to be integrated into VOR calculations—the method maybe used to focus on expected health benefits of proposed trials exclusively. However, to the extent that treatment costs are included in VOR analyses, policy makers may encounter public resistance similar to that found in incorporating costs of treatment into insurance coverage decisions such as Medicare formularies.

• Finally, VOR estimates may not help guide investment decisions in R&D if additional research is unlikely to alter recommended treatment regimens. Consequently, my colleagues and I believe it is important to emphasize that such analyses may be most helpful as part of a larger toolkit to adjudicate the additional value of research.

H&O

Can you provide some examples of how VOR is being applied today?

RC

The most widely accepted use of VOR has been for prioritizing medical R&D in the UK context, where a quasi-governmental body (NICE) employs cost-effectiveness analysis to assess coverage decisions for the national health service, and where there is an active publicly funded R&D enterprise closely linked to public health insurance provision. In the United States, most researchers have suggested that VOR analyses are helpful in understanding the health benefits of further investigation of FDA-approved therapies in new or more targeted clinical contexts. For example, my colleagues at the University of Chicago, David Meltzer and Anirban Basu, have suggested that VOR may become part of an evaluation used by policy makers to value research on heterogeneous effects of existing therapies or preferences over existing therapies in patient subpopulations. In a recent article, I have suggested that it may be useful in further personalizing medicine—identifying and prioritizing the use of biomarkers and pharmacogenomic tests to better guide patient-treatment matching and dosing decisions for therapies already developed and available in clinical practice. My colleagues and I are working on applying the method to identify, prioritize, and design phase III pancreatic cancer trials, and evaluating its practical application in the U.S. context.

H&O

What aspects of this evaluation are specific to oncology?

RC

I believe VOR has great promise in assisting oncology-based public agencies, such as the National Cancer Institute and cooperative groups, alone or in combination with private firms in this arena. The need to build up the evidence base and identify and prioritize research in oncology diagnostics and therapeutics has been a focus for many public cancer agencies. The policy motivation for this need is compelling. First, the public sector has been an active funder of R&D in oncology. The development of many new therapeutic approaches is the result of public investments in basic science and, more recently, translational R&D that has produced increases in longevity and decreases in morbidity for many types of cancer. Second, there is substantial financial risk in developing novel therapies for cancer. A recent analysis by DiMasi and Grabowski concluded that compared to traditional therapies, a higher proportion of oncology drug failures reached expensive late-stage testing. Furthermore, a recent analysis by Adams and Brantner showed that the cost of developing cancer therapies is high and steadily growing, more so than other therapeutic areas. Third, although the costs of pharmaceutical-based treatment appears to be slowing overall, and particularly for many chronic illnesses, the costs of cancer therapy is growing at double digit rates. We expect this trend to continue—cancer has just surpassed heart disease as the leading cause of death in the United States.

In light of these factors, I believe the VOR method may have immediate application to publicly funded oncology R&D as part of a scientifically grounded toolkit that government agencies charged with investing public money in translational research could use to target investment based on the potential health benefit for patients. One advantage to the method is that it gives government agencies sophisticated decision models from the public’s perspective that are analogous to the business models that for-profit biopharmaceuticals already employ to make R&D investment decisions. The method may be used as a complement to current and evolving scientific, statistical, and ethical criteria for funding decisions. The method is also complementary to evidence-based guideline development and comparative effectiveness assessments of new products, but places the emphasis on decision making in public R&D at the earliest stages of innovative activity. Consequently, the use of this method, and more broadly, more sophisticated methods for decision making under uncertainty, could help shift public debate from “how to manage access to high cost therapies once they are already developed and used in clinical practice” to “how to invest money and our substantial scientific capacity into therapies that address public health needs as a society.”