Value-Based Hypothesis Testing for Cardiac Device Clinical Trials

Medical device sponsors seeking to introduce innovative devices in the United States must clear 2 regulatory hurdles. First, they must provide evidence demonstrating reasonable assurance of safety and effectiveness for adjudication and marketing approval by the Food and Drug Administration (FDA). Second, they must apply for reimbursement coverage through the Centers for Medicare and Medicaid Services (CMS). This second step is based on a different metric that requires demonstration that treatment using the device is reasonable and necessary.

Typically, decisions for reimbursement coverage lag well behind the FDA approval for marketing and may result in substantial delays to the availability of some novel therapies. In interventional cardiology, recent examples include a 6-month wait for a coverage with evidence decision after an already-belated FDA approval for the first transcatheter aortic valve and the only recently final reimbursement decision for the Watchman (Boston Scientific, Marlborough, MA) left atrial occlusion device, almost 11 months after FDA-marketing approval. These delays arise in part from the incomplete overlap between safety/effectiveness and reasonable/necessary standards. Pivotal clinical trials are typically designed to support the safety and effectiveness assessment but may lack data necessary to characterize reasonable and necessary. Although reasonable and necessary does not have a strict regulatory definition, it connotes added healthcare value, defined as improved quality of outcome relative to cost.^1,2 Harmonizing these goals in clinical trial designs will accelerate the clinical introduction of important new technology. This study will examine the limitations of current cardiac device clinical trials and explore methods for incorporating value-based hypotheses into the design of these studies.

Noninferiority Trial Designs and Demonstrating Value

Given the frequent requirement for active treatment controls and the difficulty in proving superiority of incremental changes in new devices, noninferiority trial designs have become the mainstay of cardiac device clinical trials. In addition to demonstrating that a new device is statistically noninferior to the comparator based on an outcome difference that is clinically meaningful, these studies should also provide evidence of added value based on an important secondary outcome. The advantage of the new technology may be that it is safer, less invasive, less expensive, or superior based on secondary clinical measures of effectiveness. Any of these secondary benefits may increase the quality to cost ratio and thus add value. In contrast, a new device that is noninferior based on the primary end point but offers no other measurable advantage or has a higher cost that is out of proportion to any advantage does not add value.

Patient-Reported Outcomes as a Measure of Value

There is increasing interest in the role of patient-reported outcome measures (PROMs), such as perceived symptom burden, functional status, and health-related quality of life (HRQOL), for determining differences in clinical effectiveness. Although it is unlikely that PROMs will replace clinical outcomes of mortality and major morbidity as primary end points, it can be argued that they are often at least equally important from the patient perspective and provide an opportunity to assess possible differences between treatments that may not be apparent based on other primary clinical outcomes.

There are several concerns, however, with implementation of PROMs in cardiac device clinical trials. By definition, they represent subjective measures. As a result, to be potentially reliable, the patient and assessor should be masked to the treatment, which is frequently difficult in cardiac device clinical trials. Furthermore, data for assessing HRQOL are difficult to obtain, resulting in validity concerns because of missing data. Finally, the utility of PROMs will be limited to studies of patients with substantial symptoms where measurable improvement is likely to affect HRQOL measures. Despite these limitations, several clinical trials in interventional cardiology have used validated instruments to provide valuable outcome data for differentiating treatment effects beyond the primary clinical end points.^3–5

Instruments for assessing HRQOL include those that provide generic health, as well as disease-specific analysis. Common examples for validated generic HRQOL include the Medical Outcomes Study Short Form-36 (streamlined Short Form-12) and the EuroQOL, which assess overall physical and mental well being. Among the disease-specific instruments of interest in cardiology are the Seattle Angina Questionnaire and a variety of instruments for heart failure, with the Minnesota Living with Heart Failure Questionnaire and Kansas City Cardiomyopathy Questionnaire the most commonly used.⁶ Disease-specific HRQOL instruments have demonstrated significant differences between treatment groups even in cases where primary clinical end points were not different. For example, in the Clinical Outcomes Utilizing Revascularization and Aggressive Drug Evaluation (COURAGE) trial, scores on the Seattle Angina Questionnaire were significantly better for percutaneous coronary intervention versus medical therapy between 6 and 24 months. The import of these results was minimized, however, because the differences were small, due in part to significant improvement from baseline in both groups, and were no longer evident at 36 months.⁴ Certainly, these results might be viewed differently if early relief of angina had been regarded as an important end point for assessing healthcare value. In heart failure populations, baseline scores, changes from baseline, and differences in change scores between groups have correlated significantly with mortality and heart failure hospitalization for both the Minnesota Living with Heart Failure Questionnaire and Kansas City Cardiomyopathy Questionnaire.⁶ Specifically, in the Placement of Aortic Transcatheter Valves (PARTNER) trials of transcatheter aortic valve replacement (TAVR), the baseline Kansas City Cardiomyopathy Questionnaire demonstrated a significant association with 1-year mortality among medical therapy patients and change from the baseline Kansas City Cardiomyopathy Questionnaire at 1 and 6 months among patients undergoing TAVR quantified a significant improvement in quality of life.⁷

The benefit in HRQOL after TAVR may seem less critical as an end point given a clear mortality advantage compared with medical therapy. From a value perspective, however, improved quality of life is also important. Indeed, for many patients being evaluated for TAVR, improved duration of survival without improved quality would not be considered a worthwhile value. Assessment using heart failure HRQOL instruments will be even more essential for evaluation of percutaneous mitral valve technologies for which significant improvement in quality of life is anticipated but differences in mortality or major morbidities compared with the standard of care are unlikely.⁸

Cost–Utility Analysis, PROMs, and Estimating Value

Improved quality outcomes, including PROMs when applicable, are an essential component of the value calculation. True healthcare value can only be recognized, although, when the incremental cost of the therapy is reasonable and sustainable. Cost-effectiveness analysis is a useful technique for helping to estimate the relationship between quality of life benefit and treatment costs. Most commonly, cost-effectiveness is reported in terms of cost per quality-adjusted life year and is judged according to a prespecified, albeit arbitrary, threshold. The quality-adjusted life year is based on a factor that estimates perceived quality of life and varies from 0 for dead to 1 for perfect health. This factor or utility multiplied by the duration of life sustained generates quality-adjusted life years. In general, therapies that reduce mortality or major morbidities and can be offered at acceptable treatment risk will be cost-effective and add value, even if costs are high. However, when a measurable benefit in mortality or other life-threatening outcomes is unlikely, PROMs can still provide a method for estimating cost-effectiveness and healthcare value using validated algorithms that assign a health-state utility based on generic HRQOL instruments, such as the EuroQOL.^9,10

Value-Based Hypothesis Testing in Cardiac Device Clinical Trials

A challenge for clinical trial investigators and sponsors is whether assessment of a favorable effect on healthcare value for a new device can be projected during the early design phase. Alongside establishing parameters for end point rates and sample size for testing the primary study hypothesis, this estimate should provide convincing evidence that the new device represents improvement in the quality and cost relationship or value. In lieu of the unlikely scenario of demonstrating superiority for the primary safety and effectiveness end point with a lower estimated cost (Figure), the design must rely on secondary measures of improved quality outcomes, including PROMs, and cost estimates related to these outcomes. Establishing such a value-based hypothesis that is negotiated early with the CMS during trial design could potentially avoid the current long delay to a reimbursement decision.

Figure.

Four quadrants of a quality to cost relationship. Quadrant D represents ultimate value based on higher quality at reduced cost but is unlikely to be achieved by most new technology. Quadrant A is unacceptably poor value because of reduced quality at higher cost. Quadrants B and C are possible areas for a value-based hypothesis with constraints of acceptable cost thresholds for increasing quality (hatched area B) and acceptable quality reduction for proportionate cost savings (hatched area C).

Historically, the CMS has eschewed incorporating cost-effectiveness analyses into coverage decisions. Some of the reluctance is to avoid a perception of cost-based rationing of care for older Americans. Health policy experts have argued, however, that cost-effectiveness should not be regarded as a cost-containment tool but rather a technique for improving value.¹ Among cardiac device coverage decisions, cost-effectiveness analysis was almost certainly the underpinning for the unprecedented value-based coverage decision for drug-eluting stents ahead of FDA-marketing approval.³

There are other caveats to consider in the use of cost-effectiveness analysis as a cornerstone for establishing a value-based hypothesis. Although the concept of cost per quality-adjusted life year is helpful for measuring relative treatment differences and comparing costs among treatments as a function of society’s willingness to pay, there are substantial limitations to using these results for assessing value. These include the use of arbitrary cost thresholds, assessment of even small incremental improvements in technology as cost-effective, and a resulting failure of cost-effective procedures to prevent ongoing cost escalation for the healthcare system. These concerns raise issues for aligning cost-effective technology with the underlying objective of providing care that is affordable and sustainable. The Institute for Clinical and Economic Review has proposed the use of cost-effectiveness models along with an estimate of the projected market penetration to calculate a value-based pricing benchmark.¹¹ Such a benchmark may be a better measure of true value to a healthcare system, but there would need to be more consensus and transparency on how pricing thresholds should be established. These thresholds must consider the overall cost of the procedure to hospitals, as well as the incentives for ongoing necessary innovation by industry. Certainly, previous efforts to control costs, simply by continued reduction in procedural reimbursement, are unlikely to have long-term success.¹²

Furthermore, valid cost-effectiveness analyses have only been available after completion of the clinical trial, and the results for effect size and actual costs are known. Development of a value-based hypothesis as part of the trial design strategy would require adequate projections of effect sizes for clinical end points, including PROMs in many cases, to estimate a health quality utility. This estimated treatment effect together with estimates of total procedural cost or an established benchmark for cost-effectiveness would allow prespecifying a value-based hypothesis. Sponsors and investigators might also consider clear failure of a value-based hypothesis as a factor in decisions for ongoing evaluation of new technology. Early termination of programs for devices that are unlikely to reach the value required for successful marketing may result in overall savings for new device development. Future studies should explore options for testing value as part of device approval trials and evaluate the success of these hypotheses for predicting device coverage and ultimate marketing success.

Conclusions

Cardiac device clinical trials are frequently designed to demonstrate noninferiority for safety and effectiveness end points but may fail to obtain data necessary for showing improved quality of outcomes or healthcare value. Adopting PROMs as secondary outcomes and including cost-effectiveness estimates into the initial clinical trial designs allow testing of a value-based hypothesis even when there is not superiority for primary clinical outcomes. Approval of these design parameters by the CMS and successful results for the value-based hypothesis should support earlier coverage decisions for reimbursement.