Championing Effectiveness before Cost-Effectiveness

The proliferation of costly therapies has led to an increased focus on value. In the treatment of patients with advanced heart failure (HF), one such expensive intervention, CardioMems, is a permanently-implanted device to monitor pulmonary artery pressure and enable adjustments in therapy that could avert the need for a hospitalization. CardioMems was tested in the CardioMEMS Heart Sensor Allows Monitoring of Pressure to Improve Outcomes in NYHA Class III Heart Failure Patients (CHAMPION) Trial, a randomized, single-blind multicenter trial led by the founder of the company. It included 550 New York Heart Association (NYHA) functional Class III HF patients with ≥1 HF-related hospitalization in the past year. (1) All patients were implanted with the CardioMems device. Treatment patients had specific pulmonary artery pressure targets and treatment algorithms administered by study nurses in coordination with the local physicians, while control patients received routine HF care. The procedure was associated with a statistically significant 12% absolute reduction (44% vs 32%) in the primary efficacy endpoint of HF-related hospitalizations up to 6 months. (2) Both primary safety endpoints were also met, including no pressure-sensor failures and a 1% rate of device- or system-related complications. (2)

Shortly after FDA approval, St. Jude Medical completed its acquisition of the CardioMems company for $435 million. The CardioMems device (sensor and delivery system) is priced at approximately $17,750 plus an estimated $68 monthly for device management. Given the large eligible population of hospitalized NYHA Class III patients, the device could lead to enormous societal costs. St. Jude expects $65 million in sales of CardioMems in 2016, and plans to apply for a Medicare National Coverage Determination. (3) Meanwhile, the Centers for Medicare & Medicaid Services has authorized a supplemental New Technology Add-On Payment for CardioMems. To qualify, a new intervention must fulfill multiple criteria, most important of which is to be “an advance in medical technology that substantially improves, relative to technologies previously available, the diagnosis or treatment of Medicare beneficiaries.”(4)

Several groups have estimated the cost effectiveness of the device. In the initial publication of CardioMems’ 6-month results, the authors, which included company employees, calculated a cost of $13,979 per quality-adjusted life year (QALY). (2) However, many assumptions for that analysis are unavailable. A more recent cost-effectiveness analysis, published in December 2015 and conducted by the Institute for Clinical and Economic Review, found a cost of $57,933 per QALY. (5)

In this issue of JACC: Heart Failure, Sandhu et al. report a cost-effectiveness analysis based on 17-month CardioMems follow-up data in the CHAMPION trial that found a cost of $71,462 per QALY. (6) This is the highest cost per QALY of the 3 analyses and according to the American College of Cardiology/American Heart Association Statement on Cost/Value Methodology, this places CardioMems as an intermediate-value technology. (7,8) Sandhu et al. found that the incremental cost-effectiveness ratio depends most on CardioMems’ continuing to have a durable benefit and a device-driven reduction in hospitalizations being associated with increased survival. The analysis showed greater cost-effectiveness in HF patients with preserved (≥40%) ejection fraction and lower cost-effectiveness in a lower-risk cohort based on the Candesartan in Heart Failure-Assessment of Reduction in Mortality and Morbidity (CHARM) trials.

Confidence in CardioMems’ value, however, depends on the assumptions about CardioMems’ effectiveness – which there are reasons to question. Sensitivity analyses allow one to vary assumptions about effectiveness, but if there is no effectiveness then there is no need to calculate cost effectiveness. Because of concerns about the evidence, the FDA initially denied approval in 2011; however, based on additional analyses, the agency approved it in May 2014 for patients with NYHA Class III HF hospitalized for HF in the previous year. (9)

What is the source of the uncertainty given that there is a pivotal positive trial? CHAMPION is the single trial of the device, and it has not been marketed in any other country. (10) Further, the trial was only single-blinded, and knowledge of treatment allocation may have positively biased this study. (11) A bigger problem was concern about contamination of the trial because company-employed nurses who reviewed pulmonary artery pressure readings and provided subject-specific treatment recommendations to treatment patients went far beyond the scope of their work in addressing HF issues. (12) The FDA reviewed a sample of emails between the sponsor and investigational sites – interaction that was completely unexpected – and concluded that the communications included management suggestions that went beyond the approved study protocol. (12) This conduct may have biased the study results towards reductions in hospitalizations in the treatment group. In essence, the intervention was a combination of the device and more generalized disease management.

Additionally, even though the primary effectiveness endpoint was reported as positive, the FDA determined that statistical methods used by the device sponsor were not robust. Using an alternative bootstrap model, even 2 additional hospital stays in the treatment group would have caused the p value to exceed 0.1. (13) Regarding safety, the low rate of complications was reassuring, but since all patients received CardioMems implants, there was not a true control group for the 2 safety endpoints that were compared to pre-specified Objective Performance Criteria. (13) Further, there appeared to be a difference in benefit by sex: women in the treatment group had a non-statistically significant increase in HF hospitalizations compared with controls. The device sponsor argued that this may have been related to excessive deaths in the female controls, who then had a reduced possible study duration for HF hospitalizations. Regardless, the FDA Advisory Panel concluded that data were insufficient to assess the device’s impact on women. (13) Ultimately the FDA Advisory Panel voted that benefits did not outweigh risks in 2011, and the FDA sent a “not approvable” letter to CardioMems recommending a new prospective clinical trial.

Instead of commencing the new study, CardioMems – with FDA input – ended sponsor-site communication and followed patients longer. These additional follow-up data – collected after randomization had been revealed - led the company to conclude that the device’s benefit persisted even without sponsor interaction with individual site physicians. However, the FDA commented that these follow-up analyses were ancillary because the study’s success criteria were not defined a-priori. (14) Although p-values were provided for those new analyses, there was no attempt to adjust for the multiple comparisons and, thus, any statistically significant device-related benefits are not as rigorous as they would have been in the first (randomized) phase of the trial. (14) Further, at the time of review, 37% of initially implanted patients did not continue in the study after the initial randomized period and results for these patients were not included in the follow-up analyses. (14) To summarize, the FDA stated, “it is difficult, at best, to accurately estimate how many HF-related hospitalizations were avoided by the nurse communications.” (14) Ultimately after considering these data, the FDA’s Advisory Panel voted 7-4 that there was not a reasonable assurance that CardioMems was effective in pre-specified patients, but 6-4 (1 abstention) that the device’s benefits outweigh its risks. (15) The FDA approved CardioMems in May 2014. The best one can say is that an expert advisory committee with access to all the data delivered a split decision – and conveyed a strong lack of consensus about whether there was a benefit from the presented evidence.

Thus, CardioMems was approved with lingering uncertainty regarding the benefits underlying the cost-effectiveness data used by Sandhu et al. Additional concern may be related to quality of life, measured in the CHAMPION study using the Minnesota Living With Heart Failure Questionnaire. Sandhu et al. used only 6-month differences in quality of life because half of patient scores were missing at 12 months. The limited 12-month data demonstrated no statistically significant difference in quality of life. (14) Further, the Minnesota Living With Heart Failure Questionnaire does not ask about treatment burden, which could be high for patients given that CardioMems involves a daily measurement.

Recently published data show that less than half (246) of patients completed the most recent open-access period; 166 (30.2%) of the initial randomized patient population died – a high rate but not unexpected given that patients had advanced heart failure. What is concerning is that 69 (12.5%) of the patients withdrew consent while an additional 28 had non-compliance and 17 were lost to follow-up. (16) Indeed, the FDA stated that increased non-compliance “raises questions of whether subjects will continue to comply with the device use requirements as time progresses following implantation.” (14) Given that Sandhu et al.’s model depends foremost on a durable device benefit, the fact that patients with this permanently implanted device are withdrawing consent or being deemed non-compliant would certainly diminish its cost-effectiveness.

Sandhu et al.’s model also relied on a CardioMems-driven reduction in heart failure hospitalizations leading to a mortality benefit. If hospitalizations are prevented (a controversial assumption) but there is no survival advantage, then the cost would be more than $150,000 per QALY gained – making CardioMems a low-value intervention. CHAMPION treatment patients had a trend towards improved survival, but this did not reach statistical significance and the trial was not powered to mortality. (16) Thus, a survival benefit to CardioMems remains a crucial question.

Important uncertainties remain about CardioMems’ ability to reduce HF hospitalizations, improve quality of life, and reduce mortality. The only new planned study of the device is an observational 1200-patient post-marketing observational cohort with an estimated completion in June 2020 – (https://clinicaltrials.gov/ct2/show/NCT02279888) -which will not answer these questions. We believe that a prospective, randomized single-blind trial free of sponsor contact with individual study investigators is required in order to know, with more certainty, what the device can do. Until then, any cost-effectiveness calculations are only helpful in showing what the value would be if we knew the device had benefit. Conviction in CardioMems’ cost-effectiveness requires conviction in its effectiveness.

ABBREVIATIONS AND ACRONYMS

CHAMPION

CardioMEMS Heart Sensor Allows Monitoring of Pressure to Improve Outcomes in NYHA Class III Heart Failure Patients

CHARM

Candesartan in Heart Failure-Assessment of Reduction in Mortality and Morbidity

FDA

Food and Drug Administration

HF

heart failure

NYHA

New York Heart Association

QALY

quality-adjusted life year