The National Emphysema Treatment Trial (NETT)

Abstract

The National Emphysema Treatment Trial (NETT) was a multicenter, randomized, controlled clinical trial, comparing the efficacy of lung volume reduction surgery (LVRS) plus medical management with rehabilitation to medical management with rehabilitation in 1,218 patients with severe emphysema. The NETT was a precedent-setting collaborative effort of three government agencies: the Centers for Medicare and Medicaid Services (CMS); the National Heart, Lung, and Blood Institute of the National Institutes of Health (NIH); and the Agency for Healthcare Research and Quality (AHRQ). NETT provided Medicare beneficiaries with controlled access to a promising but unproven procedure, while scientifically valid data on the efficacy and costs were collected to guide future use, coverage decisions, and policy. NETT demonstrates that collaboration among federal agencies and among health plans, researchers, and providers can successfully fulfill their differing missions simultaneously and is a productive approach to evaluating new treatments of mutual interest.

The National Emphysema Treatment Trial (NETT) was a precedent-setting cooperative effort between the nation's largest payer, the Centers for Medicare and Medicaid Services (CMS); its premier health research agency, the National Institutes of Health (NIH); and its lead agency for health care quality, the Agency for Healthcare Research and Quality (AHRQ). The goal of the trial, consistent with the missions of NIH and AHRQ, was to provide data for evaluating the safety and effectiveness of lung volume reduction surgery (LVRS) and for guiding selection of patients. Simultaneously, Medicare beneficiaries were provided controlled access to a promising but unproven procedure, while scientifically valid data on the efficacy and costs were collected to guide future use, coverage decisions, and policy. A chronology of the events leading to the trial, and of the events of the trial itself and CMS decisions based on its results, are summarized in Table 1 and elaborated below.

TABLE 1.

CHRONOLOGY OF EVENTS

Jan 1995: Joel Cooper report on LVRS for severe emphysema (1)

Aug 1995: CMS request a technology assessment by AHRQ

Sept 1995: NHLBI workshop, “Evaluation and Research in Lung Volume Reduction Surgery (2)

Oct 1995: CMS issues a temporary billing code for LVRS

Dec 1995: Effective date of CMS national non-coverage policy for LVRS

Winter 1996: CMS initiates discussions with NHLBI about potential for collaborating on an effort to evaluate LVRS

Apr 1996: AHRQ delivers technology assessment to AHRQ

NHLBI and CMS announce plans to collaborate on multicenter randomized trial on the efficacy of LVRS

Jun 1996: NHLBI and CMS sign Memorandum of Understanding regarding the proposed clinical trial

NHLBI releases RFPs for clinical centers and a clinical coordinating center

Sept 1996: Responses to the RFP are reviewed by panel convened by NHLBI

Dec 1996: Contract awardees for clinical trial are announced

Jan 1997: Contract awardees meet and initiate work on the trial

Apr 1997: Congressional hearing to discuss mandated Report to Congress regarding CMS policy on LVRS

Aug 1997: NHLBI approves the protocol approved by the DSMB

Jan 1998: First patient is randomized in NETT

Jun 1998: CMS forwards final Report to Congress

May 2001: Protocol is modified to exclude the patients with very low FEV1 and either homogeneous emphysema or very low carbon monoxide diffusing capacity (10)

May 2003: NETT presents and publishes primary outcome (mortality) results (11) and cost effectiveness analysis results (12)

Aug 2003: CMS announces intention to cover LVRS for patients meeting criteria

Oct 2003: CMS issues revised coverage decision on LVRS

Jan 2004: Effective date of CMS decision to cover LVRS for patients meeting criteria

ORIGINS OF THE TRIAL

Following Dr. Joel Cooper's 1995 report of significant functional improvements after LVRS in 20 patients with severe emphysema (1), the procedure spread rapidly into practice despite the preliminary nature of the results, uncertain effects on morbidity and mortality, lack of consensus on surgical selection criteria, and the absence of data on long-term consequences. Coverage by Medicare of many of these surgeries was possible through their billing as “other lung procedures.”

Many in the medical community were alarmed by this rapid spread of an unproven, invasive treatment for such a sick and desperate patient population and urged the National Heart, Lung, and Blood Institute (NHLBI) of the NIH to investigate the scientific basis for the procedure. In September of 1995, the NHLBI sponsored a workshop, “Evaluation and Research in Lung Volume Reduction Surgery,” bringing together experts in pulmonary medicine, thoracic surgery, clinical trial design, and statistics to recommend a strategy for evaluating LVRS. The participants unanimously agreed that LVRS needed to be evaluated in a scientific, coordinated, cooperative fashion, and that the existing data suggested that LVRS produced substantial improvement in some subjects with severe emphysema. However, because of the rapid, uncritical dissemination of the procedure (especially to centers with limited facilities and/or lacking multidisciplinary thoracic surgical programs), the limited follow-up data, and the need to develop surgical selection criteria, the participants recommended that the Division of Lung Diseases of the NHLBI support data collection and analysis in a multicenter randomized trial with a control (nonsurgical) treatment arm (2).

Simultaneously in 1995, CMS (then the Health Care Financing Administration) staff became aware of the development and rapid growth of LVRS through their normal tracking procedures and began an internal evaluation. Finding only one published article (1) and concerned about risks to Medicare beneficiaries, CMS took the following steps: it requested a technology assessment by the AHRQ (then the Agency for Health Care Policy and Research) in August 1995; issued a temporary billing code for LVRS in October 1995; and issued a national noncoverage policy in December 1995 (3).

CMS received the report from AHRQ in April 1996. Based upon the available literature and analysis of approximately 2,800 patients from 27 institutions, the report found that “…it cannot reasonably be concluded at this time that the objective data permit a logical and a scientifically defensible conclusion regarding the risks and the benefits of LVRS as currently provided” (4).

The report cited consistent weaknesses in the evidence, including the use of case series rather than controlled studies, short duration of follow-up, high rates of attrition in the follow-up, little consensus on patient selection criteria, and insufficient documentation of benefits. According to the report, operative and perioperative mortality ranged from 5% to 10%. An analysis by CMS staff of the outcomes of 711 Medicare beneficiaries who received LVRS between October 1995 and January 1996 also found mortality rates that were higher than those reported in the published literature; 26% of patients died within the year following LVRS (5).

Noting that some percentage of patients with severe COPD did well after LVRS, but also noting the numerous unanswered questions regarding the efficacy, technique, and patient selection for LVRS, the AHRQ report recommended that CMS consider “…a prospective trial of LVRS under uniform protocol requirements with comprehensive long-term postoperative follow-up data.”

RATIONALE FOR CMS COVERAGE OF LVRS WITHIN A CLINICAL TRIAL

The basis for CMS' coverage authority is Section 1862(a) (1)(A) of Title XVIII of the Social Security Act. This section states that no payment should be made for items and services which “are not reasonable and necessary for the diagnosis and treatment of illness or injury or to improve the functioning of a malformed body member.” CMS, like other insurers, uses a rigorous, evidence-based framework to support its coverage decisions. Frequently, however, the evidence is insufficient to draw strong conclusions concerning the relative risks and benefits of services, particularly for new or evolving services. Developing coverage or noncoverage decisions in the face of inadequate evidence can expose beneficiaries to risky services that have limited benefit or deny beneficiaries a potentially beneficial service. In addition, either decision may preclude or complicate further rigorous evaluation of the service.

An alternative to the yes/no paradigm is to allow coverage while the beneficiary is participating in a structured collection of data, for example, a randomized trial. This allows beneficiaries controlled access to unproven, but promising services and facilitates collection of data that will inform subsequent coverage decisions related to the service. In 1996, paying for a service not otherwise covered by Medicare and only within the context of a clinical trial was precedent setting policy for CMS, but a number of factors, some specific to LVRS, made this a compelling strategy to pursue. First, CMS was beginning to apply the principles of evidence-based medicine as the primary framework for coverage decisions. CMS was concerned about gaps in the existing data on LVRS and the applicability to Medicare beneficiaries and was uneasy about broad coverage of LVRS in the face of these concerns. Finally, the unusually rapid spread of LVRS was worrisome in view of the lack of testing of the procedure and the inconsistent results.

The AHRQ recommendation, in conjunction with CMS' examination of the emerging literature and contact with concerned providers, resulted in the conclusion that LVRS fit the criteria necessary for an alternative coverage policy. LVRS was a service of substantial interest to Medicare; there was enough information to make some judgment about basic safety and efficacy, but not enough to make a decision about coverage, and it appeared unlikely that the data needed would be collected by the medical community on its own. By collaborating on a clinical trial, Medicare would be able to provide beneficiaries with controlled access to a promising, but unproven procedure, while ensuring the collection of data needed to inform and direct Medicare's subsequent coverage decision on LVRS.

NATIONAL EMPHYSEMA TREATMENT TRIAL

In the winter of 1996, following publication of the national noncoverage policy (4) and anticipating the AHRQ report, CMS initiated discussions with the NHLBI on the potential for cooperative efforts related to assessing LVRS. After receipt of the AHRQ report, NHLBI and CMS issued a joint press release about their plan to collaborate on a multicenter randomized trial on the efficacy of LVRS (6).

A Memorandum of Understanding (MOU) was signed by both agencies in June 1996, describing the purpose of the clinical trial, the authority of each agency to sponsor it, and the scope of work and responsibilities for each. NHLBI was responsible for negotiating, awarding, and administering all contracts and providing some administrative costs. In addition, NHLBI was to monitor program performance and appoint a Data and Safety Monitoring Board (DSMB) to monitor the accumulating data for treatment effects. The DSMB was advisory to the NHLBI regarding issues such as patient safety and treatment effects.

CMS was responsible for providing reimbursement to the participating clinical facilities for patient care costs related to services received as part of the trial. CMS agreed to structure its coverage policy so that coverage would change during the trial to accommodate any changes in protocol brought on by analysis of the accumulating data. AHRQ subsequently signed an MOU to sponsor and oversee a cost-effectiveness analysis sub-study.

On May 9 and 10, 1996, NHLBI announced the plans for the trial in the Commerce Business Daily (7, 8), informing interested parties that a Request for Proposals (RFP) would be available on or about June 3, 1996 via the NIH RFP Gopher for facilities and clinical coordinating centers interested in participating in the trial.

In September 1996, NHLBI convened a panel of experts in cardiothoracic surgery, statistics, clinical trials, and pulmonary medicine to review the proposals for the clinical centers and the clinical coordinating center. On December 20, 1996, NHLBI and CMS issued a press release announcing the contract awards to 18 clinical centers and one clinical coordinating center (9).

The Contract awardees, NHLBI, CMS, and AHRQ, first met in January 1997 to draft the trial protocol. In August 1997, with the advice of the NETT DSMB, the protocol was approved by the NHLBI to go to the individual Institutional Review Boards; the first participant was randomized in January, 1998.

While the collaboration of CMS and NIH on the NETT was generally well received by the scientific community, it was not free of controversy. Some physicians and patients believed that the existing data had already established LVRS as an effective procedure and therefore, a randomized trial was unethical and Medicare should broadly pay for LVRS as an accepted surgical intervention. These issues were raised in the Congress, and CMS was required by law to report to Congress explaining its policy. In addition, in April of 1997, Congress held a hearing to discuss the mandated Report and provide a forum for public opinion on the issue. Representatives of CMS, NIH, and AHRQ testified, along with Dr. Joel Cooper. The final Report, forwarded to Congress in June 1998, examined all evidence that had become available since the AHRQ report. The conclusion remained the same. While the existing data established some promise for LVRS as a treatment for emphysema, the evidence was insufficient to draw strong conclusions about many of the key risks and benefits to patients. Congress took no further action on this issue.

RESULTS OF THE NETT

In May 2001, the NETT protocol was modified to exclude patients with very low FEV₁ and either homogeneous emphysema or a very low carbon monoxide diffusing capacity. These patients had been found by NETT to be at high risk for death after surgery and unlikely to benefit from LVRS (10). Patients who did not meet these criteria continued to be enrolled in the trial. The primary results from the NETT and the results of the cost-effectiveness analysis were released at the American Thoracic Society meeting on May 19, 2003 and simultaneously posted on the New England Journal of Medicine web site. The print versions appeared in the May 22, 2003 issue (11, 12). The NETT showed a survival advantage for patients with both predominantly upper-lobe emphysema and low baseline exercise capacity. Patients previously reported to be at high risk and those with non–upper-lobe emphysema and high baseline exercise capacity were found to be poor LVRS candidates because of increased mortality and negligible functional gain.

The NETT's rigorous protocol, the large sample size, the prolonged and complete follow-up for mortality, the low drop-out rate, and the extensive, standardized evaluation of many functional outcomes allowed a detailed assessment of the risks and benefits of LVRS for patients with different characteristics. Coordination with CMS during the data analysis and writing of the manuscript allowed CMS to announce its intention to cover LVRS in August, 2003 (13), issue a revised coverage decision on LVRS in October, 2003 (14), and implement coverage of LVRS in January, 2004 (15).

LESSONS LEARNED

The NETT demonstrates that federal agencies can successfully fulfill their differing missions simultaneously when there is a strong mutual interest. The NETT aligned with NIH's mandate to support medical research aimed at reducing the burden of disease and was also an integral part of Medicare's coverage determination. The collaboration with AHRQ added a prospective study of the cost-effectiveness of LVRS as part of the trial.

The success of the NETT model enabled CMS to engage in a number of subsequent decisions that linked Medicare coverage to clinical studies or further data collection. In recent years these included angioplasty of the carotid with stenting, FDG-PET for suspected dementia, and prophylactic use of implantable cardiac defibrillators (ICDs) and off-label use of biologics for colorectal cancer (16). In April 2005 and July 2006, CMS issued Guidance Documents to publicly advance and explain the concept of “coverage with evidence development”—a general description of these types of policies (17). The publication of these documents reflects a sustained commitment by CMS to use their coverage decisions to improve medical evidence for all decision makers.

The NETT demonstrated that while case series provided some information about the risks and benefits of LVRS, only a randomized controlled trial provided detailed information on the relative risks of surgery compared with medical therapy according to an individual patient's baseline characteristics. Without NETT, it is unlikely that the medical community would have extended its understanding of the long-term risks and benefits of LVRS over what was known in 1996. The detailed data provided by NETT on the distribution of favorable and unfavorable outcomes of LVRS now allows patients and their doctors to make informed decisions about the likelihood of benefit.

The NETT provides a reminder of the danger of rapid diffusion into practice of unproven procedures. Without this controlled trial, it is likely that LVRS would have been applied to high-risk patients in inappropriate settings. Further, an assessment of the relative value of LVRS compared with other therapies in affording better functional health outcomes would not have been possible.

Finally, the NETT demonstrates that collaboration between health plans, researchers, and providers is a productive approach to evaluating new treatments of mutual interest. The success of the NETT model provides support for CMS' commitment to its coverage decisions that are closely coupled with evidence based development policy. With this policy, CMS collaborates closely with other federal clinical research agencies to develop evidence that judges the safety and effectiveness of clinical care and simultaneously expedites the access of eligible patient populations to potentially promising new technology.