The Role of NETT in Emphysema Research

Abstract

Lung volume reduction surgery (LVRS) is one of a long lineage of surgical approaches to emphysema. The reintroduction of this operation in the mid-1990s led to great controversy over the value of the procedure and its long-term outcomes. The National Emphysema Treatment Trial (NETT) represented an historical scientific collaboration of the National Institutes of Health (NIH), the Centers for Medicare and Medicaid Services (CMS), and the Agency for Health Research and Quality (AHRQ). NETT was designed primarily as a pivotal surgical clinical trial, but also incorporated data collection to inform health policy and cost-benefit analyses. NETT faced challenges that included practical and ethical matters, statistical design and analysis issues, and intense public and political scrutiny. The study design required the development of methods for pulmonary rehabilitation, lung imaging, and exercise testing that have become templates for current clinical and research practice. During the course of the trial, the confidential deliberations of the Data and Safety Monitoring Board (DSMB) played an important role in the ultimate success of the trial and protection of research participants. Because of the importance of the NETT outcomes, the results were disseminated to the medical community and transformed into health policy in a rapid and efficient manner. In many ways, the story of NETT serves as a model for evaluation of new surgical approaches to chronic diseases.

The National Emphysema Treatment Trial (NETT) is a landmark study. The purpose of this article is to put this study into the context of clinical research in general and on emphysema research in particular. Specifically, we will outline how the NETT originated, how challenges and obstacles to the design and conduct of the study were met, and how NETT has contributed to our current approach to emphysema research (Tables 1 and 2).

TABLE 1.

MILESTONES IN NETT

June 1996: RFP issued

Dec 1996: Funding started

Oct 1997: First patient screened

Jan 1998: First patient randomized

May 2001: High risk group found

July 2002: Last patient randomized

May 2003: Main outcomes published

Aug 2003: CMS coverage memorandum

Jan 2004: LVRS covered by CMS

Definition of abbreviations: CMS = Centers for Medicare and Medicaid Services; LVRS = lung volume reduction surgery; NETT = National Emphysema and Treatment Trial; RFP = request for proposal.

TABLE 2.

ACCOMPLISHMENTS OF NETT

Largest pulmonary surgical trial ever conducted

Policy study as well as clinical trial

Ethical issues regarding surgical trial

Set standards for rehabilitation in multisite trial

Set standards for computed tomography acquisition, data transfer, storage, and quality control

Set standards for CPX and 6-min walk

Interagency trial supported by NIH, CMS, AHRQ

Rapid dissemination of results

Scientific contributions about clinical epidemiology and pathobiology of emphysema

Definition of abbreviations: AHRQ = Agency for Health Research and Quality; CMS = Centers for Medicare and Medicaid Services; CPX = cardiopulmonary exercise test; NIH = National Institutes of Health.

HISTORY OF EMPHYSEMA SURGERY

Lung volume reduction surgery (LVRS), the treatment of interest in NETT, is only one of several surgical therapies that has been attempted in COPD (1, 2). At the beginning of the twentieth century, attempts to reduce the size of the chest wall in patients with emphysema by resection of costal cartilage were performed with little success. After World War I, many technical advances were made in the ability to perform thoracic surgery—improved anesthetic agents and monitoring, the assessment of operative risk with pulmonary function testing, the development of isolated lung ventilation, means of controlling hemorrhage, and preoperative bronchoscopic and radiographic assessments. With these advances, it became possible to resect lung tissue for suppurative and granulomatous infections as well as neoplasm with high but acceptable surgical risks. Between the 1930s and the 1960s, several thoracic surgeons attempted surgical approaches to emphysema, based on the general notion that the primary physiologic derangement of emphysema was hyperinflation of the lungs and easy collapsibility of central airways (3–5). Such approaches included surgery to enlarge the chest wall (transverse sternotomy); surgery to shrink the chest wall and collapse bullae (thoracoplasty or phrenicotomy); stimulation of growth of new lung tissue through increased pleural blood supply (pleurodesis); and mechanical stabilization of central airways (stents). Neural pathway interruption (via sympathectomy, vagotomy, carotid body resection [glomectomy], or hilar denervation) was attempted to reduce the bronchospasm, mucus secretion, and dyspnea (6). Some invasive approaches involved means of shrinking the lung to increase elastic recoil. These included radiation therapy to induce fibrosis of emphysematous lung regions, surgical bullectomy, and catheter drainage of large bullae (Monaldi procedure) (7, 8). Although many of these procedures were initially reported to have successful outcomes, none of them has remained in general use except for use of bullectomy for isolated giant bullae (9, 10).

One approach to surgical treatment of emphysema was proposed by Otto Brantigan, a Professor of Surgery and Anatomy at the University of Maryland. He concluded that removal of all of the damaged regions of the lung was not possible. However, he proposed that the resection of some portion of lung tissue would restore the diaphragm to a more normal and effective domed configuration, would tether open the airways because of increased elastic lung recoil, and would reverse compression of more normal tissue by distended bullae. Dr. Brantigan performed bilateral thoracotomy for resection of lung parenchyma (pneumectomy), usually from the apex or superior segments. The goal was to improve the functioning of the remaining lung. Between 1957 and 1961, he operated on 56 patients, reporting that most of them had substantial improvements in dyspnea. He made no systematic physiologic measurements, however, to document the objective magnitude of the improvement. Overall, however, because of the substantial 16% surgical mortality and the additional 10% post-surgical mortality, he abandoned the operation(11–13).

In 1991, Wakabayashi stimulated renewed interest in lung volume reduction surgery when he reported results of 22 patients who underwent thoracoscopic laser ablation of emphysematous tissue. He had a 10% early mortality, but survivors had a 35% increase in vital capacity, and a 43% increase in FEV₁ (14). He had similar, though less impressive, results in his next 500 cases (15). However, interest in laser ablation of emphysema decreased after 1995 because of the technical facilities required to perform the procedure and the report of late, and sometimes fatal, complications with pneumothorax. Moreover, the reintroduction of LVRS was touted to be a more effective and safer procedure (16).

The Washington University group, one of the most active lung transplantation centers at that time, reintroduced LVRS, initially conceiving of it as a method of reducing the overinflation of the native emphysematous lung following transplantation of the contralateral lung. Brantigan's vexing complications of prolonged air leak were diminished by the introduction of the pericardial strip stapler, which provided buttressing along the excision staple line. Subsequently, it was recognized that the procedure could be effective even in the absence of transplantation. The initial series of 20 patients with COPD showed dramatic benefit (17). Surgical mortality did not occur, and patients demonstrated an 86% improvement in FEV₁. This was accompanied by reduction in oxygen requirements, improved 6-minute walk distance, and testimonials of dramatic benefit. These results were presented at a national scientific meeting and widely reported in the news media along with patient testimonials. Thus, even before the publication of the peer-reviewed article, there was substantial demand for this procedure by patients desperate for improvement in their symptoms. Many hospitals developed programs for LVRS that were widely advertised to the public (18). For four months in 1995 and 1996, the Health Care Financing Administration (now Centers for Medicare Services [CMS]) issued a CPT code for the procedure. Within that period, 711 Medicare beneficiaries underwent the procedure under that CPT code, and likely many more under other procedure codes. The outcomes, assessed from the Medicare database, were sobering. After 1 year, the mortality was 26%, hospitalizations were prolonged more than 30 days in 16% of patients, and 40% of the patients required readmission to the hospital within 15 months (19).

ORIGINS OF NETT–GOVERNMENT INTERAGENCY COLLABORATION

Because of the disparate outcomes, HCFA asked for a review of the procedure by their Technology Assessment Group, which resulted in the conclusion in December 1995 that LVRS lacked evidence of efficacy. Based on this assessment, HCFA revoked the CPT code, effectively denying reimbursement for the procedure for Medicare beneficiaries. This was greeted with an outpouring of dismay from physicians, patients, and patient advocates who accused the government of making this decision for the sole purpose of saving money. CMS countered that it did not consider cost in its coverage decisions, but was barred by statute from providing payment for procedures that were not beneficial. However, some early estimates suggested that the procedure might apply to 1 to 2 million patients in the United States with a cost of $30 to 60 billion dollars, certainly bringing the value of such surgery to the attention of third-party payers.

In September of 1995, NIH-NHLBI convened a workshop to review the evidence regarding LVRS (20). The main conclusion from that workshop was that the procedure was promising, but that a clinical trial was required to prove efficacy and to determine which patients might benefit the most. Under the intense controversy and spurred by political pressure to resolve the issue, HCFA, now renamed CMS, conferred with NIH about the possibility of conducting a clinical trial to evaluate the new surgical procedure. This is believed to be the first time that CMS approached NIH to conduct a clinical trial, with the agreement that NIH would fund the research costs of the trial and CMS would fund clinical costs. CMS was permitted to fund the clinical costs of LVRS only in the context of clinical research. Thus, CMS agreed to pay for LVRS only for Medicare beneficiaries who were enrolled in a clinical trial. This approach was supported by legislation, largely intended for cancer trials, that required CMS to pay for clinical care costs for patients enrolled in NIH-sponsored trials.

In June 1996, NHLBI issued a Request for Proposals for investigators to design and conduct a clinical trial of LVRS. To assess the costs and outcomes of LVRS, the Agency for Healthcare Research and Quality (AHRQ) developed a contemporaneous proposal for conduct of an economic analysis of the procedure. Thus, without precedent, the three major government agencies involved in healthcare financing and research embarked upon the coordinated research of an innovative, costly, and potentially important treatment for emphysema.

ETHICAL CONCERNS

The stipulation that LVRS would be paid only for NETT participants lead to considerable controversy. A prominent pulmonary physician testifying before Congress suggested that this was similar to a “Tuskegee” study for the elderly—forcing retirees to participate in research to obtain medical care (21). Others raised concerns that performance of an unproven or experimental operation without informed consent was closer to the ethical breaches of the Tuskegee study (22). Because of this controversy, a blue-ribbon ethics panel, under the chairmanship of Robert Levine of Yale University, was appointed by the NIH-NHLBI to evaluate the ethical construct of NETT. This panel concluded that it was ethically sound to provide LVRS only within the context of a clinical trial, and cited the long precedent of providing new drug treatments to patients who might benefit only in the context of clinical research. Although it was recognized that most surgical procedures are not subjected to clinical trial evaluation, precedents included some of the early Veterans Administration trials of coronary artery bypass surgery.

Another issue that was presented to the panel arose from consideration of a policy to exclude the performance of LVRS outside of the NETT trial by participating hospitals, even if the surgeons were not NETT investigators. The purpose of this proposed policy was to prevent conflicts by which hospitals in the NETT might circumvent randomization of participants and perform the surgery on a private-pay basis. This policy was, in part, instigated by events such as the circulation of advertisements throughout the state of Maryland by one of the participating NETT surgeons offering LVRS on a private-pay basis. Concern was also expressed that individuals participating in NETT should be in a state of equipoise with respect to the procedure. The panel concluded that the policy was ethical under a contracting arrangement to prevent conflict of commitment as long as both parties agreed to the arrangement. There was not a specific requirement, however, that all investigators should have equipoise about the benefits of LVRS as long as there was sufficient doubt among the expert medical community. However, the panel observed that enforcement of such a policy by surgeons who did not directly participate in NETT would be impractical. Therefore, the policy was altered to exclude only those surgeons who were directly involved in NETT from performing LVRS outside of the research protocol and to require the principal NETT pulmonologist at each participating clinic to refer patients who wanted LVRS outside of NETT to other physicians for emphysema treatment. Participating hospitals were asked to keep a log of LVRS operations conducted outside of NETT to monitor whether potential research participants were being diverted. Thus, in addition to the usual issues related to protection of human research subjects, NETT specifically had to address questions about the conduct of trials of experimental surgery and the necessity for individual equipoise and commitment by investigators.

Another concern, both practical and ethical, was caused by the Medicare anti-kickback policy that forbids treating physicians and hospitals to forgive the deductible costs or the 20% Medicare co-payment. Under this policy, a physician or hospital who fails to use due diligence to collect this co-payment is subject to prosecution, fines, and imprisonment. Many of the investigators and participating hospitals thought that it was inappropriate to charge patients for participating in research, and especially unfair to charge more to those randomized to surgery than those randomized to medical treatment. This issue was never completely resolved as CMS was not able to change the statutory requirements for co-payment, and presidential memoranda and legislation permitting Medicare payments for treatment of individuals enrolled in clinical trials did not forgive this requirement. However, it was permitted for individual physicians and healthcare organizations to seek an advisory opinion from the Office of Inspector General of Health and Human Services that they would not likely be prosecuted if they did not seek co-payments from research participants. Such an opinion was granted for many of the NETT investigators, and others relied upon that precedent although they did not secure individual opinions. Most, but not all patients, were forgiven co-payment charges. In most cases in which co-payments were charged to research participants, it was covered by co-insurance, and patients were informed at the outset that they might be liable for such payments if they did not have such coverage. To date, no NETT investigators or hospitals have been prosecuted for failing to charge the research participants for co-payments.

NETT AS AN ECONOMIC AND POLICY STUDY

From its inception, NETT was designed as a clinical trial to inform health policy and guidelines as well as to determine clinical efficacy. AHRQ supported an Economic Analysis Center at the University of Washington under the direction of Scott Ramsey, M.D., Ph.D. for analysis of healthcare costs related to LVRS and the control medical arm. Direct patient costs were available through CMS databases for nearly all participants, but required development of custom software and identifier codes to associate the clinical charges with the NETT participants. Moreover, data were collected as part of the study design that permitted calculation of quality-adjusted life-years using the Quality of Well-Being instrument (QWB) (23). Indirect costs were estimated by collecting information on caregiver burden and days of incapacitation. Because these data were collected prospectively, it enabled the cost-benefit outcomes of the trial to be analyzed and published contemporaneously with the main outcome (24). Thus, the medical community had simultaneous detailed information on clinical outcomes as well as the individual and societal cost implications.

Although embedding a cost-benefit analysis into a clinical trial was not unprecedented, this was the first such NIH-sponsored clinical trial in pulmonary disease to do this from the outset. Conducting this analysis required the considerable cooperation, coordination and effort of the financial sections of CMS, the NETT data coordinating center, AHRQ, and the Economic Analysis Center. Because the charges for LVRS were paid directly by one of the sponsors, it enabled the economic analysis component to be conducted with less imputation of costs than is usual for such analyses—a frequent source of uncertainty and debate in such endeavors.

TREATMENT EFFECTS MONITORING

Data monitoring committees conduct their deliberations in secrecy, devote long hours to reviewing interim data and safety reports, and are charged with protecting the safety of study participants as well as the well-being of the general population that is affected by the disease under study. Little is published about data monitoring committees or how they make decisions, as most of their deliberations are confidential and, in many circumstances, independent of the sponsor or investigators. Often, those who participate on these committees are not given adequate credit or academic reward for their difficult job.

In NETT, the 11-member Data and Safety Monitoring Board (DSMB), chaired by John Waldhausen, M.D., had a particularly difficult task. NETT placed special requirements on this board. Because of the controversy regarding whether LVRS was being unduly denied to Medicare beneficiaries, Congress placed special oversight requirements on CMS, asking them to report regularly to Congress on the progress of NETT, the conduct of other research on LVRS, and how they planned to expand payment for the procedure outside of NETT. Because CMS did not have access to the interim study results, much of this burden fell upon the DSMB. Therefore, every three months, the coordinating center provided the DSMB a summary of the interim scientific literature related to LVRS. During the period of 1996 to 2002, nearly 200 articles were published on LVRS, and were reviewed by the monitoring committee.

Moreover, because one of the major aims of NETT was to determine which patients might particularly benefit from LVRS, the committee was charged with monitoring explanatory variables that might distinguish responders. The original goal was to prove that a certain subgroup was shown to clearly benefit from LVRS, removing them from the trial to allow Medicare coverage for those patients. The opposite occurred. An interim analysis showed that patients with very low FEV₁ (≤ 20% predicted) and either a very low diffusing capacity (≤ 20% predicted) or nonheterogeneous emphysema were at exceedingly high risk for operative death (25). When this subgroup was identified, the DSMB met in emergency session in a late-night teleconference to review the data. They recommended immediate exclusion of this subgroup from the trial.

Although it is not uncommon for a DSMB to terminate a clinical trial early because of early proof of benefit or harm, it is nearly unprecedented for a DSMB to eliminate a subgroup based on interim exploratory analyses. Because of the large group of potentially harmed or benefited subgroups, it is always possible that such a decision could be based on a statistical false positive result. In making this decision, therefore, the DSMB had to make a rapid, yet detailed review of the underlying dataset including review of outliers, secular trends, alternative explanations, center differences, and detailed case reports of adverse events. Ultimately, the decision to terminate the subgroup rested on the interplay of statistical inference, ethical considerations, and sound clinical and physiological assessments. The decision to withdraw this group had important effects on the subsequent conduct of the trial as well as CMS coverage for LVRS.

The logistics of immediate cessation of a clinical trial subgroup are complex, and involve the identification of current enrollees at risk to cancel planned randomizations and scheduled surgeries, and rapid communications with investigators, institutional review boards, patients, and the medical community. These considerations are detailed in a separate report (26).

SETTING STANDARDS FOR PULMONARY REHABILITATION

Pulmonary rehabilitation was considered by many, though not all, of the NETT investigators to be a useful component to pre-operative preparation for LVRS. It was thought that patients who successfully completed a rehabilitation program were most likely to tolerate surgery well and have the best response. Because of the well-established benefit of rehabilitation on exercise capacity, it was decided that both the medical and the surgical arm should undertake the same rehabilitation programs before randomization. To qualify for randomization, participants were required to complete a majority of the prescribed rehabilitation program, but were not required to reach any pre-specified level of exercise capacity. The design of the NETT rehabilitation program involved the completion of at least 4 of the required 16 sessions to be performed at the NETT center. However, because many of the patients lived at distance from the center, a national network of 539 rehabilitation centers across the United States were identified and certified to conduct the NETT rehabilitation protocol. Through the process of developing the NETT protocol, it was possible to develop an expert consensus on what components of a multidisciplinary procedure were essential for patients undergoing LVRS. Ultimately, the components of the NETT rehabilitation protocol have been incorporated into the policy for rehabilitation before LVRS promulgated by Medicare and implemented through JCAHO. Thus, a byproduct of NETT was to standardize components of COPD rehabilitation among a nation-wide network of providers. Although NETT required rehabilitation in both groups, it did not lead to a national CMS coverage decision for pulmonary rehabilitation for COPD outside of LVRS. This was disappointing to some proponents of pulmonary rehabilitation, although it was not unreasonable, since the rationale for rehabilitation was to condition patients for surgery. Rehabilitation was also provided to the medical arm in tandem to permit assessment of the effects of LVRS controlling for the effect of exercise conditioning.

Although many experts in LVRS touted successful pre-operative rehabilitation as a condition for undergoing the procedure, the NETT findings were surprisingly quite the opposite. Those patients who did not respond to rehabilitation and had the lowest exercise capacity were the ones who achieved the greatest benefit from LVRS. Men who could not achieve a maximum exercise of more than 40 watts and women who could not achieve a maximum exercise of more than 25 watts were at highest risk of death during medical therapy and had better survival if they undertook LVRS—particularly those with emphysema predominantly in the upper lobes. The reason that this finding had been overlooked was that patients who did not respond to rehabilitation had a relatively poor prognosis with medical treatment alone. This group showed the greatest relative improvement in survival with LVRS, although the overall prognosis was still poorer than that of other subgroups.

EXERCISE TESTING, 6-MINUTE WALKS, AND LUNG FUNCTION TESTING

As a multi-site research study, NETT had to develop detailed procedures to perform physiologic testing on participants that could be feasibly deployed at the clinical centers and could be standardized between centers and over time. Several of these procedures that had to be developed specifically for the NETT trial influenced standards for these procedures that were later promulgated among the entire medical community.

Exercise capacity, measured as maximum ergometer exercise, was a co-primary outcome of NETT. Because the investigators did not want changes in gas exchange or oxygen requirements to confound the outcomes, it was decided to test all patients using supplemental oxygen with an Fi_O2 of 30%. This required development of a customized system designed by Mr. William Slivka to deliver this oxygen while still being able to measure ventilation and gas exchange. Because of the potential for this system compromising the measurement of oxygen consumption if there was a leak across the mouthpiece valve, oxygen consumption was not used as a primary outcome. Therefore, NETT relied upon the measurement of maximum watts recorded by electronically braked ergometers. This led the investigators to explore in more detail the characteristics of the ergometers in current use. The initial design to use continuous ramp increments of work was abandoned for step increases when it was discovered that one of the widely used commercial exercise systems that claimed to be providing continuous ramp increases in exercise were actually stepping it up in 5-watt increments, but producing a record that suggested it was providing continuously increasing exercise ramp. Accordingly, a 5- or 10-watt/minute stepped incremental exercise protocol was adopted. Calibration of the ergometers proved problematic when it was found that the commercially available ergometer calibrator did not provide reproducible readings over the lower range of exercise anticipated in many of the NETT participants. Therefore, the investigators relied upon using oxygen consumption in normal control subjects (biologic calibration) to verify the ergometer accuracy and stability.

Ultimately, Medicare guidelines have required that LVRS beneficiaries undergo exercise testing in accordance with the NETT protocol, suggesting that the testing procedures designed for this research protocol will be the standard (27).

The 6-minute walk test (6MWT) had been widely used for assessment of disability in patients with cardiac and respiratory disorders, and many NETT centers had been using this test to assess outcomes (28). Comparison of procedures used at the NETT centers, however, revealed that there was a wide range of methods used for the test, though most cited the same reference (29). In development of the NETT protocol, standards were developed for the test. Many of these standards were later incorporated into the ATS standards. These included specifics of course length and layout, scripted coaching instructions, and the lack of need for a preceding practice walk (30).

Lung volume testing using body plethysmography was used in NETT as an entry criterion, but was subject to considerable variation in technical details. Development of the NETT protocol required standardization of these methods. These included performance of linked TGV–ERV–vital capacity maneuvers, and methods for calculating the subdivisions of lung volumes by subtracting the mean ERV from the mean FRC to obtain the RV. The TLC is calculated as the sum of the RV and the largest vital capacity. Subsequently some of these standards were incorporated into the ATS/ERS guidelines, and likely will be incorporated into commercial systems (31).

LUNG IMAGING IN A MULTICENTER STUDY

Severity and distribution of emphysema were entry criteria as well as proposed subgroup descriptors for NETT. Although computed tomography (CT) measure of emphysema was a mature technology at the time of initiation of NETT, there had not previously been a large clinical trial that employed these measures across a number of centers. The investigators decided to use a visual scoring method modified from that used by Nishimura (32). Criteria were developed for the methods of acquisition of CT scans that could be employed among the different platforms available at the clinical centers. A reference set of scans were provided and radiologists were certified for reading and grading of emphysema severity and distribution.

Although the primary radiographic classification was based on semiquantitative visual grading, it was apparent that there needed to be a central imaging reading center to review the quality of the scans, store the data, and to provide quantitative image analysis. The University of Iowa provided this service as the NETT Image Analysis Center (IAC) under the direction of Dr. Eric Hoffman. The transmission and storage of the large amounts of raw data acquired in a CT strained the limits of information technology at the time that the study was initiated, although present computing advances have made this routine. The IAC had to deal with a number of different reconstruction algorithms and DICOM file formats, and needed to develop custom software to de-identify the data to meet requirements of the newly enacted HIPAA regulations. Ultimately the Image Analysis Center has provided a model for subsequent NIH-sponsored multicenter emphysema studies that made use of quantitative image analysis such as FORTE (33) and the Lung Tissue Research Consortium.

STATISTICAL DESIGN CHALLENGES

At the time that NETT was initiated, there seemed to be little doubt among NETT investigators that a percentage of patients who survived LVRS for six or more months had an improvement in FEV₁. There was more doubt, however, about the durability of the effect, and the magnitude of surgical mortality, and whether those who survived surgery had improved longevity and quality of life. Therefore, the study was designed as a survival study. There were two major considerations in this design. First, there was a mandate to secure the information as quickly as possible so that CMS could make a coverage decision. Second, it was widely assumed by the investigators that large numbers of symptomatic patients who were asking for the surgery would join the trial, and that many of those randomized to medical care would seek LVRS through private funding. Thus, the initial sample size models made the assumption that NETT would enroll 2,500 patients and that 30% would crossover from medical to surgical treatment (34). The possibility of using sham surgical controls to avoid excessive crossover was easily rejected by the investigators on both ethical and practical grounds (35). The investigators, therefore, were charged with selecting participants who were truly ambivalent about their preference for surgical versus medical treatment. Because of this, it was much more difficult to enroll patients in the numbers anticipated, but ultimately only 5%, not 30%, of patients crossed over from medical to surgical therapy. In the end, 1,218 patients were enrolled, but only 5.4% of patients crossed over from medicine to surgery and only 4.6% assigned to surgery declined or were unable to have surgery. Moreover, the initial estimates of mortality were 8% per year based on published case series, and the NETT experience was 11% per year. These two factors preserved the initial power of the study, with a calculated number needed to enroll of 1,190 patients.

Interpreting Survival Analyses in Surgical Trials

The final statistical analysis plan for NETT required the consideration of several factors that are inherent in surgical versus medical trial designs. One of the assumptions of standard survival analyses is that the risk for both groups is constant over time. In reality, surgery is expected to have a higher initial mortality than medical treatment with the expectation that it will ultimately extend overall survival (36). Ultimately, this becomes a value judgment between an individual physician and patient to ascertain what initial risk is worth the possibility of long-term gain. Thus, one of the goals of NETT was to define these risks and benefits as carefully as possible for defined patient groups to inform individualized decision-making.

Differential Loss to Follow-up

Surgical versus medical trials are subject to bias and informative differential data loss insofar as patients and their physicians know exactly what treatment they are receiving. In NETT, we experienced this phenomenon in two ways. First, because the early surgical mortality might constitute a “harvesting” effect of the most impaired individuals, post-operative comparisons needed to consider this. Second, fewer individuals from the medical treatment group than the surgical treatment group attended follow-up visits. The reason for this was speculated to be either because the participants were disappointed that they did not receive the surgical treatment or because the participants were too ill to travel to the clinical center. Both of these nonrandom factors, however, certainly lead to informative loss of data that may alter the outcome analysis. Ultimately, there is no statistical method to account completely for this. However, the analysis plan in NETT used a nonparametric rank-order analysis that permitted inclusion of patients who were dead or missing into the analysis. This not only allowed all of the participants to be included in the final analysis, but permitted sensitivity analyses (e.g., assuming missing patients to be either very good or very bad outcomes) to determine whether the missing patients contributed important information to the final analysis. With this approach, we could conclude that the outcomes in NETT were not substantially influenced by informative data loss.

Subgroup Analyses

One of the goals of NETT was to evaluate the characteristics of patients that were likely to benefit from LVRS. A total of 16 pre-determined factors were analyzed during the course of the trial. In the end, the two dominant factors that showed significant interactions with treatment group assignment were the presence of upper-lobe emphysema, and poor exercise capacity following rehabilitation. The final analysis of the main results used these two dichotomous variables to provide guidance for selection of LVRS patients, based on the four possible subgroups. While this is useful for guidance of physicians and patients, the analysis of subgroups is always subject to some concern about false-positive results (37). In part, this is guarded against by the pre-specification of the proposed subgroup analyses, and the biological plausibility of the outcomes. As indicated in the editorial accompanying the main NETT outcomes (37):

Such ambiguity is an inevitable result of good clinical research. In this well-designed and well-conducted trial, the investigators went beyond an overall comparison of outcomes in the two treatment groups to search systematically for subgroups of patients who might benefit from, or be disadvantaged by, surgery. Findings from such explorations are rarely definitive, but they do offer clues for future research and guidance to clinicians.

It is not likely that another trial like NETT will be performed to test the findings in the subgroups thought to benefit, so this analysis will likely remain the main guidance for LVRS patient selection for the foreseeable future.

DISSEMINATION OF RESULTS

Because of the importance of NETT with regard to clinical care decisions as well as policy, there was considerable motivation to evaluate and disseminate NETT outcomes in a prompt fashion. In part, this was facilitated by the design of the trial that did not follow the most common practice of having each patient followed for a pre-specified time period. All patients were followed for as long as the trial was active, with a minimal duration of follow-up of 6 months. This design permitted the greatest economy of time to completion, although it weighted the outcomes more for those who were enrolled earlier.

It can often take a year or more for clinical trial results to be published after the final patient completes. When the initial finding of the high surgical risk subgroup was found, it was deemed important to disseminate this as quickly as possible, but also to vet the analysis with a rigorous peer-review process. Communicating the potential importance of the finding to the editors of a wide-circulation general medical journal and giving a preliminary notice that a manuscript would be delivered permitted the editors to pre-assign reviewers with strict timelines. After the high-risk manuscript was submitted, reviews from five peer reviewers were received in 9 days. After revisions, which were done with the direct communication with the editorial staff, the manuscript was published online within 6 weeks of the initial submission (26). Thus, NETT, in coordination with enlightened and cooperative scientific editors, was able to use innovations in information technology to produce a peer-reviewed publication in a short time frame. In this way, NETT was able to avoid the problems associated with disseminating such information publicly without adequate peer review or exposition of the underlying data.

Similarly, when the main outcome results were ready for submission, NETT was able to coordinate the detailed presentation of the results at the American Thoracic Society with the publication of the results, and the disclosure of the outcomes to CMS to facilitate a timely policy decision. The New England Journal of Medicine was able to acquire seven peer reviews of the final manuscript within 14 days. Thus, the main findings were available for clinical, scientific, and policy deliberations less than one year after the final patient was randomized into the trial.

Because CMS was well aware of the NETT design details and was kept informed of the progress toward completion of patient follow-up and plans for publication of the results, CMS was able to plan for the rapid development of a coverage decision memo within 90 days of the final results.

SCIENTIFIC ACCOMPLISHMENTS OF NETT

Because NETT recruited a very highly characterized group of patients with advanced emphysema, analysis of NETT data has provided a number of important insights into the clinical epidemiology and pathobiology of emphysema (see Appendix). Published manuscripts have investigated the demographic, psychological, and physiological correlates of disease with quality of life and functional status. Tissue specimens obtained during surgery provided important observations regarding the role of small airway pathology in emphysema and seminal observations regarding peripheral airway obstruction with mucus and cellular debris. Ancillary genetic studies hold promise for assessing determinants of emphysema development and distribution.

CONCLUSIONS

The NETT trial posed formidable organizational, logistic, ethical, scientific, and practical challenges. In dealing with these challenges, the National Emphysema Treatment Trial developed standards for the way that similar trials can be conducted in the future and provided key scientific information that benefits the many patients who would be helped by lung volume reduction surgery as well as those who would not.