Population, Clinical, and Scientific Impact of National Cancer Institute's National Clinical Trials Network Treatment Studies

PURPOSE

In the United States, the National Cancer Institute National Cancer Clinical Trials Network (NCTN) groups have conducted publicly funded oncology research for 50 years. The combined impact of all adult network group trials has never been systematically examined.

METHODS

We identified randomized, phase III trials from the adult NCTN groups, reported from 1980 onward, with statistically significant findings for ≥ 1 clinical, time-dependent outcomes. In the subset of trials in which the experimental arm improved overall survival, gains in population life-years were estimated by deriving trial-specific hazard functions and hazard ratios to estimate the experimental treatment benefit and then mapping this trial-level benefit onto the US cancer population using registry and life-table data. Scientific impact was based on citation data from Google Scholar. Federal investment costs per life-year gained were estimated. The results were derived through December 31, 2020.

RESULTS

One hundred sixty-two trials comprised of 108,334 patients were analyzed, representing 29.8% (162/544) of trials conducted. The most common cancers included breast (34), gynecologic (28), and lung (14). The trials were cited 165,336 times (mean, 62.2 citations/trial/year); 87.7% of trials were cited in cancer care guidelines in favor of the recommended treatment. These studies were estimated to have generated 14.2 million (95% CI, 11.5 to 16.5 million) additional life-years to patients with cancer, with projected gains of 24.1 million (95% CI, 19.7 to 28.2 million) life-years by 2030. The federal investment cost per life-year gained through 2020 was $326 in US dollars.

CONCLUSION

NCTN randomized trials have been widely cited and are routinely included in clinical guidelines. Moreover, their conduct has predicted substantial improvements in overall survival in the United States for patients with oncologic disease, suggesting they have contributed meaningfully to this nation's health. These findings demonstrate the critical role of government-sponsored research in extending the lives of patients with cancer.

INTRODUCTION

Cancer is a devastating group of diseases with enormous adverse impacts on population health. Cancer remains the leading cause of lost life-years in the United States, with 9.3 million years of life lost in 2019 alone.¹ For an individual with cancer, the estimated average number of life-years lost is 15.2.² Fortunately, through the combined efforts of better early detection, prevention, and improved cancer treatments, cancer mortality has begun to decrease. The annual percentage reduction in cancer mortality in the United States was 2.1% from 2015 to 2019, twice the annual rate of reduction of 1.0% from 1992 to 2001.¹ Decreasing mortality over the past 2 decades has resulted in the reduction of more than 3 million cancer-related deaths.^3-5 These combined efforts are vital given the aging of the US population and the fact that most new cancer cases occur in individuals 65 years or older.⁶

CONTEXT

• Key Objective

• The National Cancer Institute's National Cancer Clinical Trials Network (NCTN) groups have conducted publicly funded oncology research for more than 50 years. In a collaboration among the four large adult NCTN groups, we systematically evaluated the combined impact of positive randomized trials since 1980.

• Knowledge Generated

• The 162 trials that were analyzed comprised 108,334 patients. These trials were cited 165,336 times through 2020, with 87.7% of trials cited in cancer care guidelines in favor of the recommended treatment. The experimental therapies from the trials were estimated to have generated 14.2 million additional life-years to patients with cancer through 2020.

• Relevance (J.W. Friedberg)

• The impact of US NCTN trials on adult cancer outcomes cannot be overstated; this evidence should compel sustained financial investment and continued academic contributions to this valuable resource.*

  *Relevance section written by JCO Editor‐in‐Chief Jonathan W. Friedberg, MD.

The year 2021 marked the 50th anniversary of the National Cancer Act, signed into law in 1971 with the express purpose to “more effectively … carry out the national effort against cancer.”⁷ The act launched a decades-long effort to combat cancer under the guidance of the National Cancer Institute (NCI) within the National Institutes of Health. A key part of the NCI's mandate is the sponsorship of a set of large, national adult cancer network research groups that combine the efforts of physician-researchers, laboratory scientists, biostatisticians, nurses, clinical research associates, and patient advocates across academic and community cancer centers to conduct clinical trials. This NCI-sponsored National Clinical Trials Network (NCTN) coordinates and supports trials at more than 2,200 sites across the United States and internationally.⁸ These groups have been conducting research paid for by the US government for more than 5 decades, with the goal to identify new, effective treatments for patients with cancer. Significant work is also conducted in the realm of oncology population science, including cancer control and prevention. The groups' main research shares with pharmaceutical company trials the goal of identifying treatments with the potential to improve overall survival. The network groups also compare combinations of agents, test regimens in rare diseases, and assess different modalities, such as surgery and radiation.

While widely thought to conduct high-quality research with potentially meaningful results, the actual impact of all adult NCTN trials has never been systematically assessed. In a first-time collaboration combining data on positive trials conducted by the NCTN adult cancer groups, our aim was to systematically examine and characterize population, clinical, and scientific impact of the NCTN over the most recent 4 decades.

METHODS

Data

We identified randomized phase III trials from the four adult network groups: the SWOG Cancer Research Network, the Alliance for Clinical Trials in Oncology, the ECOG-ACRIN Cancer Research Group, and NRG Oncology. Primary study findings must have been reported from 1980 onward and demonstrated statistically significant results for one or more clinical, time-dependent outcomes (such as overall or progression-free survival) in favor of experimental treatment. Experimental treatments identified as beneficial but that were too toxic for study authors to recommend in the primary publication were excluded.

Information on ethical review and informed consent of participants for each trial were included in study publications. This study relied on previously published trial reports for which patient-level data were not identifiable; thus, institutional review board approval of the study was not required.

Statistical Methods

Population impact.

Population impact—defined by gains in population life-years—was estimated for all trials for which overall survival favored the experimental treatment arm, regardless of whether the benefit was statistically significant. Thus, in several cases, experimental treatment was observed to provide a statistically significant benefit for an intermediate end point (eg, progression-free survival), but a nonsignificant beneficial trend for overall survival. Life-year gains based on such trials were included to provide an empirical translation of intermediate end points into life-years. Life-year gains were also calculated for noninferiority trials if there was improved overall survival for the experimental treatment.

On the basis of a previously published method, for each trial-proven new treatment for a given type of cancer, life-years gained (LYG) at the population (Pop) level was calculated as the product of model-estimated additional life accrued to the average patient (Pt) and multiplied by the number of patients in the cancer population (N_CaPop) who would benefit from the new treatment (ie, LYG_Pop = LYG_Pt × N_CaPop).⁹ Life-years gained for the average patient (LYG_Pt) was estimated by deriving trial-specific survival function parameters depicting the difference in survival between standard and experimental treatments and mapping the benefit of the experimental treatment onto the US cancer population using national cancer registry and life-table data.⁹ For improved representativeness, rather than using the survival outcomes for the control arm for patients enrolled in the trial, the hazard rate for the control arm was estimated using cancer population survival data for incident cases during the trial enrollment period that met trial eligibility criteria. To derive the survival function for the experimental arm, we obtained the hazard ratio for the benefit of new treatment from the trial publication. The benefit of experimental treatment increased average survival during the treatment benefit period as the product of the hazard rate for the control arm and the hazard ratio. In the post-treatment benefit period, average survival for both the control and experimental arms was assumed to extend under a pattern of exponential decay until mean age-specific half-life on the basis of life-table data, a conservative assumption. Average life-years gained on the basis of the new treatment for a given individual was then calculated as the difference in the area under the survival functions (ie, Kaplan-Meier curves) between control and experimental arms from diagnosis to mean half-life.

To derive the number of patients in the cancer population to whom the new treatment would apply (N_CaPop), we matched the major cancer type, stage, tumor characteristic, prior cancer, surgery, sex, and age (ie, ≥ 18 years) eligibility criteria from the clinical trial to corresponding cancer population data using the Surveillance, Epidemiology, and End Results (SEER) program.¹⁰ The number of corresponding patients in the SEER data set was inflated by a factor of 1/P_SEER, where P_SEER is the proportion of the US population the SEER data set represented. Calculations were stratified by 14 five-year age intervals (20-25, …, 81-85, and > 85 years), since life-years vary by age, and were conducted for each year from publication of trial results through 2020.

To derive a 95% confidence limit, we iteratively sampled (using 400 iterations) the coefficient for the treatment effect from each trial, drawing from distributions on the basis of the observed point estimate and its variation under a normal distribution. In the base-case model, we assumed that the treatment benefit period was the first 5 years after diagnosis, that the overall survival treatment effect translated fully (with 100% effectiveness) to the corresponding cancer treatment population defined by the trial eligibility criteria, and that uptake of new treatments into clinical practice occurred in conjunction with the year of primary article publication. In a sensitivity analysis, we allowed the duration of treatment benefit to range from 3 to 7 years in 1-year intervals, the effectiveness parameter to vary from 80% to 120% (since generalizability may be incomplete for all patient groups, or conversely, newly proven treatments may be effectively used off-label) in 10% increments, and the year of adoption to vary from 2 years before trial publication (if adoption occurs early in conjunction with a conference presentation) to 2 years after trial publication (if uptake is delayed, especially for medically disadvantaged groups).^11-13 In an additional sensitivity analysis, to derive the hazard rate for the control arm, we used observed survival outcomes from the clinical trial rather than from cancer population data.

Clinical impact.

Clinical impact was defined by whether trial findings were included as evidence in favor of a recommended treatment in a major clinical guideline or in package inserts for US Food and Drug Administration (FDA) new drug approvals.¹⁴ The primary source was the National Comprehensive Cancer Network (NCCN) clinical practice guidelines from 1996 onward.¹⁵ Trials that supported other major guidelines (ie, ASCO and ESMO) were included to account for earlier years for which NCCN guidelines were not available. To identify whether a trial supported an FDA new drug approval, we generated a catalog of FDA-approved anticancer drugs and obtained the package inserts for any trial for which the experimental agent was included in the catalog.^14,16-18 Trials cited as pivotal in the package inserts were categorized as practice influential.

All determinations were made independently by two authors (R.V. and J.M.U.) with disagreements resolved by a third author (C.D.B.).

Scientific impact.

The primary article for a trial was the article reporting the results of the analysis for the primary protocol-specified end point. Using a bibliometric approach, scientific impact was defined by how often the primary trial report was cited through Google Scholar.^19,20 Totals were summed by year and over time. Additionally, we reported the frequency with which the primary articles were published in high impact (2-year impact factor > 10) journals on the basis of contemporary rankings.

Cost Analysis

Total federal investment funding to conduct the trials and costs per life-year gained were calculated as the sum of estimated funding for all four NCTN groups using publicly available data (Data Supplement, online only).²¹

Impact estimates for all domains were assessed through December 31, 2020.

RESULTS

Study Characteristics

Overall, 544 trials were assessed to have been conducted during the study period, and 189 trials were considered for inclusion in the analysis on the basis of determination by the study team (Data Supplement). Twenty-seven trials were excluded for the following reasons: not a treatment trial (6), positive result but too toxic to recommend (6), not a positive trial (4), positive trial but not for a time-to-event end point (ie, tumor response only; 8), and other reasons (3; Data Supplement). Therefore, 162 trials published from 1981 to 2018 comprised of 108,334 patients were analyzed, representing nearly one third (162/544, 29.8%) of trials conducted by the groups (Data Supplement). A wide variety of cancers were studied, with the most common tumors involving breast (34, 21.0%), gynecologic organs (28, 17.3%), and lungs (14, 8.6%; Table 1). Nearly all trials (155, 95.7%) had superiority designs, and most (130, 80.2%) included chemotherapy. The majority (113, 69.8%) were conducted between 1990 and 2009.

TABLE 1.

Trial Characteristics

Population Impact

Overall, 82.1% (133/161) of trials showed overall survival favoring the experimental arm to some extent, including 92 instances (56.8%) where overall survival for the experimental arm was statistically significantly superior. Through 2020, these trials were estimated to have contributed to 14.2 million (95% CI, 11.5 to 16.5 million) additional life-years (Fig 1). For the same trials, the projected estimates for 2025 and 2030 were 19.0 million (95% CI, 16.1 to 22.3 million) and 24.1 million (95% CI, 19.7 to 28.2 million) life-years gained, respectively (Fig 2).

FIG 1.

Cumulative life-years gained through 2020 by study. Each color-coded area represents cumulative life-years for 1 of 133 studies for which life-year gains were estimated.

FIG 2.

Estimated population life-year gains from National Cancer Clinical Trials Network trials for years 2020, 2025, and 2030.

In the sensitivity analysis varying trial parameters, the range was 7.8-22.9 million life-years gained, with 91.2% of estimates exceeding 10.0 million life-years (Fig 3). In sensitivity analysis that used observed outcomes from the clinical trial to derive the control arm hazard function, the estimated life-years gained through 2020 was 15.3 million, greater than our base-case estimate of 14.2 million.

FIG 3.

Life-years gained by treatment duration, effectiveness, and year of adoption. Although the base-case model assumed that the duration of treatment benefit was 5 years, in a sensitivity analysis, we allowed the treatment benefit period to vary from 3 to 7 years by 1-year intervals. Additionally, the base-case model assumed that the overall survival treatment effect translated fully, with 100% effectiveness, to the corresponding cancer treatment population defined by the trial eligibility criteria. However, the benefits of positive trials may not fully generalize to all patient groups, suggesting that the efficiency with which the efficacy estimate translates to effectiveness in the cancer population may be < 100%.¹² Conversely, newly proven treatments are commonly used off-label in populations of patients not represented in the trial.¹¹ Thus, we allowed this efficiency parameter to vary from 80% to 120% by 10% intervals. Finally, the base case assumed uptake of the new treatment occurred the same year as the primary trial publication. However, uptake of new treatments in the cancer treatment community may precede publication of the article reporting the main trial results, especially if previously presented at a major cancer conference, or uptake may not be immediate (especially for medically disadvantaged patient groups).¹³ We allowed the year of adoption to uniformly vary from 2 years before trial publication to 2 years after trial publication by 1-year intervals.

The estimated total federal investment cost to conduct the trials was $4.63 billion in US dollars (USD) in 2020, or $326 (USD) per life-year gained through 2020.

Clinical Impact

Overall, 87.7% (142/162) of trials were found to have had documented influence on cancer care guidelines, including 26 instances for both gynecologic cancers and breast cancer (Fig 4). The proportion of trials that influenced guidelines was 95.6% (108/113) for trials published after NCCN guidelines were available in 1996 and 69.4% (34/49; P < .001) before NCCN guidelines were available.

FIG 4.

Clinical impact represented by the number of trials that influenced cancer care guidelines or new FDA drug approvals by cancer type.

Scientific Impact

Primary trial results were cited 165,336 times through 2020 (mean, 62.2 citations/trial/year). More than half had 500 or more citations through 2020 (Fig 5). Trial results were frequently published in high-impact journals (146/162, 90.1%), including the Journal of Clinical Oncology (77, 47.5%), the New England Journal of Medicine (49, 30.2%), The Lancet (6, 3.7%), Blood (5, 3.1%), JAMA (3, 1.9%), Journal of the National Cancer Institute (3, 1.9%), The Lancet Oncology (2, 1.2%), and JAMA Oncology (1, 0.6%).

FIG 5.

Total number of citations by study through 2020. Each vertical line represents the citation totals for a single study, in increasing order.

DISCUSSION

This study, representing the first time that the cumulative survival benefits of phase III trials across all adult cooperative groups has been examined, demonstrates that NCI-sponsored NCTN research has contributed meaningfully to extending the lives of patients with cancer, at a low cost. The 162 trials we examined contributed to gains of an estimated 14.2 million life-years to patients with cancer in the United States at a federal investment cost of $326 (USD) per life-year gained. The same studies are projected to contribute 24.1 million life-years by 2030. Most of the trials (87.7%) influenced guideline care recommendations, and the trials contributed enormously to the scientific literature, with primary trial reports nearly all published in high-impact journals and cited more than 165,000 times through 2020.

The NCTN groups are a vital component of the scientific infrastructure of the United States. Their genesis—accelerated by the 1973 National Cancer Act—has been a key driver in reducing the mortality rate from cancer. Since 1991, the mortality rate due to cancer in the United States has decreased by 31%.²² This reduction is partly attributable to improved screening and early detection, improvements in diagnosis, and the development of prevention strategies and interventions, but advances in treatment have also been critical and have been documented in recent assessments. From 2013 through 2017, cancer death rates decreased 1.5% on average, including 2.0% for Black persons.³ Recent improvements have been particularly apparent for certain diseases such as melanoma and lung cancer.^23-26 To contextualize the findings of this study, a gain of 14.2 million life-years through 2020 due to the contributions of NCTN trials has returned 4.2% of the 336.8 million years of life lost due to cancer from 1980 to 2020 in the US population (Data Supplement).

These life-year gains were derived from a minimal federal investment. A study by Islami et al²⁷ showed that the years of life lost in 2015 (8.7 million) for persons age 16-84 years in the United States resulted in an estimated $94.4 billion (USD) in lost future earnings. Another study showed that the national expenditure for cancer care in the United States in 2015 was $183 billion (USD) and is projected to increase to $246 billion (USD) by 2030.²⁸ Set against these estimates, the investment of $4.63 billion (USD) in the conduct of clinical trials by the NCTN groups over 40 years seems comparatively small.

The mission of the NCTN groups is to change clinical practice and to improve outcomes for patients.⁸ As shown, most NCTN trials with positive clinical end points (87.7%) informed guideline care. Importantly, nearly all trials (95.6%) reported since 1996—when comprehensive NCCN guidelines became available for review—were identified as having documented influence on cancer care guidelines, suggesting the true underlying rate may be even higher than our overall estimate of 87.7%.

Trials conducted by the NCTN program impact patients with cancer in ways not included in our study. For instance, survivors from cancer can suffer lifelong consequences including morbidity, reduced quality of life, and economic hardship. NCTN trials provide improved access to protocol treatments for vulnerable patient populations that may not be routinely offered by pharmaceutical company trials.²⁹ Further, NCTN trial databases are vital resources for conducting secondary data analyses that generate new hypotheses and important insights into the mechanisms of malignancy.³⁰ A key element of the mission of the NCTN is to mentor the next generation of clinical researchers and to translate research into evidence-based practice.^8,31 Finally, the NCTN groups conduct trials to identify strategies to prevent and control cancer.

Although this study represents a first-time comprehensive evaluation of the impact of NCTN trials across important domains, it has limitations. An assessment of the impact of negative trials was not included. Negative trials also routinely guide clinical practice guidelines by identifying which treatments should not be used.¹⁴ In doing so, negative trial results can greatly limit the tremendous human and financial resources that may otherwise be committed to ineffective therapeutic approaches. Additionally, negative trials are key sources for secondary analyses, the scientific impact of which can be substantial.³² Further, the estimate of life-years gained relied on trials with improved overall survival for the experimental arm. However, trials routinely focus on earlier end points (eg, progression) and thus may not have data to fully characterize overall survival patterns. Also, in some instances, the experimental therapy is so clearly superior to standard care that a trial will be closed early; in others, the use of a cross-over design or a design without a standard-of-care arm might preclude assessment of life-years gained from treatments with demonstrable clinical benefits.^33-35 Additionally, projected estimates for 2025 and 2030 were based only on currently identified trials, with no attempt made to model how many studies will be positive in the future. From all of these perspectives, our estimate of life-years gained is likely conservative. Also, our impact metrics did not fully reflect the potential benefits of positive noninferiority or equivalency trials, which can benefit patients in terms of reduced toxicity, more convenient care delivery, and/or reduced costs without clinically meaningful reductions in outcomes. Because our study was based on overall trial findings, we were unable to estimate whether the different measures of impact differed for vulnerable patient populations, such as those with lower socioeconomic status.¹² We recognize that the conduct of a randomized clinical trial represents the culmination of a lengthy discovery process that includes initial drug discovery and early testing, the costs of which were not included in our assessment. Also, cancer guidelines frequently rely on multiple trials to inform guideline care recommendations.¹⁴ Finally, federal investment dollars do not fully cover the costs of conducting trials, including the establishment and support of institutional trial programs and the time and effort of research investigators.

In conclusion, the NCI-sponsored NCTN groups represent a vital and durable element of the scientific infrastructure of cancer clinical research in the United States. Randomized trials conducted by NCTN groups have contributed substantial gains in life-years for patients with cancer, and the studies have had a marked impact on cancer treatment guidelines and the scientific literature. Collectively, these findings demonstrate how publicly funded oncology research plays a vital role in informing clinical practice and extending the lives of patients with cancer.

AUTHOR CONTRIBUTIONS

Conception and design: Joseph M. Unger, Michael LeBlanc, Norman Wolmark, Walter J. Curran, Charles D. Blanke

Financial support: Joseph M. Unger, Michael LeBlanc

Provision of study materials or patients: Joseph M. Unger, Suzanne George, Mitchell D. Schnall, Robert S. Mannel, Sumithra J. Mandrekar

Collection and assembly of data: Joseph M. Unger, Mitchell D. Schnall, Fengmin Zhao, Mariama Bah, Riha Vaidya

Data analysis and interpretation: All authors

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

Population, Clinical, and Scientific Impact of National Cancer Institute’s National Clinical Trials Network Treatment Studies

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/jco/authors/author-center.

Open payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).