Abstract
Background
The objective of this review was to investigate trends in clinical trial design, specifically, the primary outcomes used, interpretation of results, and the magnitude of the benefits described in phase III controlled clinical trials in the first‐line treatment of patients with advanced non‐small cell lung cancer (NSCLC).
Methods
Seventy‐six trials published between 2000 and 2012 were selected from a total of 122 identified in a structured search.
Results
Overall survival (OS) was evaluated as the primary study endpoint in 50 (65.8%) trials, followed by progression‐free survival (PFS) in 15 (19.7%), and other variables, such as toxicity, quality of life (QoL), and response rate in 11 (14.5%). Ten (66.7%) out of 15 clinical trials using PFS as the primary endpoint were published between 2010 and 2012. Median overall survival (mOS) was 9.90 months (interquartile range: 3.5) with an increase of 0.384 months per year of publication (P < 0.001). A statistically significant improvement in mOS was obtained in only 13 (18.8%) trials. A total of 41 (53.9%) studies concluded that the result was positive. Of these, only 16 (39.1%) showed a statistically significant benefit in OS. QoL was assessed in 46 trials (60.5%) and of these, 10 (21.7%) reported significant improvements.
Conclusions
These findings raise important questions about how clinical benefits are measured in clinical trials in advanced NSCLC. Appropriate clinically relevant outcome variables should be established and validated, and post‐marketing studies should be requested by regulatory authorities to ensure meaningful clinical benefits in OS and QoL.
Keywords: Non‐small‐cell lung cancer, outcome assessment (health care), quality of life, retrospective study, survival analysis
Introduction
Advanced non‐small cell lung cancer (NSCLC) is the leading cause of cancer‐related death worldwide.1 Consequently, it has been the subject of extensive investigation over several decades, showing the growing interest of the oncology community and the pharmaceutical industry in this clinical entity.2 However, despite great efforts in research, the clinical benefit of new therapies has been modest,3 and treatment of lung cancer patients remains one of the greatest challenges in oncology.
Although the goal of research in cancer therapy is to improve the quantity and quality of life,4, 5 a growing number of questions have been raised with respect to the magnitude of the benefits shown in clinical trials in oncology. One of the difficulties in evaluating the clinical benefit of new cancer drugs is the numerous outcome measures used in clinical trials, and there is emerging concern regarding the design of such trials in oncology resulting from the increasing trend in measuring effectiveness through surrogate variables (i.e. progression‐free survival [PFS], time to progression [TTP], etc.).6, 7, 8, 9 The evidence supporting the use of surrogate endpoints in oncology is inconsistent6, 8, 10, 11 and, moreover, the reliability of the correlation between surrogate measures and meaningful outcomes like overall survival (OS) or quality of life (QoL) has not always been validated or is uncertain.12, 13 Although OS is still accepted as the gold standard for efficacy endpoints in lung cancer studies,14 its use as the primary endpoint appears to be declining in clinical trials investigating advanced NSCLC. Furthermore, an increasing trend toward positive interpretations of the results of clinical trials in oncology despite modest or questionable clinical benefits has also been reported.15
The aim of the study was to investigate trends in clinical trial design with respect to the primary outcomes used, the interpretation of the results, and the magnitude of the benefits described from first‐line treatment of patients with advanced NSCLC in studies published between 2000 and 2012.
Methods
Search strategy and study selection criteria
A structured search was conducted for randomized phase III clinical trials (RCTs) in the first‐line treatment of patients with advanced NSCLC published between 2000 and 2012. MEDLINE (accessed via PubMed and Ovid SP); Embase (Ovid SP); the Cochrane Central Register of Controlled Trials, Cochrane Library (CENTRAL); Evidence‐Cancer (via NHS); and TRIP (via BV‐SSPA) were searched using the following terms: lung cancer, non‐small, advanced, non‐surgical, treatment, controlled clinical trial, and phase III. To be included in the study, trials had to have at least two comparative arms of systemic chemotherapy or molecular‐targeted agents in patients with advanced NSCLC. In trials with two or more experimental arms, the arm selected for evaluation in this review was the one that obtained the highest OS or PFS, depending on the primary outcome endpoint of the trial. Only RCTs published in journals were selected. Exclusion criteria were: RCTs studying only non‐advanced disease or any stage of small‐cell lung cancer other than advanced stage; surgical or radiotherapy intervention studies or cancer screening and prevention studies; meta‐analyses or reviews reporting data from multiple RCTs; preliminary studies for which subsequent phase III studies were available; retrospective series; congress abstracts; and studies in a language other than English.
Data extraction, analysis, and management
Two investigators independently extracted data from each selected publication using a specifically designed form to record the following information: publication year, primary and secondary endpoints, assessment of QoL, median overall survival (mOS, months), median progression‐free survival (mPFS, months) per arm, and any statistically significant differences observed between treatment arms. We also recorded the authors’ conclusions on the experimental arm and summarized them as a positive result, negative result, or similar result for experimental versus control arms.
Results
Characteristics of selected trials
As shown in Figure 1, of the 122 trials retrieved for further evaluation, 46 were excluded after applying the study eligibility criteria. The 76 trials finally selected for this review (a list of the 76 clinical trials is given as online supporting information) involved a total of 40 765 patients with advanced NSCLC randomly assigned to 152 treatment arms. The characteristics of these 76 selected trials are summarized in Table 1.
Figure 1.
Flowchart depicting the trial selection process for the review.
Table 1.
Characteristics of the 76 selected clinical trials
| Variable | N (%) |
|---|---|
| Publication year | |
| 2000 | 3 (3.9) |
| 2001 | 5 (6.6) |
| 2002 | 7 (9.2) |
| 2003 | 7 (9.2) |
| 2004 | 7 (9.2) |
| 2005 | 7 (9.2) |
| 2006 | 7 (9.2) |
| 2007 | 6 (7.9) |
| 2008 | 5 (6.6) |
| 2009 | 4 (5.3) |
| 2010 | 7 (9.2) |
| 2011 | 5 (5.3) |
| 2012 | 7 (9.2) |
| No. of randomized patients | |
| < 500 | 46 (60.5) |
| 501–1000 | 17 (22.4) |
| > 1000 | 13 (17.1) |
| Median (range) | 432 (126–1725) |
| Primary endpoint | |
| OS | 50 (65.79) |
| PFS | 15 (19.74) |
| Other (RR, QoL, toxicity) | 11 (14.47) |
| Follow‐up median (range), months | 17.6 (2.7–60) |
| Not recorded | 29 |
| Authors’ conclusion | |
| Positive result | 41 (53.9) |
| Negative result | 25 (32.9) |
| Similar result | 10 (13.2) |
OS, overall survival; PFS, progression‐free survival; RR, response rate; QoL, quality of life.
First and secondary endpoints in randomized controlled trials of non‐small cell lung cancer
OS was used as a primary study endpoint in 50 (65.8%) of the 76 selected clinical trials for the first‐line treatment of advanced NSCLC published between 2000 and 2012, followed by PFS in 15 trials (19.7%), and other variables, such as toxicity, QoL, and response rate, in 11 (14.5%). It should be noted that 10 of the 15 clinical trials using PFS as the primary endpoint were published between 2010 and 2012.
OS data were available in 69 (90.8%) trials, and of these, only 13 (18.8%) reported a statistically significant OS benefit in the experimental arm. The remaining 56 (81.2%) trials did not demonstrate a statistically significant increase in OS compared to alternative treatments. We also examined total gains in OS obtained over the study period: mOS for the 76 trials was 9.90 months (interquartile range [IQR] 3.5), with a total mean improvement in OS of 0.384 months per year (P < 0.001).
PFS data were available in 67 (88.2%) of the 76 trials, and a significant difference between arms was reported in 26 (38.8%). The mPFS reported in trials during the study period was 4.90 months (IQR 1.9).
A total of 41 (53.9%) studies were considered to have concluded that the results were positive, based on the authors’ recommendations for the treatment to be adopted in clinical practice or in further studies. These positive conclusions were supported by a statistically significant result in the primary study endpoint in only 25 (61%) clinical trials. Of these, OS was the primary outcome in 11; surrogate measures, such as PFS, were the primary endpoint in a further 11 clinical trials; and finally, toxicity, QoL, and response rate were the primary endpoints in the three remaining clinical trials that were able to show statistically significant differences in the primary study endpoint.
In 16 (39%) of the 41 clinical trials that concluded that a positive result had been obtained in the experimental arm, no statistically significant differences in the primary study endpoint were shown, and the results were supported by secondary measures, such as lower toxicity (8/16), improvement in response rate (1/16), significant improvement in QoL (2/16), statistically significant improvement in PFS (3/16), and statistically significant improvement in OS (2/16).
QoL was evaluated in 46 (60.5%) trials and a statistically significant improvement in this endpoint was found in 10 (21.7%). A total of 10 different measurement instruments were identified. The specific Functional Assessment of Cancer Therapy Lung Cancer questionnaire (FACT‐LCS5) was used in 23 (50%) trials, the European Organization for Research and Treatment of Cancer Quality of Life questionnaire C30 (EORTC QLQ C‐30) in 20 (43.5%), and the specific EORTC lung cancer questionnaire (EORTC QLQ LC‐13) in 17 (37.0%).
Discussion
In recent years, there has been debate over the increasing use of surrogate variables, such as PFS, instead of the OS primary endpoint in the design of clinical trials in oncology.6, 8 In our study, 65.8% (50/76) of the trials published between 2000 and 2012 used OS as the main variable, followed by PFS in 19.7% (15/76), and other variables in 14.5% (11/76). It should be highlighted that 66% of the RCTs that included PFS as the primary endpoint were published from 2010 to 2012, the last two years of our 12‐year study period. Our work therefore confirms the increasing trend toward the use of PFS as the primary endpoint in RCTs in advanced NSCLC and the declining use of OS or QoL.
Some authors believe this trend to be the result of an increasing willingness of regulatory authorities to accept the use of surrogate variables as primary study endpoints,15 the common use of crossover trial designs and subsequent lines of treatment that may dilute the overall survival benefit, or longer follow‐up times in trials using OS as the primary endpoint.11 The Food and Drug Administration recently accepted the use of PFS as a primary endpoint in trials studying the treatment of advanced NSCLC but with reservations regarding the accuracy, reproducibility, and clinical relevance of this surrogate variable,14 and concludes that OS is still the standard clinical benefit outcome variable that should be used to establish the efficacy of a therapy in patients with advanced NSCLC. The fact is that the advantages gained from faster drug approval and patient access to new agents are questionable if they rely on results of studies based on surrogate variables that do not translate into benefits in terms of OS or QoL.
Many of the surrogate variables used as primary outcome measures in clinical trials are poorly correlated with survival or QoL, or the correlation has not been tested.12, 16, 17 Moreover, most regulatory approvals based on these surrogate variables do not usually require post‐marketing studies to be conducted to confirm or validate effectiveness and safety results that allow translating PFS benefit into meaningful clinical benefit for patients, that is, OS or QoL.
There is currently an interesting and necessary debate regarding when it should be demonstrated that a cancer drug leads to an improvement in OS or QoL. Some argue that this should occur before market authorization while others believe that in some indications, such as advanced NSCLC, a drug could receive provisional approval based on surrogate endpoints, and its benefit on OS or QoL should be demonstrated in post‐marketing trials. However, according to two recent studies, this rarely happens.18, 19 One of these studies that evaluated evidence of the OS or QoL benefits of cancer drugs approved by the European Medicines Agency stated that results reported in the post‐marketing period confirming a positive impact on survival and QoL are very rare for new cancer drugs that have been approved based on a benefit in surrogate variables.19
Our findings suggest that positive conclusions describing a meaningful clinical benefit in patient outcome are extremely common, even when no statistically significant differences in the primary study endpoint are obtained. Similarly, studies frequently reach a positive conclusion without having demonstrated a positive impact on the two most important outcomes for patients, OS and QoL. In our study, of the 41 clinical trials with positive conclusions, only 16 were based on a statistically significant OS gain.
A total of 41 (53.9%) clinical trials concluded their results in the experimental arm were positive, a figure similar to that reported by Sacher et al. in their review of phase III clinical trials in advanced NSCLC.15 They found positive conclusions in only 31% of clinical trials conducted between 1980 and 1990, compared to 53% in clinical trials carried out between 2001 and 2010. These results confirm the increasing trend toward positive interpretations of the results of clinical trials in NSCLC, even when no statistically significant differences in the main variable were shown. In our study, the percentage of trials showing a statistically significant improvement in the primary endpoint was 32.8%, a result very similar to the 31% described by Sacher et al. for the 2001–2010 study period. In this regard, Sacher et al. pointed out that the percentage of trials reporting a statistically significant benefit in the primary endpoint had not changed over time (29% in 1980–1990 and 31% in 2001–2010). However, the percentage of trials reporting a positive result despite not showing a statistically significant benefit in their primary outcome measure had clearly increased over time (30% of positive trials in 1981–1990 vs. 24% in 1991–2000 vs. 53% in 2001–2010; P = 0.001).15
QoL in cancer patients has been found to be a strong predictor of survival and toxicity. However, this outcome variable was evaluated in only 60.5% of clinical trials included in the present study. Moreover, only 21.7% of these trials reported a significant improvement in QoL. This result is better than that reported by Tanvetyanon et al. in their systematic review of QoL associated with chemotherapy for NSCLC, where only 7.1% of trials analyzed demonstrated a significant difference in QoL.20 Overall, QoL results have shown modest improvement and further research on this outcome in NSCLC patients, using more uniform methodologies, is necessary.
There are some limitations to this study. Firstly, publication bias is one of the main limitations of review studies and along with this, selection bias is one of the main criticisms of reviews and meta‐analyses. However, we minimized this bias by designing and applying a structured and well‐defined search strategy. In addition, five different databases were searched and two researchers independently selected the studies. Secondly, it was not always possible to obtain all of the data required to evaluate the methodology and outcomes of the studies. In addition, PFS and TTP were considered as surrogate endpoints of survival and as interchangeable variables, in common with other authors.10, 13, 21 Thirdly, the time period studied should be extended in order to include clinical trials investigating immunotherapy regimens. This will be taken into account in future research.
The present review shows that most of the clinical trials in advanced NSCLC published between 2000 and 2012 did not demonstrate a statistically significant improvement in OS or QoL over alternative treatments.
There is a lack of studies examining the design of clinical trials in advanced NSCLC treatment. Our findings raise important questions about how clinical benefits are measured in clinical trials in advanced NSCLC. Appropriate clinically relevant outcome variables should be established and validated, and post‐marketing studies in OS and QoL should be requested by regulatory authorities to ensure meaningful clinical benefits.
Disclosure
No authors report any conflict of interest.
Supporting information
Appendix S1. A list of the 76 phase III clinical trials included in this review is available as online supplementary material.
Acknowledgments
The authors wish to thank Claire Graham for English language editing.
References
- 1. Ferlay J, Soerjomataram I, Dikshit R et al. Cancer incidence and mortality worldwide: Sources, methods and major patterns in GLOBOCAN 2012. Int J Cancer 2015; 136: E359–86. [DOI] [PubMed] [Google Scholar]
- 2. Bariani GM, de Celis Ferrari ACR, Hoff PM, Krzyzanowska MK, Riechelmann RP. Self‐reported conflicts of interest of authors, trial sponsorship, and the interpretation of editorials and related phase III trials in oncology. J Clin Oncol 2013; 31: 2289–95. [DOI] [PubMed] [Google Scholar]
- 3. Fernández‐López C, Expósito‐Hernández J, Arrebola‐Moreno JP et al. Trends in phase III randomized controlled clinical trials on the treatment of advanced non‐small‐cell lung cancer. Cancer Med 2016; 5: 2190–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Wilson MK, Collyar D, Chingos DT et al. Outcomes and endpoints in cancer trials: Bridging the divide. Lancet Oncol 2015; 16: e43–52. [DOI] [PubMed] [Google Scholar]
- 5. Peppercorn JM, Smith TJ, Helft PR et al. American Society of Clinical Oncology statement: Toward individualized care for patients with advanced cancer. J Clin Oncol 2011; 29: 755–60. [DOI] [PubMed] [Google Scholar]
- 6. Garon E. Issues surrounding clinical trial endpoints in solid malignancies with a focus on metastatic non‐small cell lung cancer. Lung Cancer 2012; 77: 475–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Soria JC, Massard C, Le Chevalier T. Should progression‐free survival be the primary measure of efficacy for advanced NSCLC therapy? Ann Oncol 2010; 21: 2324–32. [DOI] [PubMed] [Google Scholar]
- 8. Sargent DJ, Hayes DF. Assessing the measure of a new drug: Is survival the only thing that matters? J Clin Oncol 2008; 26: 1922–3. [DOI] [PubMed] [Google Scholar]
- 9. Pilz LR, Manegold C, Schmid‐Bindert G. Statistical considerations and endpoints for clinical lung cancer studies: Can progression free survival (PFS) substitute overall survival (OS) as a valid endpoint in clinical trials for advanced non‐small‐cell lung cancer? Transl Lung Cancer Res 2012; 11: 26–35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Saad ED, Katz A. Progression‐free survival and time to progression as primary end points in advanced breast cancer: Often used, sometimes loosely defined. Ann Oncol 2009; 20: 460–4. [DOI] [PubMed] [Google Scholar]
- 11. Fenchel K, Sellmann L, Dempke WCM. Overall survival in non‐small cell lung cancer‐what is clinically meaningful? Transl Lung Cancer Res 2016; 5: 115–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Prasad V, Kim C, Burotto M, Vandross A. The strength of association between surrogate end points and survival in oncology. JAMA Intern Med 2015; 175: 1389–98. [DOI] [PubMed] [Google Scholar]
- 13. Ciani O, Davis S, Tappenden P et al. Validation of surrogate endpoints in advanced solid tumors: Systematic review of statistical methods, results, and implications for policy makers. Int J Technol Assess Health Care 2014; 30: 312–24. [DOI] [PubMed] [Google Scholar]
- 14. U.S. Department of Health and Human Services, Food and Drug Administration . Clinical Trial Endpoints for the Approval of Non‐ Small Cell Lung Cancer Drugs and Biologics Guidance for Industry. (2015,). [Cited 4 May 2017.] http://wwwfdagov/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/defaulthtm
- 15. Sacher AG, Le LW, Leighl NB. Shifting patterns in the interpretation of phase III clinical trial outcomes in advanced non‐small‐cell lung cancer: The bar is dropping. J Clin Oncol 2014; 32: 1407–11. [DOI] [PubMed] [Google Scholar]
- 16. Kim C, Prasad V. Strength of validation for surrogate end points used in the US Food and Drug Administration's approval of oncology drugs. Mayo Clin Proc 2016; 91: 713–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Kantarjian H, Rajkumar SV. Why are cancer drugs so expensive in the United States, and what are the solutions? Mayo Clin Proc 2015; 90: 500–4. [DOI] [PubMed] [Google Scholar]
- 18. Fojo T, Mailankody S, Lo A. Unintended consequences of expensive cancer therapeutics—The pursuit of marginal indications and a me‐too mentality that stifles innovation and creativity. JAMA Otolaryngol Neck Surg 2014; 140: 1225–36. [DOI] [PubMed] [Google Scholar]
- 19. Davis C, Naci H, Gurpinar E et al. Availability of evidence of benefits on overall survival and quality of life of cancer drugs approved by European Medicines Agency: Retrospective cohort study of drug approvals 2009–13. BMJ 2017; 359: j4530. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Tanvetyanon T, Soares H, Djulbegovic B et al. A systematic review of quality of life associated with standard chemotherapy regimens for advanced non‐small cell lung cancer. J Thorac Oncol 2007; 2: 1091–7. [DOI] [PubMed] [Google Scholar]
- 21. Li L, Pan Z. Progression‐free survival and time to progression as real surrogate end points for overall survival in advanced breast cancer: A meta‐analysis of 37 trials. Clin Breast Cancer 2018; 18: 63–70. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Appendix S1. A list of the 76 phase III clinical trials included in this review is available as online supplementary material.