Updating Reports of Phase 3 Clinical Trials for Cancer

Key Points

Question

How many phase 3 randomized clinical trials for cancer are updated after the initial report, and which factors are associated with the updates?

Findings

In this study, 41 of 207 phase 3 trials (20%) had initial reports that were updated; the factors associated with updating were positive trial results, larger trial size, evaluation of adjuvant therapy, and evaluation of hormonal or targeted agents. The hazard ratio for the primary end point increased in the 31 updated trials for which values were available, indicating decreased effect size.

Meaning

Updating of phase 3 trials is uncommon and should be mandated to provide mature results.

Abstract

Importance

Phase 3 randomized clinical trials (RCTs) are usually reported after a predetermined number of events (death or disease progression) have occurred, when survival curves remain poorly defined. Updated reports are important in providing mature data.

Objectives

To evaluate the proportion of phase 3 RCTs for cancer that are updated and the factors that are associated with updating them and, for updated trials, to compare initial and updated results.

Design, Setting, and Participants

This study identified reports of 2-group RCTs with a sample size of at least 100, published in 6 major journals between 1990 and 2010, that evaluated drug treatments for breast, lung, or prostate cancer. PubMed and abstracts of large cancer conferences were searched to identify updated (or earlier) reports of the same trials published up to 2019. Logistic regression was used to identify factors associated with the provision of updated reports. The hazard ratios defining the relative treatment effects for the primary and secondary end points between the initial and updated reports were compared.

Main Outcomes and Measures

Proportion of RCTs whose results are updated, factors associated with updating, and change in hazard ratio for the primary end point between initial and updated reports.

Results

A total of 207 RCTs met the inclusion criteria, and 41 (20%) were found to have updated reports. The factors significantly associated with an update included positive trial results (odds ratio [OR], 8.7 [95% CI, 3.3-23.3]), larger trial size (OR, 1.0006 [95% CI, 1.0000-1.0012]), evaluation of hormonal agents (OR, 5.8 [95% CI, 1.6-21.8]) or targeted agents (OR, 4.3 [95% CI, 1.3-14.6]) compared with chemotherapy, and evaluation of adjuvant therapy rather than therapy for advanced disease (OR, 8.0 [95% CI, 2.9-21.9]). For 31 trials for which initial and updated hazard ratios for the primary end point were available, the median hazard ratio increased from 0.66 (95% CI, 0.22-1.20) to 0.74 (95% CI, 0.32-1.19) (P < .001), indicating a decreased level of effectiveness.

Conclusions and Relevance

Only 20% of reports of phase 3 clinical trials for breast, lung, and prostate cancer were updated. Original reports of such trials are based on relatively few events, and their results are immature; more mature data indicate a decreased level of effect in updated trials. Updated reporting to provide mature, long-term results of clinical trials should be mandated.

This study evaluates the proportion of phase 3 randomized clinical trials for cancer that are updated and the factors that are associated with updating them and, for updated trials, compares initial and updated results.

Introduction

Phase 3 randomized clinical trials (RCTs) are usually analyzed after the occurrence of sufficient events to provide statistical evidence for or against a prespecified difference in outcome. The events may be death, tumor progression, or tumor recurrence, depending on whether the primary end point is overall survival, progression-free survival, or (for adjuvant therapy) disease-free survival. Survival curves in initial reports of RCTs are immature, as are reports of toxic effects, because follow-up is too short to capture many events.^1,2 Updated results can inform clinicians about the long-term effectiveness and toxic effects associated with new anticancer regimens. The aims of the present study were to evaluate (1) the probability of updating oncology RCTs, (2) the factors associated with updating of oncology RCTs, and (3) the concordance in outcomes between initial and updated reports.

Methods

We identified original articles published between 1990 and 2010 in 6 oncology journals with high impact factors (eTable in the Supplement). Eligible articles reported 2-group trials with 100 or more participants that evaluated systemic therapy for patients with breast, lung, or prostate cancer. We excluded RCTs that evaluated radiotherapy or surgical therapy alone, supportive care, screening, or prevention and reports of subgroups or nonprimary end points. As this study was based on medical literature and did not involve human participants, institutional review board approval was not required.

We searched PubMed and annual meetings of the American Society of Clinical Oncology, the European Society for Medical Oncology, and the San Antonio Breast Cancer Conference for updated reports of the trials published up to 2019 (or earlier reports if the index article was an update), using search terms that included authors of the original publication and disease indicators. For studies with multiple published updates, the most recent update was analyzed.

Data were extracted from the original and updated reports by one of the authors using Excel (Microsoft Corp); an additional author verified key elements of all extracted data. The factors of interest included disease site, adjuvant vs metastatic setting, type of treatment (chemotherapy, hormonal therapy, or targeted agents), primary and secondary end points, source of funding, sample size, primary and secondary effectiveness outcomes (reported as hazard ratios [HRs] and their statistical significance) and numbers of events defining them, proportion of participants experiencing grade 3 to 4 adverse events, and median follow-up. If there was no explicit statement defining end points, then the first end point analyzed was assumed to be primary, and the second end point analyzed was assumed to be secondary.

Outcomes were compared between initial and updated reports. Multivariable logistic regression with backward selection was used to identify the factors associated with updated reporting; the model included disease site and setting (advanced disease or adjuvant), type of therapy, impact factor of the journal of the initial report, and positive or negative outcome for the primary end point. Differences in HRs between the first and the subsequent publication were examined using the Wilcoxon signed rank test. Tests were 2-sided, and those with P < .05 are considered statistically significant. Data were analyzed using SAS, version 9.2 (SAS Institute Inc) and R, version 3.6.1 (R Foundation for Statistical Computing). Results were not adjusted for multiple comparisons and are hypothesis generating.

Results

We identified only 41 of the 207 phase 3 RCTs (20%) that had initial reports that were updated. The median follow-up in the first publications was 22 months, and for the 41 trials that were updated, the median follow-up was 36 months in the initial publication and 68 months in the subsequent publication. The characteristics of the original and updated reports are provided in the eTable in the Supplement.

The HRs reported in initial and updated reports of RCTs are listed in Table 1, with the number of events defining them and with P values for the differences in outcome with longer follow-up. Overall, the median HR for the primary end point increased from 0.66 (95% CI, 0.22-1.20) to 0.74 (95% CI, 0.32-1.19) (P < .001), with larger changes occurring in adjuvant trials. Only 8 updated reports provided information about toxic effects, and only 2 quantified the proportion of participants with grade 3 to 4 adverse events.

Table 1. Median HRs for Primary and Secondary End Points and Median Number of Events for Initial and Updated Reports Among the 41 Updated Trials.

End point	Initial report		Updated report		P value of HRa
	HR (range)	No. of events (range)	HR (range)	No. of events (range)
All trials (N = 41)
Primary end point (n = 31)	0.66 (0.22-1.20)	160 (39-973)	0.74 (0.32-1.19)	239 (53-1244)	<.001
Secondary end point (n = 26)	0.72 (0.41-1.64)	82 (20-518)	0.74 (0.51-1.25)	218 (33-1237)	.10
Early disease (n = 27)
Primary end point (n = 23)	0.66 (0.22-1.20)	136 (39-973)	0.76 (0.42-1.19)	215 (53-1244)	.001
Secondary end point (n = 17)	0.68 (0.41-1.15)	73 (20-518)	0.73 (0.51-1.12)	274 (33-1237)	.02
Advanced disease (n = 14)
Primary end point (n = 8)	0.66 (0.30-0.93)	221 (121-394)	0.66 (0.32-0.96)	332 (184-779)	.04
Secondary end point (n = 9)	0.84 (0.67-1.64)	103 (27-450)	0.81 (0.61-1.25)	178 (119-954)	.55

Abbreviation: HR, hazard ratio.

^aCalculated using the Wilcoxon signed rank test to compare HRs between initial and updated reports.

Multivariable analysis of the factors associated with updating trials is summarized in Table 2. The factors associated with updating were positive trial results (odds ratio [OR], 8.7 [95% CI, 3.3-23.3]), larger trial size (OR, 1.0006 [95% CI, 1-1.0012]), evaluation of hormonal agents (OR, 5.8 [95% CI, 1.6-21.8]) or targeted agents (OR, 4.3 [95% CI, 1.3-14.6]), and trials for early disease (OR, 8.0 [95% CI, 2.9-21.9]).

Table 2. Data on Multivariate Analysis^a.

Covariate	Odds ratio (95%CI)	Global P value
No. of patients	1.0006 (1.0000-1.0012)	.049
Regimen under evaluation
Chemotherapy	1 [Reference]	.01
Hormonal agent	5.8 (1.6-21.8)
Targeted agent	4.3 (1.3-14.6)
Setting
Advanced	1 [Reference]	<.001
Early	8.0 (2.9-21.9)
Positive study results
No	1 [Reference]	<.001
Yes	8.7 (3.3-23.3)

^aThe outcome is binary: whether the trial has an updated report or not. Includes the factors in the final model.

Discussion

Initial reports of RCTs are presented and/or published when a prespecified number of events (deaths, disease progression, or recurrence) has occurred as per a statistical plan. Even for trials of advanced cancer, in which almost all patients die of or with their disease, reporting often occurs when a limited number of events have occurred. The trial may then lead to registration and marketing of the new treatment. In these reports, the tails of the survival curves, indicating long-term benefit (or lack thereof) and long-term toxic effects, are defined poorly^1,2 but could markedly influence the balance between benefit and harm associated with the new treatment. Updated reports based on mature data better inform oncologists and patients about long-term benefits and adverse effects, but updating is rarely mandated in trial protocols.

In the present study, 207 original published reports of phase 3 RCTs were identified for breast, lung, and prostate cancer. Of these, only 41 RCTs (20%) had updated reports. Although some updates may have been missed, this number is likely small because our search terms included original authors and disease and treatment characteristics. Given the longer follow-up of adjuvant studies, the longer survival of patients with advanced breast or prostate cancer treated hormonally, and the greater interest in trials with positive results, it is not surprising that these factors were associated with producing updated reports; however, it is important to have longer follow-up for patients in all types of cancer trials.

Related studies have reported discrepancies in results between initial reports of outcomes of RCTs in meeting abstracts and subsequent publications,^3,4,5,6,7 perhaps owing to reporting of unverified data in abstracts. For 31 trials providing initial and updated values of HRs for their primary end point, we found a weakening of effect size, with the median HR increasing from 0.66 to 0.74. Changes in HRs for primary and secondary end points, and their significance levels, are unknown for the 85% of trials in our sample that were not updated or did not provide this information.

Of the 41 updated RCTs, only 8 (20%) also updated their safety results, and in only 2 of them could toxic effects be compared between the original report and its update. Underreporting of safety results can be associated with an inflated estimate of therapeutic benefit, which is a balance between improvement in duration or quality of survival and increase in toxic effects. Chronic toxic effects are often not captured in reports of clinical trials.^1,8

Limitations

Our study has limitations. First, we assessed only 2-group RCTs for breast, lung, and prostate cancer; different results might be found for other cancers. Second, we identified original articles published in 6 journals and might have missed trials for which the primary report was published elsewhere. Third, only 13% of the trials evaluated treatment with targeted agents; results might be different for current trials evaluating modern anticancer therapies, which may have a better benefit to risk ratio. Fourth, because only 20% of RCTs were updated, this analysis is liable to selection bias and may underestimate changes in the treatment effect and safety in other trials.

Conclusions

Clinicians rely on RCTs as their primary source of information to determine the most beneficial treatment plan for their patients. Clinicians need to consider the mature evidence from RCTs and not just their primary report when determining the benefits and toxic effects associated with the treatment plan for individual patients. A plan to report mature results of clinical trials (with correction for repeated analysis of data) should be included in every protocol and mandated by research ethics boards and registration agencies.

Supplement.

eTable. Randomized Clinical Trials in Oncology Published Between 1990 and 2010