Abstract
This study describes the number of interim reports of ongoing randomized trials published between 2006 and 2015, the reasons for interim reporting, and differences in the abstract conclusions between interim and final publications.
Publication of interim results of randomized trials can be consequential, potentially undermining trial integrity by unblinding or encouraging dropouts and crossovers, introducing uncertainty because results based on fewer outcomes are less precise, or overstating true treatment effects as demonstrated primarily in trials terminated early but also in ongoing trials. Nevertheless, because interim results are new and often promising, they may generate substantial interest, which can be misleading if results change. We describe the characteristics of interim publications from ongoing randomized trials and compare their consistency and prominence with those of final publications.
Methods
We searched PubMed for randomized trials (between January 1, 2006 and December 31, 2015) with “interim,” “not mature,” or “immature” in the title or abstract, excluding pilot studies, protocols, and cancer trials reporting final primary (progression-free survival) but interim secondary (overall survival) outcomes. To identify final publications, we searched PubMed, ClinicalTrials.gov, and Web of Science through December 1, 2016, and used trial registration numbers, trial and author names, randomized groups, and planned sample sizes to match with interim reports. We emailed authors of interim reports when no final publication was identified (20% [13/66] responded between January and September 2017).
Two reviewers (S.W. and L.M.S.) independently abstracted data, resolving disagreements by discussion. For interim and final publications reporting the same efficacy or safety outcome, we compared trial characteristics and prominence (journal impact factor, Altmetric attention score for news and social media) using paired difference tests (McNemar or signed rank) to generate 2-tailed P values (P < .05 was considered statistically significant) in Stata version 14.2 (StataCorp). In addition, we categorized abstract conclusions (not different, beneficial, or harmful) and compared changes between interim and final publications.
Results
Interim results were reported in 613 of 1267 screened publications. Of these, 72% (n = 442) reported on trials stopped early (for benefit, n = 105; harm, n = 67; futility, n = 224; or other problems, n = 46). The remaining 171 ongoing trials (mostly in oncology, surgery, or cardiology) reported interim efficacy or safety results (Table 1). Forty percent (n = 68) of the publications stated that the interim analysis was protocol specified but half (n = 86) provided no reason.
Table 1. Description of Interim Publications of Ongoing Trials (n = 171).
| Characteristics | No. (%)a |
|---|---|
| Reason for interim analysis | |
| Protocol prespecified | 68 (40) |
| Otherb | 17 (10) |
| None | 86 (50) |
| Journal | |
| High impact factor (≥20) | 26 (15) |
| Top-5 impact factor for general medical journal | 14 (8) |
| Altmetric attention score, median (range)c | 6 (1-531) |
| Results reported | |
| Efficacy | 155 (91) |
| Safety | 16 (9) |
| Clinical area | |
| Oncology | 48 (28) |
| Surgery | 30 (18) |
| Cardiology | 18 (11) |
| Infectious disease | 16 (9) |
| Endocrinology | 13 (8) |
| Funding | |
| Industry (solely) | 61 (36) |
| Industry (partly) | 17 (10) |
| Government | 30 (18) |
| Foundation or university | 29 (17) |
| Design | |
| Superiority | 148 (87) |
| Noninferiority | 23 (13) |
| Control group | |
| Placebo | 61 (36) |
| Active | 101 (59) |
| Placebo plus active | 9 (5) |
| Trial participants | |
| No. of participants, median (IQR) | 130 (60-468) |
| ≤50 participants | 32 (19) |
| Follow-up time, wk | |
| Median (IQR) | 48 (12-91) |
| ≤4 | 26 (15) |
| Intervention tested | |
| Drug | 81 (47) |
| Surgery or procedure | 54 (32) |
| Therapy | 26 (15) |
| Device | 10 (6) |
Abbreviation: IQR, interquartile range.
Data are expressed as No. (%) unless otherwise specified.
Examples of other reasons include response to recently released results about the same intervention and desire for publication to support upcoming research funding application.
Altmetric attention scores were available for 83 articles.
Final results were published for 61% (n = 98) of the 160 trials for which sufficient time elapsed for publication (ie, >1 year beyond trial registry–specified study completion date). Among the 73 matched pairs reporting the same efficacy or safety outcome, interim analyses had fewer participants (median, 205 vs 295 participants) and shorter follow-up (median, 48 vs 104 weeks) than final analyses (Table 2). Interim and final publications had similar prominence, and most (79%; n = 58) abstract conclusions did not change: 32 consistently concluded the intervention was not different from the comparator and 26 consistently concluded the intervention was beneficial. Twenty-one percent (n = 15) changed: 4 from not different to beneficial, 3 from not different to harmful (or possibly harmful), 6 from beneficial to not different, 1 from beneficial to harmful, and 1 from inconclusive to noninferior (see examples in Table 2). The change from beneficial to harmful occurred because the harm of high-dose imatinib, which reduced event-free survival vs the standard dose, emerged in the final publication.
Table 2. Comparison of Matched Pairs of Interim and Final Publications Both Reporting the Same Efficacy or Safety Outcomes and Examples of Changes in Abstract Conclusions Between Interim and Final Publications (n = 73)a.
| Characteristics | Publication | Difference, % (95% CI) |
P Valueb | |
|---|---|---|---|---|
| Interim | Final | |||
| Trial participants | ||||
| No. of participants, median (IQR) | 205 (89-544) | 295 (100-724) | <.001 | |
| ≤50 participants, No. (%) | 8 (11) | 5 (5) | 6 (−1 to 12) | .05 |
| Follow-up time, wk | ||||
| Median (IQR) | 48 (22-81) | 104 (52-156) | <.001 | |
| ≤4, No. (%) | 11 (15) | 5 (7) | 8 (1 to 16) | .03 |
| Journal prominence | ||||
| High impact factor (≥20), No. (%) | 16 (22) | 17 (23) | 1 (−8 to 11) | .70 |
| Top-5 impact factor for general medical journal, No. (%) | 10 (14) | 8 (11) | 3 (13 to 8) | .60 |
| Both interim and final publications in top-5 journal, No. (%) | 3 (4) | |||
| Altmetric attention score, median (IQR)c | 6 (2-20) | 5 (2-25) | .10 | |
| Examples of changes in abstract conclusiond | ||||
| Not different → beneficial | Abiraterone showed a trend toward improved overall survival in patients with metastatic castration–resistant prostate cancer (Lancet Oncology). | Treatment with abiraterone acetate prolonged overall survival vs prednisone alone by a margin that was both clinically and statistically significant (New England Journal of Medicine; NCT00887198). | ||
| Not different → harmful (or possibly harmful) | The everolimus-eluting bioresorbable scaffold (Absorb) showed similar 1-year composite secondary clinical outcomes as the everolimus-eluting metallic stent (Lancet). | A higher rate of device-oriented composite end point due to target vessel myocardial infarction, including periprocedural myocardial infarction, was observed in the everolimus-eluting bioresorbable scaffold (Absorb) group (Lancet; NCT01425281). | ||
| Beneficial → not different | The capecitabine-containing chemotherapy regimen reduced breast cancer recurrence vs a control schedule of standard agents (Lancet Oncology). | Integration of capecitabine into a regimen that contains docetaxel, epirubicin, and cyclophosphamide did not improve recurrence-free survival significantly vs a similar regimen without capecitabine (Journal of Clinical Oncology; NCT00114816). | ||
| Beneficial → harmful | First randomized phase 3 trial in patients with pretreated chronic-phase chronic myeloid leukemia demonstrating improvements in major cytogenetic response to complete cytogenetic response and major molecular response rates with high-dose imatinib therapy (Haematologica). | Standard-dose imatinib remains the standard of care for pretreated patients with chronic-phase chronic myeloid leukemia (high-dose imatinib reduced event-free survival) (Haematologica; NCT00327262). | ||
| Inconclusive → noninferior | The noninferiority criterion was not met and overall survival results were inconclusive (Lancet Oncology). | Bevacizumab plus capecitabine for HER2-negative locally recurrent or metastatic breast cancer offering good tolerability without compromising overall survival vs bevacizumab plus paclitaxel (Lancet Oncology; NCT00600340). | ||
Abbreviation: IQR, interquartile range.
Percentages may not add to total because of rounding.
P values for paired differences (McNemar test for dichotomous variables and signed rank test for continuous variables).
Altmetric attention scores were available for 37 interim and 48 final publications.
Not different indicates that the intervention is not statistically significantly different than placebo or meets noninferiority criterion for active control. Beneficial indicates that the intervention is statistically significantly better than placebo or active control. Harmful indicates that the intervention is statistically significantly worse than placebo or active control (1 of the 3 pairs of publications that changed from not different to harmful was possibly harmful because it had a P = .05, but the abstract conclusion raised concern about a possible increase in deaths due to the intervention).
Discussion
Many interim publications reported analyses without mention of prespecification or any justification. Frequent nonpublication of final results means true treatment effects often remain unknown. Interim and final publications had similar journal and media prominence, and most reached similar conclusions. However, in 21%, clinicians could have been misled about whether an intervention was beneficial, harmful, or ineffective. Limitations include underestimation of interim publications (full text not searched), the final publication rate (missed publications or misclassified trials eligible for publication), and the extent of interim to final changes (only 1 key outcome analyzed).
Interim publication should be limited to protocol-prespecified analyses performed when enough outcomes occurred for statistical stability and to scenarios least likely to undermine trial integrity (eg, crossovers cannot happen in a trial testing 2 surgical techniques after all patients have had surgery). Routinely including the word “interim” in the title and justifying the reason in the publication would help alert readers to inherent uncertainties. Journals, authors, and funders should commit to making final results accessible by linking interim publications to final reports whenever available.
Section Editor: Jody W. Zylke, MD, Deputy Editor.
References
- 1.Stephens RJ, Langley RE, Mulvenna P, Nankivell M, Vail A, Parmar MK. Interim results in clinical trials: do we need to keep all interim randomised clinical trial results confidential? Lung Cancer. 2014;85(2):116-118. [DOI] [PubMed] [Google Scholar]
- 2.Wang H, Rosner GL, Goodman SN. Quantifying over-estimation in early stopped clinical trials and the “freezing effect” on subsequent research. Clin Trials. 2016;13(6):621-631. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Bassler D, Briel M, Montori VM, et al. ; STOPIT-2 Study Group . Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis. JAMA. 2010;303(12):1180-1187. [DOI] [PubMed] [Google Scholar]
- 4.Counsell N, Biri D, Fraczek J, Hackshaw A. Publishing interim results of randomised clinical trials in peer-reviewed journals. Clin Trials. 2017;14(1):67-77. [DOI] [PubMed] [Google Scholar]
- 5.Petzer AL, Wolf D, Fong D, et al. . High-dose imatinib improves cytogenetic and molecular remissions in patients with pretreated Philadelphia-positive, BCR-ABL-positive chronic phase chronic myeloid leukemia: first results from the randomized CELSG phase III CML 11 “ISTAHIT” study. Haematologica. 2010;95(6):908-913. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Petzer AL, Fong D, Lion T, et al. . High-dose imatinib induction followed by standard-dose maintenance in pre-treated chronic phase chronic myeloid leukemia patients—final analysis of a randomized, multicenter, phase III trial. Haematologica. 2012;97(10):1562-1569. [DOI] [PMC free article] [PubMed] [Google Scholar]