The Use of Not-Negative Conclusions to Describe Results of Formally Negative Trials Presented at Oncology Meetings

Abstract

This quality improvement analysis of oral presentations of phase 3 randomized clinical trials evaluates the frequency and risks of emphasizing results of borderline significance.

Presentations at medical meetings have great resonance within the scientific community, especially following the diffusion of social media. Oral presentations are not subject to peer review, and some authors’ conclusions may not be completely justified by the results. This is particularly critical when, despite the formally negative trial result, the authors’ conclusions are not-negative. The aim of this quality improvement analysis was to describe the frequency and type of not-negative conclusions used by presenters to discuss the results of formally negative trials at recent oncology meetings.

Methods

We reviewed oral presentations of phase 3 randomized clinical trials (excluding noninferiority trials) at American Society of Clinical Oncology and European Society for Medical Oncology congresses from 2017 through 2019. We classified trials as positive (not-negative) vs negative, in terms of formal results (rejection of null hypothesis for the primary end point) and in terms of conclusions (based on the sentences used in the Conclusions section of the oral presentation). We considered the conclusions to be not-negative when, more or less explicitly, authors consider the possibility of using the experimental treatment in that setting, without clear conclusions about the study negativity. This evaluation was not blinded to the positivity or negativity of formal results. The study did not require institutional review board or ethical committee review because it did not involve human subjects.

We classified formally negative trials in different categories according to the reasons of not-negative conclusions: (1) numerically better outcome in the experimental arm, despite a nonsignificant P value, (2) emphasis on positive subgroup(s), (3) emphasis on positive secondary end point(s), and (4) noninferiority interpretation of a negative superiority trial (eTable in the Supplement).

Results

Overall, 208 randomized clinical trials were selected. Of the 91 formally negative studies, 26 (29%) had a not-negative conclusion. The proportion of negative studies with not-negative conclusions was 22%, 13%, and 47% in 2017, 2018, and 2019, respectively; 17 of 57 nonprofit studies (30%) and 9 of 34 for-profit studies (26%) have not-negative conclusions (Figure).

Figure. Proportion of Formally Negative Trials With Not-Negative Conclusions in Oral Presentations at American Society of Clinical Oncology (ASCO) and European Society for Medical Oncology (ESMO) Meetings From 2017 Through 2019.

A, Proportion of trials according to meeting. B, Proportion of trials according to study sponsor.

Within the 26 studies with a negative primary analysis, authors emphasized a numerically better outcome in the experimental arm in 13 cases (50%), the positive result in 1 or more subgroups in 12 cases (46%), and the positive result in 1 or more secondary end points in 10 cases (38%). In 7 cases (27%), authors interpreted post hoc the study as a noninferiority design.

Discussion

Each of the reasons used to underline positive aspects of a formally negative result should be presented with caution, avoiding diffusion of methodologically equivocal statements. This is true for published papers,¹ and we show that it is relevant also for meeting presentations.

Many trials had a not-negative message despite a statistically nonsignificant primary analysis. We show that the risk of wrongly emphasizing borderline significance, already described in the oncology literature,² is present also in meeting presentations.

Positive subgroup analysis of negative trials can be misleading, should not support treatment adoption, and, at best, should be hypothesis generating.^3,4 In some cases, however, such as the STAMPEDE trial testing prostate radiotherapy in patients with advanced prostate cancer,⁵ methodological strengths and weaknesses of the subgroup analysis were correctly discussed by the study presenter.

Positive analyses of secondary end points in trials with a negative primary analysis carry the same risk of false-positive results inherent in multiple testing. Furthermore, the primary end point is the measure used for the study hypothesis and should condition study interpretation.

When a trial is designed to test the superiority of an experimental treatment, post hoc interpretation of noninferiority is methodologically debatable. The noninferiority hypothesis should be prospectively planned, with a clear definition of the margin acceptable to define noninferiority.

In conclusion, we believe that more attention should be paid to the statements included in the conclusions of oral presentations at meetings, and the discussants’ role is crucial. When the primary end point is not met, the word negative should be explicitly used.

Supplement.

eTable. Oral presentations of formally negative trials at ASCO or ESMO meetings between 2017 and 2019, with not-negative authors’ conclusions.