Bringing evidence from press release to the clinic in the era of COVID-19

Abstract

The urgent need to develop effective therapeutics and disseminate information from clinical studies has led to data from clinical trials being made available by alternate methods prior to peer-reviewed publication, including press releases, social media and pre-print papers. While this allows clinicians more open access to these data, a trust has to be placed with the investigators releasing these data without the availability of scientifically rigorous peer review. The examples of results from trials studying dexamethasone and hydroxychloroquine for treatment of COVID-19 have had contrasting outcomes, including the potential for significant numbers of lives saved with the early release of results from the RECOVERY trial studying dexamethasone contrasting with unsubstantiated data being presented from trials studying hydroxychloroquine. Clinicians and researchers must maintain a healthy scepticism when reviewing results prior to peer-reviewed publication, but also consider when these opportunities may allow for early implementation of potentially lifesaving interventions for people infected with COVID-19.

Significant debate exists as to whether results of randomized clinical trials with the potential to alter clinical practice should be publicly available prior to peer review. The rapid release of evidence prior to peer review could be potentially beneficial or deleterious for patients with prime examples including data surrounding the use of dexamethasone and hydroxychloroquine.

The RECOVERY trial reported increased survival for hospitalized people with COVID-19 receiving dexamethasone, with the greatest benefit seen in people requiring invasive mechanical ventilation.¹ These results were initially released via a press release on 16 June 2020, followed by pre-print publication on 22 June and the final peer-reviewed publication on 17 July. The early release of results allowed clinicians to consider the use of this medicine prior to the peer-reviewed publication by which time guidelines in the UK and the USA had already been updated to recommend the use of dexamethasone.^1–3 If we consider the 31 day period from 16 June and 17 July an additional 153 043 deaths and 5.79 million new cases were reported globally.⁴ It is difficult to speculate on the uptake of dexamethasone with the staggered release of this information, but it may be possible to conservatively estimate the impact on mortality from this early release of results. Based on a study of over 44 000 SARS-CoV-2 PCR-confirmed cases we can estimate the proportion of people aligning with the categories that derived benefit in the RECOVERY trial. In this report from China 81% of individuals were categorized as mild, 14% as severe (i.e. dyspnoea, respiratory rate ≥30 breaths per minute, blood oxygen saturation ≤93%, partial pressure of arterial oxygen to fraction of inspired oxygen ratio <300 and/or lung infiltrates >50% within 24 to 48 h) and 5% were considered critical (i.e. respiratory failure, septic shock and/or multiple organ dysfunction or failure).⁵ These categorizations align with the RECOVERY trial, namely the severe group consistent with individuals who received oxygen therapy in the RECOVERY trial and the critical group aligned with the group who received invasive mechanical ventilation. If we apply these proportions to the number of new infections globally in the 31 day period and apply the reductions in mortality from dexamethasone to the severe and critical groups we can estimate the number of lives potentially saved for varying levels of dexamethasone uptake (Table 1). Based on 25%–50% uptake of dexamethasone over this period, 15 000–30 000 deaths were possibly prevented.

Table 1.

Global estimate of mortality with varying uptake of dexamethasone in severely and critically unwell individuals during a 31 day period from 16 June to 17 July 2020

Uptake of dexamethasone	Severe group		Critical group
	mortality estimate (95% CI)	lives saved	mortality estimate (95% CI)	lives saved
0%	212 334 (204 578–216 767)	0	112 639 (107 043–118 363)	0
25%	203 824 (191 209–214 161)	8510	104 272 (95 467–114 060)	8367
50%	195 314 (178 060–211 311)	17 020	95 906 (84 028–109 598)	16 733
75%	186 805 (164 981–208 386)	25 529	87 540 (72 634–105 088)	25 099
100%	178 295 (152 143–205 212)	34 039	79  174 (61 394–100 414)	33 465

Based on 5.79 million new infections, a 28 day mortality using standard of care of 26.2% in severe and 41.4% in critical cases and an age-adjusted absolute risk reduction of 4.2% (95% CI = 1.4%–6.7%) in severe and 12.3% (95% CI = 6.3%–17.6%) in critical cases with dexamethasone use. CIs were calculated by propagating the uncertainty from the RECOVERY trial and assuming the result is binomially distributed.

As another example, the debate around hydroxychloroquine as a treatment for COVID-19 has been highly politicized and influenced by reports with inadequate scrutiny, with unreliable data sources or making claims that cannot be substantiated. As an example, peer-reviewed science on this agent has been retracted as primary data sources could not be provided by authors to verify the results.⁶ In other circumstances, the peer-review process has identified critical shortcomings, but not led to the retraction of these publications.⁷ This highlights the critical role of medical publishers of peer-reviewed science in maintaining high levels of scientific integrity. Publication of clinical-trial data after peer review that contain spurious or misleading information has the potential to lead to harm. As an example, one meta-analysis has reported that the combination of hydroxychloroquine and azithromycin may actually increase the risk of death in people with COVID-19.⁸ Multiple large peer-reviewed randomized trials have demonstrated lack of benefit for hydroxychloroquine on clinical outcomes^9–11 and it is now not recommended for treatment of COVID-19 in international guidelines and has had its emergency-use authorization revoked by the US FDA. Despite this, multiple scientific groups continue to support the use of hydroxychloroquine for treatment of COVID-19^12,13 and the early promotion and politicization of this drug prior to the availability of peer-reviewed randomized trials has likely undermined confidence in the results from these randomized trials.

Findings from the dexamethasone RECOVERY trial are now supported by other trials.¹⁴ Research conducted by large and experienced clinical trials teams is more likely to be accurate prior to peer review, as was the case in the RECOVERY trial, and this likely led to increased survival of individuals with COVID-19 between the press release and peer-reviewed publication. The early release of study results by press release, social media or pre-print publication means everyone has the ability to read this information, and not just experts in the field. While this democratization of data can be beneficial, if the data are misinterpreted by the investigators or people consuming the information, or even found to be unreliable, it can have damaging downstream effects and undermine confidence in the scientific process. Peer review is not a guarantee of sound science, but it remains a crucial step to ensure the research is rigorous. It provides a dose of healthy scepticism to hopefully improve the content of the research and has the ability to impact conclusions drawn from the data. However, the COVID-19 pandemic has resulted in rapid review becoming commonplace, with the majority of COVID-19 publications accepted within 1 week of submission.¹⁵ While accelerating the publication of evidence this also carries the risk of undermining the quality of the COVID-19 evidence base. Ultimately, releasing results prior to peer review and acting on these data requires trust in these results. This balance between trust in the science and the rigour of peer review likely differs based on specific features of the trial, but needs to remain at the forefront of our mind when interpreting results of clinical studies and considering changes in clinical practice based on these reports.

Funding

A.J.S. is supported by an Australian National Health and Medical Research Council Early Career Fellowship (GNT1141398).

Transparency declarations

J.H.M.’s institution has received funding for the conduct of clinical trials from ViiV, Gilead and Merck. M.J.L. and A.J.S.: none to declare.