Sullied

Abstract

The recent retractions of two high‐profile articles based on unverified data from a private company jeopardizes public trust in academic research in times of crisis.

The visibility and the standing of science and scientists have grown immensely during the COVID‐19 crisis. Many heads of state heed the advice of the newly appreciated experts. The advisory panels, who influenced governments’ decisions to close or open society, are populated by epidemiologists, virologists, infectious disease researchers, medics and others. People understand that the future of society is now in the hands of those scientists who are working around the clock to develop and test vaccines against the virus that has had the greatest impact on humanity since World War II. White coats could become a fashion statement. The research community is in the spotlight, and our work and our advice carry an enormous authority that is challenged by few.

Then, there was the fiasco in June when The Lancet and the New England Journal of Medicine retracted two crucial articles. The paper in The Lancet raised safety concerns about the use of hydroxychloroquine to treat SARS‐CoV‐2 infections. The NEJM article pointed to an increase in hospitalized COVID‐19 patients who had received angiotensin‐converting enzyme (ACE) or angiotensin receptor blockers (ARBs). These are journals held in high esteem, and the original publication of these articles made headlines around the world. And then, within less than a month of grabbing attention, both studies were retracted.

The overlapping point of both articles was the source of the data. Surgisphere, an American healthcare analytics company, claimed to have collected information on COVID‐19 patients from all over the world, and eager researchers parsed and analysed the data and wrote these spectacular articles. The journals were eager to accept the papers, which would have been cited extremely highly and added to their reputation as “must‐have” publications. Now they have a different reputation. The problem is that it is not only those journals that have egg on their face: Science in general has been sullied at a time when society badly needs more factual and reliable research to navigate through the COVID‐19 crisis. It will make it easier for some politicians, commentators and believers in quackery to brush off uncomfortable data.

These retractions have exposed a number of flaws in the academic research and publication system. The starting point of the papers was data sets obtained from Surgisphere which seems to have pushed for a strategy of “big data” sourced from all over the world to grow the company. However, it may have been a case of “never mind the quality, feel the width” to quote from an old TV series based on a tailor's shop. Apparently, some of the authors had not seen any primary data. This should not happen. Of equal concern is that Surgisphere, hiding behind a thin veil of “protecting confidentiality,” refused other researchers to see and probe the data. Even if the data may not have been “rubbish,” it is still a case of “unverified data in—unverified results out.” If the primary data are not accessible to verify findings and to ensure there are no selective exclusions, then buying the data or allowing co‐authorship to the company that sourced the data can lead to an uncomfortable situation. If “confidentiality” is used to block access by others, the concept of Big Data, as the future source of wisdom to help understand complex information, is sullied.

The drive, by the authors including the company, to get a paper published in a highly visible journal—which is surely helpful for investments, careers and job prospects—seems to have made the authors ignore some basic steps in generating and presenting reliable results. The public is sensitized already by the claim that a high percentage of published papers cannot be replicated for a variety of reasons (Begley & Ellis, 2012; Baker & Dolgin, 2017). A high‐profile case, such as the retraction of these papers from leading journals, is not helpful to maintain and improve public trust on which publicly funded research relies. The reputation of research is sullied.

When these papers arrived at The Lancet and NEJM, the editors must have clearly seen their potential for visibility, citations and everything else that comes with a great story. However, the bigger the impact of the paper, the greater the need for diligence and thorough peer review. It would therefore be of great interest to see the reviewers’ comments and whether they raised questions or concerns about the underlying data. Lancet however hides behind the same thin veil as Surgisphere, claiming confidentiality as a reason for not sharing the reviewers’ reports. It is noteworthy though that, merely a week after the Lancet paper was published, 120 researchers had questioned its validity. That was quick and it shows that the reviewers selected by the journals for their expertize should have raised the same questions. If they did not, then the peer review system is sullied.

At the end of peer review, an editor has to balance the reviewers’ comments and the replies of the authors and make a choice whether or not to accept the paper. In this case, the editor chose to accept it. Maybe, this decision to publish included noble reasons. Hydroxychloroquine had created a major buzz around the world as a potential remedy for COVID‐19 patients to help them cope with the infection. The topic had even become political with US President Donald Trump publicly promoting the prophylactic properties of the drug. Publishing a paper that ostensibly demonstrated the risks of treating a SARS‐CoV‐2 infection with hydroxychloroquine would have made the peddlers of the drug look foolish. The NEJM walked a similar path with its acceptance and later retraction of a paper claiming that ACE inhibitors affect cardiac health in patients infected with COVID‐19 again based on data from Surgisphere. Was this just poor decision making by the editors, poor peer review, failure to recognize the crucial aspect that nobody checked the original data, or were all willingly duped in the rush to have a “hot” paper? The fact that there is immense pressure on authors and journals to quickly review and publish any COVID‐19 relevant research did not help either. No matter what the driver was, the outcome is that the reputations of these “leading” journals are sullied and that stain spreads to other journals.

At a time when society discovers that scientists and research are essential for managing a global health crisis, it is most unfortunate that researchers, reviewers and editors in key positions to promote the rigours of the analysis and interpretation were found to be slipshod and sloppy. It is easy to say that we all have to do better, but the genie is out of the bottle. The battle for truth over fake news has suffered a setback. The veracity of a Press Release that promotes a solution with commercial benefits will sit equal to a thoroughly researched and peer‐reviewed paper. The system has been sullied.

Frank Gannon is Emeritus Fellow and former Director of the QIMR Berghofer Medical Research Institute in Brisbane, Australia. He was the founding Senior Editor of EMBO reports from 2000 until 2009.

EMBO Reports (2020) 21: e51371