Fast news or fake news?

Abstract

Pre‐print servers have helped to rapidly publish important information during the COVID‐19 pandemic. The downside is the risk of spreading false information or fake news though.

Since the start of the COVID‐19 pandemic, hundreds of scientific papers that are related to the disease in one way or another have been uploaded to pre‐print servers for anyone to read and comment on. This has been a boon of early insights into SARS‐CoV‐2 for the research community, but there are also concerns that some of these attract more credence than they deserve. For some, pre‐print servers represent a windfall in rapid information at a crucial time for society, while for others, they are a murky meandering away from scientific rigour.

For some, pre‐print servers represent a boon in rapid information at a crucial time for society, while for others they are a murky meandering away from scientific rigour.

Speed versus quality

A prominent destination has been bioRxiv, an open access repository set up in 2013 and hosted by the Cold Spring Harbor Laboratory in New York, USA, along with its younger sister site medRxiv. The overwhelming motivation for setting up the repository was to accelerate the dissemination of research, according to co‐founder Richard Sever and editor of CSH Perspectives. “The faster you can get it out the faster people can start building on it and the faster science moves”, he explained. Pre‐print papers gave first insights into the spread of the virus in China [1], about its persistence on various surfaces 2 and warned about the importance of asymptomatic individuals and the high viral loads they can harbour 3.

All the advantages of speedier dissemination of information are writ large in a disease outbreak situation, Sever said. “Never has the importance of pre‐prints been more apparent than in the midst of a pandemic, when you simply cannot wait months and months for the work to be vetted [by journals]”, he stressed. “Speed is crucial”. Those at the frontlines agree that early insights have helped. “Scientists and authorities have had early access to important data and you don't have to wait until the publishing process is complete”, commented Clemens Wendtner, head of infectious disease at a teaching hospital of Ludwig‐Maximilians University in Munich, Germany.

Others see pre‐prints as a potentially dangerous source of misinformation that easily pollute news and social media streams, an egregious situation when public health messaging needs to be timely and accurate. “I think it is a disaster. I would get rid of these things if I had my way”, said Ralph Baric, virologist at the University of North Carolina, Chapel Hill, USA. While the scientific community has largely accepted pre‐print archives as a place to post manuscripts, Baric worries that it is an ideal format to put out fake information. “Because the instrument has been widely accepted by the scientific community, that work immediately becomes credible”, he explained.

While scientists are well aware that pre‐print manuscripts should be taken with a grain of salt, the public or the media may take it at face value. In normal times, pre‐print papers would not be scoured over for new information on a virus, but in the midst of the pandemic, everyone is eager for the newest information. “You can have controversial or unacceptable information in pre‐prints”, warned Burtram Fielding, virologist at the University of Western Cape, South Africa. His advice for non‐scientists, non‐virologists even was to wait for the peer‐reviewed paper. “The majority of journals are now fast‐tracking review for Covid‐19 publications”, Fielding noted. Indeed, journals have dramatically shortened time from submission to publication. “Most of that time was taken out of peer review time, so journals are asking peer reviewers to act really quickly”, cautioned Ivan Oransky, medical journalist and cofounder of Retraction Watch.

While scientists are well aware that pre‐print manuscripts should be taken with a grain of salt, the public or the media may take it at face value.

He also argues against putting blame on pre‐prints. “There isn't a pre‐print problem with bad information and badly done studies”, said Oransky. “There is a double standard being applied to pre‐prints”. He expressed concern that this focus on pre‐prints distracts from the real problems in scientific publishing and pointed to a recent commentary which warned that crises are no excuse for lowering scientific standards 4. This cited a National Academies Press report on the science carried out during the 2014–15 Ebola outbreak, which argued that “despite the sense of urgency, research during an epidemic is still subject to the same core scientific and ethical requirements that govern all research on human subjects”. “We don't seem to have learnt much from the Ebola outbreak”, Oransky said. He is concerned that peer‐reviewed literature might continue to insist that it “had really good filters and really effective quality control, but started publishing a lot of preliminary research that doesn't hold up”. He added: “We have already started to see retractions from peer reviewed journals”.

Questionable studies

Journals have indeed moved up a gear in publishing important papers, trying to keep pace with the demand for new information on COVID‐19. That demand is met by dipping into the deluge of pre‐print publications, many of which are on medRxiv. A paper loaded onto medRxiv on 8 March reporting that those with mild symptoms of COVID‐19 shed high levels of virus was promptly reported over the next week, often on medical news sites that did not note the preliminary nature of the findings. It was eventually published in Nature on 1 April, and largely justified the prior attention it received from the media 3.

Not all pre‐prints have had such an illustrious faith. A paper entitled “Uncanny similarity of unique inserts in the 2019‐nCoV spike protein to HIV‐1 gp120 and Gag” reported on supposed similarities between the new coronavirus and HIV. Published 31 January, it stated in the abstract that the similarities were “unlikely to be fortuitous in nature”. After a backlash among readers, the article was promptly withdrawn. For Baric, this paper highlighted the dark side of pre‐prints. His concerns go beyond the pandemic to the problems of fake news and manufactured disagreements among social groups in the USA. “This [bio‐archives] provides a legitimate vehicle for misinformation to get put out there and tear apart belief in science and scientific research”, Baric explained, noting that the HIV paper is still percolating through the Internet. “It supports this conspiracy theory stuff and idea that the virus was genetically engineered by angry scientists and that is unfortunate”.

Oransky though noted that the study was posted Friday afternoon and withdrawn Sunday afternoon. In contrast, it can take 1 or 2 years for retractions to happen in some peer‐reviewed journal even when they are provided with investigation outcomes by universities, he added. “I'm not worried [about pre‐prints]”, agreed structural biologist David Aragão at the Diamond Light Source synchrotron in the UK. “Information is a good thing. I would prefer that people read these pre‐prints than some crazy blogs on the Internet with crazy theories not based on science”.

However, pre‐print servers do take steps to ensure that submissions are appropriate scientific papers. Usually, it takes 24–48 h before a submitted paper goes on bioRxiv, 3–4 days on medRxiv, explained Sever. A basic screening process filters out opinion or commentary and checks for plagiarism. There are additional hoops for papers on medRxiv, and suspect manuscripts can be directed to outbreak experts, Sever added. “We don't want papers that say a vaccine is dangerous and then find out the paper is wrong, but in the meantime has changed public health behaviour”, he explained. He added that some computational predictions for COVID‐19 therapies have been directed towards peer review to avoid a rush on a drug that does not work or could be dangerous.

Dealing with a deluge of papers

The sheer quantity of preprints can be challenging. “There have been 50–100 publications a day in the peer reviewed literature, let alone the additional pre‐print publications”, said Nigel Curtis, clinician–scientist at Murdoch Children’s Research Institute and University of Melbourne in Australia. “There are so many reports that we need some sort of curator to sort through the papers, to review what is worthy and shows a major advance in the field”, said Tom Gallagher, coronavirus researcher at Loyola University Chicago. “The sheer volume is just too much for any one person”. To help out, the American Society for Microbiology has teamed up with coronavirus experts to vet peer‐reviewed and pre‐print papers on the disease to create a COVID‐19 Research Registry. “It is our response to the need of the research community for this resource. We have over 20 curators”, said Nguyen K. Nguyen, Director of American Academy of Microbiology at the society.

EMBO has also set up a curation platform for figures within life sciences papers called SourceData to make data within publications more easily searchable. “If you try to find a specific experiment on PubMed, it is difficult”, noted Thomas Lemberger, deputy head of scientific publications at EMBO. This curation platform could help to link pre‐prints to one another and allow readers download interconnected data from them. “We have started to pilot the curation of pre‐prints related to COVID‐19. It is feasible, though volume is an issue”, Lemberger added. Another roadblock can be publishing license which restrict re‐use of figures.

No matter the advantage of rapid dissemination, Curtis thinks the disadvantages of the burgeoning pre‐prints too often outweigh the speed. For him, the peer‐review process is tried and tested and journals are now expediting important papers. “Despite this being an emergency, it is important that we maintain scientific rigour”, Curtis stressed. He noted the value of pre‐prints getting crucial information out fast but added that that not everyone has the experience and expertise to evaluate early drafts of papers. “That's why we have peer review”, Curtis explained. “There's real danger that a lot of people don't realise that just because it is up online doesn't mean that there are not serious mistakes or flaws in it”. As Sever noted, it is therefore incumbent on journalists to check with other experts about pre‐print papers. “Amidst a pandemic, a lot more people are looking at these things than normally”, he said, so there is an onus on journalists to report these things responsibly, by noting for example that the paper is not peer reviewed and preliminary.

… there is an onus on journalists to report these things responsibly, by noting for example that the paper is not peer reviewed and preliminary.

Peer review is crucial

Indeed, most of the public do not know about pre‐print servers, peer review or how incremental scientific knowledge in reality is. “A downside is that stuff gets out there and people think it is true”, said immunologist Luke O'Neill at Trinity College Dublin, Ireland. “The whole goal of science is to publish and then get someone to repeat it. Then we know if it is really true. […] Now anybody can upload stuff and that means the quality is suddenly an issue. There is a function for journal peer review to screen out the dross, and if you don't have that, an awful lot of dross comes out”.

There is a function for journal peer review to screen out the dross, and if you don't have that, an awful lot of dross comes out.

O'Neill added that peer‐review works partly by guarding against baser human instincts. “Often people published for the glory, which is part of the human condition sadly. They will rush something out that hasn't been fully verified yet”, he said. “Then a non‐specialist might read the title, scan the abstract and not have the skill or time to dig into the data. That is the danger. Stuff that is untrue is then published”. The publicity surrounding chloroquine as a putative treatment for COVID‐19, egged on by hyperbole from US President Donald Trump, seems a classic case study in exaggerated promise on the back of scant evidence. In fact, a key paper on hydroxychloroquine was published within days of a pre‐print appearing, only to be criticised by the society that publishes the journal (https://www.isac.world/news-and-publications/official-isac-statement). It is not the only one of its kind: Oransky noted that one of the mostly highly cited medical journals just published a paper on 61 patients who received at least one dose of remdesivir, no placebo, no randomisation 5. Whether that would happen under normal circumstances is doubtful. Whether the paper will prove useful is equally doubtful. “This is unavoidable. […] We all must become our own reviewers and recognise rubbish from the nuggets of gold”, said Marc Van Ranst, virologist the Rega Institute for Medical Research in Leuven, Belgium, who takes a matter‐of‐fact view. “Rubbish will spread, rumours will spread. That is just a fact of life in 2020”.

Another issue is that many pre‐print papers are still works in progress. “I've read two or three [pre‐print] papers in my field and I can see that they are a bit preliminary and need a bit of work, but probably the bottom line is true”, O’Neill said. “Ideally, a pre‐print paper will be read by other scientists, who comment and help the authors improve it”. On the other hand, less than 10 per cent of papers on bioRxiv get comments, Sever conceded: “Public commenting is probably not at the level that one ultimately wants”.

A collaboration between EMBO and non‐profit ASAPbio created Review Commons which was launched in December 2019 to address this issue. Review Commons allows authors of pre‐prints to request peer review prior to submission to a journal. Readers can then view reviewers critiques and replies from authors, alongside the preprint manuscripts on bioRxiv. “This enriches the scientific discourse around pre‐prints by adding peer review”, Lemberger explained. With reviews in hand, authors can approach one of 17 journals onboard with the scheme, which includes eLife, EMBO Press journals and PLoS journals. Journals can make decision without having to start peer review from scratch.

Pressure to publish

With the pandemic continuing, there is even more pressure on researchers to release findings quickly, perhaps prematurely. One reason for putting up work‐in‐progress is primacy, to stake a claim for a discovery by putting it in print, before having to wait for a journal and for referees. “It encourages people to put up halfway observations that aren't complete and grab credit for concepts that they haven't actually completed”, Baric criticised. He recently loaded up a pre‐print paper on bioRxiv while it was in review at Science Translational Medicine. “Some of the postdocs I work with really want to do this”, he explained. “What I usually say is let us submit the paper and make sure it goes out for review, so at least some set of editors somewhere said ok this is high enough quality to be reviewed”. Sever believes that some valuable COVID‐19 papers may never be peer reviewed. “Things move on and authors may feel that they don't need to formally publish”, he explained.

With the pandemic continuing, there is even more pressure on researchers to release findings quickly, perhaps prematurely.

Ironically, while a pre‐print paper on COVID‐19 can be widely reported on news, websites and in magazines and journals, some authors must keep their heads down and refuse interviews if they have submitted the paper to scientific journal. “We do ask that, when we are considering a paper, that researcher don't give interviews to journalists about our research. That is our publication policy”, confirmed Magdalena Skipper, chief editor of Nature. This puts scientists in an unusual situation: they can have their paper splashed across headlines around the world, perhaps with overblown claims, yet cannot themselves talk about or contextualise their own research. “The top journals want us to deposit in the public domain even before they go to review, but at the same time we are not allowed to have a press release and they say don't talk to journalists”, said Lin‐Fa Wang, emerging infectious disease scientist at the Duke Global Health Institute in Singapore, when asked about a pre‐print he had released on SARS‐CoV‐2. Sever at bioRxiv agrees that the situation as untenable and rife with contradictions. He noted the irony in researchers being told by journals not to talk to reporters, but yet the same journals send reporters to meetings where researchers give the community early insights into their findings.

Some do see some value to the policy. Resources like bioRxiv are best suited to other researchers and public health policy experts, and not quite so much for the media, said Jeremy Rossman, virologist and honorary lecturer at the University of Kent, UK. “There is a strong argument for authors not talking to the press”, he explained. “If you have an author responding and going on social media and talking about a paper that is still under peer review, that would lend credence to this idea that it is already a published paper”.

Indeed, the homepage of medRxiv warns that “preprints are preliminary works that have not been certified by peer review” and should “not be reported in news media as established information”. Sever said this sets pre‐prints apart from other sources of information and asked that the amount of misinformation should be kept in proportion to what is available on the web. “The consequence of a cell biology paper being wrong are negligible. A few academics might waste some time. The consequence of being wrong about a therapeutic prediction is different”, he said.

A recent pre‐print paper on medRxiv looked at the early scientific literature in response to COVID‐19. “According to my empirical count, I noticed that COVID‐19 pre‐print length are shorter and often describes one or two experiments”, noted Gaetan Burgio, geneticist at the Australian National University in Canberra. He notes that an incredible amount of data is being shared in real time, but the trade‐off is that the quality of the papers is lower and some of those are truly bad. “I would call this ‘information pollution’, as many researchers are trying to get into COVID‐19 research with a variety of degree of success”, Burgio commented. “While this is focused on pre‐print, I have observed this too for peer reviewed articles in prestigious journals such as NEJM, PNAS, or Cell. No publishers are exempt”.

Notwithstanding, pre‐prints are here to stay, being accepted by most scientific publishers. “This is unstoppable”, Gallagher said. “There will always be a vehicle for transmitting findings and disseminating them independent of peer review”. Like many, he sees value in reading good research as soon as possible, but “buyer beware” he warned. Oransky said that a similar tag should be added to all scientific publications.