Moving beyond journals: the future arrives with a crash

“It’s easy to say what would be the ideal online resource for scholars and scientists: all papers in all fields, systematically interconnected, effortlessly accessible and rationally navigable, from any researcher’s desk, worldwide for free.”—Stevan Harnad

Three hundred years of print journals have bequeathed us almost the exact opposite of the ideal proposed by Harnad, one of the leading thinkers on how the internet will change science. In biomedicine thousands of journals fragment information into largely arbitrary groupings and charge users for access. The burgeoning of the world wide web (almost all researchers and three quarters of doctors in the developed world now have access) makes it inevitable that new systems of disseminating research will replace or at least supplement traditional journals. Concrete proposals for new systems are now appearing, with an ambitious one from the National Institutes of Health leading the way.¹,² We are about to enter a period of what the Austrian-American economist Joseph Schumpeter called “creative destruction,” and only some of us will still be in business at the end. And those of us who still exist will not be doing exactly what we are doing now.

What’s the problem?

The present system of disseminating research through journals has many failings (see BMJ’s website for a summary of defects in journals), but there are two main drivers of change. One, as almost always, is money. The academic community, particularly in the United States, has come to resent the money sucked out of the research system by publishers.³ Most research is funded with public money, yet the US Association of Research Libraries spends $432m (£270m) ($12 000 (£7500) for each scientist) buying research journals.⁴ In the face of declining subscriptions publishers have long been putting up their prices each year by considerably more than inflation. Something had to give.

Many researchers think that publishers do not add sufficient value to justify the large profits that some of them make. Consider the case of the traditional research journal, which contains little but research. The researchers do the research, edit the journals (often unpaid), do the peer review (almost always unpaid), often do the copy editing (again unpaid), buy the journals (often at inflated prices in thousands of dollars), read the journals, and store them. Publishers may own the journals (although often they don’t); manage the process; do the design (usually minimal); typeset, print, and distribute the journals (not processes that add much value and potentially done much more cheaply on the internet); market the journals (but often to libraries that have no choice but to have them); and sell advertising (often none). They may also sell reprints of articles—often for hundreds of thousands of dollars at a time without giving anything to authors or funders of the research.

General medical journals do much more than the stereotyped research journal we’ve depicted here, and they are usually cheaper than most research journals. But the big ones make substantial profits, and the editors of the New England Journal of Medicine and the Lancet (the latter owned by the giant publisher Reed-Elsevier) might have been more open than they were about their own competing interests when questioning the proposal from the National Institutes of Health proposal.⁵,⁶ (We declare our competing interests at the end of this article.)

The second big driver of change is what’s been called “the Balkanisation of the research literature” that arises from there being too many journals and too many publishers.Anybody who has ever attempted a systematic review knows that it’s extremely difficult to find all relevant research studies and very expensive to get copies once you do locate them.

Harold Varmus, director of the National Institutes of Health, has proposed a bold solution to these problems called “E-biomed,” a website where all biomedical research would be accessible through a powerful search engine for free.¹,² The site would have two parts: one would post research after peer review by traditional journals; the other would allow the posting of “virtually any legitimate work” on the say so of two approved reviewers. Varmus’s proposal has produced a cacophony of complaint from publishers. Our reaction is that it is not radical enough to produce a long term solution yet may be too radical to achieve implementation.

The NIH solution

A more radical step would have been to recognise the severe deficiencies of peer review for deciding what to publish.⁷,⁸ Instead, E-biomed could have used peer review to improve what is published and to start a scientific discourse around a study. Almost anything then could have been published together with commentaries. Those accessing the site could have added further comments if they so wished. The site might then have been layered by simply counting the number of hits received by studies. Our bet is the best studies would quickly move into the gold layer while the many unimportant studies would fade away. Whatever happens the National Institutes of Health should surely use this period of change to encourage research into peer review and make it more evidence based, as we and others have argued.^9–11

One major worry with Varmus’s proposal is that it might mean that the US will control what is acceptable and what isn’t in biomedical research. This anxiety can, however, be addressed by partnerships with organisations from outside the US, and there are signs that this is happening.¹² Many within the US will not be keen, however, on research dissemination being run by the federal government, and there are likely to be many political objections to Varmus’s plan—not least from the many vested interests that are threatened.

A smaller step from the BMJ and Stanford University libraries

Because Varmus’s plan is likely to get bogged down in political and commercial wrangling, we plan to press ahead with an experiment that may work because it has already worked within the high energy physics and astronomy communities. The BMJ Publishing Group intends following their lead by setting up an eprint server for clinical medicine and health research in partnership with Stanford University libraries (see BMJ’s website for details). The server will, we hope, be useful to researchers. We do not expect it to be much use to doctors who are not researchers, and nor would E-biomed be of much use to them. They are likely to continue to want to receive predigested, well presented accounts of research that matters for their practice. This is a role that journals are likely to continue to have in the future, although again the existence of the internet may allow entrepreneurs to develop better ways to do it.

Eprints (electronic preprints) are versions of articles that have been circulated via the internet before publication in a peer reviewed journal; an eprint server allows collections of these to be accessed over the world wide web. Eight years ago Paul Ginsparg set up an eprint server for high energy physics, which has now supplanted traditional physics journals as the means of first publication in the field.¹³ It has not led to the demise of peer review, and peer reviewed journals remain the destination of most eprints after review and revision. Some journals now accept submissions directly from the server, thus streamlining the peer review process. Direct reader feedback to authors is possible via the server, and Ginsparg says that “subsequent revisions frequently benefit as much or more from direct reader feedback as from the conventional referee process.”¹⁴

The experiment has clearly worked for high energy physics. Its eprint server receives about 2500 submissions a month and serves 30 000 distinct hosts a week.¹⁵ How might a similar server benefit clinical and health researchers? Locating the full text of studies on their completion would be easy. (And a next step in the experiment might be to include the registering of studies as they begin—allowing those contemplating doing some research to know what’s already under way and countering the problem of publication bias, whereby studies with “negative” findings never appear.) Gone would be the delays as journals took their time to agree a publishable version of an article, and more time to publish it. Currently, this has meant that articles published by their author’s journal of first choice are kept out of circulation for months, while those that bump down the hierarchy of journals before finding a taker could be kept out of circulation for years. Once a study has been published, Medline has been the traditional way to find it, yet Medline indexes only the top 4000 journals, and rarely provides more than an article’s abstract. To date, most research reported in the remaining journals—especially those with low circulations—might as well not have been done given its inaccessibility.

These are not necessarily poor studies that deserve to lose in this Darwinian struggle for attention—merely those that represent too modest an advance over previous work or that fail to meet the criteria of importance or interest current at the time. This matters for several reasons. Firstly, researchers whose subjects are patients have an ethical obligation to share their findings with others, as patients will usually have agreed to become research subjects in the belief that their actions would benefit others.

Secondly, when researchers are systematically reviewing the literature they need to find all relevant studies, and publication bias makes their job difficult if not impossible. Thirdly, it’s not only systematic reviewers who want to identify all relevant research: to avoid duplication of effort other researchers and funding agencies need to know what research had been completed—even if not yet published in a peer reviewed journal.

Two main arguments are advanced against eprint servers. The first is that they would silt up with poor quality information, becoming useless to researchers. This view erroneously credits peer review with being a good method of keeping poor quality work from publication, whereas the evidence suggests that with persistence, even the most flawed work will eventually find a home.⁹ Considerably less than 5% of papers appearing in current journals contain a message that is both scientifically sound and relevant to doctors.¹⁶

The second argument is that patients may come to harm through acting on the basis of poor science. Yet many unreviewed research findings already find their way into the public arena, via authors and conferences—and press conferences. The appearance of a full account on an eprint server would be far preferable to what happens now, when patients and doctors may have read a garbled account in a newspaper and have no way of accessing a full study that can be appraised. In future authors who go public on their data without posting it on an open forum where others can append their comments will raise questions about their credibility and motives.

It’s important to remember that most interventions of potential harm to patients (such as surgical operations and the prescription of drugs) would need to be initiated by doctors, who are famously conservative when it comes to changing their practice on the basis of what they’ve read. In any case visitors to the eprint server will have to read a disclaimer before entering the website, and each individual article will have a disclaimer cautioning people from acting on the basis of research that has not been peer reviewed.

That an eprint server in clinical medicine would help rather than hinder clinical and health researchers is a hypothesis that is both testable and worth testing. The eprint looks like the first substantially new form of scientific communication since the peer reviewed article, and as we’re in the business of transmitting scientific information, it makes sense for us to work to find the eprint’s right place in the new digital environment.