Why promote the findings of single research studies?

“Since when has a single scientific study constituted ‘the truth’ about anything?”¹

Scientists have known about biases in single observations for centuries.² A wealth of empirical evidence amassed across many disciplines tells us that single studies can be biased, are often seriously methodologically flawed and highly time and context dependent, and have findings that are likely to be misinterpreted and misrepresented (sometimes by the authors themselves). Increasingly it is accepted that decisions should not be based on the findings from single primary studies but rather informed by actionable messages derived from synthesised evidence based on systematic reviews.^{3 4 5} Over the past decade there has been substantial public funding of synthesised evidence and guidance to support healthcare decision making. In the United Kingdom this investment has been described as NHS research and development’s most important contribution to the global science base.⁶

Despite this investment the evidence indicates that although the transfer of research knowledge is possible its success can be variable.^{7 8} There is now renewed interest and emphasis on the gaps between research and policy and practice, nationally^{9 10} and internationally.¹¹ This emphasis on bridging the gap may be viewed as encouragement to strive even harder to promote research to ever wider audiences—to be seen to be providing a return on investment. However, it may be worth pausing to consider the benefits and costs of such activity and ask why the research community continues to place such emphasis on the promotion of the primary research study.

The emphasis on the single study drives submissions to the current UK Research Assessment Exercise (RAE). Although the RAE does recognise that original research may include systematic reviews, the practice of many academic organisations is to prioritise—or sometimes only include—primary research, under the assumption that systematic reviews are not relevant or are less important.

Most bodies that commission research also expect and demand some commitment or effort on the part of grant holders, regardless of study design, to disseminate in ways that go beyond the traditional medium of academic publication. The emphasis is on communicating and interacting with wider policy and health service audiences in ways that will facilitate uptake of research results in practice and policy. The rationale, appropriate conditions, and contexts for such interactions are rarely provided by the funding body, and nor is any acknowledgment that evidence set in context is of most value to decision makers.

Some medical journals encourage researchers to discuss their findings in the context of the existing and relevant evidence base. However, research suggests that little progress has been made by researchers and journal editors in ensuring that single studies are set in context or that the consolidated standards of reporting trials (CONSORT) recommendation is adhered to.^{12 13 14} Furthermore, every week journals issue press releases aimed at generating publicity for the most “newsworthy” studies and for the journal itself. There is some evidence that primary research studies are proportionally more likely to be included in press releases than systematic reviews.¹⁵

The emphasis on newsworthy research promotion is not restricted to academic journals. Universities, research centres, funders, and councils all use the media to promote research. Rather than representing an attempt to explain new findings in the context of existing knowledge, media strategies are used to build the corporate image of the host institution or funder, for agenda setting, and to attract and secure future funding. Using the media to promote the results of primary research can have harms and can erode trust in and understanding of science generally.^{16 17 18 19 20} More often than not the latest breakthroughs, miracle cures, and wonder drugs are based on single studies.²¹ Often buried within the press release are notes of caution and suggestions that further research is needed and that a viable treatment is actually years away. We all know that these disclaimers are rarely if ever reported prominently; and the research community, while quick to raise concerns about media accuracy, rarely questions the appropriateness of media dissemination.

The renewed interest and emphasis on the knowledge translation is to be welcomed. But more than ever there is a need for researchers, especially those conducting primary research, to consider carefully the costs and benefits of dissemination. Not every study needs to be widely disseminated. Most primary research needs to be set in context, verified, and built on, moving the field forward incrementally before it can then have wider application. Rather than being encouraged to find ever more creative ways to get research noticed, funders should encourage researchers to demonstrate that they have considered carefully the appropriateness of their plans for dissemination.

Medical journals can do more to ensure that researchers actually do discuss the findings of primary studies in the context of the existing and relevant evidence base. The Academy of Medical Sciences in London has recently argued that researchers, funders, and institutions should take greater responsibility for the accurate communication of non-experimental research.²⁰ In truth, the research community as a whole needs to be more circumspect when it comes to the active promotion of primary research. Although all research has an audience, and should be made accessible, not all research can or should have an impact on practice or policy.