Updated guidance for trusted systematic reviews: a new edition of the Cochrane Handbook for Systematic Reviews of Interventions

On a shelf in the sunny, open‐plan office of Cochrane Australia in Melbourne, there's a large, white ring‐binder that, it's fair to say, hasn't been opened in a while. It's a printed copy of the original, 1994 edition of the Cochrane Collaboration Handbook, edited by Dave Sackett, and within it the original guidance on the methods to be used. The section on preparing and maintaining systematic reviews, edited by Andy Oxman, weighs in at a total of 76 pages.

From those fairly humble beginnings ‘the Handbook’ has become the go‐to resource for those wanting a guide to current best practice in conducting systematic reviews of interventions. It has grown in depth and breadth over the years, drawing on many dozens of contributors, and it receives tens of thousands of citations.

Now we have a new edition of the Handbook, its second edition in print and sixth overall, more than a decade after the last major revision. The new edition has been extensively rewritten and its new guidance reflects a decade of development by experts in research synthesis methodology.

Much has changed since 1994. In the original Handbook, the term ‘forest plot’ does not appear (although two early variations on the plot are presented) and no empirical evidence was available to inform assessments of risk of bias. The challenge of updating reviews was addressed in only four lines of text, and apparently many Cochrane Reviews were “not much longer than a structured abstract.” A further chapter, edited by Kay Dickersin, focused on a program to establish specialized registers (now a mainstay of Cochrane Review Groups), given the very real difficulty of identifying randomized trials in Medline at the time.

The 1994 Handbook acknowledged that development of systematic reviews was in its early stages, and that in many areas only general guidance could be given. It also noted that merely by including an explicit methods section, Cochrane Reviews would be “more useful to users than the vast majority of reviews that are currently available”. Since Cochrane's initial steps 25 years ago, the Handbook has supported the organization's drive towards innovative methods and its commitment to quality, and it still aims to assist authors to produce reviews that are “more useful to users”, whether policy decision makers, consumers, or health professionals.

Cochrane Reviews should answer important questions that are relevant to decision‐making. Reflecting the breadth of these questions, guidance for meta‐analysis is now supplemented by new guidance on intervention complexity and equity, and the guidance on the use of non‐randomized studies has been extensively expanded. Decision makers often need to decide among multiple intervention options, so a major new chapter addresses network meta‐analysis to support such decisions, and Cochrane is actively encouraging the appropriate use of this methodology.

Not all challenges are new, and many of the Handbook's chapters reflect detailed reconsideration of some of the most familiar challenges. New guidance provides in‐depth support for planning the review, constructing good review questions, and grouping included studies according to their populations, interventions, and outcomes for synthesis. This planning ahead will provide more support to authors at the analysis stages of the review and is of particular assistance for reviews with high levels of heterogeneity or multiplicity of outcome measures. In addition, updated guidance on meta‐analysis and new statistical methods are supplemented by an all‐new chapter on alternatives to traditional meta‐analysis for synthesis of results across studies.

There is also revised guidance on the basics, refreshing those core methods that underpin every review. Updated guidance on identifying sources of evidence includes information on sources other than published trials (such as clinical study reports), an extended technical supplement on sources to search, an introduction to the role of technical advances such as text mining and machine learning, and prospective approaches such as living systematic reviews. Substantial developments in guidance on risk of bias assessment are reflected, with an updated overview of key concepts supported by dedicated chapters on the RoB 2 tool for assessing bias in randomized trials, the ROBINS‐I tool for assessing bias in non‐randomized studies of interventions, and a new framework for considering reporting biases and bias due to missing results in a synthesis.

The main sections of the Handbook will be relevant to all authors of systematic reviews. For authors working with Cochrane, new online‐only chapters will provide guidance specific to Cochrane Reviews, covering the planning and logistics stages of Cochrane Reviews as well as guidance on reporting and updating reviews. The new Handbook is available in book form, and is also publicly available, free of charge, at handbook.cochrane.org. Recognizing that recommending a course of action is not the same as implementing it, Cochrane also provides a range of training and other guidance for authors, editors and other contributors to systematic reviews to assist them in meeting the high standards expected (training.cochrane.org).

The new Handbook draws on the expertise of over 100 contributing methodologists and editors located around the world, and in particular the efforts of the members of Cochrane's Methods Groups over many years. These international leaders in their fields conduct research to develop the evidence base that underpins the methodological guidance in the Handbook, ensuring that the findings of Cochrane Reviews rest on strong foundations. A wide group of peer reviewers also contributed their expertise. Working with this global community over the past few years has been an immense privilege, and a labour of love for many people. We are grateful for their insights as well as their exemplary patience and dogged persistence throughout this process.

While much has changed since that 1994 edition of the Handbook, there is much in its guiding principles that we recognize and continue to emphasize today. Its advice that “whatever is done, reviewers should clearly explain what was done, and why” and that “these guidelines are not a substitute for good judgement” continues to ring true. It also remains true that, alongside the hard work, “…the rewards are great. This opportunity to remain at the cutting edge of one's field is unparalleled. The fun and learning that accompany working with a world‐wide group of like‐minded colleagues are exceptional.”

We believe this most recent revision of the Handbook will be of use to all authors, no matter how experienced in their endeavours, in meeting the ongoing challenges of providing trusted evidence to support healthcare decision making.

Feedback on this editorial and proposals for future editorials are welcome.

End Notes

Declarations of interest

The authors have completed the ICMJE form for disclosure of potential conflicts of interest. JPTH and JT co‐edited the Cochrane Handbook for Systematic Reviews of Interventions and received honoraria for that work. JC, MC, TL, MJP, and VW served as associate editors for the Cochrane Handbook for Systematic Reviews of Interventions. JC was an employee of Cochrane until September 2018. TL also reports grants from the National Eye Institute, National Institutes of Health, not related to this Editorial. MJP also reports grants from a National Health and Medical Research Council Early Career Fellowship, not related to this Editorial.

Provenance and peer review

This editorial was commissioned and was not peer reviewed.