Systematic reviews and uncertainties about the effects of treatments

Stephen Wood/Science Photo Library

One of the common criticisms made of The Cochrane Collaboration is that most of its reviews don't end with clear instructions for action by professionals, patients and policymakers. Since its inception, however, the Collaboration has made it clear that research evidence isn't everything. Other factors need to be taken into account when people decide what action – if any – is needed when making treatment and policy decisions. And it is very unusual for these other factors to be the same the world over. The Cochrane Collaboration is an international organisation, and its business is to provide reliable information, not to offer practice or policy advice as if this could be applied universally.

That said, a very common reason for the ‘Implications for practice’ sections of Cochrane Reviews leaving readers dissatisfied is that there is simply inadequate research evidence to provide reliable information to guide the choices people must make. If this is the reality, however, people are faced with two main options: either to sweep the uncertainties under the carpet so that they can be ignored, or to endeavour to reduce them by doing additional research. Sometimes the additional research should start by updating or extending the scope of existing Cochrane Reviews; sometimes by preparing new systematic reviews; and sometimes by promoting additional ‘primary’ research because reliable, up‐to‐date reviews have shown that that is what's needed.

The importance of this issue is indicated by the recognition that people have suffered and died unnecessarily as a consequence of uncertainties about the effects of treatments not being addressed . Indeed, the UK General Medical Council's advice to doctors is that they “must work with colleagues and patients … to help resolve uncertainties about the effects of treatments” .

Identifying and prioritising treatment uncertainties

To help ensure that administered treatments do more good than harm, gaps in knowledge about their effects – uncertainties – must be identified, and those deemed sufficiently important must be addressed. The UK Database of Uncertainties about the Effects of Treatments – UK DUETs (www.library.nhs.uk/duets) – harvests and publishes treatment uncertainties. The development of UK DUETs has been overseen by Mark Fenton (who has worked for many years with the Cochrane Schizophrenia Group). UK DUETs has now been incorporated into NHS Evidence (www.evidence.nhs.uk), a gateway launched in 2009 by the National Institute for Health and Clinical Excellence (NICE). The Cochrane Library is one of the richest sources of the treatment uncertainties published in UK DUETs. Indeed, the lack of clarity in some ‘Implications for research’ sections of Cochrane Reviews prompted the UK DUETs Development Group to present guidelines for improving these at the Dublin Cochrane Colloquium in 2006, and to publish the guidelines in the BMJ.

UK DUETs is accessible to anyone interested in seeing which treatment uncertainties have been identified, but UK DUETs is fundamental to the work of the James Lind Alliance (www.lindalliance.org). The James Lind Alliance is a non‐profit initiative, established in 2004, which is funded by the English National Institute of Health Research and the UK Medical Research Council. The James Lind Alliance brings patients and clinicians together in ‘Priority Setting Partnerships’ to identify and prioritise the unanswered questions about the effects of treatments that they agree are most important. Research on the effects of treatments often overlooks the shared priorities of patients and clinicians. The pharmaceutical and medical technology industries and academia play essential roles in developing and testing new treatments, but their priorities are not necessarily the same as those of patients and clinicians. Many areas of potentially important research are therefore neglected.

The James Lind Alliance Guidebook

The James Lind Alliance facilitates the creation and work of Priority Setting Partnerships focusing on specific health problems. The Partnerships:

• bring patient and clinician groups together on an equal footing;

• identify treatment uncertainties which are important to both groups;

• work with both groups jointly to prioritise the uncertainties; and

• produce a ‘top 10’ list of jointly agreed uncertainties as research questions to be presented to funders.

This week, the James Lind Alliance has launched a Guidebook (www.jlaguidebook.org/) to help others to benefit from the diverse experiences that the Alliance has acquired over the past five years . The Guidebook explains how to establish and support Priority Setting Partnerships, and the principles lying behind these methods. A key component of the Guidebook is the James Lind Alliance Protocol (jlaguidebook.ds5150.dedicated.turbodns.co.uk/jla‐guidebook.asp?val=83). This is for Priority Setting Partnerships to follow if they want to work with the James Lind Alliance.

Prioritisation and The Cochrane Collaboration

The James Lind Alliance is a UK‐based initiative and makes no pretence that the research priorities identified using its methods in the UK are necessarily those that would be chosen elsewhere. That said, the Alliance's methods appear to be of interest and relevance more widely. Indeed, a Cochrane Methods Group on priority setting has been proposed by Mona Nasser and others. There are already a few Cochrane entities among over 200 organisations and individuals currently affiliated to the James Lind Alliance. As James Lind Alliance affiliation is a cost‐free way of staying up to date with James Lind Alliance activities, additional Cochrane entities might consider affiliation. They would certainly be very welcome.

Erratum (20 May 2010): The original reference “GMC 1996” has been corrected to “GMC 2006” in the text and reference list.

Feedback on this editorial and proposals for future editorials are welcome.

End Notes

Declarations of interest

No conflicts of interest declared.