The impact of Cochrane Reviews

Research teams who prepare Cochrane Reviews are made up almost entirely of volunteers. The task is onerous, perhaps more so than people who come to this for the first time can imagine, and over time it has become more challenging, as methods aimed at improving internal and external validity advance. Therefore it is interesting to wonder what incentives there are for these teams, who continue nonetheless to complete or update over 500 reviews annually for publication in The Cochrane Library?

Firstly, it is almost certainly the case that no other journal provides such extensive support to its would‐be contributors. Whereas journals traditionally base their decision on whether to accept or reject full article ‘completed’ manuscripts, Cochrane Reviews are registered at the outset and developed in conjunction with a network of support. This includes training provided by members of the Collaboration and Cochrane Centres, guidance made available in the Cochrane Handbook for Systematic Reviews of Interventions, technology support via the information management system, and the extraordinary contribution and assistance provided by Cochrane Review Groups through their search and editorial teams, and peer reviewers. As a consequence of these support systems, many reviews that would perhaps have been rejected by another journal are nurtured through to achieving the rigorous quality requirements of published Cochrane Reviews.

However, whilst this access to support might influence the choice of Cochrane as a recipient of a given review, it is likely only a small part of the incentive for a researcher – which must be to achieve the highest possible visibility for the proposed review and to impact on clinical practice and health policy. Therefore reviews being used by national and international agencies and guidelines groups such as the World Health Organization, being used by patients, health professionals and consumers to inform decision‐making, or being incorporated within decision support applications and e‐textbooks, all meet the requirement of delivering “impact”. Reviews can demonstrate influence in other ways also; witness the effect of a recent Cochrane Review in influencing major airlines to stop marketing devices to prevent malaria that had clearly been shown to be ineffective . One can also measure impact by usage, and again the figures are encouraging – showing a 19% on year increase in visits to The Cochrane Library on the Wiley Online Library, over 1 million hits per month globally, and a search of this site every 1 second, with an abstract view every 2 seconds and a full‐text view every 3 seconds.

Finally, there is the ‘impact factor’, treasured by many, but dismissed by others as misleading and open to manipulation. The Cochrane Database of Systematic Reviews (CDSR), published within The Cochrane Library, has just received its third impact factor − 5.653. So on average each Cochrane Review is cited over five times by researchers around the globe within two years of publication. The 2009 impact factor continues a positive growth trend for CDSR, whose first impact factor was 4.654 in 2007, and then 5.182 in 2008. Whilst this trend is encouraging, the Cochrane research ‘footprint’ arguably is underestimated by the impact factor measure, given the size of the impact factor ‘denominator’, and its single focus on citations, which do not directly reflect the Cochrane goals of informing and improving healthcare decision‐making.

Individuals from over 100 countries contribute to creating Cochrane Systematic Reviews. In 2009 authors from 85 different countries cited Cochrane Reviews in their published work. Comparing the reviews that receive the most citations with those most accessed demonstrates that there is little overlap between the two groups. This might reflect differences in the priorities between researchers and other Cochrane users such as health professionals, consumers and policy‐makers. In any case, it is a reminder that impact factor is only one measure among many of overall impact. Given the international nature of the Collaboration, the range and geographical spread of the readers of The Cochrane Library, the need to actively identify those reviews that are most relevant to end users, in addition to those likely to be most cited, is very evident. A more inclusive approach to measuring impact, beyond simply the impact factor is essential if impact is to be one measure of the Collaboration's success.

Top cited reviews in 2009		Top accessed reviews globally in 2009 (full text versions)
Nicotine replacement therapy for smoking cessation	64	Interventions for treating obesity in children	10,432
Antidepressants for smoking cessation	60	Interventions for preventing falls in older people living in the community	8,904
Nicotine receptor partial agonists for smoking cessation	52	Interventions for preventing obesity in children	8,096
Interventions for enhancing medication adherence	45	Interventions for preventing falls in elderly people	7,177
Organised inpatient (stroke unit) care for stroke	45	Exercise or exercise and diet for preventing type 2 diabetes	5,814
Cooling for newborns with hypoxic ischemic encephalopathy	42	School based physical activity programs for promoting physical activity and fitness in children and adolescents aged 6–18	5,192
Antidepressants for neuropathic pain	37	Support surfaces for pressure ulcer prevention	5,071
Allergen injection immunotherapy for seasonal allergic rhinitis	34	Interventions for enhancing medication adherence	5,044
Corticosteroids for acute bacterial meningitis	32	Nicotine replacement therapy for smoking cessation	4,549
Long‐acting insulin analogues versus NPH insulin (human isophane insulin) for type 2 diabetes mellitus	32

Feedback on this editorial and proposals for future editorials are welcome.