Evidence-based medicine for consumers: a role for the Cochrane Collaboration

Abstract

Quality of health information on the Internet has been a concern since health information first began appearing on the Web. Evidence-based medicine tools, traditionally intended for physicians, may benefit consumers as they participate in making health care desisions. This article describes a rationale for Cochrane reviews as an evidence-based medicine tool for consumers. The Cochrane Collaboration, a global force for systematic literature reviews, has strict procedures for developing literature reviews. Criteria for Cochrane reviews are compared with critical evaluation skills commonly taught to consumers regarding the use of Websites. The Cochrane Collaboration's Consumer Network has established a separate Website, with review synopses written for an audience of consumers. Suggestions for further research into consumer use of the Cochrane Library and consumer involvement with the Cochrane Collaboration are discussed.

People who access health information are likely to play an active role in their health care [1]. Access to health information empowers consumers by enabling them to (1) make more knowledgeable choices in self-care, (2) more intelligently discuss medical conditions and treatments with their doctors, (3) educate themselves regarding good health practices, and (4) learn about the health care system. If access to information is to help consumers achieve better health, quality of accessible information must be considered by information producers and consumers. This paper identifies a rationale for evidence-based medicine for consumers, introduces the quality-control work of the Cochrane Collaboration, and explores consumer involvement in the Cochrane Library, relative to critical evaluation models.

EVIDENCE-BASED MEDICINE: CRITICAL THINKING FOR CONSUMERS

Librarians and educators have been teaching critical evaluation skills regarding information available on the Web since 1995 [2]. Use of the Internet to search for health information and health care supplies is projected to increase rapidly over the next five years [3]. Health care information providers have been working to develop systems for assessing and maintaining quality information available via the Internet [4–8].

Although much has been published on the subject, criteria for Website evaluation have not been standardized [9]. Intermediary evaluation tools, such as Health On the Net's (HON) Site-checker, may assist consumers in applying general Website evaluation criteria [10]. Another tool, DISCERN, guides users through the evaluation of Websites that advocate medical treatments [11]. Logos and seals of approvals are poor tools for evaluating Websites, because consumers have no way of knowing the logo has been legitimately applied or that the Website has been maintained to the approved standard.

Preevaluated sites are among the options for quality-seeking consumers. MEDLINEplus and Medical Matrix are two examples of this type of offering [12, 13]. These sites offer convenient alternatives for people who have little time or desire to evaluate sites for themselves. Preevaluated sites may be helpful in providing introductory or background information.

In consultation with physicians, consumers may use evidence-based medicine as part of their medical decision-making process. Evidence-based medicine is the practice of systematically locating, appraising, and applying high-quality research in an effort to make sound clinical decisions [14]. Evidence-based medicine may appeal to consumers who are frustrated with a lack of standards among Websites or who wish to move beyond the evaluation of individual Websites and general education materials.

An increasing interest in evidence-based medicine for consumers has been emerging as part of a growing interest in decision support for patients [15] and an increased movement toward accountability for health services [16]. Explanations of epidemiological phrases, such as “relative risk reduction,” can help patients evaluate treatment options. Armed with an understanding of epidemiological phrases and tools for evidence-based medicine, patients can search for reading materials most relevant to their concerns. They must be taught, however, that their newfound knowledge is best placed in context under the guidance of a health professional, rather than self-applied.

While many consumers do not have a choice in their health care, the principles behind choice are worth pursuing for all health care consumers [17]. Some physicians and health educators argue against evidence-based medicine for consumers, citing low literacy levels or lack of interest on the part of consumers in risk analysis [18]. These arguments conflict with informed consent policies and the wishes of a growing number of proactive health consumers [19, 20].

The Cochrane Library [21] offers structured literature reviews to support evidence-based medicine. These reviews lack the arbitrary nature of many published reviews in that the terms of the literature search and criteria for inclusion and exclusion of studies have been identified. The collaboration has expanded its audience to consumers in the form of review synopses. These synopses may appeal to an increasing number of consumers who seek health information and are unable to digest lengthy documents written for physicians.

INTRODUCTION TO THE COCHRANE LIBRARY

The international Cochrane Collaboration, initiated by Archie Cochrane in the United Kingdom and formally announced in 1993, produces information for people who make health care decisions. Through its Database of Systematic Reviews, the collaboration provides opportunities for consumers to access, as well as organize, high-quality health information. The Cochrane Database of Systematic Reviews, publicly available since 1995, is a growing set of high-quality literature reviews. Currently numbering about 1,235, these literature reviews are highly structured summaries of carefully selected articles on health-related topics. Randomized controlled trials are a major emphasis of the database. The data from these trials are sometimes combined, using a statistical technique known as meta-analysis, to confirm and amplify the power of the findings. The full text of the reviews requires a subscription. Individual reviews may be purchased from Cinahl Information Systems [22]. Abstracts of the reviews can be accessed for free via the Internet [23].

The Cochrane Database of Systematic Reviews is part of the Cochrane Library. The library includes four other sections: Database of Abstracts of Reviews of Effectiveness, The Cochrane Controlled Trials Register, the Cochrane Review Methodology Database, and the National Health Service (NHS) Economic Evaluation Database. These sections help to support the material included in the Database of Systematic Reviews. The Cochrane Library is published four times per year. Material published in the library is cumulative, rather than sequential.

The Cochrane Collaboration Handbook serves to guide the development of the reviews [24]. Review Manager, also known as RevMan, is the collaboration's software, designed to assist in constructing the reviews [25].

The appendix shows the anatomy of a systematic review for the Cochrane Database, which is described here in brief. There are four parts to the cover sheet: Title, Reviewers, Sources of Support, and What's New. The names of the review group and the names and contact information for the individual reviewers are listed. The twenty-seven components to the text of the review include detailed descriptions of the studies, criteria for inclusion in the database, analyses of the quality of the studies, and considerations for applied use or research. The objectives of the reviews are stated, along with background information, materials and methods involved in the review, and conclusions regarding the import of the review and studies discussed. References and a list of tables and figures follow the main body of text. Citations to the studies reviewed are given, along with citations to studies that have been considered for inclusion but not used. Reasons for excluding potentially relevant studies are described. Tables summarize each trial, its methodology, and its results. Tables summarizing meta-analyses are included as warranted.

The collaboration aims for a comprehensive selection of high-quality studies to include in the database. Extensive searches are conducted to find relevant, high-quality studies. Search methods range from querying databases (e.g., MEDLINE) to combing gray literature (e.g., conference proceedings), to reading hand-selected journals, to looking for unpublished clinical trials. Nine key principles are collaborating, building the enthusiasm of individuals, avoiding duplication, minimizing bias, keeping up to date, striving for relevance, promoting access, and ensuring quality and continuity drive the collaboration [26]. The first two principles, collaborating and building the enthusiasm of individuals, promote a team of people with different backgrounds and skill levels. Hence, a role exists for consumers, to be described in the third section of this paper.

Approximately fifty review groups produce and update systematic reviews for the database [27]. Reviewers receive no financial compensation or gifts directly from the collaboration. Reviewers must disclose other sponsoring agencies. The collaboration may help facilitate access to funding from outside sources [28]. Reviewers agree to update their group area at least once per year and to find other reviewers if they are unable to do so.

Cochrane Centres support review groups, as well as others involved in the collaboration. Other collaboration participants include field members, methods groups, an organizationwide steering group, and Cochrane Consumer Network members. Field members serve in an advisory capacity to relevant review groups. Field members help select trials for the Cochrane Controlled Trials Register and may serve as reviewers. Methods groups provide policy advice, support, and training regarding quality issues in the collaboration. Methods groups also develop and maintain the Cochrane Review Methodology Database, help edit the Cochrane Handbook, and provide consultation on software development. The steering group facilitates long-range planning.

CONSUMER USE OF THE COCHRANE LIBRARY RELATIVE TO EVALUATION SKILLS

How many consumers access health information from the Cochrane Library is unknown. Low consumer use of fee-based portions of the library is likely, however. Low utilization of the library may be due to the wide availability of free health information over the Internet, little media attention given to the library, and lack of critical evaluation skills. Another reason few consumers may access the Cochrane Library is they may have difficulty understanding its content. The National Institute for Literacy reports that 22% of Americans are functionally illiterate [29], meaning they may have difficulty completing as simple a task as finding an intersection on a street map.

Until May 2000, a reading ability similar to that of clinicians was necessary to understand the information in the Cochrane Library. The Cochrane Database of Systematic Reviews now includes review synopses, written at a level specifically for consumers. Reviewers, review groups, or members of the Consumer Network may author the synopses. Methodologies used for authoring the synopses are under evaluation.

Critical evaluation skills most commonly taught to consumers have some similarities to quality considerations in Cochrane review groups. Potential for bias, currency of information, scope of information, veracity, and Website design are commonly posed as important considerations in Website evaluation [30]. Potential for bias, methodology to increase the likelihood of accurate conclusions, currency, and scope are factors considered in study selection for Cochrane reviews.

Much distinguishes Cochrane reviews, however, from health Websites that pass most or all chosen evaluation criteria. Cochrane reviews are highly structured, with carefully selected studies comprising the backbone of each review. Cochrane reviews are readily traceable to randomized controlled trials, most of which have been published in primary literature. Topical Websites designed for the public rarely include citations to more than one original study, let alone a randomized controlled trial. Unless they carry out a separate literature search, users are likely to rely on author reputation and comparison to other reputable sites as gauges of truthfulness.

CONSUMER INVOLVEMENT WITH THE COCHRANE COLLABORATION

One consequence of the Internet is that consumers are now playing a role in the production of health information as well as its consumption. This role may be in the form of gathering and maintaining links [31]. Consumers may also have roles as parts of elaborate review processes. In 1995, the Cochrane Collaboration registered a Consumer Network as part of the collaboration [32].

The Cochrane Consumer Network serves to structure consumer activity in the collaboration. Each review group strives to include members who voice consumer concerns regarding topics to be reviewed, appropriate questions to answer in the review, and ways to portray results. There are three levels at which consumers may take part in the development of reviews: as part of the review team helping to develop a synopsis, as referees, or as evaluators through the comments and criticisms section. A set of instructions, similar to those in the handbook, guides consumers through the refereeing process [33].

Consumers have recently begun an effort to supplement each review with a synopsis, written for a nonexpert audience. Some very active participants contribute to a Consumer Network Newsletter [34]. Perspectives from consumer advocates from around the world are offered in this moving publication. The Cochrane Consumer Network has recently become active in facilitating access to reproductive health information in developing countries. The Reproductive Health Library is the fruit of a joint partnership between the Cochrane Collaboration and the World Health Organization. The Cochrane Consumer Network sends out free copies of material in the library to network members and seeks input from developing countries [35]. The network now has its own Website, currently undergoing restructuring, sponsored by Medibank of Australia [36].

CONCLUSIONS

Consumers bring a unique, personal perspective to the efforts of the Cochrane Collaboration. They have a vested interest in making high-quality medical information available to the public. Through grass roots efforts, such as the Cochrane Consumer Network, global access to health information may become a reality. Studies of the network's recent expansion may help to gather baseline data appropriate for goal setting and evaluation.

A study on consumer access to the Cochrane Database of Systematic Reviews could help to compare cost and literacy as factors in consumer access to the Cochrane Library. Some groups could be given access to the library on a trial basis and queried regarding its usefulness and comprehensibility. Other groups could be given access specifically to the consumer-oriented synopses. Groups with specific health concerns would be good candidates for inclusion in this type of study.

More thought needs to be given to the work of consumers as referees. The idea of consumers assisting with the development of synopses may be a better use of consumer efforts than refereeing the reviews due to differing levels of familiarity with medical information. Assistance from health literacy educators could facilitate the development of quality consumer-level synopses. Recent efforts to involve consumers in the development of evidence-based medical information show that the Cochrane Consumer Network is well on its way to becoming an integral part of the Cochrane Collaboration.

APPENDIX

Anatomy of a systematic review for the Cochrane Database†

2.1 Outline of a Cochrane Review

• Cover sheet:

• Title

• Reviewers

• Sources of support

• What's New

• Text of review:

• Synopsis

• Abstract

• Background

• Objectives

• Search strategy

• Selection criteria

• Data collection and analysis

• Main results

• Reviewers' conclusions

• Background

• Objectives

• Criteria for selecting studies for this review

• Types of studies

• Types of participants

• Types of interventions

• Types of outcome measures

• Search strategy for identification of studies

• Methods of the review

• Description of studies

• Methodological quality

• Results

• Discussion

• Reviewers' conclusions

• Implications for practice

• Implications for research

• Acknowledgements

• Conflicts of interest

• References:

• References to studies

• Included studies

• Excluded studies

• Studies awaiting assessment

• Ongoing studies

• Other references

• Additional references

• Other published versions of this review

• Tables and figures:

• Characteristics of included studies

• Characteristics of excluded studies

• Characteristics of ongoing studies

• Comparisons, data and graphs

• Additional tables