Systematic reviews and meta-analyses can help busy family physicians keep up to date with the medical literature by objectively summarizing large bodies of evidence in a standardized and concise manner. Systematic reviews involve the application of explicit criteria to identify, critically appraise, and synthesize all high-quality research evidence across multiple studies that address specific, clearly articulated, clinical questions.
A meta-analysis is a special type of systematic review that additionally uses statistical methods to quantitatively combine and summarize the results of several individual studies. The proliferation of systematic reviews over the past 20 years has been accompanied by the development of numerous guides and tools to improve the methodologic quality of these reviews.1–3
When properly conducted, systematic reviews and meta-analyses are considered the highest level of evidence and are increasingly used to inform medical decision making, clinical practice guidelines, and health care policy. The evidence pyramid in Figure 1 shows the relative authority of various types of medical research. It is important to note that not all clinical questions can be answered through randomized controlled trials (RCTs), either because of practical or ethical issues, and thus evidence from studies using other designs should also be considered. This is particularly true as far as family medicine is concerned, as many routine practices and procedures used by family physicians have never been subjected to evaluations using the RCT methodology.
Figure 1.
Evidence pyramid: Hierarchy of various types of medical research.
The challenges faced by many medical specialists to upgrade and retain their knowledge pale in comparison to those faced by most family physicians. MEDLINE, while one of the largest online databases of biomedical abstracts, is not the only database containing evidence of relevance to family physicians. MEDLINE alone indexes approximately 5200 journals that are jointly responsible for adding more than 13 000 articles each week. Systematic reviews and meta-analyses can provide an important strategy that facilitates transition from information overload to knowledge synthesis.
Questions and answers
What makes systematic reviews systematic?
Unlike traditional narrative reviews, in which a content expert writes about a particular topic, systematic reviews use explicit and reproducible criteria designed to reduce bias. This process includes a comprehensive search strategy (which frequently involves electronic and manual searches) of all potentially relevant articles, including unpublished and “gray” literature, and the application of predetermined inclusion and exclusion criteria in the selection of articles for review. The included studies are critically appraised in terms of methodologic rigour: data are abstracted and synthesized, and results are summarized and interpreted using a standardized approach. Frequently, all these steps and decisions are made independently by more than one reviewer to further reduce bias. The usefulness of the end product depends largely on whether a clear and relevant clinical question was asked and the extent to which methodologic rigour has been used to minimize error and bias.
How are they helpful?
It is highly unusual for a single study, no matter how well designed it is, to provide a definitive answer to a clinical question. Because of the cumulative nature of science, even widely accepted evidence might be challenged or refuted over time. Systematic reviews can assist family physicians in examining and better understanding inconsistencies and evolution of evidence over time. When properly conducted, systematic reviews provide an unbiased and cumulative state of the scientific knowledge around a specific therapy, treatment, or clinical approach. By pooling the results of several studies, meta-analyses can provide more statistically precise summary treatment effects and can help to explain heterogeneity between the results of individual studies.
Are systematic reviews “magic bullets”?
While systematic reviews are regarded as providing the highest level of medical evidence, there are also some controversies surrounding how they are conducted and their ultimate worth. One criticism is the disproportional reliance on RCTs that tend to produce a robust but very narrow evidence base. There are also concerns that not all systematic reviews are equally reliable4 and that many reviews require frequent updates or they rapidly become obsolete. 5 In addition, many systematic reviews conclude that there is not enough evidence to support or discard the use of a particular intervention or therapy.6 Some of the strategies to address these concerns include meta-narrative reviews,7 extension of search strategies beyond traditional databases, development of more efficient search strategies, and development of explicit policies or tools to determine how frequently reviews should be updated.
How to find systematic reviews?
Many health care journals, including Canadian Family Physician, publish systematic reviews, but the best-known source is the Cochrane Collaboration, in which a group of more than 15 000 methodologic and content experts systematically review randomized trials addressing a range of health topics. As of January 2009, there were almost 4000 completed reviews and almost 2000 pending protocols for additional reviews. Box 1 shows some additional sources of systematic reviews and meta-analyses.
Box 1. Database sources for systematic reviews and meta-analyses.
The following databases are good sources for systematic reviews and meta-analyses:
|
CINAHL—Cumulative Index to Nursing and Allied Health Literature, CRD—Centre for Reviews and Dissemination, DARE—Database of Abstracts of Reviews of Effects, EMBASE—Excerpta Medica, HTA—Health Technology Assessment, NHS EED—National Health Service Economic Evaluation Database, TRIP—Turning Research into Practice.
Bottom line
Systematic reviews can assist the decision-making process by objectively summarizing large numbers of studies, recognizing current knowledge gaps, and identifying beneficial or harmful therapies and interventions.
Hypothesis is a quarterly series in Canadian Family Physician, coordinated by the Section of Researchers of the College of Family Physicians of Canada. The goal is to explore clinically relevant research concepts for all CFP readers. Submissions are invited from researchers and nonresearchers. Ideas or submissions can be submitted on-line at http://mc.manuscriptcentral.com/cfp or through the CFP website www.cfp.ca under “Authors.”
Footnotes
Competing interests
None declared
References
- 1.Oxman AD, Cook DJ, Guyatt GH. Users’ guides to the medical literature. VI. How to use an overview. JAMA. 1994;272(17):1367–71. doi: 10.1001/jama.272.17.1367. [DOI] [PubMed] [Google Scholar]
- 2.Greenhalgh T. Papers that summarise other papers (systematic reviews and meta-analyses) BMJ. 1997;315(7109):672–5. doi: 10.1136/bmj.315.7109.672. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126(5):376–80. doi: 10.7326/0003-4819-126-5-199703010-00006. [DOI] [PubMed] [Google Scholar]
- 4.Savoie I, Helmer D, Green CJ, Kazanjian A. Beyond Medline: reducing bias through extended systematic review search. Int J Technol Assess Health Care. 2003;19(1):168–78. doi: 10.1017/s0266462303000163. [DOI] [PubMed] [Google Scholar]
- 5.Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D.How quickly do systematic reviews go out of date? A survival analysis Ann Intern Med 20071474224–33. Epub 2007 Jul 16. [DOI] [PubMed] [Google Scholar]
- 6.Petticrew M. Why certain systematic reviews reach uncertain conclusions. BMJ. 2003;326(7392):756–8. doi: 10.1136/bmj.326.7392.756. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R.Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review Soc Sci Med 2005612417–30. Epub 2005 Jan 26. [DOI] [PubMed] [Google Scholar]