Systematic Reviews and Meta-Analyses in Pediatric Practice

• You are seeing an 11 month old infant with bronchiolitis. A medical student rotating through your practice asks you if the empirical evidence supports prescribing albuterol. In between patients, you search the medical literature, and find hundreds of articles written on this topic. You would like one paper that summarizes the existing evidence regarding the effectiveness of beta2-agonists for treating bronchiolitis.

• You are seeing an otherwise healthy 3 year old boy who has had 4 days of infectious diarrhea. His mother would like to give him lactobacillus and asks your opinion. You do not have time to find and read multiple papers on this topic, but would like to provide a sound, evidence-based answer to the child's mother.

Pediatric medical research continues to grow exponentially, and because of the vast quantity of articles, keeping abreast of current literature may be overwhelming. Additionally, individual studies often are constrained by small sample sizes which limits the ability to detect significant relationships, and by recruitment from a single site which limits generalizeability. Rigorous literature reviews are therefore needed to synthesize available information and to guide practice.

Systematic reviews and meta-analyses identify, in an organized and objective manner, existing studies about a particular clinic question, and then answer the clinical question by synthesizing the relevant data in each included study. This synthesis of multiple studies provides clinicians with a robust means by which to make informed clinical decisions. Systematic reviews summarize existing studies in a descriptive way, while meta-analyses use statistics to combine the results from each included study and generate a single summary statistic. Using one systematic review/meta-analysis about the efficacy of beta2-agonists (Flores 1997) and one meta-analysis about lactobacillus therapy (Van Niel 2002), this brief will discuss the uses and limitations of systematic reviews and meta-analyses.

What is a systematic review and how do I know that it has been conducted rigorously?

A systematic review uses a well-defined approach to locate and summarize articles related to a clinical question. Rigorously conducted systematic reviews should include the following: 1) focus on a specific clinical question such as “Are beta2-agonists effective in decreasing respiratory distress for infants with bronchiolitis?”; 2) a thorough and objective search of the scientific literature guided by a formal and prescribed protocol; this search protocol should be well-documented and reproducible, and should include searches of multiple databases (such as MedLine, PsycInfo and CINAHL) and use of additional sources of information (such as manual search of bibliographies and consultation with content experts). Use of an explicit search strategy minimizes biases in finding and including relevant studies. The goal is to find all potentially relevant articles; 3) a clear explanation of which studies were included and which were excluded. For example, the article about beta2-agonists included only randomized control trials (Flores 1997); 4) a descriptive results section summarizing the findings from the included studies, and addressing the study quality of included studies.

What is a meta-analysis and how do I know that it has been conducted rigorously?

A meta-analysis pools the statistical information from a systematically collected group of articles and reports an overall summary statistic that incorporates all of the data in all of the included articles. For example, in the lactobacillus meta-analysis, the authors combined the statistical data from nine studies, and found that on average lactobacillus decreased the duration of infectious diarrhea by 0.7 days (Van Niel 2002).

A meta-analysis should meet all of the requirements listed for a systematic review. Additionally, a meta-analysis should: 1) present a summary statistic using data from all of the included studies. For this pooled estimate, studies with more subjects should be given greater weight; 2) test the homogeneity versus heterogeneity of the included studies (sample, intervention, outcome) to determine whether combining the statistics from the studies is possible. For example, it may be inappropriate to combine results of one study of beta2-agonists whose reported outcome was maternal report of infant's symptoms with another study whose reported outcome was reduction in days of hospital admission; and 3) assess for publication bias. Often, positive studies are more likely to be published than negative studies. There are statistical tests and diagrams that help researchers determine whether smaller studies with negative findings are potentially missing from the review.

What are the advantages of systematic reviews and meta-analyses over individual studies?

Because both systematic reviews and meta-analyses pool results from multiple studies, their findings offer a compilation of evidence that potentially has greater power to inform clinical decisions than would an individual study.

If a meta-analysis provides statistical results, why does anyone ever do just a systematic review?

Researchers may conduct a systematic review rather than a meta-analysis because 1) the existing research on the given clinical question is too limited to do a meta-analysis; 2) the individual studies related to the study question are so different that averaging their results is not meaningful. For example, in the beta-agonist article, the authors conclude that the outcomes of the inpatient studies were too diverse to combine the statistics. (Flores 1997)

What are the limitations of systematic reviews and meta-analyses?

The most significant limitation of both systematic reviews and meta-analyses is commonly described as “garbage in, garbage out.” In other words, if the quality of the studies included in the systematic review or meta-analysis is poor, then the summary conclusions are similarly inadequate.

I have heard of the Cochrane Library. What is it and how do I access it?

The Cochrane Library is an international, non-profit organization of independent researchers who compile systematic reviews on a wide range of medical topics. These medical topics include a range of pediatric health issues such as “drugs for preventing migraine headaches in children” and “feed thickener for infants with gastroesophageal reflux.” The Cochrane Library can be accessed at the following website: http://www3.interscience.wiley.com/cgi-bin/mrwhome/106568753/HOME; depending on your affiliation, a fee may be charged to access the reviews.