Systematic review and meta-analysis: a brief introduction

Abstract

Learning objectives:

1. Understand the basic structure and parts of a systematic review

2. Be able to read and critically appraise a published systematic review

What is a systematic review?

A systematic review is a manuscript that provides up to date information regarding a specific topic. A systematic review is different from a narrative review or an opinion piece in the following ways:

1. A systematic review should initially start with a specific and clearly outlined study question.

2. A systematic review begins only after a protocol is prepared and approved by the review group. Therefore, unlike a narrative review, aspects of the review process are defined even before searching the literature.

3. The protocol should contain the following key points:

   • Search strategy—The search strategy should be clearly outlined, and the exact search strategy provided. More than 1 search engine should be used with at least 3 recommended. Years searched, language, and type of articles should all be defined a priori.
   • Review of articles for inclusion—Articles reviewed should be evaluated by at least 2 independent reviewers in a blinded manner. Disagreements regarding article inclusion should be resolved as a group.
   • Reasons for article exclusion—It is important for a systematic review to clearly state which articles were reviewed in greater detail and why they were finally excluded.

4. Along with presenting the results of included studies, a systematic review should also provide an overview of the quality of each study included in the review. This information provides readers information regarding the strength of evidence for each study.

Why is this important?

Healthcare needs to adapt to the evolving medical evidence. However, it is very challenging for any individual to be able to keep abreast with the rapidly growing literature available on a specific topic. A systematic review, therefore, should provide the reader with an unbiased up to date summary of evidence. The review should be clear, specific, transparent, and reproducible. Such a review provides support to people making decisions regarding healthcare delivery and approaches. Unlike an opinion piece, a good systematic review should present the available evidence objectively, without bias, and the authors’ opinion should not influence the results or conclusions provided in the manuscript.

Systematic reviews continue to be a very important part of published literature and their numbers are increasing. A search in PubMed demonstrates that the number of systematic reviews published annually has rapidly increased in the past 30 years (Fig. 1A). In the area of cardiovascular sciences, this type of article has seen a rapid increase in the past 10 years, with approximately 1000 systematic reviews being published in 2020 (Fig. 1B).

Fig. 1.

A The annual number of published systematic reviews has increased rapidly over the past 30 years. In 2020 alone, approximately 20,000 systematic reviews were published. B In cardiology, the past 10 years has witnessed a rapid increase in systematic reviews. Last year, 1000 systematic review papers were published in PubMed in cardiovascular sciences

What is a meta-analysis?

A meta-analysis is the statistical extension of a systematic review. In a meta-analysis, statistical techniques are used to combine the results of included studies and present a single pooled estimate for the result. Therefore, along with providing the results for each included study, the meta-analysis possibly increases the strength of evidence by mathematically combining all these results. Depending upon the study question, a meta-analysis may or may not be conducted as part of the review process. Some study questions, for example, if two drug therapies or treatments are being compared and the end points are clinical events, can be pooled as a meta-analysis more easily. The meta-analysis of randomized controlled trials is often performed to increase the strength of evidence above that provided by any single study. Randomized trials are designed and powered to determine the treatment effect for a composite clinical end point. Therefore, a meta-analysis of such studies can provide stronger evidence regarding each component of that composite end point. Depending upon the statistical methods used, each study contributes a specific proportion to the final estimate.

However, a meta-analysis does not make a flawed systematic review more objective. Authors should carefully evaluate studies to ensure that they are similar enough that their results can be pooled together.

If there are differences in study quality, often, sub-group analyses are performed to provide results of only those studies that are deemed high quality. Sensitivity analyses are further performed to evaluate the robustness of presented results. Finally, authors should evaluate and report the heterogeneity obtained on pooling studies. Heterogeneity is a numerical estimate of the observed differences between pooled studies. Hence, if heterogeneity is high, then that implies that the results obtained in the pooled studies are dissimilar from each other. In this case, the authors should perform further analyses and present possible reasons why such heterogeneity is observed.

What can be pooled in a meta-analysis?

Depending upon the end points relevant to that specific study question, various values can be combined to present a single pooled estimate. For categorial end points (called dichotomous outcomes), for example, in-hospital mortality, stroke, or myocardial infarction, authors can combine the odds ratio, risk ratio, or risk difference reported in each individual study. As a measure of uncertainty regarding the estimates, it is important to present confidence intervals along with the pooled values. Continuous values, for example, cardiopulmonary bypass times or hospital length of stay, can also be pooled together. If such variables are combined, it is important that all have the same unit of measurement. Meta-analyses can also be conducted for long-term end points like all-cause mortality. In this case, the hazard ratio is often used as the estimate that is pooled together. Bayesian techniques can also be applied to pooling data from studies. These methods may allow for more useful interpretation of results by presenting results in a more practical manner.

What is a network meta-analysis?

A network meta-analysis is the natural extension of a conventional meta-analysis when more than 2 treatment options need to be compared. Figure 2 provides a simple graph of a network meta-analysis. We observe that 4 studies compare Control with Treatment A; also, 4 studies compare Control with Treatment B and only 1 study compares Treatment A and Treatment B. However, using a network meta-analysis, we can combine estimates from all these studies to obtain a comparison between Treatment A and Treatment B. Therefore, rather than only 1 study, we are using evidence from all 9 studies, thus increasing the strength of our evidence. While this is a very simple example, in published literature, networks often consist of many treatments and this method allows authors to compare treatments for which direct evidence is very limited.

Fig. 2.

A flowchart to depict the use of network meta-analyses. In published literature, data comparing Treatments A and B is directly available in only 1 study. However, in a network meta-analysis, using both direct and indirect evidence, evidence from 9 studies can be pooled together to compare Treatments A and B. Within the limitations of interpretation, this may increase the strength of the available evidence

In the next few installments of this series, we will go into more details regarding how to understand graphs used to report meta-analyses, and points to consider when reading a meta-analysis.

This was an initial overview regarding systematic reviews and meta-analyses. It provides the reader with a basic understanding of systematic reviews, the important points to consider when formulating a systematic review, and the introductory concepts of a meta-analysis.

Funding

None.

Declarations

IRB approval

Not required as this is an editorial.

Ethical committee approval

Not required being a review article. This paper complies with ethical standards of the journal.

Conflict of interest

The authors do not declare any conflict of interest.

Informed consent

This is an editorial and no patient information is present. Therefore, there is no need for informed consent.

Human and animal rights statement

Not applicable.