Young GI angle: How to learn & conduct meta‐analysis: Tips & tricks for the emerging researcher

INTRODUCTION

Meta‐analysis is a statistical method that involves the pooling of data from independent studies on the same subject. ¹ It is widely accepted as a “high‐quality” form of research in the scientific world, as evidenced by the fact that a simple Google Scholar search of the term “meta‐analysis” yields over 3.6 million results. Meta‐analyses are often conducted in conjunction with what's called a “systematic review.” While in a systematic review, a “qualitative” summary of published literature is presented; in a meta‐analysis, a quantitative outcome is reported.

In this article, we share our personal experiences in learning and conducting a meta‐analysis, lessons we learned in the process and how to be mindful, as emerging researchers, while conducting a scientifically sound and clinically applicable meta‐analysis.

TIPS AND TRICKS

Choosing a clinically relevant research topic

Meta‐analysis cannot be necessarily considered as “novel” research. However, the most important part of any meta‐analysis is to phrase the right question. For example, a recently published meta‐analysis in UEG Journal investigated whether adjuncts to bowel preparation for colonoscopy led to improved patient experience and resulted in superior bowel cleanliness. ² The PICO format (Patient problem, Intervention, Comparison, Outcomes) is commonly used. This provides clinical context, focus, and often serves as a title for the manuscript. A good clinical question is one that arises at the patient's bedside. Authors may also choose a new or emerging topic, for example, COVID‐19 and assess its implications on patient outcomes. ³

tip: Network with GI faculty and fellows on Twitter. Brainstorm to arrive at a great idea! Use #GITwitter.

Preliminary literature search

While choosing a clinically relevant topic is important, one must also consider whether the same question has been answered by a previously published meta‐analysis. Our approach is to search the key terms of the study on Google‐Scholar and PubMed. Using the term “meta‐analysis” during a literature search would allow researchers to study other meta‐analyses published on the same topic. It is useful at this step to save the studies' individual and search links, as these can be used to help guide your medical librarian in refining their extensive literature search, if the study idea seems feasible.

tip: Check “related articles” on Google‐Scholar, and “similar articles” on PubMed.

Study outcomes

Based on the study question, the patient population of interest and the intervention being investigated must be clearly identified. A clear, unambiguous patient sample and intervention will help in pooling information across studies and formulate a clear outcome data (frequently called “event” data in statistics). Primary study outcomes guide the meta‐analysis outcomes. For example, if the primary study reports proportions, then the meta‐analysis would report pooled proportions. It would be an error to calculate pooled ratios if the primary study outcome is in proportions. Outcomes stratified by the patient population and interventions will additionally help with reducing heterogeneity and help in formulating subgroups within the patient population.

tip: Pooled study outcomes are usually formulated from the primary study outcomes.

The team

Co‐authors

Preferred Reporting Items for Systematic Reviews and Meta‐Analysis (PRISMA) and Meta‐Analysis of Observational Studies (MOOSE) are evidence‐based items for reporting in systematic reviews and meta‐analyses. ⁴ , ⁵ Its main aim is to help standardize and protocolize meta‐analysis study. Per guidelines, at least two authors are needed to independently review the literature and select studies. This is ideally done with the help of an experienced medical librarian. At least two authors must independently gather data, independently perform risk of bias assessment and a third author settles any conflicts that may arise. A statistician is obviously needed to guide with statistical analysis, but more importantly to limit errors and misinterpretations of the results, as we will discuss next. A senior mentor to help with finalizing your manuscript is indispensable. Additionally, collaborating with a key opinion leader in the area of your research can add invaluable clinical perspective to your data. Beginners should review the PRISMA statement and browse published meta‐analyses to see the commonality of format, style and use of subheadings, examples of which are available in UEG Journal. ⁶ , ⁷

tip: Present both PRISMA and MOOSE checklists in your study.

Statistician

Ideally, a statistician should be part of any research study. Meta‐analysis statistics is easy to learn; many self‐directed guides are available in the form of books, YouTube video lectures and workshops (see Helpful Resources section below). Additionally, online resources such as Cochrane interactive learning are available for conducting an intervention‐based review. A statistician can help ensure the validity of methods and formulate a statistical analysis plan. Depending on the type of studies included (cohort, case‐control or randomized controlled trials) the statistical methodology will vary. Reporting pooled odds ratio instead of risk ratio or relative risk are commonly encountered errors. High heterogeneity being equated to a poor meta‐analysis study and not reporting 95% prediction intervals are other errors in meta‐analysis that hopefully can be avoided with the help of a statistician. ⁸

tip: Set aside time with a qualified biostatistician and learn from them!

Medical librarian & literature search

Once the idea is mature and ready to roll, a formal, full, and exhaustive literature search should be carried out by an experienced medical librarian. A one‐on‐one discussion with the librarian to ensure the right search terms and appropriate keywords is helpful. PRISMA guidelines mandate that at least three major databases must be searched to ensure a wholesome search of the literature. A study reference manager such as Refworks, Mendeley, Zotero or Endnote can be used to transfer the literature search results to the study team.

tip: Involve a medical librarian to facilitate your literature search.

Research protocol

A research protocol must be written, discussed with the team and edited before final approval. Study features like the research question, study outcomes, inclusion and exclusion criteria, patient population, databases to be searched, etc., must be confirmed before registering the study on an established registry for meta‐analysis like PROSPERO, and before sending the search instructions to the medical librarian. Additionally, it is helpful to check if a similar study has already been registered or undertaken on PROSPERO. It is always a good idea to have the research protocol checked by the statistician in your team.

tip: You can prepare a protocol from scratch just by following the steps in PROSPERO.

Study selection

A selection process based on screening the title of the articles is typically done first, followed by the abstracts. A final detailed reading of the shortlisted articles is done to select the studies of interest. To prevent bias and ensure thoroughness, at least two authors must do this process independently. Differences in the study selection will always exist. The medical librarian typically would alert the differences in the study selection and arrange for a meeting among the authors to reconcile any discrepancies.

tip: Familiarize with a reference manager, for example, Endnote or Refworks.

Data collection

This is the “heart and soul” step of your study. Meticulous and accurate data collection is paramount, and this must be done independently by two study authors. Typically, any data collection table or working sheet would work. Multiple data collection software exists which also helps with basic data computations, Microsoft Excel, Google‐Drive Sheets are some examples. Other less known ones that can easily generate data‐visualization plots are Tableau, Microsoft Power‐BI. Always make sure to mention the data sheet software you used in your manuscript's “Methods” section.

tip: Use standard column headings (study, design, age, gender, etc.) in the excel sheet, these then are submitted as tables along with the manuscript.

The rows of the datasheet in a meta‐analysis are almost always the details of the studies included in the meta‐analysis (study first author, year, reference). A trivial, but important information to pay attention to is to ensure that the author name and year of publication match the information on the reference list. The columns on the sheet depend on the data to be collected. Study and patient characteristics are other column variables that are typically generated. Attention must be exercised to collect the variables that make the most clinical sense to the study question in hand. These values can be very important for meta‐regression analysis if appropriate for the study question.

tip: Ensure that study name, year of publication are identical to the study information in the reference list.

Finally, the data important for the “study outcomes” forms the rest of the columns of the sheet. Information on the study follow‐up time is important to collect and report. It is helpful to highlight these columns on the sheet for easy assimilation of the information when discussing with other team members of the study. Moreover, these data points on the sheet can be finally used as the “tables” in your manuscript. Furthermore, as the study moves along, multiple edits of the data sheet might be warranted. During this process, it's good practice to save the collected data by duplicating the existing data sheet before editing and deleting unwanted or unnecessary columns/or rows. During this process, clarity on the study outcomes starts to become more apparent.

tip: Do not ever delete collected data. Make a copy of the data sheet and continue editing on the new sheet.

Statistical model

All meta‐analysis research that includes database search, must be done by the “random‐effects” model. Some authors use the technique of using a “fixed‐effect” if no heterogeneity and “random” in the presence of heterogeneity. Although this does not cause a variation in the results being reported, it is conceptually wrong to retrospectively select a study model based on heterogeneity results. Therefore, our approach is to always use a “random‐effects” model in meta‐analyses performed on study data derived from database literature search.

tip: Always use a random‐effects model.

Statistical analysis

Multiple easy‐to‐use meta‐analysis statistical packages exist for data analysis and can generate Forest plots. Having a statistician in your team can help in this crucial yet easy step of your meta‐analysis research. Especially if you plan to present 95% prediction interval (different from 95% confidence interval) data to explain heterogeneity and more so if executing a meta‐regression analysis. ⁹ I ²% values are arbitrarily sub grouped to define the level of heterogeneity. Subgroups analyses, meta‐regression analysis can be very helpful supportive calculations that can demonstrate key findings to make your paper attractive and clinically useful. Not infrequently, results of supportive (or secondary) analysis can help guide future research questions.

tip: Learn one statistical software well and know it in detail. Report heterogeneity data and study weights on the forest plot image.

Risk of bias assessment

Bias! Every study is biased when conceived. Although a meta‐analysis is an analysis of already published studies, it is still important to assess bias on the selected studies and report them. Multiple validated tools are available to test and report the risk of bias analysis. Based on the study type and outcomes, the tool can differ. For case‐control or cohort studies, the Newcastle Ottawa scale is applicable and easy to use. ¹⁰ Its various components of assessment are self‐explanatory. Scores are assigned based on points or stars to calculate and understand the extent of risk of bias. For meta‐analyses that include all randomized controlled trials (RCTs), the Jadad scale or the Cochrane Collaboration tool for risk of bias assessment can be used. ¹¹ , ¹² Quality assessment of diagnostic accuracy studies (or QUADAS) is used when pooled diagnostic accuracy parameters are being studied. ¹³

tip: incorporate risk of bias assessment at the data collection phase to extract data efficiently from each included study.

Interpretation

Always present the results of your primary outcomes first. Although the majority of meta‐analysis studies focus on the overall pooled results, the focus of result interpretation must be on the heterogeneity and validation of the results. This helps the reader to understand the clinical application of your results. It is understandable that when analysing results of studies published in various countries, with varying population demographics, the pooled results might not be accurately applicable to the patient visiting you in your endoscopy suite. Thus, subgroup analysis, assessing and discussing the heterogeneity of your results in a dedicated “validation of results” section helps the reader.

Additionally, contrary to common belief, a high heterogeneity does not mean the meta‐analysis is bad. The key is to attempt additional analyses to understand reasons for heterogeneity and present the findings.

tip: High heterogeneity is not always bad. Validation and explanation of heterogeneity is the key in a meta‐analysis.

CONCLUSION

The aim of this article was to give an overall superficial idea to the emerging researcher who is keen to explore the field of meta‐analysis. These are summarized above and in Figure 1. We hope the tips and resources mentioned in this article helps and motivates more people to make the initial move and work towards successful execution of their meta‐analysis.

FIGURE 1.

Beginner's guide for learning and performing meta‐analysis. Ax, Assessment; CMA, comprehensive meta‐analysis software; MAs, meta‐analyses; PICOs, Population, Intervention, Comparator and Outcome; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta‐Analysis

HELPFUL RESOURCES

Comprehensive Meta‐Analysis (CMA) workshops and lecture videos on Youtube.

Common mistakes in Meta‐Analysis book by Michael Borenstein.

Heterogeneity in systematic reviews and meta‐analysis: how to read between the numbers. Gastrointest Endosc. 2019;89(4):902–3.

Irfan‐View photo editing software, for free and easy to use image editing and formatting.

Cochrane interactive training: https://training.cochrane.org/interactivelearning.

Cochrane RevMan (free meta‐analysis app): https://training.cochrane.org/online‐learning/core‐software/revman.

CONFLICT OF INTEREST

Authors report no conflicts of interest.

Mohan BP, Chandan S, Siau K. Young GI angle: how to learn & conduct meta‐analysis: Tips & tricks for the emerging researcher. United European Gastroenterol J. 2022;10(9):1031–5. 10.1002/ueg2.12322

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are openly available in UEG Journal at https://doi.org/10.1002/ueg2.12322.