Problems with Early Systematic Reviews: The Case of Coronavirus Disease 2019 (COVID-19) in Pregnancy

Abstract

Introduction

Rapid dissemination of findings regarding the Coronavirus Disease 2019 (COVID-19) and its potential effects on pregnancy is crucial to support understanding and development of recommendations for optimization of obstetrics care. However, much of the current studies published are in the form of case reports or case series which can be prone to biases. Other factors also further complicate attempts to analyze data accurately. Hence, this evaluation hopes to highlight some of these problems and provide suggestions to help clinicians mitigate and make reasonable conclusions when reading the abundant yet limited body of evidence when furthering their research efforts.

Methods

Studies regarding COVID-19 and pregnancy were searched on databases such as PubMed, EMBASE, Scopus, the Cochrane Library. Manual search of references of select articles were also undertaken. Apart from summarizing study limitations identified by authors, the characteristics of current literature and systematic reviews were also evaluated to identify potential factors affecting accuracy of subsequent analysis.

Results

Factors such as innate biasness in study design of current literature, duplicate reporting, differing inclusion criteria of systematic reviews, scarce data, inadequate follow-up period and limitations of systematic reviews have been shown to hinder the ability for accurate data extrapolation.

Discussion

Unless additional studies are conducted in identified areas of data scarcity and a common list of factors affecting accuracy of data analysis are taken into account when developing recommendations, discrepancies will continue to arise and accurate data analysis and valid systematic reviews will be precluded.

Significance Statement

Information regarding COVID-19 and pregnancy are rapidly disseminated. However, the additional quantity of evidence does not translate to a similar improvement in the quality of evidence for systematic reviews as current evidence may be at risk of bias. Discrepancies arise when data is scarce and when different factors affecting biasness are omitted in data interpretation. To our knowledge, no current studies have specifically addressed factors affecting the accuracy and value of data analysis. This paper identifies some of these factors in order to provide shared knowledge of areas where caution should be exercised when interpreting such findings.

Introduction

Despite ongoing research efforts over the last eight months, much remains unknown about the Coronavirus Disease 2019 (COVID-19). Learning about COVID-19 infection and its potential effects on pregnancy and perinatal outcomes is crucial to guide obstetrical management during this disease outbreak. As such, many obstetric units have been expeditiously publishing data from their local studies in an attempt to rapidly disseminate information on COVID-19 in pregnancy. To date, most of these publications have been in the form of case reports and case series. While these reports provide valuable information which have helped to guide early management in this rapidly evolving global pandemic, they are not ranked highly in the traditional hierarchy of evidence (Royal College of Obstetricians and Gynaecologists 2020). Discrepancies in recommendations especially in areas with scarce data thus arise. This commentary aims to highlight some of the potential problems and limitations one may encounter when reviewing the existing literature.

Methods

A search was done on PubMed, EMBASE, Scopus and the Cochrane Library electronic databases. References of select articles for studies describing COVID-19 and pregnancy were also screened. This manuscript is not based upon clinical study or patient data. Since December 2019 when the infection was first identified, more than 41,000 MEDLINE-indexed papers have been published on COVID-19. Up to 31st July 2020, at least 29 systematic reviews on COVID-19 and pregnancy with varying conclusions have been published (Abdollahpour and Khadivzadeh 2020; Akhtar et al. 2020; Ashraf et al. 2020; Capobianco et al. 2020; Della Gatta et al. 2020; Deniz and Tezer 2020; Di Mascio et al. 2020; Dubey et al. 2020; Elshafeey et al. 2020; Hessami et al. 2020; Juan et al. 2020; Kasraeian et al. 2020; Kotlyar et al. 2020; Lopes de Sousa et al. 2020; Matar et al. 2020; Melo and Araújo 2020; Muhidin et al. 2020; Segars et al. 2020; Silva et al. 2020; Smith et al. 2020; Thomas et al. 2020a, b; Trippella et al. 2020; Trocado et al. 2020; Turan et al. 2020; Walker et al. 2020; Yang et al. 2020; Yoon et al. 2020; Zaigham and Andersson 2020). Factors which are likely to cause bias and thus, inaccuracy in subsequent analysis in systematic reviews were then identified and compiled.

Results

All the systematic reviews included in our analysis acknowledge the limitation that their review is limited to largely case series and case reports. Apart from the innate bias of case series and case reports, the current literature is complicated by inconsistent reporting and small sample size which further impede efforts to apply the findings to the larger population (Ashraf et al. 2020; Capobianco et al. 2020; Della Gatta et al. 2020; Di Mascio et al. 2020; Dubey et al. 2020; Hessami et al. 2020; Kotlyar et al. 2020; Lopes de Sousa et al. 2020; Matar et al. 2020; Sayre et al. 2017; Smith et al. 2020; Thomas et al. 2020a, b; Trippella et al. 2020; Trocado et al. 2020; Turan et al. 2020; Walker et al. 2020; Yang et al. 2020; Yoon et al. 2020; Zaigham et al. 2020). Duplicate reporting overemphasizes findings. They can occur when overlaps in patient series are not clearly stated, when the same case is reported by both the admitting and receiving hospitals or when different aspects of the same case are reported, making it difficult for screening by direct comparison of the clinical findings (Di Mascio et al. 2020; Elshafeey et al. 2020; Juan et al. 2020). Additionally, variations in management and healthcare resources exists between countries (Turan et al. 2020). Genetic and environmental factors can also influence the natural history of disease (Capobianco et al. 2020) and thus, these factors limit generalizability of the findings of systematic reviews (Dubey et al. 2020). Exclusion of studies not published in the English language may further cause bias (Matar et al. 2020; Muhidin et al. 2020; Thomas et al. 2020a, b).

Discrepancies arise when differing recommendations are made based on researchers’ personal experiences, synthesis of limited COVID-19 studies or studies done during previous disease outbreaks. Factors identified include the study design of existing literature, duplicate reporting, inadequate follow-up period and limitations of systematic reviews.

In most studies, the short follow-up period and inclusion of predominantly patients in their third trimester of pregnancy, might lead to an overestimation of risks such as preterm birth whilst underestimating longitudinal risks such as fetal growth restriction (Di Mascio et al. 2020; Thomas et al. 2020a, b; Yang et al. 2020). Increased risk of preterm birth may be confounded by other factors such as the socioeconomic characteristics of patients, decisions to expedite delivery to optimize maternal condition and psychological stress during pregnancy. (Melo et al. 2020) In reports where indication for delivery was not clearly specified, extrapolation of data about the rate of spontaneous versus iatrogenic preterm birth was near-impossible. (Kasraeian et al. 2020) Hence, the lack of comparable controls negates attributing these complications and outcomes solely to COVID-19 infection. (Trocado et al. 2020) Moreover, most studies included only patients with laboratory confirmed positive quantitative reverse transcriptase polymerase chain reaction (qRT-PCR) assay. However, the viral nucleic acid test has a false-negative rate of up to 30% (Calda et al. 2020). Asymptomatic carriers of COVID-19 were only picked up incidentally during universal screening (Breslin et al. 2020). Thus, the actual prevalence of COVID-19 amongst pregnant women, and the associated COVID-19 pregnancy outcomes and complications might be underreported (Kotlyar et al. 2020; Matar et al. 2020).

In the face of an evolving disease, systematic reviews are only accurate and updated up to the point of submission for publication (Elshafeey et al. 2020; Segars et al. 2020). Additionally, the study protocol of systematic reviews typically excludes papers published in the format of commentaries, opinions, expert reviews, and letters to editors. Therefore, important insights from these studies might not be captured in systematic reviews. Evaluation of the quality of included studies must not be missed, even if performed during a pandemic, to ensure readers are not misled about the findings of the systematic review (Capobianco et al. 2020).

Discussion

Rapid dissemination of information regarding COVID-19 is necessary for clinicians to keep abreast of the latest developments, thereby guiding and updating clinical management as deemed appropriate. However, reviewers should bear in mind the limitations and interpret findings with caution. At present, the conclusions drawn from the current body of evidence may be at risk of bias.

To mitigate biasness, better quality evidence from large laboratories and standardized registries are important. In the absence of such databases, strict study protocol should be implemented, with special attention paid to studies originating from countries with high COVID-19 related research output. Recognition of the admitting and receiving hospitals might also help to identify the possibility of duplicate reporting. Establishment of carefully governed national and international registries or an international standard for publication of data can help to address some of these issues.

Nonetheless, we are immensely grateful for the tremendous ongoing research efforts during this challenging time of a global pandemic. As we continue the fight against COVID-19, we look forward to more rigorous research studies and higher level of evidence which will help us more reliably better our understanding in managing COVID-19 in pregnancy.

Author Contributions

XLA: Preparation, manuscript original writing and editing. SPC, MSQC and SS: supervision, manuscript review and edit. JCSL: Conceptualization, supervision, manuscript review and edit.

Funding

No funding or sponsorships were received for this study or publication of this article.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest. All named authors meet the International Committee of Medical Journal Editors (ICMJE) criteria for authorship for this article, take responsibility for the integrity of the work as a whole, and have given their approval for this version to be published.

Ethical Approval

No ethics approval applicable for this commentary.