Abstract
Objectives and background
Wei et al. have published a meta-analysis (MA), which aimed to evaluate the association between SARS-CoV-2 infection during pregnancy and adverse pregnancy outcomes. Using classical random-effects model, they found that SARS-CoV-2 infection was associated with preeclampsia, preterm birth and stillbirth. Performing MA with low event rates or with few studies may be challenging insofar as MA relies on several within and between-study distributional assumptions. The objective was to assess the robustness of the results provided by Wei et al.
Methods
We performed a sensitivity analysis using frequentist and Bayesian meta-analysis methods. We also estimated fragility indexes.
Results
For eclampsia, the confidence intervals of most frequentist models contain 1. All beta-binomial models (Bayesian) lead to credible intervals containing 1. The prediction interval, based on DL method, ranges from 0.75 to 2.38. The fragility index is 2 for the DL method. For preterm, the confidence (credible) intervals exclude 1. The prediction interval is broad, ranging from 0.84 to 20.61. The fragility index ranges from 27 to 10. For stillbirth, the confidence intervals of most frequentist models contain 1. Six Bayesian MA models lead to credible intervals containing 1. The prediction interval ranges from 0.52 to 8.49. The fragility index is 3.
Conclusion
Given the available data and the results of our broad sensitivity analysis, we can suggest that SARS-CoV-2 infection during pregnancy is associated with preterm, and that it may be associated with preeclampsia. For stillbirth, more data are needed as none of the Bayesian analyses are conclusive.
Keywords: SARS-CoV-2, Eclampsia, Preterm, Stillbirth, Meta-analysis
1. Background
Wei et al. [1] recently published a systematic review and meta-analysis (MA) which aimed to “evaluate the association between SARS-Cov-2 infection during pregnancy and adverse pregnancy outcomes”.
The authors show that SARS-CoV-2 infection is associated with preeclampsia, preterm birth and stillbirth. While they stated that they used Mantel-Haenszel method, the results of Figure 2 are related to a random-effects inverse-variance model, with DerSimonian-Laird estimate of tau2 and continuity correction.
Higgin's I2 was used to assess heterogeneity. Although this approach is widely mentioned, the point estimate I2 should be interpreted cautiously when a MA has few studies [2], and the confidence interval should be given.
MA relies on several within and between-study distributional assumptions that are sometimes hidden [3].
Several methods are available to assign weights in meta-analyses, e.g. Mantel-Haenszel for fixed-effect MA or inverse-variance for fixed-effect or random-effects MA. For random-effects MA, there are several ways to estimate between-study variance, e.g. DerSimonian and Laird (DL), restricted maximum likelihood (REML). Finally, there are different ways to estimate the confidence interval for the summary effect (e.g. Wald or Hartung-Knapp-Sidik-Jonkman (HKSJ) method). The same applies to the confidence interval for between-study variance (e.g. Q-Profile method). See [4], [5] for more information.
Performing MA with low event rates or with few studies, or both, may be challenging. For example, in this case estimating between-study heterogeneity is difficult, and inaccurate estimation of this heterogeneity may lead to overly narrow confidence intervals.
Statistical methods can never completely resolve the issue of sparse data. Different methods may yield different results, and using a suboptimal approach may lead to erroneous conclusions [6].
2. Methods
In order to assess the robustness of the results reported by Wei et al. [1], we performed a sensitivity analysis using a range of frequentist and Bayesian meta-analysis methods. We also estimated fragility indexes (and Fragility Quotient, which is fragility index divided by the total study sample size). As an intuitive measure of the robustness of a trial, it is a number indicating how many patients would be required to convert a trial from being statistically significant to non-significant [7]. This index has been adapted for MA [8] and is currently being refined methodologically [9]. We estimated FI with the DL method and restricted maximum likelihood with Knapp-Hartung (KNHA) test.
Wei et al. [1] reported three outcomes: preeclampsia, preterm and stillbirth. We focused on preeclampsia and stillbirth, for which data are sparse.
For frequentist MA, several methods were used [10], [11], including that of Hemni and Copas, which was found to be less sensitive to publication bias than the DL method [12], [13] (see tables). Although not optimal, continuity correction (0.5) was applied to studies with zero cells, the objective being to compare our results with the work of Wei et al. [1].
In accordance with IntHout et al. [14], we have also presented prediction interval, but only for the model used by Wei et al. [1]. It is the interval within the effect size of which a new study would be categorized, if this study were selected at random from the same population as the studies already included in the MA.
For Bayesian MA, we used a binomial-normal hierarchical model (BNHM), i.e., modelling probabilities of success in each group [15], instead of modelling estimates of log odds-ratios directly (normal-normal model) for the between-trial heterogeneity. We also used a beta-binomial (BB) model, which has shown good statistical properties for meta-analysis of sparse data [16], [17]. With these approaches, no continuity correction (of any type) is required, insofar as, unlike the commonly used normal-normal hierarchical model, these models are based on exact distributional assumptions. Several priors were used as sensitivity analysis [18], [19].
Statistical analyses were performed with Stata (frequentist framework) and R software (Bayesian framework).
3. Results
All estimates are shown in Table 1, Table 2, Table 3, Table 4 .
Table 1.
Sensitivity analysis using different methods to estimate the association between Covid-19 and preeclampsia.
| Model/method | Odds ratio (95% CI) | I2 (95% CI) | Tau2 (95% CI) |
|---|---|---|---|
| Patients with Covid-19 vs. without Covid-19 | |||
| Frequentist | |||
| DL | 1.33 (1.03 to 1.73) | 30.9% (0.0 to 64.3) | 0.05 |
| DL-pred | 1.33 (0.75 to 2.38) | ||
| Perm | 1.33 (0.95 to 1.66) | 30.9% | 0.05 |
| Boot | 1.34 (1.02 to 1.78) | 31.0% (0.0 to 64.5) | 0.06 |
| REML | 1.36 (1.00 to 1.85) | 30.9% (0.0 to 64.3) | 0.09 (0.00 to 0.93) |
| MP | 1.35 (1.01 to 1.80) | 30.9% (0.0 to 64.3) | 0.07 (0.00 to 0.87) |
| SJ | 1.44 (0.94 to 2.20) | 30.9% (0.0 to 64.3) | 0.28 (0.00 to 0.87) |
| RO | 1.03 (0.98 to 1.81) | 30.9% | 0.05 |
| HKSJ | 1.33 (0.98 to 1.81) | 30.9% | 0.05 |
| KR (oim) | 1.36 (0.75 to 2.45) | 44.5% (0.0 to 85.5) | 0.09 (0.00 to 0.67) |
| KR (eim) | 1.36 (0.81 to 2.29) | 44.5% (0.0 to 85.5) | 0.09 (0.00 to 0.67) |
| PL-B | 1.33 (0.84 to 2.54) | 0.0% (0.0 to 75.9) | 0.0 (0.00 to 0.36) |
| PL-S | 1.33 (0.86 to 2.52) | 0.0% (0.0 to 75.9) | 0.0 (0.00 to 0.36) |
| HC | 1.33 (0.45 to 3.90) | 30.9% (0.0 to 64.3) | 0.05 |
| IVHet | 1.33 (0.98 to 1.99) | 30.9% | 0.05 |
| Sens (60) | 1.40 (0.97 to 2.02) | 60% 2(6.47 to 78.24) | 0.17 |
| Sens (75) | 1.46 (0.92 to 2.30) | 75% (75 to 85.5) | 0.34 |
| Bayesian | |||
| BNHM (WIP) | 1.40 (1.02 to 2.25) | ||
| BB (half normal) | 1.62 (0.88 to 3.00) | ||
| BB (uniform) | 1.60 (0.89 to 2.94) | ||
| BB (half Cauchy) | 1.60 (0.88 to 2.97) | ||
| BNHM (Vag) | 1.40 (1.02 to 2.18) | ||
| BN (t) | 1.47 (1.00 to 2.50) | ||
| Patients with severe vs mild Covid-19 | |||
| Frequentist | |||
| DL | 4.16 (1.55 to 11.15) | ||
| DL-pred | 4.16 (0.84 to 20.61) | ||
| Perm | Not computed: number of studies is below 6 | ||
| Boot | 4.16 (1.55 to 11.16) | 0.08% (0.0 to 79.2) | 0.001 |
| REML | 4.16 (1.55 to 11.15) | 0% (0.0 to 77.2) | 0 (0 to 4.65) |
| MP | 4.16 (1.55 to 11.15) | 0% (0.0 to 74)* | 0 (0 to 3.90) |
| SJ | 4.16 (1.55 to 11.15) | 0% | 0 |
| RO | 4.16 (1.11 to 15.59) | 0% | 0 |
| HKSJ | 4.16 (1.03 to 16.80) | 0% | 0 |
| KR (oim) | 4.16 (0.69 to 25.0) | 0% (0.0 to 77.2) | 0 (0 to 4.65) |
| KR (eim) | 4.16 (0.08 to 226.82) | 0% (0.0 to 77.2) | 0 (0 to 4.65) |
| PL-B | 4.16 | 0% (0.0 to 66.9) | 0 (0 to 2.78) |
| PL-S | 4.16 (1.23 to 27.24) | 0% (0.0 to 66.9) | 0 (0 to 2.78) |
| HC | 4.16 (1.66 to 10.40) | 0% | 0 |
| IVHet | 4.16 (1.55 to 11.15) | ||
| Sens(60) | 5.30 (1.01 to 27.80) | 60% (0.00 to 85.04) | 2.06 |
| Sens(75) | 5.60 (0.692 to 45.38) | 75% (38.36 to 89.86) | 4.12 |
| Bayesian | |||
| BNHM (WIP) | 4.48 (1.51 to 13.46) | ||
| BB (half normal) | 3.38 (0.92 to 13.06) | ||
| BB (uniform) | 3.29 (0.91 to 12.55) | ||
| BB (half Cauchy) | 3.38 (0.91 to 13.46) | ||
| BNHM (Vag) | 4.71 (1.55 to 14.44) | ||
| BN(t) | 8.10 (0.65 to 29.04) |
CI: confidence interval (frequentist) or credible (Bayesian); I2: Higgin's I2; Tau2: between-study variance; Boot: Bootstrapped DL; DL: DerSimoniam-Laird); DL-pred: prediction interval with DL; HKSJ: Hartung-Knapp-Sidik-Jonkman; HC: Henmi and Copas; IVHet: Inverse-variance heterogeneity; KR: Kenward-Roger; MP: Mandel-Paule; Perm: permutation; PL: Partial Likelihood (-B: with Bartlett's correction; -S: with Skovgaard's correction); REML: Restricted Maximum Likelihood; RO: DL with SJ robust variance estimator; SJ: Sidik-Jonkman; Sens: sensitivity analysis (percentage of I2); BB: Beta-Binomial; BNHM: Binomial-Normal Hierarchical Model; Vag: Vague prior; BN(t): Binomial Normal (t distribution); WIP: Weakly Informative Prior.
Table 2.
Sensitivity analysis using different methods to estimate the association between Covid-19 and stillbirth.
| Model/method | Odds ratio (95% CI) | I2 (95% CI) | Tau2 (95% CI) |
|---|---|---|---|
| Patients with Covid-19 vs. without Covid-19 | |||
| Frequentist | |||
| DL | 2.11 (1.14 to 3.90) | 24.02% (0.00 to 67.77) | 0.15 |
| DL-pred | 2.11 (0.52 to 8.49) | ||
| Perm | 2.10 (0.98 to 4.24) | 24.02% (0.00 to 67.77) | 0.15 |
| Boot | 2.16 (1.08 to 4.32) | 28.94% (0.00 to 70.86) | 0.24 |
| REML | 2.07 (1.16 to 3.67) | 19.5% (0.00 to 87.2) | 0.12 (0.00 to 3.30) |
| MP | 2.12 (1.12 to 4.0) | 26% (0.00 to 93.5) | 0.17 (0.05 to 6.94) |
| SJ | 2.11 (1.12 to 3.96) | 25.2% | 0.16 |
| RO | 2.10 (0.99 to 4.46) | 24.0% | 0.15 |
| HKSJ | 2.10 (0.93 to 4.77) | 24.0% | 0.15 |
| KR (oim) | 2.07 (0.00 to 9.3e + 27) | 19.5% (0.00 to 87.2) | 0.12 (0.00 to 3.30) |
| KR (eim) | 2.07 (0.00 to?) | 19.5% (0.00 to 87.2) | 0.12 (0.00 to 3.30) |
| PL-B | 1.80(0.57 to 8.18) | 0.0% (0.0 to 71.3) | 0.00 (0.00 to 1.20) |
| PL-S | 1.80(0.61 to 6.74) | 0.0% (0.0 to 71.3) | 0.00 (0.00 to 1.20) |
| HC | 1.80 (0.52 to 6.29) | 24.0% | 0.15 |
| IVHet | 1.80 (0.87 to 3.75) | 24.0% | 0.15 |
| Sens (60) | 2.21 (0.85 to 5.76) | 60.0% (1.82 to 83.70) | 0.73 |
| Sens (75) | 2.18 (0.65 to 7.24) | 75.0% (43.39 to 88.96) | 1.45 |
| Bayesian | |||
| BNHM (WIP) | 1.84 (0.92 to 3.63) | ||
| BB (half normal) | 1.39 (0.46 to 3.74) | ||
| BB (uniform) | 1.40 (0.47 to 3.86) | ||
| BB (half Cauchy) | 1.39 (0.46 to 3.82) | ||
| BNHM (Vag) | 1.86 (0.98 to 3.60) | ||
| BN (Vag) | 1.96 (0.38 to 5.24) |
CI: confidence interval (frequentist) or interval (Bayesian); I2: Higgin's I2; Tau2: between-study variance; Boot: Bootstrapped DL; DL: DerSimoniam-Laird); DL-pred: prediction interval with DL; HKSJ: Hartung-Knapp-Sidik-Jonkman; HC: Henmi and Copas; IVHet: Inverse-variance heterogeneity; KR: Kenward-Roger; MP: Mandel-Paule; Perm: permutation; PL: Partial Likelihood (-B: with Bartlett's correction; -S: with Skovgaard's correction); REML: Restricted Maximum Likelihood; RO: DL with SJ robust variance estimator; SJ: Sidik-Jonkman; Sens: sensitivity analysis (percentage of I2); BB: Beta-Binomial; BNHM: Binomial-Normal Hierarchical Model; VAG: Vague prior; BN(t): Binomial Normal (t distribution); WIP: Weakly Informative Prior.
Table 3.
Sensitivity analysis using different methods to estimate the association between Covid-19 and preterm birth.
| Model/method | Odds ratio (95% CI) | I2 (95% CI) | Tau2 (95% CI) |
|---|---|---|---|
| Patients with Covid-19 vs without Covid-19 | |||
| Frequentist | |||
| DL | 1.82 (1.38 to 2.39) | 63.47% (39.42 to 77.97) | 0.16 |
| DL-pred | 1.82 (0.75 to 4.43) | ||
| Perm | 1.82 (1.41 to 2.55) | 63.47% (39.42 to 77.97) | |
| Boot | 1.82 (1.38 to 2.42) | 62.78% (38.15 to 77.61) | 0.17 |
| REML | 1.82 (1.39 to 2.38) | 62.2% (25.2 to 85.6) | 0.15 (0.03 to 0.59) |
| MP | 1.81 (1.39 to 2.36) | 60.4% (20.2 to 86.7) | 0.14 (0.023 to 0.59) |
| SJ | 1.81 (1.40 to 2.33) | 57.1% | 0.12 |
| RO | 1.82 (1.36 to 2.42) | 63.5% | 0.16 |
| HKSJ | 1.82 (1.34 to 2.47) | 63.5% | 0.16 |
| KR (oim) | 1.82 (1.38 to 2.39) | 62.2% (25.2 to 85.6) | 0.15 (0.03 to 0.53) |
| KR (eim) | 1.82 (1.38 to 2.39) | 62.2% (25.2 to 85.6) | 0.15 (0.03 to 0.53) |
| PL-B | 1.81 (1.36 to 2.48) | 58.9% (23.0 to 83.9) | 0.13 (0.025 to 0.47) |
| PL-S | 1.81 (1.37 to 2.45) | 58.9% (23.0 to 83.9) | 0.13 (0.025 to 0.47) |
| HC | 1.38 (0.70 to 2.71) | 63.5% | 0.17 |
| IVHet | 1.38 (0.74 to 2.58) | 63.5% | 0.16 |
| Sens (60) | 1.82 (1.39 to 2.35) | 60.0% (32.93 to 76.14) | 0.13 |
| Sens (75) | 1.85 (1.34 to 2.55) | 75.0% (60.43 to 85.20) | 0.27 |
| Bayesian | |||
| BNHM (WIP) | 1.80 (1.36 to 2.46) | ||
| BB (half normal) | 1.70 (1.17 to 2.51) | ||
| BB (uniform) | 1.70 (1.14 to 2.48) | ||
| BB (half Cauchy) | 1.71 (1.18 to 2.51) | ||
| BNHM (Vag) | 1.80 (1.36 to 2.46) | ||
| BN (t) | 1.28 (1.34 to 2.49) | ||
| Patients with symptomatic vs asymptomatic Covid-19 | |||
| Frequentist | |||
| DL | 2.29 (1.49 to 3.53) | 57.2% (10.16 to 79.63) | 0.21 |
| DL-pred | 2.29 (0.69 to 7.20) | ||
| Perm | 2.29 (1.53 to 3.51) | 57.2% (10.16 to 79.63) | 0.21 |
| Boot | 2.28 (1.50 to 3.46) | 51.49% (0.00 to 77.27) | 0.19 |
| REML | 2.25 (1.51 to 3.34) | 49.6% (10.7 to 84.5) | 0.15 (0.019 to 0.85) |
| MP | 2.19 (1.53 to 3.14) | 40.5% (2.5 to 89.0) | 0.11 (0.004 to 1.26) |
| SJ | 2.27 (1.50 to 3.43) | 53.6% | 0.18 |
| RO | 2.29 (1.43 to 3.62) | 57.2% | 0.21 |
| HKSJ | 2.29 (1.49 to 3.55) | 57.2% | 0.21 |
| KR (oim) | 2.25 (1.41 to 3.58) | 49.6% (10.7 to 84.5) | 0.15 (0.019 to 0.85) |
| KR (eim) | 2.25 (1.29 to 3.93) | 49.6% (10.7 to 84.5) | 0.15 (0.019 to 0.85) |
| PL-B | 2.22 (1.43 to 3.82) | 44.8% (7.6 to 80.9) | 0.13 (0.013 to 0.66) |
| PL-S | 2.22 (1.52 to 3.73) | 44.8% (7.6 to 80.9) | 0.13 (0.013 to 0.66) |
| HC | 1.82 (0.92to 3.6) | 57.2% | 0.21 |
| IVHet | 1.82 (1.08 to 3.07) | 57.2% | 0.21 |
| Sens(60) | 2.31 (1.48 to 3.61) | 60.0% (16.78 to 80.77) | 0.23 |
| Sens(75) | 2.41 (1.37 to 4.23) | 60.0% (51.71 to 87.06) | 0.47 |
| Bayesian | |||
| BNHM (WIP) | 2.41 (1.58 to 3.93) | ||
| BB (half normal) | 2.34 (1.39 to 4.01) | ||
| BB (uniform) | 2.36 (1.40 to 4.10) | ||
| BB (half Cauchy) | 2.36 (1.43 to 3.97) | ||
| BNHM (Vag) | 2.41 (1.57 to 3.97) | ||
| BN (t) | 2.53 (1.45 to 4.33) | ||
| Patients with severe vs mild Covid-19 | |||
| Frequentist | |||
| DL | 4.29 (2.41 to 7.63) | 61.4% (23.10 to 80.63) | 0.47 |
| DL-pred | 4.29 (0.77 to 23.89) | ||
| Perm | 4.29 (2.08 to 8.47) | 61.4% (23.10 to 80.63) | 0.47 |
| Boot | 4.30 (2.40 to 7.70) | 61.17% (22.54 to 80.53) | 0.49 |
| REML | 4.29 (2.42 to 7.62) | 61.3% (14.3 to 88.5) | 0.47 (0.05 to 2.26) |
| MP | 4.30 (2.40 to 7.70) | 62.4% (16.0 to 90.9) | 0.49 (0.06 to 2.92) |
| SJ | 4.30 (2.41 to 7.67) | 62.0% | 0.48 |
| RO | 4.29 (2.21 to 8.32) | ||
| HKSJ | 4.29 (2.19 to 8.40) | 61.4% | 0.47 |
| KR (oim) | 4.29 (2.12 to 8.69) | 61.3% (14.3 to 88.5) | 0.46 (0.05 to 2.26) |
| KR (eim) | 4.29 (2.12 to 8.68) | 61.3% (14.3 to 88.5) | 0.46 (0.05 to 2.26) |
| PL-B | 4.26 (2.23 to 8.67) | 56.2% (8.60 to 85.9) | 0.38 (0.03 to 1.79) |
| PL-S | 4.26 (2.23 to?) | 56.2% (8.60 to 85.9) | 0.38 (0.03 to 1.79) |
| HC | 4.14 (1.98 to 8.67) | 61.4% | 0.47 |
| IVHet | 4.14 (2.24to 7.64) | 61.4% | 0.47 |
| Sens(60) | 4.28 (2.47 to 7.53) | 60.0% (19.87 to 80.3) | 0.44 |
| Sens(75) | 4.40 (2.16 to 8.96) | 75.0% (53.39 to 86.59) | 0.88 |
| Bayesian | |||
| BNHM (WIP) | 4.53 (2.61 to 8.17) | ||
| BB (half normal) | 3.46 (1.89 to 6.30) | ||
| BB (uniform) | 3.49 (1.91 to 6.23) | ||
| BB (half Cauchy) | 3.49 (1.91 to 6.23) | ||
| BNHM (Vag) | 4.57 (2.59 to 8.25) | ||
| BN (Vag) | 4.61 (1.97 to 9.08) |
CI: confidence interval (frequentist) or interval (Bayesian); I2: Higgin's I2; Tau2: between-study variance; Boot: Bootstrapped DL; DL: DerSimoniam-Laird); DL-pred: prediction interval with DL; HKSJ: Hartung-Knapp-Sidik-Jonkman; HC: Henmi and Copas; IVHet: Inverse-variance heterogeneity; KR: Kenward-Roger; MP: Mandel-Paule; Perm: permutation; PL: Partial Likelihood (-B: with Bartlett's correction; -S: with Skovgaard's correction); REML: Restricted Maximum Likelihood; RO: DL with SJ robust variance estimator; SJ: Sidik-Jonkman; Sens: sensitivity analysis (percentage of I2); BB: Beta-Binomial; BNHM: Binomial-Normal Hierarchical Model; VAG: Vague prior; BN(t): Binomial Normal (t distribution); WIP: Weakly Informative Prior.
Table 4.
fragility index and fragility quotient.
| Diseases | Statistical method | Fragility index | Fragility quotient (%) |
|---|---|---|---|
| Eclampsia | |||
| Covid-19 vs without Covid-19 | DL | 2 | 0 |
| REML-KNHA | 2 | 0 | |
| Severe vs mild Covid | DL | 2 | 0.4 |
| REML-KNHA | 1 | 0.2 | |
| Stillbirth | DL | 3 | 0.0 |
| REML-KNHA | 1 | 0.0 | |
| Preterm | |||
| Covid-19 vs without Covid-19 | DL | 27 | 0.0 |
| REML-KNHA | 18 | 0.0 | |
| Symptomatic vs asymptomatic Covid-19 | DL | 17 | 0.4 |
| REML-KNHA | 10 | 0.2 | |
| Severe vs mild Covid-19 | DL | 20 | 1.4 |
| REML-KNHA | 13 | 0.9 |
DL: Dersimonian-Laird; REML: Restricted Maximum Likelihood (KNHA: Knapp- Hartung).
In some cases, no estimation was possible, one example being permutations, when fewer than six studies were included in the meta-analysis. Under these conditions, Bayesian estimates are more appropriate.
For eclampsia in general (comparing patients with Covid-19 vs. without Covid-19), the confidence intervals of most frequentist MA models contain 1 (Table 1).
The prediction interval, based on the DL method, ranges from 0.75 to 2.38.
The fragility index is 2 for the DL method used by the authors, as is is with the REML-HKSJ method.
As for the Bayesian approach, all three beta-binomial models of MA lead to credible intervals containing 1. For binomial normal models, credible intervals do not contain one, but the upper boundary of the interval is very close to 1.
For eclampsia, when comparing patients with severe vs mild Covid-19, frequentist sensitivity analysis confirms the authors’ results, except for one model out of 12 (Kenward-Roger's model).
The prediction interval, based on the DL method, ranges from 0.84 to 20.61.
The fragility index is 2 for the DL method and 1 with the REML-HKSJ method.
When heterogeneity of 75% is taken into account, the difference between the two groups is no longer significant.
Concerning the Bayesian analysis, only the binomial-normal models confirmed the initial results reported by Wei et al. [1]. None of the three beta binomial models resulted in credible intervals excluding 1.
For preterm (global analysis comparing patients with Covid-19 vs. without Covid-19, or subgroup analysis comparing patients with symptomatic vs. asymptomatic Covid-19), all frequentist models used in our sensitivity analysis led to confidence or credible intervals excluding 1, except for the inverse variance heterogeneity model and the Hemni-Copas method (Table 3).
However, the prediction interval based on the DL method is very broad, ranging from 0.84 to 20.61.
The fragility index ranges from 27 (global analysis with the DL method) to 10 (subgroup analysis comparing symptomatic vs asymptomatic patients, with the REML-HKNHA method). Details on these fragility indexes and quotients are given in Table 4.
All Bayesian credible intervals also exclude 1.
Finally for stillbirth, the confidence intervals of most frequentist meta-analysis (MA) models contain 1 (Table 2).
The prediction interval based on the DL method ranges from 0.52 to 8.49.
The fragility index is 3 for the DL method and 3 with the REML-HKNHA method.
As for the Bayesian approach, all six MA models lead to credible intervals containing one.
4. Discussion
We performed a large-scale sensitivity study taking into account different approaches (frequentist and Bayesian) and different statistical models, the objective being to estimate several parameters including odds ratio, I2, Tau and their respective confidence intervals.
With the exception of preterm infants, and regardless of the frequentist or Bayesian statistical approach applied, the confidence intervals of odds ratios frequently overlap 1 and consequently show no association between Covid-19 and eclampsia or stillbirth. The prediction intervals are very wide and for all three endpoints (eclampsia, stillbirth and preterm) contain 1. Finally, the fragility indexes for eclampsia and stillbirth are only 1, 2 or 3.
Concerning our Bayesian sensitivity analysis, our results are robust as regards variation of the priors.
Some authors have proposed the use of generalized linear mixed model (GLMM) to perform MA with sparse data [20]. We have not used GLMM. Here again, several approaches are available. We considered that our sensitivity analysis was sufficiently broad. Finally, we did not take into account the quality of the studies via risk-of-bias analysis, which is possible through meta-regression.
5. Conclusion
Given the available data and the results of our broad sensitivity analysis, we can suggest that SARS-CoV-2 infection during pregnancy is associated with preterm, and may be associated with preeclampsia. For stillbirth, more data are needed as none of the Bayesian analyses are conclusive.
Disclosure of interest
The authors declare that they have no competing interest.
Authors’ contributions
Ludwig Serge Aho Glele (LSAG) and Emmanuel Simon (EM) contributed to the conception and design of the study.
LSAG performed the statistical analysis.
LSAG, EM, Philippe Kadhel (PK) and Paul Sagot (PS) drafted the manuscript.
Camille Bouit (CM), Maeva Serrand (MS), Laurence Filipuzzi (LS) and Karine Astruc (KA) reviewed the manuscript.
LSAG, EM, CM, MS, LF, KA, PK and PS contributed to the interpretation of the data, gave final approval of the version to be published and agreed to be accountable for all aspects of the work.
Ethical approval
This study is a meta-analysis, for which ethical approval is not required.
References
- 1.Wei S.Q., Bilodeau-Bertrand M., Liu S., Auger N. The impact of COVID-19 on pregnancy outcomes: a systematic review and meta-analysis. CMAJ. 2021 doi: 10.1503/cmaj.202604. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.von Hippel P.T. The heterogeneity statistic I2 can be biased in small meta-analyses. BMC Med Res Methodol. 2015:15. doi: 10.1186/s12874-015-0024-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Jackson D., White I.R. When should meta-analysis avoid making hidden normality assumptions? Biometrical Journal Biometrische Zeitschrift. 2018;60:1040. doi: 10.1002/bimj.201800071. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Veroniki A.A., Jackson D., Viechtbauer W., Bender R., Bowden J., Knapp G., et al. Methods to estimate the between-study variance and its uncertainty in meta-analysis. Res Synth Methods. 2016;7:55–79. doi: 10.1002/jrsm.1164. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Deeks J.J., Higgins J.P.T., Altman D.G. In: 2nd ed. Higgins J.P.T., Thomas J., Chandler J., Cumpston M., Li T., Page M.J., et al., editors. Wiley-Blackwell; Hoboken, NJ: 2019. Analysing data and undertaking meta-analyses; pp. 241–284. (Cochrane Handbook for Systematic Reviews of Interventions). [chapter 10] [Google Scholar]
- 6.Efthimiou O. Practical guide to the meta-analysis of rare events. Evid-Based Ment Health. 2018;21:72–76. doi: 10.1136/eb-2018-102911. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Walsh M., Srinathan S.K., McAuley D.F., Mrkobrada M., Levine O., Ribic C., et al. The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index. J Clin Epidemiol. 2014;67:622–628. doi: 10.1016/j.jclinepi.2013.10.019. [DOI] [PubMed] [Google Scholar]
- 8.Atal I., Porcher R., Boutron I., Ravaud P. The statistical significance of meta-analyses is frequently fragile: definition of a fragility index for meta-analyses. J Clin Epidemiol. 2019;111:32–40. doi: 10.1016/j.jclinepi.2019.03.012. [DOI] [PubMed] [Google Scholar]
- 9.Lin L. Factors that impact fragility index and their visualizations. J Eval Clin Pract. 2021;27:356–364. doi: 10.1111/jep.13428. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Seide S.E., Röver C., Friede T. Likelihood-based random-effects meta-analysis with few studies: empirical and simulation studies. BMC Med Res Methodol. 2019;19:16. doi: 10.1186/s12874-018-0618-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Veroniki A.A., Jackson D., Bender R., Kuss O., Langan D., Higgins J.P.T., et al. Methods to calculate uncertainty in the estimated overall effect size from a random-effects meta-analysis. Res Synth Methods. 2019;10:23–43. doi: 10.1002/jrsm.1319. [DOI] [PubMed] [Google Scholar]
- 12.Henmi M., Hattori S., Friede T. A confidence interval robust to publication bias for random-effects meta-analysis of few studies. ArXiv. 2020 doi: 10.1002/jrsm.1482. [200207598 Stat] [DOI] [PubMed] [Google Scholar]
- 13.Henmi M., Copas J.B. Confidence intervals for random effects meta-analysis and robustness to publication bias. Stat Med. 2010;29:2969–2983. doi: 10.1002/sim.4029. [DOI] [PubMed] [Google Scholar]
- 14.IntHout J., Ioannidis J.P.A., Rovers M.M., Goeman J.J. Plea for routinely presenting prediction intervals in meta-analysis. BMJ Open. 2016;6 doi: 10.1136/bmjopen-2015-010247. [e010247] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Hamza T.H., van Houwelingen H.C., Stijnen T. The binomial distribution of meta-analysis was preferred to model within-study variability. J Clin Epidemiol. 2008;61:41–51. doi: 10.1016/j.jclinepi.2007.03.016. [DOI] [PubMed] [Google Scholar]
- 16.Mathes T., Kuss O. A comparison of methods for meta-analysis of a small number of studies with binary outcomes. Res Synth Methods. 2018;9:366–381. doi: 10.1002/jrsm.1296. [DOI] [PubMed] [Google Scholar]
- 17.Mathes T., Kuss O. Beta-binomial models for meta-analysis with binary outcomes: Variations, extensions, and additional insights from econometrics. Res Methods Med Health Sci. 2021;2:82–89. [Google Scholar]
- 18.Günhan B.K., Röver C., Friede T. Random-effects meta-analysis of few studies involving rare events. Res Synth Methods. 2020;11:74–90. doi: 10.1002/jrsm.1370. [DOI] [PubMed] [Google Scholar]
- 19.Röver C., Bender R., Dias S., Schmid C.H., Schmidli H., Sturtz S., et al. On weakly informative prior distributions for the heterogeneity parameter in Bayesian random-effects meta-analysis. Res Syn Meth 2021. 2021 doi: 10.1002/jrsm.1475. [DOI] [PubMed] [Google Scholar]
- 20.Ju K., Lin L., Chu H., Cheng L.-L., Xu C. Laplace approximation, penalized quasi-likelihood, and adaptive Gauss-Hermite quadrature for generalized linear mixed models: towards meta-analysis of binary outcome with sparse data. BMC Med Res Methodol. 2020;20:152. doi: 10.1186/s12874-020-01035-6. [DOI] [PMC free article] [PubMed] [Google Scholar]