To the editor:
We would like to appreciate your opinions and comments by Prof. Gou-Fu Li and Guo Yu with respect to our systematic review of acid suppressant use and the incidence and severe outcomes of COVID-19. We have replied on a point-by-point basis, with emphasis especially on aspects of our choice of statistical and methodological evaluation, and indicate that it is not necessary to change the conclusions we have drawn.
First, regarding the exclusion of non-peer-reviewed (“unpublished”) studies, “Cochrane Handbook for Systematic Reviews of Interventions” by Higgins and Green had made it clear that the inclusion of data from unpublished studies can introduce bias and the studies that can be located might be an unrepresentative sample of all unpublished studies. In addition, unpublished studies might be of lower methodological quality than published studies [1]. Before analyzing only published studies, we did perform a meta-analysis including unpublished studies in advance. These unpublished studies included three case–control studies [2, 3, 4] and two retrospective studies [5, 6]. This analysis including these unpublished studies yielded primary results similar to those after including only published studies. Specifically, a significant association between PPI use and severe outcomes of COVID-19 was observed (OR/RR = 1.47, 95% CI, 1.20–1.80, I2 = 75.7%) and there was no significant association between PPI use and incidence of COVID-19 (OR/RR = 1.10, 95% CI, 0.77–1.56, I2 = 84.6%). Moreover, no publication bias was found in our meta-analysis on the effect of PPI use on adverse outcomes as evidenced by the symmetrical Begg’s funnel plots and a P-value of 0.57 for the Egger’s test. We note to avoid misunderstanding that the time of analysis including unpublished studies was March 2021. In addition, while acknowledging the urgency of COVID-19, it is reasonable to exclude unpublished studies from the time of our search considering the points mentioned above.
Additionally, the leave-one-out method used as sensitivity analysis is no longer recommended in Cochrane systematic reviews. While meta-analyses increase the statistical power and the precision of ascertaining the studied effect, excluding any single study most likely will change the pooled effect estimate numerically, potentially significantly. Furthermore, inclusion of repeated analyses using a leave-one-out approach increases the likelihood of observing a statistical significant by chance alone due to Type I error inflation [7].
Finally, although several of the unpublished studies have now been published in peer-reviewed journals, as noted in the letter to the editor, they were unpublished at the time the manuscript was submitted, we could not include them in our study. Furthermore, two of the papers [8, 9] indicated in the letter that have been withdrawn after strict peer-review did not examine the impact of PPI or H2RA use on COVID-19 outcomes.
We acknowledge that the Newcastle–Ottawa Scale (NOS) used to rate the methodological quality of observational studies included in our meta-analysis has not yet been published in a peer-reviewed journal [10]. However, the NOS is one of the most commonly used tools for evaluating the methodological quality of studies in a systematic review [11]. Furthermore, use of an unpublished tool to examine the quality of included studies and inclusion of published studies in a meta-analysis are two separate issues.
In the entire manuscript, indeed we reported odds ratio/relative risk (OR/RR)s but erroneously described these as HRs, and this should be changed in the entire manuscript.
Finally, the standard Cochrane method clearly indicates that meta-regression can be implemented for categorical variables with multiple levels by implementing dummy variables with a referent category [1]. Examples of this can be found in the tables presented in several meta-analyses [12, 13]. Though not specifically stated in our Methods, we did use dummy variables for the application of meta-regression.
Author contribution
JHK and HBK conceived and designed the reply. HBK and BJW drafted the manuscript.
BJW corrected English grammar. All authors took part in the interpretation of the results and approved the version of the reply. JHK had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Declarations
Competing interests
The authors declare no competing of interest.
Conflict of interest
The authors declare no conflict of interest.
Footnotes
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Higgins JPT, Green S (editors) (2011) Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. Available from www.handbook.cochrane.org. Accessed 14 October 2021
- 2.Blanc F, Waechter C, Vogel T et al (2020) Preprint. 10.20944/preprints202005.0016.v1
- 3.McKeigue PM, Kennedy S, Weir A et al (2020) Associations of severe COVID-19 with polypharmacy in the REACT-SCOT case-control study. medRxiv. 10.1101/2020.07.23.20160747
- 4.Li J, Cao J, Cai P et al (2020) Risk factors of secondary infections in severe and critical patients hospitalized with COVID-19: a case-control study. Preprint. Research Square. 10.21203/rs.3.rs-55958/v1
- 5.Ullah AZMD, Sivapalan L, Kocher HM, Chelala C (2020) COVID-19 in patients with hepatobiliary and pancreatic diseases in East London: a single-centre cohort study. medRxiv. 10.1101/2020.09.07.20189621
- 6.Corcoles AV, Gracia ES, Gondar OO, et al. Use of distinct anti-hypertensive drugs and risk for COVID-19 among hypertensive people: a population-based cohort study in Southern Catalonia. Spain J Clin Hypertens (Greenwich) 2020;22(8):1379–1388. doi: 10.1111/jch.13948. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Higgins J. Commentary: heterogeneity in meta-analysis should be expected and appropriately quantified. Int J Epidemiol. 2008;37:1158–1160. doi: 10.1093/ije/dyn204. [DOI] [PubMed] [Google Scholar]
- 8.Mehra MR, Desai SS, Ruschitzka F, Patel AN (2020) "RETRACTED: Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis." Lancet Online ahead of print. 10.1016/S0140-6736(20)31180-6 [DOI] [PMC free article] [PubMed] [Retracted]
- 9.Mehra MR, Desai SS, Kuy SR, Henry TD, Patel AN (2020) Cardiovascular disease, drug therapy, and mortality in Covid-19. N Engl J Med 382:e102. 10.1056/NEJMoa2007621 [DOI] [PMC free article] [PubMed] [Retracted]
- 10.Wells GA, Shea B, O’Connell D (2014) The Newcastle-Ottawa Scale (NOS) for assessing the quality of non randomised studies in meta-analyses. http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp. Accessed 24 July 2021
- 11.Luchini C, Stubbs B, Solmi M, Veronese N (2017) Assessing the quality of studies in meta-analyses: advantages and limitations of the Newcastle Ottawa Scale. World J Meta-Anal 5(4):80–84. 10.13105/wjma.v5.i4.80
- 12.Hiller TWR, O'Sullivan DE, Brenner DR, Peters CE, King WD. Solar ultraviolet radiation and breast cancer risk: a systematic review and meta-analysis. Environ Health Perspect. 2020;128:16002. doi: 10.1289/EHP4861. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Nascimento GG, Leite FRM, Vestergaard P, Scheutz F, López R (2018) Does diabetes increase the risk of periodontitis?. A systematic review and meta-regression analysis of longitudinal prospective studies. Acta Diabetol 55:653–667. 10.1007/s00592-018-1120-4 [DOI] [PubMed]