1.
Dear Editor,
We read the recent article with great interest by Wang and Jonas [1], who assessed the likelihood of severe COVID‐19 outcomes among people living with HIV/AIDS (PLHIV) with various comorbidities using both frequentist and Bayesian meta‐analysis approaches. Findings from this systematic review are important for PLHIV with coexisting diabetes, hypertension, cardiovascular disease, respiratory disease and chronic kidney disease as they are at a higher risk of developing severe COVID‐19 outcomes. However, we identified several methodological issues related to planning, conduct, and analytical methods and its reporting that limits the acceptability and generalizability of the findings from this study and could mislead in clinical decision making.
Authors should have used the Preferred Reporting Items for Systematic reviews and Meta‐Analyses (PRISMA) 2020 [2] for reporting and presenting their flow diagram instead of using the PRISMA extension for network meta‐analysis. Similarly, authors did not report the International Prospective Register of Systematic Reviews registration information of this systematic review, which is essential to avoid duplication and transparency. Without a registered protocol, we will never know about what was planned and what has been presented. These are essential features when you conceptualize and report a systematic review. Such practice questions the integrity of overall conduct and reliability of findings. Further, no information about the model selection (i.e. fixed or random) was provided for frequentist meta‐analysis, and both were estimated for Bayesian meta‐analysis. However, fixed or random effect model must be selected as a priori based on the differences in the way studies were conducted and study characteristics [3]. Authors did not present frequentist meta‐analysis results with the prediction interval, which is a common practice to allow more informative inferences in meta‐analyses [4, 5]. The prediction interval reflects the expected range of true effects in similar future studies over different settings. Authors have wrongly planned and tested (using meta‐regression which is suitable to explore heterogeneity in estimates) publication bias, which will mislead to readers. Publication bias should not be assessed when included studies are less than 10 in a meta‐analysis due to low power as recommended in the literature [6].
Both frequentist and Bayesian methods have been applied to generate effect estimates, which is unusual considering both methods to generate same effect estimates as they are fundamentally different in their nature [7]. Authors discussed meta‐analysis under two different setups, viz. frequentist and Bayesian perspective techniques, but the reasoning being using both complementary methodologies, comparison and evaluation of the performances can be more elaborated for better clarity to readers. Authors have used the prior predictive distribution for sensitivity analysis and based on that, they selected half‐normal distribution for further analysis. But prior predictive distribution is considered in the model before taking the observation and is very sensitive to the choice of prior [8]. In order to assess the performance of the prior in association of the given data, one should use the posterior predictive probability [9]. The most appropriate criteria for model selection will be Bayes factor [10]. The inferential procedure presented in the manuscript needs more elaboration on the specification and selection of priors as well as the suitability of the appropriate model. Authors are also lacking on the several aspects of reporting in the methods and results (such as Markov chain Monte Carlo (MCMC) chain convergence and resolution, model diagnostics and reproducible codes) of a Bayesian analysis as recommended in the literature, which is essential for transparency and robustness [11].
We believe that authors should address the points raised and the overall purpose of the presented points, this will only improve the conduct and reporting of frequentist and Bayesian methods in meta‐analysis to benefit researchers at large.
Sincerely,
Ram Bajpai
Vivek Verma
Gyan Prakash Singh
COMPETING INTERESTS
The authors have no competing interests to declare.
AUTHORS’ CONTRIBUTIONS
RB, VV and GPS drafted, read and reviewed the final version of letter.
ACKNOWLEDGEMENTS
RB is affiliated to the National Institute for Health and Care Research (NIHR) Applied Research Collaboration (ARC) West Midlands. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.
REFERENCES
- 1. Wang H, Jonas KJ. The likelihood of severe COVID‐19 outcomes among PLHIV with various comorbidities: a comparative frequentist and Bayesian meta‐analysis approach. J Int AIDS Soc. 2021;. 24:e25841. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;. 372:n71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Borenstein M, Hedges LV, Higgins JP, Rothstein HR. A basic introduction to fixed‐effect and random‐effects models for meta‐analysis. Res Synth Methods. 2010;. 1(2):97–111. [DOI] [PubMed] [Google Scholar]
- 4. IntHout J, Ioannidis JPA, Rovers MM, Goeman JJ. Plea for routinely presenting prediction intervals in meta‐analysis. BMJ Open. 2016;. 6:e010247. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Nagashima K, Noma H, Furukawa TA. Prediction intervals for random‐effects meta‐analysis: a confidence distribution approach. Stat Methods Med Res. 2018;. 28(6):1689–702. [DOI] [PubMed] [Google Scholar]
- 6. Sterne JAC, Sutton AJ, Ioannidis JPA, Terrin N, Jones DR, Lau J, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta‐analyses of randomised controlled trials. BMJ. 2011;. 343:d4002. [DOI] [PubMed] [Google Scholar]
- 7. Spiegelhalter DJ, Myles JP, Jones DR, Abrams KR. Methods in health service research. An introduction to Bayesian methods in health technology assessment. BMJ. 1999;. 319(7208):508–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Gelman A, Simpson D, Betancourt M. The prior can often only be understood in the context of the likelihood. Entropy. 2017;. 19(10):555. [Google Scholar]
- 9. Lewis PO, Xie W, Chen MH, Fan Y, Kuo L. Posterior predictive Bayesian phylogenetic model selection. Syst Biol. 2014;. 63(3):309–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Kass RE, Raftery AE. Bayes factors. J Am Stat Assoc. 1995;. 90(430):773–95. [Google Scholar]
- 11. Kruschke JK. Bayesian analysis reporting guidelines. Nat Hum Behav. 2021;. 5:1282–91. [DOI] [PMC free article] [PubMed] [Google Scholar]