Reply to “Conceptual interpretation and clinical applicability of A systematic review and meta-analysis about prognostic value of Apolipoproteins in COVID-19 patients”

Dear Editor

We thank Sundari Rajagopal M et al. for their interest in our systematic review and metanalysis. In response to their objections, we noted these points:

1. Interpretation of statistical significance or estimated effect size

We disagree with what the authors mentioned about interpreting the odds ratio (OR) of the ApoB/A1 ratio. In the interpretation of a measure of association estimated by meta-analysis (MA), there are two considerations: statistical significance, evaluated using p-values and confidence intervals (CI), and clinical significance, evaluated using direction and magnitude of effect (strength of association) [1]. In some studies, the results may be statistically significant but not clinically relevant. However, clinical significance cannot be established without statistical significance. In this way, it would not be correct to interpret the clinical significance of the ApoB/ApoA1 ratio because its effect measure is not statistically significant according to the reported CI and p-value. Furthermore, we have noted that the references used by the authors to support their claims come from the self-citation of letters to the editors rather than articles about methodological support.

Regarding the argument of the number of studies, in the Cochrane Manual of Systematic Reviews, there is no consensus or agreement on the minimum number of studies necessary to establish clinical or statistical significance [2]. They mentioned that the measures of association obtained are the indicators to evaluate the clinical significance of the results. Therefore, the clinical significance is evaluated with the indicators mentioned above, the direction of the effect and its magnitude.

Empirically, effects that accumulate a benefit or harm greater than 30% could be considered clinically relevant in dichotomous outcomes; however, there is no consensus on a cut-off point to establish the clinical relevance of the results. According to the guidelines rating the quality of evidence (GRADE), a long effect is defined as a risk ratio (RR) <0.5 or RR> 2, however, similar definitions for OR are not mentioned. In our study, the ApoB/ApoA1 ratio results do not meet the criteria mentioned above for statistical significance, so its clinical significance cannot be interpreted.

2. Mortality should be Hazard Ratio instead of Odds ratio or Standarized Mean Difference

At first, it was thought to use only mean differences (MD), but we decided to report OR as only one effect size using the Chinn method due to included studies in the systematic review reported OR and MD [3]. The OR is an adequate measure when follow-ups are short, as in the case of studies included in our meta-analysis. It is essential to use all the available information for the quantitative synthesis, so the conversion from MD to OR is a strength of our meta-analysis.

The prognostic value of any biomarker could be described as OR, risk ratio (RR) or Hazard Ratio (HR) in prognostic model studies, those measures are valid, and their interpretation could be very similar [4]. Our study did not use HR because it was not reported in available primary studies. We decided not to convert OR to HR for avoiding misestimate effect measures.

The HR is preferred for mortality studies because they include the rate of change over time; However, some prognostic models assess mortality using OR and are valid, such as those developed in COVID-19, to predict mortality in hospitalized patients [5]. The standardized mean difference is not useable in any prognostic model of mortality, and likewise, due to the difficulty that clinicians have in its interpretation, it is not advisable to be reported.

Regarding the argument of the number of studies, in the Cochrane Manual of Systematic Reviews, there is no consensus or agreement on the minimum number of studies necessary to establish clinical or statistical significance.

3. Publication bias indicators

Sundari Rajagopal M et al. have a terms confusion, what publication bias assess is small study effects and not “small or missing studies” as they mention. There are different statistical tests to assess publication bias (Egger, Begg, Harbord, Peters, etc.), and according to the Cochrane manual, they all have low statistical power to establish the existence of publication bias. The Duval et al. test, the trim-and-fill method, is a method to assess publication bias and after determining the existence of publication bias. The Cochrane manual does not recommend using the Begg test because it has lower statistical power than the Egger test. In addition, when the data set is small, a study recommended using Spearman's rho instead of Kendall's tau in the Begg test for avoiding misestimation [6]. The Harbord test is preferred for binary outcomes; however, this test is recommended in MA from randomized clinical trials [7]. In our study, we used the Egger test and the trim-and-fill method according to the recommendations of the Cochrane Manual. Thus, the indiscriminate use of different publication bias tests does not add any value to the clinical interpretation of the results, as stated by Sundari Rajagopal M et al.

Authors contributions

Juan R. Ulloque-Badaracco: Conceptualization, and Writing - Original Draft. Enrique A. Hernandez-Bustamante: Conceptualization, and Writing - Original Draft. Percy Herrera-Añazco: Conceptualization, and Writing - Original Draft. Vicente A. Benites-Zapata: Conceptualization, Writing - Original Draft and Visualization and Supervision.

Declaration of competing interest

No conflicts of interest for all authors.