Letter to the Editor: Association is not prediction — choosing the right tools for diagnostic evidence synthesis

Dear Editor

While odds ratios (ORs) are commonly used to quantify the strength of association between a biomarker and an outcome, they should not be mistaken for evidence of predictive utility [1]. An OR expresses how much more likely an outcome is to occur in the presence of a given factor, but this is a retrospective, group-level measure. It does not assess a test’s ability to accurately classify individual patients. In diagnostic contexts, prediction requires specific accuracy metrics – sensitivity, specificity, predictive values, likelihood ratios, and AUCs. Confusing association with prediction can lead to overstated conclusions and misguide clinical decisions.

Likewise, demonstrating a significant difference in mean biomarker values between groups (e.g., complicated vs. uncomplicated appendicitis) does not imply diagnostic performance. Although such findings may suggest association or provide exploratory insight, they offer no information about a biomarker’s individual-level discriminative power. Mean differences remain descriptive; without validation through threshold analysis or accuracy metrics, they are hypothesis-generating at best.

In this context, I offer additional insights into the methodology used in the recent meta-analysis by Shi et al. [2], which evaluates hyponatremia as a predictor of complicated appendicitis.

Firstly, regarding the review design, the eligibility criteria are questionable. Relevant exclusions – such as renal, cardiac, or endocrine disorders (e.g., SIADH, heart failure, chronic kidney disease), medications that alter sodium levels (e.g., diuretics, corticosteroids), pregnancy, and gastrointestinal malignancies – were not clearly defined, despite their known impact on serum sodium and potential for confounding. Moreover, the authors stated that in cases of overlapping datasets, they selected the study with the largest sample size – a choice that risks significant bias if overlapping patient populations are not properly clarified with the original investigators. There are also notable discrepancies between the registered protocol (PROSPERO CRD42023482911) and the final publication. For instance, while the protocol listed PubMed, Embase, and Web of Science as sources, the final manuscript includes Scopus as well – an apparent post hoc inclusion. Additionally, PROSPERO prespecified subgroup analyses based on sodium cutoff values were inadequately explored. Reported thresholds in the review ranged from 134 to 136.5 mEq/L across studies, representing a clinically meaningful variability that is likely to contribute to heterogeneity and threshold effects. However, this issue was neither modeled nor sufficiently discussed.

Secondly, regarding the methodological framework, reviews that evaluate diagnostic performance are guided by a well-established methodological and statistical framework, characterized by the use of dedicated risk of bias tools (e.g., QUADAS-2) [3], reporting checklists (e.g., PRISMA-DTA) [4], and specialized modeling approaches. Within this framework, the present study also raises concerns. The authors relied on the Newcastle–Ottawa Scale (NOS), which is inappropriate in this context, followed the standard PRISMA guidelines rather than the tailored PRISMA-DTA, and applied inverse variance-weighted random-effects meta-analyses of mean differences and ORs – an approach with well-known limitations. Classical inverse-variance methods (e.g., DerSimonian-Laird) for comparing mean differences are suboptimal in this context and, as noted, may only serve an exploratory role, since they fail to capture diagnostic performance. Still, even when such modeling is chosen, they are increasingly regarded as inferior to more robust alternatives, especially under high heterogeneity [5]. Proper diagnostic test accuracy (DTA) assessment requires dedicated hierarchical models that simultaneously account for sensitivity and specificity. Without such modeling the conclusions about predictive utility are methodologically unsupported [6,7].

A striking example of the pitfalls of relying on ORs as diagnostic proxies is provided by Symeonidis et al. [8]. Despite an impressive OR of 42 (Review´s Figure 4), the reported AUC by the original authors was only 0.793—indicative of moderate-to-good performance. This exemplifies how a strong association does not imply high diagnostic accuracy. Unfortunately, such misinterpretations are increasingly common in the recent literature [9], further underscoring the need for adherence to proper DTA methodology.

Thirdly, regarding transparency and reproducibility, there are inconsistencies in study coding and data extraction, and a notable lack of attention to clinical heterogeneity. The study by Maqbool et al. provides a paradigmatic example [10]. Although the review authors extracted data from the “complicated appendicitis” group (n = 23) (Maqbool et al. Table 4), they did not verify the actual composition of this group based on the subgroup data reported in Maqbool´s Table 3. A closer examination shows it consisted of suppurative appendicitis (n = 18), perforated/gangrenous appendicitis (n = 2), and appendicitis with hyperplasia (n = 3). The inclusion of suppurative appendicitis—a histologically less severe form that is frequently classified as non-complicated in the literature—within the “complicated” category introduces a substantial misclassification bias. Likewise, the decision to include an ill-defined and non-standard subgroup such as appendicitis with hyperplasia further compounds the problem, as it represents a heterogeneous entity with unclear clinical significance and no consistent place in conventional classifications. This definitional flaw explains why, in the forest plot of the meta-analysis (the review’s Figure 2), the study by Maqbool et al. is the only one showing an anomalous result—reporting markedly higher sodium levels in the complicated appendicitis group than in the non-complicated appendicitis group—directly contradicting the overall direction of effect in the rest of the literature. By pooling this study without addressing these discrepancies and its consequent outlier result, Shi et al. compromise the validity of the meta-analytic findings, as different spectra of disease severity are conflated under a single label. Additionally, Turhan et al. [11] were incorrectly labeled as “Tarun et al.” in the forest plots, which complicates traceability and hinders reproducibility efforts. Lastly, there is a mismatch between the reported sodium threshold in the original study – defined by the authors as ≤ 137.5 mEq/L – and the value cited in the extraction table of the meta-analysis ( < 135 mEq/L), introducing further uncertainty regarding the validity of the pooled estimates.

Given these issues, I believe the article’s conclusion – that hyponatremia “can be a useful marker for predicting complicated appendicitis” – is overly optimistic and not supported by the presented methodology.

Funding Statement

No funding received.

Disclosure statement

The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

Reviewer disclosure

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose

Writing disclosure

No writing assistance was utilized in the production of this manuscript.