Data Extraction Errors and Diagnostic Conclusions of Meta-Analyses: A Systematic Review

Zixing Huang; Yuan Tian; Ping Gong; Tong Zhang; Hehan Tang

doi:10.1001/jamanetworkopen.2026.40850

. 2026 Oct 29;9(10):e2640850. doi: 10.1001/jamanetworkopen.2026.40850

Data Extraction Errors and Diagnostic Conclusions of Meta-Analyses

A Systematic Review

Zixing Huang ¹, Yuan Tian ², Ping Gong ³, Tong Zhang ³, Hehan Tang ^1,^✉

¹Department of Radiology, West China Hospital of Sichuan University, Chengdu, China

²Department of Orthopedics, The First Hospital of China Medical University, China Medical University, Shenyang, China

³School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, China

Accepted for Publication: August 31, 2026.

Published: October 29, 2026. doi:10.1001/jamanetworkopen.2026.40850

Open Access: This is an open access article distributed under the terms of the CC-BY-NC-ND License, which does not permit alteration or commercial use, including those for text and data mining, AI training, and similar technologies. © 2026 Huang Z et al. JAMA Network Open.

^✉

Corresponding Author: Hehan Tang, PhD, Department of Radiology, West China Hospital of Sichuan University, Sichuan University, Chengdu 610000, China (lanny.320@163.com).

Author Contributions: Drs Tang and Huang had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs Huang and Tian contributed equally to this work.

Concept and design: Huang, Gong, Zhang, Tang.

Acquisition, analysis, or interpretation of data: Huang, Tian, Gong, Tang.

Drafting of the manuscript: Huang, Tian, Gong, Zhang.

Critical review of the manuscript for important intellectual content: Huang, Gong, Tang.

Statistical analysis: Huang, Tian, Gong, Zhang.

Administrative, technical, or material support: Huang, Tian.

Supervision: Huang, Tang.

Conflict of Interest Disclosures: None reported.

Funding/Support: This study was supported by the Science and Technology Support Program of Sichuan Province (grant No. 2024YFFK0260).

Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Data Sharing Statement: See Supplement 2.

Additional Contributions: We acknowledge volunteers in data extraction: Jianzhou Liu, BA, and Kailing Cui, BA (Shanxi Medical University); Siyu Wang, BA, and Qicheng Wang, BA (Chongqing Medical University); Jianuo Hou, MA, Sirui Zhangm, MA, and Siyu Zhao, MA (Southwest University of Science and Technology); Zifeng Chen, BA, and Yintao Ye, BA (North Sichuan Medical College); Chunyu Zhang, MA (Inner Mongolia Medical University); Xiaoyi Gong, BA (Xinjiang Second Medical College), for helping with data collection; and Ruiyang Zhang, BA (Anhui Medical University). All individuals named above received compensation for their contributions on data extraction from primary studies.

^✉

Corresponding author.

PMCID: PMC13720697 PMID: 42954991

This systematic review evaluates data extraction errors in diagnostic meta-analyses and extent to which such errors affect results of meta-analyses.

Key Points

Question

How common are data extraction errors in diagnostic meta-analyses, and to what extent do they affect results of meta-analyses?

Findings

In this systematic review including 46 systematic reviews, 879 diagnostic comparisons, and 6173 primary studies, data extraction could not be reproduced in 338 of 6173 trials, and 34 of 46 Cochrane reviews and 137 of 414 diagnostic meta-analyses had at least 1 data extraction error. No significant systematic bias was detected when such errors were corrected.

Meaning

These findings suggest that adherence to a rigorous data extraction process remains a priority, and authors of systematic reviews should balance the robustness and interpretability when selecting diagnostic performance measures.

Abstract

Importance

Systematic reviews (SRs) are often characterized as being inherently reproducible; however, errors arising from the evidence synthesis process can compromise this assumption and lead to reproducibility failure.

Objective

To investigate the data extraction errors in diagnostic meta-analyses and the extent to which such errors affect the results of meta-analyses.

Evidence Review

We included SRs with diagnostic meta-analyses from the Cochrane Library filtered by diagnostic up to May 2023. Data used in the original meta-analyses were extracted from the included SRs and independently reproduced from primary studies. Data extraction errors were identified by comparing datasets and classified by potential mechanism. To assess the consequences, we reanalyzed the meta-analyses using error-corrected data with the same effect estimators and methods. The primary outcomes were the proportions of data extraction errors at the trial, meta-analysis, and SR levels. The secondary outcome was the consequences for diagnostic performance measures (eg, sensitivity, specificity, diagnostic odds ratio [DOR], and area under the curve [AUC]). Analysis was conducted between March and June 2026.

Findings

A total of 46 SRs, 879 diagnostic comparisons, and 6173 primary studies were included. Data extraction could not be reproduced in 338 studies (5.5%) and affected 137 diagnostic meta-analyses (33.1%) and 34 SRs (73.9%). Numerical error (ie, incorrect values) and inferential error (ie, incorrect derivation from statistics) were the most prevalent error types, accounting for 166 of 412 (40.3%) and 140 of 412 (34.0%) errors, respectively. Mismatching error (ie, mismatching data to groups; 69 of 412 errors [16.7%]) and conceptual error (ie, misaligned reference details; 37 of 412 [9.0%]) were also identified. Among 137 reproduced meta-analyses, large changes were observed in 0.7% meta-analyses for sensitivity (absolute differences, >0.20), 3.6% for DOR (fold-changes, >5.0), and 2.9% for AUC (absolute differences, >0.10). Regression analysis of error location showed that combined false-positive and true-negative errors were associated with higher likelihood of moderate or large DOR changes (OR, 1.11; 95% CI, 1.01-1.22).

Conclusions and Relevance

In this SR of diagnostic meta-analyses in Cochrane reviews, data extraction errors were common and differentially affected performance measures, highlighting the need to implement rigorous data extraction processes and to consider careful selection of performance measures.

Introduction

Reproducibility underpins the credibility of scientific knowledge. At its core, it requires that the same data and analysis are used to generate the same or reasonably similar results¹; when this condition is not met, either due to unavailable data or analytic code or errors arising from the original or reproduction process, reproducibility failure occurs.^2,3 Such failure increases uncertainty about the validity of the original findings, and this crisis of confidence has extended across multiple scientific disciplines,^4,5,6 including medicine.^7,8,9 In health care, reproducibility is more important than ever because such unease propagates beyond individual trials into the broader evidence ecosystem. In this context, evidence synthesis has been proposed as a key mechanism to mitigate inconsistent and conflicting findings and to provide summary effects; its explicit and transparent methodology is widely assumed to render it valid and inherently reproducible.

A growing body of reproducibility studies has challenged this assumption, examining whether evidence synthesis procedures could be fully reproduced. One study by Nguyen et al¹⁰ found that 14.0% of meta-analyses were not fully reproduced due to missing information (eg, analytic code). Beyond issues of data and code inaccessibility, only a limited number of studies have attempted to examine the errors arising from review process (ie, search AND data extraction) and assess the consequences of these discrepancies on the findings of meta-analyses.^11,12,13,14 Among these steps, data extraction process represents one of the most vulnerable steps, affecting 85% of systematic reviews (SRs) and 67% of meta-analyses.¹⁵ Gøtzsche et al¹⁶ reported that 63% of meta-analyses of continuous outcomes had at least one data extraction error. Once incorporated into the synthesis, such errors affect the effect estimates and undermine the validity of conclusions.

Understanding the mechanisms of data extraction errors is therefore critical for identifying their sources, as these errors differ in nature across different types of underlying data. The distribution of data extraction errors differs between binary and continuous outcomes; for example, errors are prevalent in the extraction of standard deviation, often arising from back-calculation when data are missing.^16,17 Meanwhile, when data tend to be rare and sparse, such as serious harms, extraction errors are more likely to distort effect estimates; approximately 10% of meta-analyses of binary outcomes for medication-harms were substantially affected.¹⁵ However, existing evidence has been largely restricted to data from intervention studies with binary outcomes, leaving data extraction errors among interdependent measures (eg, false positive [FP]) in diagnostic test accuracy studies largely unexplored. Among these measures, FP counts are often sparse and approach 0, such that even minor errors could distort the diagnostic estimators and mislead final conclusions.

To address this gap, we leveraged the open-access contingency table data in the Cochrane Library to examine the extent and consequences of data extraction errors in diagnostic meta-analyses. We aimed to propose an empirical taxonomy of data extraction errors to characterize their underlying mechanisms, and to explore the susceptibility of diagnostic performance measures to these errors.

Methods

This study was a systematic review of Cochrane reviews that aimed at investigating the incidence and consequences of data extraction errors in diagnostic effect estimates of meta-analyses. The reporting of this study followed the Reporting Guideline for Overviews of Reviews of Healthcare interventions (PRIOR) guideline¹⁸ whenever possible. As this study involved only secondary analysis of publicly available data and did not involve patient-level information, institutional review board approval was not required in accordance with 45 CFR §46.

Data Sources and Literature Screening

We focused on diagnostic meta-analyses in Cochrane reviews, as their publicly accessible trial-level data provide a benchmark for evaluating data extraction errors and facilitating reproducibility.¹⁹ In brief, we searched the Cochrane Library filtered by the diagnostic review type, including SRs that used imaging examinations (eg, radiograph) as the index test up to May 2023, and only the latest version of reviews was included. To enable reproducibility, we included only SRs that reported 2-by-2 tables for each included study. Eligible reviews were further restricted to those with meta-analyses synthesizing at least 5 trials to reduce variability arising from random error. Additionally, diagnostic comparisons from these reviews that were not synthesized quantitatively were also included, given that errors could occur regardless of whether data were incorporated into a meta-analysis.

Two authors (P.G. and T.Z.) screened titles and abstracts independently using the online application Rayyan,²⁰ and records excluded by both authors were removed. Full texts of the remaining records were then retrieved and assessed for eligibility with documented detailed reasons for exclusions. Any disagreements were resolved through discussion until consensus was achieved.

Data Collection

The information was collected at 3 levels: the trial level, the meta-analysis level, and the systematic review level. At the trial level, the data were obtained from Review Manager 5 files, which included author, publication year, and metadata, namely true positive (TP), true negative (TN), false negative (FN), and FP in each meta-analysis; at the meta-analysis level, diagnostic performance measures were extracted, including sensitivity, specificity, diagnostic odds ratio (DOR), and area under the curve (AUC); at the systematic review level, we collected authors, country of corresponding authors, year of publication, number of electronic databases searched, number of included trials, number of meta-analyses, quality assessment tools, funding sources (eg, nonprofit), diseases, and updates (yes vs no). Moreover, we collected details of the data extraction process based on the reporting of each systematic review: the number of data extractors involved, whether data templates were used (yes vs not reported), the professional background of data extractors (eg, clinicians), and the methods used to resolve disagreements (eg, third-party adjudication). Data extraction was conducted by paired volunteers independently with cross-checking, and further checked by a lead author (P.G.).

Data Reproduction

Volunteers were trained before formal data extraction, and only those achieving more than 90% accuracy in a pilot study proceeded. To identify the data extraction errors, 11 trained volunteers were required to re-extract the same outcomes included in meta-analyses from the primary studies. Paired volunteers identified the outcomes in primary studies independently and resolved the conflicts. If the trial data reported in the original database were not identical to the data from the primary studies, it was classified as a data extraction error. For each identified error, we recorded the location where the discrepancy occurred between the primary and reproduced extracted data (ie, TP, TN, FP, or FN) and the reasons for nonreproducibility. If full-text sources of primary studies were not accessible, original data were retained under a conservative assumption of no error. A core author (P.G.) and senior methodologists (Z.H. and H.T.) verified and judged whether the errors were attributed to systematic review authors and proposed potential error classifications. Any issues arising from this process were resolved by consensus and group discussion (eFigure 1 in Supplement 1).

Outcomes

The primary outcomes were the proportions of data extraction errors at the trial level, the meta-analysis level, and the systematic review level. The secondary outcomes were the changes in diagnostic performance measures before and after correction (eg, sensitivity, specificity, DOR, and AUC) when the original meta-analyses were reanalyzed using the corrected data, with changes classified as low, moderate, or large based on predefined thresholds to enhance clinical interpretability.

Statistical Analysis

We summarized baseline characteristics of included SRs, with binary outcomes reported as frequencies and proportions and continuous outcomes reported as medians and IQRs. For primary outcomes, we reported data extraction errors at the trial level, the meta-analysis level, and the systematic review level, defined as follows: the proportion of trials in which data extraction errors were identified; the proportion of meta-analyses containing at least 1 trial with a data extraction error; and the proportion of SRs containing at least 1 meta-analysis with a data extraction error. To account for potential clustering arising from different conflict-resolving approaches in data extraction practices, we treated each systematic review as a cluster and estimated the adjusted error rates (AERs) using a generalized linear mixed model (GLMM) with a random intercept for each systematic review.

For the secondary outcomes, we reproduced the meta-analyses with data extraction errors using error-corrected data and the same analytical methods and re-estimated diagnostic performance (eg, sensitivity, specificity, DOR, and AUC). Bland-Altman plots were used to assess the agreement between estimates before and after correction by plotting the difference against their mean to assess systematic bias and variability. Depending on the bounded scale and clinical interpretability of each diagnostic performance measure, we predefined different thresholds for each metric (eFigure 7 and eFigure 8 in Supplement 1). For sensitivity or specificity, absolute changes were classified as low (≤0.05), moderate (>0.05), or large (>0.20) after error correction. For DOR, absolute changes were classified as low (0.50-2.00), moderate (0.20-0.50 or 2.00-5.00), or large (<0.20 or >5.00), considering its multiplicative nature and reflecting fold changes on the log scale. For AUC, absolute changes were classified as low (≤0.05), moderate (>0.05), or large (>0.10) given its constrained distribution. We summarized the proportions of the magnitude of absolute changes classified as low, moderate, and large.

We conducted an exploratory regression analysis to explore association between error locations and the likelihood of moderate or large change in meta-analysis effect estimates. We used a GLMM with a logit link, with magnitude of changes for each diagnostic performance coded as response variable (1 indicates moderate or large changes and 0 indicates low changes) and error locations coded as explanatory variable (1 indicates error present and 0 indicates error absent). For each diagnostic performance measure, only the related error locations were included in the regression (eg, including TP and FN errors for sensitivity), whereas a robust variance estimate was used by treating meta-analysis as a cluster. To explore combined effects of multiple errors within a trial, interaction terms between error locations were included to assess their joint effects. We reported the odds ratio (OR) with 95% CIs as the effect estimate, owing to its portability across baseline risk.²¹

When original data were unavailable, we treated them as missing values. As a sensitivity analysis, meta-analyses with missing values were excluded and the proportions of data extraction errors were re-estimated. We used Stata/SE version 16.0 (StataCorp LLC) to reproduce meta-analyses and conduct the data analysis with a significance level of α = .05. All analysis was conducted between March and June 2026.

Results

A total of 46 SRs that included 879 diagnostic comparisons and 6173 trials were included^{22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67} (Figure 1); and Table 1 presents the characteristics of included reviews. All review protocols drafted a plan for data extraction, and 43 reviews (93.5%) had 2 data extractors. Only 5 reviews (10.9%) reported the professional background of data extractors, and 17 (37.0%) piloted the standard data extraction form. To address conflicts in data extraction, 21 reviews (45.7%) employed adjudication by a third party, 12 reviews (26.1%) resolved conflicts by discussion, and 10 (21.7%) used consensus or arbitration.

Flowchart of systematic review screening and inclusion counts for diagnostic meta-analyses. Light gray vertical flowchart with rectangular boxes connected by gray arrows. Top box: 176 Cochrane database of systematic review for systematic reviews filtered by diagnostic on May 2023. Downward arrow to second box: 176 Records screened by titles and abstracts. From this second box, a rightward arrow leads to a side box: 113 Records excluded. A downward arrow from the second box leads to the next main box: 63 Full-text articles assessed for eligibility. From this box, a rightward arrow leads to a side box listing exclusions: 17 Excluded; 7 Included less than 5 trials in meta-analyses; 6 No available 2 × 2 table; 4 Systematic reviews without meta-analyses. A downward arrow continues to the next main box: 46 Included by 2 independent reviewers. Downward arrow to the next main box: 46 Systematic reviews of imaging-based diagnostic tests, including at least 1 meta-analysis of at least 5 trials and providing 2 × 2 table data. Downward arrow to the next main box: 46 Systematic reviews with 879 diagnostic test accuracy comparisons including 6173 trials. From this box, a rightward arrow leads to a side box: 465 Comparisons with 713 trials not synthesized quantitatively. A final downward arrow leads to the bottom main box: 46 Systematic reviews with 414 meta-analyses including 5460 trials were reproduced to assess impact of data extraction errors on conclusion.

Table 1. Basic Characteristics of Included Diagnostic Systematic Reviews (n = 46).

Basic characteristics	Systematic reviews, No. (%) (N = 46)
Databases searched, median (IQR)	5 (4-7)
Meta-analyses in review, median (IQR)	4 (2-9)
Trials included in review, median (IQR)	24 (14-47)
Geographic area^a
Northern America	5 (10.9)
Asia	3 (6.5)
Eastern European	33 (71.7)
Year of publication
Before 2015	5 (10.9)
2015-2020	29 (63.0)
After 2020	12 (26.1)
Quality assessment
QUADAS	7 (15.2)
QUADAS-2	39 (84.8)
Source of funding
Nonindustry funding	38 (82.6)
No funding	6 (13.0)
No information	2 (4.3)
Update of systematic reviews
Yes	7 (15.2)
No	39 (84.8)
No. of data extractors
2	43 (93.5)
3	3 (6.5)
Background of data extractors
Clinician or methodologist	5 (10.9)
Unclear	41 (89.1)
Resolution for conflicts in data extraction
Third-party adjudication	21 (45.7)
Consensus or arbitration	10 (21.7)
Discussion	12 (26.1)
Unclear	3 (6.5)
Pilot test for standard form
Yes	17 (37.0)
Unclear	29 (63.0)

Open in a new tab

Abbreviation: QUADAS, Quality Assessment of Diagnostic Accuracy Studies.

^{^a}

Geographic areas were classified by country of corresponding authors. Geographic areas were classified by the country of the corresponding authors, taking into account economic and ethnic characteristics.

Reproducibility and Classification of Data Extractions

At the trial level, 338 of 6173 trials (5.5%; AER, 5.5%; 95% CI, 4.9% to 6.0%) could not be reproduced. At the meta-analysis level, 178 of 879 diagnostic comparisons (20.3%; AER, 22.9%; 95% CI, 15.3% to 30.4%) had at least 1 trial with data extraction errors; among comparisons without quantitative synthesis, 44 of 465 of those (8.8%; AER, 8.8%; 95% CI, 6.2% to 11.4%) had at least 1 data extraction error, whereas among comparisons with quantitative synthesis, 137 of 414 of those meta-analyses (33.1%; AER, 35.1%; 95% CI, 25.8% to 44.4%) had at least 1 data extraction error, and the proportion of errors within meta-analyses ranged from 6.5% to 16.7%, with a median (IQR) value of 1 (1-3). At the systematic review level, 34 of 46 SRs^{22,24,25,26,27,28,29,30,31,33,34,39,40,41,44,45,47,48,49,50,51,52,53,54,55,56,57,59,60,61,63,64,66,67} (73.9%; AER, 73.9%; 95% CI, 61.2% to 86.6%) had at least 1 affected meta-analysis; the proportion of affected meta-analyses ranged from 3.5% to 100%, with a median (IQR) value of 50.0% (26.3% to 75.0%) (eFigure 6 in Supplement 1).

Based on the predefined error taxonomy, we classified these errors into 4 types: conceptual errors, mismatching errors, numerical errors, and inferential errors. Table 2 presents detailed definitions and examples of each error type.^68,69,70,71 Among 338 trials with data extraction errors, 412 error events were identified; 264 trials (78.1%) were classified as single errors and 74 trials (21.9%) as mixed errors. Numerical errors were the most frequent, accounting for 166 (40.3%), and inferential errors were the second most prevalent, accounting for 140 (34.0%). Mismatching errors accounted for 69 (16.7%) and conceptual errors for 37 (9.0%). The distribution of data extraction errors is presented in Figure 2.

Table 2. Description and Example of Data Extraction Error Types.

Type of error	Description	Example
Conceptual error	Conceptual errors occur when outcome definitions or data structures are misunderstood, resulting in inappropriate extraction or classification of data.	In a meta-analysis evaluating endoscopic ultrasonography for gastric cancer T staging,³¹ pathological staging served as the reference standard. The correct 2 × 2 table was TP = 4, FP = 1, FN = 0, and TN = 6.⁶⁸ During data extraction, a patient with pathological T3 disease was misclassified as negative, leading to an incorrect increase in TN to 7.
Mismatching error	Mismatching errors refer to incorrect alignment between extracted data and their corresponding group labels, resulting in misclassification of contingency table cells.	In a meta-analysis evaluating optical coherence tomography for early primary open-angle glaucoma,²⁵ the original study included 34 patients with primary open-angle glaucoma and 42 controls.⁶⁹ During data extraction, the case and control groups were mistakenly reversed, leading to misclassified contingency data (TP = 31, FP = 2, FN = 11, and TN = 32). As a result, the derived groups with and without disease were inverted (42 vs 34), inconsistent with the original study population.
Numerical error	Numerical errors refer to incorrect extraction or transcription of numerical values from primary studies, leading to inaccurate entries in the dataset.	In a meta-analysis evaluating magnetic resonance cholangiopancreatography for detecting common bile duct stones,⁴⁵ the original study reported TP = 37, FP = 2, FN = 4, and TN = 52.⁷⁰ During data extraction, FP and FN were mistakenly transposed, leading to misclassification of contingency table cells (TP = 37, FP = 4, FN = 2, and TN = 52).
Inferential error	Inferential errors arise when data are derived or reconstructed incorrectly from reported statistics, leading to inaccurate estimation of underlying values.	In a meta-analysis evaluating dobutamine stress echocardiography for coronary artery disease,²⁴ the original study reported summary measures (sensitivity 88% and specificity 62%) and sample size information (38 patients, including 17 with disease).⁷¹ The correct 2 × 2 table was TP = 15, FP = 8, FN = 2, and TN = 13. During data reconstruction, FP and FN were mistakenly reversed, leading to misclassification of contingency table cells.

Open in a new tab

Abbreviations: FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Grouped bar chart of studies percent by error type and error category. Single-panel grouped bar chart with vertical axis labeled Studies, percent, ranging from 0 to 80 with tick marks every 10 and light horizontal gridlines. Horizontal axis has five category labels: Types of error, n equals 412; Conceptual error, n equals 69; Mismatching errors, n equals 140; Numerical errors, n equals 166; Inferential error, n equals 37. A legend in the upper right titled Error contains three colored keys: Single in dark teal, Mixed in orange, and Overall in light blue gray. For Types of error, two bars appear: Single near 78 percent and Mixed near 22 percent; no Overall bar is present in this group. For Conceptual error, three bars appear: Single near 15 percent, Mixed near 20 percent, and Overall near 17 percent. For Mismatching errors, three bars appear: Single near 39 percent, Mixed near 24 percent, and Overall near 34 percent. For Numerical errors, three bars appear: Single near 45 percent, Mixed near 32 percent, and Overall near 40 percent. For Inferential error, three bars appear: Single near 1 percent, Mixed near 24 percent, and Overall near 9 percent. Bars are outlined in a dark stroke and arranged side by side within each category group.

Data Extraction Errors and Effect Estimates

The consequences of data extraction errors for effect estimates was examined across 137 meta-analyses using error-addressed data and summarized as simple means of the changes in diagnostic performance measures (Figure 3). For sensitivity, 1 meta-analysis (0.7%) had large changes and 7 meta-analyses (5.1%) had moderate changes, with a mean difference of −0.00 (95% limits of agreement, −0.07 to 0.07). For specificity, 1 meta-analysis (0.7%) had moderate changes, with a mean difference of −0.00 (95% limits of agreement, −0.03 to 0.03) (eFigure 2 in Supplement 1). For DOR, 5 meta-analyses (3.6%) had large changes, and 3 (2.1%) had moderate changes, with a mean difference of −0.04 (95% limits of agreement, −0.59 to 0.56) on the log scale and relative DOR of 0.96. For AUC, 4 meta-analyses (2.9%) had large changes, and 4 meta-analyses (2.9%) had moderate changes, with a mean difference of −0.00 (95% limits of agreement, −0.14 to 0.13) (eFigure 3 in Supplement 1).

Horizontal stacked bars by metric, x axis Studies percent from 90 to 100. Single-panel horizontal stacked bar chart with four rows labeled on the left as A U C, D O R, Specificity, and Sensitivity. A boxed legend centered near the top labeled Impact contains three color keys: Low in dark blue-gray, Moderate in orange, and Large in pale blue-gray. The horizontal axis along the bottom labeled Studies, percent spans from 90 to 100 with tick marks at each integer; light vertical gridlines align with the ticks. Each row contains one stacked bar starting at 90 and ending at 100, with segment boundaries outlined in black. For A U C, the Low segment extends from 90 to about 94 point 2, the Moderate segment from about 94 point 2 to about 97 point 0, and the Large segment from about 97 point 0 to 100. For D O R, the Low segment extends from 90 to about 94 point 2, the Moderate segment from about 94 point 2 to about 96 point 4, and the Large segment from about 96 point 4 to 100. For Specificity, the Low segment extends from 90 to about 99 point 3, followed by a Moderate segment from about 99 point 3 to 100; no Large segment is visible in this row. For Sensitivity, the Low segment extends from 90 to about 94 point 2, the Moderate segment from about 94 point 2 to about 99 point 3, and the Large segment from about 99 point 3 to 100. — AUC indicates area under the curve; DOR, diagnostic odds ratio.

Among the 137 reproduced meta-analyses, 51 (17.3%) had only a single error cell and 243 (82.7%) had multiple error cells (eFigure 4 in Supplement 1). Regression analysis revealed no significant independent single error cells, while an exploratory analysis indicated that the interaction effect between FP and TN errors was associated with a higher likelihood of moderate or large changes in DOR (OR, 1.11; 95% CI, 1.01 to 1.22) (eFigure 5 in Supplement 1).

Sensitivity Analysis

For 477 of 6173 trials (7.7%), data accuracy could not be verified through full-text sources. After treating them as missing values, AERs remained robust: 5.9% at the trial level, 30.1% at the meta-analysis level, and 73.9% at the systematic review level.

Discussion

In this SR of diagnostic meta-analyses in Cochrane reviews, our results suggested that data extraction errors were prevalent in diagnostic meta-analyses, with 73.9% of SRs containing at least 1 affected meta-analysis and 33.3% of diagnostic meta-analyses containing at least 1 affected trial. However, the consequences of data extraction errors for effect estimates remained limited, with no evidence of systematic bias detected; large changes were observed in 0.7% to 3.6% across diagnostic performance measures after error correction. Among these measures, DOR was the most susceptible, while specificity demonstrated the greatest robustness.

The classification of error types reflects 2 distinct mechanisms: data estimation error arising from back-calculation from reported statistics, and data extraction error in the extraction process. These findings highlighted the importance of directly reporting raw diagnostic cells, as recommended by Standards for Reporting of Diagnostic Accuracy reporting guidelines,⁷² which may be essential to address such issues at the source (trial-level). Conceptual errors arise from ambiguous outcome definitions across primary studies. This is analogous to the diagnostic reference standard bias described by Kellerhuis et al,⁷³ where discrepancies between expert assessments and study criteria propagate systematic errors into extracted data. For instance, when diagnostic criteria for hepatocellular carcinoma nodules differ between primary studies and SRs, such as arterial phase enhancement followed by rapid washout, 2 nodules without arterial enhancement were incorrectly classified as true positives,⁷⁴ reflecting the complexity of consistently applying such criteria during data extraction.

Our findings suggest that diagnostic performance measures differ in their susceptibility to data extraction errors, with DOR being particularly vulnerable, as evidenced by 3.6% of meta-analyses showing large changes. In our data, the DOR amplifies errors through its multiplicative structure, by which errors in FP and FN can be amplified by the denominator, and errors in TP and TN can be magnified by the numerator.⁷⁴ However, sensitivity and specificity appeared to confer greater robustness to meta-analyses, as these measures are bounded between 0 and 1, inherently limiting the consequences of extraction errors. In addition, the susceptibility of DOR was compounded by sparse and imbalanced data—counts of FP were small (median [IQR], 5 [1-21]); however, counts of TN were substantial (median [IQR], 50 [22-119]), which may further amplify the consequences of extraction errors on DOR estimates. While consequences of a single error were limited, the combined errors increased the likelihood of leveraging the conclusions. Our findings suggested that researchers may need to consider not only clinical interpretability but also the potential vulnerability of diagnostic performance measures to data extraction errors.

Strengths and Limitations

To our knowledge, this study is the largest reproducibility study to assess data extraction errors in diagnostic meta-analyses, providing direct empirical evidence on their incidence and consequences for effect estimates. Furthermore, this study provides an empirical taxonomy for data extraction errors specific to diagnostic meta-analyses, which offers a theoretical basis to better understand the mechanisms of errors in evidence synthesis practice.

However, several limitations should be acknowledged. First, our database covered only Cochrane reviews that used imaging examinations as the index test, given its relatively high quality of reporting and transparent process¹⁹; the true incidence of such errors may be underestimated in other publications. Second, 477 of 6173 trials (4.7%) had inaccessible full-text sources and were retained as error-free under a conservative assumption, which may lead to a slight underestimation of overall error rates. Third, the consequences of data extraction errors were restricted to 4 diagnostic performance measures without examining other outputs (eg, heterogeneity, prediction regions, and threshold effects), which are equally important for result interpretation in diagnostic evidence.

Conclusions

In this systematic review of Cochrane reviews, data extraction errors were common in diagnostic meta-analyses and could compromise the validity of evidence synthesis. Their consequences were limited but varied across performance measures, reflecting differences in mathematical structure and data dependence. These findings underscore the importance of transparent reporting of raw diagnostic data and careful selection of performance measures.

Supplement 1.

eFigure 1. Flowchart of Data Extraction Errors Identification

eTable 1. Detailed Reasons and Lists for Exclusions in Full-Text Screening

eTable 2. Inclusion Lists for Eligible Systematic Reviews With Diagnostic Meta-Analyses

eFigure 2. Bland- Altman Plot Comparing Reproduced and Original Meta-Analysis of Sensitivity and Specificity

eFigure 3. Bland-Altman Plot Comparing Reproduced and Original Meta-Analysis of Diagnostic Odds Ratio (DOR) and Area Under Curve (AUC)

eFigure 4. The Distribution of Sites on Data Extraction Errors

eFigure 5. Association Between Error Locations and Diagnostic Performance Measures

eFigure 6. Association Between Number of Included Studies Within Each Systematic Review, Studies With Data Extraction Errors and Percentages of Errors

eFigure 7. Sensitivity Analysis for Thresholds for Diagnostic Performance Measures

eFigure 8. Sensitivity Analysis for Thresholds for Diagnostic Performance Measures

eTable 3. Changes of Reproduced Meta-Analyses Using Error-Addressed Data in Sensitivity

eTable 4. Changes of Reproduced Meta-Analyses Using Error-Addressed Data in Specificity

eTable 5. Changes of Reproduced Meta-Analyses Using Error-Addressed Data in Diagnostic Odd Ratio

eTable 6. Changes of Reproduced Meta-Analyses Using Error-Addressed Data in Area Under the Curve

jamanetwopen-e2640850-s001.pdf^{(2MB, pdf)}

Supplement 2.

Data Sharing Statement

jamanetwopen-e2640850-s002.pdf^{(16.2KB, pdf)}

References

1.Barba LA. Terminologies for reproducible research. Arxiv. Preprint posted online February 9, 2018; doi: 10.48550/arXiv.1802.03311 [DOI]
2.Siddaway AP, Wood AM, Hedges LV. How to do a systematic review: a best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses. Annu Rev Psychol. 2019;70:747-770. doi: 10.1146/annurev-psych-010418-102803 [DOI] [PubMed] [Google Scholar]
3.Nosek BA, Hardwicke TE, Moshontz H, et al. Replicability, robustness, and reproducibility in psychological science. Annu Rev Psychol. 2022;73:719-748. doi: 10.1146/annurev-psych-020821-114157 [DOI] [PubMed] [Google Scholar]
4.Camerer CF, Dreber A, Forsell E, et al. Evaluating replicability of laboratory experiments in economics. Science. 2016;351(6280):1433-1436. doi: 10.1126/science.aaf0918 [DOI] [PubMed] [Google Scholar]
5.Liu L, Jones BF, Uzzi B, Wang D. Data, measurement and empirical methods in the science of science. Nat Hum Behav. 2023;7(7):1046-1058. doi: 10.1038/s41562-023-01562-4 [DOI] [PubMed] [Google Scholar]
6.Camerer CF, Dreber A, Holzmeister F, et al. Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nat Hum Behav. 2018;2(9):637-644. doi: 10.1038/s41562-018-0399-z [DOI] [PubMed] [Google Scholar]
7.Jaljuli I, Kafkafi N, Giladi E, et al. A multi-lab experimental assessment reveals that replicability can be improved by using empirical estimates of genotype-by-lab interaction. PLoS Biol. 2023;21(5):e3002082. doi: 10.1371/journal.pbio.3002082 [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Kafkafi N, Agassi J, Chesler EJ, et al. Reproducibility and replicability of rodent phenotyping in preclinical studies. Neurosci Biobehav Rev. 2018;87:218-232. doi: 10.1016/j.neubiorev.2018.01.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
9.Wang SV, Sreedhara SK, Schneeweiss S; REPEAT Initiative . Reproducibility of real-world evidence studies using clinical practice data to inform regulatory and coverage decisions. Nat Commun. 2022;13(1):5126. doi: 10.1038/s41467-022-32310-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Nguyen PY, McKenzie JE, Alqaidoom Z, Hamilton DG, Moher D, Page MJ. Reproducibility of meta-analytic results in systematic reviews of interventions: meta-research study. BMJ Med. 2025;4(1):e002024. doi: 10.1136/bmjmed-2025-002024 [DOI] [PMC free article] [PubMed] [Google Scholar]
11.Rethlefsen ML, Brigham TJ, Price C, et al. Systematic review search strategies are poorly reported and not reproducible: a cross-sectional metaresearch study. J Clin Epidemiol. 2024;166:111229. doi: 10.1016/j.jclinepi.2023.111229 [DOI] [PubMed] [Google Scholar]
12.Bonetti AF, Tonin FS, Lucchetta RC, Pontarolo R, Fernandez-Llimos F. Methodological standards for conducting and reporting meta-analyses: ensuring the replicability of meta-analyses of pharmacist-led medication review. Res Social Adm Pharm. 2022;18(2):2259-2268. doi: 10.1016/j.sapharm.2021.06.002 [DOI] [PubMed] [Google Scholar]
13.Hamilton DG, McKenzie JE, Nguyen PY, et al. Evaluation of the replicability of systematic reviews with meta-analyses of the effects of health interventions. Res Synth Methods. 2026;17(3):538-556. doi: 10.1017/rsm.2025.10064 [DOI] [PMC free article] [PubMed] [Google Scholar]
14.Ford AC, Guyatt GH, Talley NJ, Moayyedi P. Errors in the conduct of systematic reviews of pharmacological interventions for irritable bowel syndrome. Am J Gastroenterol. 2010;105(2):280-288. doi: 10.1038/ajg.2009.658 [DOI] [PubMed] [Google Scholar]
15.Xu C, Yu T, Furuya-Kanamori L, et al. Validity of data extraction in evidence synthesis practice of adverse events: reproducibility study. BMJ. 2022;377:e069155. doi: 10.1136/bmj-2021-069155 [DOI] [PMC free article] [PubMed] [Google Scholar]
16.Gøtzsche PC, Hróbjartsson A, Maric K, Tendal B. Data extraction errors in meta-analyses that use standardized mean differences. JAMA. 2007;298(4):430-437. doi: 10.1001/jama.298.4.430 [DOI] [PubMed] [Google Scholar]
17.Xu C, Doi SAR, Zhou X, Lin L, Furuya-Kanamori L, Tao F. Data reproducibility issues and their potential impact on conclusions from evidence syntheses of randomized controlled trials in sleep medicine. Sleep Med Rev. 2022;66:101708. doi: 10.1016/j.smrv.2022.101708 [DOI] [PubMed] [Google Scholar]
18.Gates M, Gates A, Pieper D, et al. Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement. BMJ. 2022;378:e070849. doi: 10.1136/bmj-2022-070849 [DOI] [PMC free article] [PubMed] [Google Scholar]
19.Moseley AM, Elkins MR, Herbert RD, Maher CG, Sherrington C. Cochrane reviews used more rigorous methods than non-Cochrane reviews: survey of systematic reviews in physiotherapy. J Clin Epidemiol. 2009;62(10):1021-1030. doi: 10.1016/j.jclinepi.2008.09.018 [DOI] [PubMed] [Google Scholar]
20.Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210. doi: 10.1186/s13643-016-0384-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
21.Doi SA, Furuya-Kanamori L, Xu C, et al. The odds ratio is “portable” across baseline risk but not the relative risk: time to do away with the log link in binomial regression. J Clin Epidemiol. 2022;142:288-293. doi: 10.1016/j.jclinepi.2021.08.003 [DOI] [PubMed] [Google Scholar]
22.Brazzelli M, Sandercock PA, Chappell FM, et al. Magnetic resonance imaging versus computed tomography for detection of acute vascular lesions in patients presenting with stroke symptoms. Cochrane Database Syst Rev. 2009;(4):CD007424. doi: 10.1002/14651858.CD007424.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
23.Virgili G, Menchini F, Casazza G, et al. Optical coherence tomography (OCT) for detection of macular oedema in patients with diabetic retinopathy. Cochrane Database Syst Rev. 2015;1(1):CD008081. doi: 10.1002/14651858.CD008081.pub3 [DOI] [PMC free article] [PubMed] [Google Scholar]
24.Wang LW, Fahim MA, Hayen A, et al. Cardiac testing for coronary artery disease in potential kidney transplant recipients. Cochrane Database Syst Rev. 2011;(12):CD008691. doi: 10.1002/14651858.CD008691.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
25.Michelessi M, Lucenteforte E, Oddone F, et al. Optic nerve head and fibre layer imaging for diagnosing glaucoma. Cochrane Database Syst Rev. 2015;(11):CD008803. [DOI] [PMC free article] [PubMed] [Google Scholar]
26.Lenza M, Buchbinder R, Takwoingi Y, Johnston RV, Hanchard NC, Faloppa F. Magnetic resonance imaging, magnetic resonance arthrography and ultrasonography for assessing rotator cuff tears in people with shoulder pain for whom surgery is being considered. Cochrane Database Syst Rev. 2013;(9):CD009020. doi: 10.1002/14651858.CD009020.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
27.Josephson CB, White PM, Krishan A, Al-Shahi Salman R. Computed tomography angiography or magnetic resonance angiography for detection of intracranial vascular malformations in patients with intracerebral haemorrhage. Cochrane Database Syst Rev. 2014;(9):CD009372. doi: 10.1002/14651858.CD009372.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
28.Schmidt-Hansen M, Baldwin DR, Hasler E, Zamora J, Abraira V, Roqué I Figuls M. PET-CT for assessing mediastinal lymph node involvement in patients with suspected resectable non-small cell lung cancer. Cochrane Database Syst Rev. 2014;(11):CD009519. doi: 10.1002/14651858.CD009519.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
29.Nisenblat V, Bossuyt PM, Farquhar C, Johnson N, Hull ML. Imaging modalities for the non-invasive diagnosis of endometriosis. Cochrane Database Syst Rev. 2016;2(2):CD009591. doi: 10.1002/14651858.CD009591.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
30.Mocellin S, Pasquali S. Diagnostic accuracy of endoscopic ultrasonography (EUS) for the preoperative locoregional staging of primary gastric cancer. Cochrane Database Syst Rev. 2015;(2):CD009944. doi: 10.1002/14651858.CD009944.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
31.Rud B, Vejborg TS, Rappeport ED, Reitsma JB, Wille-Jørgensen P. Computed tomography for diagnosis of acute appendicitis in adults. Cochrane Database Syst Rev. 2019;(11):CD009977. doi: 10.1002/14651858.CD009977.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
32.Mallee WH, Wang J, Poolman RW, et al. Computed tomography versus magnetic resonance imaging versus bone scintigraphy for clinically suspected scaphoid fractures in patients with negative plain radiographs. Cochrane Database Syst Rev. 2015;(6):CD010023. doi: 10.1002/14651858.CD010023.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
33.Best LM, Rawji V, Pereira SP, Davidson BR, Gurusamy KS. Imaging modalities for characterising focal pancreatic lesions. Cochrane Database Syst Rev. 2017;4(4):CD010213. doi: 10.1002/14651858.CD010213.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
34.Abraha I, Luchetta ML, De Florio R, et al. Ultrasonography for endoleak detection after endoluminal abdominal aortic aneurysm repair. Cochrane Database Syst Rev. 2017;6(6):CD010296. doi: 10.1002/14651858.CD010296.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
35.Gurusamy KS, Giljaca V, Takwoingi Y, et al. Endoscopic retrograde cholangiopancreatography versus intraoperative cholangiography for diagnosis of common bile duct stones. Cochrane Database Syst Rev. 2015;(2):CD010339. doi: 10.1002/14651858.CD010339.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
36.Zhang S, Smailagic N, Hyde C, et al. (11)C-PIB-PET for the early diagnosis of Alzheimer’s disease dementia and other dementias in people with mild cognitive impairment (MCI). Cochrane Database Syst Rev. 2014;(7):CD010386. [DOI] [PMC free article] [PubMed] [Google Scholar]
37.Pavlov CS, Casazza G, Nikolova D, et al. Transient elastography for diagnosis of stages of hepatic fibrosis and cirrhosis in people with alcoholic liver disease. Cochrane Database Syst Rev. 2015;1(1):CD010542. doi: 10.1002/14651858.CD010542.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
38.Smailagic N, Vacante M, Hyde C, Martin S, Ukoumunne O, Sachpekidis C. ¹⁸F-FDG PET for the early diagnosis of Alzheimer’s disease dementia and other dementias in people with mild cognitive impairment (MCI). Cochrane Database Syst Rev. 2015;1(1):CD010632. doi: 10.1002/14651858.CD010632.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
39.Shaikh N, Spingarn RB, Hum SW. Dimercaptosuccinic acid scan or ultrasound in screening for vesicoureteral reflux among children with urinary tract infections. Cochrane Database Syst Rev. 2016;7(7):CD010657. doi: 10.1002/14651858.CD010657.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
40.Mattioni A, Cenciarelli S, Eusebi P, et al. Transcranial Doppler sonography for detecting stenosis or occlusion of intracranial arteries in people with acute ischaemic stroke. Cochrane Database Syst Rev. 2020;2(2):CD010722. doi: 10.1002/14651858.CD010722.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
41.Van’t Hoog A, Viney K, Biermann O, Yang B, Leeflang MM, Langendam MW. Symptom- and chest-radiography screening for active pulmonary tuberculosis in HIV-negative adults and adults with unknown HIV status. Cochrane Database Syst Rev. 2022;3(3):CD010890. [DOI] [PMC free article] [PubMed] [Google Scholar]
42.van Mens TE, Scheres LJ, de Jong PG, Leeflang MM, Nijkeuter M, Middeldorp S. Imaging for the exclusion of pulmonary embolism in pregnancy. Cochrane Database Syst Rev. 2017;1(1):CD011053. doi: 10.1002/14651858.CD011053.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
43.Nieuwenhuis LL, Hermans FJ, Bij de Vaate AJM, et al. Three-dimensional saline infusion sonography compared to two-dimensional saline infusion sonography for the diagnosis of focal intracavitary lesions. Cochrane Database Syst Rev. 2017;5(5):CD011126. doi: 10.1002/14651858.CD011126.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
44.van Gruting IM, Stankiewicz A, Thakar R, Santoro GA, IntHout J, Sultan AH. Imaging modalities for the detection of posterior pelvic floor disorders in women with obstructed defaecation syndrome. Cochrane Database Syst Rev. 2021;9(9):CD011482. [DOI] [PMC free article] [PubMed] [Google Scholar]
45.Giljaca V, Gurusamy KS, Takwoingi Y, et al. Endoscopic ultrasound versus magnetic resonance cholangiopancreatography for common bile duct stones. Cochrane Database Syst Rev. 2015;(2):CD011549. doi: 10.1002/14651858.CD011549 [DOI] [PMC free article] [PubMed] [Google Scholar]
46.Abrigo JM, Fountain DM, Provenzale JM, et al. Magnetic resonance perfusion for differentiating low-grade from high-grade gliomas at first presentation. Cochrane Database Syst Rev. 2018;1(1):CD011551. doi: 10.1002/14651858.CD011551.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
47.Dinnes J, Deeks JJ, Chuchu N, et al. ; Cochrane Skin Cancer Diagnostic Test Accuracy Group . Dermoscopy, with and without visual inspection, for diagnosing melanoma in adults. Cochrane Database Syst Rev. 2018;12(12):CD011902. doi: 10.1002/14651858.CD011902.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
48.D’Souza N, Hicks G, Beable R, Higginson A, Rud B. Magnetic resonance imaging (MRI) for diagnosis of acute appendicitis. Cochrane Database Syst Rev. 2021;12(12):CD012028. [DOI] [PMC free article] [PubMed] [Google Scholar]
49.Wennmacker SZ, Lamberts MP, Di Martino M, Drenth JP, Gurusamy KS, van Laarhoven CJ. Transabdominal ultrasound and endoscopic ultrasound for diagnosis of gallbladder polyps. Cochrane Database Syst Rev. 2018;8(8):CD012233. doi: 10.1002/14651858.CD012233.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
50.Jahanfar S, Ho JJ, Jaafar SH, et al. Ultrasound for diagnosis of birth weight discordance in twin pregnancies. Cochrane Database Syst Rev. 2021;3(3):CD012553. [DOI] [PMC free article] [PubMed] [Google Scholar]
51.Alldred SK, Takwoingi Y, Guo B, et al. First trimester ultrasound tests alone or in combination with first trimester serum tests for Down’s syndrome screening. Cochrane Database Syst Rev. 2017;3(3):CD012600. doi: 10.1002/14651858.CD012600 [DOI] [PMC free article] [PubMed] [Google Scholar]
52.Drost FH, Osses DF, Nieboer D, et al. Prostate MRI, with or without MRI-targeted biopsy, and systematic biopsy for detecting prostate cancer. Cochrane Database Syst Rev. 2019;4(4):CD012663. doi: 10.1002/14651858.CD012663.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
53.Stengel D, Leisterer J, Ferrada P, Ekkernkamp A, Mutze S, Hoenning A. Point-of-care ultrasonography for diagnosing thoracoabdominal injuries in patients with blunt trauma. Cochrane Database Syst Rev. 2018;12(12):CD012669. doi: 10.1002/14651858.CD012669.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
54.Van Hoving DJ, Griesel R, Meintjes G, Takwoingi Y, Maartens G, Ochodo EA. Abdominal ultrasound for diagnosing abdominal tuberculosis or disseminated tuberculosis with abdominal involvement in HIV-positive individuals. Cochrane Database Syst Rev. 2019;9(9):CD012777. doi: 10.1002/14651858.CD012777.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
55.Dinnes J, Ferrante di Ruffano L, Takwoingi Y, et al. ; Cochrane Skin Cancer Diagnostic Test Accuracy Group . Ultrasound, CT, MRI, or PET-CT for staging and re-staging of adults with cutaneous melanoma. Cochrane Database Syst Rev. 2019;7(7):CD012806. doi: 10.1002/14651858.CD012806.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
56.Tsujimoto Y, Kumasawa J, Shimizu S, et al. Doppler trans-thoracic echocardiography for detection of pulmonary hypertension in adults. Cochrane Database Syst Rev. 2022;5(5):CD012809. [DOI] [PMC free article] [PubMed] [Google Scholar]
57.Jindal A, Ctori I, Virgili G, Lucenteforte E, Lawrenson JG. Non-contact tests for identifying people at risk of primary angle closure glaucoma. Cochrane Database Syst Rev. 2020;5(5):CD012947. [DOI] [PMC free article] [PubMed] [Google Scholar]
58.Chan KK, Joo DA, McRae AD, et al. Chest ultrasonography versus supine chest radiography for diagnosis of pneumothorax in trauma patients in the emergency department. Cochrane Database Syst Rev. 2020;7(7):CD013031. [DOI] [PMC free article] [PubMed] [Google Scholar]
59.Cassola N, Baptista-Silva JC, Nakano LC, et al. Duplex ultrasound for diagnosing symptomatic carotid stenosis in the extracranial segments. Cochrane Database Syst Rev. 2022;7(7):CD013172. [DOI] [PMC free article] [PubMed] [Google Scholar]
60.Ferrante di Ruffano L, Takwoingi Y, Dinnes J, et al. ; Cochrane Skin Cancer Diagnostic Test Accuracy Group . Computer-assisted diagnosis techniques (dermoscopy and spectroscopy-based) for diagnosing skin cancer in adults. Cochrane Database Syst Rev. 2018;12(12):CD013186. doi: 10.1002/14651858.CD013186 [DOI] [PMC free article] [PubMed] [Google Scholar]
61.Dinnes J, Deeks JJ, Saleh D, et al. ; Cochrane Skin Cancer Diagnostic Test Accuracy Group . Reflectance confocal microscopy for diagnosing cutaneous melanoma in adults. Cochrane Database Syst Rev. 2018;12(12):CD013190. doi: 10.1002/14651858.CD013190 [DOI] [PMC free article] [PubMed] [Google Scholar]
62.Nadarevic T, Giljaca V, Colli A, et al. Computed tomography for the diagnosis of hepatocellular carcinoma in adults with chronic liver disease. Cochrane Database Syst Rev. 2021;10(10):CD013362. [DOI] [PMC free article] [PubMed] [Google Scholar]
63.Fraquelli M, Nadarevic T, Colli A, et al. Contrast-enhanced ultrasound for the diagnosis of hepatocellular carcinoma in adults with chronic liver disease. Cochrane Database Syst Rev. 2022;9(9):CD013483. [DOI] [PMC free article] [PubMed] [Google Scholar]
64.Ebrahimzadeh S, Islam N, Dawit H, et al. ; Cochrane COVID-19 Diagnostic Test Accuracy Group . Thoracic imaging tests for the diagnosis of COVID-19. Cochrane Database Syst Rev. 2022;5(5):CD013639. [DOI] [PMC free article] [PubMed] [Google Scholar]
65.Macey R, Walsh T, Riley P, et al. Transillumination and optical coherence tomography for the detection and diagnosis of enamel caries. Cochrane Database Syst Rev. 2021;1(1):CD013855. [DOI] [PMC free article] [PubMed] [Google Scholar]
66.Walsh T, Macey R, Riley P, et al. Imaging modalities to inform the detection and diagnosis of early caries. Cochrane Database Syst Rev. 2021;3(3):CD014545. [DOI] [PMC free article] [PubMed] [Google Scholar]
67.Nadarevic T, Colli A, Giljaca V, et al. Magnetic resonance imaging for the diagnosis of hepatocellular carcinoma in adults with chronic liver disease. Cochrane Database Syst Rev. 2022;5(5):CD014798. [DOI] [PMC free article] [PubMed] [Google Scholar]
68.Botet JF, Lightdale CJ, Zauber AG, et al. Preoperative staging of gastric cancer: comparison of endoscopic US and dynamic CT. Radiology. 1991;181(2):426-432. doi: 10.1148/radiology.181.2.1924784 [DOI] [PubMed] [Google Scholar]
69.Fang Y, Pan YZ, Li M, Qiao RH, Cai Y. Diagnostic capability of Fourier-Domain optical coherence tomography in early primary open angle glaucoma. Chin Med J (Engl). 2010;123(15):2045-2050. [PubMed] [Google Scholar]
70.Boraschi P, Gigoni R, Braccini G, Lamacchia M, Rossi M, Falaschi F. Detection of common bile duct stones before laparoscopic cholecystectomy. Evaluation with MR cholangiography. Acta Radiol. 2002;43(6):593-598. doi: 10.1034/j.1600-0455.2002.430610.x [DOI] [PubMed] [Google Scholar]
71.Cai Q, Serrano R, Kalyanasundaram A, Shirani J. A preoperative echocardiographic predictive model for assessment of cardiovascular outcome after renal transplantation. J Am Soc Echocardiogr. 2010;23(5):560-566. doi: 10.1016/j.echo.2010.03.008 [DOI] [PubMed] [Google Scholar]
72.Bossuyt PM, Reitsma JB, Bruns DE, et al. ; STARD Group . STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015;351:h5527. doi: 10.1136/bmj.h5527 [DOI] [PMC free article] [PubMed] [Google Scholar]
73.Kellerhuis BE, Jenniskens K, Schuit E, Hooft L, Moons KGM, Reitsma JB. Drivers of bias in diagnostic test accuracy estimates when using expert panels as a reference standard: a simulation study. BMC Med Res Methodol. 2025;25(1):106. doi: 10.1186/s12874-025-02557-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
74.Glas AS, Lijmer JG, Prins MH, Bonsel GJ, Bossuyt PM. The diagnostic odds ratio: a single indicator of test performance. J Clin Epidemiol. 2003;56(11):1129-1135. doi: 10.1016/S0895-4356(03)00177-X [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials