Successful Publication of Systematic Review and Meta-Analysis of Studies Evaluating Diagnostic Test Accuracy

As a research synthesis methodology, systematic review and meta-analysis generates an evidence-based systematic summary of a specific topic by objectively evaluating all relevant primary studies available in the literature. Meta-analysis is a part of the methodology and uses statistical methods to integrate the results of primary research studies. Recently, systematic review and meta-analysis appears to be increasingly used in the field of diagnostic test accuracy (DTA) studies (1). The methods used for systematic review and meta-analysis of DTA studies somewhat differ from those used for therapeutic/interventional studies and are presumably less well known to clinical researchers as the contents are still evolving. Successful publication of a systematic review and meta-analysis of DTA studies would require a sound understanding of the process and methods involved. Through a 2-part conceptual review by Kim et al. (2) and Lee et al. (3) published in the November-December 2015 issue of the Korean Journal of Radiology, we would like to introduce our readers to the currently recommended methodologies used to perform systematic review and meta-analysis of DTA studies.

Part I (2) of the 2-part review explains the overall process and aims to provide a practical step-wise guide on how to conduct, report, and critically appraise a systematic review and meta-analysis of DTA studies, as summarized below.

Step 1: Defining the research questions and developing inclusion/exclusion criteria. The research questions should be specified clearly before beginning the systematic review and the inclusion/exclusion criteria for literature search should be made accordingly. The structured patient (or population)/intervention/comparator/outcomes (PICO) framework is recommended although it may not apply seamlessly to some DTA studies due to their differences in design from therapeutic/interventional studies.

Step 2: Systematic search and selection of the literature. The literature search should include multiple resources extensively and should be conducted in MEDLINE and EMBASE at minimum. Specific reasons for inclusion and exclusion of articles and the corresponding article numbers should be clearly recorded. The literature search also needs to be up-to-date.

Step 3: Assessing the quality of studies. The Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) (4) is currently recommended for the articles to be included in a systematic review of DTA studies.

Step 4: Data extraction and management. Data should be extracted from individual articles using a standardized form in order to ensure that all relevant data are collected, to minimize any errors, and allow evaluation of the accuracy of data.

Step 5: Analysis and data synthesis. Data analysis and synthesis involve tabulation of study characteristics, graphical plotting of the results, and use of various statistical methods. For DTA studies, it is important to carefully check and explore study heterogeneity (the variability across studies), particularly the threshold effect, before performing any meta-analytic synthesis. Unlike therapeutic/interventional studies, Higgins' I² statistic or Cochran's Q alone may not be informative enough. Publication bias should be assessed using an appropriate method.

Step 6: Presentation of results for publication. Reporting of a systematic review and meta-analysis of DTA studies should follow the Preferred Reporting Items of Systematic Reviews and Meta-Analyses (PRISMA) guidelines (5).

Part II (3) further elaborates on the statistical methods used for meta-analysis of DTA studies, providing non-mathematic conceptual explanations and highlighting methodological differences from the methods used for therapeutic/interventional meta-analysis. A meta-analysis of DTA studies jointly analyzes a pair of outcome measures, i.e., sensitivity and specificity, which are generally inversely correlated and affected by the threshold effect. Also, DTA studies typically show large study heterogeneity. In order to address these characteristics of DTA studies appropriately, random-effects hierarchical models are currently recommended for meta-analysis of DTA studies as endorsed by authoritative bodies such as the Diagnostic Test Accuracy Working Group of the Cochrane Collaboration or the Agency for Healthcare Research and Quality. There are currently 2 analytical models available for the hierarchical modeling: the bivariate model and the hierarchical summary receiver operating characteristic (HSROC) model. The bivariate model is preferred for estimating point summary values of sensitivity and specificity, as well as for evaluating how their expected values may vary with study level covariates; whereas, the HSROC model is favored for estimating the SROC curve for assessing test accuracy and determining how the curve's position and shape may vary with study level covariates.

Korean Journal of Radiology published systematic review and meta-analysis articles only occasionally in the past (6,7,8,9) and encourages further publication of appropriately-conducted and written systematic review and meta-analysis in the journal. The 2 review articles (2,3) could be useful methodological references for those who wish to perform a systematic review and meta-analysis of DTA studies. Korean Journal of Radiology will also refer to the principles explained in the review articles as well as in this editorial when evaluating the submitted manuscripts.