Skip to main content
PMC home page
BMJ Open logoLink to BMJ Open
. 2017 Dec 26;7(12):e018175. doi: 10.1136/bmjopen-2017-018175

Comparing the diagnostic accuracy of five common tumour biomarkers and CA19-9 for pancreatic cancer: a protocol for a network meta-analysis of diagnostic test accuracy

Long Ge 1, Bei Pan 2, Fujian Song 3, Jichun Ma 4, Dena Zeraatkar 5, Jianguo Zhou 6, Jinhui Tian 7,8
PMCID: PMC5770961  PMID: 29282264

Abstract

Introduction

Surgical resection is the only curative treatment for patients with resectable pancreatic cancer. Unfortunately, 80%–85% of patients present with locally advanced or metastatic unresectable pancreatic cancer at the time of diagnosis. Detection of pancreatic cancer at early stages remains a great challenge due to lack of accurate detection tests. Recommendations in existing clinical practice guidelines on early diagnosis of pancreatic cancer are inconsistent and based on limited evidence. Most of them endorse measuring serum CA19-9 as a complementary test, but also state that it is not recommended for diagnosing early pancreatic cancer. There are currently no other tumour-specific markers recommended for diagnosing early pancreatic cancer. This study aims to evaluate and compare the accuracy of five common tumour biomarkers (CA242, carcino-embryonic antigen (CEA)), CA125, microRNAs and K-ras gene mutation) and CA19-9 and their combinations for diagnosing pancreatic cancer using network meta-analysis method, and to rank these tests using a superiority index.

Methods and analysis

PubMed, EMBASE and the Cochrane Central Register of Controlled Trials will be searched from inception to April 2017. The search will include the above-mentioned tumour biomarkers for diagnosing pancreatic cancer, including CA19-9. The risk of bias for each study will be independently assessed as low, moderate or high using criteria adapted from the Quality Assessment of Diagnostic Accuracy Studies 2. Network meta-analysis will be performed using STATA V.12.0 and R software V.3.4.1. The competing diagnostic tests will be ranked by a superiority index.

Ethics and dissemination

Ethical approval and patient consent are not required since this study is a network meta-analysis based on published studies. The results of this network meta-analysis will be submitted to a peer-reviewed journal for publication.

PROSPERO registration number

CRD42017064627.

Keywords: tumor biomarkers, ca19-9, pancreatic cancer, network meta-analysis, sensitivity and specificity, protocol

Strengths and limitations of this study.

  • To the best of our knowledge, this will be the first diagnostic network meta-analysis comparing different tumour biomarkers combined with or without CA19-9 for pancreatic cancer.

  • Current network meta-analysis will compare simultaneously the accuracy of multiple tests within and between studies and rank the diagnostic tests using a superiority index.

  • Our results of this network meta-analysis will help clinicians and patients to select appropriate diagnostic test for pancreatic cancer.

  • Our results will be limited by both the quantity and quality of the studies available for review.

  • Our subgroup analyses of network meta-analysis will be based on the reporting of baseline characteristics of included original studies; some expected characteristics of patients may not be reported adequately.

Introduction

Pancreatic cancer is the fourth leading cause of cancer death in the USA.1 The American Cancer Society estimates that there will be 53 670 newly diagnosed pancreatic cancers in the USA in 2017, and that 43 090 will die from the disease.1 Despite decades of effort in detection and management of pancreatic cancer, the 5-year survival rate is only about 4%.2 The number of patients with pancreatic cancer is currently increasing year by year and is predicted to become the second leading cause of cancer death in the USA by 2030.3

Systemic chemotherapy has been demonstrated to prolong survival in patients with resectable or metastatic pancreatic cancer,4–6 although surgical resection is the only curative treatment.2 Unfortunately, 80%–85% of patients present with locally advanced or metastatic unresectable pancreatic cancer at the time of diagnosis.2 Detection of pancreatic cancer at early stages remains a great challenge due to lack of specific detection tests.7

Many investigations have been conducted to find the appropriate serum and imaging biomarkers to help early detection of pancreatic cancer.8 Currently, several biomarkers (such as carcinoembryonic antigen, CA19-9, CA125, microRNAs, etc) have been proposed for pancreatic cancer detection, although the clinical applicability of these tests remains unclear.9 The recommendations in existing clinical practice guidelines on early diagnosis of pancreatic cancer are inconsistent and based on limited evidence.10 Most of them endorse measuring serum CA19-9 as a complementary test, but also stated that it is not useful for diagnosing early pancreatic cancer or for screening.10 There are currently no other tumour-specific markers recommended for diagnosing early pancreatic cancer.10

Tumour markers, imaging approaches or combination of the two might be the future of pancreatic cancer screening.11 A combination of serum CA19-9 and CEA has been reported to increase specificity to 84% compared with CA19-9 alone, and CA19-9 combined with CA125 improved sensitivity.12 Meta-analyses have also shown that the combined tests of CA19-9 plus CA242, or CA19-9 plus K-ras gene mutation or endoscopic retrograde cholangiopancreatography plus endoscopic ultrasonography could be of better diagnostic value than individual tests.13–16 Moreover, a combination of microRNAs and CA19-9 was more accurate, especially in early pancreatic cancer screening.17 18 However, it is still unclear which individual test or combined test is the best for diagnosing pancreatic cancer based on currently available studies.

Network meta-analysis has been used to extend conventional meta-analyses on multiple treatments (ie, three or more) for a given condition.19 An attractive feature of network meta-analysis is the ranking of interventions using rank probabilities and rankograms. Similarly, there are often multiple candidate tests for diagnosing a particular disease outcome in a diagnostic test accuracy study.20 In order to present an overall picture, network meta-analysis (mainly refers to indirect comparison) has been proposed by some researchers to simultaneously compare the accuracy of multiple tests within and between studies and rank the diagnostic tests using diagnostic OR (DOR) and a superiority index.20–26

This study aims to evaluate and compare the accuracy of five common tumour biomarkers (CA242, CEA, CA125, microRNAs and K-ras gene mutation) and CA19-9 and their combinations for diagnosing pancreatic cancer using network meta-analysis method, and to rank these tests using superiority index.

Methods

Design and registration

We will conduct a network meta-analysis of diagnostic test accuracy. We have registered the protocol on the international prospective register of systematic review (PROSPERO).27 We will follow the Preferred Reporting Items for Systematic Reviews and Meta-Analyses28 statements for reporting our systematic review.

Information sources

PubMed, EMBASE and the Cochrane Central Register of Controlled Trials will be searched from inception to April 2017. The search strategies will be developed by GL and TJH who are experienced information specialists. The references of relevant systematic reviews/meta-analyses will be searched to identify additional potential studies.

Search strategy

The search terms will include: pancreatic neoplasm, pancreas neoplasms, pancreas neoplasm, pancreas cancers, pancreas cancer, pancreatic cancer, pancreatic cancers, CA199, carbohydrate antigen 199, sensitivity and specificity. Full details of the search strategies can be found in online supplementary appendix 1.

Supplementary file 1

bmjopen-2017-018175supp001.docx (13.4KB, docx)

Eligibility criteria

Eligibility criteria are as follows: (1) index tests include either CA19-9, CA242, CEA, CA125, microRNAs and K-ras gene mutation, or combinations thereof; (2) at least two index tests per study, one of them being CA19-9; (3) report or provide sufficient information to allow us to calculate the true positive (TP), false positive (FP), true negative (TN) and false negative (FN) values; (4) case–control, cross-sectional or cohort designs; there will be no limitations on language of publication, year of publication, publication status or stage of pancreatic cancer.

Study selection and data extraction

Initial search records will be imported into ENDNOTE X6 literature management software, then the titles and abstracts of records will be screened to identify potential trials according to eligibility criteria. Next, full-text versions of all potentially relevant trials will be obtained and reviewed to ensure eligibility.

A standard data extraction form will be created using Microsoft Excel 2013 (Microsoft, Redmond, Washington, USA, www.microsoft.com) to collect data of interest, which include eligible studies characteristics (eg, name of first author, year of publication, country in which the study was conducted, gold standard, index tests), patients characteristics (male, mean age, sample, method, cut-off level, risk factors of pancreatic cancer) and outcomes (TP, FP, FN, TN).

Study selection and data extraction will be performed by one reviewer (LG), and will be checked by other reviewers (BP, JT). Any conflicts will be resolved by discussion.

Quality evaluation

Two reviewers will independently assess the risk of bias for each study as low, moderate or high using criteria adapted from Quality Assessment of Diagnostic Accuracy Studies 2,29 and conflicts will be resolved by discussion.

Geometry of the network

We will draw network plots using R software V.3.4.1. In network plots, the size of the nodes is proportional to the number of studies evaluating a test, and thickness of the lines between the nodes is proportional to the number of direct comparisons between tests. The network is connected because there exists at least one study evaluating a given test together with at least one of the other remaining tests.20 A loop connecting three tests indicates that there is at least one study comparing the three targeted tests simultaneously.

Network meta-analysis

Pairwise meta-analyses

We will perform pairwise meta-analyses for pooled sensitivity (SEN), specificity (SPE), positive likelihood ratio, negative likelihood ratio, DOR and area under the summary receiver operating characteristic curve using bivariate mixed-effects regression modelling with STATA V.12.0 (Stata). The between-study variance will be calculated var logitSEN and logitSPE.30 31 The proportion of heterogeneity due to the threshold effect among the included studies will be calculated by the squared correlation coefficient estimated from the between-study covariance variable in the bivariate model.32 The heterogeneity between each study will be estimated using the Q value and the inconsistency index (I2 test, and the values of 25%, 50% and 75% for the I2 will be indicative of low, moderate and high statistical heterogeneity, respectively.33

Subgroup analyses for each biomarker will be planned on the basis of the country in which the study was conducted, stage of pancreatic cancer, cut-off level, risk factors of pancreatic cancer and risk of bias.

The Deek’s funnel plot will be applied to evaluate the potential publication bias where there are more than 10 studies available for an index test.34

Indirect comparisons between competing diagnostic tests

Using CA19-9 as common reference test, we will calculate relative diagnostic outcomes between index tests by analysis of variance model in R software V.3.4.1,20 including relative SEN, relative SPE and relative DOR.

Ranking of competing diagnostic tests

Ranking of interventions is an attractive feature of network meta-analysis. Currently, it is still challenging to rank competing diagnostic tests. Some researchers consider DOR as a indicator of ranking of competing diagnostic tests25; however, the measure cannot distinguish between tests with high sensitivity but low specificity or vice versa. Alternatively, the superiority index introduced by Deutsch et al26 gives more weight to tests performing relatively well on both diagnostic accuracy measures and less weight on tests performing poorly on both diagnostic measures or tests performing better on one measure but poorly on the other.20 The superiority index ranges from 0 to ∞, and tends towards ∞ and 0 as the number of tests to which the target test is superior and inferior increases, respectively, and the superiority index tending to one indicates that the tests are equal.20

Ethics and dissemination

Ethical issues

Ethical approval and patient consent are not required since this is a network meta-analysis based on published studies.

Publication plan

This protocol has been registered on the international prospective register of systematic review (PROSPERO).27 The results of this network meta-analysis will be submitted to a peer-reviewed journal for publication.

Supplementary Material

Reviewer comments
bmjopen-2017-018175.reviewer_comments.pdf (135.1KB, pdf)
Author's manuscript
bmjopen-2017-018175.draft_revisions.pdf (886.6KB, pdf)

Acknowledgments

We are grateful for the helpful reviewer comments on this paper.

Footnotes

Contributors: LG, BP and JT: plan and design the research. LG, BP, JM and JZ tested the feasibility of the study. FS, DZ and JT provided methodological advice, polished and revised the manuscript. LG and JT wrote the manuscript; all authors approved the final version of the manuscript.

Competing interests: None declared.

Provenance and peer review: Not commissioned; externally peer reviewed.

References

  • 1.Siegel RL, Miller KD, Jemal A, et al. . Cancer statistics, 2017. CA Cancer J Clin 2017;67:7–30. 10.3322/caac.21387 [DOI] [PubMed] [Google Scholar]
  • 2.Vincent A, Herman J, Schulick R, et al. . Pancreatic cancer. Lancet 2011;378:607–20. 10.1016/S0140-6736(10)62307-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Rahib L, Smith BD, Aizenberg R, et al. . Projecting cancer incidence and deaths to 2030: the unexpected burden of thyroid, liver, and pancreas cancers in the United States. Cancer Res 2014;74:2913–21. 10.1158/0008-5472.CAN-14-0155 [DOI] [PubMed] [Google Scholar]
  • 4.Oettle H, Neuhaus P, Hochhaus A, et al. . Adjuvant chemotherapy with gemcitabine and long-term outcomes among patients with resected pancreatic cancer: the CONKO-001 randomized trial. JAMA 2013;310:1473–81. 10.1001/jama.2013.279201 [DOI] [PubMed] [Google Scholar]
  • 5.Conroy T, Desseigne F, Ychou M, et al. . FOLFIRINOX versus gemcitabine for metastatic pancreatic cancer. N Engl J Med 2011;364:1817–25. 10.1056/NEJMoa1011923 [DOI] [PubMed] [Google Scholar]
  • 6.Von Hoff DD, Ervin T, Arena FP, et al. . Increased survival in pancreatic cancer with nab-paclitaxel plus gemcitabine. N Engl J Med 2013;369:1691–703. 10.1056/NEJMoa1304369 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Liu R, Chen X, Du Y, et al. . Serum microRNA expression profile as a biomarker in the diagnosis and prognosis of pancreatic cancer. Clin Chem 2012;58:610–8. 10.1373/clinchem.2011.172767 [DOI] [PubMed] [Google Scholar]
  • 8.Chang JC, Kundranda M. Novel diagnostic and predictive biomarkers in pancreatic adenocarcinoma. Int J Mol Sci 2017;18:667 10.3390/ijms18030667 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Herreros-Villanueva M, Gironella M, Castells A, et al. . Molecular markers in pancreatic cancer diagnosis. Clin Chim Acta 2013;418:22–9. 10.1016/j.cca.2012.12.025 [DOI] [PubMed] [Google Scholar]
  • 10.Takaori K, Bassi C, Biankin A, et al. . International Association of Pancreatology (IAP)/European Pancreatic Club (EPC) consensus review of guidelines for the treatment of pancreatic cancer. Pancreatology 2016;16:14–27. 10.1016/j.pan.2015.10.013 [DOI] [PubMed] [Google Scholar]
  • 11.Gemmel C, Eickhoff A, Helmstädter L, et al. . Pancreatic cancer screening: state of the art. Expert Rev Gastroenterol Hepatol 2009;3:89–96. 10.1586/17474124.3.1.89 [DOI] [PubMed] [Google Scholar]
  • 12.Kamisawa T, Wood LD, Itoi T, et al. . Pancreatic cancer. Lancet 2016;388:73–85. 10.1016/S0140-6736(16)00141-0 [DOI] [PubMed] [Google Scholar]
  • 13.Zhang Y, Yang J, Li H, et al. . Tumor markers CA19-9, CA242 and CEA in the diagnosis of pancreatic cancer: a meta-analysis. Int J Clin Exp Med 2015;8:11683–91. [PMC free article] [PubMed] [Google Scholar]
  • 14.Gu J, Wang D, Huang Y, et al. . Diagnostic value of combining CA 19-9 and K-ras gene mutation in pancreatic carcinoma: a meta-analysis. Int J Clin Exp Med 2014;7:3225–34. [PMC free article] [PubMed] [Google Scholar]
  • 15.Li H, Hu Z, Chen J, et al. . Comparison of ERCP, EUS, and ERCP combined with EUS in diagnosing pancreatic neoplasms: a systematic review and meta-analysis. Tumour Biol 2014;35:8867–74. 10.1007/s13277-014-2154-z [DOI] [PubMed] [Google Scholar]
  • 16.Goonetilleke KS, Siriwardena AK. Systematic review of carbohydrate antigen (CA 19-9) as a biochemical marker in the diagnosis of pancreatic cancer. Eur J Surg Oncol 2007;33:266–70. 10.1016/j.ejso.2006.10.004 [DOI] [PubMed] [Google Scholar]
  • 17.Liu J, Gao J, Du Y, et al. . Combination of plasma microRNAs with serum CA19-9 for early detection of pancreatic cancer. Int J Cancer 2012;131:683–91. 10.1002/ijc.26422 [DOI] [PubMed] [Google Scholar]
  • 18.Schultz NA, Dehlendorff C, Jensen BV, et al. . MicroRNA biomarkers in whole blood for detection of pancreatic cancer. JAMA 2014;311:392–404. 10.1001/jama.2013.284664 [DOI] [PubMed] [Google Scholar]
  • 19.Li L, Catalá-López F, Alonso-Arroyo A, et al. . The global research collaboration of network meta-analysis: a social network analysis. PLoS One 2016;11:e0163239 10.1371/journal.pone.0163239 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Nyaga VN, Aerts M, Arbyn M. ANOVA model for network meta-analysis of diagnostic test accuracy data. Stat Methods Med Res 2016:096228021666918 10.1177/0962280216669182 [DOI] [PubMed] [Google Scholar]
  • 21.Lin YH, Lin HH, Liu HM, et al. . Diagnostic performance of CT and MRI on the detection of symptomatic intracranial dural arteriovenous fistula: a meta-analysis with indirect comparison. Neuroradiology 2016;58:753–63. 10.1007/s00234-016-1696-8 [DOI] [PubMed] [Google Scholar]
  • 22.Purkayastha S, Athanasiou T, Tekkis PP, et al. . Magnetic resonance colonography vs computed tomography colonography for the diagnosis of colorectal cancer: an indirect comparison. Colorectal Dis 2007;9:100–11. 10.1111/j.1463-1318.2006.01126.x [DOI] [PubMed] [Google Scholar]
  • 23.Weng WW, Dong MJ, Zhang J, et al. . A systematic review of MRI, scintigraphy, FDG-PET and PET/CT for diagnosis of multiple myeloma related bone disease – which is best? Asian Pac J Cancer Prev 2014;15:9879–84. 10.7314/APJCP.2014.15.22.9879 [DOI] [PubMed] [Google Scholar]
  • 24.Zhang JH, Ge L, Tian JH, et al. . Diagnostic value of PET/CT and CT for para-aortic metastatic lymph nodes in patients with cervical cancer: a meta-analysis and indirect comparison. J Lanzhou Univ 2015;41:34–42. [Google Scholar]
  • 25.Glas AS, Lijmer JG, Prins MH, et al. . The diagnostic odds ratio: a single indicator of test performance. J Clin Epidemiol 2003;56:1129–35. 10.1016/S0895-4356(03)00177-X [DOI] [PubMed] [Google Scholar]
  • 26.Deutsch R, Mindt MR, Xu R, et al. . Quantifying relative superiority among many binary-valued diagnostic tests in the presence of a gold standard. J Data Sci 2009;7:161–77. [Google Scholar]
  • 27.Ge L, Tian JH, Pan B, et al. . Common tumor biomarkers based-on CA199 in the diagnosis of pancreatic adenocarcinoma: a network meta-analysis of diagnostic test accuracy PROSPERO: CRD42017064627. 2017. http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42017064627
  • 28.Moher D, Liberati A, Tetzlaff J, et al. . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg 2010;8:336–41. 10.1016/j.ijsu.2010.02.007 [DOI] [PubMed] [Google Scholar]
  • 29.Whiting PF, Rutjes AW, Westwood ME, et al. . QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011;155:529–36. 10.7326/0003-4819-155-8-201110180-00009 [DOI] [PubMed] [Google Scholar]
  • 30.Reitsma JB, Glas AS, Rutjes AW, et al. . Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 2005;58:982–90. 10.1016/j.jclinepi.2005.02.022 [DOI] [PubMed] [Google Scholar]
  • 31.Higgins JP, Thompson SG, Deeks JJ, et al. . Measuring inconsistency in meta-analyses. BMJ 2003;327:557–60. 10.1136/bmj.327.7414.557 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Harbord RM, Deeks JJ, Egger M, et al. . A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics 2007;8:239–51. 10.1093/biostatistics/kxl004 [DOI] [PubMed] [Google Scholar]
  • 33.Dwamena BA. MIDAS: Statamodule for meta-analytical integration of diagnostic accuracy studies. 2017. http://econpapers.repec.org/software/bocbocode/s456880.htm (accessed 8 May 2017).
  • 34.Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol 2005;58:882–93. 10.1016/j.jclinepi.2005.01.016 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary file 1

bmjopen-2017-018175supp001.docx (13.4KB, docx)

Reviewer comments
bmjopen-2017-018175.reviewer_comments.pdf (135.1KB, pdf)
Author's manuscript
bmjopen-2017-018175.draft_revisions.pdf (886.6KB, pdf)

Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES