Abstract
Background:
Research reproducibility is vital for translation of epidemiologic findings. However, repeated studies of the same question may be undertaken without enhancing existing knowledge. To identify settings in which additional research is or is not warranted, we adapted research synthesis metrics to determine number of additional observational studies needed to change the inference from an existing meta-analysis.
Methods:
The fail-safe number (FSN) estimates number of additional studies of average weight and null effect needed to drive a statistically significant meta-analysis to null (P≥0.05). We used conditional power to determine number of additional studies of average weight and equivalent heterogeneity to achieve 80% power in an updated meta-analysis to detect the observed summary estimate as statistically significant. We applied these metrics to a curated set of 98 meta-analyses on biomarkers and cancer risk.
Results:
Both metrics were influenced by number of studies, heterogeneity, and summary estimate size in the existing meta-analysis. For the meta-analysis on H. pylori and gastric cancer with 15 studies (OR=2.29; 95% CI 1.71–3.05), FSN was 805 studies, supporting futility of further study. For the meta-analysis on dehydroepiandrosterone sulfate and prostate cancer with 7 studies (OR=1.29; 95% CI 0.99–1.69), 5 more studies would be needed for 80% power, suggesting further study could change inferences.
Conclusions:
Along with traditional assessments, these metrics could be used by stakeholders to decide whether additional studies addressing the same question are needed.
Impact:
Systematic application of these metrics could lead to more judicious use of resources and acceleration from discovery to population-health impact.
Keywords: fail-safe number, conditional power, meta-analyses, epidemiology, observational study, cohort, risk, biomarker, cancer
Introduction
Translation of cancer etiology, risk, prognosis, and prediction biomarkers into prevention and control strategies relies, in part, on the ability to reproduce associations. However, repetitive investigations of established biomarker-cancer associations that do not contribute meaningful additional information to the existing evidence base – e.g., fill remaining knowledge gaps, provide substantial clinical or public health support for an association, or have the potential to improve biological understanding – may be inefficient and a waste of resources (1–3).
To address these concerns, we adapted an application of existing clinical trial and research synthesis metrics – the fail-safe number (FSN) (4) and conditional power (5) – to determine whether or not further investigation of cancer relevant biomarkers may provide meaningful contribution to the existing evidence. In its original application, Rosenthal (6) introduced the FSN to quantify the impact of selectively unpublished research on the existing meta-analysis. The FSN indicates the number of unpublished studies with an average null effect (e.g., P≥0.05) needed to be included in an updated meta-analysis to drive a statistically significant summary estimate in the existing meta-analysis (e.g., P<0.05) to a statistically non-significant summary estimate (e.g., to P≥0.05) in the updated meta-analysis. We adapted the FSN for observational epidemiology studies to determine whether the inference from an existing meta-analysis for a statistically significant exposure-outcome association, will likely change to a null association with the addition of further research to update the meta-analysis. In its original application, conditional power was used to guide the design of clinical trials based on effect size and sample size of an existing trial or meta-analysis. In the context of observational epidemiology and assuming a statistically non-significant existing meta-analysis, we adapted conditional power calculation to determine the feasibility of conducting the necessary number of future studies with sufficient power to detect a significant association of a certain size in the updated meta-analysis (5).
We applied FSN and conditional power to a collection of 98 existing meta-analyses (7) of associations between non-genomic cancer biomarkers and multiple types of cancer. More detailed illustration of their use is provided using data on a well-established biomarker-cancer relationship (i.e., H. pylori and gastric cancer) and an uncertain biomarker-cancer association (i.e., androgens and prostate cancer).
Methods
FSN and conditional power were applied to findings from 98 biomarker-cancer meta-analyses(8–44) (Table 1) published in 37 reports that were curated by Tsilidis et al. after a comprehensive PubMed search of meta-analyses of epidemiologic studies on biomarkers and cancer risk published between 1966 and 2010 (7). The purpose of that study was to evaluate whether evidence of excess statistical significance could be detected in such studies that would be indicative of publication bias.
Table 1.
Results for Rosenberg’s and Orwin’s FSN and conditional power for the 98 meta-analysis
| Area | Author & Year | Cancer | Biomarker | No. of studies | No. cases & controls | I2 | Fixed-effect |
Random-effects |
|||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| OR (95% CI) | FSN1 | FSN2 | M3 | OR (95% CI) | FSN1 | M4 | |||||||
| Diet | Chen 2010(8) | BrCA | 1α,25(OH)2 vitamin D | 3 | 3627 | 47 | 1.02 (0.81–1.29) | NA | NA | 858 | 0.99 (0.68–1.44) | NA | 27210 |
| Diet | Saadatian-Elahi 2004(9) | BrCA | Arachidonic acid | 5 | 2226 | 0 | 0.89 (0.65–1.22) | NA | NA | 79 | 0.89 (0.65–1.22) | NA | 181 |
| Diet | Saadatian-Elahi 2004(9) | BrCA | Linoleic acid | 8 | 3081 | 60 | 0.88 (0.69–1.12) | NA | NA | 67 | 0.85 (0.57–1.26) | NA | 457 |
| Diet | Saadatian-Elahi 2004(9) | BrCA | MUFA | 5 | 2291 | 67 | 1.33 (0.98–1.81) | NA | NA | 4 | 1.44 (0.82–2.53) | NA | 31 |
| Diet | Saadatian-Elahi 2004(9) | BrCA | Palmitic acid | 7 | 2802 | 59 | 1.04 (0.81–1.35) | NA | NA | 621 | 1.05 (0.69–1.58) | NA | 6656 |
| Diet | Saadatian-Elahi 2004(9) | BrCA | Palmitoleic acid | 2 | 798 | 81 | 1.09 (0.68–1.74) | NA | NA | 123 | 1.26 (0.41–3.89) | NA | 301 |
| Diet | Saadatian-Elahi 2004(9) | BrCA | SFA | 6 | 2570 | 0 | 1.05 (0.79–1.39) | NA | NA | 410 | 1.05 (0.79–1.39) | NA | 1430 |
| Diet | Saadatian-Elahi 2004(9) | BrCA | Stearic acid | 7 | 2802 | 14 | 0.93 (0.71–1.23) | NA | NA | 200 | 0.93 (0.69–1.26) | NA | 937 |
| Diet | Saadatian-Elahi 2004(9) | BrCA | α-Linolenic acid | 8 | 3444 | 39 | 0.82 (0.65–1.03) | NA | NA | 12 | 0.80 (0.59–1.08) | NA | 39 |
| Diet | Saadatian-Elahi 2004(9) | BrCA | n-3 PUFA | 8 | 2946 | 37 | 0.79 (0.60–1.03) | NA | NA | 11 | 0.79 (0.56–1.11) | NA | 51 |
| Diet | Saadatian-Elahi 2004(9) | BrCA | n-6 PUFA | 7 | 2667 | 16 | 0.75 (0.55–1.03) | NA | NA | 36 | 0.75 (0.53–1.06) | NA | 369 |
| Diet | Chen 2010(8) | BrCA | 25(OH) vitamin D | 7 | 11330 | 86 | 0.58 (0.51–0.66) | 230 | 75 | NA | 0.55 (0.38–0.80) | 29 | NA |
| Diet | Saadatian-Elahi 2004(9) | BrCA | Docosahexanoic acid | 7 | 3262 | 36 | 0.76 (0.59–0.99) | 5 | 106 | NA | 0.73 (0.53–1.02) | NA | 9 |
| Diet | Saadatian-Elahi 2004(9) | BrCA | Eicosapentanoic acid | 5 | 2291 | 0 | 0.91 (0.87–0.95) | 48 | 88 | NA | 0.91 (0.87–0.95) | 48 | NA |
| Diet | Buck 2010(10) | BrCA | Enterolactone | 12 | 7710 | 71 | 0.84 (0.74–0.96) | 24 | 200 | NA | 0.79 (0.61–1.02) | NA | 14 |
| Diet | Larsson 2007(11) | BrCA | Folate | 6 | 3584 | 41 | 0.69 (0.53–0.90) | 18 | 79 | NA | 0.67 (0.46–1.00) | 6 | NA |
| Diet | Saadatian-Elahi 2004(9) | BrCA | Oleic acid | 9 | 3723 | 70 | 0.83 (0.71–0.98) | 14 | 144 | NA | 0.99 (0.70–1.38) | NA | 184370 |
| Diet | Larsson 2010(12) | CRC | Vitamin B6 | 4 | 2307 | 0 | 0.52 (0.38–0.71) | 31 | 39 | NA | 0.52 (0.38–0.71) | 31 | NA |
| Diet | Yin 2009(13) | Colon CA | 25(OH) vitamin D | 7 | 2944 | 46 | 0.77 (0.59–1.00) | 7 | 103 | NA | 0.78 (0.53–1.13) | NA | 46 |
| Diet | Gallicchio 2008(14) | Lung CA | α-carotene | 5 | 5618 | 53 | 0.91 (0.69–1.19) | NA | NA | 65 | 0.88 (0.59–1.33) | NA | 438 |
| Diet | Gallicchio 2008(14) | Lung CA | B-cryptoxanthin | 5 | 5618 | 75 | 0.87 (0.62–1.21) | NA | NA | 44 | 0.82 (0.40–1.69) | NA | 529 |
| Diet | Gallicchio 2008(14) | Lung CA | Lutein/zeaxanthin | 4 | 5066 | 11 | 0.95 (0.68–1.33) | NA | NA | 342 | 0.95 (0.67–1.36) | NA | 1192 |
| Diet | Zhuo 2004(15) | Lung CA | Selenium | 6 | 2687 | 42 | 0.80 (0.63–1.02) | NA | NA | 6 | 0.77 (0.56–1.08) | NA | 17 |
| Diet | Gallicchio 2008(14) | Lung CA | B-carotene | 10 | 37629 | 41 | 0.83 (0.73–0.94) | 31 | 160 | NA | 0.84 (0.66–1.07) | NA | 36 |
| Diet | Gallicchio 2008(14) | Lung CA | Carotenoids | 4 | 7803 | 45 | 0.70 (0.50–0.97) | 5 | 53 | NA | 0.70 (0.44–1.11) | NA | 12 |
| Diet | Gallicchio 2008(14) | Lung CA | Lycopene | 4 | 5294 | 0 | 0.71 (0.51–0.99) | 5 | 54 | NA | 0.71 (0.51–0.99) | 5 | NA |
| Diet | Yin b 2009(16) | PrCA | 25(OH) vitamin D | 11 | 7806 | 26 | 1.03 (0.97–1.10) | NA | NA | 82 | 1.03 (0.95–1.11) | NA | 536 |
| Diet | Collin 2010(17) | PrCA | Folate | 7 | 9920 | 39 | 1.04 (0.98–1.11) | NA | NA | 25 | 1.11 (0.96–1.28) | NA | 17 |
| Diet | Collin 2010(17) | PrCA | Total Homocysteine | 4 | 7015 | 14 | 0.93 (0.74–1.17) | NA | NA | 77 | 0.91 (0.70–1.19) | NA | 123 |
| Diet | Collin 2010(17) | PrCA | Vitamin B12 | 6 | 9401 | 45 | 1.09 (1.03–1.14) | 24 | 127 | NA | 1.10 (1.01–1.19) | 10 | NA |
| Diet | Simon 2009(18) | PrCA | α-Linolenic acid | 6 | 2361 | 16 | 1.51 (1.17–1.94) | 26 | 181 | NA | 1.54 (1.16–2.06) | 21 | NA |
| Environment | Khanjani 2007(19) | BrCA | Cis-nonachlor | 3 | 1387 | 0 | 1.09 (0.72–1.64) | NA | NA | 137 | 1.09 (0.72–1.64) | NA | 290 |
| Environment | Lopez-Cervantes 2004(20) | BrCA | DDT | 24 | 11369 | 17 | 0.97 (0.87–1.09) | NA | NA | 668 | 0.97 (0.85–1.11) | NA | 8663 |
| Environment | Khanjani 2007(19) | BrCA | Dieldrin | 5 | 3223 | 43 | 1.18 (0.89–1.58) | NA | NA | 26 | 1.15 (0.77–1.69) | NA | 288 |
| Environment | Khanjani 2007(19) | BrCA | Trans-nonachlor | 6 | 3248 | 0 | 0.86 (0.68–1.07) | NA | NA | 23 | 0.86 (0.68–1.07) | NA | 35 |
| Environment | Khanjani 2007(19) | BrCA | Oxychlordane | 5 | 2718 | 51 | 0.75 (0.57–0.98) | 4 | 73 | NA | 0.77 (0.51–1.14) | NA | 38 |
| Environment | Veglia 2008(21) | CA (cur smokers) | DNA adducts | 8 | 916 | 94 | 3.88 (3.31–4.54) | 1146 | 628 | NA | 3.76 (1.75–8.05) | 39 | NA |
| Environment | Veglia 2008(21) | CA (for smokers) | DNA adducts | 7 | 632 | 0 | 0.94 (0.71–1.25) | NA | NA | 291 | 0.94 (0.71–1.25) | NA | 1041 |
| Environment | Veglia 2008(21) | CA (nev smokers) | DNA adducts | 9 | 564 | 79 | 1.20 (0.88–1.64) | NA | NA | 41 | 1.64 (0.72–3.77) | NA | 103 |
| IGF/insulin | Pisani 2008(22) | BrCA | C-peptide | 11 | 3517 | 64 | 1.26 (1.07–1.48) | 27 | 269 | NA | 1.35 (1.01–1.81) | 11 | NA |
| IGF/insulin | Morris 2006(23) | CRC | IGFBP-3 | 7 | 3501 | 60 | 1.00 (0.77–1.30) | NA | NA | NA | 0.98 (0.64–1.51) | NA | 47178 |
| IGF/insulin | Pisani 2008(22) | CRC | C-peptide | 12 | 5542 | 54 | 1.36 (1.15–1.62) | 64 | 322 | NA | 1.51 (1.14–1.99) | 39 | NA |
| IGF/insulin | Pisani 2008(22) | CRC | Glucose | 11 | 1381129 | 47 | 1.19 (1.07–1.32) | 49 | 257 | NA | 1.28 (1.06–1.54) | 26 | NA |
| IGF/insulin | Rinaldi 2010(24) | CRC | IGF-1 | 11 | 7828 | 0 | 1.07 (1.01–1.14) | 17 | 230 | NA | 1.07 (1.01–1.14) | 17 | NA |
| IGF/insulin | Morris 2006(23) | CRC | IGF-2 | 3 | 1685 | 0 | 1.95 (1.26–3.00) | 11 | 117 | NA | 1.95 (1.26–3.00) | 11 | NA |
| IGF/insulin | Pisani 2008(22) | Endometrial CA | C-peptide | 4 | 862 | 69 | 1.09 (0.74–1.62) | NA | NA | 141 | 1.18 (0.57–2.43) | NA | 642 |
| IGF/insulin | Chen 2009(25) | Lung CA | IGF-1 | 6 | 12515 | 41 | 1.05 (0.80–1.37) | NA | NA | 361 | 0.98 (0.68–1.41) | NA | 21602 |
| IGF/insulin | Chen 2009(25) | Lung CA | IGFBP-3 | 6 | 12515 | 67 | 0.89 (0.68–1.15) | NA | NA | 54 | 0.96 (0.59–1.56) | NA | 10376 |
| IGF/insulin | Pisani 2008(22) | Pancreas CA | C-peptide | 2 | 692 | 0 | 1.70 (1.11–2.61) | 4 | 68 | NA | 1.70 (1.11–2.61) | 4 | NA |
| IGF/insulin | Pisani 2008(22) | Pancreas CA | Glucose | 5 | 1334539 | 0 | 1.98 (1.67–2.35) | 152 | 198 | NA | 1.98 (1.67–2.35) | 152 | NA |
| IGF/insulin | Rowlands 2009(26) | PrCA | IGFBP-1 | 3 | 1553 | 92 | 0.93 (0.80–1.09) | NA | NA | 72 | 1.20 (0.65–2.22) | NA | 251 |
| IGF/insulin | Rowlands 2009(26) | PrCA | IGFBP-2 | 5 | 2670 | 78 | 1.07 (0.95–1.21) | NA | NA | 36 | 1.18 (0.90–1.54) | NA | 56 |
| IGF/insulin | Rowlands 2009(26) | PrCA | IGFBP-3 | 29 | 17160 | 81 | 0.97 (0.93–1.01) | NA | NA | 80 | 0.88 (0.79–0.98) | 57 | NA |
| IGF/insulin | Rowlands 2009(26) | PrCA | IGF-1 | 42 | 19347 | 88 | 1.18 (1.14–1.23) | 1497 | 974 | NA | 1.21 (1.07–1.36) | 159 | NA |
| IGF/insulin | Rowlands 2009(26) | PrCA | IGF-1/BP-3 | 11 | 9677 | 80 | 1.07 (1.02–1.13) | 30 | 230 | NA | 1.10 (0.97–1.24) | NA | 46 |
| IGF/insulin | Rowlands 2009(26) | PrCA | IGF-2 | 10 | 2797 | 77 | 1.24 (1.12–1.36) | 81 | 242 | NA | 1.17 (0.93–1.47) | NA | 75 |
| IGF/insulin | Key 2010(27) | postmenopausal BrCA | IGF-1 | 15 | 8185 | 0 | 1.30 (1.13–1.49) | 92 | 385 | NA | 1.30 (1.13–1.49) | 92 | NA |
| IGF/insulin | Key 2010(27) | postmenopausal BrCA | IGFBP-3 | 15 | 8012 | 31 | 1.21 (1.04–1.41) | 32 | 357 | NA | 1.22 (1.01–1.49) | 16 | NA |
| IGF/insulin | Key 2010(27) | premenopausal BrCA | IGFBP-3 | 11 | 5927 | 0 | 0.99 (0.83–1.19) | NA | NA | 7367 | 0.99 (0.83–1.19) | NA | 50352 |
| IGF/insulin | Key 2010(27) | premenopausal BrCA | IGF-1 | 11 | 6033 | 29 | 1.18 (1.00–1.40) | 10 | 255 | NA | 1.21 (0.98–1.49) | NA | 12 |
| Infection | Gutierrez 2006(28) | Bladder CA | HPV (DNA) | 13 | 657 | 6 | 2.29 (1.37–3.84) | 53 | 597 | NA | 2.30 (1.33–4.00) | 45 | NA |
| Infection | Gutierrez 2006(28) | Bladder CA | HPV (no DNA) | 3 | 379 | 0 | 2.98 (1.65–5.40) | 18 | 180 | NA | 2.98 (1.65–5.40) | 18 | NA |
| Infection | Zhao 2008(29) | CRC | H. pylori | 14 | 3581 | 58 | 1.41 (1.22–1.65) | 127 | 391 | NA | 1.49 (1.16–1.90) | 57 | NA |
| Infection | Mandelblatt 1999(30) | Cervical CA | HPV | 12 | 3657 | 27 | 8.07 (6.49–10.0) | 2338 | 1978 | NA | 8.08 (6.04–10.8) | 1249 | NA |
| Infection | Zhang 1994(31) | Cervical CA | T. vaginalis | 2 | 65764 | 0 | 1.88 (1.29–2.74) | 9 | 75 | NA | 1.88 (1.29–2.74) | 9 | NA |
| Infection | Islami 2008(32) | ESCC | H. pylori | 9 | 3664 | 73 | 1.08 (0.92–1.27) | NA | NA | 73 | 1.10 (0.78–1.55) | NA | 1356 |
| Infection | Islami 2008(32) | ESCC | cagA (H. pylori) | 4 | 2327 | 0 | 1.01 (0.79–1.27) | NA | NA | NA | 1.01 (0.79–1.27) | NA | NA |
| Infection | Islami 2008(32) | Esophageal adeno CA | H. pylori | 13 | 3730 | 15 | 0.56 (0.48–0.67) | 275 | 136 | NA | 0.57 (0.47–0.69) | 207 | NA |
| Infection | Islami 2008(32) | Esophageal adeno CA | cagA (H. pylori) | 5 | 1472 | 17 | 0.41 (0.29–0.59) | 54 | 37 | NA | 0.41 (0.28–0.62) | 42 | NA |
| Infection | Huang 2003(33) | Gastric CA | H. pylori | 15 | 5054 | 76 | 2.05 (1.79–2.35) | 805 | 615 | NA | 2.29 (1.71–3.05) | 224 | NA |
| Infection | Huang 2003(33) | Gastric CA | cagA (H. pylori) | 10 | 3831 | 85 | 2.65 (2.29–3.05) | 888 | 531 | NA | 2.87 (1.95–4.22) | 137 | NA |
| Infection | Zhuo 2008(34) | Laryngeal CA | H. pylori | 3 | 357 | 0 | 2.02 (1.27–3.23) | 10 | 121 | NA | 2.02 (1.27–3.23) | 10 | NA |
| Infection | Hobbs 2006(35) | Larynx CA | HPV | 8 | 1133 | 50 | 1.71 (1.11–2.64) | 17 | 281 | NA | 2.01 (0.96–4.22) | NA | 6 |
| Infection | Donato 1998(36) | Liver CA | HBV (HCV-) | 28 | 9199 | 86 | 17.9 (15.7–20.5) | 24939 | 10279 | NA | 21.9 (14.9–32.3) | 3464 | NA |
| Infection | Donato 1998(36) | Liver CA | HBV + HCV | 9 | 2437 | 37 | 65.0 (35.0–121) | 784 | 12315 | NA | 61.2 (27.0–139) | 440 | NA |
| Infection | Donato 1998(36) | Liver CA | HCV (HBV-) | 26 | 7694 | 86 | 16.8 (14.1–20.0) | 13151 | 9822 | NA | 20.3 (12.2–33.7) | 1924 | NA |
| Infection | Zhuo 2009(37) | Lung CA | H. pylori | 4 | 430 | 79 | 2.31 (1.46–3.65) | 22 | 185 | NA | 3.24 (1.11–9.41) | 6 | NA |
| Infection | Hobbs 2006(35) | Oral CA | HPV | 8 | 3976 | 62 | 1.68 (1.36–2.08) | 76 | 274 | NA | 1.99 (1.17–3.38) | 17 | NA |
| Infection | Hobbs 2006(35) | Oropharynx CA | HPV | 5 | 2199 | 56 | 3.01 (2.11–4.30) | 93 | 300 | NA | 4.31 (2.07–8.95) | 35 | NA |
| Infection | Taylor 2005(38) | PrCA | HPV | 9 | 4864 | 35 | 1.37 (1.11–1.69) | 31 | 246 | NA | 1.52 (1.12–2.06) | 23 | NA |
| Infection | Hobbs 2006(35) | Tonsil CA | HPV | 8 | 380 | 0 | 15.1 (6.78–33.4) | 173 | 2471 | NA | 15.1 (6.78–33.4) | 173 | NA |
| Infection | Wang 2007(39) | early Gastric CA | H. pylori | 15 | 16698 | 83 | 4.83 (4.27–5.48) | 4639 | 1467 | NA | 3.38 (2.15–5.32) | 197 | NA |
| Inflammation | Heikkila 2009(40) | CA | Interleukin-6 | 4 | 6785 | 21 | 1.01 (0.92–1.11) | NA | NA | 718 | 1.01 (0.90–1.12) | NA | 3321 |
| Inflammation | Heikkila 2009(40) | CA | C-reactive protein | 14 | 74545 | 73 | 1.09 (1.05–1.13) | 150 | 299 | NA | 1.10 (1.02–1.18) | 35 | NA |
| Inflammation | Tsilidis 2008(41) | CRC | C-reactive protein | 8 | 39145 | 51 | 1.10 (1.02–1.18) | 20 | 172 | NA | 1.12 (1.01–1.25) | 12 | NA |
| Sex hormones | Barba 2009(42) | PrCA | 2OHE1 | 2 | 536 | 0 | 0.76 (0.45–1.28) | NA | NA | 13 | 0.76 (0.45–1.28) | NA | 15 |
| Sex hormones | Roddam 2008(43) | PrCA | A-diol G | 8 | 5488 | 24 | 1.12 (0.96–1.31) | NA | NA | 17 | 1.15 (0.95–1.38) | NA | 28 |
| Sex hormones | Roddam 2008(43) | PrCA | D4 | 6 | 4211 | 0 | 1.02 (0.85–1.21) | NA | NA | 994 | 1.02 (0.85–1.21) | NA | 3995 |
| Sex hormones | Roddam 2008(43) | PrCA | DHEA-S | 7 | 3024 | 17 | 1.22 (0.98–1.53) | NA | NA | 8 | 1.29 (0.99–1.68) | NA | 5 |
| Sex hormones | Roddam 2008(43) | PrCA | DHT | 7 | 2455 | 0 | 0.88 (0.69–1.11) | NA | NA | 41 | 0.88 (0.69–1.11) | NA | 80 |
| Sex hormones | Roddam 2008(43) | PrCA | E2 | 9 | 5225 | 0 | 0.92 (0.78–1.09) | NA | NA | 62 | 0.92 (0.78–1.09) | NA | 162 |
| Sex hormones | Roddam 2008(43) | PrCA | Free E2 | 8 | 4778 | 0 | 0.97 (0.82–1.16) | NA | NA | 1173 | 0.97 (0.82–1.16) | NA | 5279 |
| Sex hormones | Roddam 2008(43) | PrCA | Free T | 14 | 9365 | 0 | 1.12 (0.98–1.27) | NA | NA | 18 | 1.12 (0.98–1.27) | NA | 20 |
| Sex hormones | Roddam 2008(43) | PrCA | T | 17 | 10324 | 0 | 0.98 (0.87–1.10) | NA | NA | 502 | 0.98 (0.87–1.10) | NA | 3602 |
| Sex hormones | Barba 2009(42) | PrCA | 16α-OHE1 | 2 | 536 | 0 | 1.82 (1.08–3.05) | 3 | 73 | NA | 1.82 (1.08–3.05) | 3 | NA |
| Sex hormones | Barba 2009(42) | PrCA | 2OHE1/16α-OHE1 | 2 | 536 | 0 | 0.52 (0.31–0.89) | 4 | 19 | NA | 0.52 (0.31–0.89) | 4 | NA |
| Sex hormones | Roddam 2008(43) | PrCA | SHBG | 15 | 9702 | 0 | 0.86 (0.76–0.97) | 32 | 249 | NA | 0.86 (0.76–0.97) | 32 | NA |
| Sex hormones | Key 2002(44) | postmenopausal BrCA | E2 | 9 | 2365 | 42 | 1.29 (1.14–1.45) | 72 | 227 | NA | 1.26 (1.07–1.49) | 27 | NA |
IGF, insulin-like growth factor; CRC, colorectal cancer; IGFBP, insulin-like growth factor binding protein; CA, cancer; BrCA, breast cancer; PrCA, prostate cancer; ESCC, esophageal squamous cell carcinoma; T, testosterone; E2, estradiol; DHT, dihydrotestosterone; A-diol G, androstanediol glucuronide; DHEA-S, dehydroepiandrosterone sulfate; D4, androstenedione; SHBG, sex hormone binding globulin; E1, estrone; SFA, total saturated fatty acids; MUFA, monounsaturated fatty acids; PUFA, polyunsaturated fatty acids; H. pylori, Helicobacter pylori; HPV, human papillomavirus; HBV, hepatitis B virus; HCV, hepatitis C virus; T. vaginalis, Trichomonas vaginalis; DDT, dichlorodiphenyltrichloroethane; Cur, current; For, former; Nev, never; NA, non-statistically significant meta-analyses not applicable to the FSN, and statistically significant meta-analyses not applicable to the conditional power analysis.
Rosenberg’s FSN – the number of future studies averaging null effect and average weight to reduce the summary OR to null
Orwin’s FSN – the number of future studies averaging null effect to reduce the summary OR to 1.05
Number of future studies of average weight and no between-study heterogeneity needed to be included in the updated meta-analysis to achieve 80% power to detect the observed fixed-effect summary OR
Number of future studies of average weight and average between-study heterogeneity need to be included in the updated meta-analysis to achieve 80% power to detect the observed random-effects summary OR
The 98 meta-analyses included a median of seven studies (range 2–42) and described associations between a diverse range of non-genomic biomarkers and cancer risk including: Insulin-like growth factor(IGF)/insulin markers (21 meta-analyses); sex hormones (13 meta-analyses); dietary markers (31 meta-analyses); inflammatory markers (3 meta-analyses); infectious agents (22 meta-analyses); and environmental markers (8 meta-analyses). The most common cancer sites include breast (28 meta-analyses); prostate (24 meta-analyses); lung (10 meta-analyses); and colorectal (8 meta-analyses). Previously, using the primary study data from the studies included in each of the 98 meta-analyses, Tsilidis et al. (7) calculated summary estimates using fixed-effect and random-effects models and corresponding 95% confidence intervals, and I2. Based on random-effects models, 44 (45%) of the meta-analyses reported statistically significant summary odds ratios (OR), whereas based on fixed-effect models 54 (55%) of the meta-analyses reported statistically significant summary ORs.
Fail-Safe Number:
For the statistically significant meta-analyses, we used Rosenberg’s version of the FSN (4) (a refinement of Rosenthal’s FSN (6)) to quantify the number of future studies with an average null effect and average weight (i.e., inverse variance), needed to drive the existing meta-analysis summary estimate to null in the updated meta-analysis (for this work: P≥0.05). To overcome the restriction of statistical significance, we used Orwin’s FSN (45) to calculate the number of future studies with an average null effect (OR=1.00) needed to reduce the updated summary effect to a range of estimates (OR=1.05; 1.10; 1.25; 1.50; and 2.00) for the updated meta-analysis. Additional details of FSN calculation are presented in Supplemental Methods. FSN is not applicable to non-statistically significant summary estimates.
Conditional power:
For the non-statistically significant meta-analyses, we calculated conditional power to determine the number of future studies needed to achieve sufficient power to detect a statistically significant summary estimate when added to the observed non-statistically significant meta-analysis (P≥0.05). We set the minimum power to 0.8 and took a pragmatic approach declaring an alternative hypothesis for the updated meta-analysis equivalent to the observed summary OR, and assumed the future studies were of average weight as those included in the observed meta-analysis. Our conditional power analyses were based on two approaches described by Roloff et al. (5) We implemented the first approach in the non-statistically significant fixed-effect meta-analyses, where we assumed that no heterogeneity is present between the studies included in the existing meta-analysis (I2=0%) and that the future studies will not introduce heterogeneity. In approach 2, focusing on the non-statistically significant random-effects meta-analyses, we fixed the between-study heterogeneity in the future studies to be equivalent to the heterogeneity in the existing meta-analysis. Additional details of conditional power calculation are presented in Supplemental Methods
From the list of 98 meta-analyses, we selected two exemplar scenarios: 1) a well-established causal biomarker-cancer relationship supported by evidence-based classification as a Group 1 carcinogen (i.e., H. pylori and gastric cancer risk) (46) and 2) a biomarker-cancer association with strong biological rationale, but several methodologic concerns leading to an uncertain biomarker-cancer association (i.e., androgens and prostate cancer). We provide these two examples both to describe the application of these adapted methods and how their use can be used in practice to inform the need for future research to be able to fill knowledge gaps and improve biological understanding. For both scenarios, we interpret the number of future studies needed determined by FSN for H. pylori and gastric cancer or by conditional power for androgens and prostate cancer within the context of the existing evidence (e.g., the number, sample size, and heterogeneity of the findings).
We calculated Rosenberg’s and Orwin’s FSNs and the two conditional power approaches in STATA version 13 (STATA Corp, College Station, TX).
Results
FSN.
Among the 54 statistically significant fixed-effect (median number of studies 9 [range 2–42]; median I2=42%) and 44 statistically significant random-effects (median number of studies 9 [range 2–42]; median I2=36%) meta-analyses, median FSN (Rosenberg) was 31.5 studies (range 3.2–24,939) for the fixed-effect meta-analyses, and 31.1 studies (range 3.2–3,464) for the random-effects meta-analyses.
The influence of between-study heterogeneity on Rosenberg’s FSN is illustrated by comparing the FSN between the fixed-effect and random-effects summary estimates from the same meta-analysis (SFigure 1). The median FSN was larger for meta-analyses with extreme heterogeneity (I2>80% (47)); 1497 and 148 for fixed-effect and random-effects meta-analyses, respectively, compared to 53 and 45 for fixed-effect and random-effects meta-analyses with low heterogeneity (I2: 1–29% (47)). The FSN was larger for the fixed-effect than for the random-effects meta-analyses, which is consistent with the assumption of no between-study heterogeneity in fixed-effect meta-analyses that results in more precise summary estimates (48) (SFigure 1). Among meta-analyses with similar between-study heterogeneity (0%, 1–29%, 30–59%, 60–80%, >80%), meta-analyses that included more studies tended to have a higher FSN (SFigure 2) as a result of more precise summary estimates.
Rosenberg’s FSN was larger when the summary estimates observed in the existing meta-analyses were higher (SFigure 3). The influence of summary estimate size in the existing meta-analysis and in the future studies is further illustrated with Orwin’s FSN, which does not take into account within- or between-study heterogeneity. Therefore, we only considered the values of Orwin’s FSN for fixed-effect meta-analyses. Orwin’s FSN was larger for smaller updated summary estimates (SFigure 4). To reduce the updated summary OR to 1.05 among 38 meta-analyses with an existing summary OR>1.05, the median of Orwin’s FSN was 271 studies, whereas to reduce the updated summary OR to 2.00 among meta-analyses with an existing summary OR>2.00 the median FSN was 33 studies. As for Rosenberg’s FSN, which is based on statistical significance, Orwin’s FSN, which is based on effect size, also indicates that a larger number of future studies is required for existing meta-analyses with larger as opposed to smaller summary ORs.
Conditional power.
We used two approaches under a variety of assumptions to conduct conditional power analysis. In the first approach, we assumed no between-study heterogeneity in the existing and updated meta-analyses, and accordingly, used only the 18 fixed-effect meta-analyses with a statistically non-significant summary OR>1.01. With a median power of 15% (range 0.5–50%) for the existing meta-analyses, a median of 78 studies (range 4–994) of average weight with no between-study heterogeneity would need to be included in the updated meta-analysis to achieve 80% power to detect the summary OR as statistically significant.
In the second approach, we assumed equivalent between-study heterogeneity in the future studies as in the existing meta-analysis, and accordingly used the 21 random-effects meta-analyses with a statistically non-significant summary OR>1.01. With a median power of 21% (range 6–47%) for the existing meta-analyses, a median of 103 studies (range 5–6,656) of average weight and equivalent between-study heterogeneity as in the existing meta-analysis would need to be included in the updated meta-analysis to achieve 80% power to detect the summary OR as statistically significant.
The greater number of future studies required to achieve 80% for the random-effects compared with fixed-effect meta-analysis is consistent with their differing assumptions about between-study heterogeneity incorporated into the two approaches (SFigure 5). By taking into account the between-study heterogeneity, our second approach incorporated additional uncertainty into the summary estimates, thereby increasing the number of future studies needed. In the both fixed-effect and random-effects meta-analyses, the number of future studies needed was smaller for larger than for smaller summary estimates (SFigure 5).
Application of the FSN: H. pylori and gastric cancer.
In 1994, the International Agency for Research on Cancer (IARC) classified Helicobacter pylori as a Group 1 carcinogen (46). At the time, the evidence supporting IARC’s classification included four cohort studies and nine case-control studies of H. pylori infection and gastric cancer risk. Since the initial classification, the accumulation of evidence is sufficient that the relationship is now considered well established. This is reflected in the greater than 2-fold increase in risk of gastric cancer described in the meta-analysis of 15 studies with more than 5,000 cases and controls reported by Huang et al. (33) Rosenberg’s FSN indicates 805 future studies would be required to reduce the reported fixed-effect summary OR of 2.05 (95% CI 1.79–2.35; I2=76%) to null (P≥0.05) and 224 future studies based on the random-effects meta-analysis (summary OR=2.29; 95% CI 1.71–3.05; I2=76%). Based on Orwin’s FSN, a total of 615 future studies averaging null effect (OR=1.00) would be required to drive the observed fixed-effect summary OR of 2.05 to an essentially null OR of 1.05. The implementation of each FSN to the example of H. pylori and gastric cancer illustrates the futility of further investigation of the association between H. pylori and gastric cancer, while the large between-study heterogeneity (I2=76%) suggests the need for further subgroup analysis to determine sources of heterogeneity (e.g., method of detection of H. pylori infection, adjustment for confounding, or geographic/ethnic differences in strength of the association). To this end, the geographic and ethnic differences in the distribution of gastric cancer led to further investigations that revealed a stronger association between H. pylori infection and gastric cancer in studies conducted in populations with diets high in salt-preserved foods, suggesting dietary salt may modify the pathogenic effect of H. pylori infection on gastric cancer (49, 50). The role of a high salt diet as a potential modifier of the effect of H. pylori is supported by additional laboratory research that identified cagA gene expression in H. pylori, a marker of higher risk of gastric cancer, is upregulated by dietary salt intake (51). These findings further illustrate the importance of examining subgroups or different populations once the main effect of the etiologic cancer biomarker has been established, especially in the context of extreme heterogeneity which can help identify high-risk populations and can provide additional understanding of the underlying biology of the biomarker cancer association (e.g., effect modification).
Application of conditional power: Androgens and prostate cancer.
In 1993, the Prostate Cancer Prevention Trial was launched to test the hypothesis that finasteride, a drug that blocks the conversion of testosterone into dihydrotestosterone (DHT), can prevent prostate cancer (52). The trial was stopped early in 2003 when an interim analysis found a 25% reduction in the period prevalence of prostate cancer in the treatment group receiving finasteride (53). This finding provided additional evidence supporting the underlying hypothesis that DHT is an etiologic factor in prostate cancer. However, several methodological challenges encountered in population-based epidemiologic investigations including adequacy of measuring circulating hormones, difficulty integrating multiple components of the androgen pathway, difficulty in incorporating clinical and population health important outcomes, and detection bias (e.g., differential opportunity to be screened with PSA by exposure; and differential detection of prostate cancer in PSA-based prostate cancer screening due to the association between androgens and PSA concentration), have contributed to the inconsistent reports on the associations between circulating androgens and prostate cancer incidence (54). Using study-specific estimates for components in the androgen pathway and prostate cancer from a pooled analysis of harmonized primary data, (43) Tsilidis et al. (7) calculated fixed-effect and random-effects summary estimates (Table 2). For the six components of the androgen pathway that were not statistically significant in fixed-effect meta-analyses (with I2=0% and a median number of studies of 8.5), conditional power indicated that 18 to 1173 future studies of average weight as those included in the existing meta-analysis would be required to achieve 80% power to detect the summary OR in the updated meta-analysis (Table 2). For these comparisons, the large number of future studies needed to achieve sufficient power – more than twice as many studies as included in the existing meta-analyses – of the same average weight – totaling tens of thousands of cases and controls among the future studies (Table 2) – may not be within reach of existing resources, and points to a situation where further research should be aimed at overcoming the methodologic challenges mentioned above (54) to fill important evidence gaps with respect to androgens and prostate cancer.
Table 2.
Results of conditional power for 9 meta-analyses of circulating androgens concentrations and prostate cancer risk
| Comparison | No. included studies | No. cases & controls | I2 | Fixed-effect1 |
Random-effects1 |
||||
|---|---|---|---|---|---|---|---|---|---|
| Odds ratio | 95% CI | Future studies2 | Odds ratio | 95% CI | Future studies3 | ||||
| SHBG | 15 | 9702 | 0 | 0.86 | 0.76 – 0.97 | 1 | 0.86 | 0.76 – 0.97 | 1 |
| Free T | 14 | 9365 | 0 | 1.12 | 0.98 – 1.27 | 18 | 1.12 | 0.98 – 1.27 | 20 |
| DHT | 7 | 2455 | 0 | 0.88 | 0.69 – 1.11 | 41 | 0.88 | 0.69 – 1.11 | 80 |
| E2 | 9 | 5225 | 0 | 0.92 | 0.78 – 1.09 | 62 | 0.92 | 0.78 – 1.09 | 162 |
| T | 17 | 10324 | 0 | 0.98 | 0.87 – 1.10 | 502 | 0.98 | 0.87 – 1.10 | 3602 |
| D4 | 6 | 4211 | 0 | 1.02 | 0.85 – 1.21 | 994 | 1.02 | 0.85 – 1.21 | 3995 |
| Free E2 | 8 | 4778 | 0 | 0.97 | 0.82 – 1.16 | 1173 | 0.97 | 0.82 – 1.16 | 5279 |
| DHEA-S | 7 | 3024 | 17 | 1.22 | 0.98 – 1.53 | 8 | 1.29 | 0.99 – 1.68 | 5 |
| A-diol G | 8 | 5488 | 24 | 1.12 | 0.96 – 1.31 | 17 | 1.15 | 0.95 – 1.38 | 28 |
T, testosterone; E2, estradiol; DHT, dihydrotestosterone; A-diol G, androstanediol glucuronide; DHEA-S, dehydroepiandrosterone sulfate; D4, androstenedione; SHBG, sex hormone binding globulin.
Fixed-effect and random-effects estimates reported by Tsilidis et al. (7) calculated from study-specific estimates for individual components in the androgen pathway and prostate cancer from Roddam et al. (43)
Number of future studies of average weight as studies included in observed meta-analysis needed to achieve 80% in updated meta-analysis determined by conditional power assuming no between-study heterogeneity
Number of future studies of average weight and equivalent between-study heterogeneity as studies included in observed meta-analysis needed to achieve 80% power in updated meta-analysis determined by conditional power assuming equivalent between-study heterogeneity in updated meta-analysis
In the case of the random-effects meta-analysis with 7 included studies evaluating the association between dehydroepiandrosterone sulfate (DHEA-S) and prostate cancer (summary OR=1.29; 95% CI 0.99 to 1.68; I2=17%), the 5 future studies required to achieve 80% power to detect the observed summary OR may be within reach of existing resources, and points to a scenario where additional research could provide a meaningful contribution to the existing meta-analysis. However, we caution against the inappropriate interpretation of applying conditional power to the example of DHEA-S and prostate cancer incidence. Our approach assumed that the number of future studies are of the average weight of those already included in the existing meta-analysis and that they will not introduce additional between-study heterogeneity into the updated meta-analysis. However, this assumption may not be realistic; with respect to molecular epidemiologic investigations, measurement error in the index biomarker assay may introduce between-study heterogeneity. Further, relying on the number of needed studies does not guarantee that a future study will be informative. Whether to conduct future studies on DHEA-S and prostate cancer must also take into consideration the composition of the existing evidence base (e.g., existing study population characteristics and prostate cancer case mix) and failure to consider the methodological issues previously cited as factors leading to inconsistent associations could also lead to uninformative research.
Discussion
We adapted two established metrics – the fail-safe number (FSN) (4) and conditional power (5) – to quantify the impact of future investigations on the inferences drawn in existing meta-analyses. Both metrics provide a heuristic approach to inform whether continued investigation is warranted versus sufficient evidence is available to establish or refute an exposure-outcome association. Our motivation to adapt the application of these metrics is to be able to quantify the impact of further investigation of the same association as the primary research question. However, the application of these metrics should not be interpreted as stopping research all together, but rather, to focus future research to address current evidence gaps and improve biologic understanding of the biomarker-cancer association by evaluating new or improved methods to measure the biomarker or using other markers correlated and more specific to the studied biomarker, evaluating clinically meaningful outcomes, and reducing heterogeneity and imprecision in the observed associations by investigating the biomarker-cancer relationship in important subpopulations. When further research does not add information to the existing literature, unnecessary and wasteful research may be undertaken (55). We envision the application of these metrics along with traditional assessments of study quality (e.g., STROBE,(56) PRISMA,(57)) causal criteria (58), and remaining knowledge gaps (e.g., subgroup associations) by stakeholders engaged in translational epidemiologic research including principal investigators, funding agencies, grant reviewers, journal editors, and peer-reviewers to make more informed decisions about the need for additional research. While our application of FSN and conditional power focused on observational studies of etiologic biomarkers and cancer risk, these methods are equally applicable to other epidemiologic study designs including randomized trials as well as non-biomarker exposures and other important outcomes such as mortality, and prognosis.
FSN can be calculated using several common meta-analysis software packages and calculation of conditional power is straightforward (See Supplemental Methods) but requires a number of assumptions (e.g., heterogeneity, effect size, and study weights) that influence how the corresponding metrics are interpreted, thus informing the impact of future research. We applied these metrics to 98 meta-analyses of observational epidemiologic studies evaluating the associations between non-genomic biomarkers and cancer risk to demonstrate the ability of these metrics to identify situations where future research may or may not provide a meaningful contribution to an updated meta-analysis. When adapting the application of these metrics, the patterns of the output of the FSN and conditional power analysis are consistent with the underlying computation of each metric. For example, FSN appears to increase with decreasing heterogeneity, increasing number of included studies, and increasing magnitude of summary estimates. For conditional power, the number of additional studies appears to decrease with increasing magnitude of summary estimates.
To our knowledge no method has been introduced to directly quantify the expected impact of further observational epidemiologic research on the current evidence base. While our motivation was to explore whether the FSN and conditional power could be used to quantify the impact of future research, additional work is needed to incorporate these metrics into a formal framework for deciding whether additional epidemiologic studies addressing the same question are needed. Such a framework might include cutpoints or ranges for defining whether the number of future studies needed is too large to make additional work worthwhile. We do not envision that the framework would rely on cutpoints alone: considerations that could be incorporated into the framework beyond a cutpoint might include feasibility and cost as well as implications for policy, and clinical and public health recommendations. Such a framework could encompass aspects of the Value of Information approach to deciding cost-effectiveness, which has been described for improving research prioritization and reducing waste (59).
We recognize that application of these adapted methods to existing meta-analyses is not the only strategy to minimizing the problem of repetitive research. Facilitating and encouraging the publication of null results that can be included in meta-analyses such that the null results are interpreted alongside the relevant evidence is a direct way investigators and stakeholders can minimize the production of redundant uninformative research (60). An alternative approach is a coordinated effort among individual investigators to determine which exposures require additional investigations, to share and pool their data and biospecimens, to standardize an exposure’s measurement and harmonize the outcome and covariate data, all while ensuring optimal study design and minimizing selection and information bias. Using this approach, research on particular exposures is prioritized through consensus, exposure-outcome associations can be investigated in subpopulations of the pooled studies, and power is maximized. This practice-based approach has been used over the past 15 years by large consortia, including the NCI Cohort Consortium (>50 cohorts with 7 million participants) (https://epi.grants.cancer.gov/Consortia/cohort.html#overview) and the Early Detection Research Network (https://edrn.nci.nih.gov) both supported by the National Cancer Institute (NCI), and the Endogenous Hormones, Nutritional Biomarkers and Prostate Cancer Collaborative Group (35 studies with biomarker data on 23,000 men with prostate cancer and 35,000 controls) (https://www.ceu.ox.ac.uk/research/endogenous-hormones-nutritional-biomarkers-and-prostate-cancer). We view the approach that we describe herein as complementary to the practice-based approach.
In summary, we show how FSN and conditional power can be adapted to quantify the impact of future investigations of a specified exposure and outcome on the current evidence base summarized in the corresponding meta-analysis. To illustrate the utility of these approaches, we applied them to meta-analyses of biomarkers and cancer risk. The systematic application of these metrics by researchers, funding agencies, and grant reviewers when considering future research, journal editors, and peer-reviewers when considering the novelty and impact of submitted manuscripts, could lead to more judicious use of resources and acceleration along the translational continuum from discovery to population-health impact.
Supplementary Material
Acknowledgments:
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the National Institutes of Health. We appreciate Dr. Muin Khoury’s helpful comments during the conduct of this work.
Funding: Dr. Marrone was supported by National Cancer Institute grant T32 93140 (Platz). Dr. Platz was supported by NCI Cancer Center Support Grant P30 CA006973 (Nelson). Dr. Joshu was supported by the Prostate Cancer Foundation.
Footnotes
Conflicts of interest: The authors declare that they have no competing financial interests related to this paper.
References
- 1.Kern SE. Why your new cancer biomarker may never work: recurrent patterns and remarkable diversity in biomarker failures. Cancer Res. 2012;72:6097–101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383:166–75. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gulmezoglu AM, et al. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383:156–65. [DOI] [PubMed] [Google Scholar]
- 4.Rosenberg MS. The file-drawer problem revisited: a general weighted method for calculating the fail-safe number in meta-analysis. Evolution. 2005;59:464–68. [PubMed] [Google Scholar]
- 5.Roloff V, Higgins JP, Sutton AJ. Planning future studies based on the conditional power of a meta-analysis. Stat Med. 2013;32:11–24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Rosenthal R The file drawer problem and tolerance for null results. Psycholical Bulletin. 1979;86:638–41. [Google Scholar]
- 7.Tsilidis KK, Papatheodorou SI, Evangelou E, Ioannidis JP. Evaluation of excess statistical significance in meta-analyses of 98 biomarker associations with cancer risk. J Natl Cancer Inst. 2012;104:1867–78. [DOI] [PubMed] [Google Scholar]
- 8.Chen P, Hu P, Xie D, Qin Y, Wang F, Wang H. Meta-analysis of vitamin D, calcium and the prevention of breast cancer. Breast Cancer Res Treat. 2010;121:469–77. [DOI] [PubMed] [Google Scholar]
- 9.Saadatian-Elahi M, Norat T, Goudable J, Riboli E. Biomarkers of dietary fatty acid intake and the risk of breast cancer: a meta-analysis. Int J Cancer. 2004;111:584–91. [DOI] [PubMed] [Google Scholar]
- 10.Buck K, Zaineddin AK, Vrieling A, Linseisen J, Chang-Claude J. Meta-analyses of lignans and enterolignans in relation to breast cancer risk. Am J Clin Nutr. 2010;92:141–53. [DOI] [PubMed] [Google Scholar]
- 11.Larsson SC, Giovannucci E, Wolk A. Folate and risk of breast cancer: a meta-analysis. J Natl Cancer Inst. 2007;99:64–76. [DOI] [PubMed] [Google Scholar]
- 12.Larsson SC, Orsini N, Wolk A. Vitamin B6 and risk of colorectal cancer: a meta-analysis of prospective studies. JAMA. 2010;303:1077–83. [DOI] [PubMed] [Google Scholar]
- 13.Yin L, Grandi N, Raum E, Haug U, Arndt V, Brenner H. Meta-analysis: longitudinal studies of serum vitamin D and colorectal cancer risk. Aliment Pharmacol Ther. 2009;30:113–25. [DOI] [PubMed] [Google Scholar]
- 14.Gallicchio L, Boyd K, Matanoski G, Tao XG, Chen L, Lam TK, et al. Carotenoids and the risk of developing lung cancer: a systematic review. Am J Clin Nutr. 2008;88:372–83. [DOI] [PubMed] [Google Scholar]
- 15.Zhuo H, Smith AH, Steinmaus C. Selenium and lung cancer: a quantitative analysis of heterogeneity in the current epidemiological literature. Cancer Epidemiol Biomarkers Prev. 2004;13:771–8. [PubMed] [Google Scholar]
- 16.Yin L, Raum E, Haug U, Arndt V, Brenner H. Meta-analysis of longitudinal studies: Serum vitamin D and prostate cancer risk. Cancer Epidemiol. 2009;33:435–45. [DOI] [PubMed] [Google Scholar]
- 17.Collin SM, Metcalfe C, Refsum H, Lewis SJ, Zuccolo L, Smith GD, et al. Circulating folate, vitamin B12, homocysteine, vitamin B12 transport proteins, and risk of prostate cancer: a case-control study, systematic review, and meta-analysis. Cancer Epidemiol Biomarkers Prev. 2010;19:1632–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Simon JA, Chen YH, Bent S. The relation of alpha-linolenic acid to the risk of prostate cancer: a systematic review and meta-analysis. Am J Clin Nutr. 2009;89:1558s–64s. [DOI] [PubMed] [Google Scholar]
- 19.Khanjani N, Hoving JL, Forbes AB, Sim MR. Systematic review and meta-analysis of cyclodiene insecticides and breast cancer. J Environ Sci Health C Environ Carcinog Ecotoxicol Rev. 2007;25:23–52. [DOI] [PubMed] [Google Scholar]
- 20.Lopez-Cervantes M, Torres-Sanchez L, Tobias A, Lopez-Carrillo L. Dichlorodiphenyldichloroethane burden and breast cancer risk: a meta-analysis of the epidemiologic evidence. Environ Health Perspect. 2004;112:207–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Veglia F, Loft S, Matullo G, Peluso M, Munnia A, Perera F, et al. DNA adducts and cancer risk in prospective studies: a pooled analysis and a meta-analysis. Carcinogenesis. 2008;29:932–6. [DOI] [PubMed] [Google Scholar]
- 22.Hyper-insulinaemia Pisani P. and cancer, meta-analyses of epidemiological studies. Arch Physiol Biochem. 2008;114:63–70. [DOI] [PubMed] [Google Scholar]
- 23.Morris JK, George LM, Wu T, Wald NJ. Insulin-like growth factors and cancer: no role in screening. Evidence from the BUPA study and meta-analysis of prospective epidemiological studies. Br J Cancer. 2006;95:112–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Rinaldi S, Cleveland R, Norat T, Biessy C, Rohrmann S, Linseisen J, et al. Serum levels of IGF-I, IGFBP-3 and colorectal cancer risk: results from the EPIC cohort, plus a meta-analysis of prospective studies. Int J Cancer. 2010;126:1702–15. [DOI] [PubMed] [Google Scholar]
- 25.Chen B, Liu S, Xu W, Wang X, Zhao W, Wu J. IGF-I and IGFBP-3 and the risk of lung cancer: a meta-analysis based on nested case-control studies. J Exp Clin Cancer Res. 2009;28:89. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Rowlands MA, Gunnell D, Harris R, Vatten LJ, Holly JM, Martin RM. Circulating insulin-like growth factor peptides and prostate cancer risk: a systematic review and meta-analysis. Int J Cancer. 2009;124:2416–29. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Key TJ, Appleby PN, Reeves GK, Roddam AW. Insulin-like growth factor 1 (IGF1), IGF binding protein 3 (IGFBP3), and breast cancer risk: pooled individual data analysis of 17 prospective studies. Lancet Oncol. 2010;11:530–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Gutierrez J, Jimenez A, de Dios Luna J, Soto MJ, Sorlozano A. Meta-analysis of studies analyzing the relationship between bladder cancer and infection by human papillomavirus. J Urol. 2006;176:2474–81; discussion 81. [DOI] [PubMed] [Google Scholar]
- 29.Zhao YS, Wang F, Chang D, Han B, You DY. Meta-analysis of different test indicators: Helicobacter pylori infection and the risk of colorectal cancer. Int J Colorectal Dis. 2008;23:875–82. [DOI] [PubMed] [Google Scholar]
- 30.Mandelblatt JS, Kanetsky P, Eggert L, Gold K. Is HIV infection a cofactor for cervical squamous cell neoplasia? Cancer Epidemiol Biomarkers Prev. 1999;8:97–106. [PubMed] [Google Scholar]
- 31.Zhang ZF, Begg CB. Is Trichomonas vaginalis a cause of cervical neoplasia? Results from a combined analysis of 24 studies. Int J Epidemiol. 1994;23:682–90. [DOI] [PubMed] [Google Scholar]
- 32.Islami F, Kamangar F. Helicobacter pylori and esophageal cancer risk: a meta-analysis. Cancer Prev Res (Phila). 2008;1:329–38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Huang JQ, Zheng GF, Sumanac K, Irvine EJ, Hunt RH. Meta-analysis of the relationship between cagA seropositivity and gastric cancer. Gastroenterology. 2003;125:1636–44. [DOI] [PubMed] [Google Scholar]
- 34.Zhuo XL, Wang Y, Zhuo WL, Zhang XY. Possible association of Helicobacter pylori infection with laryngeal cancer risk: an evidence-based meta-analysis. Arch Med Res. 2008;39:625–8. [DOI] [PubMed] [Google Scholar]
- 35.Hobbs CG, Sterne JA, Bailey M, Heyderman RS, Birchall MA, Thomas SJ. Human papillomavirus and head and neck cancer: a systematic review and meta-analysis. Clin Otolaryngol. 2006;31:259–66. [DOI] [PubMed] [Google Scholar]
- 36.Donato F, Boffetta P, Puoti M. A meta-analysis of epidemiological studies on the combined effect of hepatitis B and C virus infections in causing hepatocellular carcinoma. Int J Cancer. 1998;75:347–54. [DOI] [PubMed] [Google Scholar]
- 37.Zhuo WL, Zhu B, Xiang ZL, Zhuo XL, Cai L, Chen ZT. Assessment of the relationship between Helicobacter pylori and lung cancer: a meta-analysis. Arch Med Res. 2009;40:406–10. [DOI] [PubMed] [Google Scholar]
- 38.Taylor ML, Mainous AG 3rd, Wells BJ. Prostate cancer and sexually transmitted diseases: a meta-analysis. Fam Med. 2005;37:506–12. [PubMed] [Google Scholar]
- 39.Wang C, Yuan Y, Hunt RH. The association between Helicobacter pylori infection and early gastric cancer: a meta-analysis. Am J Gastroenterol. 2007;102:1789–98. [DOI] [PubMed] [Google Scholar]
- 40.Heikkila K, Harris R, Lowe G, Rumley A, Yarnell J, Gallacher J, et al. Associations of circulating C-reactive protein and interleukin-6 with cancer risk: findings from two prospective cohorts and a meta-analysis. Cancer Causes Control. 2009;20:15–26. [DOI] [PubMed] [Google Scholar]
- 41.Tsilidis KK, Branchini C, Guallar E, Helzlsouer KJ, Erlinger TP, Platz EA. C-reactive protein and colorectal cancer risk: a systematic review of prospective studies. Int J Cancer. 2008;123:1133–40. [DOI] [PubMed] [Google Scholar]
- 42.Barba M, Yang L, Schunemann HJ, Sperati F, Grioni S, Stranges S, et al. Urinary estrogen metabolites and prostate cancer: a case-control study and meta-analysis. J Exp Clin Cancer Res. 2009;28:135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Roddam AW, Allen NE, Appleby P, Key TJ. Endogenous sex hormones and prostate cancer: a collaborative analysis of 18 prospective studies. J Natl Cancer Inst. 2008;100:170–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Key T, Appleby P, Barnes I, Reeves G. Endogenous sex hormones and breast cancer in postmenopausal women: reanalysis of nine prospective studies. J Natl Cancer Inst. 2002;94:606–16. [DOI] [PubMed] [Google Scholar]
- 45.Orwin RG. A fail-safe n for effct size in meta-analysis. American Educational Research Association. 1983;8:157–59. [Google Scholar]
- 46.Schistosomes, liver flukes and Helicobacter pylori. IARC Working Group on the Evaluation of Carcinogenic Risks to Humans. Lyon, 7–14 June 1994. IARC Monogr Eval Carcinog Risks Hum. 1994;61:1–241. [PMC free article] [PubMed] [Google Scholar]
- 47.Higgins JP GSe. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from http://www.cochrane-handbook.org/. [Google Scholar]
- 48.Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Introduction to meta-analysis: Wiley; 2009. [Google Scholar]
- 49.Tsugane S, Sasazuki S. Diet and the risk of gastric cancer: review of epidemiological evidence. Gastric Cancer. 2007;10:75–83. [DOI] [PubMed] [Google Scholar]
- 50.Gaddy JA, Radin JN, Loh JT, Zhang F, Washington MK, Peek RM Jr., et al. High dietary salt intake exacerbates Helicobacter pylori-induced gastric carcinogenesis. Infect Immun. 2013;81:2258–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Loh JT, Torres VJ, Cover TL. Regulation of Helicobacter pylori cagA expression in response to salt. Cancer Res. 2007;67:4709–15. [DOI] [PubMed] [Google Scholar]
- 52.Brawley OW, Thompson IM. Chemoprevention of prostate cancer. Urology. 1994;43:594–9. [DOI] [PubMed] [Google Scholar]
- 53.Thompson IM, Goodman PJ, Tangen CM, Lucia MS, Miller GJ, Ford LG, et al. The influence of finasteride on the development of prostate cancer. N Engl J Med. 2003;349:215–24. [DOI] [PubMed] [Google Scholar]
- 54.Platz EA, Giovannucci E. The epidemiology of sex steroid hormones and their signaling and metabolic pathways in the etiology of prostate cancer. J Steroid Biochem Mol Biol. 2004;92:237–53. [DOI] [PubMed] [Google Scholar]
- 55.Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374:86–9. [DOI] [PubMed] [Google Scholar]
- 56.Vandenbroucke JP, von Elm E, Altman DG, Gotzsche PC, Mulrow CD, Pocock SJ, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. Int J Surg. 2014;12:1500–24. [DOI] [PubMed] [Google Scholar]
- 57.Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151:264–9, w64. [DOI] [PubMed] [Google Scholar]
- 58.Hill AB. THE ENVIRONMENT AND DISEASE: ASSOCIATION OR CAUSATION? Proc R Soc Med. 1965;58:295–300. [PMC free article] [PubMed] [Google Scholar]
- 59.Minelli C, Baio G. Value of Information: A Tool to Improve Research Prioritization and Reduce Waste. PLoS Med. 2015;12:e1001882. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, Ioannidis JP, et al. Biomedical research: increasing value, reducing waste. Lancet. 2014;383:101–4. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.