Skip to main content
PMC home page
Advances in Nutrition logoLink to Advances in Nutrition
. 2021 Sep 2;13(1):48–65. doi: 10.1093/advances/nmab095

Evaluating Concordance of Bodies of Evidence from Randomized Controlled Trials, Dietary Intake, and Biomarkers of Intake in Cohort Studies: A Meta-Epidemiological Study

Jessica Beyerbach 1, Julia Stadelmaier 2, Georg Hoffmann 3, Sara Balduzzi 4, Nils Bröckelmann 5, Lukas Schwingshackl 6,✉
PMCID: PMC8803500  PMID: 34308960

ABSTRACT

We aimed to identify and compare empirical data to determine the concordance of diet–disease effect estimates of bodies of evidence (BoE) from randomized controlled trials (RCTs), dietary intake, and biomarkers of dietary intake in cohort studies (CSs). The Cochrane Database of Systematic Reviews and MEDLINE were searched for systematic reviews (SRs) of RCTs and SRs of CSs that investigated both dietary intake and biomarkers of intake published between 1 January 2010 and 31 December 2019. For matched diet–disease associations, the concordance between results from the 3 different BoE was analyzed using 2 definitions: qualitative (e.g., 95% CI within a predefined range) and quantitative (test hypothesis on the z score). Moreover, the differences in the results coming from BoERCTs, BoECSs dietary intake, and BoECSs biomarkers were synthesized to get a pooled ratio of risk ratio (RRR) across all eligible diet–disease associations, so as to compare the 3 BoE. Overall, 49 diet–disease associations derived from 41 SRs were identified and included in the analysis. Twenty-four percent, 10%, and 39% of the diet–disease associations were qualitatively concordant comparing BoERCTs with BoECSs dietary intake, BoERCTs with BoECSs biomarkers, and comparing both BoE from CSs, respectively; 88%, 69%, and 90% of the diet–disease associations were quantitatively concordant comparing BoERCTs with BoECSs dietary intake, BoERCTs with BoECSs biomarkers, and comparing both BoE from CSs, respectively. The pooled RRRs comparing effects from BoERCTs with effects from BoECSs dietary intake were 1.09 (95% CI: 1.06, 1.13) and 1.18 (95% CI: 1.10, 1.25) compared with BoECSs biomarkers. Comparing both BoE from CSs, the difference in the results was also small (RRR: 0.92; 95% CI: 0.88, 0.96). Our findings suggest that BoE from RCTs and CSs are often quantitatively concordant. Prospective SRs in nutrition research should include, whenever possible, BoE from RCTs and CSs on dietary intake and biomarkers of intake to provide the whole picture for an investigated diet–disease association.

Keywords: meta-epidemiological, dietary intake, biomarkers of intake, concordance, randomized controlled trials, cohort studies

Statement of Significance: Our findings suggest that bodies of evidence from randomized controlled trials and cohort studies are often concordant.

Introduction

The Global Burden of Disease (GBD) Study Group showed that noncommunicable diseases (NCDs) accounted for nearly 75% of deaths worldwide (1), and evidence from prospective cohort studies (CSs) showed that suboptimal diet accounted for 22% of all deaths worldwide (2). Bodies of evidence (BoE) from CSs with clinical outcomes provide valuable insights into diet–disease relations and are the most important evidence source of GBD reports and dietary guidelines (2, 3). However, nutritional epidemiology has been criticized for providing potentially less trustworthy findings (4). Therefore, limitations of CSs, such as residual confounding and measurement error, need to be considered (4). In CSs, self-reported dietary assessment methods are often used, but have limitations impacting validity and reliability. Dietary biomarkers provide objective verification of self-reported dietary intakes, and can complement and strengthen credibility of diet–disease associations (5). On the other hand, RCTs, if well designed and well conducted, give robust answers to the research questions they address and are widely accepted as the ideal methodology for causal inference (6). However, dietary RCTs often suffer from inherent methodological limitations. In the past, several RCTs comparing dietary interventions with placebo or control interventions have failed to replicate the inverse associations between dietary intake/biomarkers of dietary intake and risk for NCDs found in large-scale CSs (7–10). For example, RCTs found no evidence for a beneficial effect of vitamin E and cardiovascular disease (11). On the contrary, some consistent findings between CSs and RCTs have been reported as well (e.g., omega-3 and stroke risk) (12), but to the best of our knowledge, no systematic evaluation of concordance between the 3 BoE has been conducted so far. This meta-epidemiological study aims to do exactly this, to identify and compare empirical data to determine the extent to which diet–disease effect estimates of BoE from RCTs, CSs on dietary intake, and CSs on biomarkers of dietary intake are concordant or discordant.

Methods

This meta-epidemiological study was planned, written, and reported in adherence to guidelines for reporting meta-epidemiological methodology research (13). Inclusion criteria [patients/population, intervention/exposure, comparator, and outcome (PI/ECO)] are described in Table 1.

TABLE 1.

Detailed description of inclusion criteria

Inclusion criteria
Population Generally healthy participants (adults).
Intervention/exposure (dietary intake and biomarkers of dietary intake) 1. Dietary pattern: e.g., Mediterranean diet, Dietary Approaches to Stop Hypertension (DASH), low-carbohydrate diet.2. Food groups: the following food groups (macro-level) and foods (micro-level) were considered: e.g., grains, vegetables, fruit, milk and dairy products, meat, processed meat, fish, eggs, nuts, chocolate, oils. 3. Macronutrients: carbohydrate (starch, fructose, glucose, sucrose); fat: e.g., n–3 fatty acids (EPA, DHA, ɑ-linolenic acid); n–6 fatty acids (linoleic acid); monounsaturated fat; protein (e.g., amino acids).. 4. Micronutrients: vitamins: B-carotene; vitamins A, E, C (ascorbic acid), and D (cholecalciferol, ergocalciferol); B vitamins (thiamin, riboflavin, niacin, pyridoxine, cobalamin, folic acid); minerals: magnesium, calcium, selenium, sodium, potassium, iron, zinc, copper, iodine.5. Other: fiber (psyllium, inulin, cellulose); probiotics; prebiotics; and synbiotics.
Control/comparison 1. Low (no) intake and status level of the above interventions/exposure. 2. Placebo/usual care.
Outcomes For example, all-cause mortality, cardiovascular disease, coronary heart disease (myocardial infarction, ischemic heart disease, and acute coronary syndrome), stroke, cancer, type 2 diabetes, dementia, fractures, age-related macular degeneration, anthropometric outcomes; important intermediate disease markers such systolic blood pressure and diastolic blood pressure, fasting glucose, and LDL cholesterol.
Study design 1. Systematic reviews of randomized controlled trials.2. Matching systematic reviews of cohort studies investigating both dietary intake and biomarkers of dietary intake: cohort studies (if available, prospective cohorts were preferred).
Open in a new tab

Identification of systematic reviews of RCTs

Search strategy

We searched for systematic reviews (SRs) of RCTs, considering the inclusion criteria, in the Cochrane Database of Systematic Reviews (Supplemental Appendix 1) and MEDLINE (hand search), published within 1 January 2010 and 31 December 2019 (Supplemental Figure 1). Screening of titles/abstracts was done by 1 reviewer (LS), and then in the second stage, all potentially relevant full papers were screened for inclusion by 2 reviewers independently (JB, LS) using an inclusion/exclusion form specifically developed for the present study. By hand-searching, 17 additional SRs of RCTs were identified [12 of them were included (12, 14–24) and one of them was published in 2020 (16)]. Discrepancies were resolved by an additional reviewer.

Identification of matching SRs of CSs

Search strategy

After all potentially relevant SRs of RCTs were identified, we searched for matching SRs of CSs. First, we screened all eligible SRs of RCTs, whether or not they also included CSs. Second, we conducted searches for matching SRs of CSs (only SRs of CSs were included, which investigated diet–disease associations on both dietary intake and biomarkers of intake for the same outcome) in MEDLINE, published within the last 10 y (Supplemental Appendix 2 and Supplemental Figure 2). We selected a time period of 10 y to ensure comparability between the 3 BoE. No language restriction was used. Screening of titles/abstracts was conducted by 1 reviewer (LS), and then, in the second stage, all potentially relevant full papers were screened for inclusion by 2 reviewers (LS, JB). By hand-searching, an additional 3 matching SRs of CSs were identified (24–26) [one of them was published in 2020 (26)]. The most appropriate (investigating similar PI/ECO) and comprehensive (most recent) matching SRs of CSs were selected for inclusion. For each eligible SR of RCTs we matched a maximum of 3 outcomes for a given intervention/exposure. Furthermore, in the Cochrane Reviews, the selection of outcomes was based on the ranking in the Summary of Findings tables (from top to bottom).

Data extraction

We extracted the following data for each included diet–disease association of a BoERCTs, BoECSs dietary intake, and BoECSs biomarkers: name of first author, year of publication, intervention/exposure, outcome, effect estimates [risk ratio (RR), HR, OR, 95% CI], type of comparison (e.g., high vs. low, dose-response), number of studies, number of participants, and number of cases included. The data were extracted by 2 reviewers (LS, JB) using a piloted data extraction form.

In cases where a BoE reported effect estimates based on a pool of studies of variable design (i.e., case-control, cross-sectional studies, retrospective cohort studies, or quasi-RCTs), we recalculated the pooled effect estimates by excluding non-cohort/non-RCT studies, while retaining the CSs/RCTs fulfilling our inclusion criteria. The meta-analyses were recalculated by combining the RRs of the corresponding study designs based on a random-effects model using the DerSimonian-Laird method (27). Using an inverse-variance method, the SE for the log-transformed RRs was calculated and interpreted as an estimated variance of log-transformed RR to weight each study. Also, if an intervention in a BoE of RCTs (e.g., low vs. high sodium) and an exposure in a BoE of CSs (e.g., high vs. low sodium) investigated opposite comparisons, we recalculated the risk estimates, respectively (e.g., low vs. high sodium). The analyses were conducted using the Review Manager by the Cochrane Collaboration (version 5.4) (28).

Methodological quality

The methodological quality of eligible SRs was evaluated using AMSTAR 2 (A Measurement Tool to Assess systematic Reviews, version 2) (29). Each SR was assessed based on 16 predefined items and rated in 1 of 4 categories (high, moderate, low, or critically low) according to the presence of critical and noncritical weaknesses. This assessment was done by 1 reviewer (JS).

Statistical analysis

The concordance between results from the eligible BoERCTs, BoECSs dietary intake, and BoECSs biomarkers was assessed using 2 definitions (qualitative, quantitative). We defined as qualitatively concordant effect estimates of the outcome-specific BoERCTs, BoECSs dietary intake, and BoECSs biomarkers that were statistically significant (at the 0.05 level) and were in the same direction (e.g., all RRs suggesting lower risk of disease). We defined qualitative concordant also effect estimates that were both not statistically significant with the 95% CI fully within the range of 0.80 to 1.25 (30). We also performed a secondary qualitative concordance analysis, considering a wider range for the CI (i.e., 0.70 to 1.30). If the effect estimate of the BoERCTs was expressed with a different measure than the effect estimate of a BoECSs dietary intake, or BoECSs biomarkers, the appropriate conversion formulas were used to have the estimates expressed in the same measure [e.g., RR and OR: the relevant formula requires an assumed control risk (ACR): Inline graphic] (31). The measure of quantitative concordance between effects estimates of the outcome-specific BoERCTs, BoECSs dietary intake, and BoECSs biomarkers was calculated as follows (32, 33):

graphic file with name TM0002.gif (1)

 

graphic file with name TM0003.gif (2)

 

graphic file with name TM0004.gif (3)

We defined quantitative concordant results if the P value associated to the z was ≥0.017—that is, 0.5/3 (i.e., applying a Bonferroni correction). Moreover, we synthesized the differences in the results coming from BoERCTs, BoECSs dietary intake, and BoECSs biomarkers to get a pooled difference across all eligible outcome pairs and compare the 3 BoE. These were expressed as ratio of risk ratios (RRRs) (33). By using BoECSs dietary intake as the reference group, we examined the pooled estimate to see whether there was a relative larger (effect BoERCTs > effect BoECSs dietary intake) or smaller (effect BoERCTs < effect BoECSs dietary intake) estimate coming from BoE of RCTs. This procedure was adopted for the other 2 comparisons: BoERCTs vs. BoECSs biomarkers, and BoECSs biomarkers vs. BoECSs dietary intake, respectively. We conducted a priori planned subgroup analyses: type of intervention/exposure, outcome. Finally, we also conducted a sensitivity analysis by using risk estimates for BoE from RCTs from SRs of CSs that included also RCTs. The pooled estimates were obtained through a random-effects meta-analysis model. We assessed heterogeneity through the I2 and τ2 statistics (34, 35). The τ2 was estimated by the Paule and Mandel method (36, 37). Furthermore, the 95% prediction intervals (PIs) were obtained in order to show the range of possible values for the difference in the results between the different BoE that might be observed in future comparisons. These meta-analyses were performed using the R package meta (38).

Results

Overall, 20 SRs of RCTs (12, 14–24, 39–46) and 25 matching SRs of CSs were included in this study (12, 19, 24–26, 39, 47–65). One Cochrane Review and 3 SRs of RCTs also included CSs (12, 19, 24, 39). Forty-nine diet–disease associations were included. Ten pooled estimates from 6 SRs were recalculated (24, 39, 49, 51, 60, 62). The number of primary studies contributing to the 49 diet–disease outcome pairs ranged from 1 to 64 (median: 5) for BoE from RCTs, and between 1 and 16 (median: 7) for BoE from CSs. The total number of participants ranged from 122 to 211,957 for BoE from RCTs, and from 1414 to 1,012,099 for BoE from CSs.

The interventions/exposures investigated in the identified SRs can be categorized into micronutrients (n = 33), fatty acids (n = 15), and phytonutrients (n = 1) and the outcomes cluster included the following: cancer (n = 19), cardiovascular disease (n = 19), overall mortality (n = 7), diabetes (n = 2), neurodegenerative disease (n = 1), and pregnancy outcomes (n = 1).

Tables 2 and 3 show the concordance between the summary effects of the 3 BoE using the 2 definitions. Supplemental Tables 2–4 shows the general study characteristics.

TABLE 2.

Analysis of qualitative concordance of the 49 included diet–disease outcomes1

Qualitative concordance
Study (reference) Intervention/exposure category Outcome category Summary measure Direction of effect (RCTs) Significance (RCTs) Direction of association: CS dietary intake Significance (CSs) Direction of association: CS biomarkers Significance (CSs) RCTs vs. CSs: dietary intake RCTs vs. CSs: biomarker CS biomarker vs. CS dietary intake Overall
Abdelhamid et al. 2018 (40) + Chowdhury et al. 2014 (47) Omega-3 Cardiovascular disease RR Decreasing Not sign Decreasing Sign Decreasing Not sign Not concordant Not concordant Not concordant Not concordant
Abdelhamid et al. 2018 (40) + Chowdhury et al. 2014 (47) Omega-3 Cardiovascular mortality RR Decreasing Not sign Decreasing Not sign Decreasing Sign Not concordant Not concordant Not concordant Not concordant
Abdelhamid et al. 2018 (40) + Pan et al. 2012 (48) α-Linolenic acid Cardiovascular disease RR Decreasing Not sign Decreasing Not sign Decreasing Not sign Concordant Not concordant Not concordant Not concordant
Adler et al. 2014 (41) + Aburto et al. 2013 (49) Low sodium All-cause mortality RR Decreasing Not sign Decreasing Not sign Decreasing Not sign Not concordant Concordant Not concordant Not concordant
Adler et al. 2014 (41) + Aburto et al. 2013 (49) Low sodium Cardiovascular disease RR Decreasing Not sign Decreasing Not sign Decreasing Not sign Not concordant Not concordant Not concordant Not concordant
Adler et al. 2014 (41) + Aburto et al. 2013 (49) Low sodium Cardiovascular mortality RR Decreasing Not sign Decreasing Not sign Decreasing Not sign Not concordant Not concordant Not concordant Not concordant
Bjelakovic et al. 2012 (42) + Aune et al. 2018 (50) Vitamin C All-cause mortality RR Increasing Not sign Decreasing Sign Decreasing Sign Not concordant Not concordant Concordant Not concordant
Bjelakovic et al. 2012 (42) + Aune et al. 2018 (50) Vitamin E All-cause mortality RR Increasing Not sign Decreasing Not sign Decreasing Not sign Concordant Not concordant Not concordant Not concordant
Bjelakovic et al. 2012 (42) + Aune et al. 2018 (50) β-Carotene All-cause mortality RR Increasing Not sign Decreasing Sign Decreasing Sign Not concordant Not concordant Concordant Not concordant
Bjelakovic et al. 2014 (46) + Hossain et al. 2019 (51) Vitamin D Breast cancer RR Decreasing Not sign Decreasing Not sign Increasing Not sign Concordant Not concordant Not concordant Not concordant
Bjelakovic et al. 2014 (46) + Touvier et al. 2011 (52) Vitamin D Colorectal cancer RR Increasing Not sign Decreasing Sign Decreasing Sign Not concordant Not concordant Concordant Not concordant
Bjelakovic et al. 2014 (46) + Zhang et al. 2015 (53) Vitamin D Lung cancer RR Decreasing Not sign Decreasing Not sign Decreasing Sign Not concordant Not concordant Not concordant Not concordant
Brown et al. 2019 (14) + Wu et al. 2012 (54) Omega-3 Type 2 diabetes RR Same effect Not sign Increasing Not sign Decreasing Not sign Concordant Concordant Concordant Concordant
Brown et al. 2019 (14) + Wu et al. 2012 (54) α-Linolenic acid Type 2 diabetes RR Decreasing Not sign Decreasing Not sign Decreasing Not sign Not concordant Not concordant Not concordant Not concordant
Chowdhury et al. 2012 (12) + Chowdhury et al. 2012 (12) Omega-3 Stroke RR Increasing Not sign Decreasing Not sign Increasing Not sign Concordant Concordant Concordant Concordant
Druesne-Pecollo et al. 2010 (15) + Aune et al. 2012 (55) β-Carotene Breast cancer RR Decreasing Not sign Decreasing Sign Decreasing Not sign Not concordant Not concordant Not concordant Not concordant
Druesne-Pecollo et al. 2010 (15) + Aune et al. 2018 (50) β-Carotene Cancer RR Increasing Not sign Decreasing Not sign Decreasing Sign Concordant Not concordant Not concordant Not concordant
Hanson et al. 2020 (16) + Alexander et al. 2015 (56) Omega-3 Prostate cancer RR Increasing Not sign Same effect Not sign Increasing Not sign Concordant Not concordant Concordant Not concordant
Hanson et al. 2020 (16) + Cao et al. 2016 (57) Omega-6 Breast cancer RR Same effect Not sign Increasing Not sign Decreasing Not sign Not concordant Not concordant Not concordant Not concordant
Hanson et al. 2020 (16) + Cao et al. 2016 (57) PUFA Breast cancer RR Increasing Not sign Increasing Not sign Decreasing Not sign Not concordant Not concordant Not concordant Not concordant
Hanson et al. 2020 (16) + Fu et al. 2015 (58) α-Linolenic acid Prostate cancer RR Increasing Not sign Decreasing Not sign Same effect Not sign Not concordant Not concordant Concordant Not concordant
Hanson et al. 2020 (16) + Zheng et al. 2013 (59) Omega-3 Breast cancer RR Increasing Not sign Decreasing Sign Decreasing Not sign Not concordant Not concordant Not concordant Not concordant
Hooper et al. 2018 (43) + Chowdhury et al. 2014 (47) Omega-6 Cardiovascular disease RR Decreasing Not sign Decreasing Not sign Decreasing Not sign Concordant Concordant Concordant Concordant
Hooper et al. 2018 (43) + Li et al. 2020 (26) Omega-6 All-cause mortality RR Same effect Not sign Decreasing Sign Decreasing Sign Not concordant Not concordant Concordant Not concordant
Hooper et al. 2018 (43) + Li et al. 2020 (26) Omega-6 Cardiovascular mortality RR Increasing Not sign Decreasing Sign Decreasing Sign Not concordant Not concordant Concordant Not concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) Vitamin C Cardiovascular disease RR Decreasing Not sign Decreasing Sign Decreasing Sign Not concordant Not concordant Concordant Not concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) Vitamin C Cardiovascular mortality RR Increasing Not sign Decreasing Sign Decreasing Sign Not concordant Not concordant Concordant Not concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) Vitamin C Stroke RR Decreasing Not sign Decreasing Sign Decreasing Sign Not concordant Not concordant Concordant Not concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) Vitamin E Cardiovascular disease RR Decreasing Not sign Decreasing Not sign Decreasing Not sign Not concordant Not concordant Not concordant Not concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) Vitamin E Cardiovascular mortality RR Decreasing Not sign Same effect Not sign Decreasing Not sign Concordant Concordant Concordant Concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) Vitamin E Stroke RR Decreasing Not sign Decreasing Not sign Decreasing Sign Not concordant Not concordant Not concordant Not concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) β-Carotene Cardiovascular disease RR Increasing Not sign Decreasing Not sign Decreasing Sign Concordant Not concordant Not concordant Not concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) β-Carotene Coronary heart disease RR Increasing Not sign Decreasing Sign Decreasing Sign Not concordant Not concordant Concordant Not concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) β-Carotene Stroke RR Increasing Not sign Decreasing Sign Decreasing Not sign Not concordant Not concordant Not concordant Not concordant
Jenkins et al. 2018 (18) + Hunnicutt et al. 2014 (60) Iron Cardiovascular mortality RR Decreasing Not sign Decreasing Not sign Same effect Not sign Not concordant Not concordant Not concordant Not concordant
Jenkins et al. 2018 (18) + Hunnicutt et al. 2014 (60) Iron Myocardial infarction RR Decreasing Not sign Decreasing Not sign Increasing Sign Not concordant Not concordant Not concordant Not concordant
Khan et al. 2019 (17) + Chen et al. 2016 (61) Omega-3 All-cause mortality RR Decreasing Not sign Decreasing Sign Decreasing Sign Not concordant Not concordant Concordant Not concordant
Moazzen et al. 2018 (19) + Moazzen et al. 2018 (19) Folate Colorectal cancer RR Increasing Not sign Decreasing Not sign Decreasing Sign Not concordant Not concordant Not concordant Not concordant
Park et al. 2017 (20) + Wu et al. 2020 (62) Vitamin A/carotenoids Bladder cancer RR Decreasing Not sign Increasing Not sign Decreasing Not sign Not concordant Not concordant Not concordant Not concordant
Rees et al. 2013 (45) + Jayedi et al. 2018 (63) Selenium All-cause mortality RR Decreasing Not sign Decreasing Sign Decreasing Sign Not concordant Not concordant Concordant Not concordant
Rutjes et al. 2018 (44) + Doets et al. 2013 (64) B-vitamins Dementia/MCI RR Increasing Not sign Decreasing Not sign Same effect Not sign Not concordant Not concordant Concordant Not concordant
Schwingshackl et al. 2017 (22) + Aune et al. 2018 (50) Vitamin C Cancer RR Decreasing Not sign Decreasing Sign Decreasing Sign Not concordant Not concordant Concordant Not concordant
Schwingshackl et al. 2017 (22) + Aune et al. 2018 (50) Vitamin E Cancer RR Increasing Not sign Increasing Not sign Decreasing Sign Concordant Not concordant Not concordant Not concordant
van Die et al. 2014 (21) + Applegate et al. 2018 (25) Isoflavones Prostate cancer RR Decreasing Sign Decreasing Sign Decreasing Not sign Concordant Not concordant Not concordant Not concordant
Vinceti 2018 (39) + Vinceti 2018 (39) Selenium Cancer mortality RR Decreasing Not sign Decreasing Not sign Decreasing Not sign Not concordant Not concordant Not concordant Not concordant
Vinceti et al. 2018 (39) + Vinceti et al. 2018 (39) Selenium Lung cancer RR Increasing Not sign Increasing Not sign Decreasing Not sign Not concordant Not concordant Not concordant Not concordant
Vinceti et al. 2018 (39) + Vinceti et al. 2018 (39) Selenium Prostate cancer RR Decreasing Not sign Decreasing Not sign Decreasing Sign Not concordant Not concordant Not concordant Not concordant
Vollset et al. 2013 (23) + Wang et al. 2014 (65) Folate Prostate cancer RR Increasing Not sign Increasing Not sign Increasing Sign Not concordant Not concordant Not concordant Not concordant
Zhao et al. 2017 (24) + Zhao et al. 2017 (24) Iron Gestational diabetes RR Decreasing Not sign Increasing Not sign Increasing Sign Not concordant Not concordant Not concordant Not concordant
Open in a new tab

1CS, cohort study; MCI, mild cognitive impairment; RCT, randomized controlled trial; RR, risk ratio; sign, significant.

TABLE 3.

Analysis of quantitative concordance of the 49 included diet–disease outcomes1

Intervention/ exposure category RCTs vs. CS dietary intake (reference) RCTs vs. CS biomarkers (reference) CS biomarkers vs. CS dietary intake (reference) Overall quantitative concordance
Study (reference) Outcome category Summary measure z P Quantitative concordance z P Quantitative concordance z P Quantitative concordance
Abdelhamid et al. 2018 (40) + Chowdhury et al. 2014 (47) Omega-3 Cardiovascular disease RR 2.1078 0.0350 Concordant 1.1194 0.2630 Concordant −0.2267 0.8207 Concordant Concordant
Abdelhamid et al. 2018 (40) + Chowdhury et al. 2014 (47) Omega-3 Cardiovascular mortality RR 0.4107 0.6813 Concordant 2.1138 0.0345 Concordant −1.3531 0.1760 Concordant Concordant
Abdelhamid et al. 2018 (40) + Pan et al. 2012 (48) α-Linolenic acid Cardiovascular disease RR 0.2619 0.7934 Concordant 0.6939 0.4877 Concordant −0.5427 0.5873 Concordant Concordant
Adler et al. 2014 (41) + Aburto et al. 2013 (49) Low sodium All-cause mortality RR 0.0635 0.9493 Concordant 0.2061 0.8367 Concordant −0.0641 0.9489 Concordant Concordant
Adler et al. 2014 (41) + Aburto et al. 2013 (49) Low sodium Cardiovascular disease RR −0.6326 0.5270 Concordant −0.9437 0.3453 Concordant 0.2262 0.8211 Concordant Concordant
Adler et al. 2014 (41) + Aburto et al. 2013 (49) Low sodium Cardiovascular mortality RR −1.0100 0.3125 Concordant −1.4874 0.1369 Concordant 0.5481 0.5836 Concordant Concordant
Bjelakovic et al. 2012 (42) + Aune et al. 2018 (50) Vitamin C All-cause mortality RR 3.9226 0.0001 Not concordant 5.9248 0.0000 Not concordant −3.2194 0.0013 Not concordant Not concordant
Bjelakovic et al. 2012 (42) + Aune et al. 2018 (50) Vitamin E All-cause mortality RR 1.2837 0.1993 Concordant 1.3085 0.1907 Concordant −0.8978 0.3693 Concordant Concordant
Bjelakovic et al. 2012 (42) + Aune et al. 2018 (50) β-Carotene All-cause mortality RR 6.1042 0.0000 Not concordant 3.7773 0.0002 Not concordant −1.7237 0.0848 Concordant Not concordant
Bjelakovic et al. 2014 (46) + Hossain et al. 2019 (51) Vitamin D Breast cancer RR 0.4315 0.6661 Concordant −0.2332 0.8156 Concordant 0.3874 0.6984 Concordant Concordant
Bjelakovic et al. 2014 (46) + Touvier et al. 2011 (52) Vitamin D Colorectal cancer RR 1.6070 0.1080 Concordant 1.5081 0.1315 Concordant 0.6722 0.5015 Concordant Concordant
Bjelakovic et al. 2014 (46) + Zhang et al. 2015 (53) Vitamin D Lung cancer RR −0.2370 0.8126 Concordant 0.2989 0.7650 Concordant −0.6984 0.4850 Concordant Concordant
Brown et al. 2019 (14) + Wu et al. 2012 (54) Omega-3 Type 2 diabetes RR −0.4019 0.6877 Concordant 0.5668 0.5708 Concordant −1.1198 0.2628 Concordant Concordant
Brown et al. 2019 (14) + Wu et al. 2012 (54) α-Linolenic acid Type 2 diabetes RR −0.8097 0.4181 Concordant −0.6565 0.5115 Concordant −0.5302 0.5960 Concordant Concordant
Chowdhury et al. 2012 (12) + Chowdhury et al. 2012 (12) Omega-3 Stroke RR 1.8137 0.0697 Concordant −0.1124 0.9105 Concordant 1.5307 0.1258 Concordant Concordant
Druesne-Pecollo et al. 2010 (15) + Aune et al. 2012 (55) β-Carotene Breast cancer RR 0.4454 0.6560 Concordant 1.1240 0.2610 Concordant −0.9923 0.3211 Concordant Concordant
Druesne-Pecollo et al. 2010 (15) + Aune et al. 2018 (50) β-Carotene Cancer RR 2.0646 0.0390 Concordant 3.4857 0.0005 Not concordant −1.7517 0.0798 Concordant Not concordant
Hanson et al. 2020 (16) + Alexander et al. 2015 (56) Omega-3 Prostate cancer RR 1.2777 0.2014 Concordant 0.3130 0.7543 Concordant 0.9106 0.3625 Concordant Concordant
Hanson et al. 2020 (16) + Cao et al. 2016 (57) Omega-6 Breast cancer RR −0.0950 0.9243 Concordant 0.1724 0.8631 Concordant −1.3013 0.1932 Concordant Concordant
Hanson et al. 2020 (16) + Cao et al. 2016 (57) PUFA Breast cancer RR 0.2402 0.8102 Concordant 1.3714 0.1703 Concordant −1.4291 0.1530 Concordant Concordant
Hanson et al. 2020 (16) + Fu et al. 2015 (58) α-Linolenic acid Prostate cancer RR 0.9125 0.3615 Concordant 0.8782 0.3798 Concordant 0.7336 0.4632 Concordant Concordant
Hanson et al. 2020 (16) + Zheng et al. 2013 (59) Omega-3 Breast cancer RR 1.9849 0.0472 Concordant 1.4817 0.1384 Concordant 0.1044 0.9169 Concordant Concordant
Hooper et al. 2018 (43) + Chowdhury et al. 2014 (47) Omega-6 Cardiovascular disease RR −0.1039 0.9172 Concordant 0.2928 0.7697 Concordant −0.5744 0.5657 Concordant Concordant
Hooper et al. 2018 (43) + Li et al. 2020 (26) Omega-6 All-cause mortality RR 1.9263 0.0541 Concordant 1.7234 0.0848 Concordant −0.1586 0.8740 Concordant Concordant
Hooper et al. 2018 (43) + Li et al. 2020 (26) Omega-6 Cardiovascular mortality RR 1.2241 0.2209 Concordant 1.6912 0.0908 Concordant −1.3087 0.1906 Concordant Concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) Vitamin C Cardiovascular disease RR 2.4667 0.0136 Not concordant 2.9465 0.0032 Not concordant −1.9764 0.0481 Concordant Not concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) Vitamin C Cardiovascular mortality RR 2.3164 0.0205 Concordant 3.1514 0.0016 Not concordant −1.8327 0.0669 Concordant Not concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) Vitamin C Stroke RR 0.9535 0.3404 Concordant 3.2193 0.0013 Not concordant −3.0516 0.0023 Not concordant Not concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) Vitamin E Cardiovascular disease RR 0.8055 0.4205 Concordant 1.1061 0.2687 Concordant −0.6999 0.4840 Concordant Concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) Vitamin E Cardiovascular mortality RR −1.4151 0.1570 Concordant −0.5498 0.5825 Concordant −0.4416 0.6588 Concordant Concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) Vitamin E Stroke RR 1.1189 0.2632 Concordant 2.9136 0.0036 Not concordant −1.8969 0.0578 Concordant Not concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) β-Carotene Cardiovascular disease RR 0.5698 0.5688 Concordant 3.2045 0.0014 Not concordant −2.3571 0.0184 Concordant Not concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) β-Carotene Coronary heart disease RR 3.8766 0.0001 Not concordant 2.5008 0.0124 Not concordant 0.0000 1.0000 Concordant Not concordant
Jenkins et al. 2018 (18) + Aune et al. 2018 (50) β-Carotene Stroke RR 2.8591 0.0042 Not concordant 2.0692 0.0385 Concordant 0.1108 0.9117 Concordant Not concordant
Jenkins et al. 2018 (18) + Hunnicutt et al. 2014 (60) Iron Cardiovascular mortality RR −0.3417 0.7326 Concordant −0.5920 0.5539 Concordant 0.4799 0.6313 Concordant Concordant
Jenkins et al. 2018 (18) + Hunnicutt et al. 2014 (60) Iron Myocardial infarction RR −1.0240 0.3058 Concordant −1.9709 0.0487 Concordant 2.7827 0.0054 Not concordant Not concordant
Khan et al. 2019 (17) + Chen et al. 2016 (61) Omega-3 All-cause mortality RR 1.6165 0.1060 Concordant 2.6479 0.0081 Not concordant −1.8688 0.0617 Concordant Not concordant
Moazzen et al. 2018 (19) + Moazzen et al. 2018 (19) Folate Colorectal cancer RR 0.6171 0.5372 Concordant 2.5404 0.0111 Not concordant −1.9757 0.0482 Concordant Not concordant
Park et al. 2017 (20) + Wu et al. 2020 (62) Vitamin A Bladder cancer RR −1.4637 0.1433 Concordant 1.0497 0.2939 Concordant −2.1549 0.0312 Concordant Concordant
Rees et al. 2013 (45) + Jayedi et al. 2018 (63) Selenium All-cause mortality RR 3.1535 0.0016 Not concordant 2.9296 0.0034 Not concordant −1.6293 0.1033 Concordant Not concordant
Rutjes et al. 2018 (44) + Doets et al. 2013 (64) B-vitamins Dementia/MCI RR 0.1027 0.9182 Concordant 0.0510 0.9593 Concordant 0.9551 0.3395 Concordant Concordant
Schwingshackl et al. 2017 (22) + Aune et al. 2018 (50) Vitamin C Cancer RR 2.0316 0.0422 Concordant 3.9609 0.0001 Not concordant −2.4631 0.0138 Not concordant Not concordant
Schwingshackl et al. 2017 (22) + Aune et al. 2018 (50) Vitamin E Cancer RR 0.2019 0.8400 Concordant 5.4212 0.0000 Not concordant −3.7900 0.0002 Not concordant Not concordant
van Die et al. 2014 (21) + Applegate et al. 2018 (25) Isoflavones Prostate cancer RR −1.8202 0.0687 Concordant −2.0097 0.0445 Concordant 0.8122 0.4167 Concordant Concordant
Vinceti et al. 2018 (39) + Vinceti et al. 2018 (39) Selenium Cancer mortality RR −0.5342 0.5932 Concordant 0.6774 0.4982 Concordant −1.5258 0.1271 Concordant Concordant
Vinceti et al. 2018 (39) + Vinceti et al. 2018 (39) Selenium Lung cancer RR −1.0873 0.2769 Concordant 0.6450 0.5189 Concordant −1.7522 0.0797 Concordant Concordant
Vinceti et al. 2018 (39) + Vinceti et al. 2018 (39) Selenium Prostate cancer RR −0.6808 0.4960 Concordant 0.3723 0.7096 Concordant −1.3649 0.1723 Concordant Concordant
Vollset et al. 2013 (23) + Wang et al. 2014 (65) Folate Prostate cancer RR 1.4144 0.1573 Concordant −0.4831 0.6290 Concordant 2.1435 0.0321 Concordant Concordant
Zhao et al. 2017 (24) + Zhao et al. 2017 (24) Iron Gestational diabetes RR −1.1649 0.2440 Concordant −2.9681 0.0030 Not concordant −0.1269 0.8990 Concordant Not concordant
Open in a new tab

1CS, cohort study; MCI, mild cognitive impairment; RCT, randomized controlled trial; RR, risk ratio.

Methodological quality of the included SRs

Of the 42 identified SRs, 27 studies (66%) were classified as critically low and 7 (17%) as low, whereas 2 (5%) were classified as moderate and 5 (12%) as high quality. Nine of the SRs including RCTs (45%) and 21 of the SRs including CSs (84%) were rated as critically low. Most SRs did not provide a list of excluded studies (n = 29; 71%) and did not report the presence or registration of a review protocol (n = 25; 61%). Moreover, 31 SRs (76%) did not provide funding information of the original studies, and 20 (49%) did not account for risk of bias in individual studies when discussing the results. Results of the quality assessment are presented in detail in Supplemental Table 5.

Qualitative concordance

In 98% (48/49) of the BoERCTs no statistically significant effect was observed, whereas 65% (32/49) from BoECSs dietary intake and 53% (26/49) from BoECSs biomarkers showed no statistically significant effect. Using the first definition of concordance, 12 (24%), 5 (10%), and 19 (39%) out of 49 eligible of the diet–disease associations were qualitatively concordant comparing BoERCTs with BoECSs dietary intake, BoERCTs with BoECSs biomarkers, and comparing both BoE from CSs, respectively (Table 2). Eight percent (4/49) of the diet–disease associations were qualitatively concordant considering all 3 BoE simultaneously. Considering a wider range for the CIs (i.e., 0.70 to 1.30), 35%, 20%, and 47% of the diet–disease associations were qualitatively concordant comparing BoERCTs with BoECSs dietary intake, BoERCTs with BoECSs biomarkers, and comparing both BoE from CSs, respectively.

Quantitative concordance

Using the second definition (calculated as z score), 88%, 69%, and 90% of the diet–disease associations were quantitatively concordant comparing BoERCTs with BoECSs dietary intake, BoERCTs with BoECSs biomarkers, and comparing both BoE from CSs, respectively (Table 3). Sixty-five percent (32/49) of the diet–disease associations were quantitatively concordant considering all 3 BoE simultaneously.

Pooled estimate

In order to compare overall effect estimates between the RCTs and CSs we calculated the pooled estimates using BoECSs dietary intake and BoECSs biomarkers as the reference group, which showed that overall BoERCTs had larger estimates compared with that of CSs (RRR: 1.09; 95% CI: 1.06, 1.13; I2 = 44%; PI: 0.96 to 1.24; and RRR: 1.18; 95% CI: 1.10, 1.25; I2 = 60%; PI: 0.83 to 1.66) (Figures 1 and 2). By using BoECSs dietary intake as the reference group, we showed that the pooled estimate was a relative smaller estimate (RRR: 0.92; 95% CI: 0.88, 0.96; I2 = 58%; PI: 0.74 to 1.14) coming from BoECSs biomarkers (Figure 3).

FIGURE 1.

FIGURE 1

Open in a new tab

Forest plot of comparisons between bodies of evidence from RCTs vs. cohort studies (on dietary intake: reference) as pooled RRRs. CS, cohort study; diet, dietary intake; MCI, mild cognitive impairment; RCT, randomized controlled trial; RR, risk ratio; RRR, ratio of risk ratio.

FIGURE 2.

FIGURE 2

Open in a new tab

Forest plot of comparisons between bodies of evidence from RCTs vs. cohort studies (on biomarkers of dietary intake: reference) as pooled RRRs. Biom, biomarkers; CS, cohort study; MCI, mild cognitive impairment; RCT, randomized controlled trial; RR, risk ratio; RRR, ratio of risk ratio.

FIGURE 3.

FIGURE 3

Open in a new tab

Forest plot of comparisons between bodies of evidence from cohort studies (on biomarkers of dietary intake) vs. cohort studies (on dietary intake: reference) as pooled RRRs. Biom, biomarkers; CS, cohort study; diet, dietary intake; MCI, mild cognitive impairment; RCT, randomized controlled trial; RR, risk ratio; RRR, ratio of risk ratio.

In subgroup analyses stratified by intervention type, the relative larger estimate was driven by micronutrient comparisons (BoERCTs vs. BoECSs dietary intake RRR: 1.09; 95% CI: 1.05, 1.14; I2 = 56%; PI: 0.93 to 1.28; BoERCTs vs. BoECSs biomarkers RRR: 1.19; 95% CI: 1.10, 1.30; I2 = 67%; PI: 0.79 to 1.81; BoECSs biomarkers vs. BoECSs dietary intake RRR: 0.89; 95% CI: 0.84, 0.95; I2 = 67%; PI: 0.69 to 1.15) (Supplemental Figures 3–5). Stratifying by outcome type, the estimates coming from the 3 BoE were relatively more different for overall mortality (BoERCTs vs. BoECSs dietary intake RRR: 1.14; 95% CI: 1.08, 1.21; I² = 67%; PI: 0.97 to 1.33; BoERCTs vs. BoECSs biomarkers RRR: 1.29; 95% CI: 1.15, 1.46; I² = 62%; PI: 0.91 to 1.84; BoECSs biomarkers vs. BoECSs dietary intake RRR: 0.87; 95% CI: 0.81, 0.94; I² = 9%; PI: 0.77 to 0.98) (Supplemental Figures 6–8).

The sensitivity analysis confirmed the findings of the primary analysis (Supplemental Figures 9 and 10).

Discussion

Summary of findings

This is the first meta-epidemiological study to identify and compare empirical data to determine the extent to which diet–disease association estimates of BoE from RCTs and CSs on dietary intake and biomarkers of intake are concordant. Of the 49 eligible diet–disease associations included, few were qualitatively concordant; this might be related to the fact that most of the BoE of RCTs reported not statistically significant results, whereas one-third and one-half of the BoE from CSs on dietary and biomarkers of intake, respectively, showed no statistically significant effect. More than 70% of the diet–disease associations were quantitatively concordant. By using both BoE from CSs as the reference category, the pooled estimate showed small relative larger estimates coming from BoE of RCTs, and comparing both BoE from CSs yielded also similar effects. The relative larger estimate in BoE of RCTs was mainly driven by comparing micronutrient comparisons. The majority of the eligible SRs (66%) were classified as critically low, whereas only 17% were moderate- or high-quality evidence based on the AMSTAR 2 criteria.

Comparison with other studies

We could not identify any similar meta-epidemiological study comparing the concordance between BoE from RCTs, CSs on dietary intake, and CSs on biomarkers of dietary intake. However, a review published in 2013 identified 34 diet–disease outcome pairs comparing of SRs of RCTs or a large single RCT (>1000 participants) compared with SRs of case-control or CSs or a large single observational study (>5000 participants). Comparable to our findings, 6 of 34 diet–disease associations (18%) were qualitatively concordant and 12 diet–disease associations (35%) were quantitatively discordant (32). In contrast to our study, the authors of this review included a smaller sample of diet–disease pairs, did not include solely findings from SRs (8 out of 34 associations derived from single studies), did not pool the effect estimate to generate an RRR, and did not differentiate between dietary intake and biomarkers of dietary intake.

Our findings are also in line with a statement by Satija and colleagues (66), which argued that, more often than not, when RCTs are able to successfully examine diet–disease relations, their results are remarkably in line with those of CSs. In the medical field, Anglemyer et al. (67) observed that there is little difference between the results obtained from RCTs and observational studies (cohort and case-control studies). Eleven out of 14 estimates were quantitatively concordant (79%). Moreover, although not significant, the point estimates suggest that BoE from RCTs may have a relative larger estimate than those obtained in observational studies (RRR: 1.08; 95% CI: 0.96, 1.22), which is similar to our findings (RRR: 1.09; 95% CI: 1.06, 1.13; and RRR: 1.18; 95% CI: 1.10, 1.25).

Implications for the broader research field

There has been a long debate regarding what constitutes best evidence in nutrition research, and whether it emerges from RCTs, which are considered the ideal methodology for causal inference and in which the effects of a dietary change on disease or intermediate disease markers are evaluated (68, 69). However, most dietary interventional RCTs are of short duration and often do not target patient-relevant outcomes such as morbidity or mortality. CSs, on the other hand, provide less-robust information regarding causality, but are usually considered more applicable for nutrition research. Biomarkers allow for objective measurement of intake without any bias due to self-reporting. Biomarkers as defined by the Biomarker Definitions Working Group show “characteristics that are objectively measured and evaluated as an indicator of normal biological processes, pathogenic processes, or pharmacologic responses to a therapeutic intervention” (70). If the required accuracy is given, they can differentiate between specific diseases as well as their severity, and serve to predict the likely outcome or the effectiveness of a therapy (71). At best, they can be used as surrogate markers replacing patient-relevant clinical endpoints (72). Thus, the indicators termed biomarkers in the present study are rather descriptors of dietary conditions and therefore have limitations.

For example, many dietary patterns, foods groups, or nutrients are not sensitive for or lack specific biomarkers, may not be reliable indicators of individual long-term intake, and are often expensive to measure. Therefore, a food-frequency questionnaire is the most common choice for measuring dietary intake in CSs (73). In our study, the biomarkers included were circulating fatty acids, sodium urinary excretion, status of vitamins (e.g., vitamins E, C, D), ferritin, micronutrients status such as folic acid and selenium, and circulating genistein, all of which are considered relevant biomarkers of dietary intake (74).

Confirmation of research findings or rather lack of confirmatory data is not a problem exclusively reserved for nutritional sciences. In recent years, different projects dedicated to the reproducibility of study data in the medical fields of oncology and cardiovascular disease have shown that it can be quite hard to verify preclinical observations (75–77). To check for concordance between different study designs as done in the present study might represent a useful tool to increase the reliability of studies in nutritional sciences.

Since BoE from CSs on dietary intake and biomarkers of intake can complement BoE from RCTs, and vice versa, our meta-epidemiological study provides a first insight that integration of all these BoE in nutrition evidence syntheses is recommended.

Strengths and limitations

This meta-epidemiological study has several strengths. First, we included a large sample of diet–disease associations (n = 49), which were based on >400 RCTs and >550 CSs, both study designs considered as the most trustworthy in nutrition research (6); second, the various statistical analyses conducted, such as recalculating 10 pooled estimates, converting ORs to RRs, the qualitative and quantitative assessment of concordance, and pooling the estimates across all diet–disease pairs; and finally, the exploration of potential sources of concordance due to subgroup analyses for types of intervention/exposure and outcomes was an additional strength of this study.

Limitations of this study are as follows: first, although we included a large sample of diet–disease pairs, including (and pooling) the totality of evidence of available diet–disease associations might provide different results; second, the definitions of qualitative and quantitative concordance used have some limitations as well (definition of qualitative concordance was relatively strict). However, for the qualitative assessment we also included nonsignificant findings based on the imprecision criteria by the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) Working Group (30), previously not used, as well as the results considering a less strict definition. Finally, we did not explore all potential reasons of concordance, such as dietary adherence or validity and reliability of the dietary biomarkers in the underlying RCTs, or risk of bias of primary studies.

Conclusions

Our findings suggest that BoE from RCTs and CSs are often quantitatively concordant. Prospective SRs in nutrition research should include, whenever possible, BoE from RCTs and CSs on dietary intake and biomarkers of intake to provide the whole picture for an investigated diet–disease association.

Supplementary Material

nmab095_Supplemental_File
Click here for additional data file. (4.2MB, pdf)

ACKNOWLEDGEMENTS

All authors substantially contributed to the concept, data collection and analysis, or preparation of the manuscript.

Notes

Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - Projektnummer 459430615 and Forschungskommission der Medizinischen Fakultät Freiburg. The funders had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report, or decision to submit the article for publication.

Author disclosures: The authors report no conflicts of interest. LS is a member of the Editorial Board of Advances in Nutrition and a member of the GRADE Working Group.

Supplemental Appendixes 1 and 2, Supplemental Figures 1–10, Supplemental Tables 1–5, and Supplemental References are available from the “Supplemental data” link in the online posting of the article and from the same link in the online table of contents at https://academic.oup.com/advances/.

Abbreviations used: ACR, assumed control risk; BoE, bodies of evidence; CS, cohort study; GBD, Global Burden of Disease; NCD, noncommunicable disease; PI, prediction interval; PI/ECO, patients/population, intervention/exposure, comparator, and outcome; RCT, randomized controlled trial; RR, risk ratio; RRR, ratio of risk ratio; SR, systematic review.

Contributor Information

Jessica Beyerbach, Institute for Evidence in Medicine, Medical Center–University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany.

Julia Stadelmaier, Institute for Evidence in Medicine, Medical Center–University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany.

Georg Hoffmann, Department of Nutritional Sciences, University of Vienna, Vienna, Austria.

Sara Balduzzi, Institute of Medical Biometry and Statistics, Medical Center–University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany.

Nils Bröckelmann, Institute for Evidence in Medicine, Medical Center–University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany.

Lukas Schwingshackl, Institute for Evidence in Medicine, Medical Center–University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany.

References

  • 1. GBD 2017 Causes of Death Collaborators . Global, regional, and national age-sex-specific mortality for 282 causes of death in 195 countries and territories, 1980–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2018;392(10159):1736–88.. Epub 2018 Nov 8. Erratum in: Lancet. 2019 Jun 22;393(10190):e44. Erratum in: Lancet. 2018 Nov 17;392(10160):2170. PMID: 30496103; PMCID: PMC6227606. doi: 10.1016/S0140-6736(18)32203-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Afshin  A, Sur  PJ, Fay  KA, Cornaby  L, Ferrara  G, Salama  JS, Mullany  EC, Abate  KH, Abbafati  C, Abebe  Z  et al.  Health effects of dietary risks in 195 countries, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet North Am Ed. 2019;393(10184):1958–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. US Department of Health and Human Services; US Department of Agriculture . 2015–2020 Dietary guidelines for Americans. 8th ed. 2015. Available from[Internet]: https://health.gov/dietaryguidelines/2015/guidelines/ (accessed 30 June 2019). [Google Scholar]
  • 4. Ioannidis  JA. The challenge of reforming nutritional epidemiologic research. JAMA. 2018;320(10):969–70. [DOI] [PubMed] [Google Scholar]
  • 5. Clarke  ED, Rollo  ME, Pezdirc  K, Collins  CE, Haslam  RL. Urinary biomarkers of dietary intake: a review. Nutr Rev. 2020;78(5):364–81. [DOI] [PubMed] [Google Scholar]
  • 6. Pan  A, Lin  X, Hemler  E, Hu  FB. Diet and cardiovascular disease: advances and challenges in population-based studies. Cell Metab. 2018;27(3):489–96. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. Humphrey  LL, Fu  R, Rogers  K, Freeman  M, Helfand  M. Homocysteine level and coronary heart disease incidence: a systematic review and meta-analysis. Mayo Clin Proc. 2008;83(11):1203–12. [DOI] [PubMed] [Google Scholar]
  • 8. Koushik  A, Hunter  DJ, Spiegelman  D, Anderson  KE, Buring  JE, Freudenheim  JL, Goldbohm  RA, Hankinson  SE, Larsson  SC, Leitzmann  M  et al.  Intake of the major carotenoids and the risk of epithelial ovarian cancer in a pooled analysis of 10 cohort studies. Int J Cancer. 2006;119(9):2148–54. [DOI] [PubMed] [Google Scholar]
  • 9. Rapola  JM, Virtamo  J, Ripatti  S, Huttunen  JK, Albanes  D, Taylor  PR, Heinonen  OP. Randomised trial of alpha-tocopherol and beta-carotene supplements on incidence of major coronary events in men with previous myocardial infarction. Lancet North Am Ed. 1997;349(9067):1715–20. [DOI] [PubMed] [Google Scholar]
  • 10. Stampfer  MJ, Hennekens  CH, Manson  JE, Colditz  GA, Rosner  B, Willett  WC. Vitamin E consumption and the risk of coronary disease in women. N Engl J Med. 1993;328(20):1444–9. [DOI] [PubMed] [Google Scholar]
  • 11. Heart Outcomes Prevention  Evaluation Study Investigators, Yusuf  S, Dagenais  G, Pogue  J, Bosch  J, Sleight  P. Vitamin E supplementation and cardiovascular events in high-risk patients. N Engl J Med. 2000;342(3):154–60. [DOI] [PubMed] [Google Scholar]
  • 12. Chowdhury  R, Stevens  S, Gorman  D, Pan  A, Warnakula  S, Chowdhury  S, Ward  H, Johnson  L, Crowe  F, Hu  FB  et al.  Association between fish consumption, long chain omega 3 fatty acids, and risk of cerebrovascular disease: systematic review and meta-analysis. BMJ. 2012;345:e6698. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13. Murad  MH, Wang  Z. Guidelines for reporting meta-epidemiological methodology research. Evid Based Med. 2017;22(4):139–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. Brown  TJ, Brainard  J, Song  F, Wang  X, Abdelhamid  A, Hooper  L. Omega-3, omega-6, and total dietary polyunsaturated fat for prevention and treatment of type 2 diabetes mellitus: systematic review and meta-analysis of randomised controlled trials. BMJ. 2019;366:14697. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15. Druesne-Pecollo  N, Latino-Martel  P, Norat  T, Barrandon  E, Bertrais  S, Galan  P, Hercberg  S. Beta-carotene supplementation and cancer risk: a systematic review and metaanalysis of randomized controlled trials. Int J Cancer. 2010;127(1):172–84. [DOI] [PubMed] [Google Scholar]
  • 16. Hanson  S, Thorpe  G, Winstanley  L, Abdelhamid  AS, Hooper  L, Abdelhamid  A, Ajabnoor  S, Alabdulghafoor  F, Alkhudairy  L, Biswas  P  et al.  Omega-3, omega-6 and total dietary polyunsaturated fat on cancer incidence: systematic review and meta-analysis of randomised trials. Br J Cancer. 2020;122(8):1260–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. Khan  SU, Khan  MU, Riaz  H, Valavoor  S, Zhao  D, Vaughan  L, Okunrintemi  V, Riaz  IB, Khan  MS, Kaluski  E  et al.  Effects of nutritional supplements and dietary interventions on cardiovascular outcomes: an umbrella review and evidence map. Ann Intern Med. 2019;171(3):190–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18. Jenkins  DJA, Spence  JD, Giovannucci  EL, Kim  YI, Josse  R, Vieth  R, Blanco Mejia  S, Viguiliouk  E, Nishi  S, Sahye-Pudaruth  S  et al.  Supplemental vitamins and minerals for CVD prevention and treatment. J Am Coll Cardiol. 2018;71(22):2570–84. [DOI] [PubMed] [Google Scholar]
  • 19. Moazzen  S, Dolatkhah  R, Tabrizi  JS, Shaarbafi  J, Alizadeh  BZ, de Bock  GH, Dastgiri  S. Folic acid intake and folate status and colorectal cancer risk: a systematic review and meta-analysis. Clin Nutr. 2018;37(6):1926–34. [DOI] [PubMed] [Google Scholar]
  • 20. Park  SJ, Myung  SK, Lee  Y, Lee  YJ. Effects of vitamin and antioxidant supplements in prevention of bladder cancer: a meta-analysis of randomized controlled trials. J Korean Med Sci. 2017;32(4):628–35. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. van Die  MD, Bone  KM, Williams  SG, Pirotta  MV. Soy and soy isoflavones in prostate cancer: a systematic review and meta-analysis of randomized controlled trials. BJU Int. 2014;113(5b):E119–30. [DOI] [PubMed] [Google Scholar]
  • 22. Schwingshackl  L, Boeing  H, Stelmach-Mardas  M, Gottschald  M, Dietrich  S, Hoffmann  G, Chaimani  A. Dietary supplements and risk of cause-specific death, cardiovascular disease, and cancer: a systematic review and meta-analysis of primary prevention trials. Adv Nutr. 2017;8(1):27–39. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23. Vollset  SE, Clarke  R, Lewington  S, Ebbing  M, Halsey  J, Lonn  E, Armitage  J, Manson  JE, Hankey  GJ, Spence  JD  et al.  Effects of folic acid supplementation on overall and site-specific cancer incidence during the randomised trials: meta-analyses of data on 50,000 individuals. Lancet North Am Ed. 2013;381(9871):1029–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24. Zhao  L, Lian  J, Tian  J, Shen  Y, Ping  Z, Fang  X, Min  J, Wang  F. Dietary intake of heme iron and body iron status are associated with the risk of gestational diabetes mellitus: a systematic review and meta-analysis. Asia Pac J Clin Nutr. 2017;26(6):1092–106. [DOI] [PubMed] [Google Scholar]
  • 25. Applegate  CC, Rowles  JL, Ranard  KM, Jeon  S, Erdman  JW. Soy consumption and the risk of prostate cancer: an updated systematic review and meta-analysis. Nutrients. 2018;10(1):40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26. Li  J, Guasch-Ferre  M, Li  Y, Hu  FB. Dietary intake and biomarkers of linoleic acid and mortality: systematic review and meta-analysis of prospective cohort studies. Am J Clin Nutr. 2020;112(1):150–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27. DerSimonian  R, Laird  N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177–88. [DOI] [PubMed] [Google Scholar]
  • 28. Review Manager (RevMan) [computer program]. Version 5.4. The Cochrane Collaboration; 2020. [Google Scholar]
  • 29. Shea  BJ, Reeves  BC, Wells  G, Thuku  M, Hamel  C, Moran  J, Moher  D, Tugwell  P, Welch  V, Kristjansson  E  et al.  AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30. Guyatt  GH, Oxman  AD, Kunz  R, Brozek  J, Alonso-Coello  P, Rind  D, Devereaux  PJ, Montori  VM, Freyschuss  B, Vist  G  et al.  GRADE guidelines 6. Rating the quality of evidence—imprecision. J Clin Epidemiol. 2011;64(12):1283–93. [DOI] [PubMed] [Google Scholar]
  • 31. Higgins  JPT, Thomas  J, Chandler  J, Cumpston  M, Li  T, Page  MJ, Welch  VA, editors. Cochrane handbook for systematic reviews of interventions. Version 6.0 (updated July 2019). Cochrane Collaboration: 2019. Available from[Internet]: www.training.cochrane.org/handbook. [Google Scholar]
  • 32. Moorthy  D, Chung  M, Lee  J, Yu  WW, Lau  J, Trikalinos  TA. AHRQ technical reviews. In: Concordance between the findings of epidemiological studies and randomized trials in nutrition: an empirical evaluation and citation analysis. Nutritional Research Series. Vol. 6. Rockville (MD): Agency for Healthcare Research and Quality; 2013. https://www.ncbi.nlm.nih.gov/books/NBK138246/. [PubMed] [Google Scholar]
  • 33. Altman  DG, Bland  JM. Interaction revisited: the difference between two estimates. BMJ. 2003;326(7382):219. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34. Riley  RD, Higgins  JPT, Deeks  JJ. Interpretation of random effects meta-analyses. BMJ. 2011;342:d549. [DOI] [PubMed] [Google Scholar]
  • 35. Higgins  JP, Thompson  SG, Deeks  JJ, Altman  DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36. Veroniki  AA, Jackson  D, Viechtbauer  W, Bender  R, Bowden  J, Knapp  G, Kuss  O, Higgins  JP, Langan  D, Salanti  G. Methods to estimate the between-study variance and its uncertainty in meta-analysis. Res Synth Methods. 2016;7(1):55–79. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37. Paule  RC, Mandel  J. Consensus values and weighting factors. National Institute of Standards and Technology; 1982. https://nvlpubs.nist.gov/nistpubs/jres/087/jresv87n5p377_A1b.pdf. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Balduzzi  S, Rücker  G, Schwarzer  G. How to perform a meta-analysis with R: a practical tutorial. Evid Based Ment Health. 2019;22(4):153–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. Vinceti  M, Filippini  T, Del Giovane  C, Dennert  G, Zwahlen  M, Brinkman  M, Zeegers  MP, Horneber  M, D'Amico  R, Crespi  CM. Selenium for preventing cancer. Cochrane Database Syst Rev. 2018;1:CD005195. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40. Abdelhamid  AS, Brown  TJ, Brainard  JS, Biswas  P, Thorpe  GC, Moore  HJ, Deane  KH, AlAbdulghafoor  FK, Summerbell  CD, Worthington  HV  et al.  Omega-3 fatty acids for the primary and secondary prevention of cardiovascular disease. Cochrane Database Syst Rev. 2018;11:CD003177. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41. Adler  AJ, Taylor  F, Martin  N, Gottlieb  S, Taylor  RS, Ebrahim  S. Reduced dietary salt for the prevention of cardiovascular disease. Cochrane Database Syst Rev. 2014(12):CD009217. doi: 10.1002/14651858.CD009217.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42. Bjelakovic  G, Nikolova  D, Gluud  LL, Simonetti  RG, Gluud  C. Antioxidant supplements for prevention of mortality in healthy participants and patients with various diseases. Cochrane Database Syst Rev. 2012(3):CD007176. doi: 10.1002/14651858.CD007176.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43. Hooper  L, Al-Khudairy  L, Abdelhamid  AS, Rees  K, Brainard  JS, Brown  TJ, Ajabnoor  SM, O'Brien  AT, Winstanley  LE, Donaldson  DH  et al.  Omega-6 fats for the primary and secondary prevention of cardiovascular disease. Cochrane Database Syst Rev. 2018;11:CD011094. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44. Rutjes  AW, Denton  DA, Di Nisio  M, Chong  LY, Abraham  RP, Al-Assaf  AS, Anderson  JL, Malik  MA, Vernooij  RW, Martinez  G  et al.  Vitamin and mineral supplementation for maintaining cognitive function in cognitively healthy people in mid and late life. Cochrane Database Syst Rev. 2018;12:CD011906. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45. Rees  K, Hartley  L, Day  C, Flowers  N, Clarke  A, Stranges  S. Selenium supplementation for the primary prevention of cardiovascular disease. Cochrane Database Syst Rev. 2013;1:CD009671. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46. Bjelakovic  G, Gluud  LL, Nikolova  D, Whitfield  K, Krstic  G, Wetterslev  J, Gluud  C. Vitamin D supplementation for prevention of cancer in adults. Cochrane Database Syst Rev. 2014;6:CD007469. doi: 10.1002/14651858.CD007469.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47. Chowdhury  R, Warnakula  S, Kunutsor  S, Crowe  F, Ward  HA, Johnson  L, Franco  OH, Butterworth  AS, Forouhi  NG, Thompson  SG  et al.  Association of dietary, circulating, and supplement fatty acids with coronary risk: a systematic review and meta-analysis. Ann Intern Med. 2014;160(6):398–406. [DOI] [PubMed] [Google Scholar]
  • 48. Pan  A, Chen  M, Chowdhury  R, Wu  JH, Sun  Q, Campos  H, Mozaffarian  D, Hu  FB. alpha-Linolenic acid and risk of cardiovascular disease: a systematic review and meta-analysis. Am J Clin Nutr. 2012;96(6):1262–73. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49. Aburto  NJ, Ziolkovska  A, Hooper  L, Elliott  P, Cappuccio  FP, Meerpohl  JJ. Effect of lower sodium intake on health: systematic review and meta-analyses. BMJ. 2013;346:f1326. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50. Aune  D, Keum  N, Giovannucci  E, Fadnes  LT, Boffetta  P, Greenwood  DC, Tonstad  S, Vatten  LJ, Riboli  E, Norat  T. Dietary intake and blood concentrations of antioxidants and the risk of cardiovascular disease, total cancer, and all-cause mortality: a systematic review and dose-response meta-analysis of prospective studies. Am J Clin Nutr. 2018;108(5):1069–91. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51. Hossain  S, Beydoun  MA, Beydoun  HA, Chen  X, Zonderman  AB, Wood  RJ. Vitamin D and breast cancer: a systematic review and meta-analysis of observational studies. Clin Nutr ESPEN. 2019;30:170–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52. Touvier  M, Chan  DS, Lau  R, Aune  D, Vieira  R, Greenwood  DC, Kampman  E, Riboli  E, Hercberg  S, Norat  T. Meta-analyses of vitamin D intake, 25-hydroxyvitamin D status, vitamin D receptor polymorphisms, and colorectal cancer risk. Cancer Epidemiol Biomarkers Prev. 2011;20(5):1003–16. [DOI] [PubMed] [Google Scholar]
  • 53. Zhang  L, Wang  S, Che  X, Li  X. Vitamin D and lung cancer risk: a comprehensive review and meta-analysis. Cell Physiol Biochem. 2015;36(1):299–305. [DOI] [PubMed] [Google Scholar]
  • 54. Wu  JH, Micha  R, Imamura  F, Pan  A, Biggs  ML, Ajaz  O, Djousse  L, Hu  FB, Mozaffarian  D. Omega-3 fatty acids and incident type 2 diabetes: a systematic review and meta-analysis. Br J Nutr. 2012;107(S2):S214–27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55. Aune  D, Chan  DS, Vieira  AR, Navarro Rosenblatt  DA, Vieira  R, Greenwood  DC, Norat  T. Dietary compared with blood concentrations of carotenoids and breast cancer risk: a systematic review and meta-analysis of prospective studies. Am J Clin Nutr. 2012;96(2):356–73. [DOI] [PubMed] [Google Scholar]
  • 56. Alexander  DD, Bassett  JK, Weed  DL, Barrett  EC, Watson  H, Harris  W. Meta-analysis of long-chain omega-3 polyunsaturated fatty acids (LCω-3PUFA) and prostate cancer. Nutr Cancer. 2015;67(4):543–54. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57. Cao  Y, Hou  L, Wang  W. Dietary total fat and fatty acids intake, serum fatty acids and risk of breast cancer: a meta-analysis of prospective cohort studies. Int J Cancer. 2016;138(8):1894–904. [DOI] [PubMed] [Google Scholar]
  • 58. Fu  YQ, Zheng  JS, Yang  B, Li  D. Effect of individual omega-3 fatty acids on the risk of prostate cancer: a systematic review and dose-response meta-analysis of prospective cohort studies. J Epidemiol. 2015;25(4):261–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59. Zheng  JS, Hu  XJ, Zhao  YM, Yang  J, Li  D. Intake of fish and marine n-3 polyunsaturated fatty acids and risk of breast cancer: meta-analysis of data from 21 independent prospective cohort studies. BMJ. 2013;346:f3706. [DOI] [PubMed] [Google Scholar]
  • 60. Hunnicutt  J, He  K, Xun  P. Dietary iron intake and body iron stores are associated with risk of coronary heart disease in a meta-analysis of prospective cohort studies. J Nutr. 2014;144(3):359–66. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61. Chen  GC, Yang  J, Eggersdorfer  M, Zhang  W, Qin  LQ. N-3 long-chain polyunsaturated fatty acids and risk of all-cause mortality among general populations: a meta-analysis. Sci Rep. 2016;6:28165. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62. Wu  S, Liu  Y, Michalek  JE, Mesa  RA, Parma  DL, Rodriguez  R, Mansour  AM, Svatek  R, Tucker  TC, Ramirez  AG. Carotenoid intake and circulating carotenoids are inversely associated with the risk of bladder cancer: a dose-response meta-analysis. Adv Nutr. 2020;11(3):630–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63. Jayedi  A, Rashidy-Pour  A, Parohan  M, Zargar  MS, Shab-Bidar S. Dietary antioxidants, circulating antioxidant concentrations, total antioxidant capacity, and risk of all-cause mortality: a systematic review and dose-response meta-analysis of prospective observational studies. Adv Nutr. 2018;9(6):701–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64. Doets  EL, van Wijngaarden  JP, Szczecinska  A, Dullemeijer  C, Souverein  OW, Dhonukshe-Rutten  RA, Cavelaars  AE, van't Veer  P, Brzozowska  A, de Groot  LC. Vitamin B12 intake and status and cognitive function in elderly people. Epidemiol Rev. 2013;35:2–21. [DOI] [PubMed] [Google Scholar]
  • 65. Wang  R, Zheng  Y, Huang  JY, Zhang  AQ, Zhou  YH, Wang  JN. Folate intake, serum folate levels, and prostate cancer risk: a meta-analysis of prospective studies. BMC Public Health. 2014;14:1326. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66. Satija  A, Stampfer  MJ, Rimm  EB, Willett  W, Hu  FB. Perspective: are large, simple trials the solution for nutrition research?. Adv Nutr. 2018;9(4):378–87. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67. Anglemyer  A, Horvath  HT, Bero  L. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. Cochrane Database Syst Rev. 2014;4:MR000034. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68. Maki  KC, Slavin  JL, Rains  TM, Kris-Etherton  PM. Limitations of observational evidence: implications for evidence-based dietary recommendations. Adv Nutr. 2014;5(1):7–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69. Schwingshackl  L, Schünemann  HJ, Meerpohl  JJ. Improving the trustworthiness of findings from nutrition evidence syntheses: assessing risk of bias and rating the certainty of evidence. Eur J Nutr. 2020. Online ahead of print, doi: 10.1007/s00394-020-02464-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70. Biomarkers Definitions Working Group . Biomarkers and surrogate endpoints: preferred definitions and conceptual framework. Clin Pharmacol Ther. 2001;69(3):89–95.. https://pubmed.ncbi.nlm.nih.gov/11240971/. [DOI] [PubMed] [Google Scholar]
  • 71. Katz  R. Biomarkers and surrogate markers: an FDA perspective. NeuroRx. 2004;1(2):189–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 72. Temple  R. Are surrogate markers adequate to assess cardiovascular disease drugs?. JAMA. 1999;282(8):790–5. [DOI] [PubMed] [Google Scholar]
  • 73. Satija  A, Yu  E, Willett  WC, Hu  FB. Understanding nutritional epidemiology and its role in policy. Adv Nutr. 2015;6(1):5–18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74. Sri Harsha  PSC, Wahab  RA, Garcia-Aloy  M, Madrid-Gambin  F, Estruel-Amades  S, Watzl  B, Andrés-Lacueva  C, Brennan  L. Biomarkers of legume intake in human intervention and observational studies: a systematic review. Genes Nutr. 2018;13:25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 75. Prinz  F, Schlange  T, Asadullah  K. Believe it or not: how much can we rely on published data on potential drug targets?. Nat Rev Drug Discovery. 2011;10(9):712. [DOI] [PubMed] [Google Scholar]
  • 76. Errington  TM, Iorns  E, Gunn  W, Tan  FE, Lomax  J, Nosek  BA. An open investigation of the reproducibility of cancer biology research. Elife. 2014;3:e04333. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 77. Begley  CG, Ellis  LM. Raise standards for preclinical cancer research. Nature. 2012;483(7391):531–3. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

nmab095_Supplemental_File
Click here for additional data file. (4.2MB, pdf)

Articles from Advances in Nutrition are provided here courtesy of American Society for Nutrition

RESOURCES