Skip to main content
The BMJ logoLink to The BMJ
. 2021 Sep 15;374:n1864. doi: 10.1136/bmj.n1864

Evaluating agreement between bodies of evidence from randomised controlled trials and cohort studies in nutrition research: meta-epidemiological study

Lukas Schwingshackl 1,, Sara Balduzzi 2, Jessica Beyerbach 1, Nils Bröckelmann 1, Sarah S Werner 1, Jasmin Zähringer 1, Blin Nagavci 1, Joerg J Meerpohl 1 ,3
PMCID: PMC8441535  PMID: 34526355

Abstract

Objective

To evaluate the agreement between diet-disease effect estimates of bodies of evidence from randomised controlled trials and those from cohort studies in nutrition research, and to investigate potential factors for disagreement.

Design

Meta-epidemiological study.

Data sources

Cochrane Database of Systematic Reviews, and Medline.

Review methods

Population, intervention or exposure, comparator, outcome (PI/ECO) elements from a body of evidence from cohort studies (BoE(CS)) were matched with corresponding elements of a body of evidence from randomised controlled trials (BoE(RCT)). Pooled ratio of risk ratios or difference of mean differences across all diet-disease outcome pairs were calculated. Subgroup analyses were conducted to explore factors for disagreement. Heterogeneity was assessed through I2 and τ2. Prediction intervals were calculated to assess the range of possible values for the difference in the results between evidence from randomised controlled trials and evidence from cohort studies in future comparisons.

Results

97 diet-disease outcome pairs (that is, matched BoE(RCT) and BoE(CS)) were identified overall. For binary outcomes, the pooled ratio of risk ratios comparing estimates from BoE(RCT) with BoE(CS) was 1.09 (95% confidence interval 1.04 to 1.14; I2=68%; τ2=0.021; 95% prediction interval 0.81 to 1.46). The prediction interval indicated that the difference could be much more substantial, in either direction. We further explored heterogeneity and found that PI/ECO dissimilarities, especially for the comparisons of dietary supplements in randomised controlled trials and nutrient status in cohort studies, explained most of the differences. When the type of intake or exposure between both types of evidence was identical, the estimates were similar. For continuous outcomes, small differences were observed between randomised controlled trials and cohort studies.

Conclusion

On average, the difference in pooled results between estimates from BoE(RCT) and BoE(CS) was small. But wide prediction intervals and some substantial statistical heterogeneity in cohort studies indicate that important differences or potential bias in individual comparisons or studies cannot be excluded. Observed differences were mainly driven by dissimilarities in population, intervention or exposure, comparator, and outcome. These findings could help researchers further understand the integration of such evidence into prospective nutrition evidence syntheses and improve evidence based dietary guidelines.

Introduction

The Global Burden of Disease study group showed that non-communicable diseases accounted for 73% of deaths worldwide.1 According to the Global Burden of Disease study, which is based on evidence from prospective cohort studies, suboptimal diet accounted for 22% of all deaths worldwide, and 15% of all disability adjusted life years.2

The Global Burden of Disease studies and dietary guidelines are predominantly based on bodies of evidence (BoE) from cohort studies,3 although evidence from randomised controlled trials exists as well. Cohort studies with patient relevant outcomes provide valuable insights into associations between diet and disease.2 4 However, nutrition research, predominantly nutritional epidemiology, has been criticised for providing potentially less trustworthy estimates of diet associated risks or benefits.5 Therefore, limitations such as residual confounding and measurement errors of cohort studies in nutrition research need to be considered in depth.5 On the other hand, randomised controlled trials, if well designed and well conducted, give robust answers to the research questions under consideration and are widely accepted as the ideal methodology for causal inference.6 However, dietary trials often have methodological limitations, such as small sample sizes, short intervention periods, as well as blinding and low compliance issues.7

In the past, several randomised controlled trials comparing dietary interventions with placebo or control interventions have failed to replicate the (presumably protective) associations between dietary factors and risk for non-communicable diseases found in large cohort studies.8 9 10 11 For example, randomised controlled trials found no beneficial effect of fibre intake on colorectal cancer risk,12 or of vitamin E on cardiovascular diseases.13 In other instances, consistent findings between cohort studies and randomised controlled trials have been reported (eg, Mediterranean diet and risk of cardiovascular disease and type 2 diabetes),14 but to the best of our knowledge no systematic evaluation of agreement, with an investigation on factors for disagreement, between the two BoE has ever been conducted.14 15 This meta-epidemiological study aims to determine the extent to which estimates between diet and disease based on BoE from randomised controlled trials are in agreement with those estimates based on BoE from cohort studies, and further investigate reasons behind any disagreement. These findings will allow us to better understand and explore the possible integration of both BoE in prospective nutrition evidence syntheses.

Methods

This meta-epidemiological study was planned, written, and reported in adherence to guidelines for reporting meta-epidemiological research.16 The inclusion criteria (patients or population, intervention or exposure, comparator, and outcome (PI/ECO)) are described in box 1.

Box 1. Detailed description of inclusion criteria, by population.

Intervention or exposure

  • Dietary pattern: for example, Mediterranean diet, Dietary approaches to Stop Hypertension, low carbohydrate diet

  • Food groups (macro-level) and foods (micro-level): for example, grains, vegetables, fruit, milk and dairy products, meat, processed meat, fish, eggs, nuts, chocolate, oils

  • Macronutrients:

    • Carbohydrates: starch, fructose, glucose, sucrose

    • Fat: for example, n-3 fatty acids (eicosapentaenoic acid, docosahexaenoic acid, α linolenic acid), n-6 fatty acids (linoleic acid), monounsaturated fat

    • Protein: for example, amino acids

  • Micronutrients:

    • Vitamins: β carotene; vitamins A, E, C (ascorbic acid), and D (cholecalciferol, ergocalciferol); B vitamins (thiamine, riboflavin, niacin, pyridoxine, cobalamin, folic acid)

    • Minerals: magnesium, calcium, selenium, sodium, potassium, iron, zinc, copper, iodine

  • Other: fibre (psyllium, inulin, cellulose), probiotics, prebiotics, and synbiotics

Control or comparison

  • Low (no) intake (status) level of the above interventions or exposure

  • Placebo or usual care

Outcomes

  • For example, all cause mortality, cardiovascular disease, coronary heart disease (myocardial infarction, ischaemic heart disease, and acute coronary syndrome), stroke, cancer, type 2 diabetes, dementia, fractures, age related macular degeneration, anthropometric outcomes, important intermediate disease markers such systolic blood pressure, and diastolic blood pressure, fasting glucose, and low density lipoprotein cholesterol

Study design

  • Systematic reviews of randomised controlled trials

  • Matching systematic reviews of cohort studies (if available prospective cohort studies were preferred)

Identification of systematic reviews of randomised controlled trials

We searched the Cochrane Database of Systematic Reviews, for systematic reviews of randomised controlled trials, published between 1 January 2010 and 31 December 2019 (supplementary appendix 1). Titles and abstracts were screened by one reviewer (LS), and subsequently all potentially relevant full texts were screened and assessed by two reviewers independently (LS, JZ). Discrepancies were resolved by a third reviewer (JJM).

Identification of matching systematic reviews of cohort studies

After identifying all potentially relevant systematic reviews of randomised controlled trials, we searched for matching systematic reviews of cohort studies. Firstly, we screened all eligible Cochrane reviews, to determine whether they also included cohort studies. Secondly, we conducted searches for systematic reviews of cohort studies in Medline, published within the past 10 years (1 January 2010 and 31 December 2019; supplementary appendix 2). We selected a period of 10 years to ensure comparability between the two BoE. No language restriction was used. Titles and abstracts was screened by one reviewer (LS), after which relevant full texts were screened by two reviewers independently (LS, JZ). Supplementary hand searches identified two additional matching systematic reviews of cohort studies.17 18 We included the best matching (that is, investigating similar PI/ECO categories, see below) and most comprehensive (that is, most recent) systematic reviews of cohort studies for inclusion.

Matching bodies of evidence according to PI/ECO criteria

For all potentially eligible BoE of cohort studies, two reviewers judged whether each PI/ECO element matched those of the corresponding BoE of randomised controlled trials according to three definitions (supplementary table 1): more or less identical (very closely matched), similar but not identical (closely matched), or broadly similar (matched, but less close).19 Differences in reviewer ratings of one level disagreement were resolved by discussion (we considered the broader similarity rating for the overall PI/ECO rating); a third reviewer adjudicated the overall PI/ECO match rating for differences of more than two levels. Based on these criteria, we classified each comparison of effect estimates from randomised controlled trials and cohort studies for a given outcome according to the same three definitions: more or less identical, similar but not identical, and broadly similar. For each eligible systematic review of randomised controlled trials, we matched a maximum of six outcomes (maximum three patient relevant outcomes; and maximum three intermediate disease outcomes) for a given intervention or exposure. Selection of outcomes was based on the ranking in the summary of findings tables in the identified Cochrane reviews (from top to bottom).

Data extraction

We extracted data for every eligible BoE pair (BoE from a randomised controlled trial and matched BoE from a cohort study) related to the association between diet and disease (eg, all cause mortality, cardiovascular disease, stroke, type 2 diabetes). These data included the name of first author, year of publication, description of population (eg, disease status), age range, intervention or exposure (eg, dietary pattern, food group, food, macronutrient, micronutrient), description of comparator (eg, placebo, lowest intake or status category, control diet), definition of outcome, study design (parallel, crossover, factorial (for randomised controlled trials); prospective, nested case-control studies, case cohort studies (for cohort studies)), effect estimates (risk ratio, hazard ratio, odds ratio, mean differences, 95% confidence interval), type of comparison (eg, high v low, dose-response), number of studies included, sample size, number of cases, duration of intervention or exposure (range), risk of bias or study quality ratings, and certainty of evidence rating. Data were extracted by three reviewers (LS, JB, or SSW) using a piloted data extraction form.

Where a BoE reported effect estimates based on a pool of studies of variable design (that is, case-control, cross sectional studies, retrospective cohort studies, or quasi-randomised controlled trials), we recalculated the pooled effect estimates by excluding non-cohort studies and non-randomised controlled trials. Also, if an intervention in a BoE of randomised controlled trials (eg, low v high sodium) and an exposure in a BoE of cohort studies (eg, high v low sodium) investigated opposite comparisons, we recalculated the risk estimates, respectively (eg, low v high sodium). Moreover, where a BoE reported effect estimates based on dietary intake and dietary supplements, nutrient status (eg, plasma selenium status) and dietary intake, or nutrient status and dietary supplements, we recalculated effect estimates whenever feasible to improve comparability between exposures in cohort studies and interventions in randomised controlled trials. For example, if a meta-analysis of randomised controlled trials investigated the effect of selenium supplements, and the authors of the matched meta-analysis of cohort studies combined plasma selenium status with selenium supplements, we excluded the studies with plasma selenium status and recalculated the effect estimates only based on the studies with selenium supplements.

Statistical analysis

If the effect estimate of the BoE from randomised controlled trials was expressed in a different measure than the effect estimate of a BoE from cohort studies, we used the appropriate conversion formulas in order to express both estimates in the same measure—that is, risk ratios for binary outcomes and mean differences for continuous outcomes. The relevant formula to transform an odds ratio to a risk ratio requires an assumed control risk:

RR=(OR÷(1−ACR×(1−OR))

Where RR=risk ratio, OR=odds ratio, and ACR=assumed control risk.20

Ten meta-analyses of cohort studies, included in seven systematic reviews,21 22 23 24 25 26 27 used an odds ratio as a summary measure (supplementary table 2); the median comparator group risk from the included studies was used20 for the assumed control risk required for transformation of each pooled odds ratio. If these data were not directly available in the meta-analyses of cohort studies, we used the median comparator group risk from the studies included in the corresponding meta-analyses of randomised controlled trials.

In six analyses (each including only one study) coming from two systematic reviews of randomised controlled trials,28 29 the results were expressed using hazard ratios; we did not consider the hazard ratios, but went back to the primary studies and extracted relevant data in order to obtain a risk ratio (that is, the number of randomised patients and number of patients with the outcomes of interest, in each arm; supplementary table 2).

To compare the two BoEs (that is, from randomised controlled trials and cohort studies), we synthesised the differences in the results coming from all eligible outcome pairs. Binary outcomes were expressed as ratio of risk ratios,30 while continuous outcomes were expressed as differences of mean differences. By using the BoE of cohort studies as the reference group, we examined the pooled estimate to determine a relatively larger or smaller estimate from the BoE of randomised controlled trials (that is, effect of BoE of trials > effect of BoE of cohort studies, or effect of BoE of trials < effect of BoE of cohort studies). For example, a risk ratio from randomised controlled trials of 0.95 and a risk ratio from cohort studies of 0.90 would result in a ratio of risk ratios of 1.06; whereas a risk of 1.00 in cohort studies compared with a risk ratio of 1.06 in randomised controlled trials would also result in a ratio of risk ratios of 1.06. Therefore, the ratio of risk ratios should not be interpreted as larger or smaller treatment effects in one type of study (eg, randomised controlled trials), but only as differences between the two BoEs; and the direction of difference depends on direction of effect of the underlying BoEs.

We conducted a priori planned subgroup analyses: type of dietary intervention or exposure, outcome, and PI/ECO similarity degree (more or less identical, similar but not identical, and broadly similar). We also conducted two post hoc sensitivity analyses excluding highly correlated outcomes. Firstly, we did a conservative sensitivity analysis including only one outcome per comparison (that is, the outcome with the largest number of randomised controlled trials) from each Cochrane review. Secondly, we did a sensitivity analysis including outcomes based on their ranking in the summary of findings tables in the identified Cochrane reviews (from top to bottom). For example, for the α linolenic acid intervention or exposure, the outcomes of coronary heart disease, cardiovascular disease, and cardiovascular mortality are likely to be highly correlated. Because cardiovascular mortality was mentioned first in the summary of findings table, cardiovascular mortality was chosen to be included, while the other two outcomes were excluded (supplementary table 3). Finally, a sensitivity analysis was also performed for Cochrane reviews that included both randomised controlled trials and cohort studies.

We obtained pooled estimates through a random effects meta-analysis model.31 We assessed heterogeneity through the I2 and τ2 statistics.31 32 The τ2 statistic was estimated by the Paule and Mandel method,33 which is the recommended method for binary outcomes and performs well also with continuous ones.34 Furthermore, we calculated 95% prediction intervals to show the range of possible values for the difference between BoEs of randomised controlled trials and those of cohort studies that might be observed in future comparisons. We conducted all the meta-analyses using the R package meta.35

Patient and public involvement

We did not involve patients or members of the public when we selected the research question, designed the study, interpreted the results, or wrote the manuscript. Although there was no direct patient and public involvement in this paper owing to the methodological design of our study, we asked a member of the public to read our manuscript after submission.

Results

The literature search identified 333 systematic reviews (Cochrane reviews) of randomised controlled trials, of which 65 full texts were assessed for inclusion (supplementary fig 1, and supplementary table 4), and 33 were included in this study.26 28 29 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 We found 3318 systematic reviews of cohort studies, from which 46 systematic reviews of cohort studies (with matching systematic reviews of randomised controlled trials) were included (supplementary fig 2, and supplementary table 4).17 18 21 22 23 24 25 27 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 Two of the Cochrane reviews contained also cohort studies and were therefore included in this study.26 36

Overall, we included 97 diet-disease outcome pairs of randomised controlled trials and cohort studies (that is, estimates based on BoE from trials matched with those based on BoE from cohort studies related to the association between diet and disease; supplementary table 5). We recalculated 34 pooled estimates from 21 systematic reviews.21 24 25 26 27 40 48 56 61 71 72 77 80 83 86 89 91 92 96 101 102 The number of primary studies contributing to the 97 diet-disease outcome pairs ranged from 1 to 64 (median 6) for BoE from randomised controlled trials, and from 1 to 68 (median 7) for BoE from cohort studies (overall >950 trials and >750 cohort studies). The total number of participants ranged from 56 to 211 957 for BoE from randomised controlled trials, and from 2563 to 1 797 670 for BoE from cohort studies. Of the identified 97 diet-disease outcome pairs, 83 were included in the meta-analysis (71 binary, 12 continuous). We could not include 14 diet-disease outcome pairs in the meta-analysis (reasons in supplementary table 6).

The interventions or exposures investigated in the identified systematic reviews could be categorised into micronutrients (n=47), dietary approach (n=19), fatty acids (n=17), food groups (n=5), fibre (n=4), phytonutrients (n=3), and food (n=2). Across the BoE of randomised controlled trials, the intervention was either given in the form of dietary supplements (n=43), dietary intake (n=38), or both (n=16). Interventions on intake were mainly attempts to modify dietary intake via dietary advice or dietary counselling to reduce, for example, fat or sodium intake, but dietary adherence to these interventions was mainly not assessed in the primary systematic reviews. Across the BoE of cohort studies, the exposure measured was dietary intake (n=69), nutrient status (n=16), dietary supplements (n=8), dietary intake and dietary supplements (n=2), or dietary intake and nutrient status (n=2).

The type of intake or exposure between both BoEs was the same for dietary intake across 36 diet-disease outcome pairs and for dietary supplements across eight diet-disease outcome pairs, respectively. The diseases clusters included cardiovascular disease (n=22), intermediate disease markers (n=22), pregnancy outcomes (n=17), all cause mortality (n=15), cancer (n=12), eye disease (n=3), neurodegenerative disease (n=3), bone health (n=2), and type 2 diabetes (n=1). All Cochrane reviews evaluated risk of bias, whereas the Newcastle-Ottawa scale was the most often used instrument to evaluate study quality for BoE of cohort studies (n=48; mean rating 7.5). Certainty of evidence was rated for 48 BoE of randomised controlled trials using GRADE: very low (n=5), low (n=16), moderate (n=14), and high (n=13). For 10 BoE of cohort studies rated (for two outcomes NutriGrade104 was used), the certainty of evidence was measured: very low (n=8), low (n=1), moderate (n=1). Detailed study characteristics including effect estimates, description of population, age, description of intervention or comparator, outcomes, range study length, and risk of bias or study quality of primary studies included in each diet disease pair are given in supplementary tables 7-12.

Similarities

Of 97 diet-disease outcome pairs, none was rated as more or less identical, 57 (59%) were similar but not identical, and 40 (41%) were broadly similar. Interventions or exposures rated as broadly similar accounted for most PI/ECO dissimilarities overall (n=17/40; 42.5%). Of 83 diet-disease outcome pairs included in the meta-analysis, 57 (69%) were similar but not identical and 26 (31%) were broadly similar. Interventions or exposures rated as broadly similar accounted for most PI/ECO dissimilarities overall (n=17/26; 65%). Supplementary table 13 shows additional information.

Statistical heterogeneity

Across individual meta-analyses of randomised controlled trials, the mean I2 was 21% (τ2=0.018), whereas the median I2 was 2% (τ2=0). The heterogeneity (I2) was lower for binary outcomes (mean I2=19%; median I2=0%) than for continuous outcomes (I2=31%; I2=23%). Across individual meta-analyses of cohort studies, the mean I2 was 47% (τ2=0.023), whereas the median I2 was 54% (τ2=0.01). The heterogeneity was lower for binary outcomes (mean I2=44%; median I2=48%) than for continuous outcomes (I2=81%; I2=86%; supplementary table 14).

Pooled estimate

Overall, 83 diet-disease outcome pairs were included in the meta-analysis. For binary outcomes, 71 pairs were included. The treatment effects were more often larger in the BoE of cohort studies (n=44) than in the BoE of randomised controlled trials (n=25), and for two outcome pairs the treatment effects were of similar magnitude (supplementary table 5). The risk ratio was <1 across 64 BoE from cohort studies, whereas the risk ratio was ≥1 across seven. The risk ratio was <1 across 48 BoE from randomised controlled trials, whereas it was ≥1 in 23 BoE from randomised controlled trials. For continuous outcomes, the treatment effects were more often larger in the BoE of randomised controlled trials (n=7) than in the BoE of cohort studies (n=5). For eight outcomes, we observed a risk ratio difference greater than 0.25 between the BoE from randomised controlled trials compared with the BoE from cohort studies (but only two instances showed a strong difference >0.5).

The pooled estimate, using the BoE of cohort studies as the reference group, showed that on average the BoE of randomised controlled trials had slightly different estimates compared to that of cohort studies (ratio of risk ratios 1.09 (95% confidence interval 1.04 to 1.14); 95% prediction interval 0.81 to 1.46; fig 1 and table 1). The prediction interval indicated that the difference could be much more substantial, in either direction. Substantial heterogeneity (I2=68%; τ2=0.021) was observed. When the BoE from cohort studies with a risk ratio <1 versus ≥1 were analysed separately, the ratios of risk ratios were 1.12 (95% confidence interval 1.07 to 1.17; I2=60%; τ2=0.016; 95% prediction interval 0.87 to 1.45; n=64; supplementary fig 3) and 0.89 (0.79 to 1.00; 44%; 0.013; 0.64 to 1.24; n=7; supplementary fig 4), respectively.

Fig 1.

Fig 1

Forest plot of comparisons between bodies of evidence from randomised controlled trials versus those from cohort studies for binary outcomes as pooled ratio of risk ratios. CS=cohort study; RCT=randomised controlled trial; RR=risk ratio; omega 3=omega 3 fatty acid; omega 6=omega 6 fatty acid; Abdelhamid 2018a=reference 37; Abdelhamid 2018b=reference 38; Bjelakovic 2014a=reference 43; Bjelakovic 2014=reference 42; Chowdhury 2014a=reference 67; Chowdhury 2014b=reference 75; Hooper 2015b=reference 51; Rees 2013b=reference 58; Zhang 2016a=reference 93

Table 1.

Overview of main results for binary outcomes (n=71 diet-disease outcome pairs)

Ratio of risk ratios (95% CI) Heterogeneity (I2 (%); τ2) 95% prediction interval
Main analysis 1.09 (1.04 to 1.14) 68; 0.021 (0.81 to 1.46)
Stratified by overall PI/ECO similarity degree
 More or less identical
 Similar but not identical 1.05 (1.00 to 1.10) 61; 0.016 (0.81 to 1.36)
 Broadly similar 1.20 (1.10 to 1.30) 62; 0.020 (0.88 to 1.63)
Stratified by type of dietary intervention/exposure
 Fatty acids 1.05 (1.00 to 1.10) 26; 0.002 (0.94 to 1.17)
 Micronutrients 1.14 (1.06 to 1.22) 69; 0.031 (0.79 to 1.63)
 Dietary approach 0.99 (0.90 to 1.09) 61; 0.010 (0.77 to 1.27)
Stratified by type of intake/exposure (randomised controlled trials v cohort studies)
 Intake v intake 0.98 (0.93 to 1.04) 4; 0.00 (0.90 to 1.07)
 Supplements v supplements 1.08 (0.98 to 1.20) 0 (0.95 to 1.23)
 Intake and supplements v intake 1.06 (0.99 to 1.14) 62, 0.007 (0.86 to 1.30)
 Supplements v intake 1.07 (0.95 to 1.21) 74; 0.049 (0.65 to 1.75)
 Supplements v status 1.29 (1.17 to 1.42) 54; 0.018 (0.94 to 1.77)
Stratified by type of outcomes
 Cardiovascular disease 1.05 (0.99 to 1.12) 57; 0.010 (0.85 to 1.31)
 All cause mortality 1.17 (1.11 to 1.23) 75; 0.006 (0.99 to 1.39)
 Cancer 1.07 (0.98 to 1.16) 20; 0.007 (0.86 to 1.31)
 Pregnancy outcomes 0.93 (0.75 to 1.15) 70; 0.092 (0.46 to 1.88)

I2=inconsistency; PI/ECO=population, intervention or exposure, comparator, outcome.

For continuous outcome pairs (n=12), we observed no differences between randomised controlled trials and cohort studies, apart from smaller systolic and diastolic blood pressure estimates in the BoE of randomised controlled trials. The pooled difference of mean differences was −1.95 mm Hg (95% confidence interval −3.84 to −0.06; I2=59%; τ2=1.64; 95% prediction interval −22.33 to 18.43) for systolic blood pressure estimates and −2.36 mm Hg (−3.16 to −1.57); I2=0%; τ2=0; −3.16 to −1.57) for diastolic blood pressure estimates (fig 2).

Fig 2.

Fig 2

Forest plot of comparisons between bodies of evidence from randomised controlled trials versus those from cohort studies for continuous outcomes as pooled difference of mean differences. CS=cohort study; RCT=randomised controlled trial; MD=mean difference; Rees 2013a=reference 57

Sensitivity analyses excluding highly correlated outcomes

The first sensitivity analysis, where only one outcome (with the largest number of randomised controlled trials) was chosen from each Cochrane review (n=31), confirmed the findings of the primary analysis (ratio of risk ratios 1.14 (95% confidence interval 1.06 to 1.22); I2=72%; τ2=0.027; 95% prediction interval 0.81 to 1.61; supplementary fig 5). In the second sensitivity analysis, 50 diet-disease outcome pairs were included in the meta-analysis and showed also similar results (1.12 (1.06 to 1.18); I2=68%; τ2=0.023; 0.82 to 1.52; supplementary fig 6).

Subgroup analyses

Subgroup analyses showed that estimates were marginally different in BoE of randomised controlled trials compared to BoE of cohort studies for PI/ECO matched outcomes pairs that were similar but not identical (ratio of risk ratios 1.05 (95% confidence interval 1.00 to 1.10); I2=61%; τ2=0.016; 95% prediction interval 0.81 to 1.36) and substantially in disagreement for those pairs that were broadly similar (1.20 (1.10 to 1.30); I2=62%; τ2=0.020; 0.88 to 1.63; fig 3 and fig 4). Regarding specific PI/ECO components, the dissimilarity in intervention or exposure explained most of the differences. The broadly similar category showed substantial disagreement (1.29 (1.18 to 1.41); I2=52%; τ2=0.015; 0.97 to 1.71), whereas the more or less identical category led to estimates highly in agreement (0.98 (0.91 to 1.04); I2=7%; τ2=0.00; 0.88 to 1.09; supplementary fig 7).

Fig 3.

Fig 3

Forest plot of comparisons between bodies of evidence from randomised controlled trials versus cohort studies for binary outcomes as pooled ratio of risk ratios, stratified by the similarity degree of PI/ECO category (similar but not identical). CS=cohort study; PI/ECO=population, intervention or exposure, comparator, outcome; RCT=randomised controlled trial; RR=risk ratio; omega 3=omega 3 fatty acid; omega 6=omega 6 fatty acid; Abdelhamid 2018a=reference 37; Abdelhamid 2018b=reference 38; Bjelakovic 2014b=reference 42; Chowdhury 2014a=reference 67; Hooper 2015b=reference 51; Rees 2013b=reference 58

Fig 4.

Fig 4

Forest plot of comparisons between bodies of evidence from randomised controlled trials versus cohort studies for binary outcomes as pooled ratio of risk ratios, stratified by the similarity degree of PI/ECO category (broadly similar). CS=cohort study; PI/ECO=population, intervention or exposure, comparator, outcome; RCT=randomised controlled trial; RR=risk ratio; MCI=mild cognitive impairment; omega 3=omega 3 fatty acid; Abdelhamid 2018a=reference 37; Bjelakovic 2014a=reference 43; Bjelakovic 2014b=reference 42; Chowdhury 2014a=reference 67; Chowdhury 2014b=reference 75; Rees 2013b=reference 58; Zhang 2016a=reference 93

Subgroup analyses by type of dietary intervention or exposure showed different results between the two BoEs for micronutrient comparisons (mainly dietary supplements; ratio of risk ratios 1.14 (95% confidence interval 1.06 to 1.22); I2=69%; τ2=0.031; 95% prediction interval 0.79 to 1.63), whereas no differences for all other types of intervention were observed (supplementary fig 8). After observing substantial heterogeneity for several types of comparison for intervention or exposure (dietary approaches, τ2=0.01, I2=61%; micronutrients, τ2=0.03, I2=69%), we further explored it, by considering the type of intake or exposure (supplementary fig 9). We noticed that when the type of intake of the interventions and exposures was the same in both BoE, the estimates were similar (for dietary intake, ratio of risk ratios 0.98 (95% confidence interval 0.93 to 1.04); I2=4%; τ2=0.00; 95% prediction interval 0.90 to 1.07; for dietary supplements, 1.08 (0.98 to 1.20; I2=0%; τ2=0.00; 0.95 to 1.23); in both cases, no heterogeneity was observed). The comparison of dietary intake and dietary supplements in randomised controlled trials versus dietary intake in cohort studies also showed similar estimates (ratio of risk ratios 1.06 (95% confidence interval 0.99 to 1.14); I2=62%; τ2=0.007; 95% prediction interval 0.86 to 1.30).

Heterogeneity was present when considering low fat dietary approaches. Focusing on dietary fatty acids only (n-3, n-6, and polyunsaturated fatty acids), we observed no heterogeneity (supplementary fig 10). Moreover, the comparisons between dietary supplements in randomised controlled trials versus dietary intake in cohort studies showed similar estimates but substantial heterogeneity and a wide prediction interval (ratio of risk ratios 1.07 (95% confidence interval 0.95 to 1.21); I2=74%; τ2=0.049; 95% prediction interval 0.65 to 1.75). By excluding pregnancy outcomes (because all other comparisons focused on non-communicable diseases), and β carotene and vitamin A comparisons (for the outcome of mortality), which are known to increase mortality at higher doses in randomised controlled trials,105 heterogeneity disappeared (supplementary fig 11). The comparisons of dietary supplements versus nutrient status was judged to have the lowest similarity degree for intervention or exposure and also showed substantial differences between randomised controlled trials and cohort studies (1.29 (1.17 to 1.42); I2=54%; τ2=0.018; 0.94 to 1.77). Heterogeneity was driven by vitamin D comparisons (supplementary fig 9). After stratifying the analysis by outcome type, we observed differences for overall mortality (1.17 (1.11 to 1.23); I2=75%; τ2=0.006; 0.99 to 1.39), bone health (1.46 (1.16 to 1.84); I2=67%; τ2=0.019; 1.02 to 2.08), and eye disease (1.14 (1.03 to 1.26); I2=36%; τ2=0.003; 0.44 to 2.96; supplementary fig 12).

The findings of the subgroup analyses are supported by sensitivity analyses excluding outcomes that are likely to be highly correlated (supplementary figs 13-17), and when BoE from cohort studies with a risk ratio <1 were analysed separately (supplementary figs 18-22). The subgroup analyses for BoE from cohort studies with a RR ≥1, need to be interpreted with caution due to the very small number of comparisons (n=7) (supplementary figs 23-27).

Additional analyses

We also performed a multi-level meta-analysis, considering the pairs as grouping factor, and the findings of the primary analysis were confirmed (ratio of risk ratios 1.08 (95% confidence interval 1.02 to 1.13); I2=68%). The sensitivity analysis comparing BoEs from randomised controlled trials versus cohort studies of the two Cochrane reviews (based on six outcomes) also confirmed the findings of the primary analysis (supplementary fig 28).

Discussion

Summary of findings

This large meta-epidemiological study identified and compared empirical data to determine the extent to which diet-disease association estimates of BoE from randomised controlled trials and cohort studies are in agreement. Overall, 97 diet-disease outcome pairs were identified and 83 were suitable for meta-analysis. No outcome pair was rated as more or less identical, according to PI/ECO similarity. On average, the difference in the pooled results between the two BoEs was small, but given that prediction intervals are wide and statistical heterogeneity was in part substantial in cohort studies, important differences or potential bias in individual comparisons or individual studies cannot be excluded.

We investigated possible factors for the observed heterogeneity, finding that PI/ECO dissimilarities, in particular the comparisons of dietary supplements in randomised controlled trials and nutrient status in cohort studies, explained most of the differences. When the type of intake or exposure between both BoE was identical, the estimates were similar (and the analysis showed low statistical heterogeneity). For pooled estimate of continuous outcomes, no differences were observed between randomised controlled trials and cohort studies, except for smaller systolic and diastolic blood pressure estimates in the BoE of trials.

Comparison with other studies

Nutrition field

A technical review published in 2013 identified 34 diet-disease outcome pairs of systematic reviews of randomised controlled trials and large, single randomised controlled trials (>1000 participants) versus systematic reviews of case-control or cohort studies and one large observational study (>5000 participants).15 Similar to our findings, 22 (65%) of 34 diet-disease outcome pairs were in the same direction, and had no evidence of significant disagreement (z score not statistically significant).15 By comparison, our study included a larger sample of outcome pairs and a larger number of participants. We also thoroughly matched PI/ECO criteria, pooled the effect estimate to generate a ratio of risk ratios and difference of mean differences, and also investigated the possible factors of disagreement.

Trepanowski and Ioannidis106 recently argued that many prominent epidemiological associations (including highly cited studies on α tocopherol, β carotene, vitamin C, vitamin D, selenium, calcium, and low fat diets) have not been corroborated by large randomised controlled trials or meta-analyses.107 108 Their statement, however, is not based on a systematic evaluation, and does not accord with our findings, where pooling BoEs of randomised controlled trials and cohort studies showed on average minor differences. On the contrary, Satija and colleagues14 argued that, when randomised controlled trials are able to successfully examine diet-disease relations, their results are more often in line with those of cohort studies. Our findings seem to accord with Satija and colleagues’ conclusions, although the pooled estimate showed some differences between both BoEs, and the prediction intervals were wide.

Medical field

Anglemyer et al109 conducted a methodological Cochrane review, including systematic reviews and overviews of reviews in different medical fields, which showed little difference between the results obtained from randomised controlled trials and observational studies (cohort and case-control studies). Their result when comparing BoE from randomised controlled trials with BoE from observational studies (ratio of odds ratios 1.08 (95% confidence interval 0.96 to 1.22)) is similar to our findings (ratio of risk ratios 1.09 (1.04 to 1.14)). The difference in the estimates, in terms of point estimate, was more in disagreement with pharmacological studies (ratio of odds ratios 1.17 (0.95 to 1.43)). This difference corresponds to our findings regarding micronutrient interventions (mainly as dietary supplements), where randomised controlled trials showed differences compared with cohort studies (ratio of risk ratios 1.14 (1.06 to 1.22)). However, the methodological review by Anglemyer et al did not conduct PI/ECO matching, did not calculate 95% prediction intervals, and did not differentiate various types of intervention and outcomes.

When comparing our results with findings from meta-epidemiological studies investigating the impact of design features of randomised controlled trials, the magnitude of differences were similar. For example, lack of reporting of adequate random sequence generation, allocation sequence concealment, and double blinding tend to overestimate intervention effects (ratio of odds ratiosranging from 0.87 to 0.93).110 The extent of overestimation was lower for objective outcomes (eg, mortality) and therefore unlikely to be influenced by knowledge of the intervention received.110 A recent meta-epidemiological study of 142 meta-analyses found no evidence for difference in treatment effect between randomised controlled trials with and without patients, healthcare providers, or outcome assessors blinded to treatment.111 The impact of design features of randomised controlled trials and cohort studies has not yet been explored in the field of nutrition using meta-epidemiological methods.

Potential implications

What constitutes best evidence in nutrition research has been debated extensively, and whether it comes from randomised controlled trials, which are considered the ideal methodology for causal inference and in which the effects of a dietary change on disease or intermediate disease markers are evaluated experimentally.112 However, most randomised controlled trials of dietary interventions are short and do often not target patient relevant outcomes such as morbidity or mortality. Further limitations are the difficulty of inducing and maintaining dietary changes in the long term, and the low adherence to a specific dietary regimen that often occurs.7 Moreover, although several long term trials have been conducted (eg, the Women’s Health Initiative Dietary Modification trial,113 or the PREDIMED study114), the costs of such large scale dietary trials are challenging.115 Cohort studies, on the other hand, provide methodologically less robust information regarding causality, but are usually considered more applicable for nutrition research.

In general, the two BoE (trials v cohort studies) often differ in terms of study populations (inclusion and exclusion criteria, comparison group), different exposure levels (dose, duration, sources), different outcomes, and different sample sizes and follow-up durations, as shown in our study. For example, in randomised controlled trials of supplements such as selenium, participants might already have an adequate selenium status.116 117 Observational studies, on the other hand, usually include participants with a broader range of selenium status.118 Therefore, the comparison of risk ratios from trials and cohort studies might not be a perfect match.

Randomised controlled trials are experimental studies, where participants are usually given fixed intake levels in the form of dietary supplements or where investigators try to modify dietary intake via dietary advice or dietary counselling.112 Cohort studies are observational and do not actively intervene in the behaviour of study participants, and participant groups are based on reported intake (or status) of study participants (eg, higher v lower sodium intake), thus implying a variable difference in intake levels.112 In each case, the estimated risk ratio reflects the direction of the effect and an indication of the strength of the association. In our study, BoE of randomised controlled trials were predominantly based on dietary supplements (n=43) and dietary intake (n=38), whereas BoE of cohort studies investigated mainly dietary intake (n=69). In randomised controlled trials of dietary supplements, which are similar to drug trials, study participants are randomly assigned into study arms, thus balancing measured and unmeasured potential confounders across the comparison groups, allowing differences in the outcome measure to be attributed to the dietary intervention.20 By contrast, cohort studies are prone to residual confounding, and the direction and magnitude of risk ratio is influenced by the variables included in the statistical models built to estimate the effect, and by the potential measurement error of dietary factors (which is also a problem in long term randomised controlled trials on dietary intake) and all other factors.112

Despite these circumstances, our study matched PI/ECO similarities between the two most important study designs in nutrition research. Because of the above described differences, no diet-disease outcome pair was rated as more or less identical. However, when the type of intake or exposure between both BoEs was identical, the estimates were similar (and the analysis showed low statistical heterogeneity). Such PI/ECO dissimilarities are often present even between studies with the same design, which contributes to statistical heterogeneity in the primary meta-analyses.20 A meta-epidemiological study of meta-analyses such as ours could further increase the complexity of heterogeneity, but the exploration of statistical heterogeneity among comparisons between different types of dietary intake or exposure yielded plausible explanations in our study.

At the systematic review level, the established approach to evaluate the credibility of results from primary studies is risk-of-bias assessment. In our study, all Cochrane reviews used the Cochrane risk-of-bias tool, whereas for cohort studies the Newcastle-Ottawa scale was mainly used, as reported elsewhere.104 Risk-of-bias assessment is an integral part of the GRADE approach, which rates the certainty of evidence based on a BoE.119 120 According to the GRADE approach, the certainty of evidence is initially determined by study design: a BoE from randomised controlled trials starts with high certainty, whereas a BoE from observational studies starts with low certainty owing to confounding and selection bias (if the ROBINS tool is used, both BoEs start as high certainty).119 120 Use of the GRADE approach, especially in relation to the risk-of-bias assessment, is challenging and could lead to excessive downgrading. For example, GRADE users might inappropriately double count the risk of confounding and selection bias by downgrading the initial certainty of the BoE to low, followed by further downgrading due to unknown confounders.119 121 When GRADE was used, very low and high certainty of evidence ratings accounted for 10% and 27% of ratings for BoE of randomised controlled trials, respectively, compared to 80% and 0% for BoE of cohort studies, respectively. In this regard, we could show in a recent methodological survey that very low and high certainty of evidence ratings accounted for 61% and 1% of ratings in systematic reviews of observational studies, respectively, compared to 16% and 5% in systematic reviews of randomised controlled trials, respectively.122

A recent cross sectional study has shown that very few Cochrane nutrition reviews include observational studies (2%),123 which has been criticised.124 BoE from cohort studies can strengthen or complement BoE from randomised controlled trials, and vice versa, so our meta-epidemiological study provides support for integration based on thorough assessment of PI/ECO similarities, and could be a starting point for future work.

Strengths and limitations

Our study had several strengths. Firstly, we included a large sample of diet-disease outcome pairs (n=97), based on more than 950 randomised controlled trials and 750 cohort studies, with both study designs considered as the most reliable in nutrition research.6 Secondly, the conducted PI/ECO matching process was novel, and provided important insights in our understanding of which factors are associated with disagreement. Thirdly, the data extraction was extensive, retrieving information on the description of interventions or exposures and comparators, population, study design, risk of bias of the primary studies, and certainty of the evidence for each diet-disease pair. Fourthly, we conducted various statistical analyses, such as recalculating 34 pooled estimates, converting odds ratios and hazard ratios into risk ratios, including binary and continuous outcomes, and pooling the estimates across all diet-disease outcome pairs. Finally, the exploration of factors potentially associated with disagreement through a priori planned subgroup analyses for PI/ECO dissimilarities, types of intervention, intake, exposure, and outcomes was an additional strength of this study.

The present study also had several limitations. Firstly, we only searched the Cochrane Database of Systematic Reviews to identify systematic reviews of randomised controlled trials, after which we screened all eligible Cochrane reviews to see if they also included cohort studies. It would have been ideal if Cochrane reviews also included cohort studies to ensure better comparability in terms of systematic review methodological approaches (eg, search strategy, risk-of-bias assessment, or GRADE rating). However, only two Cochrane reviews contained also cohort studies. Therefore, we searched and matched systematic reviews of cohort studies retrieved from Medline, which might have affected the validity of our findings. However, a sensitivity analysis comparing BoE from randomised controlled trials versus cohort studies of the two Cochrane reviews (based on six outcomes) confirmed the findings of the primary analysis. Secondly, the meta-analyses in the present research could have themselves had limitations, from the primary data and how the evidence has been summarised; therefore, readers should consider the original studies for more detailed information.

Thirdly, although the PI/ECO matching process was conducted by two reviewers, subjectivity cannot be ruled out completely. A quantitative PI/ECO matching approach might have been more objective but has yet to be developed. Another limitation, particularly for the BoE from cohort studies, is that some studies were included multiple times, and from the systematic reviews, the same original studies were used with the same exposure but for different outcomes. The sensitivity analysis where only one outcome (with the largest number of randomised controlled trials) was chosen from each Cochrane review (n=31) confirmed the findings of the primary analysis. Moreover, sensitivity analyses excluding outcomes that were likely to be highly correlated showed similar findings as the primary analysis (ratio of risk ratio 1.12 (95% confidence interval 1.06 to 1.18)). We also did a multi-level meta-analysis considering the pairs as grouping factor, which confirmed the findings of the primary analysis. A further limitation was that we did not explore other potential factors for disagreement, such as dietary adherence in the included primary randomised controlled trials.

Finally, the impact of potential bias in cohort studies should be considered at three levels: generally causing a systematic bias; causing bias in individual comparisons; and causing bias in individual studies, hence leading to heterogeneity in individual meta-analyses. Considering these three levels in order, our findings implied no strong indication for systematic bias overall (meta-analytical ratio of risk ratios close to 1), but this did not exclude an important risk for bias in individual comparisons, or in individual studies. The ratio of risk ratios in individual comparisons was often not equal to 1; in fact, we observed a different direction (eg, ratio of risk ratios <1 in 24 of 71 comparisons) and magnitude (eg, four comparisons showed a ratio of risk ratios <0.75, and 13 comparisons showed a ratio of risk ratios >1.25). Furthermore, the prediction intervals indicated that the difference between the two BoEs as shown in our study (ratio of risk ratios 1.09) could be much more substantial in either direction (95% prediction interval 0.81 to 1.46). By exploring factors associated with those differences, we found that PI/ECO dissimilarity (mainly in type of intake) was the main driver. Finally, statistical heterogeneity was higher in the individual meta-analyses of cohort studies (mean I2=47%; τ2=0.023) than in those of randomised controlled trials (mean I2=21%; τ2=0.018), possibly due potential bias in individual cohort studies.

Conclusion

On average, the difference in the pooled results between the two BoEs was small, but with wide prediction intervals and some substantial statistical heterogeneity in cohort studies, important differences or potential bias in individual comparisons or individual studies cannot be excluded. We investigated possible factors for the observed heterogeneity, finding that PI/ECO dissimilarities, especially for the comparisons of dietary supplements in randomised controlled trials and nutrient status in cohort studies. When the type of intake or exposure between both BoEs was identical, the estimates were similar (and the analysis showed low statistical heterogeneity). Nevertheless, the comparison between randomised controlled trials and cohort studies should be interpreted very carefully.

These findings provide valuable insights towards better understanding the integration of both BoEs of randomised controlled trials and cohort studies in prospective nutrition evidence syntheses. Considering that few Cochrane nutrition reviews include cohort studies, and that most of the evidence in nutrition actually comes from cohort studies, evidence based guidance is urgently needed on how to incorporate and possibly integrate both BoEs in nutrition evidence syntheses. We consider that this information is needed not only for research purposes but also for decision making processes and tailoring better evidence based dietary guidelines.

What is already known on this topic:

  • Previously, several randomised controlled trials comparing dietary with control interventions have failed to replicate the (presumably protective) associations between dietary factors and risk of non-communicable diseases observed in large scale cohort studies

  • However, some consistent findings between cohort studies and randomised controlled trials have been also reported

  • Systematic evaluation of the two bodies of evidence between trials and cohort studies, with an investigation on factors for disagreement, has not yet been conducted

What this study adds

  • The difference in results between the two study designs was small

  • However, with wide prediction interval and some substantial statistical heterogeneity in cohort studies, differences or potential bias cannot be excluded

  • When the type of intake or exposure was identical between the bodies of evidence from randomised controlled trials and cohort studies, estimates were similar and the analysis showed low statistical heterogeneity

Web extra.

Extra material supplied by authors

Web appendix: Data supplement

schl063523.ww.pdf (10.5MB, pdf)

Contributors: LS, SB, JB, NB, SSW, JZ, and JJM designed the research. LS and SB analysed the data and wrote the first draft of the paper. LS, SB, JB, NB, SSW, JZ, BN, and JJM interpreted the data, read the manuscript, and approved the final version. LS and JJM are guarantors. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding: Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - Projektnummer 459430615 and Forschungskommission der Medizinischen Fakultät Freiburg. The funders had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report, or decision to submit the article for publication.

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: support from the Deutsche Forschungsgemeinschaft and Forschungskommission der Medizinischen Fakultät Freiburg for the submitted work; LS is a member of the GRADE working group; JJM is a member of the GRADE working group and director of the Freiburg GRADE centre; no other relationships or activities that could appear to have influenced the submitted work.

The lead authors (the manuscript’s guarantors) affirm that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.

Dissemination to participants and related patient and public communities: Results from our meta-epidemiological study will be disseminated through a press release (Cochrane Germany), the website (eg, Cochrane Germany, Cochrane Nutrition), blog (eg, wissenwaswirkt.org), and social media (Twitter). Results dissemination will also be targeted to health professionals, including dietitians, nutritionists, physicians, guidelines developers, and scientists.

Provenance and peer review: Not commissioned; externally peer reviewed.

Ethics statements

Ethical approval

Not required.

Data availability statement

Data were extracted from published meta-analyses, all of which are available and accessible.

References

  • 1.GBD 2017 Causes of Death Collaborators . Global, regional, and national age-sex-specific mortality for 282 causes of death in 195 countries and territories, 1980-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 2018;392:1736-88. 10.1016/S0140-6736(18)32203-7.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Afshin A, Sur PJ, Fay KA, et al. GBD 2017 Diet Collaborators . Health effects of dietary risks in 195 countries, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 2019;393:1958-72. 10.1016/S0140-6736(19)30041-8.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Kromhout D, Spaaij CJK, de Goede J, Weggemans RM. The 2015 Dutch food-based dietary guidelines. Eur J Clin Nutr 2016;70:869-78. 10.1038/ejcn.2016.52.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.US Department of Health and Human Services and US Department of Agriculture. 2015-2020 Dietary Guidelines for Americans. 8th ed. December 2015. https://health.gov/our-work/food-nutrition/2015-2020-dietary-guidelines/guidelines/.
  • 5.Ioannidis JPA. The challenge of reforming nutritional epidemiologic research. JAMA 2018;320:969-70. 10.1001/jama.2018.11025.  [DOI] [PubMed] [Google Scholar]
  • 6.Pan A, Lin X, Hemler E, Hu FB. Diet and Cardiovascular Disease: Advances and Challenges in Population-Based Studies. Cell Metab 2018;27:489-96. 10.1016/j.cmet.2018.02.017.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Schwingshackl L, Schünemann HJ, Meerpohl JJ. Improving the trustworthiness of findings from nutrition evidence syntheses: assessing risk of bias and rating the certainty of evidence. Eur J Nutr 2020. 10.1007/s00394-020-02464-1.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Humphrey LL, Fu R, Rogers K, Freeman M, Helfand M. Homocysteine level and coronary heart disease incidence: a systematic review and meta-analysis. Mayo Clin Proc 2008;83:1203-12. 10.4065/83.11.1203.  [DOI] [PubMed] [Google Scholar]
  • 9.Koushik A, Hunter DJ, Spiegelman D, et al. Intake of the major carotenoids and the risk of epithelial ovarian cancer in a pooled analysis of 10 cohort studies. Int J Cancer 2006;119:2148-54. 10.1002/ijc.22076.  [DOI] [PubMed] [Google Scholar]
  • 10.Rapola JM, Virtamo J, Ripatti S, et al. Randomised trial of alpha-tocopherol and beta-carotene supplements on incidence of major coronary events in men with previous myocardial infarction. Lancet 1997;349:1715-20. 10.1016/S0140-6736(97)01234-8.  [DOI] [PubMed] [Google Scholar]
  • 11.Stampfer MJ, Hennekens CH, Manson JE, Colditz GA, Rosner B, Willett WC. Vitamin E consumption and the risk of coronary disease in women. N Engl J Med 1993;328:1444-9. 10.1056/NEJM199305203282003.  [DOI] [PubMed] [Google Scholar]
  • 12.Schatzkin A, Lanza E, Corle D, et al. Polyp Prevention Trial Study Group . Lack of effect of a low-fat, high-fiber diet on the recurrence of colorectal adenomas. N Engl J Med 2000;342:1149-55. 10.1056/NEJM200004203421601.  [DOI] [PubMed] [Google Scholar]
  • 13.Yusuf S, Dagenais G, Pogue J, Bosch J, Sleight P, Heart Outcomes Prevention Evaluation Study Investigators . Vitamin E supplementation and cardiovascular events in high-risk patients. N Engl J Med 2000;342:154-60. 10.1056/NEJM200001203420302.  [DOI] [PubMed] [Google Scholar]
  • 14.Satija A, Stampfer MJ, Rimm EB, Willett W, Hu FB. Perspective: Are Large, Simple Trials the Solution for Nutrition Research? Adv Nutr 2018;9:378-87. 10.1093/advances/nmy030.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Moorthy D, Chung M, Lee J, et al. AHRQ Technical Reviews. Concordance Between the Findings of Epidemiological Studies and Randomized Trials in Nutrition: An Empirical Evaluation and Citation Analysis: Nutritional Research Series. Vol 6. Agency for Healthcare Research and Quality (US), 2013. [PubMed] [Google Scholar]
  • 16.Murad MH, Wang Z. Guidelines for reporting meta-epidemiological methodology research. Evid Based Med 2017;22:139-42. 10.1136/ebmed-2017-110713.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Li J, Guasch-Ferré M, Li Y, Hu FB. Dietary intake and biomarkers of linoleic acid and mortality: systematic review and meta-analysis of prospective cohort studies. Am J Clin Nutr 2020;112:150-67. 10.1093/ajcn/nqz349.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Jayedi A, Rashidy-Pour A, Parohan M, Zargar MS, Shab-Bidar S. Dietary Antioxidants, Circulating Antioxidant Concentrations, Total Antioxidant Capacity, and Risk of All-Cause Mortality: A Systematic Review and Dose-Response Meta-Analysis of Prospective Observational Studies. Adv Nutr 2018;9:701-16. 10.1093/advances/nmy040.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Bassler D, Briel M, Montori VM, et al. STOPIT-2 Study Group . Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis. JAMA 2010;303:1180-7. 10.1001/jama.2010.310.  [DOI] [PubMed] [Google Scholar]
  • 20.Higgins JP, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Reviews of Interventions Version 6.1. 2020. https://training.cochrane.org/handbook.
  • 21.Chia AR, Chen LW, Lai JS, et al. Maternal Dietary Patterns and Birth Outcomes: A Systematic Review and Meta-Analysis. Adv Nutr 2019;10:685-95. 10.1093/advances/nmy123.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Goodwill AM, Szoeke C. A Systematic Review and Meta-Analysis of The Effect of Low Vitamin D on Cognition. J Am Geriatr Soc 2017;65:2161-8. 10.1111/jgs.15012.  [DOI] [PubMed] [Google Scholar]
  • 23.Hu L, Zhang Y, Wang X, et al. Maternal Vitamin D Status and Risk of Gestational Diabetes: a Meta-Analysis. Cell Physiol Biochem 2018;45:291-300. 10.1159/000486810.  [DOI] [PubMed] [Google Scholar]
  • 24.Mijatovic-Vukas J, Capling L, Cheng S, et al. Associations of Diet and Physical Activity with Risk for Gestational Diabetes Mellitus: A Systematic Review and Meta-Analysis. Nutrients 2018;10:E698. 10.3390/nu10060698.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Tous M, Villalobos M, Iglesias L, Fernández-Barrés S, Arija V. Vitamin D status during pregnancy and offspring outcomes: a systematic review and meta-analysis of observational studies. Eur J Clin Nutr 2020;74:36-53. 10.1038/s41430-018-0373-x.  [DOI] [PubMed] [Google Scholar]
  • 26.Vinceti M, Filippini T, Del Giovane C, et al. Selenium for preventing cancer. Cochrane Database Syst Rev 2018;1:CD005195. 10.1002/14651858.CD005195.pub4.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Yuan Y, Tai W, Xu P, et al. Association of maternal serum 25-hydroxyvitamin D concentrations with risk of preeclampsia: a nested case-control study and meta-analysis. J Matern Fetal Neonatal Med 2021;34:1576-85. 10.1080/14767058.2019.1640675.  [DOI] [PubMed] [Google Scholar]
  • 28.Al-Khudairy L, Flowers N, Wheelhouse R, et al. Vitamin C supplementation for the primary prevention of cardiovascular disease. Cochrane Database Syst Rev 2017;3:CD011114. 10.1002/14651858.CD011114.pub2.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Rees K, Takeda A, Martin N, et al. Mediterranean-style diet for the primary and secondary prevention of cardiovascular disease. Cochrane Database Syst Rev 2019;3:CD009825. 10.1002/14651858.CD009825.pub3.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Altman DG, Bland JM. Interaction revisited: the difference between two estimates. BMJ 2003;326:219. 10.1136/bmj.326.7382.219.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Riley RD, Higgins JP, Deeks JJ. Interpretation of random effects meta-analyses. BMJ 2011;342:d549. 10.1136/bmj.d549.  [DOI] [PubMed] [Google Scholar]
  • 32.Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ 2003;327:557-60. 10.1136/bmj.327.7414.557.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Paule RC, Mandel J. Consensus values and weighting factors. National Institute of Standards and Technology, 1982 10.6028/jres.087.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Veroniki AA, Jackson D, Viechtbauer W, et al. Methods to estimate the between-study variance and its uncertainty in meta-analysis. Res Synth Methods 2016;7:55-79. 10.1002/jrsm.1164.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Balduzzi S, Rücker G, Schwarzer G. How to perform a meta-analysis with R: a practical tutorial. Evid Based Ment Health 2019;22:153-60. 10.1136/ebmental-2019-300117.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Jin H, Leng Q, Li C. Dietary flavonoid for preventing colorectal neoplasms. Cochrane Database Syst Rev 2012;(8):CD009350. 10.1002/14651858.CD009350.pub2.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Abdelhamid AS, Brown TJ, Brainard JS, et al. Omega-3 fatty acids for the primary and secondary prevention of cardiovascular disease. Cochrane Database Syst Rev 2018;11:CD003177. 10.1002/14651858.CD003177.pub4.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Abdelhamid AS, Martin N, Bridges C, et al. Polyunsaturated fatty acids for the primary and secondary prevention of cardiovascular disease. Cochrane Database Syst Rev 2018;11:CD012345. 10.1002/14651858.CD012345.pub3.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Adler AJ, Taylor F, Martin N, Gottlieb S, Taylor RS, Ebrahim S. Reduced dietary salt for the prevention of cardiovascular disease. Cochrane Database Syst Rev 2014;(12):CD009217. 10.1002/14651858.CD009217.pub3.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Avenell A, Mak JC, O’Connell D. Vitamin D and vitamin D analogues for preventing fractures in post-menopausal women and older men. Cochrane Database Syst Rev 2014;(4):CD000227. 10.1002/14651858.CD000227.pub4.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Bjelakovic G, Nikolova D, Gluud LL, Simonetti RG, Gluud C. Antioxidant supplements for prevention of mortality in healthy participants and patients with various diseases. Cochrane Database Syst Rev 2012;(3):CD007176. 10.1002/14651858.CD007176.pub2.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Bjelakovic G, Gluud LL, Nikolova D, et al. Vitamin D supplementation for prevention of cancer in adults. Cochrane Database Syst Rev 2014;(6):CD007469. 10.1002/14651858.CD007469.pub2.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Bjelakovic G, Gluud LL, Nikolova D, et al. Vitamin D supplementation for prevention of mortality in adults. Cochrane Database Syst Rev 2014;(1):CD007470. 10.1002/14651858.CD007470.pub3.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Cormick G, Ciapponi A, Cafferata ML, Belizán JM. Calcium supplementation for prevention of primary hypertension. Cochrane Database Syst Rev 2015;(6):CD010037. 10.1002/14651858.CD010037.pub2.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.El Dib R, Gameiro OL, Ogata MS, et al. Zinc supplementation for the prevention of type 2 diabetes mellitus in adults with insulin resistance. Cochrane Database Syst Rev 2015;(5):CD005525. 10.1002/14651858.CD005525.pub3.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Hartley L, Igbinedion E, Holmes J, et al. Increased consumption of fruit and vegetables for the primary prevention of cardiovascular diseases. Cochrane Database Syst Rev 2013;(6):CD009874. 10.1002/14651858.CD009874.pub2.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Hartley L, May MD, Loveman E, Colquitt JL, Rees K. Dietary fibre for the primary prevention of cardiovascular disease. Cochrane Database Syst Rev 2016;(1):CD011472. 10.1002/14651858.CD011472.pub2.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Hemmingsen B, Gimenez-Perez G, Mauricio D, Roqué I Figuls M, Metzendorf MI, Richter B. Diet, physical activity or both for prevention or delay of type 2 diabetes mellitus and its associated complications in people at increased risk of developing type 2 diabetes mellitus. Cochrane Database Syst Rev 2017;12:CD003054. 10.1002/14651858.CD003054.pub4.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Hofmeyr GJ, Lawrie TA, Atallah AN, Torloni MR. Calcium supplementation during pregnancy for preventing hypertensive disorders and related problems. Cochrane Database Syst Rev 2018;10:CD001059. 10.1002/14651858.CD001059.pub5.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Hooper L, Summerbell CD, Thompson R, et al. Reduced or modified dietary fat for preventing cardiovascular disease. Cochrane Database Syst Rev 2012;(5):CD002137. 10.1002/14651858.CD002137.pub3.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Hooper L, Martin N, Abdelhamid A, Davey Smith G. Reduction in saturated fat intake for cardiovascular disease. Cochrane Database Syst Rev 2015;(6):CD011737. 10.1002/14651858.CD011737.  [DOI] [PubMed] [Google Scholar]
  • 52.Hooper L, Abdelhamid A, Bunn D, Brown T, Summerbell CD, Skeaff CM. Effects of total fat intake on body weight. Cochrane Database Syst Rev 2015;(8):CD011834. 10.1002/14651858.CD011834.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Hooper L, Al-Khudairy L, Abdelhamid AS, et al. Omega-6 fats for the primary and secondary prevention of cardiovascular disease. Cochrane Database Syst Rev 2018;11:CD011094. 10.1002/14651858.CD011094.pub4.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Kelly SA, Hartley L, Loveman E, et al. Whole grain cereals for the primary or secondary prevention of cardiovascular disease. Cochrane Database Syst Rev 2017;8:CD005051. 10.1002/14651858.CD005051.pub3.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Mathew MC, Ervin AM, Tao J, Davis RM. Antioxidant vitamin supplementation for preventing and slowing the progression of age-related cataract. Cochrane Database Syst Rev 2012;(6):CD004567. 10.1002/14651858.CD004567.pub2.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Palacios C, Trak-Fellermeier MA, Martinez RX, et al. Regimens of vitamin D supplementation for women during pregnancy. Cochrane Database Syst Rev 2019;10:CD013446. 10.1002/14651858.CD013446.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Rees K, Dyakova M, Wilson N, Ward K, Thorogood M, Brunner E. Dietary advice for reducing cardiovascular risk. Cochrane Database Syst Rev 2013;(12):CD002128. 10.1002/14651858.CD002128.pub5.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Rees K, Hartley L, Day C, Flowers N, Clarke A, Stranges S. Selenium supplementation for the primary prevention of cardiovascular disease. Cochrane Database Syst Rev 2013;(1):CD009671. 10.1002/14651858.CD009671.pub2.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Rutjes AW, Denton DA, Di Nisio M, et al. Vitamin and mineral supplementation for maintaining cognitive function in cognitively healthy people in mid and late life. Cochrane Database Syst Rev 2018;12:CD011906. 10.1002/14651858.CD011906.pub2.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Sydenham E, Dangour AD, Lim WS. Omega 3 fatty acid for the prevention of cognitive decline and dementia. Cochrane Database Syst Rev 2012;(6):CD005379. 10.1002/14651858.CD005379.pub3.  [DOI] [PubMed] [Google Scholar]
  • 61.Tieu J, Shepherd E, Middleton P, Crowther CA. Dietary advice interventions in pregnancy for preventing gestational diabetes mellitus. Cochrane Database Syst Rev 2017;1:CD006674. 10.1002/14651858.CD006674.pub3.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Usinger L, Reimer C, Ibsen H. Fermented milk for hypertension. Cochrane Database Syst Rev 2012;(4):CD008118. 10.1002/14651858.CD008118.pub2.  [DOI] [PubMed] [Google Scholar]
  • 63.Yao Y, Suo T, Andersson R, et al. Dietary fibre for the prevention of recurrent colorectal adenomas and carcinomas. Cochrane Database Syst Rev 2017;1:CD003430. 10.1002/14651858.CD003430.pub2.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Keats EC, Haider BA, Tam E, Bhutta ZA. Multiple-micronutrient supplementation for women during pregnancy. Cochrane Database Syst Rev 2019;3:CD004905. 10.1002/14651858.CD004905.pub6.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.De-Regil LM, Peña-Rosas JP, Fernández-Gaxiola AC, Rayco-Solon P. Effects and safety of periconceptional oral folate supplementation for preventing birth defects. Cochrane Database Syst Rev 2015;(12):CD007950. 10.1002/14651858.CD007950.pub3.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66.Wan Y, Zheng J, Wang F, Li D. Fish, long chain omega-3 polyunsaturated fatty acids consumption, and risk of all-cause mortality: a systematic review and dose-response meta-analysis from 23 independent prospective cohort studies. Asia Pac J Clin Nutr 2017;26:939-56. 10.6133/apjcn.072017.01.  [DOI] [PubMed] [Google Scholar]
  • 67.Chowdhury R, Warnakula S, Kunutsor S, et al. Association of dietary, circulating, and supplement fatty acids with coronary risk: a systematic review and meta-analysis. Ann Intern Med 2014;160:398-406. 10.7326/M13-1788.  [DOI] [PubMed] [Google Scholar]
  • 68.Schlesinger S, Neuenschwander M, Schwedhelm C, et al. Food Groups and Risk of Overweight, Obesity, and Weight Gain: A Systematic Review and Dose-Response Meta-Analysis of Prospective Studies. Adv Nutr 2019;10:205-18. 10.1093/advances/nmy092.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69.Wei J, Hou R, Xi Y, et al. The association and dose-response relationship between dietary intake of α-linolenic acid and risk of CHD: a systematic review and meta-analysis of cohort studies. Br J Nutr 2018;119:83-9. 10.1017/S0007114517003294.  [DOI] [PubMed] [Google Scholar]
  • 70.Zhu Y, Bo Y, Liu Y. Dietary total fat, fatty acids intake, and risk of cardiovascular disease: a dose-response meta-analysis of cohort studies. Lipids Health Dis 2019;18:91. 10.1186/s12944-019-1035-2.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Aburto NJ, Ziolkovska A, Hooper L, Elliott P, Cappuccio FP, Meerpohl JJ. Effect of lower sodium intake on health: systematic review and meta-analyses. BMJ 2013;346:f1326. 10.1136/bmj.f1326.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 72.Leyvraz M, Chatelan A, da Costa BR, et al. Sodium intake and blood pressure in children and adolescents: a systematic review and meta-analysis of experimental and observational studies. Int J Epidemiol 2018;47:1796-810. 10.1093/ije/dyy121.  [DOI] [PubMed] [Google Scholar]
  • 73.Aune D, Keum N, Giovannucci E, et al. Dietary intake and blood concentrations of antioxidants and the risk of cardiovascular disease, total cancer, and all-cause mortality: a systematic review and dose-response meta-analysis of prospective studies. Am J Clin Nutr 2018;108:1069-91. 10.1093/ajcn/nqy097.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74.Feng Y, Cheng G, Wang H, Chen B. The associations between serum 25-hydroxyvitamin D level and the risk of total fracture and hip fracture. Osteoporos Int 2017;28:1641-52. 10.1007/s00198-017-3955-x.  [DOI] [PubMed] [Google Scholar]
  • 75.Chowdhury R, Kunutsor S, Vitezova A, et al. Vitamin D and risk of cause specific death: systematic review and meta-analysis of observational cohort and randomised intervention studies. BMJ 2014;348:g1903. 10.1136/bmj.g1903.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 76.Han J, Guo X, Yu X, et al. 25-Hydroxyvitamin D and Total Cancer Incidence and Mortality: A Meta-Analysis of Prospective Cohort Studies. Nutrients 2019;11:E2295. 10.3390/nu11102295.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 77.Hossain S, Beydoun MA, Beydoun HA, Chen X, Zonderman AB, Wood RJ. Vitamin D and breast cancer: A systematic review and meta-analysis of observational studies. Clin Nutr ESPEN 2019;30:170-84. 10.1016/j.clnesp.2018.12.085.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 78.Zhang L, Wang S, Che X, Li X. Vitamin D and lung cancer risk: a comprehensive review and meta-analysis. Cell Physiol Biochem 2015;36:299-305. 10.1159/000374072.  [DOI] [PubMed] [Google Scholar]
  • 79.Jayedi A, Zargar MS. Dietary calcium intake and hypertension risk: a dose-response meta-analysis of prospective cohort studies. Eur J Clin Nutr 2019;73:969-78. 10.1038/s41430-018-0275-y.  [DOI] [PubMed] [Google Scholar]
  • 80.Fernández-Cao JC, Warthon-Medina M, H Moran V, et al. Zinc Intake and Status and Risk of Type 2 Diabetes Mellitus: A Systematic Review and Meta-Analysis. Nutrients 2019;11:E1027. 10.3390/nu11051027.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 81.Schwingshackl L, Schwedhelm C, Hoffmann G, et al. Food Groups and Risk of Hypertension: A Systematic Review and Dose-Response Meta-Analysis of Prospective Studies. Adv Nutr 2017;8:793-803. 10.3945/an.117.017178.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 82.Schwingshackl L, Bogensberger B, Hoffmann G. Diet Quality as Assessed by the Healthy Eating Index, Alternate Healthy Eating Index, Dietary Approaches to Stop Hypertension Score, and Health Outcomes: An Updated Systematic Review and Meta-Analysis of Cohort Studies. J Acad Nutr Diet 2018;118:74-100.e11. 10.1016/j.jand.2017.08.024.  [DOI] [PubMed] [Google Scholar]
  • 83.Newberry SJ, Chung M, Shekelle PG, et al. Vitamin D and Calcium: A Systematic Review of Health Outcomes (Update). Evid Rep Technol Assess (Full Rep) 2014;(217):1-929. 10.23970/AHRQEPCERTA217.  [DOI] [PubMed] [Google Scholar]
  • 84.Seidelmann SB, Claggett B, Cheng S, et al. Dietary carbohydrate intake and mortality: a prospective cohort study and meta-analysis. Lancet Public Health 2018;3:e419-28. 10.1016/S2468-2667(18)30135-X.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 85.Noto H, Goto A, Tsujimoto T, Noda M. Low-carbohydrate diets and all-cause mortality: a systematic review and meta-analysis of observational studies. PLoS One 2013;8:e55030. 10.1371/journal.pone.0055030.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 86.Sartorius K, Sartorius B, Madiba TE, Stefan C. Does high-carbohydrate intake lead to increased risk of obesity? A systematic review and meta-analysis. BMJ Open 2018;8:e018449. 10.1136/bmjopen-2017-018449.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 87.de Souza RJ, Mente A, Maroleanu A, et al. Intake of saturated and trans unsaturated fatty acids and risk of all cause mortality, cardiovascular disease, and type 2 diabetes: systematic review and meta-analysis of observational studies. BMJ 2015;351:h3978. 10.1136/bmj.h3978.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 88.Pan A, Chen M, Chowdhury R, et al. α-Linolenic acid and risk of cardiovascular disease: a systematic review and meta-analysis. Am J Clin Nutr 2012;96:1262-73. 10.3945/ajcn.112.044040.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 89.Ye EQ, Chacko SA, Chou EL, Kugizaki M, Liu S. Greater whole-grain intake is associated with lower risk of type 2 diabetes, cardiovascular disease, and weight gain. J Nutr 2012;142:1304-13. 10.3945/jn.111.155325.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 90.Jiang H, Yin Y, Wu CR, et al. Dietary vitamin and carotenoid intake and risk of age-related cataract. Am J Clin Nutr 2019;109:43-54. 10.1093/ajcn/nqy270.  [DOI] [PubMed] [Google Scholar]
  • 91.Kastorini CM, Milionis HJ, Esposito K, Giugliano D, Goudevenos JA, Panagiotakos DB. The effect of Mediterranean diet on metabolic syndrome and its components: a meta-analysis of 50 studies and 534,906 individuals. J Am Coll Cardiol 2011;57:1299-313. 10.1016/j.jacc.2010.09.073.  [DOI] [PubMed] [Google Scholar]
  • 92.Xiang S, Dai Z, Man C, Fan Y. Circulating Selenium and Cardiovascular or All-Cause Mortality in the General Population: a Meta-Analysis. Biol Trace Elem Res 2020;195:55-62. 10.1007/s12011-019-01847-8.  [DOI] [PubMed] [Google Scholar]
  • 93.Zhang X, Liu C, Guo J, Song Y. Selenium status and cardiovascular diseases: meta-analysis of prospective observational studies and randomized controlled trials. Eur J Clin Nutr 2016;70:162-9. 10.1038/ejcn.2015.78.  [DOI] [PubMed] [Google Scholar]
  • 94.Soltani S, Jayedi A, Shab-Bidar S, Becerra-Tomás N, Salas-Salvadó J. Adherence to the Mediterranean Diet in Relation to All-Cause Mortality: A Systematic Review and Dose-Response Meta-Analysis of Prospective Cohort Studies. Adv Nutr 2019;10:1029-39. 10.1093/advances/nmz041.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 95.Rosato V, Temple NJ, La Vecchia C, Castellan G, Tavani A, Guercio V. Mediterranean diet and cardiovascular disease: a systematic review and meta-analysis of observational studies. Eur J Nutr 2019;58:173-91. 10.1007/s00394-017-1582-0.  [DOI] [PubMed] [Google Scholar]
  • 96.Doets EL, van Wijngaarden JP, Szczecińska A, et al. Vitamin B12 intake and status and cognitive function in elderly people. Epidemiol Rev 2013;35:2-21. 10.1093/epirev/mxs003.  [DOI] [PubMed] [Google Scholar]
  • 97.Zhang Y, Chen J, Qiu J, Li Y, Wang J, Jiao J. Intakes of fish and polyunsaturated fatty acids and mild-to-severe cognitive impairment risks: a dose-response meta-analysis of 21 cohort studies. Am J Clin Nutr 2016;103:330-40. 10.3945/ajcn.115.124081.  [DOI] [PubMed] [Google Scholar]
  • 98.Soedamah-Muthu SS, Verberne LD, Ding EL, Engberink MF, Geleijnse JM. Dairy consumption and incidence of hypertension: a dose-response meta-analysis of prospective cohort studies. Hypertension 2012;60:1131-7. 10.1161/HYPERTENSIONAHA.112.195206.  [DOI] [PubMed] [Google Scholar]
  • 99.Ben Q, Sun Y, Chai R, Qian A, Xu B, Yuan Y. Dietary fiber intake reduces risk for colorectal adenoma: a meta-analysis. Gastroenterology 2014;146:689-699.e6. 10.1053/j.gastro.2013.11.003.  [DOI] [PubMed] [Google Scholar]
  • 100.Aune D, Chan DS, Lau R, et al. Dietary fibre, whole grains, and risk of colorectal cancer: systematic review and dose-response meta-analysis of prospective studies. BMJ 2011;343:d6617. 10.1136/bmj.d6617.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 101.Feng Y, Wang S, Chen R, Tong X, Wu Z, Mo X. Maternal folic acid supplementation and the risk of congenital heart defects in offspring: a meta-analysis of epidemiological observational studies. Sci Rep 2015;5:8506. 10.1038/srep08506.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 102.Blencowe H, Cousens S, Modell B, Lawn J. Folic acid to reduce neonatal mortality from neural tube disorders. Int J Epidemiol 2010;39(Suppl 1):i110-21. 10.1093/ije/dyq028.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 103.Wolf HT, Hegaard HK, Huusom LD, Pinborg AB. Multivitamin use and adverse birth outcomes in high-income countries: a systematic review and meta-analysis. Am J Obstet Gynecol 2017;217:404.e1-30. 10.1016/j.ajog.2017.03.029.  [DOI] [PubMed] [Google Scholar]
  • 104.Schwingshackl L, Knüppel S, Schwedhelm C, et al. Perspective: NutriGrade: A Scoring System to Assess and Judge the Meta-Evidence of Randomized Controlled Trials and Cohort Studies in Nutrition Research. Adv Nutr 2016;7:994-1004. 10.3945/an.116.013052.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 105.Schwingshackl L, Boeing H, Stelmach-Mardas M, et al. Dietary Supplements and Risk of Cause-Specific Death, Cardiovascular Disease, and Cancer: A Systematic Review and Meta-Analysis of Primary Prevention Trials. Adv Nutr 2017;8:27-39. 10.3945/an.116.013516.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 106.Trepanowski JF, Ioannidis JPA. Perspective: Limiting Dependence on Nonrandomized Studies and Improving Randomized Trials in Human Nutrition Research: Why and How. Adv Nutr 2018;9:367-77. 10.1093/advances/nmy014.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 107.Miller PE, Perez V. Low-calorie sweeteners and body weight and composition: a meta-analysis of randomized controlled trials and prospective cohort studies. Am J Clin Nutr 2014;100:765-77. 10.3945/ajcn.113.082826.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 108.Mente A, de Koning L, Shannon HS, Anand SS. A systematic review of the evidence supporting a causal link between dietary factors and coronary heart disease. Arch Intern Med 2009;169:659-69. 10.1001/archinternmed.2009.38.  [DOI] [PubMed] [Google Scholar]
  • 109.Anglemyer A, Horvath HT, Bero L. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. Cochrane Database Syst Rev 2014;(4):MR000034. 10.1002/14651858.MR000034.pub2.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 110.Savović J, Jones HE, Altman DG, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med 2012;157:429-38. 10.7326/0003-4819-157-6-201209180-00537.  [DOI] [PubMed] [Google Scholar]
  • 111.Moustgaard H, Clayton GL, Jones HE, et al. Impact of blinding on estimated treatment effects in randomised clinical trials: meta-epidemiological study. BMJ 2020;368:l6802. 10.1136/bmj.l6802.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 112.Maki KC, Slavin JL, Rains TM, Kris-Etherton PM. Limitations of observational evidence: implications for evidence-based dietary recommendations. Adv Nutr 2014;5:7-15. 10.3945/an.113.004929.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 113.Howard BV, Manson JE, Stefanick ML, et al. Low-fat dietary pattern and weight change over 7 years: the Women’s Health Initiative Dietary Modification Trial. JAMA 2006;295:39-49. 10.1001/jama.295.1.39.  [DOI] [PubMed] [Google Scholar]
  • 114.Estruch R, Ros E, Salas-Salvadó J, et al. PREDIMED Study Investigators . Primary Prevention of Cardiovascular Disease with a Mediterranean Diet Supplemented with Extra-Virgin Olive Oil or Nuts. N Engl J Med 2018;378:e34. 10.1056/NEJMoa1800389.  [DOI] [PubMed] [Google Scholar]
  • 115.Lucey A, Heneghan C, Kiely ME. Guidance for the design and implementation of human dietary intervention studies for health claim submissions. Nutr Bull 2016;41:378-94. 10.1111/nbu.12241. [DOI] [Google Scholar]
  • 116.Lü J, Zhang J, Jiang C, Deng Y, Özten N, Bosland MC. Cancer chemoprevention research with selenium in the post-SELECT era: Promises and challenges. Nutr Cancer 2016;68:1-17. 10.1080/01635581.2016.1105267.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 117.Rayman MP. Selenium and human health. Lancet 2012;379:1256-68. 10.1016/S0140-6736(11)61452-9.  [DOI] [PubMed] [Google Scholar]
  • 118.Garland M, Morris JS, Stampfer MJ, et al. Prospective study of toenail selenium levels and cancer among women. J Natl Cancer Inst 1995;87:497-505. 10.1093/jnci/87.7.497.  [DOI] [PubMed] [Google Scholar]
  • 119.Schünemann HJ, Cuello C, Akl EA, et al. GRADE Working Group . GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence. J Clin Epidemiol 2019;111:105-14. 10.1016/j.jclinepi.2018.01.012.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 120.Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016;355:i4919. 10.1136/bmj.i4919.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 121.Schwingshackl L, Knüppel S, Schwedhelm C, et al. Reply to JJ Meerpohl et al. Adv Nutr 2017;8:790-1. 10.3945/an.117.016469.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 122.Werner SS, Binder N, Toews I, Schünemann HJ, Meerpohl JJ, Schwingshackl L. Use of GRADE in evidence syntheses published in high-impact-factor nutrition journals: A methodological survey. J Clin Epidemiol 2021;135:54-69. 10.1016/j.jclinepi.2021.02.010.  [DOI] [PubMed] [Google Scholar]
  • 123.Naude CE, Durao S, Harper A, Volmink J. Scope and quality of Cochrane reviews of nutrition interventions: a cross-sectional study. Nutr J 2017;16:22. 10.1186/s12937-017-0244-7.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 124.Truswell AS. Some problems with Cochrane reviews of diet and chronic disease. Eur J Clin Nutr 2005;59(Suppl 1):S150-4, discussion S195-6. 10.1038/sj.ejcn.1602189.  [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Web appendix: Data supplement

schl063523.ww.pdf (10.5MB, pdf)

Data Availability Statement

Data were extracted from published meta-analyses, all of which are available and accessible.


Articles from The BMJ are provided here courtesy of BMJ Publishing Group

RESOURCES