Limitations of Observational Evidence: Implications for Evidence-Based Dietary Recommendations^,

Abstract

Data from randomized controlled trials (RCTs) provide the strongest evidence for establishing relations between exposures, including dietary exposures, and health outcomes. However, not all diet and health outcome relations can be practically or ethically evaluated by using RCTs; therefore, many dietary recommendations are supported by evidence primarily from observational data, particularly those from prospective cohort studies. Although such evidence is of critical importance, limitations are often underappreciated by nutrition scientists and policymakers. This editorial review is intended to 1) highlight some of these limitations of observational evidence for diet-disease relations, including imprecise exposure quantification, collinearity among dietary exposures, displacement/substitution effects, healthy/unhealthy consumer bias, residual confounding, and effect modification; and 2) advocate for greater caution in the communication of dietary recommendations for which RCT evidence of clinical event reduction after dietary intervention is not available.

Introduction

Public health policy recommendations for nutrition are based on expert review of the totality of scientific evidence. Over time, the process of evaluating the totality of evidence has become more rigorous and, in recent years, more transparent. The principles of evidence-based medicine (EBM)⁶ have been adopted, in which a hierarchal approach to the evaluation of evidence is applied, with meta-analyses, systematic reviews, and randomized controlled trials (RCTs) considered the strongest types of evidence. As reviewed recently by Blumberg et al. (1), the Institute of Medicine first used the EBM approach beginning with the 1997 Dietary Reference Intakes, followed shortly thereafter by the 2005 Dietary Guidelines for Americans.

The RCT is the strongest study design for drawing causal inferences regarding relations between exposures, including dietary exposures, and health outcomes. However, not all diet and health relations can be practically or ethically evaluated in RCTs (2, 3). For example, it is possible that some dietary components affect disease risk over decades of chronic exposure (4). Pathologies of many diet-related diseases develop over extended periods, which would potentially require considerable subject burden and expense to evaluate in an RCT spanning more than just a few years. Consequently, rather than clinical event data (i.e., the ideal endpoint results), biomarkers of disease risk are often used to provide evidence of causal relations between diet and disease. Furthermore, the appropriateness of RCTs to adequately assess all diet-disease relations has been challenged due to the complex nature of conducting dietary interventions (1). It is widely acknowledged that causal inferences based on results from observational data that are also supported by results from RCTs evaluating the effects of interventions on clinical events represent more persuasive evidence than those drawn from observational data alone. Nevertheless, in many cases, the best available evidence may be that from intervention trials evaluating the effects of a dietary change on disease surrogates or recognized risk markers, due to limited or absent RCT data on clinical events.

According to the EBM hierarchy, the strongest study design secondary to RCTs is the prospective cohort study (Fig. 1). Although observational in nature, the prospective cohort has an important advantage over other observational designs in that the measurement of a dietary exposure precedes the development of clinical signs and symptoms of the disease, allowing the temporal sequence of the relation to be more firmly established and minimizing the risk of recall bias (cases with the disease recalling certain exposures more clearly than controls). Furthermore, prospective cohorts allow investigators an opportunity to evaluate dietary patterns and disease outcomes over long periods of time. However, just as with other types of evidence, prospective cohort studies have limitations that are important to understand in order to assess the implications of such findings for formulating public health recommendations.

FIGURE 1.

Hierarchy of research evidence. Reproduced with permission from University of Illinois at Chicago (5).

It serves public health to make recommendations on the basis of the best available evidence. Currently, a heavy reliance on observational data exists, particularly those from prospective cohort studies, for many dietary recommendations. In the authors’ view, there is a widespread underappreciation among scientists and policymakers regarding the limitations of observational data for establishing cause-effect relations between dietary exposures and health outcomes. The intent of the present review is not to minimize the importance of observational data or prospective cohort studies, because we believe that such data are critical elements of the evidence base; instead, our objective is to emphasize the limitations of these investigations and to advocate for greater caution in the communication of dietary recommendations based primarily on results from observational analyses that have not been confirmed through well-designed clinical outcomes trials.

Observational Evidence and Dietary Recommendations

Dietary guidance in the United States is primarily based on the periodic evaluations of nutrients and other dietary substances performed by the Institute of Medicine, known as the DRIs. Recommendations from the Institute of Medicine are used in the development of nutrition policy and diet planning for schools, prisons, hospitals, and nursing homes. In addition, the evidence reviewed within the DRIs serves as the framework for the Dietary Guidelines for Americans, which are issued and updated every 5 y by the USDA and the U.S. Department of Health and Human Services. Unlike the DRIs, which establish reference values for nutrients important to human health and are used primarily by health care professionals, the Dietary Guidelines place an emphasis on food-based recommendations for public use (www.myplate.gov), including federal food policies, standards for schools, and food assistance programs (www.dietaryguidelines.gov). As such, the release of the Dietary Guidelines every 5 y attracts a great deal of attention in the nutrition community and beyond.

Distilling a body of scientific evidence into dietary recommendations that promote health and reduce disease risk is challenging, particularly with regard to the evaluation and interpretation of the totality of scientific evidence. Well-informed experts may draw different conclusions after reviewing the same body of literature. For example, dietary recommendations in the United States advise restricting dietary cholesterol, whereas many other countries no longer include recommendations regarding cholesterol intake (6).

Transparency and the use of systematic reviews have been put in place to assuage some of these concerns. For the Dietary Guidelines specifically, the Nutrition Evidence Library (www.NEL.gov) was formed to provide a mechanism for critical review and evaluation of the strength of the scientific evidence to support each of the guidelines (7). For the 2010 report, the process for specific research questions (but not for the guidelines themselves) involved critiquing and grading available studies according to quality, and based on this analysis the strength of the overall body of evidence is ranked on a scale ranging from “strong” to “grade not assignable” for cases in which there is insufficient evidence to draw a reasonable conclusion (intermediate ratings include “moderate” and “limited”). The scale considers the number of studies, number of participants studied, design and quality of studies, consistency of findings across investigations, magnitude of the effect, and generalizability to the U.S. population. It should be noted that a given rating does not imply the presence of a relation, because there may be, for example, strong evidence for a positive, inverse, or no relation between a dietary exposure and disease outcome(s).

Despite the rigor in the Nutrition Evidence Library process, it is unclear whether the systematic evaluation and subsequent grading of the evidence is applied consistently across diet-disease relations. For example, the strength of the evidence for the relation between consumption of milk and milk products and bone health in children is rated as “moderate,” supported by a meta-analysis and systematic review that included ≥20 RCTs on total bone mineral content in children (8), in addition to observational evidence. The evidence regarding the relation between consumption of milk and milk products and type 2 diabetes risk was also rated as “moderate” but was based on only 4 prospective cohort studies (7).

It is acknowledged within the body of the Dietary Guidelines document that for some diet-disease relations, scientific conclusions are based on observational studies due to the absence of RCTs. This is an issue that plagues the study of nutrition. There exists very little RCT evidence for many diet-disease relations. For some health outcomes (e.g., many cancers), it would be impractical or prohibitively expensive to conduct large-scale clinical outcomes trials. There are also ethical considerations inherent to studying diet and disease relations. As Blumberg et al. (1) highlighted, RCTs for nutrient-disease relations are often limited by the lack of an adequate control group. It would be unethical to deprive a control group of an essential nutrient to compare disease outcomes with an active group receiving the nutrient. In addition, in a reasonable well-nourished participant sample, there may be an insufficient number of individuals with low enough intake to increase disease risk, resulting in inadequate separation between the control and active treatment groups to demonstrate an effect or to demonstrate an effect in a period of time practical for inclusion in an RCT. In addition, nutrient insufficiency/deficiency is only 1 model to study diet-disease associations. Others may result from excess intakes of some dietary components (e.g., energy and refined carbohydrates), which may have adverse health effects that take decades to manifest. As a result, prospective cohort studies may be the strongest evidence available.

Methodologic Limitations of Observational Data

Chance, bias, and confounding must always be considered as possible explanations for an association between an exposure and an outcome (4). A full review of the strengths and limitations of observational evidence in assessment of diet-disease relations is beyond the scope of this review. However, there are several issues inherent to the study of dietary exposures that warrant highlighting (9). Examples in the present review are drawn largely from the area of nutrition and cardiovascular disease (CVD) for several reasons. CVDs are common sources of morbidity and mortality in the United States and have thus been studied extensively. In addition, risk factors for CVD events have been well characterized, and the authors have all been involved in the investigation of the influence of dietary interventions on CVD risk and/or risk factors.

Measurement error.

It is widely acknowledged that dietary intake tools are associated with substantial error and therefore represent imperfect measures of exposure (10). In some instances, this represents random error, which produces imprecise estimates for exposure and disease diagnosis, and will tend to bias diet exposure-disease relation results toward the null (11, 12). In other cases, the error is systematic, such as the known tendency for participants recording dietary data to underreport energy intake (13–15). Furthermore, the degree of underestimation appears to increase with increasing BMI (16), which can affect estimates for intakes of other nutrients. In addition, there is a known tendency for certain types of foods to be underreported, including desserts, sugar-sweetened beverages, and alcoholic beverages (15, 17). Changes in the food supply and dietary habits over time, incomplete information in food and nutrient databases, and changes in commercial food products due to formula optimization that are not rapidly reflected in nutrient databases may also contribute to inaccurate assessments of dietary exposures. For example, the composition of some products may change with the relative costs of different ingredients, resulting in fluidity of the formulas used in manufacturing to minimize production costs.

In many cohorts, dietary exposures are measured years before the onset of the disease under study. Without periodic dietary assessment and application of complex, time-dependent exposure and covariate analysis, changes in diet over time may confound diet-disease associations (18–20). Finally, categories of foods constructed from FFQ data such as “vegetables” or “red meat” may include many types of foods and cooking methods, which are defined differently by various investigators, resulting in limited comparability across studies.

Collinearity.

The highly correlated nature of nutrients and dietary components adds complexity to the interpretation of results from observational studies. For example, dietary fiber is commonly found in foods that are also rich in magnesium and B vitamins, making it difficult to establish the independence of associations between fiber intake and diseases. This can be a problem where nutrient databases lack complete information about the nutrient content of foods, and where other sources (e.g., drinking water for magnesium) may make accurate assessment difficult in geographically diverse samples. Intakes of foods or nutrients may also be associated with nonnutrient food components that are bioactive but not fully represented in nutrient databases (e.g., flavonoids and isoflavones). Multivariate statistical models used to adjust for confounding may result in “overadjustment” when dietary exposures are highly correlated, or produce markedly different point estimates of the exposure-disease association depending on which correlated variables are included in the model. Reports from observational studies investigating a particular diet-disease relation vary markedly in the potential confounders and effect modifiers included in statistical models, even within the same cohort.

Displacement/substitution effects.

An association between the intake of a food or nutrient and a disease outcome could be an indication of a harmful or protective effect of the dietary exposure under study, or it could reflect the result of the displacement of other food(s) and/or nutrient(s). Because dietary intake is constrained by energy consumption, eating more of a food or beverage containing metabolizable energy will leave less room within the daily diet for other foods and/or beverages in the absence of weight or body composition changes. Therefore, an association between higher intake of a food or nutrient and a disease outcome might be attributable to a favorable or harmful effect of the exposure or the displacement of the food or nutrient under study.

Healthy or unhealthy consumer bias.

Intake of a food, or category of food, may be associated with other nondietary variables that may be difficult or impossible to fully adjust for in statistical modeling (21). This phenomenon may account, at least in part, for one of the most dramatic instances in which results from observational and RCT evidence diverged: the association between use of postmenopausal estrogen-progestin therapy and risk of coronary heart disease (CHD).

In the years before the publication of the results from intervention studies of postmenopausal estrogen-progestin therapy in postmenopausal women (22, 23), a large and consistent body of observational evidence had accumulated that showed that use of estrogen-progestin therapy (or estrogen without a progestin) was associated with a 40–60% lower incidence of CHD events (24–26). In 1 prominent example published in the New England Journal of Medicine (27), investigators reported that current users of estrogen-progestin therapy had a relative risk of major coronary disease that was 61% lower than that of never-users (RR: 0.39; 95% CI: 0.19, 0.78) after multivariate adjustment for potential confounders among postmenopausal women participating in the Nurses’ Health Study. Women who used estrogen-progestin therapy differed in a number of ways from those who did not use such therapy (Table 1), but the lower risk of coronary disease persisted after adjustment for numerous potential confounders. In contrast, in the Women’s Health Initiative RCT (22), in which >16,000 postmenopausal women were randomly assigned to receive estrogen-progestin therapy with one of the most commonly prescribed postmenopausal hormone therapy formulations in the United States at the time the study was initiated (0.625 mg/d conjugated equine estrogens + 2.5 mg/d medroxyprogesterone acetate) or placebo, CHD incidence after mean follow-up of 5.2 y was 29% higher among women assigned to estrogen-progestin therapy (HR: 1.29; nominal 95% CI: 1.02, 1.63) relative to placebo.

TABLE 1.

Characteristics of study participants in the Nurses’ Health Study in 1990 according to postmenopausal hormone use¹

Characteristic	Never used hormones (n = 27,034)	Current use, estrogen with progestin (n = 6224)	Difference2
			%
Parental myocardial infarction before the age of 60 y, %	29.6	20.6	−30
Hypertension, %	32.9	27.3	−17
Diabetes mellitus, %	5.8	2.7	−53
High serum cholesterol3, %	35.6	41.6	17
Moderate smoker4, %	9.4	4.6	−51
Bilateral oophorectomy, %	4.2	8.9	112
Past use of oral contraceptives, %	30.6	46.4	52
Multivitamin use, %	24.6	42.2	72
Vitamin E use, %	9.5	18.1	91
Aspirin use, %	33.6	48.3	44
Mean age, y	60.1	56.7	−6
Mean age at menopause, y	50.9	49.2	−3
Mean BMI, kg/m2	26.3	24.3	−8
Mean alcohol consumption, g/d	4.7	6.0	28
Mean consumption of saturated fat, g/d	31.2	41.4	33

¹Adapted with permission from Grodstein et al. (27).

²Calculated as 100 × (value in current-user group – value in never-used group)/value in never-used group.

³Defined as ≥200 mg/dL.

⁴Defined as 15–24 cigarettes/d.

Many factors may have contributed to the discrepant findings between the observational and RCT data in this instance, including older mean age of trial participants than women in whom postmenopausal hormone therapy was generally initiated in clinical practice (28). However, it is likely that women who used estrogen-progestin therapy during the 1970s and early 1990s differed in important ways from those who did not, and such differences could not be fully accounted for in extensive and well-executed multivariate analyses. The divergence between the observational and RCT data suggests the presence of unmeasured confounders, or residual confounding from variables that were measured, in the observational study results. It should also be pointed out that medication use is an exposure that is likely measured with greater accuracy than many dietary variables.

It is common for numerous variables to differ between those with low and high intakes of foods with a reputation for being “healthy” (e.g., whole grains, dietary fibers, fruit and vegetables, antioxidant vitamins) or “unhealthy” (e.g., red meat, sugar-sweetened beverages, quick-service meals). Pan et al. (29) reported a multivariate adjusted HR of 1.45 (95% CI: 1.30, 1.63) for cardiovascular mortality for the fifth versus the first quintile of total red meat consumption in the Nurses’ Health Study cohort after adjustment for “major lifestyle and dietary risk factors.” However, comparing characteristics of those in the extreme quintiles of total red meat consumption reveals marked differences beyond red meat intake, including differences of ≥20% for reported physical activity, energy intake, fish consumption, and prevalence values for smoking and elevated cholesterol (Table 2). The reported results are consistent with the widely held view that red meat consumption increases risk of CVD mortality. Nevertheless, because participants in observational cohorts self-select their level of the dietary exposure, red meat in this example, and those selecting low or high consumption differ in numerous attributes, one must consider the possibility of “unhealthy user” effects.

TABLE 2.

Characteristics of study participants in the Nurses’ Health Study 1980–2008 according to average red meat consumption¹

	Red meat intake
Characteristic	Lowest quintile	Highest quintile	Difference2
			%
Median red meat intake, servings/d	0.5	3.1	520
Family history of early MI, %	19.4	19.0	−2
Hypertension, %	15.2	16.4	8
Diabetes mellitus, %	1.6	2.9	81
High serum cholesterol3, %	6.0	4.7	−22
Current smoker, %	25.5	31.6	24
Current hormone use, %	20.6	20.7	1
Aspirin use, %	43.2	49.1	14
Mean age, y	47.3	46.0	−3
Mean BMI, kg/m2	23.9	24.7	3
Mean caloric intake, kcal/d	1202	2030	69
Current multivitamin use, %	37.9	32.3	−15
Mean alcohol consumption, g/d	5.8	6.6	14
Activity, MET-h/wk	16.9	12.4	−27

¹n = 83,644. Adapted with permission from Pan et al. (29). MET-h, metabolic equivalent task hours; MI, myocardial infarction.

²Calculated as 100 × (value in highest quintile – value in lowest quintile)/value in lowest quintile.

³Defined as ≥200 mg/dL.

Confounding and effect modification.

A common limitation in the investigation of dietary exposures and risk of cardiovascular events is that some nondietary risk factors may not be measured, or may be measured imprecisely (e.g., adjustment for a major risk factor such as hypercholesterolemia or hypertension as a dichotomous variable: no vs. yes), resulting in residual confounding. As an example, cardiorespiratory fitness and physical activity have both been shown to be strong predictors of CVD risk and are often effect modifiers for other risk factors (30). However, most large-scale cohorts that have investigated dietary exposures and CVD risk have not measured cardiorespiratory fitness, and physical activity is generally assessed with a questionnaire that provides an imprecise estimate of habitual physical activity.

The influence of a dietary exposure may vary according to characteristics of the individual, thus producing different effects in subgroups of the population studied. A low-fat, high-carbohydrate diet may produce favorable changes in the CVD risk factor profile in those with normal insulin sensitivity but exacerbate “atherogenic dyslipidemia” in those with insulin resistance or the metabolic syndrome (31).

Genetic assessments are beginning to yield information about subgroups that may differ in response to various dietary and nondietary factors. For example, concentrations of long-chain omega-3 FAs in RBC phospholipids (a surrogate for tissue concentrations) appear to vary as a result of polymorphisms in the encoding genes for desaturase enzymes that are responsible for conversion of precursor molecules (e.g., α-linolenic acid, 18:3n−3) into the longer-chain n−3 (ω-3) FAs EPA (20:5n−3) and DHA (22:6n−3) (32). The possible implication of this finding is that individuals with some FA desaturase variants may have greater potential for a favorable effect of consuming longer-chain n−3 FAs than those with variants who are synthesizing more of these compounds in vivo, particularly in light of the relatively low intakes of long-chain n−3 PUFAs that have been hypothesized to provide protection against ischemia-triggered ventricular arrhythmia (33). However, this hypothesis will need to be verified in prospective RCTs.

Research on the influence of gut microbiota on risks of certain diseases (e.g., obesity) and responses to some interventions (e.g., consumption of fermentable dietary fibers or gastric by-pass surgery) suggests the potential for responses to dietary exposures that are dependent on the individual’s gut microbiome (34, 35). These examples illustrate how the relation between a dietary component or pattern, and a disease may vary according to the prevalence of genotypes or phenotypes that influence response within the population under study.

Most diet-disease relations represent relatively modest associations.

Most associations between dietary exposures and disease outcomes are modest, with RR or HR estimates <2.0 (or >0.50 for inverse associations). The closer a point estimate for an association is to the null (OR, RR, or HR of 1.0), the greater is the likelihood that alternative explanations such as confounding or bias could account for a statistically significant association. The discussion above covers some sources of bias and confounding that can complicate the evaluation of dietary exposures and disease risk. In addition, many dietary exposures may be investigated within a cohort as potential predictors of a disease outcome. Therefore, a high potential exists for type I statistical errors (false positives), particularly when such investigations are from exploratory data mining, and not prespecified hypotheses (36). Published reports often fail to distinguish between exploratory and prespecified analyses.

RCT Results Have Been Mixed When Testing Dietary Hypotheses Generated from Observational Data

Recommendations are based implicitly or explicitly on a judgment of causality between a dietary exposure and 1 or more health outcomes. Hill (37) proposed 9 criteria for judging causality. The most important of these are strength of association, consistency of association, dose-response, biologic plausibility, and concordance with available data from other sources, including experimental evidence, especially that from clinical trials (38). There are numerous examples in nutrition in which hypotheses that appeared very promising based on results from observational and mechanistic studies to establish biologic plausibility were not unequivocally supported by subsequent RCT data. Several examples are summarized below for hypotheses related to dietary exposures and CHD risk to illustrate this point.

The failure of RCTs to confirm a benefit for a dietary intervention does not necessarily indicate that the hypothesis being tested is not valid. Baseline susceptibility and nutritional status, small effect size, and limited duration of exposure have been raised as factors that may lead to failure of a dietary intervention to produce the anticipated benefit in an RCT, even if the underlying hypothesis is valid (39–43). Nevertheless, the authors view recommendations for which hypotheses generated from observational and mechanistic studies have been confirmed by results from RCTs to be more compelling than those for which benefits have not been demonstrated in RCTs with clinical event endpoints.

B-vitamin supplementation, homocysteine, and CHD risk.

A meta-analysis of 29 observational studies showed a consistent, positive association between elevated plasma homocysteine and CHD risk (44). In light of this association, and the demonstrated effect of B-vitamin supplementation to lower homocysteine concentrations, there was considerable excitement about the use of folic acid and cobalamin supplementation for their potential to provide cardioprotection by lowering blood concentrations of homocysteine. However, 8 large RCTs failed to demonstrate reduced cardiovascular events or mortality with folic acid and cobalamin supplementation, despite substantial lowering of homocysteine concentrations, generally to mean concentrations in the normal range (39, 41, 45–51).

Antioxidant vitamins supplementation and CHD risk.

Oxidative modification of lipoprotein particles in the subendothelial space is believed to be a central feature of the process by which atherosclerotic lesions develop (52). Results from observational studies have generally shown higher antioxidant vitamin intake to be associated with lower CHD risk (53–55). For example, a meta-analysis of 15 cohort studies published before May 2007 reported pooled estimates for RR for the top third versus the bottom third of vitamin C intake of 0.85 (95% CI: 0.73, 0.95) and vitamin E intake of 0.78 (95% CI: 0.63, 0.89). Results from a number of RCTs have failed to demonstrate that supplementation of the diet with antioxidant vitamins individually, or in combinations, reduces CVD events (56–59).

Polyunsaturated fat and CHD risk.

Results from large cohort studies have often suggested an inverse association between the intakes of PUFAs and CHD risk. For example, Oh et al. (18) reported a multivariable adjusted RR of 0.77 (95% CI: 0.62, 0.95) comparing the highest with the lowest quintiles of linoleic acid (18:2n−6) intake after 20 y of follow-up in the Nurses’ Health Study. Jakobsen et al. (60) estimated from a meta-analysis of 11 cohort studies that substitution of 5% of energy from SFAs with PUFAs (of which linoleic acid is the major constituent in the U.S. diet) was associated with an HR in multivariate analyses for coronary events of 0.87 (95% CI: 0.77, 0.97) and for coronary death of 0.74 (95% CI: 0.61, 0.89).

In a meta-analysis of results from RCTs, Mozaffarian et al. (61) found a 10% reduction in CHD risk (RR: 0.90; 95% CI: 0.83, 0.97) for each 5% increase in energy from PUFAs, without evidence for statistical heterogeneity between results from 8 RCTs (Q-statistic P = 0.13; I² = 37%). Meta-regression identified study duration as an independent determinant of risk reduction (P = 0.017), with studies of longer duration showing greater benefits. In contrast, Ramsden et al. (62) performed a meta-analysis of results from RCTs and reported that those which provided linoleic acid as the main intervention showed a trend toward an increase in CHD mortality with a pooled HR estimate of 1.33 (95% CI: 0.99, 1.79), whereas the trials that included an intervention that was a mixture of ω-3 and ω-6 PUFAs showed a trend toward lower CHD mortality, with an HR of 0.81 (95% CI: 0.64, 1.03). When pooled, the HR was 0.98 (95% CI: 0.82, 1.19), suggesting no clear evidence of benefit regarding CHD mortality. These results illustrate the complexity of these issues and may prompt a reexamination of the body of evidence relating intake of specific PUFAs and CHD risk.

Mediterranean diet and CHD risk—an example of alignment between observational and RCT findings.

The examples above illustrate some of the difficulties that have been experienced in confirming promising hypotheses derived from observational studies regarding dietary interventions for CHD risk reduction. This should not be interpreted that the authors view observational evidence as flawed or uninformative. Quite the contrary, we believe that observational data can be extremely useful, and the strongest recommendations should be reserved for those cases in which findings from observational investigations and RCTs align.

The recent publication of results from the Prevención con Dieta Mediterránea (PREDIMED) study underscores this point. Findings from prospective cohort studies have consistently reported that a traditional Mediterranean eating pattern characterized by high intakes of olive oil, nuts, fruit, vegetables, and mostly whole-grain cereals, along with moderate consumption of fish, poultry, and wine, and low consumption of sweets, dairy products, and processed meats is associated with low risk of CVD morbidity and mortality (63). The results from the PREDIMED primary prevention trial support those from the Lyon Heart Study (64), a secondary prevention trial. Both studies showed significantly lower CVD event rates among those randomly assigned to receive instruction on following a Mediterranean-type eating pattern and provided with some foods such as extra-virgin olive oil, mixed nuts (i.e., walnuts, almonds, and hazelnuts), or a high–linolenic acid spread to support this eating pattern compared with control groups who received instruction to follow a diet low in fat and saturated fat (63–65).

Implications for Evidence-Based Reviews and Meta-Analyses

Given the different types of issues potentially affecting point estimates from observational and RCT data reviewed above, and the frequency with which results from RCTs have failed to confirm hypotheses suggested by observational evidence, the authors urge those preparing evidence-based reviews and/or meta-analyses to clearly separate observational and RCT results when presenting summaries of the relations between dietary exposures and disease outcomes. Although presentation of a pooled point estimate may be justified in some cases in which the point estimates from collections of observational and RCT results show sufficient concordance, the rationale for such pooling should, in the authors’ view, always be explicitly stated.

Unintended Consequences of Dietary Guidance

Dietary guidance is issued with the intent of improving health and reducing disease risk. Recommendations need to reflect the state of the science and be effectively communicated to the audience in which the recommendations are directed (66). This is no small task. However, an important and often underappreciated aspect of developing dietary guidance is the interpretation and implementation of dietary recommendations by the public. In other words, in practice, how is dietary behavior altered in response to dietary recommendations, if at all, and what is the effect of this change on the intended goals—improving public health and reducing disease risk?

It is possible that dietary guidance has produced unintended consequences on dietary intake and, subsequently, disease risk. In 1980, the first Dietary Guidelines for Americans were released with 7 specific recommendations “to stay healthy,” including “avoid too much fat, saturated fat, and cholesterol” (67). This was followed by the more explicit message in 1985 of consuming ≤30% of energy from fat, with an emphasis on selecting more fruit, vegetables, and grain products (68). The food industry responded to increased consumer demand for low-fat foods by developing thousands of new products lower in fat. In an effort to maintain palatability of such foods, fat-replacement strategies often required increasing the sweetness, and fat was replaced by a combination of sugars and processed starches. In the years that followed, dietary fat as a percentage of calories decreased slightly from ~36% in the late 1970s to ~33% in 2000 (69). However, this change is attributed primarily to an increase in total energy intake, of which the majority was an increase in carbohydrate of ~65 g/d from 1971 to 2000. Although one cannot unequivocally conclude a causal relation, the temporal association between increased processed carbohydrate intake and the increase in the prevalence of overweight and obesity in the United States is striking. At the very least, dietary recommendations to reduce fat intake did not decrease caloric intake as was intended and, in turn, favorably influence the prevalence of obesity in the United States.

Clinical Trials to Assess the Implementation of Dietary Recommendations

Although the response of the food industry to changes in dietary recommendations has the potential to alter food intake patterns, the authors nevertheless believe that much more testing of the implementation of dietary recommendations should be completed to assess how consumers given dietary advice, consistent with proposed recommendations, will change their eating behaviors. This could be accompanied by assessments of key indicators of health status, including body weight, serum lipid and blood pressure levels, etc. Such testing would ideally be carried out in healthy individuals as well as in those with common health conditions such as obesity, hypercholesterolemia, metabolic syndrome, prediabetes, hypertension, etc. Furthermore, such testing would help to identify unintended and unanticipated consequences such as increasing total energy intake while simultaneously reducing the percentage of energy from dietary fat. In the authors’ view, such testing, along with the completion of more clinical trials to evaluate the influence of dietary interventions on actual disease incidence, would strengthen the process of generating dietary recommendations that more fully reflect EBM principles.

Conclusions

Observational investigations, particularly prospective cohort studies, provide critically important information for identifying diet-disease relations. However, observational studies are inherently limited by lack of randomization of exposure; therefore, it is difficult to rule out bias and confounding as possible alternative explanations for diet-disease associations. Because observational evidence for a diet-disease association is subject to a number of limitations including imprecise exposure measurement, collinearity of dietary exposures, displacement/substitution effects, and healthy or unhealthy consumer bias, it is not surprising that a number of associations with relatively consistent support from prospective cohort study results failed to be confirmed in RCTs conducted to test dietary interventions based on such data. The authors do not wish to minimize the importance of observational data, but we believe that a widespread underappreciation of the limitations of such data exists, which may be used as the primary evidence for the development of dietary recommendations.

In our view, the strongest recommendations should be reserved for areas in which observational and RCT results align. We also believe that dietary recommendations and proposed recommendations would be strengthened considerably by the completion of more RCTs to evaluate the impact of dietary advice on diet quality and markers for disease risk in healthy individuals, as well in as those with common health conditions (e.g., obesity, metabolic syndrome, and hypertension) that confer increased risk of chronic diseases. Both controlled feeding studies to directly assess the physiologic impact of dietary interventions on disease risk markers and trials to evaluate how consumers actually implement dietary advice are needed for the evaluation of dietary recommendations.

Finally, although it is impractical or prohibitively expensive to test many proposed diet-disease relations, the authors believe that a larger number of clinical trials need to be undertaken to test dietary interventions and evaluate the full range of risks and benefits for reducing incidence of adverse disease outcomes, including cardiovascular events, new onset diabetes, and certain types of cancer. The recently published results from the PREDIMED dietary intervention trial illustrate the feasibility of undertaking such studies for selected disease outcomes (63).