Unscientific Beliefs about Scientific Topics in Nutrition^,,

Abstract

Humans interact with food daily. Such repeated exposure creates a widespread, superficial familiarity with nutrition. Personal familiarity with nutrition from individual and cultural perspectives may give rise to beliefs about food not grounded in scientific evidence. In this summary of the session entitled “Unscientific Beliefs about Scientific Topics in Nutrition,” we discuss accumulated work illustrating and quantifying potentially misleading practices in the conduct and, more so, reporting of nutrition science along with proposed approaches to amelioration. We begin by defining “unscientific beliefs” and from where such beliefs may come, followed by discussing how large bodies of nutritional epidemiologic observations not only create highly improbable patterns of association but implausible magnitudes of implied effect. Poor reporting practices, biases, and methodologic issues that have distorted scientific understandings of nutrition are presented, followed by potential influences of conflicts of interest that extend beyond financial considerations. We conclude with recommendations for improving the conduct, reporting, and communication of nutrition-related research to ground discussions in evidence rather than solely on beliefs.

Introduction

Nutrition research merits the same rigor used in understanding other domains in which science is used, but daily interactions with food and cultural practices surrounding diet seem to lead to widespread nutritional beliefs based on conjecture, anecdote, and intuition more than sound science. Many individuals have beliefs about foods and nutrition that are not necessarily grounded in empirical evidence, and such beliefs, when held by scientists, likely influence the lens through which they report their findings. In the session entitled “Unscientific Beliefs about Scientific Topics in Nutrition” at the ASN Scientific Sessions and Annual Meeting at Experimental Biology 2014 we discussed the following: 1) the factors that may promote such beliefs; 2) biases in nutrition science literature; 3) how sources of potential conflicts of interest (financial and otherwise) may influence reporting; and 4) strategies to enhance the quality of research, research reporting, and interpretation.

What Is an “Unscientific Belief”?

Adopting simple dictionary definitions, science is “the process of understanding the world through experimentation and observation,” whereas beliefs are “feelings that something is true.” Thus, the former represents an ideal of discovering truth that exists separate from the knower, whereas beliefs are internally held understandings filtered through one’s world view. By “unscientific belief,” we indicate that something is held as generalizable fact without substantial scientific supporting evidence, but we fully recognize that “substantial evidence” itself is at least partially subjective. Previously, some of us described 3 classifications of beliefs about scientific topics: 1) myths: “beliefs held to be true despite substantial refuting evidence” 2) presumptions: “beliefs held to be true for which convincing evidence does not confirm or disprove their truth” and 3) facts: “propositions backed by sufficient evidence to consider them empirically proved for practical purposes” (1). These definitions represent a spectrum of interpretations of scientific evidence, which implies that our collective certainty in a proposition is necessarily dependent on the strength of belief in the evidence. Presumptions should be considered understudied, not false, whereas myths should be considered generally false, acknowledging that there may be specific settings or individuals for which they could be true.

This dynamic becomes important on the individual level because most humans experience and make decisions about food daily, with numerous personal, cultural, and religious traditions formed around dietary behaviors. This is exemplified by the increased use of phrases used to demonize particular foods, such as “toxic” or “junk,” or the advent of biblical, historical, or geographical diets that often instantiate beliefs about diets independent of scientific data. As humans, scientists are subject to biases that may affect research reporting and thereby conflate science and beliefs.

Associations in Epidemiology: Too Good to be True?

According to the Scopus database, >25 million papers appeared in major scientific journals from 1996 to 2011. Each paper can include or discuss anywhere from a few up to many millions of results. In this vast compendium of information, biomedical research has the lion’s share, and observational associations are a major component of this literature, outnumbering experimental studies by ∼20:1. However, the validity and credibility of much of this observational association literature is questionable.

One of us recently published a systematic cookbook review on the theme of “diet causes cancer” (2). Opening a popular cookbook, 50 food ingredients were randomly selected to ask how many were associated with increased or decreased cancer risk in scientific literature. Association studies were found for 40 of the 50; studies likely would have been identified for the other 10 ingredients if their biochemical constituents were searched for (e.g., vanillin instead of vanilla). Statistically significant associations abounded. The distribution of Z-scores (reflecting P values) showed a big gap around nonsignificant results. Some of this gap was accounted for in respective meta-analyses of these associations, but much remained. Most effect sizes (point estimates) of the published associations were inconsistent with common sense: RRs of ≥2 per serving were common. If taken literally, this literature means that, if we increase or decrease (as appropriate) intake of any of several nutrients by 2 servings/d, cancer will almost disappear worldwide; manipulating the uptake of a single nutrient suffices.

We demonstrated repeatedly excess significance bias in observational studies. This includes primary results that become statistically significant because of “vibration of effects” when they should have been null and results that are null or negative that are suppressed. Occasionally, having too many published studies with significant results on the same question may represent allegiance to a nutritional zeitgeist, not proof of replication.

Reporting Practices That May Perpetuate Beliefs beyond the Scientific Evidence

Some of these correlations may reflect genuine, causal associations, but the vast majority of them are plausibly due to confounding or biases leading to excess statistical significance. Significant results are attractive for many reasons: genuine scientific interest, financial bias, and allegiance to hypotheses and expectations, to name a few. For any individual study, identifying such biases is often impossible; however, at the aggregate level of an entire field or a subfield, patterns can emerge that call the body of literature into question. The media are often blamed for distorting nutrition research, but in some circumstances, distortions in lay media were tracked to the original articles and press releases on which the stories are based. In other cases, research methodology itself can reinforce unscientific beliefs. In 1 example, researchers created a survey to evaluate participant knowledge about behaviors detrimental or beneficial for obesity; several items they identified as true were identified as myths or presumptions (1). In other cases, research methods commit nominal fallacies, such as the Healthy Eating Index that is more accurately a U.S. Federal Dietary Guidance Conformance Index. Reifying these abstractions by repeatedly stating them is related to social–psychologic phenomena, such as confirmation bias, the mere exposure effect, and cognitive dissonance (3), and thus perpetuates unscientific beliefs.

Financial and Nonfinancial Sources of Bias

Nutrition scientists reporting research have a responsibility to be truthful so that information can be used properly by science, society, and appropriate policies to influence health. Unfortunately, many scientists are faced with perceived and real sources of conflicts of interest, which may either negatively influence science or serve as fodder for critics to wantonly dismiss well-conducted science. Financial conflicts are the most discussed, with financial ties to industries serving as the source of many expressions of concern regarding potential distortions of research reporting. However, scientists may also have financial conflicts unrelated to industries: 1) questioning government-supported nutritional norms may influence funding; 2) popular diet books written by scientists may influence their propensity to defend their personal brand through research publications; and 3) the pressure to publish in “high-impact” journals for promotion, tenure, or bonuses may influence decisions on whether and where to publish.

There is also the potential for nonfinancial sources of bias to influence nutrition studies in detrimental ways, including personal biases, political views, promotion opportunities, and allegiance to the “norm,” although it is often impossible to determine whether these biases occur in any specific instance and whether they occur unconsciously or deliberately. “White-hat bias” is 1 form of nonfinancial bias, defined as bias leading to distortion of research-based information in the service of what may be perceived as righteous ends (4). Some examples of white-hat bias include the following: 1) selectively citing only the portion of results that favor a particular viewpoint (also called “unbalanced citation”); 2) inappropriate inclusion or exclusion of data in reviews; 3) miscommunication within research conclusions, press releases, or media about research results; 4) publication bias (in this case, only publishing results that are perceived to match a preconceived beneficial effect); and 5) only making conclusions about results consistent with the hypothesis (3). Each of these distortions was described in other ways, but white-hat bias seems predicated on researchers’ beliefs that such distortions will improve human health. Instead, these distortions harm the health of science by impairing scientific integrity and damaging public trust.

Practices Needed to Improve the Conduct and Reporting of Nutrition Science

Several considerations may help buttress the scientific integrity of our field. Foremost may be emphasizing to scientists-in-training that science is a discipline whose sine qua non is the unwavering pursuit of truth through objective research conduct and communication. Also, replication initiatives should be welcomed as a means of determining generalizability of results, not viewed as witch hunts. The use of meta-methods can help determine when issues exist and how the field is improving in totality. Several additional specific steps that individual scientists, institutions, and journals can take to improve nutrition science include the following.

Registration.

Study registration helps ensure that the initial purpose of a study is faithfully reported in the final research output and must be taken into consideration during peer review to be of value. Registration “bait and switch” happens when primary endpoints become secondary endpoints and vice versa, whereas subsequent papers focus on tertiary analyses without acknowledging the original study purpose. Thorough a priori registration of scientific studies must include the following: 1) data to be collected; 2) statements of primary and secondary endpoints; and 3) prespecified data analysis plans. Because of evidence that statistical designs are sometimes chosen to elicit significant effects through data transformation, outlier removal, quantile fishing, and other sources of P value ”fiddling” (5), data analysis plans are necessary. Selection of endpoints or analysis plans post hoc should be clearly stated so that results can be interpreted with appropriate caution.

Multiple-comparison transparency.

It takes only 59 uncorrected comparisons to have a 95% chance of at least 1 significant P value. [Assuming that all tests are null and independent, Pr (at least 1 significant test) = 1 − Pr (all nonsignificant tests) = 1 − (0.95)x, when x = 59, Pr (at least 1 significant test) > 0.95.] Therefore, multiple-comparison correction is important, particularly with large datasets in which myriad tests can be conducted. If multiple comparisons are not corrected, readers should be made aware of how many tests were conducted, permitting readers to evaluate the plausibility that significant effects were found by chance.

Adherence to guidelines.

Guidelines for study types including systematic reviews, randomized controlled trials, and observational work were aggregated by the EQUATOR network (6). Instead of being viewed as bureaucratic obstacles to publishing, the guidelines should be used to help better communicate research, aid other scientists in understanding methods used, and increase the chances of replication.

Equitable standards for all research.

Even with the faults of most human research, animal and in vitro studies are often deficient in the most elementary conditions necessary for publication of human trials, including true randomization, allocation concealment, blinding, and intent-to-treat analyses.

Moving Forward: Celebrating the Best of Nutrition Science

Great study designs, exciting results, and new understandings of the intricacies of nutrition are executed, discovered, and communicated daily. Let us collectively celebrate the most well-conducted nutrition science and encourage those who have done due diligence in upholding the rigor on which this field is built, regardless of whether the results match our personal beliefs or favorite hypotheses. With so many values surrounding food, including hedonic aspects, religious considerations, and cultural traditions, expecting everyone to base all of their food decisions on scientific evidence is naive at best. However, it is imperative that we establish and communicate what is and is not known in nutrition science in the most accurate and “unvarnished” manner possible if we hope to encourage scientific beliefs about nutrition.