Self-report–based estimates of energy intake offer an inadequate basis for scientific conclusions

Dear Sir:

It has been 23 y since one of us (DAS) examined data from 9 studies that compared self-reported energy intakes (EIs) with measurements of EI made by use of the doubly labeled water (DLW) method. At that time, we detected substantial biases and inaccuracies in self-reported EIs such that we concluded, “Because the greatest bias was observed in obese subjects, current methods for self-reported energy intake are not recommended for use in obesity research” (1). Despite this, yet another new publication has used self-report data and arrived at the conclusion that “after decades of increases, mean energy intake has decreased significantly since 2003–2004” (2). Although the CDC has led a historic effort to reverse the obesity epidemic, that does not justify their investigators using 40 y of self-reported EI data from the NHANES unless some reason exists to conclude that the accuracy of dietary reporting has substantially improved since the publication of those earlier caveats. Unfortunately, this is not the case. Rather, other reports would suggest that bias in reporting of energy intake may even have increased (3). Hence, the conclusions of Ford and Dietz (2) cannot be supported by the EI data they present.

Are the ei data from nhanes valid?

The average values for EI in the NHANES data set range from a low of 1972 kcal/d to a high of 2267 kcal/d. These values are not physiologically plausible. With the advent of the DLW method, it became possible to compare EI with objectively measured total energy expenditure and hence energy requirements for weight maintenance (4). The Observing Protein and Energy Nutrition study in 484 middle-aged, healthy adult subjects used the National Cancer Institute’s 5-pass 24-h recall system, a protocol almost identical to the NHANES method for collecting dietary data that has been in use for more than a decade, and found that self-reported intakes averaged 2170 kcal/d. This figure is very similar to the 2220 kcal/d reported by Ford and Dietz for that same 1999–2000 period. Biomarker total energy expenditure, however, averaged 2532 kcal/d, indicating a 14% underreporting of EI by self-report. This finding is not an outlier. Systematic underreporting of EI has been found in almost all validations of EI against objectively measured energy expenditure, thus repeatedly showing that self-reported EI is invalid (5, 6). Indeed, we can apply thermodynamic models (7, 8) to express the self-reported EI reduction of 98 kcal/d between 2001–2005 and 2007–2010 in terms of weight loss. For example, 2 separate body weight calculators (http://bwsimulator.niddk.nih.gov/ and http://www.pbrc.edu/research-and-faculty/calculators/sswcp/) consistently predicted an improbable population-wide weight loss of ∼3.5 kg resulting from this self-reported reduction in EI.

Might the reporting error have increased in recent years?

Not only are self-reported EI data subject to too much systematic error (bias) to be used as a research tool to track energy balance, but we think it is plausible that the magnitude of the bias may have increased in recent years. Furthermore, it is likely that the reduction in self-reported EI does not reflect a reduction in actual EI. The degree of underreporting is positively related to BMI, at least partially under volitional control, and is likely motivated by social desirability (9). Furthermore, as has been shown, interventions that emphasize the importance of some behaviors can lead to increases in reporting error because the participants modify their reports in the desired direction independently of actual behavior change (10). In other words, we “teach” subjects the relevant socially acceptable responses (11). As such, ubiquitous media and public health messages about the importance of combating obesity (12), in which the primary focus has been on eating less, may explain the increasing downward biases in self-reported EI. It is also likely that increased underreporting of foods high in sugar and fat has played a role (3). The increased downward bias in self-reported EI may parallel the increased downward bias in self-reported weight exhibited in recent years (13).

Conclusions

In conclusion, the data analyzed by Ford and Dietz do not justify the conclusion that EI has decreased among US adults in recent years. We recognize that there is great interest in studying food intake among free-living individuals in large samples. We also recognize that in such large studies, self-report may be the only measurement tool that is practical and may well provide data in regard to dietary patterns. Although Ford and Dietz acknowledged the limitations of self-reported food intake, the common argument that it is the best available method does not make it adequate. Erroneous conclusions derived from self-reported EI may adversely affect policy decisions involving obesity.

Lord Kelvin once said, “In physical science a first essential step in the direction of learning any subject is to find principles of numerical reckoning and practicable methods for measuring some quality connected with it. I often say that when you can measure what you are speaking about and express it in numbers you know something about it” (14). More than 20 y ago when DLW data became available, the assumption that self-reports of EI could be used to generate valid conclusions came to an end. Going forward, we should accept that self-reported EI is fatally flawed and we should stop publishing inaccurate and misleading EI data. With the advent of new validated tools for estimating EI, such as thermodynamic models (7, 8), remote sensing devices (15, 16), and remote food photography (17), we should work toward applying, improving, and extending these methods for measuring EI in free-living persons rather than continuing to rely on self-report.