We Agree That Self-Reported Energy Intake Should Not Be Used as a Basis for Conclusions about Energy Intake in Scientific Research^,,,

Dear Editor:

The valuable article by Subar et al. (1) on self-reported food data is commendable for both its erudition and collegial tone. Because their study was prompted, in part, by one of our own (2), we write to correct one factual point, to clarify substantial points of agreement between our groups, and to suggest where there is room for additional dialogue.

With regard to the clarification, Subar et al. state, “Recent reports have asserted that, because of energy underreporting, dietary self-report data suffer from measurement error so great that findings from all dietary surveillance and observational studies are useless for informing public health policy or investigating diet-health relations” and cite 5 references, including ours (2). However, in our article, we limited our conclusions about the value of self-reported food intake data only to the invalidity of self-report estimates of energy intake (SREI) as bases for conclusions about actual intakes.

With regard to points of agreement, we were delighted that Subar et al. recommend that investigators “do not use self-reported energy intake as a measure of true energy intake.” This is equivalent to our article’s central thesis, which stated, “It is time to move from the common view that self-reports of EI [energy intake] …are imperfect, but nevertheless deserving of use, to a view commensurate with the evidence that self-reports of EI …are so poor that they are wholly unacceptable for scientific research on EI….” The conclusions of Subar et al. and our conclusions about the nonvalue of SREI are in agreement. We also agree with Subar et al. that the field should “continue to develop, evaluate, and further expand methods of dietary assessment, including dietary biomarkers and methods using new technologies” (1).

We also endorse the suggestion by Subar et al. that, “Currently, the optimal method for estimating EI distributions at the population level is to administer DLW [doubly labeled water] in at least a subset representative of the population to permit measurement error adjustment” (1). It is plausible that if this approach is combined with multiple imputation methods (3), where the “true values” (or DLW-derived values as proxies) of EI are treated as missing data for the subjects for whom only SREI is available and if those chosen to receive the DLW are chosen at random, then it may be possible to obtain reasonably accurate answers about EI. More study to optimize this and related approaches (4, 5) is warranted.

There are other points on which we do not agree. We do not agree with Subar et al. when they state, “What does it mean if an association with a health outcome for a nutrient or food group is or is not found? Usually, dietary measurement error causes associations to be underestimated, and although a certain amount of residual confounding can occur, this is usually not sufficient to create spurious associations. A strong signal, therefore, is likely to be true, especially when consistent across studies” (1). First, there is clear empirical evidence that residual confounding can indeed create the kind of associations typically observed in nutrition epidemiology studies (6). Second, we know of no evidence to show generally that, “Usually, dietary measurement error causes associations to be underestimated.” Although true under some specific circumstances (e.g., when measurement error is random and associations are linear and estimated via bivariate linear correlations), more generally measurement error can create, diminish, exaggerate, or change the sign of associations depending on its magnitude, its association with other factors, and the nature of the statistical model fit. Third, consistency across studies can occur because a bias induced by measurement error, confounding, or other factors is consistently replicated.

We also disagree with some broad statements about the demonstrated value of self-report data, because those statements assume facts not in evidence and make logical leaps. For example, Subar et al. offer that there is “amassed evidence that shows that self-report dietary intake data can be successfully used to inform dietary guidance and public health policy” and that findings of “associations between dietary patterns and health outcomes indicate the relevance of self-report dietary data for assessing intakes and relating them to important health outcomes.” Such statements assert that the mere act of having used the data to influence policy serves as justification for the continued use of such data. Tradition is not evidence of accuracy or value. In contrast, if success and relevance entail some aspect of accuracy, we are unclear what the empirical evidence for the purported success is. These statements of the utility of self-report data are therefore ipse dixit assertions rather than logically sound arguments whose empirical components are established. We conclude by noting our fundamental point of agreement: that the use of self-report–based estimates of EI as measures of true EI should be discontinued.