Reply to E Archer and SN Blair^,

Dear Editor:

We thank Drs. Archer and Blair for commenting on our article (1), which was motivated in part by commentary by them (2) and others (3, 4) that dismissed an entire field of research based almost completely on the concern that a single 24-h dietary recall (24HR) provides inaccurate estimates of energy intake that are not “physiologically plausible.” In our article, we outlined 9 points that can be considered in judging the utility of dietary assessment data, in assessing diet-disease relations, and in drawing inferences from research results. In their response, Archer and Blair allege that there are “intractable systematic biases” in the NHANES data. However, they have not presented evidence to establish the nature of these alleged systematic biases. As we noted under the seventh point in our article, knowing the specific nature of biases provides essential information regarding their effect and offers opportunities for improving methods of risk estimation.

Far from being silent on the matter of drawing inferences based on these kinds of data (and not just from a single 24HR, as in the NHANES), we quoted directly from Archer et al. (2) and then responded to their and others’ criticisms of self-report dietary data in a systematic manner. Under our first point, we readily acknowledged errors in dietary self-report and provided a variety of solutions that we and others have devised and applied. Whole sections of our article were devoted to acquainting readers with understanding the nature of errors and describing methods for adjustment that, in turn, allow for predicting “hard” biological endpoints (i.e., “constructs”).

We also questioned the specific cutoffs that Archer and Blair used to judge implausibility and pointed out the statistical properties of repeat, as opposed to single, measures of daily dietary intake. When taken into account, repeat measures can provide estimates of intraperson variability that can be used to inform analyses using these 24HR-derived data. It is well known to methodologists in this field that a single 24HR is not adequate to characterize an individual’s usual diet (5). This is due to the relatively large day-to-day variation in dietary intake of most people. Beaton and colleagues (6–8) demonstrated that between 42% and 52% of the variation in energy intake was due to within-subject (i.e., day-to-day) variability, findings subsequently reproduced in our work (9–11). The information from additional days of intake provides an estimate of intraperson variability that can be used to assess the distribution of usual intake for a population (12). Furthermore, when intake estimates are averaged over the 2 (or more) days of intake, intraperson variability is reduced as more extreme values are “pulled” in toward the mean. Oddly, when examining the impact of a second day of data, Archer and Blair chose to analyze it as a single day rather than combining it with the first 24HR, a standard practice with both dietary (9, 10, 13–15) and physical activity (16, 17) data. As would be expected by anyone familiar with using such data, the observed results for the second day alone are similar to the first day. In our article, we cited the study by Moshfegh et al. (18), which reported on using three 24HRs (coupled with an improved interview process), and found lower levels of underreporting. In the article cited by Archer and Blair, Freedman et al. (19) reported that averaging intakes across 3 d provides improved estimates of intake over estimates based on a single 24HR. Additional days of information will virtually always improve estimation (9, 10, 13–15). This point is well accepted in many contexts, not just in dietary assessment.

Archer and Blair reject our criticism that they incorrectly applied the Goldberg cutoff for identifying under-reporters. We are not criticizing their computation; after all, it is simple arithmetic. The salient points that we wish to make are these. First, any choice of cutoff is arbitrary in the absence of data on individuals’ metabolic needs. This is stressed by Goldberg and Black and colleagues in their seminal work (20, 21), which Archer and Blair cite as the basis for their decision. Second, “new values … for each element of the Goldberg equation” were suggested by Black (22) in an article published 9 y later and 13 y before Archer and Blair’s article. Black also discussed the need to consider “within-subject variation in energy intake” and “other sources of variation [that] are increased in the light of new data” and that “the effect of these changes is to widen the confidence limits and reduce the sensitivity of the cut-off.” Failing to account for intraperson variability reduces sensitivity for identifying under-reporters at the individual level. Third, applying results from an algorithm based on fitting mean values in order to judge individual intake estimates is methodologically flawed.

The contention that all data derived from such dietary assessment methods are uninformative, because energy intakes appear to be underestimated, is fundamentally unsound. We countered this claim in our article, but it is worth reiterating here. There is a voluminous literature on diet and health based on self-report data that is remarkable for its consistency across studies, one of the criteria for judging causality as spelled out by Hill (23). Of course, if one understands how risk estimates are computed from dietary self-report data in epidemiologic research, it makes sense that simple “calibration errors” described by Archer and Blair will have minimal effect on risk estimation. To illustrate with an admittedly superficial example, if we just subtract a constant 50% of energy intake from each individual’s estimated intake, every value would be “implausible,” but it would have no biasing effect on risk estimates obtained by comparing quantiles of exposure, as is done in a typical epidemiologic study.

Archer and Blair’s dismissal of statistical and other techniques to adjust for biases in self-report is inconsistent with the vast majority of literature on the topic. Many of the advances in interpreting exposure assessment, such as understanding within-person systematic and random measurement error, design and analysis approaches to adjust or account for errors, and use of biomarkers of exposure, owe their development in part to the known limitations of self-report dietary assessment methods. This includes an understanding that errors are neither perfectly correlated across food items within individuals’ diets nor uniform within populations.

In their 2013 article, Archer et al. (2) alleged that energy intake values from a single 24HR are too low; therefore, foods that were eaten went unreported (i.e., omissions). Archer and Blair now allege that there is a problem with “false memory” (i.e., intrusions). It is known that dietary self-report data reflect both omissions and intrusions and these tend to vary according to respondent characteristics. As we stated in our article, any measure represents a combination of truth (signal) and error (noise). Our task is to identify the noise and control or account for it to the greatest extent possible. We do this “before the fact” through appropriate study design [and not just reacting to the reflexive demand to conduct randomized controlled trials as the only way to study relations between diet and health (3)] and data collection protocols, such as ensuring the best procedures for interviewing. We do it “after the fact” by measuring and controlling for identified sources of bias (e.g., social desirability). Over the years, it has been shown that errors do follow observable patterns and therefore can be controlled analytically. In our own work we find that the correlation between unadjusted energy intake estimates from 24HRs with energy expenditure from doubly labeled water (24) is much higher than what Archer and Blair cite. This also is true for data from the OPEN (Observing Protein and Energy Nutrition) study (19).

Archer and Blair appear to suggest that measurement of absolute intake of nutrients is synonymous with “dietary patterns.” Just as self-report biases reflect complex human behaviors, the relation between total caloric intake and intakes of specific nutrients is complicated. Although there is a tendency to consume greater amounts of micronutrients as total intake increases (resulting in a positive correlation), there is a countervailing tendency for “health-conscious” eaters to increase nutrient density with decreasing energy density of the diet (resulting in a negative correlation) (25, 26). To account for some of the correlation with nutrient intake, adjusting for energy intake has now become the standard approach in epidemiologic studies. Doing so generally dampens the effects of under- or overreporting of energy intake and improves risk estimates substantially (27–29). In the OPEN study analyses, Freedman et al. (19) demonstrated improved associations for food-frequency–based estimates of protein intake with urinary nitrogen excretion after adjustment for energy intake.

In their response to our article, Archer and Blair condemn virtually the entire field of nutritional epidemiology because of its reliance on self-report dietary data. In doing so, they have ignored a large body of research that runs counter to their beliefs. With regard to the assertion of Archer and Blair that we are acting in “defense of the status quo,” we stand guilty as charged if they mean that we assert, with empirical evidence, that there is value in dietary data. Rather than fixating on an observation that is well known to generations of researchers (i.e., that a single 24HR provides inaccurate estimates of individuals’ usual energy intake) to condemn an entire field of research, we and many others have made meaningful improvements to the only methods feasible for use in large-scale epidemiologic and clinical studies and for population-level surveillance. This includes a long track record of examining the data, acknowledging problems, improving existing dietary assessment methods, inventing entirely new approaches to increase the value of information obtained from such data, and developing methods to improve the analysis of dietary data and interpretation of results from those analyses. Continuing such efforts is needed to improve the assessment of diet in order to deepen the understanding of the role of diet in human health.