Re: “Application of a repeat-measure biomarker measurement error model to 2 validation studies: Examination of the effect of within-person variation in biomarker measurements”

The paper by Preis et al. (1) contains potentially misleading statements concerning the impact of within-person biomarker variation on the Observing Protein and Energy Nutrition (OPEN) Study results.

The major theme of the paper is estimation of deattenuation factors and correlations between intakes reported on a food frequency questionnaire or 24-hour diet recall and true usual intake. A second theme is estimating correlations between person-specific systematic errors in the food frequency questionnaire and 24-hour diet recall.

Two models, labeled “(1)” in the paper and another, unlabeled, that we refer to as “model 2,” are considered. Model 1 assumes that 24-hour diet recalls provide unbiased measures of usual intake, whereas the biomarkers are biased. Model 2 assumes the reverse. We see no reason to consider model 1 for the cases of doubly labeled water and urinary nitrogen. Previous feeding studies with urinary nitrogen (2–4) and indirect calorimetry studies with doubly labeled water (3, 5–7) have found no appreciable bias in these recovery biomarkers (8). Although model 1 has been previously used (8, 9), it was only in studies with concentration biomarkers known to be biased.

Claiming that doubly labeled water within-person variation is underestimated in the OPEN Study, Preis et al. (1) use an alternative estimate from the Automated Multiple-Pass Method (AMPM) Validation Study to reanalyze the OPEN Study’s deattenuation factors in their Table 5. However, as shown in the Appendix (1), the deattenuation factor does not depend on biomarker within-person variation under model 2. As a result, the authors’ estimated deattenuation factors for the OPEN Study food frequency questionnaire under model 2 (0.07 for energy, 0.16 for protein, and 0.33 for protein density) are similar to those reported by Kipnis et al. (10) (0.080 and 0.039 for energy, 0.156 and 0.137 for protein, and 0.404 and 0.316 for protein density (men and women, respectively)). Small differences are probably due to minor analytical differences from Kipnis et al. (10) (use of the second vs. the first food frequency questionnaire and the combining of analyses for men and women).

We agree that the level of biomarker within-person variation does affect correlations between self-report and true usual intake and also that the estimated within-person variation could be sensitive to time between repeats. However, we see little evidence that this affected the OPEN Study results. The correlations for the OPEN Study reported in Table 4 (using the AMPM Validation Study within-person variation) were 0.25 for energy, 0.30 for protein, and 0.36 for protein density. Kipnis et al. (10) reported 0.199 and 0.098 for energy (men and women), 0.323 and 0.298 for protein, and 0.431 and 0.356 for protein density. The energy estimates of Kipnis et al. are slightly lower than the authors’ estimate of 0.25. However, epidemiologic analyses rarely include energy alone. Therefore, protein density is more relevant, and the larger AMPM Validation Study doubly labeled water within-person variance appears to have negligible impact on the protein density correlation. Thus, the results do not support the conclusion that the 2-week period between doubly labeled water repeats in the OPEN Study led to “underestimation of the FFQ’s [food frequency questionnaire’s] validity” (1, p. 684).

Finally, Preis et al. (1) suggest in their Figure 1 that the short period between doubly labeled water repeats exaggerated the correlation between food frequency questionnaire and 24-hour diet recall systematic errors. Recalculation using the AMPM Validation Study biomarker within-person variation gave correlations of 0.31 for energy, 0.25 for protein, and 0.59 for protein density. Kipnis et al. (10) reported 0.45 and 0.28 for energy (men and women), 0.18 and 0.24 for protein, and 0.40 and 0.94 for protein density. Averaging over men and women gives values of 0.365, 0.21, and 0.67—not materially different from those reported by Preis et al.

Although the timing of repeated biomarker measurements in a validation study deserves careful consideration, we think Preis et al. greatly exaggerated its potential impact with regard to previously reported estimates from the OPEN Study.