Combining a Food Frequency Questionnaire With 24-Hour Recalls to Increase the Precision of Estimation of Usual Dietary Intakes—Evidence From the Validation Studies Pooling Project

Abstract

Improving estimates of individuals’ dietary intakes is key to obtaining more reliable evidence for diet-health relationships from nutritional cohort studies. One approach to improvement is combining information from different self-report instruments. Previous work evaluated the gains obtained from combining information from a food frequency questionnaire (FFQ) and multiple 24-hour recalls (24HRs), based on assuming that 24HRs provide unbiased measures of individual intakes. Here we evaluate the same approach of combining instruments but base it on the better assumption that recovery biomarkers provide unbiased measures of individual intakes. Our analysis uses data from the 5 large validation studies included in the Validation Studies Pooling Project: the Observing Protein and Energy Nutrition Study (1999–2000), the Automated Multiple-Pass Method validation study (2002–2004), the Energetics Study (2006–2009), the Nutrition Biomarker Study (2004–2005), and the Nutrition and Physical Activity Assessment Study (2007–2009). The data included intakes of energy, protein, potassium, and sodium. Under a time-varying usual-intake model analysis, the combination of an FFQ with 4 24HRs improved correlations with true intake for predicted protein density, potassium density, and sodium density (range, 0.39–0.61) in comparison with use of a single FFQ (range, 0.34–0.50). Absolute increases in correlation ranged from 0.02 to 0.26, depending on nutrient and sex, with an average increase of 0.14. Based on unbiased recovery biomarker evaluation for these nutrients, we confirm that combining an FFQ with multiple 24HRs modestly improves the accuracy of estimates of individual intakes.

Measurement error in self-reported dietary intakes limits the reliability of results from nutritional cohort studies (1). Approaches to improving the accuracy of dietary intake estimates include reporting intake using new technologies (2, 3), combining self-reports with biomarkers (4), and combining different self-report instruments (5). We focus on the last.

Carroll et al. (5) considered combining information from multiple 24-hour recalls (24HRs) and a food frequency questionnaire (FFQ), using data from the Eating at America’s Table Study (6). Their evaluation assumed that 24HRs unbiasedly measure an individual’s intake, and they showed that their results were probably robust to departures from this assumption.

Here, we also evaluate combining 24HRs with an FFQ, but under the better-founded assumption that recovery biomarkers unbiasedly measure individual intake (7). We use data from 5 large validation studies with recovery biomarkers as reference instruments, comprising the Validation Studies Pooling Project (8).

Also motivating this work was previous work using the time-varying intake model (9), which showed that correlations between intakes reported using 24HRs and true usual intake were lower than previously estimated, because of the close proximity in time of biomarker and 24HR assessments in some studies (10). This raised doubts regarding the benefit accruing from adding 24HR assessments to an FFQ. Using recovery biomarker and self-report data, we estimated correlations of intakes estimated from self-report instruments and their combinations with true longer-term usual intakes for energy, protein, potassium, and sodium and their densities. We report that combining instruments led to increased correlation of estimated intakes with true intakes, and below we discuss the implications of these findings.

Determining the optimal combination of 24HR and FFQ data requires knowing the error models of the two instruments, which in turn requires recovery biomarker measurements. Here, we focus on cohort studies having FFQ and 24HR data but no biomarker data. The FFQ and 24HR data would then be combined, as the best option available, under the (usually erroneous) assumption that 24HRs are unbiased. We use recovery biomarkers to provide an unbiased assessment of this method of combining instruments.

METHODS

The Validation Studies Pooling Project

Investigators in 5 large (>200 participants) validation studies using recovery biomarkers agreed to pool their data, aiming, through common analysis, to clarify the nature and magnitude of reporting errors in FFQs and 24HRs (8, 11). The 5 studies in the Validation Studies Pooling Project included diverse populations within the United States. The Observing Protein and Energy Nutrition (OPEN) Study (12) and the Automated Multiple-Pass Method (AMPM) validation study (13) included volunteers aged 40–69 years (OPEN) or 30–69 years (AMPM) residing in Maryland. The Energetics Study included younger white and African-American adults residing in California (14). The Nutrition Biomarker Study (NBS) (15) and the Nutrition and Physical Activity Assessment Study (NPAAS) (16) included postmenopausal women, mostly aged ≥60 years and residing throughout the United States, in the Women’s Health Initiative Dietary Modification Trial and the Women’s Health Initiative Observational Study, respectively. Further details are provided elsewhere (12–16).

At least 1 FFQ was administered to each participant. This analysis includes only the first administration. The FFQs queried about dietary intake over the past year (OPEN, Energetics, AMPM) or the past 3 months (NBS, NPAAS). One of 3 FFQs was used: the Harvard FFQ (17) (AMPM), the Women’s Health Initiative FFQ (18, 19) (NBS, NPAAS), or the Diet History Questionnaire (6) (OPEN, Energetics).

Each study included 2 or more 24HR assessments, administered to all participants in 4 studies and to a 20% subset in the NBS. An interviewer-administered multiple-pass method (4 studies) or a Web-based self-administered 24HR (Energetics) was used. Further details are provided elsewhere (12–16).

Each study measured doubly labeled water for energy intake (20) and collected 24-hour urine samples for measuring nitrogen (21), potassium (22), and sodium (23) intakes. Details on the laboratories and methods used are provided in Web Appendix 1 (available at https://academic.oup.com/aje).

Urinary nitrogen level (in grams) was divided by 0.81 to convert the data to dietary nitrogen values (21) and then multiplied by 6.25 to convert the data to dietary protein values. Urinary potassium level was divided by 0.8 to convert the data to dietary potassium values (24), and urinary sodium level was divided by 0.86 to convert the data to dietary sodium values (25).

The timing of measurements varied across studies (Web Figure 1). The FFQ was administered at entry in 4 studies but 1–14 months after entry in the AMPM study. In all studies, doubly labeled water was measured at 1–14 days, and 24-hour urine samples were collected during the same period. The 24HRs were administered on the following days: OPEN, days 1 and 61; Energetics, days 1, 3, 5, 8, 11, 14, 30, and 60; AMPM, days 1, 5–6, and 10–11; NBS, around 180 days (twice in a 20% subsample); NPAAS, days 14–104 (3 administrations).

These measurements were supplemented by substudies, of varying size, carried out to examine the reliability of self-reports and biomarkers. The duration of time between main and substudy administrations ranged from 2 weeks in OPEN to approximately 6 months in Energetics, NBS, and NPAAS and 10–23 months in AMPM. In OPEN, only the doubly labeled water measurement was repeated; in other studies, all biomarker measurements and self-reports were repeated. For example, in NBS and NPAAS, the entire study protocol was repeated in a 20% subsample. Our analysis included repeat biomarker and 24HR assessments.

Statistical methods

We report on 7 dietary components: energy intake and intakes of protein, potassium, and sodium and their densities (ratios to energy intake).

We predicted targeted true usual intake (defined here as the 12-month average) using different self-report instruments. A measure of the goodness of a prediction is its correlation with truth. The correlation measures how well the prediction orders individuals according to their true intake and relates to loss of statistical power (26). Low values (e.g., <0.4) are undesirable, although there is no sharp cutoff.

The time-varying intake model accounts for serial correlation in individuals’ intakes and proximity of self-report assessments to biomarker assessments (9). We describe the model briefly below. For technical details, see Web Appendix 2 and Freedman et al. (9).

Each sex and dietary component was modeled separately. Data on dietary variables were logarithmically transformed. The model comprised 4 meta-analysis submodels with study-specific parameters. The first 3 submodels specified linear regression relationships between the biomarker, 24HR, and FFQ, respectively, and true intake, the explanatory variable. The fourth submodel specified how true intake varied over time. The time axis was divided into 90-day subperiods.

Biomarker submodel

In the biomarker submodel, biomarkers were assumed to measure true intake on the previous day (or for energy during the 10- to 14-day assessment) without bias (intercept = 0, slope = 1) but with independent random error. The error variance was estimated through repeat assessments performed within the same subperiod and was assumed to be equal across studies, because of insufficient replications to provide study-specific estimates.

24HR submodel

In the 24HR submodel, 24HR-reported intake was assumed to measure true intake on the same day with systematic intake-related bias. Person-specific bias and within-person random-error terms were included, as in Kipnis et al. (27). Within-person random-error terms were assumed to be mutually independent and independent of all other terms in the model.

FFQ submodel

In the FFQ submodel, FFQ-reported intake was assumed to measure true average intake over the past year with systematic intake-related bias. With no FFQ replications, person-specific bias and within-person random-error terms could not be separately estimated and were combined as a total error term. This total error was assumed to be correlated with the 24HR person-specific bias.

Time-varying true intake submodel

In the time-varying true intake submodel, we assumed that 1) an individual’s true intake varied over time, but the group average and variance, on a single day and in each subperiod, remained constant within-study; 2) the ratio of the single-day intake variance to the 90-day average intake variance (the intake-variance ratio) was common across studies; and 3) the correlation structure between usual intakes in different subperiods was common across studies and was autoregressive (AR) of order 1 (AR(1)) or compound symmetry, decided according to the best model fit, using Akaike’s Information Criterion.

Model parameters were estimated by means of maximum likelihood, assuming log biomarker, 24HR, and FFQ values to be normally distributed, using the CALIS procedure in SAS (SAS Institute, Inc., Cary, North Carolina) (28).

Instruments were combined through a prediction (or calibration) equation for dietary intake, using 24HRs as the reference method. Via estimated model parameters, we estimated correlations of these predictions with truth, separately for each study, sex, and dietary component, for 24HRs, the FFQ, or a combination. We present results for a single FFQ and 1, 2, 4, or 8 24HRs with or without an FFQ. The estimates depend on the timing of self-reports. The timing assumed is shown in Web Table 1, and the calculation methods are given in Web Appendix 2. Study-specific estimates are presented in the Web material (specified below), and across-study summaries (weighted averages) are presented in the main paper.

RESULTS

Details on the models fitted and parameter estimates are provided in Web Tables 2–12.

Table 1 shows, for each dietary component, the estimated correlations with truth, averaged across studies, for a single FFQ, different numbers of 24HR administrations, and their combination. Correlations increased with the number of 24HR administrations, but with diminishing benefits. Similarly, adding an FFQ to one or more 24HRs improved correlations, but benefits diminished with increasing number of 24HRs. Benefits of combining 24HRs with an FFQ were seen for both absolute intakes and densities, but they appeared smaller for women than for men. Estimated correlations remained low (<0.40) for energy intake among men and women and for sodium intake and sodium density among women. For other dietary components, correlations above 0.5 were attained for some combinations, but for protein density in women the maximum was 0.48. Study-specific correlations are presented in Web Tables 13–20.

Table 1.

Correlation Coefficients for the Correlation of Intakes of Selected Dietary Components (Predicted^a From Various Combinations of Self-Report Instruments^b) With Truth, Combined Over 5 Validation Studies^c

Dietary Component, Sex, and Use of FFQ (Y/N)d	No. of 24HRs Administered
	0	1	2	4	8
Energy intake
Male
Y	0.15	0.28	0.32	0.36	0.38
N		0.28	0.33	0.36	0.39
Female
Y	0.22	0.24	0.25	0.26	0.28
N		0.18	0.22	0.24	0.27
Protein intake
Male
Y	0.30	0.42	0.48	0.53	0.57
N		0.37	0.45	0.52	0.56
Female
Y	0.35	0.47	0.52	0.57	0.62
N		0.40	0.49	0.56	0.61
Potassium intake
Male
Y	0.48	0.54	0.58	0.60	0.63
N		0.44	0.52	0.57	0.61
Female
Y	0.39	0.53	0.58	0.63	0.66
N		0.47	0.56	0.62	0.66
Sodium intake
Male
Y	0.21	0.36	0.43	0.48	0.53
N		0.34	0.42	0.48	0.53
Female
Y	0.19	0.22	0.24	0.27	0.30
N		0.17	0.22	0.26	0.29
Protein density
Male
Y	0.40	0.47	0.51	0.56	0.60
N		0.35	0.44	0.52	0.59
Female
Y	0.45	0.46	0.47	0.47	0.48
N		0.27	0.34	0.39	0.44
Potassium density
Male
Y	0.48	0.53	0.57	0.61	0.64
N		0.44	0.53	0.60	0.65
Female
Y	0.50	0.54	0.56	0.57	0.58
N		0.39	0.46	0.52	0.56
Sodium density
Male
Y	0.36	0.46	0.52	0.57	0.63
N		0.38	0.47	0.56	0.62
Female
Y	0.34	0.36	0.37	0.39	0.41
N		0.20	0.26	0.34	0.40

Abbreviations: FFQ, food frequency questionnaire; 24HR, 24-hour recall; N, no; Y, yes.

^a The predictions were formed under the assumption that 24HRs are unbiased measures of intake.

^b An FFQ, 24HRs (1, 2, 4, or 8), or a combination of these.

^c The Observing Protein and Energy Nutrition Study (1999–2000), the Energetics Study (2006–2009), the Automated Multiple-Pass Method validation study (2002–2004), the Nutrition Biomarker Study (2004–2005), and the Nutrition and Physical Activity Assessment Study (2007–2009).

^d Whether or not an FFQ report was used in predicting usual intake.

We considered the “least” combination (the combination with the smallest number of self-report administrations) yielding a correlation at least 90% of that achieved by using 1 FFQ plus 8 24HRs (Table 2). One FFQ plus 3 or 4 24HRs provided the majority of the benefit derived from combining instruments. See Web Figures 2–15 for graphs of the relative correlations achieved by using different combinations of instruments.

Table 2.

“Least” Combination^a of an FFQ and 24HRs Achieving a Correlation of at Least 90% of the Correlation for 1 FFQ + 8 24HRs in Assessment of Selected Dietary Intakes, by Sex, in 5 Validation Studies^b

Dietary Component	No. of FFQs + No. of 24HRs
	Men	Women
Energy intake	0 + 3	1 + 2
Protein intake	1 + 3	1 + 3
Potassium intake	1 + 2	1 + 3
Sodium intake	0 + 4	1 + 4
Protein density	1 + 4	1 + 0
Potassium density	1 + 3	1 + 1
Sodium density	1 + 4	1 + 2

Abbreviations: FFQ, food frequency questionnaire; 24HR, 24-hour recall.

^a The combination with the smallest number of self-report administrations yielding a correlation at least 90% of that achieved by using 1 FFQ plus 8 24HRs.

^b The Observing Protein and Energy Nutrition Study (1999–2000), the Energetics Study (2006–2009), the Automated Multiple-Pass Method validation study (2002–2004), the Nutrition Biomarker Study (2004–2005), and the Nutrition and Physical Activity Assessment Study (2007–2009).

Using multiple 24HRs with an FFQ, the predicted dietary intake used in the analysis also provides estimated risk parameters that have less attenuation due to measurement error. As an example, using 1 FFQ plus 4 administrations of a 24HR to predict dietary intake, the attenuation coefficients are low for energy, are considerably higher for absolute intakes of protein and potassium than the 0.3–0.4 values typically seen when no adjustment for measurement error is made, and are quite close to 1.0 (no attenuation) for nutrient densities (Web Table 21).

DISCUSSION

Adding repeat administrations of a 24HR to an FFQ increased the accuracy of prediction of dietary intake, but with diminishing returns as the number of 24HR administrations increased. One FFQ plus 4 24HR administrations gave at least 90% of the maximum correlation. Using this combination, correlations between predicted and true intakes were 0.39–0.61 for nutrient densities compared with 0.35–0.51 for a single FFQ. Absolute increases in correlation ranged from 0.02 to 0.26, depending on nutrient and sex, with an average increase of 0.14. These results strengthen those of Carroll et al. (5), being based on unbiased recovery biomarker evaluation.

Improvements in correlation from adding a single FFQ to 4 administrations of a 24HR were mostly marginal. However, the dietary components we studied are not episodically consumed. Carroll et al. (5) also reported only small increases in correlations for such components.

The correlations obtained by combining 24HRs with an FFQ leave room for considerable improvement. Clearly, continued research into ways of assessing dietary intake is needed, so as to increase correlations to more than 0.7.

Neuhouser et al. (15) showed that for energy and absolute protein intake, correlations can be considerably increased by including in the prediction equation body mass index (weight (kg)/height (m)²) and other personal characteristics that are sources of systematic reporting error. More extensive data analysis confirms this and extends the result to absolute sodium intake and potassium density, with modest gains for potassium intake, protein density, and sodium density (8, 11). Such prediction equations hold promise for further gains in the precision of intake measurement.

However, such prediction is effective only when it is derived from biomarker references. Here, we have focused on studies collecting data from an FFQ and 24HRs, where prediction equations are based on 24HRs as reference instruments. Such equations do not capture the importance of personal characteristics such as body mass index (29). Consequently, prediction equations based on 24HRs as the reference method do not greatly improve correlations with truth (Web Tables 22 and 23). Biomarker-based prediction equations are preferable, but they require cohort studies to include validation substudies with such biomarkers and are limited to the few nutrients with suitable biomarkers.

A central assumption behind our modeling is that recovery biomarkers are unbiased for individual intake and that their errors are random. We discuss this assumption in Web Appendix 2.

The results presented here are for the limited set of dietary components that have recovery biomarkers. Carroll et al. (5) have shown that combining 24HRs and an FFQ can lead to improvements across a wider range of dietary components. Together, this suggests that while the gains we have demonstrated will not apply to all dietary components, they will apply to many nutrients and foods.

Abbreviations

AMPM

Automated Multiple-Pass Method

FFQ

food frequency questionnaire

24HR

24-hour recall

NBS

Nutrition Biomarker Study

NPAAS

Nutrition and Physical Activity Assessment Study

OPEN

Observing Protein and Energy Nutrition