Measurement Error Affecting Web- and Paper-Based Dietary Assessment Instruments: Insights From the Multi-Cohort Eating and Activity Study for Understanding Reporting Error

Sharon I Kirkpatrick; Richard P Troiano; Brian Barrett; Christopher Cunningham; Amy F Subar; Yikyung Park; Heather R Bowles; Laurence S Freedman; Victor Kipnis; Eric B Rimm; Walter C Willett; Nancy Potischman; Donna Spielgelman; David J Baer; Dale A Schoeller; Kevin W Dodd

doi:10.1093/aje/kwac026

. 2022 Feb 8;191(6):1125–1139. doi: 10.1093/aje/kwac026

Measurement Error Affecting Web- and Paper-Based Dietary Assessment Instruments: Insights From the Multi-Cohort Eating and Activity Study for Understanding Reporting Error

Sharon I Kirkpatrick ^✉, Richard P Troiano, Brian Barrett, Christopher Cunningham, Amy F Subar, Yikyung Park, Heather R Bowles, Laurence S Freedman, Victor Kipnis, Eric B Rimm, Walter C Willett, Nancy Potischman, Donna Spielgelman, David J Baer, Dale A Schoeller, Kevin W Dodd

PMCID: PMC9393065 PMID: 35136928

Abstract

Few biomarker-based validation studies have examined error in online self-report dietary assessment instruments, and food records (FRs) have been considered less than food frequency questionnaires (FFQs) and 24-hour recalls (24HRs). We investigated measurement error in online and paper-based FFQs, online 24HRs, and paper-based FRs in 3 samples drawn primarily from 3 cohorts, comprising 1,393 women and 1,455 men aged 45–86 years. Data collection occurred from January 2011 to October 2013. Attenuation factors and correlation coefficients between reported and true usual intake for energy, protein, sodium, potassium, and respective densities were estimated using recovery biomarkers. Across studies, average attenuation factors for energy were 0.07, 0.07, and 0.19 for a single FFQ, 24HR, and FR, respectively. Correlation coefficients for energy were 0.24, 0.23, and 0.40, respectively. Excluding energy, the average attenuation factors across nutrients and studies were 0.22 for a single FFQ, 0.22 for a single 24HR, and 0.51 for a single FR. Corresponding correlation coefficients were 0.31, 0.34, and 0.53, respectively. For densities (nutrient expressed relative to energy), the average attenuation factors across studies were 0.37, 0.17, and 0.50, respectively. The findings support prior research suggesting different instruments have unique strengths that should be leveraged in epidemiologic research.

Keywords: 24-hour recall, dietary assessment, food frequency questionnaire, food record, recovery biomarkers, validation

Abbreviations

24HR: 24-hour recall
ASA24: Automated Self-Administered 24-hour Dietary Assessment Tool
BMI: body mass index
DLW: doubly labeled water
FFQ: food frequency questionnaire
FR: food record
IDATA: Interactive Diet and Activity Tracking in AARP study
MEASURE: Multi-Cohort Eating and Activity Study for Understanding Reporting Error
MLVS: Men’s Lifestyle Validation Study
WLVS: Women’s Lifestyle Validation Study

The challenges measurement error poses to examining relationships between long-term dietary intake and health have been extensively described (1–3). Since the 1980s, validation studies leveraging recovery biomarkers (4) have enhanced our understanding of the sources and extent of error in intake estimates of energy and a few nutrients based on self-report (5–12). Prior studies highlight that 24-hour recalls (24HRs) and food records (FRs) are affected by random error more than are food frequency questionnaires (FFQs), whereas the opposite is true for systematic error (5–12).

These studies have informed approaches to mitigate effects of error by adjusting for random error, calibrating a self-report instrument to a more accurate instrument or to biomarker data (9, 13–15), or combining self-report instruments (16, 17). However, online self-administered 24HRs and FFQs are increasingly available (18–20), whereas few biomarker-based validation studies have examined their error properties (21–24). Further, FRs have been studied less than FFQs and 24HRs in such studies.

The Multi-Cohort Eating and Activity Study for Understanding Reporting Error (MEASURE) was conducted to examine the error properties of multiple administrations of online 24HRs, online and paper-based FFQs, and paper-based weighed and unweighed FRs. The analyses reported here investigated error affecting data collected by each instrument, with a focus on energy, protein, potassium, and sodium, and corresponding densities.

METHODS

Validation studies and their populations

MEASURE comprises 3 studies. The Interactive Diet and Activity Tracking in AARP study (IDATA; 1,151 consented participants) was conducted in a sample of AARP (formerly, American Association of Retired Persons) members aged 50–74 years residing in the Pittsburgh, Pennsylvania, area (24). Participants visited a study center 3 times over 1 year and completed monthly tasks at home. The Women’s Lifestyle Validation Study (WLVS; 851 consented participants) included a sample of women aged 45–86 years from across the United States, drawn from the Nurses’ Health Study (n = 121,701, recruited in 1976) and Nurses’ Health Study II (n = 116,671, recruited in 1989 (25, 26)) (27–29). The Men’s Lifestyle Validation Study (MLVS; 671 consented participants) included men aged 46–82 years who were participants in the Health Professionals Follow-up Study (n = 51,529, recruited in 1986 (30)), and Pilgrim Health Care members from the Boston area (28). WLVS and MLVS participants completed tasks at home. For each study, eligibility criteria included having high-speed internet access and being free of major diseases.

The IDATA study was approved by the National Cancer Institute Special Studies Institutional Review Board and is registered on ClinicalTrials.gov (NCT03268577). The WLVS and MLVS were approved by the human subjects committees of the Harvard T. H. Chan School of Public Health and Brigham and Women’s Hospital. WLVS was conducted from January 2011 to March 2012, IDATA from March 2012 to October 2013, and MLVS from August 2012 to October 2013.

Due to variations in instruments across studies, described below, and prior evidence of gender-specific differences in measurement error structure (11, 12), 4 strata reflecting IDATA women, IDATA men, WLVS, and MLVS were analyzed separately.

Measures

A suite of measures, including 24HRs, FFQs, FRs, and recovery biomarkers, was implemented (Table 1). In IDATA, participants were enrolled in 4 waves with identical data collection activities over 12 months but varied timing to reduce the influence of seasonal variation in intake and manage the volume of study center visits. Similarly, WLVS and MLVS participants were randomized into 4 waves that completed the assessments in different orders over approximately 1 year.

Table 1.

Self-Report Dietary Assessment Tools and Biomarkers Administered in the Multi-Cohort Eating and Activity Study for Understanding Reporting Error^a, United States, January 2011 to October 2013

	FFQ		24HR		FR		DLW ^b		24-Hour Urine
Study	Version	No. and Timing	Version	No. and Timing	Version	No. and Timing	Laboratory	No. and Timing	Laboratory	No. and Timing
IDATA	Diet History Questionnaire, 134 food items and 8 supplement items (online)	2 (1 year apart)	ASA24-2011 (online)	6 (2 months apart)	4-day (paper)	2 (6 months apart)	University of Wisconsin’s Isotope Ratio Mass Spectrometry Core	2 (6 months apart)	USDA Agricultural Research Service Food Components and Health Laboratory	2 (6 months apart)
WLVS	Harvard semiquantitative FFQ, 152 food items and 10 supplement items (paper)^c	2 (1 year apart)	ASA24-Beta (2009) (online)	4 (2–3 months apart)	7-day (paper), weighed	2 (4–8 months apart)	Pennington Biomedical Research Center	2 (6–12 months apart)	Tufts University Human Nutrition Research Center on Aging Nutrition Evaluation Laboratory	4 (3 months apart)
MLVS	Harvard semiquantitative FFQ, 152 food items and 10 supplement items (paper)^c	2 (1 year apart)	ASA24-2011 (online)	4 (2–3 months apart)	7-day (paper), weighed	2 (4–8 months apart)	Pennington Biomedical Research Center	2 (6–12 months apart)	Tufts University Human Nutrition Research Center on Aging Nutrition Evaluation Laboratory	4 (3 months apart)

Open in a new tab

Abbreviations: 24HR, 24-hour recall; ASA24, Automated Self-Administered 24-Hour Dietary Assessment Tool; DLW, doubly labeled water; FFQ, food frequency questionnaire; FR, food record; IDATA, Interactive Diet and Activity Tracking in AARP; MLVS, Men’s Lifestyle Validation Study; USDA, US Department of Agriculture; WLVS, Women’s Lifestyle Validation Study.

^a Sample sizes for each study, nutrient, and instrument combination are available in Web Tables 1–4.

^b Repeat administrations of DLW were performed in subsets for each study (see Web Tables 1–4).

^c In WLVS and MLVS, an online FFQ was also administered. Prior analyses suggested no differences in validity between the 2 formats; only the paper-based FFQ administered in WLVS and MLVS is considered here.

Self-report dietary assessment instruments.

The Automated Self-Administered 24-Hour Dietary Assessment Tool (ASA24) (18) was used to collect 24HRs. IDATA participants completed up to 6 recalls; WLVS and MLVS participants completed up to 4. WLVS used the Beta version of ASA24 (released in 2009), IDATA and the MLVS used ASA24-2011 (31). Participants were prompted by e-mail and robotic telephone calls to complete unannounced recalls on randomly assigned days, every 2–3 months depending on the study. Data were cleaned using established procedures for incomplete recalls, extreme values, and known issues (32). Energy and nutrient intake were calculated using the US Department of Agriculture’s Food and Nutrient Database for Dietary Surveys (FNDDS) (33), version 1.0 for ASA24-Beta and 4.1 for ASA24-2011 (31). ASA24-2011 captured supplement intake and was linked to the National Health and Nutrition Examination Survey Dietary Supplement Database, 2007–2008 (31). The Beta version did not assess supplement intake.

Participants completed FFQs at 2 time points. Within IDATA, the online version of the National Cancer Institute’s Diet History Questionnaire II (DHQ II) (20, 34) was administered; WLVS and MLVS administered the paper-based Harvard semiquantitative FFQ (SFFQ) (27, 29, 35). The DHQ II includes 134 food items and 8 dietary supplements and captures frequency and portion sizes of foods consumed in the past 12 months. The SFFQ includes 152 items and 10 dietary supplements and queries the frequency of consumption of a specified portion or serving size of specific foods in the past 12 months. Frequencies reported were multiplied by nutrient contents per portion size based on FNDDS (33), versions 1.0, 2.0, and 3.0 for the DHQ II and the Harvard Food Composition Table (largely based on US Department of Agriculture data) for the SFFQ.

Finally, IDATA participants completed 2 administrations of a 4-day FR on paper, whereas WLVS and MLVS participants completed 2 administrations of a 7-day FR on paper. The FRs captured foods, beverages, and supplements. IDATA participants received instructions and examples, and the 4-day FRs were coded by trained coders using the US Department of Agriculture’s Survey Net and FNDDS (33), version 4.1. Those in WLVS and MLVS received an Escali food scale (Escali Corporation, Burnsville, Minnesota) and ruler, an instructional DVD, and instructions via telephone (29). They were prompted to measure and report gram weights before and after eating, provide recipes for home-prepared foods, and submit labels for store brand products. Completed 7-day FRs were reviewed by dietitians and participants were recontacted if a section or page was missed. Energy and nutrient intake estimates were calculated using the Nutrition Data System for Research, 2011 (36).

Biomarkers.

Doubly labeled water (DLW) was administered as a biomarker of energy expenditure over a 10–14 day period. Assuming individuals are in energy balance, DLW can be used as a biomarker of energy intake (37). To assess intakes of nitrogen (38), potassium, and sodium (39, 40), 24-hour urine samples were collected. Urinary values for nitrogen, sodium, and potassium were calculated by multiplying urinary concentration by total urine volume. Urinary nitrogen in grams was divided by 0.81 to convert to dietary nitrogen, and multiplied by 6.25 to obtain protein intake in grams per day. Urinary values were divided by 0.86 for sodium (41) and 0.80 for potassium (42) to convert them to intake in milligrams per day. Examining the completeness of 24-hour urine collection using para-amino-benzoic-acid results has been shown to be unnecessary (43). It was expected that low values of urinary biomarkers caused by missing voids would be identified in the outlier detection process noted below.

A random subsample of participants who previously completed biomarker assessments in each study completed repeat measures of DLW and 24-hour urine collections (Table 1). The numbers of replicates by instrument and strata are summarized in Web Tables 1–4 (available at https://doi.org/10.1093/aje/kwac026).

Covariates.

Covariates harmonized across studies include age, race (identified as African American or not), and body mass index (BMI). Height and weight were measured in IDATA and self-reported in MLVS and WLVS, and were used to calculate BMI (weight (kg)/height (m)²) (44).

Statistical analysis

Analyses were conducted using SAS, version 9.4 (SAS Institute, Cary, North Carolina) (45).

Absolute intakes of energy, protein, potassium, and sodium, along with corresponding densities, were considered. Densities reflect composition of the diet and tend to be affected by measurement error less than absolute intakes (11, 12). Protein density is the ratio of energy from protein to total energy intake. Potassium and sodium densities are the ratio of nutrient intake to energy intake per day. Because of the small number of replicate DLW measures, biomarker densities involving energy were computed using the first DLW administration. Random error in biomarker densities is more likely to be driven by day-to-day variation in the numerator than by the fortnight-to-fortnight variation in energy reflected by DLW, so this approach was taken to improve the stability of parameter estimates. The sodium-to-potassium ratio was also examined.

Self-reported intake, biomarker, and BMI values were log-transformed because intake and BMI values were right skewed and recovery biomarkers are assumed to be unbiased on the logarithmic scale (11, 12). Quantile-quantile plots confirmed that the log-transformed distributions were approximately normal.

Preliminary analyses.

Changes in group mean-estimated energy intake over time were examined for each self-reported measure and biomarker. For ASA24, weekday/weekend effects were also examined. No consistent pattern was apparent and no adjustments were made.

Individuals with body weight over 150 kg (1 man and 3 women in IDATA, 1 man in MLVS, and 3 women in WLVS) and individuals with >20% weight change between successive reports (8 men and 8 women in IDATA, 10 men in MLVS, and 14 women in WLVS) were excluded, the former for having potentially misreported (in kilograms instead of pounds, for example) and the latter because the energy biomarker is valid only for weight-stable individuals.

Potential outliers were then examined (Web Appendix 1) resulting in removal of up to 3.8% (average 0.8%) of observations and up to 10.7% (average 1.5%) of individuals across nutrients and strata (Web Tables 1–4). Exclusions were most common for the densities, because either numerator or denominator could be an outlier.

Descriptive analyses.

For each stratum, mean (standard error) age, the proportion of participants identifying as African American, and mean (standard error) BMI were calculated. For each nutrient and density, geometric means (95% confidence limits) were calculated by strata and instrument, based on the first administration.

Measurement error modeling.

An adaptation of Kipnis et al.’s measurement error model (5) was used; it expresses self-reported intake as a sum of: 1) a linear relationship with true intake that holds on average across persons, 2) a person-specific deviation from the linear relationship, and 3) random error (Web Appendix 2). The first term models the flattened-slope phenomenon in which individuals with high intake tend to underreport and individuals with low intake tend to overreport. The second term represents an individual’s level of misreporting compared with the population tendency. The third term reflects random error across repeated observations. The model is appropriate because self-reported dietary data do not follow a classical measurement error structure (i.e., random error only) (5, 46). In contrast, the model expresses biomarker measurements as a sum of an individual’s usual intake and random error only.

In practice, when modeling associations between intake and health outcomes, adjustment for confounders is included. Our primary analyses included confounders commonly used in diet-health models, namely, age and BMI (modeled as continuous variables). We thus consider variation not explained by age and BMI (i.e., the conditional variance in true intake) in the calculation of attenuation factors and correlation coefficients (Web Appendix 2). For each nutrient and stratum, the models for the self-report instruments and biomarkers were fitted jointly using the maximum likelihood procedure implemented in SAS PROC NLMIXED (SAS Institute), allowing correlations among the person-specific bias terms for the self-report instruments. The models were fitted using all available administrations of the self-report instruments.

Attenuation factors and Pearson correlation coefficients (hereafter shortened to correlations) between self-reported intake and true (usual) intake were estimated from the model, using biomarkers as unbiased reference instruments, with adjustment for random variation in the biomarker. The attenuation factor is the slope of the regression of true intake on reported intake and the confounders, and describes the multiplicative bias in the estimated regression coefficient in a diet-disease model that uses self-reported rather than true intake. Values of the attenuation factor far from 1 are undesirable, although adjustments can be made to mitigate the resulting bias in situations in which there is a calibration substudy that includes unbiased reference instruments. The squared correlation measures the loss of statistical power to detect associations due to measurement error. Low values of the correlation are undesirable since measurement error adjustment does not generally recover power.

Complete adjustment for the effects of all errors (random and systematic) requires a recovery biomarker or other unbiased reference measure. However, in the absence of a reference measure, a partial correction for within-person random error can be performed when replicate measurements are available. If all participants have the same “large” number of replicates, using the within-person average rather than a single replicate dampens the attenuating and power-reducing effects of random error (3, 47). Alternatively, statistical techniques, such as regression calibration (48), can be applied even if only a few replicates are available and for only a representative subsample. Attenuation factors and correlations were estimated for a single administration of each instrument, and for within-person averages of 2 FFQs and FRs and 4, 6, and 12 24HRs. The model also provided estimates of the attenuation factors that would pertain if repeat self-report administrations were adjusted for random error using regression calibration (instead of taking within-person averages).

We first present simple averages across strata for 1) energy, 2) absolute nutrients, and 3) corresponding densities as a summary of the performance of different instrument types. Grouping energy versus absolute nutrients versus densities was based on prior research indicating that energy is most poorly measured regardless of instrument and that densities are better measured than absolute nutrients (11, 12). P values for 2-sided t-test comparisons of attenuation factors and correlations were computed. Testing was limited to averages across strata for each of energy, absolute nutrients, and densities, and was not adjusted for multiple comparisons. Attenuation factors and correlations are then presented for each nutrient and stratum combination, enabling qualitative comparisons of online versus paper and weighed versus unweighed assessment modalities.

RESULTS

Table 2 provides an overview of participants within each stratum by age, race, and BMI. The geometric means for energy, protein, potassium, sodium, and the respective densities, based on the first administration of each measure, are provided in Web Tables 5–6.

Table 2.

Characteristics of Participants in the Multi-Cohort Eating and Activity Study for Understanding Reporting Error^a, United States, January 2011 to October 2013

	Women		Men
Characteristic	IDATA (n = 543)	WLVS (n = 850)	IDATA (n = 547)	MLVS (n = 908)
Age, years^b	62.4 (0.03)	62.2 (0.03)	64.0 (0.02)	67.3 (0.03)
Body mass index^b^,^c	27.8 (0.02)	26.4 (0.02)	28.4 (0.02)	26.1 (0.01)
African American^d	10.5	7.4	3.8	1.5

Open in a new tab

Abbreviations: IDATA, Interactive Diet and Activity Tracking in AARP; MLVS, Men’s Lifestyle Validation Study; WLVS, Women’s Lifestyle Validation Study.

^a The analytical sample for each instrument and nutrient combination reported in subsequent tables varies due to the exclusion of outliers (see Web Tables 1–4).

^b Values are expressed as mean (standard error).

^c Weight (kg)/height (m)².

^d Values are expressed as %.

On average across strata, the attenuation factor for energy was similar for 1 FFQ and 1 24HR (P = 0.59); in comparison, the attenuation factor for a single multiday food record was substantially larger (P < 0.0001 for both tests) (Table 3). Averaging multiple recalls or records improved the attenuation factor for energy to a greater extent than averaging multiple FFQs. On average across studies, the highest attenuation factor observed for energy was 0.30, estimated for regression calibration–adjusted food records. For correlations, the estimate for a single multiday FR (0.40) was significantly better than for 12 24HRs (0.35) (P = 0.03) and for 2 FFQs (0.26) (P < 0.0001).

Table 3.

Across-Strata Average Attenuation and Correlation Factors^a for Reported Intakes of Energy, Absolute Nutrients, and Associated Densities in the Multi-Cohort Eating and Activity Study for Understanding Reporting Error, United States, January 2011 to October 2013

	Energy				Absolute Nutrients				Densities
	Attenuation		Correlation		Attenuation		Correlation		Attenuation		Correlation
Instrument and No. or Adjustment	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE
FFQ, single	0.07	0.01	0.24	0.04	0.22	0.01	0.31	0.01	0.37	0.01	0.34	0.01
FFQ, 2	0.08	0.01	0.26	0.04	0.26	0.01	0.33	0.02	0.49	0.02	0.39	0.01
FFQ, adjusted^b	0.10	0.01			0.31	0.01			0.80	0.05
ASA24, single	0.07	0.01	0.23	0.03	0.22	0.01	0.34	0.01	0.17	0.01	0.25	0.01
ASA24, 4	0.13	0.01	0.32	0.04	0.43	0.01	0.48	0.01	0.34	0.01	0.35	0.01
ASA24, 6	0.14	0.01	0.33	0.04	0.49	0.02	0.51	0.01	0.39	0.02	0.38	0.02
ASA24, 12	0.16	0.02	0.35	0.04	0.56	0.02	0.55	0.02	0.45	0.02	0.40	0.02
ASA24, adjusted^b	0.18	0.02			0.66	0.02			0.53	0.02
FR, single	0.19	0.01	0.40	0.04	0.51	0.01	0.53	0.01	0.50	0.01	0.47	0.01
FR, 2	0.23	0.01	0.45	0.04	0.63	0.02	0.59	0.01	0.65	0.02	0.54	0.01
FR, adjusted^b	0.30	0.02			0.82	0.02			0.94	0.04

Open in a new tab

Abbreviations: ASA24, Automated Self-Administered 24-Hour Dietary Assessment Tool 24-hour recall; FFQ, food frequency questionnaire; FR, food record; SE, standard error.

^a Estimated using a measurement error model that included age and body mass index. Absolute nutrients were protein, potassium, and sodium, and associated densities are for protein density, potassium density, sodium density, and sodium:potassium. Studies included were Interactive Diet and Activity Tracking in AARP, the Men’s Lifestyle Validation Study, and the Women’s Lifestyle Validation Study.

^b Refers to attenuation factors that would pertain if repeat self-report administrations were adjusted for random error using regression calibration.

Across absolute nutrients, attenuation factors for regression-calibrated instruments were 0.31 for FFQs, 0.66 for 24HRs, and 0.82 for FRs (Table 3). For the densities, a single FR outperformed 12 24HRs (P = 0.003 for attenuation factors and <0.001 for correlations) and 1 FFQ (P < 0.001 for attenuation factors and <0.001 for correlations) (Table 3).

Tables 4–8 provide the attenuation factors and correlations for each nutrient and stratum combination, with indication of the instances in which the self-report instruments were online (IDATA FFQ and all 24HRs) and weighed (WLVS and MLVS 7-day FRs). For energy, the weighed 7-day FR used in WLVS and MLVS appeared to somewhat outperform the unweighed 4-day FR used in IDATA—the highest attenuation factor was 0.40 for the 7-day FR in MLVS (Table 4). For protein, the attenuation factors closest to 1 were observed for the regression calibration–adjusted weighed 7-day FR in WLVS and MLVS followed by the regression calibration–adjusted 4-day FR in IDATA and the regression calibration–adjusted 24HRs (Table 5). Similarly, the highest correlations for protein were for the weighed 7-day FR, with similar correlations for 6 or 12 ASA24 recalls and 2 multiday unweighed 4-day FRs in some instances. The attenuation factors closest to 1 for potassium and sodium were also observed for the regression calibration–adjusted FR, followed by the regression calibration–adjusted 24HRs (Tables 6 and 7), with similar correlations for multiple 24HRs and 2 4-day FRs. Overall, for the absolute nutrients, the regression calibration–adjusted, weighed 7-day FR yielded the attenuation factor closest to 1 in 2 of 3 cases (protein in WLVS and potassium in WLVS), with the attenuation factor closest to 1 for sodium observed for the unweighed 4-day FR among IDATA men (Tables 5–7).

Table 4.

Attenuation and Correlation Factors^a for Reported Intakes of Energy in the Multi-Cohort Eating and Activity Study for Understanding Reporting Error, United States, January 2011 to October 2013

	Women								Men
	IDATA Women				WLVS				IDATA Men				MLVS
	Attenuation		Correlation		Attenuation		Correlation		Attenuation		Correlation		Attenuation		Correlation
Instrument andNo. or Adjustment	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE
FFQ, single	0.05^b	0.02	0.18^b	0.07	0.05	0.01	0.14	0.04	0.07^b	0.02	0.32^b	0.12	0.12	0.02	0.30	0.04
FFQ, 2	0.06^b	0.02	0.20^b	0.07	0.06	0.02	0.15	0.04	0.08^b	0.02	0.34^b	0.13	0.14	0.02	0.33	0.04
FFQ, adjusted^c	0.07^b	0.03			0.07	0.02			0.09^b	0.03			0.17	0.02
ASA24, single	0.07^b	0.02	0.23^b	0.05	0.06^b	0.01	0.17^b	0.04	0.07^b	0.02	0.28^b	0.09	0.08^b	0.01	0.24^b	0.03
ASA24, 4	0.13^b	0.03	0.31^b	0.07	0.11^b	0.02	0.24^b	0.05	0.12^b	0.03	0.39^b	0.13	0.15^b	0.02	0.33^b	0.04
ASA24, 6	0.14^b	0.03	0.33^b	0.08	0.12^b	0.03	0.25^b	0.05	0.14^b	0.03	0.41^b	0.13	0.17^b	0.02	0.34^b	0.05
ASA24, 12	0.16^b	0.03	0.35^b	0.08	0.14^b	0.03	0.27^b	0.05	0.15^b	0.04	0.43^b	0.14	0.19^b	0.03	0.37^b	0.05
ASA24, adjusted^c	0.18^b	0.04			0.16^b	0.03			0.17^b	0.04			0.21^b	0.03
FR, single^d	0.13	0.03	0.31	0.07	0.21	0.02	0.41	0.04	0.12	0.03	0.38	0.13	0.28	0.02	0.51	0.03
FR, 2^d	0.17	0.03	0.36	0.08	0.26	0.02	0.45	0.04	0.16	0.04	0.44	0.14	0.33	0.02	0.55	0.04
FR, adjusted^c^,^d	0.27	0.06			0.32	0.03			0.22	0.05			0.40	0.03

Open in a new tab

Abbreviations: ASA24, Automated Self-Administered 24-Hour Dietary Assessment Tool 24-hour recall; FFQ, food frequency questionnaire; FR, food record; IDATA, Interactive Diet and Activity Tracking in AARP; MLVS, Men’s Lifestyle Validation Study; SE, standard error; WLVS, Women’s Lifestyle Validation Study.

^a Attenuation and correlation factors were estimated using a measurement error model that included age and body mass index.

^b Self-reported intakes were collected using online (vs. paper-based) instruments.

^c Refers to attenuation factors that would pertain if repeat self-report administrations were adjusted for random error using regression calibration.

^d The FR was weighed in WLVS and MLVS and unweighed in IDATA.

Table 8.

Attenuation and Correlation Factors^a for Sodium-Potassium Ratio From Reported Intakes in the Multi-Cohort Eating and Activity Study for Understanding Reporting Error, United States, January 2011 to October 2013

	Women								Men
	IDATA Women				WLVS				IDATA Men				MLVS
	Attenuation		Correlation		Attenuation		Correlation		Attenuation		Correlation		Attenuation		Correlation
Instrument andNo. or Adjustment	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE
FFQ, single	0.43^b	0.06	0.44^b	0.06	0.60	0.05	0.50	0.04	0.51^b	0.06	0.50^b	0.05	0.59	0.05	0.49	0.04
FFQ, 2	0.50^b	0.07	0.47^b	0.06	0.70	0.06	0.54	0.04	0.58^b	0.06	0.53^b	0.05	0.69	0.06	0.53	0.04
FFQ, adjusted^c	0.60^b	0.08			0.84	0.07			0.67^b	0.07			0.82	0.07
ASA24, single	0.27^b	0.03	0.38^b	0.04	0.26^b	0.03	0.35^b	0.03	0.28^b	0.03	0.37^b	0.03	0.39^b	0.03	0.47^b	0.03
ASA24, 4	0.60^b	0.06	0.58^b	0.05	0.60^b	0.06	0.54^b	0.05	0.65^b	0.06	0.56^b	0.05	0.77^b	0.05	0.66^b	0.04
ASA24, 6	0.70^b	0.07	0.62^b	0.06	0.71^b	0.07	0.58^b	0.05	0.76^b	0.07	0.61^b	0.05	0.86^b	0.06	0.70^b	0.04
ASA24, 12	0.84^b	0.09	0.68^b	0.06	0.85^b	0.09	0.64^b	0.05	0.92^b	0.09	0.67^b	0.05	0.98^b	0.07	0.75^b	0.04
ASA24, adjusted^c	1.05^b	0.11			1.07^b	0.13			1.16^b	0.12			1.14^b	0.09
FR, single^d	0.53	0.06	0.52	0.05	0.64	0.04	0.64	0.03	0.68	0.06	0.61	0.04	0.66	0.04	0.67	0.03
FR, 2^d	0.71	0.07	0.61	0.06	0.80	0.04	0.71	0.03	0.89	0.07	0.70	0.05	0.78	0.04	0.74	0.03
FR, adjusted^c^,^d	1.07	0.13			1.06	0.06			1.27	0.12			0.98	0.05

Open in a new tab

^a Attenuation and correlation factors were estimated using a measurement error model that included age and body mass index.

^b Self-reported intakes were collected using online (vs. paper-based) instruments.

^c Refers to attenuation factors that would pertain if repeat self-report administrations were adjusted for random error using regression calibration.

^d The FR was weighed in WLVS and MLVS and unweighed in IDATA.

Table 5.

Attenuation and Correlation Factors^a for Reported Intakes of Protein and Protein Density in the Multi-Cohort Eating and Activity Study for Understanding Reporting Error, United States, January 2011 to October 2013

	Women								Men
	IDATA Women				WLVS				IDATA Men				MLVS
	Attenuation		Correlation		Attenuation		Correlation		Attenuation		Correlation		Attenuation		Correlation
Instrument andNo. or Adjustment	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE
Protein
FFQ, single	0.16^b	0.03	0.29^b	0.05	0.25	0.03	0.32	0.04	0.19^b	0.03	0.34^b	0.05	0.28	0.03	0.34	0.04
FFQ, 2	0.20^b	0.03	0.32^b	0.05	0.29	0.03	0.34	0.04	0.22^b	0.03	0.37^b	0.05	0.33	0.04	0.38	0.04
FFQ, adjusted^c	0.25^b	0.04			0.35	0.04			0.26^b	0.04			0.42	0.05
ASA24, single	0.18^b	0.02	0.30^b	0.03	0.21^b	0.02	0.33^b	0.03	0.21^b	0.02	0.34^b	0.03	0.19^b	0.02	0.30^b	0.03
ASA24, 4	0.38^b	0.04	0.43^b	0.05	0.46^b	0.04	0.49^b	0.04	0.45^b	0.04	0.50^b	0.04	0.42^b	0.04	0.45^b	0.04
ASA24, 6	0.43^b	0.05	0.46^b	0.05	0.54^b	0.05	0.53^b	0.04	0.51^b	0.05	0.54^b	0.04	0.48^b	0.05	0.48^b	0.04
ASA24, 12	0.51^b	0.06	0.50^b	0.05	0.64^b	0.07	0.58^b	0.05	0.60^b	0.06	0.58^b	0.05	0.56^b	0.06	0.52^b	0.05
ASA24, adjusted^c	0.61^b	0.07			0.78^b	0.09			0.73^b	0.07			0.68^b	0.08
FR, single^d	0.41	0.05	0.42	0.05	0.66	0.04	0.62	0.03	0.42	0.04	0.45	0.04	0.70	0.04	0.65	0.03
FR, 2^d	0.53	0.06	0.48	0.05	0.80	0.04	0.68	0.03	0.53	0.05	0.51	0.05	0.82	0.04	0.70	0.03
FR, adjusted^c^,^d	0.75	0.09			1.00	0.06			0.72	0.08			0.98	0.05
Protein density
FFQ, single	0.34^b	0.06	0.32^b	0.05	0.31	0.04	0.28	0.03	0.23^b	0.06	0.23^b	0.06	0.34	0.04	0.32	0.04
FFQ, 2	0.57^b	0.09	0.41^b	0.06	0.46	0.06	0.34	0.04	0.33^b	0.08	0.28^b	0.07	0.48	0.06	0.38	0.04
FFQ, adjusted^c	1.76^b	0.49			0.88	0.13			0.62^b	0.16			0.84	0.11
ASA24, single	0.09^b	0.03	0.14^b	0.04	0.13^b	0.02	0.22^b	0.04	0.11^b	0.03	0.19^b	0.04	0.10^b	0.02	0.17^b	0.03
ASA24, 4	0.17^b	0.05	0.20^b	0.06	0.27^b	0.05	0.32^b	0.05	0.21^b	0.05	0.26^b	0.06	0.20^b	0.04	0.24^b	0.05
ASA24, 6	0.19^b	0.06	0.21^b	0.06	0.30^b	0.05	0.34^b	0.06	0.24^b	0.06	0.28^b	0.06	0.22^b	0.05	0.25^b	0.05
ASA24, 12	0.22^b	0.07	0.23^b	0.07	0.35^b	0.06	0.36^b	0.06	0.27^b	0.06	0.30^b	0.07	0.25^b	0.05	0.27^b	0.05
ASA24, adjusted^c	0.26^b	0.08			0.41^b	0.07			0.31^b	0.07			0.30^b	0.06
FR, single^d	0.39	0.07	0.35	0.06	0.58	0.04	0.52	0.03	0.33	0.06	0.31	0.05	0.57	0.04	0.49	0.03
FR, 2^d	0.54	0.09	0.41	0.06	0.74	0.05	0.59	0.03	0.46	0.08	0.37	0.06	0.70	0.05	0.55	0.04
FR, adjusted^c^,^d	0.86	0.16			1.02	0.07			0.78	0.15			0.90	0.07

Open in a new tab

^a Attenuation and correlation factors were estimated using a measurement error model that included age and body mass index.

^b Self-reported intakes were collected using online (vs. paper-based) instruments.

^c Refers to attenuation factors that would pertain if repeat self-report administrations were adjusted for random error using regression calibration.

^d The FR was weighed in WLVS and MLVS and unweighed in IDATA.

Table 6.

Attenuation and Correlation Factors^a for Reported Intakes of Potassium and Potassium Density in the Multi-Cohort Eating and Activity Study for Understanding Reporting Error, United States, January 2011 to October 2013

	Women								Men
	IDATA Women				WLVS				IDATA Men				MLVS
	Attenuation		Correlation		Attenuation		Correlation		Attenuation		Correlation		Attenuation		Correlation
Instrument andNo. or Adjustment	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE
Potassium
FFQ, single	0.26^b	0.04	0.33^b	0.05	0.32	0.03	0.41	0.04	0.25^b	0.04	0.36^b	0.05	0.43	0.04	0.50	0.04
FFQ, 2	0.30^b	0.05	0.36^b	0.05	0.36	0.04	0.43	0.04	0.29^b	0.04	0.38^b	0.05	0.48	0.04	0.53	0.04
FFQ, adjusted^c	0.35^b	0.06			0.42	0.04			0.33^b	0.05			0.56	0.05
ASA24, single	0.31^b	0.03	0.40^b	0.04	0.28^b	0.02	0.45^b	0.03	0.30^b	0.03	0.45^b	0.03	0.29^b	0.02	0.43^b	0.03
ASA24, 4	0.54^b	0.05	0.53^b	0.05	0.51^b	0.04	0.60^b	0.04	0.55^b	0.04	0.60^b	0.04	0.51^b	0.04	0.57^b	0.04
ASA24, 6	0.59^b	0.06	0.56^b	0.05	0.56^b	0.04	0.63^b	0.04	0.60^b	0.05	0.63^b	0.04	0.56^b	0.04	0.59^b	0.04
ASA24, 12	0.66^b	0.06	0.59^b	0.05	0.63^b	0.05	0.67^b	0.04	0.67^b	0.05	0.66^b	0.04	0.62^b	0.05	0.62^b	0.04
ASA24, adjusted^c	0.73^b	0.07			0.70^b	0.06			0.75^b	0.06			0.69^b	0.06
FR, single^d	0.52	0.05	0.50	0.04	0.68	0.03	0.74	0.03	0.52	0.04	0.57	0.04	0.68	0.04	0.67	0.03
FR, 2^d	0.64	0.06	0.56	0.05	0.77	0.04	0.79	0.03	0.63	0.05	0.63	0.04	0.77	0.04	0.71	0.03
FR, adjusted^c^,^d	0.85	0.09			0.88	0.04			0.80	0.07			0.87	0.05
Potassium density
FFQ, single	0.40^b	0.06	0.37^b	0.06	0.34	0.04	0.31	0.04	0.33^b	0.06	0.31^b	0.06	0.42	0.05	0.37	0.04
FFQ, 2	0.52^b	0.08	0.42^b	0.06	0.47	0.06	0.36	0.04	0.42^b	0.08	0.35^b	0.07	0.54	0.06	0.42	0.04
FFQ, adjusted^c	0.76^b	0.13			0.74	0.10			0.59^b	0.12			0.74	0.09
ASA24, single	0.16^b	0.04	0.21^b	0.05	0.17^b	0.03	0.28^b	0.04	0.13^b	0.03	0.20^b	0.05	0.20^b	0.02	0.31^b	0.04
ASA24, 4	0.26^b	0.06	0.27^b	0.06	0.30^b	0.04	0.38^b	0.05	0.21^b	0.05	0.26^b	0.06	0.33^b	0.04	0.40^b	0.05
ASA24, 6	0.28^b	0.07	0.28^b	0.07	0.33^b	0.05	0.40^b	0.05	0.23^b	0.06	0.27^b	0.07	0.35^b	0.04	0.42^b	0.05
ASA24, 12	0.30^b	0.07	0.29^b	0.07	0.37^b	0.05	0.42^b	0.06	0.25^b	0.06	0.28^b	0.07	0.38^b	0.05	0.44^b	0.05
ASA24, adjusted^c	0.33^b	0.08			0.41^b	0.06			0.27^b	0.07			0.42^b	0.05
FR, single^d	0.54	0.07	0.44	0.05	0.58	0.04	0.59	0.03	0.40	0.06	0.36	0.05	0.60	0.04	0.55	0.03
FR, 2^d	0.70	0.09	0.50	0.06	0.67	0.04	0.64	0.04	0.53	0.08	0.41	0.06	0.70	0.05	0.59	0.04
FR, adjusted^c^,^d	1.00	0.14			0.81	0.05			0.81	0.14			0.83	0.06

Open in a new tab

^a Attenuation and correlation factors were estimated using a measurement error model that included age and body mass index.

^b Self-reported intakes were collected using online (vs. paper-based) instruments.

^c Refers to attenuation factors that would pertain if repeat self-report administrations were adjusted for random error using regression calibration.

^d The FR was weighed in WLVS and MLVS and unweighed in IDATA.

Table 7.

Attenuation and Correlation Factors^a for Reported Intakes of Sodium and Sodium Density in the Multi-Cohort Eating and Activity Study for Understanding Reporting Error, United States, January 2011 to October 2013

	Women								Men
	IDATA Women				WLVS				IDATA Men				MLVS
	Attenuation		Correlation		Attenuation		Correlation		Attenuation		Correlation		Attenuation		Correlation
Instrument andNo. or Adjustment	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE
Sodium
FFQ, single	0.12^b	0.04	0.19^b	0.06	0.10	0.03	0.18	0.05	0.13^b	0.04	0.20^b	0.06	0.18	0.04	0.25	0.05
FFQ, 2	0.14^b	0.05	0.21^b	0.07	0.12	0.04	0.19	0.06	0.15^b	0.05	0.21^b	0.07	0.21	0.04	0.27	0.05
FFQ, adjusted^c	0.17^b	0.06			0.14	0.04			0.18^b	0.05			0.25	0.05
ASA24, single	0.16^b	0.03	0.26^b	0.04	0.14^b	0.02	0.28^b	0.04	0.18^b	0.03	0.26^b	0.04	0.18^b	0.02	0.29^b	0.04
ASA24, 4	0.35^b	0.06	0.38^b	0.06	0.30^b	0.05	0.41^b	0.06	0.36^b	0.06	0.37^b	0.06	0.39^b	0.05	0.43^b	0.05
ASA24, 6	0.40^b	0.07	0.40^b	0.07	0.35^b	0.06	0.44^b	0.07	0.41^b	0.06	0.39^b	0.06	0.45^b	0.06	0.46^b	0.06
ASA24, 12	0.46^b	0.08	0.44^b	0.07	0.41^b	0.07	0.48^b	0.08	0.48^b	0.07	0.42^b	0.06	0.53^b	0.07	0.50^b	0.06
ASA24, adjusted^c	0.56^b	0.10			0.50^b	0.09			0.56^b	0.09			0.64^b	0.09
FR, single^d	0.30	0.06	0.32	0.06	0.41	0.04	0.54	0.05	0.40	0.06	0.39	0.05	0.45	0.04	0.53	0.04
FR, 2^d	0.41	0.08	0.38	0.07	0.54	0.05	0.62	0.05	0.55	0.08	0.46	0.06	0.54	0.05	0.58	0.05
FR, adjusted^c^,^d	0.64	0.13			0.78	0.07			0.88	0.14			0.68	0.06
Sodium Density
FFQ, single	0.34^b	0.08	0.29^b	0.07	0.24	0.04	0.29	0.05	0.25^b	0.08	0.20^b	0.06	0.25	0.05	0.24	0.05
FFQ, 2	0.54^b	0.13	0.37^b	0.08	0.33	0.06	0.33	0.06	0.36^b	0.12	0.24^b	0.08	0.34	0.07	0.28	0.05
FFQ, adjusted^c	1.35^b	0.38			0.49	0.09			0.65^b	0.22			0.52	0.10
ASA24, single	0.11^b	0.04	0.16^b	0.06	0.11^b	0.03	0.21^b	0.05	0.07^b	0.04	0.10^b	0.05	0.15^b	0.03	0.23^b	0.04
ASA24, 4	0.22^b	0.08	0.24^b	0.08	0.22^b	0.05	0.30^b	0.07	0.14^b	0.07	0.14^b	0.07	0.30^b	0.05	0.33^b	0.06
ASA24, 6	0.25^b	0.09	0.25^b	0.09	0.24^b	0.06	0.31^b	0.07	0.15^b	0.07	0.15^b	0.07	0.34^b	0.06	0.35^b	0.06
ASA24, 12	0.29^b	0.10	0.27^b	0.09	0.28^b	0.07	0.34^b	0.08	0.17^b	0.08	0.16^b	0.08	0.40^b	0.07	0.37^b	0.06
ASA24, adjusted^c	0.35^b	0.12			0.33^b	0.08			0.20^b	0.10			0.47^b	0.09
FR, single^d	0.33	0.08	0.30	0.07	0.44	0.04	0.50	0.05	0.21	0.08	0.16	0.06	0.55	0.05	0.54	0.04
FR, 2^d	0.48	0.11	0.36	0.08	0.62	0.06	0.59	0.05	0.34	0.13	0.21	0.08	0.70	0.06	0.61	0.05
FR, adjusted^c^,^d	0.84	0.21			1.01	0.10			0.85	0.35			0.97	0.09

Open in a new tab

^a Attenuation and correlation factors were estimated using a measurement error model that included age and body mass index.

^b Self-reported intakes were collected using online (vs. paper-based) instruments.

^c Refers to attenuation factors that would pertain if repeat self-report administrations were adjusted for random error using regression calibration.

^d The FR was weighed in WLVS and MLVS and unweighed in IDATA.

For each of protein density, potassium density, and sodium density, the attenuation factors closest to 1 and highest correlations were observed for FRs, but similar attenuation factors and correlations were yielded by FFQs. The attenuation factors for the sodium:potassium ratio were similar for the regression calibration–adjusted 24HRs and FRs (weighed or unweighed), whereas the attenuation factors were somewhat closer to zero for the regression calibration–adjusted FFQ, with the paper-based FFQ in WLVS and MLVS appearing to outperform the online FFQ used in IDATA (Table 8). The attenuation factor was substantially larger than 1 for protein density and sodium density estimated by the regression calibration–adjusted online FFQ among IDATA women and sodium:potassium ratio estimated by regression calibration–adjusted FRs among IDATA men, indicating that adjusting for random error overcorrects.

For reference, selected results from models excluding age and BMI are summarized in Web Tables 7–11. These can be used to compare with earlier studies that did not include covariates in the measurement error model.

DISCUSSION

Enhancing understanding of the effects of measurement error in self-reported intake informs strategies to improve our understanding of the diet and health nexus (3, 49, 50). The Validation Studies Pooling Project leveraged data from 5 studies to provide insights into the error properties of commonly-used self-report instruments (11, 12). The FFQs studied were self-administered using paper and pencil, 4 of the 5 studies used interviewer-administered 24HRs, and FRs were not considered (11, 12). Herein, we applied consistent analytical methods to multiple studies that used a combination of online and paper-and-pencil FFQs, online 24HRs, and 2 versions of FRs. Our findings are in line with prior research in terms of the magnitude and types of error affecting different instruments, as well as varying levels of reporting error across nutrients. In particular, estimates of absolute intake from FFQ, whether administered online or using paper-and-pencil, are affected by systematic measurement error to a greater extent than estimates from 24HRs and FRs. Nonetheless, the FFQs perform moderately well for densities, highlighting the importance of tailoring data collection and analysis methods to the research question.

Estimated attenuation factors and correlations for a single online FFQ were similar to those observed for paper-based FFQs in prior validation studies (11, 12). Across strata, online and paper-based FFQs were used, with the online instruments tending to produce attenuation factors closer to zero and smaller correlations. The differences may be owed to variations in the samples as well as the specific FFQs used. Nonetheless, the patterns in performance of FFQs relative to 24HRs and FRs and for absolute nutrients versus densities were largely consistent regardless of mode. The feasibility offered by online FFQs may outweigh differences in performance related to mode.

All 24HRs considered were administered online using ASA24. Estimated attenuation factors and correlations for a single online 24HR were similar to those observed for interviewer-administered recalls in prior studies (11, 12) Also consistent with prior research using interviewer-administered 24HRs (5–12), the systematic components of measurement error were smaller relative to those for FFQs, while the contributions of within-person random error were larger (as expected for short-term instruments). Given their greater feasibility for large-scale studies, online recalls may be preferable to interviewer-administered recalls. Compared with interviewer-administered recalls, online recall systems also enable, for the same cost, collection of detailed dietary intake data from considerably larger substudies to allow calibration to better mitigate the effects of error in FFQs used as main instruments.

Consistent with prior research that has considered multiday FRs (49, 51, 52), records (sometimes with only 1 administration) outperformed both FFQs and 24HRs and, not surprisingly, the weighed 7-day FR appeared to outperform the unweighed 4-day FR. The 7-day FR involved the collection of recipes and labels and was reviewed with participants by study staff following completion and thus may not be realistic for most epidemiologic research. The FRs considered were paper-based. It is an open question as to whether online FRs, collected using ASA24 or other online platforms, can achieve performance similar to more intensive paper-based FRs, potentially opening up new avenues to improve nutritional epidemiology.

For both 24HRs and FRs, it is important to design studies to enable accounting for within-person random error to avoid losing the gains afforded by the lower systematic error compared with FFQs. For all self-report instruments, the findings also underscore the necessity of accounting for the extent of error in sample size calculations since power lost due to error generally cannot be recovered (3). The correlation factors presented are informative for such calculations since their squared values are inversely proportional to the loss of power.

In some cases, the estimated attenuation factors that would pertain if repeat self-report administrations were adjusted for random error were larger than 1. This occurs when the flattened slope component of error (which exaggerates associations) overcomes the remaining person-specific bias component (which, along with random error, attenuates associations). Even when the flattened slope and other components balance out (i.e., when the net attenuation factor is nearly 1), this does not mean the instrument is error-free, only that the health outcome–usual intake regression slope can be estimated with minimal bias.

As shown in prior analyses (11, 12), estimation of absolute energy intake is poor across self-report instruments. This is the case for energy even when drawing upon multiple administrations of short-term instruments and adjusting for random error, with the possible exception of weighed food records. Our findings support the recommendation to avoid relying on estimates of absolute energy intake based on self-reported data (53) and, in studies in which energy balance is central, to instead use direct indicators such as weight and changes in weight. Other nutrients, especially densities (based on self-reported energy from the same instrument), are less affected by error, suggesting stronger inferences are possible—this now well-documented finding is consistent with approaches to account for self-reported energy within diet-disease models (50) and the shift within nutritional epidemiology toward considering dietary composition (54). Further research studying diet-disease relationships in the presence of multiple error-prone dietary exposures is warranted to support this shift.

The variation in the specific instruments used in the studies comprising MEASURE could mean that inferences relying on averages mask the effects of such heterogeneity. The Beta version of ASA24, used in WLVS, had known issues and did not query supplement intake (potentially affecting estimated potassium intake). Nonetheless, the pattern of results in this stratum does not diverge substantially from the other strata. The underlying databases differed across instruments and strata, with potential implications for the alignment of true and reported intake.

Statistical modeling and related assumptions were used to make inferences about how self-reported intake compared with true long-term intake because it is not possible to measure true individual long-term intake. The use of recovery biomarkers to assess the validity of the self-report measures assumes they are unbiased for true long-term individual intake. We attempted to exclude individuals not in energy balance, but modest imbalances over 2 weeks are difficult to detect, with potential implications for the accuracy of estimated energy intake based on DLW. The assumption of unbiasedness may be more defensible for DLW as well as protein and potassium based on 24-hour urines than for sodium; however, Freedman et al. (12) suggest that it is plausible that urinary sodium excretion complies with the assumptions for recovery biomarkers. Finally, correlations between intake estimates from short-term instruments may be overestimated and those from FFQs underestimated relative to long-term true intake because the biomarker assessments were in closer alignment with the periods assessed by the 24HRs and FRs versus the FFQs.

The different modalities of dietary assessment instruments compared here were not implemented in the same samples; thus, comparisons of online versus paper and weighed versus unweighed modalities may be confounded by heterogeneity in the samples, including predominantly health professionals in WLVS and MLVS. The WLVS and MLVS height and weight data were self-reported, likely resulting in some underestimation of BMI (55). Further, the results may not be generalizable to a population with a substantially different BMI distribution. Our analyses did not consider race/ethnicity because the harmonized race/ethnicity variable was limited and the samples had little diversity.

The findings support prior research suggesting different instruments have unique strengths that should be leveraged in epidemiologic research. In particular, FFQs offer value for understanding densities and establishing probable diet-disease relationships, whereas online systems provide flexibility to researchers to collect 24HRs and FRs in large-scale studies to enable calibration of FFQs and improve quantitative estimates of diet-disease relationships.

Supplementary Material

Web_Material_kwac026

Click here for additional data file.^{(684.2KB, pdf)}

ACKNOWLEDGMENTS

Author affiliations: School of Public Health Sciences, University of Waterloo, Waterloo, Ontario, Canada (Sharon I. Kirkpatrick); Epidemiology and Genomics Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, Bethesda, Maryland, United States (Richard P. Troiano, Amy F. Subar); Information Management Services, Inc., Rockville, Maryland, United States (Brian Barrett, Christopher Cunningham, Laurence S. Freedman); Division of Public Health Sciences, Department of Surgery, Washington University School of Medicine, St. Louis, Missouri, United States (Yikyung Park); Biometry Research Group, Division of Cancer Prevention, National Cancer Institute, Bethesda, Maryland, United States (Heather R. Bowles, Victor Kipnis, Kevin W. Dodd); Biostatistics and Biomathematics Unit, Gertner Institute for Epidemiology and Health Policy Research, Tel Hashomer, Israel (Laurence S. Freedman); Department of Nutrition, Harvard T. H. Chan School of Public Health, Boston, Massachusetts, United States (Eric B. Rimm, Walter C. Willett), Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, United States (Eric B. Rimm, Walter C. Willett); Channing Division of Network Medicine, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts, United States (Eric B. Rimm, Walter C. Willett); Population Studies Program, Office of Dietary Supplements, National Institutes of Health, Bethesda, Maryland, United States (Nancy Potischman); Yale School of Public Health, New Haven, Connecticut, United States (Donna Spiegelman); Department of Statistics and Data Science, Yale School of Public Health, New Haven, Connecticut, United States (Donna Spiegelman); Agricultural Research Service, US Department of Agriculture, Beltsville, Maryland, United States (David J. Baer); Biotechnology Center, University of Wisconsin, Madison, Wisconsin, United States (Dale A. Schoeller); and Nutritional Sciences, University of Wisconsin, Madison, Wisconsin, United States (Dale A. Schoeller).

The Interactive Diet and Activity Tracking in AARP study (IDATA) was funded by the National Cancer Institute (contracts HHSN261201000087I, HHSN26100003, and N02-PC-64406, with contributions from the Office of Dietary Supplements, the National Heart, Lung and Blood Institute, the National Institute of Diabetes and Digestive and Kidney Diseases, the Office of Behavioral and Social Science Research, the Eunice Kennedy Shriver National Institute of Child Health and Human Development, and National Institute on Alcohol Abuse and Alcoholism, and shared instrumentation grant RR020915 to University of Wisconsin for purchase of the isotope ratio mass spectrometer for the urine analysis). The Women’s Lifestyle Validation Study (WLVS) and Men’s Lifestyle Validation Study (MLVS) were supported by National Institutes of Health (grants UM1-CA186107, UM1-CA176726, UM1-CA167552, P01-CA055075-18S1, and U01-CA152904). The present analyses were funded by the National Cancer Institute.

Proposals to access Interactive Diet and Activity Tracking in AARP study data and biospecimens can be submitted at https://cdas.cancer.gov/idata/.

Conflict of interest: none declared.

REFERENCES

1. Beaton G. Approaches to analysis of dietary data: relationship between planned analyses and choice of methodology. Am J Clin Nutr. 1994;59(1):253S–261S. [DOI] [PubMed] [Google Scholar]
2. Beaton GH, Burema J, Ritenbaugh C. Errors in the interpretation of dietary assessments. Am J Clin Nutr. 1997;65(4 Suppl):1100S–1107S. [DOI] [PubMed] [Google Scholar]
3. Freedman LS, Schatzkin A, Midthune D, et al. . Dealing with dietary measurement error in nutritional cohort studies. J Natl Cancer Inst. 2011;103(14):1086–1092. [DOI] [PMC free article] [PubMed] [Google Scholar]
4. Kaaks R, Ferrari P, Ciampi A, et al. . Uses and limitations of statistical accounting for random error correlations, in the validation of dietary questionnaire assessments. Public Health Nutr. 2002;5(6A):969–976. [DOI] [PubMed] [Google Scholar]
5. Kipnis V, Subar AF, Midthune D, et al. . Structure of dietary measurement error: results of the OPEN biomarker study. Am J Epidemiol. 2003;158(1):14–16. [DOI] [PubMed] [Google Scholar]
6. Subar AF, Kipnis V, Troiano RP, et al. . Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN study. Am J Epidemiol. 2003;158(1):1–13. [DOI] [PubMed] [Google Scholar]
7. Schatzkin A, Kipnis V, Carroll RJ, et al. . A comparison of a food frequency questionnaire with a 24-hour recall for use in an epidemiological cohort study: results from the biomarker-based Observing Protein and Energy Nutrition (OPEN) study. Int J Epidemiol. 2003;32(6):1054–1062. [DOI] [PubMed] [Google Scholar]
8. Moshfegh AJ, Rhodes DG, Baer DJ, et al. . The US Department of Agriculture Automated Multiple-Pass Method reduces bias in the collection of energy intakes. Am J Clin Nutr. 2008;88(2):324–332. [DOI] [PubMed] [Google Scholar]
9. Neuhouser ML, Tinker L, Shaw PA, et al. . Use of recovery biomarkers to calibrate nutrient consumption self-reports in the Women’s Health Initiative. Am J Epidemiol. 2008;167(10):1247–1259. [DOI] [PubMed] [Google Scholar]
10. Huang Y, Horn L, Tinker LF, et al. . Measurement error corrected sodium and potassium intake estimation using 24-hour urinary excretion. Hypertension. 2014;63(2):238–244. [DOI] [PMC free article] [PubMed] [Google Scholar]
11. Freedman LS, Commins JM, Moler JE, et al. . Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for energy and protein intake. Am J Epidemiol. 2014;180(2):172–188. [DOI] [PMC free article] [PubMed] [Google Scholar]
12. Freedman LS, Commins JM, Moler JE, et al. . Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for potassium and sodium intake. Am J Epidemiol. 2015;181(7):473–487. [DOI] [PMC free article] [PubMed] [Google Scholar]
13. Tinker LF, Sarto GE, Howard BV, et al. . Biomarker-calibrated dietary energy and protein intake associations with diabetes risk among postmenopausal women from the Women’s Health Initiative. Am J Clin Nutr. 2011;94(6):1600–1606. [DOI] [PMC free article] [PubMed] [Google Scholar]
14. Prentice RL, Shaw PA, Bingham SA, et al. . Biomarker-calibrated energy and protein consumption and increased cancer risk among postmenopausal women. Am J Epidemiol. 2009;169(8):977–989. [DOI] [PMC free article] [PubMed] [Google Scholar]
15. Freedman LS, Midthune D, Carroll RJ, et al. . Using regression calibration equations that combine self-reported intake and biomarker measures to obtain unbiased estimates and more powerful tests of dietary associations. Am J Epidemiol. 2011;174(11):1238–1245. [DOI] [PMC free article] [PubMed] [Google Scholar]
16. Carroll RJ, Midthune D, Subar AF, et al. . Taking advantage of the strengths of 2 different dietary assessment instruments to improve intake estimates for nutritional epidemiology. Am J Epidemiol. 2012;175(4):340–347. [DOI] [PMC free article] [PubMed] [Google Scholar]
17. Freedman LS, Midthune D, Arab L, et al. . Combining a food frequency questionnaire with 24-hour recalls to increase the precision of estimation of usual dietary intakes—evidence from the Validation Studies Pooling Project. Am J Epidemiol. 2018;187(10):2227–2232. [DOI] [PMC free article] [PubMed] [Google Scholar]
18. Subar AF, Kirkpatrick SI, Mittl B, et al. . The Automated Self-Administered 24-hour dietary recall (ASA24): a resource for researchers, clinicians, and educators from the National Cancer Institute. J Acad Nutr Diet. 2012;112(8):1134–1137. [DOI] [PMC free article] [PubMed] [Google Scholar]
19. Carter MC, Albar SA, Morris MA, et al. . Development of a UK online 24-h dietary assessment tool: myfood24. Nutrients. 2015;7(6):4016–4032. [DOI] [PMC free article] [PubMed] [Google Scholar]
20. National Cancer Institute . Diet History Questionnaire II (DHQ II) for U.S. and Canada. https://epi.grants.cancer.gov/dhq2/. Updated July 24, 2020. Accessed January 5, 2021.
21. Wark PA, Hardie LJ, Frost GS, et al. . Validity of an online 24-h recall tool (myfood24) for dietary assessment in population studies: comparison with biomarkers and standard interviews. BMC Med. 2018;16(1):136. [DOI] [PMC free article] [PubMed] [Google Scholar]
22. Lafrenière J, Lamarche B, Laramée C, et al. . Validation of a newly automated web-based 24-hour dietary recall using fully controlled feeding studies. BMC Nutr. 2017;3:34. [DOI] [PMC free article] [PubMed] [Google Scholar]
23. Subar AF, Potischman N, Dodd KW, et al. . Performance and feasibility of recalls completed using the Automated Self-Administered 24-hour Dietary Assessment Tool in relation to other self-report tools and biomarkers in the Interactive Diet and Activity Tracking in AARP (IDATA) Study. J Acad Nutr Diet. 2020;120(11):1805–1820. [DOI] [PMC free article] [PubMed] [Google Scholar]
24. Park Y, Dodd KW, Kipnis V, et al. . Comparison of self-reported dietary intakes from the Automated Self-Administered 24-h recall, 4-d food records, and food-frequency questionnaires against recovery biomarkers. Am J Clin Nutr. 2018;107(1):80–93. [DOI] [PMC free article] [PubMed] [Google Scholar]
25. Colditz GA, Manson JE, Hankinson SE. The Nurses’ Health Study: 20-year contribution to the understanding of health among women. J Womens Health. 1997;6(1):49–62. [DOI] [PubMed] [Google Scholar]
26. Colditz GA, Hankinson SE. The Nurses’ Health Study: lifestyle and health among women. Nat Rev Cancer. 2005;5(5):388–396. [DOI] [PubMed] [Google Scholar]
27. Yuan C, Spiegelman D, Rimm EB, et al. . Validity of a dietary questionnaire assessed by comparison with multiple weighed dietary records or 24-hour recalls. Am J Epidemiol. 2017;185(7):570–584. [DOI] [PMC free article] [PubMed] [Google Scholar]
28. Chomistek AK, Yuan C, Matthews CE, et al. . Physical activity assessment with the ActiGraph GT3X and doubly labeled water. Med Sci Sport Exerc. 2017;49(9):1935–1944. [DOI] [PMC free article] [PubMed] [Google Scholar]
29. Yuan C, Spiegelman D, Rimm EB, et al. . Relative validity of nutrient intakes assessed by questionnaire, 24-hour recalls, and diet records as compared with urinary recovery and plasma concentration biomarkers: findings for women. Am J Epidemiol. 2018;187(5):1051–1063. [DOI] [PMC free article] [PubMed] [Google Scholar]
30. Wilson KM, Kasperzyk JL, Rider JR, et al. . Coffee consumption and prostate cancer risk and progression in the Health Professionals Follow-up Study. J Natl Cancer Inst. 2011;103(11):876–884. [DOI] [PMC free article] [PubMed] [Google Scholar]
31. National Cancer Institute . Comparison of ASA24 versions. https://epi.grants.cancer.gov/asa24/comparison.html. Updated December 15, 2020. Accessed January 5, 2021.
32. National Cancer Institute . ASA24 known issues & workarounds. https://epi.grants.cancer.gov/asa24/resources/issues.html. Updated January 24, 2020. Accessed January 5, 2021.
33. US Department of Agriculture . Food and nutrient database for dietary surveys. https://www.ars.usda.gov/northeast-area/beltsville-md-bhnrc/beltsville-human-nutrition-research-center/food-surveys-research-group/docs/fndds/. Updated July 15, 2020. Accessed January 5, 2021.
34. Subar AF, Thompson FE, Kipnis V, et al. . Comparative validation of the Block, Willett, and National Cancer Institute food frequency questionnaires: the Eating at America’s Table Study. Am J Epidemiol. 2001;154(12):1089–1099. [DOI] [PubMed] [Google Scholar]
35. Willett WC, Sampson L, Stampfer MJ, et al. . Reproducibility and validity of a semiquantitative food frequency questionnaire. Am J Epidemiol. 1985;122(1):51–65. [DOI] [PubMed] [Google Scholar]
36. University of Minnesota Nutrition Coordinating Centre . Nutrition Data System for Research (NDSR) Software. Minneapolis, MN: University of Minnesota Nutrition Coordinating Centre; 2020. [Google Scholar]
37. Schoeller DA, Hnilicka JM. Reliability of the doubly labeled water method for the measurement of total daily energy expenditure in free-living subjects. J Nutr. 1996;126(1):348S–354S. [PubMed] [Google Scholar]
38. Bingham SA. Urine nitrogen as a biomarker for the validation of dietary protein intake. J Nutr. 2003;133(3):921S–924S. [DOI] [PubMed] [Google Scholar]
39. Mickelsen O, Makdani D, Gill JL, et al. . Sodium and potassium intakes and excretions of normal men consuming sodium chloride or a 1:1 mixture of sodium and potassium chlorides. Am J Clin Nutr. 1977;30(12):2033–2040. [DOI] [PubMed] [Google Scholar]
40. Luft FC, Fineberg NS, Sloan RS. Estimating dietary sodium intake in individuals receiving a randomly fluctuating intake. Hypertension. 1982;4(6):805–808. [DOI] [PubMed] [Google Scholar]
41. Holbrook JT, Patterson KY, Bodner JE, et al. . Sodium and potassium intake and balance in adults consuming self-selected diets. Am J Clin Nutr. 1984;40(4):786–793. [DOI] [PubMed] [Google Scholar]
42. Freedman LS, Midthune D, Carroll RJ, et al. . Adjustments to improve the estimation of usual dietary intake distributions in the population. J Nutr. 2004;134(7):1836–1843. [DOI] [PubMed] [Google Scholar]
43. Subar AF, Midthune D, Tasevska N, et al. . Checking for completeness of 24-h urine collection using Para-amino benzoic acid not necessary in the Observing Protein and Energy Nutrition study. Eur J Clin Nutr. 2013;67(8):863–867. [DOI] [PubMed] [Google Scholar]
44. Centers for Disease Control and Prevention . Body mass index (BMI). https://www.cdc.gov/healthyweight/assessing/bmi/index.html. Reviewed September 17, 2020. Accessed January 5, 2021.
45. SAS Institute . Statistical Analysis Software, version 9.4. Cary, NC. https://support.sas.com/software/94/. Accessed February 21, 2022. [Google Scholar]
46. Carroll RJ. Measurement Error in Nonlinear Models: A Modern Perspective. 2nd ed. New York, NY: Chapman & Hall/CRC; 2006. [Google Scholar]
47. Dodd KW, Guenther PM, Freedman LS, et al. . Statistical methods for estimating usual intake of nutrients and foods: a review of the theory. J Am Diet Assoc. 2006;106(10):1640–1650. [DOI] [PubMed] [Google Scholar]
48. Spiegelman D, McDermott A, Rosner B. Regression calibration method for correcting measurement-error bias in nutritional epidemiology. Am J Clin Nutr. 1997;65(4):1179S–1186S. [DOI] [PubMed] [Google Scholar]
49. Prentice RL, Mossavar-Rahmani Y, Huang Y, et al. . Evaluation and comparison of food records, recalls, and frequencies for energy and protein assessment by using recovery biomarkers. Am J Epidemiol. 2011;174(5):591–603. [DOI] [PMC free article] [PubMed] [Google Scholar]
50. Willett WC. Nutritional Epidemiology. 3rd ed. New York, NY: Oxford University Press; 2013. [Google Scholar]
51. Bingham SA, Gill C, Welch A, et al. . Validation of dietary assessment methods in the UK arm of EPIC using weighed records, and 24-hour urinary nitrogen and potassium and serum vitamin c and carotenoids as biomarkers. Int J Epidemiol. 1997;26(suppl 1):S137–S151. [DOI] [PubMed] [Google Scholar]
52. McKeown NM, Day NE, Welch AA, et al. . Use of biological markers to validate self-reported dietary intake in a random sample of the European Prospective Investigation Into Cancer United Kingdom Norfolk cohort. Am J Clin Nutr. 2001;74(2):188–196. [DOI] [PubMed] [Google Scholar]
53. Subar AF, Freedman LS, Tooze JA, et al. . Addressing current criticism regarding the value of self-report dietary data. J Nutr. 2015;145(12):2639–2645. [DOI] [PMC free article] [PubMed] [Google Scholar]
54. Reedy J, Subar A, George S, et al. . Extending methods in dietary patterns research. Nutrients. 2018;10(5):571. [DOI] [PMC free article] [PubMed] [Google Scholar]
55. Gorber SC, Tremblay M, Moher D, et al. . A comparison of direct vs. self-report measures for assessing height, weight and body mass index: a systematic review. Obes Rev. 2007;8(4):307–326. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Web_Material_kwac026

Click here for additional data file.^{(684.2KB, pdf)}

[ref1] 1. Beaton G. Approaches to analysis of dietary data: relationship between planned analyses and choice of methodology. Am J Clin Nutr. 1994;59(1):253S–261S. [DOI] [PubMed] [Google Scholar]

[ref2] 2. Beaton GH, Burema J, Ritenbaugh C. Errors in the interpretation of dietary assessments. Am J Clin Nutr. 1997;65(4 Suppl):1100S–1107S. [DOI] [PubMed] [Google Scholar]

[ref3] 3. Freedman LS, Schatzkin A, Midthune D, et al. . Dealing with dietary measurement error in nutritional cohort studies. J Natl Cancer Inst. 2011;103(14):1086–1092. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref4] 4. Kaaks R, Ferrari P, Ciampi A, et al. . Uses and limitations of statistical accounting for random error correlations, in the validation of dietary questionnaire assessments. Public Health Nutr. 2002;5(6A):969–976. [DOI] [PubMed] [Google Scholar]

[ref5] 5. Kipnis V, Subar AF, Midthune D, et al. . Structure of dietary measurement error: results of the OPEN biomarker study. Am J Epidemiol. 2003;158(1):14–16. [DOI] [PubMed] [Google Scholar]

[ref6] 6. Subar AF, Kipnis V, Troiano RP, et al. . Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN study. Am J Epidemiol. 2003;158(1):1–13. [DOI] [PubMed] [Google Scholar]

[ref7] 7. Schatzkin A, Kipnis V, Carroll RJ, et al. . A comparison of a food frequency questionnaire with a 24-hour recall for use in an epidemiological cohort study: results from the biomarker-based Observing Protein and Energy Nutrition (OPEN) study. Int J Epidemiol. 2003;32(6):1054–1062. [DOI] [PubMed] [Google Scholar]

[ref8] 8. Moshfegh AJ, Rhodes DG, Baer DJ, et al. . The US Department of Agriculture Automated Multiple-Pass Method reduces bias in the collection of energy intakes. Am J Clin Nutr. 2008;88(2):324–332. [DOI] [PubMed] [Google Scholar]

[ref9] 9. Neuhouser ML, Tinker L, Shaw PA, et al. . Use of recovery biomarkers to calibrate nutrient consumption self-reports in the Women’s Health Initiative. Am J Epidemiol. 2008;167(10):1247–1259. [DOI] [PubMed] [Google Scholar]

[ref10] 10. Huang Y, Horn L, Tinker LF, et al. . Measurement error corrected sodium and potassium intake estimation using 24-hour urinary excretion. Hypertension. 2014;63(2):238–244. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref11] 11. Freedman LS, Commins JM, Moler JE, et al. . Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for energy and protein intake. Am J Epidemiol. 2014;180(2):172–188. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref12] 12. Freedman LS, Commins JM, Moler JE, et al. . Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for potassium and sodium intake. Am J Epidemiol. 2015;181(7):473–487. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref13] 13. Tinker LF, Sarto GE, Howard BV, et al. . Biomarker-calibrated dietary energy and protein intake associations with diabetes risk among postmenopausal women from the Women’s Health Initiative. Am J Clin Nutr. 2011;94(6):1600–1606. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref14] 14. Prentice RL, Shaw PA, Bingham SA, et al. . Biomarker-calibrated energy and protein consumption and increased cancer risk among postmenopausal women. Am J Epidemiol. 2009;169(8):977–989. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref15] 15. Freedman LS, Midthune D, Carroll RJ, et al. . Using regression calibration equations that combine self-reported intake and biomarker measures to obtain unbiased estimates and more powerful tests of dietary associations. Am J Epidemiol. 2011;174(11):1238–1245. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref16] 16. Carroll RJ, Midthune D, Subar AF, et al. . Taking advantage of the strengths of 2 different dietary assessment instruments to improve intake estimates for nutritional epidemiology. Am J Epidemiol. 2012;175(4):340–347. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref17] 17. Freedman LS, Midthune D, Arab L, et al. . Combining a food frequency questionnaire with 24-hour recalls to increase the precision of estimation of usual dietary intakes—evidence from the Validation Studies Pooling Project. Am J Epidemiol. 2018;187(10):2227–2232. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref18] 18. Subar AF, Kirkpatrick SI, Mittl B, et al. . The Automated Self-Administered 24-hour dietary recall (ASA24): a resource for researchers, clinicians, and educators from the National Cancer Institute. J Acad Nutr Diet. 2012;112(8):1134–1137. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref19] 19. Carter MC, Albar SA, Morris MA, et al. . Development of a UK online 24-h dietary assessment tool: myfood24. Nutrients. 2015;7(6):4016–4032. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref20] 20. National Cancer Institute . Diet History Questionnaire II (DHQ II) for U.S. and Canada. https://epi.grants.cancer.gov/dhq2/. Updated July 24, 2020. Accessed January 5, 2021.

[ref21] 21. Wark PA, Hardie LJ, Frost GS, et al. . Validity of an online 24-h recall tool (myfood24) for dietary assessment in population studies: comparison with biomarkers and standard interviews. BMC Med. 2018;16(1):136. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref22] 22. Lafrenière J, Lamarche B, Laramée C, et al. . Validation of a newly automated web-based 24-hour dietary recall using fully controlled feeding studies. BMC Nutr. 2017;3:34. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref23] 23. Subar AF, Potischman N, Dodd KW, et al. . Performance and feasibility of recalls completed using the Automated Self-Administered 24-hour Dietary Assessment Tool in relation to other self-report tools and biomarkers in the Interactive Diet and Activity Tracking in AARP (IDATA) Study. J Acad Nutr Diet. 2020;120(11):1805–1820. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref24] 24. Park Y, Dodd KW, Kipnis V, et al. . Comparison of self-reported dietary intakes from the Automated Self-Administered 24-h recall, 4-d food records, and food-frequency questionnaires against recovery biomarkers. Am J Clin Nutr. 2018;107(1):80–93. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref25] 25. Colditz GA, Manson JE, Hankinson SE. The Nurses’ Health Study: 20-year contribution to the understanding of health among women. J Womens Health. 1997;6(1):49–62. [DOI] [PubMed] [Google Scholar]

[ref26] 26. Colditz GA, Hankinson SE. The Nurses’ Health Study: lifestyle and health among women. Nat Rev Cancer. 2005;5(5):388–396. [DOI] [PubMed] [Google Scholar]

[ref27] 27. Yuan C, Spiegelman D, Rimm EB, et al. . Validity of a dietary questionnaire assessed by comparison with multiple weighed dietary records or 24-hour recalls. Am J Epidemiol. 2017;185(7):570–584. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref28] 28. Chomistek AK, Yuan C, Matthews CE, et al. . Physical activity assessment with the ActiGraph GT3X and doubly labeled water. Med Sci Sport Exerc. 2017;49(9):1935–1944. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref29] 29. Yuan C, Spiegelman D, Rimm EB, et al. . Relative validity of nutrient intakes assessed by questionnaire, 24-hour recalls, and diet records as compared with urinary recovery and plasma concentration biomarkers: findings for women. Am J Epidemiol. 2018;187(5):1051–1063. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref30] 30. Wilson KM, Kasperzyk JL, Rider JR, et al. . Coffee consumption and prostate cancer risk and progression in the Health Professionals Follow-up Study. J Natl Cancer Inst. 2011;103(11):876–884. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref31] 31. National Cancer Institute . Comparison of ASA24 versions. https://epi.grants.cancer.gov/asa24/comparison.html. Updated December 15, 2020. Accessed January 5, 2021.

[ref32] 32. National Cancer Institute . ASA24 known issues & workarounds. https://epi.grants.cancer.gov/asa24/resources/issues.html. Updated January 24, 2020. Accessed January 5, 2021.

[ref33] 33. US Department of Agriculture . Food and nutrient database for dietary surveys. https://www.ars.usda.gov/northeast-area/beltsville-md-bhnrc/beltsville-human-nutrition-research-center/food-surveys-research-group/docs/fndds/. Updated July 15, 2020. Accessed January 5, 2021.

[ref34] 34. Subar AF, Thompson FE, Kipnis V, et al. . Comparative validation of the Block, Willett, and National Cancer Institute food frequency questionnaires: the Eating at America’s Table Study. Am J Epidemiol. 2001;154(12):1089–1099. [DOI] [PubMed] [Google Scholar]

[ref35] 35. Willett WC, Sampson L, Stampfer MJ, et al. . Reproducibility and validity of a semiquantitative food frequency questionnaire. Am J Epidemiol. 1985;122(1):51–65. [DOI] [PubMed] [Google Scholar]

[ref36] 36. University of Minnesota Nutrition Coordinating Centre . Nutrition Data System for Research (NDSR) Software. Minneapolis, MN: University of Minnesota Nutrition Coordinating Centre; 2020. [Google Scholar]

[ref37] 37. Schoeller DA, Hnilicka JM. Reliability of the doubly labeled water method for the measurement of total daily energy expenditure in free-living subjects. J Nutr. 1996;126(1):348S–354S. [PubMed] [Google Scholar]

[ref38] 38. Bingham SA. Urine nitrogen as a biomarker for the validation of dietary protein intake. J Nutr. 2003;133(3):921S–924S. [DOI] [PubMed] [Google Scholar]

[ref39] 39. Mickelsen O, Makdani D, Gill JL, et al. . Sodium and potassium intakes and excretions of normal men consuming sodium chloride or a 1:1 mixture of sodium and potassium chlorides. Am J Clin Nutr. 1977;30(12):2033–2040. [DOI] [PubMed] [Google Scholar]

[ref40] 40. Luft FC, Fineberg NS, Sloan RS. Estimating dietary sodium intake in individuals receiving a randomly fluctuating intake. Hypertension. 1982;4(6):805–808. [DOI] [PubMed] [Google Scholar]

[ref41] 41. Holbrook JT, Patterson KY, Bodner JE, et al. . Sodium and potassium intake and balance in adults consuming self-selected diets. Am J Clin Nutr. 1984;40(4):786–793. [DOI] [PubMed] [Google Scholar]

[ref42] 42. Freedman LS, Midthune D, Carroll RJ, et al. . Adjustments to improve the estimation of usual dietary intake distributions in the population. J Nutr. 2004;134(7):1836–1843. [DOI] [PubMed] [Google Scholar]

[ref43] 43. Subar AF, Midthune D, Tasevska N, et al. . Checking for completeness of 24-h urine collection using Para-amino benzoic acid not necessary in the Observing Protein and Energy Nutrition study. Eur J Clin Nutr. 2013;67(8):863–867. [DOI] [PubMed] [Google Scholar]

[ref44] 44. Centers for Disease Control and Prevention . Body mass index (BMI). https://www.cdc.gov/healthyweight/assessing/bmi/index.html. Reviewed September 17, 2020. Accessed January 5, 2021.

[ref45] 45. SAS Institute . Statistical Analysis Software, version 9.4. Cary, NC. https://support.sas.com/software/94/. Accessed February 21, 2022. [Google Scholar]

[ref46] 46. Carroll RJ. Measurement Error in Nonlinear Models: A Modern Perspective. 2nd ed. New York, NY: Chapman & Hall/CRC; 2006. [Google Scholar]

[ref47] 47. Dodd KW, Guenther PM, Freedman LS, et al. . Statistical methods for estimating usual intake of nutrients and foods: a review of the theory. J Am Diet Assoc. 2006;106(10):1640–1650. [DOI] [PubMed] [Google Scholar]

[ref48] 48. Spiegelman D, McDermott A, Rosner B. Regression calibration method for correcting measurement-error bias in nutritional epidemiology. Am J Clin Nutr. 1997;65(4):1179S–1186S. [DOI] [PubMed] [Google Scholar]

[ref49] 49. Prentice RL, Mossavar-Rahmani Y, Huang Y, et al. . Evaluation and comparison of food records, recalls, and frequencies for energy and protein assessment by using recovery biomarkers. Am J Epidemiol. 2011;174(5):591–603. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref50] 50. Willett WC. Nutritional Epidemiology. 3rd ed. New York, NY: Oxford University Press; 2013. [Google Scholar]

[ref51] 51. Bingham SA, Gill C, Welch A, et al. . Validation of dietary assessment methods in the UK arm of EPIC using weighed records, and 24-hour urinary nitrogen and potassium and serum vitamin c and carotenoids as biomarkers. Int J Epidemiol. 1997;26(suppl 1):S137–S151. [DOI] [PubMed] [Google Scholar]

[ref52] 52. McKeown NM, Day NE, Welch AA, et al. . Use of biological markers to validate self-reported dietary intake in a random sample of the European Prospective Investigation Into Cancer United Kingdom Norfolk cohort. Am J Clin Nutr. 2001;74(2):188–196. [DOI] [PubMed] [Google Scholar]

[ref53] 53. Subar AF, Freedman LS, Tooze JA, et al. . Addressing current criticism regarding the value of self-report dietary data. J Nutr. 2015;145(12):2639–2645. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref54] 54. Reedy J, Subar A, George S, et al. . Extending methods in dietary patterns research. Nutrients. 2018;10(5):571. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref55] 55. Gorber SC, Tremblay M, Moher D, et al. . A comparison of direct vs. self-report measures for assessing height, weight and body mass index: a systematic review. Obes Rev. 2007;8(4):307–326. [DOI] [PubMed] [Google Scholar]

PERMALINK

Measurement Error Affecting Web- and Paper-Based Dietary Assessment Instruments: Insights From the Multi-Cohort Eating and Activity Study for Understanding Reporting Error

Sharon I Kirkpatrick

Richard P Troiano

Brian Barrett

Christopher Cunningham

Amy F Subar

Yikyung Park

Heather R Bowles

Laurence S Freedman

Victor Kipnis

Eric B Rimm

Walter C Willett

Nancy Potischman

Donna Spielgelman

David J Baer

Dale A Schoeller

Kevin W Dodd

Abstract

Abbreviations

METHODS

Validation studies and their populations

Measures

Table 1.

Self-report dietary assessment instruments.

Biomarkers.

Covariates.

Statistical analysis

Preliminary analyses.

Descriptive analyses.

Measurement error modeling.

RESULTS

Table 2.

Table 3.

Table 4.

Table 8.

Table 5.

Table 6.

Table 7.

DISCUSSION

Supplementary Material

ACKNOWLEDGMENTS

REFERENCES

Associated Data

Supplementary Materials

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases