Abstract
We pooled data from 5 large validation studies (1999–2009) of dietary self-report instruments that used recovery biomarkers as referents, to assess food frequency questionnaires (FFQs) and 24-hour recalls (24HRs). Here we report on total potassium and sodium intakes, their densities, and their ratio. Results were similar by sex but were heterogeneous across studies. For potassium, potassium density, sodium, sodium density, and sodium:potassium ratio, average correlation coefficients for the correlation of reported intake with true intake on the FFQs were 0.37, 0.47, 0.16, 0.32, and 0.49, respectively. For the same nutrients measured with a single 24HR, they were 0.47, 0.46, 0.32, 0.31, and 0.46, respectively, rising to 0.56, 0.53, 0.41, 0.38, and 0.60 for the average of three 24HRs. Average underreporting was 5%–6% with an FFQ and 0%–4% with a single 24HR for potassium but was 28%–39% and 4%–13%, respectively, for sodium. Higher body mass index was related to underreporting of sodium. Calibration equations for true intake that included personal characteristics provided improved prediction, except for sodium density. In summary, self-reports capture potassium intake quite well but sodium intake less well. Using densities improves the measurement of potassium and sodium on an FFQ. Sodium:potassium ratio is measured much better than sodium itself on both FFQs and 24HRs.
Keywords: attenuation factors, calibration models, dietary measurement error, food frequency questionnaire, 24-hour recall, underreporting
Studies of dietary intake and its relationship to health outcomes often use self-reported intakes (1). Assessing the validity of dietary self-report instruments is needed to reliably interpret the results. Dietary intake recovery biomarkers (2) that provide accurate assessments of short-term intakes are available for just 4 dietary components: energy, protein, potassium, and sodium. A series of large validation studies with recovery biomarkers, starting with the Observing Protein and Energy Nutrition (OPEN) Study (3), have been conducted in the United States. In 2009, investigators from 5 such studies agreed to pool their data for common analysis (the Validation Studies Pooling Project), so as to characterize with greater precision the nature and magnitude of reporting errors in food frequency questionnaires (FFQs) and 24-hour recalls (24HRs) and investigate personal characteristics associated with such errors. Results regarding energy and protein have already been reported (4). Here, we present results regarding potassium and sodium intakes. Low potassium and high sodium intakes (5–8) have been associated with elevated blood pressure in epidemiologic studies and clinical trials and are important to public health.
METHODS
Validation studies and their populations
The 5 studies, described in more detail elsewhere (4), differed in their emphasis and in the populations studied. The OPEN Study was conducted in adult volunteers aged 40–69 years residing in suburban Maryland (3). In the Energetics Study, white and African-American young adults in Los Angeles, California, were studied (9). In a study conducted to validate the US Department of Agriculture's Automated Multiple-Pass Method (AMPM), Moshfegh et al. (10) recruited a population similar to that in OPEN. The Nutrition Biomarker Study (NBS) and the Nutrition and Physical Activity Assessment Study (NPAAS) included participants from study centers spread across the United States as part of the Women's Health Initiative Dietary Modification Trial and Observational Study, respectively (11, 12), and included only women, mostly over age 60 years. Further details are provided in Table 1. Each study proposal received institutional review board approval.
Table 1.
Characteristics of 5 Validation Studies of Dietary Self-Report Instruments and Their Populations
| First Author, Year (Reference No.) | Study | Dates of Study | Sponsoring Organization | No. of Participants | Mean Age, years (SD) | Sex, % men | Mean Body Mass Indexa (SD) | Race/Ethnicity, % |
Education, % |
||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Non-Hispanic White | Black | College Degree | Postgraduate Study | ||||||||
| Subar, 2003 (3) | OPEN Study | 1999–2000 | National Cancer Institute | 484 | 53.4 (8.3) | 54 | 27.9 (5.3) | 83 | 6 | 54 | 32 |
| Arab, 2010 (9) | Energetics Study | 2006–2009 | University of California, Los Angeles | 263 | 37.8 (12.6) | 36 | 26.8 (6.2) | 49 | 51 | 81 | 15 |
| Moshfegh, 2008 (10) | AMPM | 2002–2004 | US Department of Agriculture | 524 | 49.5 (10.9) | 50 | 26.6 (4.6) | 77 | 13 | 54 | 39 |
| Neuhouser, 2008 (11) | NBS | 2004–2005 | Women's Health Initiative | 544 | 70.9 (6.3) | 0 | 28.2 (5.5) | 83 | 11 | 40 | 31 |
| Prentice, 2011 (12) | NPAAS | 2007–2009 | Women's Health Initiative | 450 | 70.5 (6.0) | 0 | 28.5 (6.4) | 64 | 18 | 38 | 38 |
Abbreviations: AMPM, Automated Multiple-Pass Method validation study; NBS, Nutrition Biomarker Study; NPAAS, Nutrition and Physical Activity Assessment Study; OPEN, Observing Protein and Energy Nutrition; SD, standard deviation.
a Weight (kg)/height (m)2.
Self-report instruments
The participants in each study completed an FFQ. Although some repeat administrations were performed in OPEN, AMPM, NBS, and NPAAS, we analyzed only the first administration here. Three FFQs were used: the Diet and Health Questionnaire (13) in OPEN and Energetics, the Harvard FFQ (14) in AMPM, and the Women's Health Initiative FFQ (15) in NBS and NPAAS.
Each study also included 2 or more 24HR assessments administered on nonconsecutive days in either all participants (4 of 5 studies) or a 20% subset of participants (NBS) (Table 2). Different 24HRs were used (Table 2). The 24HR energy data were assessed for possible drift over repeat assessments. In one study (Energetics, which had 8 repeat non-interviewer-assisted assessments), a marked drift was detected after the fourth assessment. Additionally, since the first assessment was thought to be on the participants’ learning curve, only the second, third, and fourth 24HR assessments were analyzed in this study.
Table 2.
Self-Report Instruments and Biomarkers Used in 5 Validation Studies of Dietary Self-Report Instruments
| First Author, Year (Reference No.) | Studya | FFQ |
24HR |
% of Patients Engaging in 24HR | DLW Laboratory | % of Patients Undergoing Repeat DLW Assessments | Time Between Repeat DLW Assessments | Urinary Potassium and Sodium |
Time Between Repeat Urine Assessments | % of Patients Undergoing a Repeat Set of Urine Assessments | ||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Type of FFQ | No. of Assessments | Type of 24HR | No. of Assessments | Laboratory | Method | No. of Assessments in a Set | ||||||||
| Subar, 2003 (3) | OPEN Study | DHQ | 2 | AMPM, version 1 | 2 | 100 | University of Wisconsin | 5 | 2 weeks | Medical Research Council | FP | 2 | 12 days | 0 |
| Arab, 2010 (9) | Energetics Study | DHQ | 1 | DietDay (Centrax Corporation, Chicago, Illinois) | 8 | 100 | University of Wisconsin | 23 | 6 months | University of California, Los Angeles | ISE | 2 | 10 days | 23 |
| Moshfegh, 2008 (10) | AMPM | Harvard FFQ | 1 | AMPM, version 2 | 3 | 100 | US Department of Agriculture | 11 | 10–23 months | MedStar's Penn Medical Laboratory (Hyattsville, Maryland) | ISE | 2 | 5 days | 11 |
| Neuhouser, 2008 (11) | NBS | WHI | 1 | AMPM with NDSRb | 2 | 20 | University of Wisconsin | 20 | 6 months | Pharmaceutical Product Development LLC (Wilmington, North Carolina) | ISE | 1 | 5 months (1 urine assessment per set) | 20 |
| Prentice, 2011 (12) | NPAAS | WHI | 1 | AMPM with NDSR | 3 | 100 | University of Wisconsin | 20 | 6 months | University of Washington | ISE | 1 | 6 months (1 urine assessment per set) | 20 |
Abbreviations: AMPM, Automated Multiple-Pass Method validation study; DHQ, Diet History Questionnaire; DLW, doubly labeled water; FFQ, food frequency questionnaire; FP, flame photometry; 24HR, 24-hour recall; ISE, ion-selective electrode; NBS, Nutrition Biomarker Study; NDSR, Nutrition Data System for Research; NPAAS, Nutrition and Physical Activity Assessment Study; OPEN, Observing Protein and Energy Nutrition; WHI, Women's Health Initiative.
a For study dates, see Table 1.
b Software from the University of Minnesota Nutrition Coordinating Center (Minneapolis, Minnesota).
Biomarkers
Recovery biomarkers were measured in all 5 studies. Measurements included doubly labeled water for energy intake (16) and 24-hour urinary potassium and sodium levels for potassium and sodium intake (17, 18) (Table 2). The doubly labeled water method (Table 2) assesses energy expenditure over a 10- to 14-day period and, assuming individuals to be in energy balance, measures average daily energy intake over this period (16). Urinary levels of potassium and sodium (total amounts in 24-hour urine samples) assess potassium and sodium intake over a 24-hour period (17, 18). Urinary potassium and sodium levels were measured at a Medical Research Council laboratory (Cambridge, United Kingdom) using the flame photometry method (OPEN) and at a University of California, Los Angeles (Los Angeles, California) laboratory (Energetics—potassium), a US Department of Agriculture (Washington, DC) laboratory (Energetics—sodium), the MedStar Health Research Institute's Penn Medical Laboratory (Hyattsville, Maryland) (AMPM), a Pharmaceutical Product Development LLC (Wilmington, North Carolina) laboratory (NBS), and a University of Washington (Seattle, Washington) laboratory (NPAAS) using the ion-selective electrode method (Table 2). Three studies included repeat determinations in the main protocol, made approximately 5 days apart; NBS and NPAAS included repeat determinations in a 20% reliability subsample, approximately 6 months later. These repeat determinations were included in our analyses.
Urinary potassium values were divided by 0.8 to convert them to dietary potassium values (19). Urinary sodium values were divided by 0.86 to convert them to dietary sodium values (20).
Reliability substudies
Each study included a substudy (of varying sizes) to examine within-person variation in self-reports and biomarkers. The time between initial administration and repeat administrations varied considerably between studies, as did the extent of the repeat data collection (Table 2). In OPEN, only doubly labeled water measurements were repeated, while the other studies repeated both biomarker measurements and self-reports.
Relative timing of the measurements in the main studies
In all studies except AMPM, the FFQ was administered at the beginning of the main study. In AMPM, the FFQ was administered 1–14 months after the beginning of the main study. In OPEN, a second FFQ was administered 60 days after the first. In all studies, doubly labeled water was administered at 1–14 days, and 24-hour urine samples were collected during the same period. The 24HRs were administered on the following days: OPEN, days 1 and 61; Energetics, days 1, 3, 5, 8, 11, 14, 30, and 60; AMPM, days 1, 5–6, and 10–11; NBS, none in the main study; and NPAAS, 3 occasions over a 14- to 104-day period.
Statistical methods
We report on 5 dietary components: potassium, potassium density, sodium, sodium density, and sodium:potassium ratio (SPR). Densities are defined as the ratio of nutrient intake (mg/day) to energy intake (thousands of kcal/day) and are studied because they represent energy-adjusted quantities (1). They are often reported with less relative error than are absolute intakes (21). SPR has been proposed as a risk factor for cardiovascular disease (22).
All dietary intake and biomarker variables were logarithmically transformed so that their distributions were approximately normal, as confirmed by visualization of quantile-quantile plots.
Recent research (23) indicates that para-aminobenzoic acid need not be used to assess the completeness of 24-hour urine collection. Urinary potassium and sodium values from analysis were excluded only if the participant reported missing 2 or more voids during the 24-hour collection. Exclusion of outliers is described in Web Appendix 1 (available at http://aje.oxfordjournals.org/).
Since repeated 24HR assessments were administered in all studies, we investigated reporting characteristics for a single 24HR and for two and three 24HRs (where available), using mean log reported intake as the value reported from multiple assessments.
Analyses were based on the following premises: 1) recovery biomarker levels provide, on a logarithmic scale, unbiased estimates of short-term intake; 2) within-person variation is independent of personal characteristics; and 3) in the 5 studies evaluated, intake did not vary systematically with time. Together, these assumptions imply that recovery biomarkers are unbiased for measurement of longer-term usual intake.
We investigated several characteristics of dietary reporting error, including reporting bias, attenuation factors, and correlation coefficients for correlations between reported and true usual intake. Reporting bias was estimated as the mean difference between log reported intake (first administration) and the log biomarker value, reexpressed as relative bias by exponentiation.
The attenuation factor (usually between 0 and 1) is the multiplicative bias or shrinkage factor in the estimated regression coefficient when a health outcome is regressed on continuous self-reported intake rather than true dietary intake, in the absence of systematic reporting biases dependent on factors related to the health outcome. It was estimated in each study as the slope in the linear regression of the log biomarker value on log first reported intake. To accommodate the multiple biomarker determinations, we employed linear mixed models (24), with a random intercept for participants (Web Appendix 2). Across-study average attenuation factors were weighted by the inverses of their variances.
The coefficient for correlation between reported and true usual intake is used to measure the loss, due to measurement error, of statistical power to detect diet-health associations. With simple disease models, which have a single normally distributed exposure that is categorized, it can also be used to deattenuate relative risks between the categories (25). It was estimated as the correlation between the first reported intake and the biomarker value, adjusted for within-person variation in the biomarker using a method similar to that of Rosner and Willett (26) (Web Appendix 3). Low values of the attenuation factor and of correlation between reported intake and true intake (e.g., less than 0.4) are undesirable, although there is no sharp cutoff.
We investigated the role of personal characteristics in modifying reporting bias and attenuation. We examined sex, age (<40, 40–49, 50–59, 60–69, 70–79, or ≥80 years), body mass index (weight (kg)/height (m)2; log-transformed), race/ethnicity (black, white/other), and education (high school, college, postgraduate study). Their influence on bias was measured through the coefficients of linear regressions of reporting bias (reported intake minus biomarker value) on these characteristics. Their influence on attenuation was assessed via the coefficients of the interaction of the characteristics with reported intake in linear regressions of log biomarker value on these variables. Calibration equations for predicting true usual intake were obtained from regressions of the log biomarker value on log reported intake and personal characteristics. The accuracy of prediction was measured by means of the multiple correlation coefficient (R2), adjusted for within-person variation in the biomarker (12) (Web Appendix 3), and was contrasted with the R2 obtained when the self-report instrument was the only predictor.
All analyses were performed as meta-analyses, with study entered as a dummy variable into the regression model (Web Appendix 2). Between-study heterogeneity was assessed through interactions of study with other terms in the model, and was quantified in terms of the ratio of between-study variance to total variance, I2 (27). Between-sex heterogeneity in attenuation factors was assessed through interactions between sex and self-reports. The statistical significance of coefficients was tested using 2-sided t tests or F tests. Although P < 0.05 was used as a guide for statistical significance, the tables present many P values that were unadjusted for multiple testing. Conclusions were drawn based on the consistency of results across studies, as well as the P values themselves. Statistical analyses were implemented in SAS, version 9 (SAS Institute, Inc., Cary, North Carolina) (28).
RESULTS
Reporting bias
Geometric mean intakes (Table 3) and relative bias (Table 4) are shown for FFQs and single 24HRs in Tables 3 and 4. Results for the mean of two or three 24HRs were very similar to those for a single 24HR. Self-report mean values for potassium were similar to biomarker values for some studies but were underestimated by 10%–15% for NBS and NPAAS. Potassium density was overestimated by approximately 35% by FFQ reports, mainly because of energy underestimation (Web Tables 1–8), and by about 15% by 24HR reports. Sodium intake was underestimated by 23%–52% by FFQ report in all studies except Energetics (2%–9%), and it was also underestimated by 23%–28% by 24HR report in NBS and NPAAS. Density of sodium intake was overestimated by approximately 30% by FFQ report in Energetics but underestimated by about 30% in AMPM. It was overestimated by about 10% by 24HR report in OPEN and Energetics, but not in the other studies. SPR was underestimated by 33%–47% using an FFQ in OPEN and AMPM but only by 12%–17% in Energetics, NBS, and NPAAS. Using a 24HR, SPR was underestimated by less than 10%, on average.
Table 3.
Average Intakes of Potassium and Sodium, Their Densities, and Their Ratio in 5 Validation Studies of Dietary Self-Report Instruments, by Study, Sex, and Instrument
| Intake Type, Sex, and Instrument | Studya |
|||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| OPEN Study |
Energetics Study |
AMPM |
NBSb |
NPAASb |
||||||
| GM | 95% CI | GM | 95% CI | GM | 95% CI | GM | 95% CI | GM | 95% CI | |
| Potassium, mg/day | ||||||||||
| Men | ||||||||||
| Biomarker | 3,465 | 3,337, 3,599 | 3,516 | 3,290, 3,757 | 3,449 | 3,294, 3,611 | ||||
| 24HRc | 3,372 | 3,223, 3,529 | 3,438 | 3,088, 3,826 | 3,402 | 3,264, 3,546 | ||||
| FFQ | 3,323 | 3,170, 3,484 | 3,512 | 3,239, 3,808 | 2,991 | 2,870, 3,116 | ||||
| Women | ||||||||||
| Biomarker | 2,688 | 2,555, 2,828 | 2,296 | 2,151, 2,450 | 2,615 | 2,494, 2,741 | 2,918 | 2,835, 3,004 | 2,690 | 2,590, 2,795 |
| 24HR | 2,702 | 2,563, 2,850 | 2,573 | 2,362, 2,802 | 2,621 | 2,510, 2,736 | 2,588 | 2,432, 2,753 | 2,359 | 2,272, 2,449 |
| FFQ | 2,798 | 2,667, 2,935 | 2,581 | 2,382, 2,797 | 2,684 | 2,563, 2,811 | 2,532 | 2,457, 2,608 | 2,441 | 2,350, 2,536 |
| Potassium Density, mg/1,000 kcald | ||||||||||
| Men | ||||||||||
| Biomarker | 1,225 | 1,177, 1,274 | 1,189 | 1,105, 1,280 | 1,213 | 1,155, 1,273 | ||||
| 24HR | 1,349 | 1,302, 1,397 | 1,212 | 1,135, 1,294 | 1,372 | 1,321, 1,425 | ||||
| FFQ | 1,671 | 1,621, 1,724 | 1,585 | 1,510, 1,664 | 1,551 | 1,509, 1,594 | ||||
| Women | ||||||||||
| Biomarker | 1,194 | 1,132, 1,259 | 1,018 | 954, 1,087 | 1,202 | 1,142, 1,265 | 1,404 | 1,362, 1,448 | 1,320 | 1,266, 1,375 |
| 24HR | 1,408 | 1,346, 1,473 | 1,228 | 1,154, 1,306 | 1,347 | 1,154, 1,306 | 1,687 | 1,602, 1,776 | 1,526 | 1,481, 1,573 |
| FFQ | 1,836 | 1,773, 1,900 | 1,568 | 1,503, 1,636 | 1,625 | 1,583, 1,668 | 1,728 | 1,695, 1,763 | 1,667 | 1,633, 1,703 |
| Sodium, mg/day | ||||||||||
| Men | ||||||||||
| Biomarker | 4,502 | 4,287, 4,727 | 3,692 | 3,371, 4,043 | 4,648 | 4,421, 4,886 | ||||
| 24HR | 4,446 | 4,258, 4,643 | 4,010 | 3,506, 4,587 | 4,176 | 3,982, 4,379 | ||||
| FFQ | 3,070 | 2,920, 3,227 | 3,377 | 3,077, 3,706 | 2,188 | 2,088, 2,293 | ||||
| Women | ||||||||||
| Biomarker | 3,310 | 3,126, 3,503 | 2,555 | 2,345, 2,783 | 3,494 | 3,330, 3,666 | 3,263 | 3,155, 3,373 | 3,056 | 2,933, 3,183 |
| 24HR | 3,337 | 3,153, 3,532 | 2,580 | 2,354, 2,827 | 3,184 | 3,034, 3,342 | 2,437 | 2,275, 2,611 | 2,358 | 2,268, 2,451 |
| FFQ | 2,308 | 2,186, 2,436 | 2,459 | 2,270, 2,662 | 1,851 | 1,762, 1,945 | 2,394 | 2,318, 2,472 | 2,383 | 2,286, 2,484 |
| Sodium Density, mg/1,000 kcal | ||||||||||
| Men | ||||||||||
| Biomarker | 1,571 | 1,500, 1,645 | 1,237 | 1,115, 1,373 | 1,618 | 1,539, 1,700 | ||||
| 24HR | 1,763 | 1,707, 1,821 | 1,434 | 1,323, 1,554 | 1,674 | 1,617, 1,734 | ||||
| FFQ | 1,568 | 1,535, 1,601 | 1,538 | 1,480, 1,598 | 1,132 | 1,105, 1,160 | ||||
| Women | ||||||||||
| Biomarker | 1,484 | 1,405, 1,567 | 1,148 | 1,054, 1,250 | 1,613 | 1,537, 1,693 | 1,593 | 1,541, 1,646 | 1,493 | 1,433, 1,555 |
| 24HR | 1,708 | 1,642, 1,776 | 1,231 | 1,149, 1,318 | 1,630 | 1,575, 1,686 | 1,604 | 1,523, 1,689 | 1,535 | 1,491, 1,581 |
| FFQ | 1,519 | 1,479, 1,560 | 1,484 | 1,440, 1,530 | 1,122 | 1,094, 1,151 | 1,651 | 1,628, 1,675 | 1,637 | 1,611, 1,665 |
| Sodium:Potassium Ratio | ||||||||||
| Men | ||||||||||
| Biomarker | 1.31 | 1.24, 1.38 | 1.09 | 0.96, 1.24 | 1.34 | 1.26, 1.42 | ||||
| 24HR | 1.32 | 1.26, 1.38 | 1.14 | 1.03, 1.27 | 1.21 | 1.16, 1.27 | ||||
| FFQ | 0.93 | 0.90, 0.96 | 0.98 | 0.92, 1.03 | 0.73 | 0.71, 0.76 | ||||
| Women | ||||||||||
| Biomarker | 1.22 | 1.13, 1.31 | 1.12 | 1.03, 1.23 | 1.34 | 1.26, 1.42 | 1.13 | 1.09, 1.18 | 1.13 | 1.08, 1.18 |
| 24HR | 1.21 | 1.15, 1.28 | 1.00 | 0.91, 1.11 | 1.21 | 1.15, 1.27 | 0.95 | 0.89, 1.02 | 1.00 | 0.96, 1.04 |
| FFQ | 0.83 | 0.80, 0.86 | 0.96 | 0.93, 1.00 | 0.69 | 0.67, 0.72 | 0.96 | 0.93, 0.98 | 0.98 | 0.96, 1.01 |
Abbreviations: AMPM, Automated Multiple-Pass Method validation study; CI, confidence interval; FFQ, food frequency questionnaire; GM, geometric mean; 24HR, 24-hour recall; NBS, Nutrition Biomarker Study; NPAAS, Nutrition and Physical Activity Assessment Study; OPEN, Observing Protein and Energy Nutrition.
a For study dates, see Table 1.
b NBS and NPAAS included only women.
c Single administration of a 24HR; data were taken from the first recall, except for the Energetics Study, where the second recall was used.
d 1 kcal = 4.184 kJ.
Table 4.
Bias in Reported Intakes of Potassium and Sodium, Their Densities, and Their Ratio in 5 Validation Studies of Dietary Self-Report Instruments, by Study, Sex, and Instrument
| Intake Type, Sex, and Instrument | Studya |
Average RBc, % | P Valued | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| OPEN Study |
Energetics Study |
AMPM |
NBSb |
NPAASb |
||||||||
| RBe, % | 95% CI | RB, % | 95% CI | RB, % | 95% CI | RB, % | 95% CI | RB, % | 95% CI | |||
| Potassium | ||||||||||||
| Men | ||||||||||||
| 24HRf | −1 | −5, 4 | 6 | −2, 15 | −2 | −6, 3 | 0 | 0.28 | ||||
| FFQ | −2 | −7, 2 | 10 | 1, 19 | −15 | −19, −10 | −6 | <0.001 | ||||
| Women | ||||||||||||
| 24HR | 1 | −5, 7 | 10 | 3, 18 | 0 | −5, 6 | −12 | −19, −5 | −13 | −16, −9 | −4 | <0.001 |
| FFQ | 5 | −1, 11 | 11 | 4, 20 | 3 | −3, 9 | −13 | −17, −10 | −10 | −14, −6 | −5 | <0.001 |
| Potassium Density | ||||||||||||
| Men | ||||||||||||
| 24HR | 13 | 8, 18 | 17 | 8, 26 | 13 | 8, 18 | 13 | 0.72 | ||||
| FFQ | 41 | 35, 47 | 49 | 39, 61 | 26 | 21, 32 | 36 | <0.001 | ||||
| Women | ||||||||||||
| 24HR | 19 | 13, 26 | 18 | 11, 26 | 12 | 7, 18 | 21 | 12, 31 | 16 | 11, 21 | 16 | 0.39 |
| FFQ | 56 | 48, 64 | 52 | 43, 60 | 35 | 29, 41 | 22 | 19, 26 | 26 | 21, 31 | 33 | <0.001 |
| Sodium | ||||||||||||
| Men | ||||||||||||
| 24HR | −4 | −10, 1 | 8 | −2, 19 | −8 | −13, −2 | −4 | 0.029 | ||||
| FFQ | −34 | −38, −30 | −9 | −18, 1 | −52 | −55, −49 | −39 | <0.001 | ||||
| Women | ||||||||||||
| 24HR | −3 | −10, 5 | 2 | −7, 11 | −8 | −14, −1 | −28 | −35, −21 | −23 | −27, −19 | −13 | <0.001 |
| FFQ | −33 | −38, −28 | −2 | −10, 7 | −46 | −49, −42 | −27 | −30, −23 | −23 | −27, −18 | −28 | <0.001 |
| Sodium Density | ||||||||||||
| Men | ||||||||||||
| 24HR | 9 | 3, 14 | 16 | 7, 27 | 5 | 0, 10 | 8 | 0.12 | ||||
| FFQ | −4 | −8, 1 | 26 | 16, 35 | −28 | −31, −25 | −12 | <0.001 | ||||
| Women | ||||||||||||
| 24HR | 13 | 6, 20 | 8 | 0, 16 | 2 | −4, 8 | 0 | −9, 9 | 3 | −2, 8 | 5 | 0.086 |
| FFQ | −1 | −6, 5 | 31 | 23, 39 | −30 | −33, −26 | 4 | 0, 7 | 10 | 6, 15 | 1 | <0.001 |
| Sodium:Potassium Ratio | ||||||||||||
| Men | ||||||||||||
| 24HR | −5 | −9, 1 | −1 | −10, 8 | −6 | −11, −1 | −5 | 0.65 | ||||
| FFQ | −33 | −36, −29 | −17 | −24, −9 | −43 | −46, −40 | −35 | <0.001 | ||||
| Women | ||||||||||||
| 24HR | −4 | −11, 3 | −9 | −16, −2 | −8 | −14, −2 | −18 | −26, −9 | −11 | −15, −6 | −9 | 0.16 |
| FFQ | −36 | −39, −32 | −14 | −19, −7 | −47 | −50, −44 | −15 | −18, −12 | −12 | −16, −9 | −24 | <0.001 |
Abbreviations: AMPM, Automated Multiple-Pass Method validation study; CI, confidence interval; FFQ, food frequency questionnaire; 24HR, 24-hour recall; NBS, Nutrition Biomarker Study; NPAAS, Nutrition and Physical Activity Assessment Study; OPEN, Observing Protein and Energy Nutrition; RB, relative bias.
a For study dates, see Table 1.
b NBS and NPAAS included only women.
c Average weighted by the inverse of the variance.
d P for heterogeneity across studies.
e % RB = 100 × exponential (mean log self-report − mean log biomarker value) − 100; negative values indicate underreporting.
f Single administration of a 24HR; data were taken from the first recall, except for the Energetics Study, where the second recall was used.
There were no personal characteristics that on both self-report instruments were consistently associated with reporting bias in potassium intake (Web Tables 9 and 10). In contrast to potassium, underreporting of sodium was strongly associated with higher body mass index and also, for FFQ reports, with being black, being male, and having a high school education versus a college education or higher.
Attenuation factors and correlations of reported intake with true usual intake
Attenuation factors for FFQ-reported potassium were higher than those previously reported for protein and energy (4) (Table 5), with an average of 0.25–0.30. For a single 24HR, they were somewhat higher, with an average of approximately 0.30–0.35. Using the mean of two 24HR administrations increased the attenuation factor to about 0.45, and using the mean of three 24HRs increased it to 0.50.
Table 5.
Attenuation Factors for Reported Intakes of Potassium and Sodium, Their Densities, and Their Ratio in 5 Validation Studies of Dietary Self-Report Instruments
| Intake Type, Sex, and Instrument | Studya |
P Valueb | I2c | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| OPEN Study |
Energetics Study |
AMPM |
NBSd |
NPAASd |
Averagee |
|||||||||
| AF | 95% CI | AF | 95% CI | AF | 95% CI | AF | 95% CI | AF | 95% CI | AF | 95% CI | |||
| Potassium | ||||||||||||||
| Men | ||||||||||||||
| 1 × 24HR | 0.31 | 0.22, 0.40 | 0.08 | −0.04, 0.19 | 0.47 | 0.37, 0.58 | 0.30 | 0.24, 0.36 | <0.001 | 0.92 | ||||
| 2 × 24HR | 0.44 | 0.34, 0.54 | 0.10 | −0.03, 0.23 | 0.59 | 0.49, 0.69 | 0.42 | 0.35, 0.48 | <0.001 | 0.94 | ||||
| 3 × 24HR | 0.11 | −0.05, 0.26 | 0.70 | 0.59, 0.81 | 0.49 | 0.40, 0.58 | <0.001 | 0.97 | ||||||
| FFQ | 0.30 | 0.22, 0.39 | 0.15 | −0.02, 0.32 | 0.38 | 0.26, 0.50 | 0.30 | 0.24, 0.37 | 0.10 | 0.57 | ||||
| Women | ||||||||||||||
| 1 × 24HR | 0.32 | 0.23, 0.42 | 0.21 | 0.12, 0.30 | 0.46 | 0.34, 0.57 | 0.37 | 0.23, 0.52 | 0.42 | 0.34, 0.51 | 0.35 | 0.31, 0.40 | 0.003 | 0.75 |
| 2 × 24HR | 0.47 | 0.35, 0.58 | 0.33 | 0.23, 0.43 | 0.65 | 0.53, 0.78 | 0.46 | 0.31, 0.61 | 0.51 | 0.40, 0.61 | 0.47 | 0.42, 0.52 | 0.002 | 0.76 |
| 3 × 24HR | 0.37 | 0.26, 0.47 | 0.68 | 0.55, 0.82 | 0.56 | 0.45, 0.68 | 0.51 | 0.44, 0.58 | 0.001 | 0.86 | ||||
| FFQ | 0.22 | 0.10, 0.33 | 0.12 | 0.01, 0.22 | 0.26 | 0.14, 0.37 | 0.32 | 0.24, 0.39 | 0.25 | 0.17, 0.33 | 0.25 | 0.20, 0.29 | 0.05 | 0.57 |
| Potassium Density | ||||||||||||||
| Men | ||||||||||||||
| 1 × 24HR | 0.37 | 0.25, 0.49 | 0.19 | −0.02, 0.41 | 0.56 | 0.44, 0.69 | 0.43 | 0.35, 0.51 | 0.008 | 0.79 | ||||
| 2 × 24HR | 0.60 | 0.47, 0.74 | 0.31 | 0.06, 0.56 | 0.72 | 0.59, 0.85 | 0.62 | 0.53, 0.71 | 0.016 | 0.76 | ||||
| 3 × 24HR | 0.28 | 0.03, 0.54 | 0.82 | 0.68, 0.96 | 0.69 | 0.56, 0.81 | <0.001 | 0.92 | ||||||
| FFQ | 0.50 | 0.36, 0.64 | 0.35 | 0.06, 0.65 | 0.69 | 0.49, 0.89 | 0.53 | 0.43, 0.64 | 0.14 | 0.50 | ||||
| Women | ||||||||||||||
| 1 × 24HR | 0.44 | 0.33, 0.55 | 0.28 | 0.15, 0.41 | 0.61 | 0.48, 0.75 | 0.30 | 0.11, 0.48 | 0.34 | 0.22, 0.45 | 0.40 | 0.34, 0.46 | 0.003 | 0.75 |
| 2 × 24HR | 0.60 | 0.47, 0.73 | 0.42 | 0.26, 0.58 | 0.84 | 0.68, 1.01 | 0.39 | 0.18, 0.60 | 0.51 | 0.35, 0.67 | 0.57 | 0.49, 0.64 | 0.001 | 0.78 |
| 3 × 24HR | 0.44 | 0.29, 0.60 | 0.89 | 0.71, 1.08 | 0.62 | 0.44, 0.79 | 0.63 | 0.54, 0.73 | 0.001 | 0.85 | ||||
| FFQ | 0.51 | 0.36, 0.66 | 0.59 | 0.41, 0.77 | 0.77 | 0.55, 0.98 | 0.48 | 0.37, 0.60 | 0.50 | 0.33, 0.67 | 0.54 | 0.47, 0.61 | 0.21 | 0.31 |
| Sodium | ||||||||||||||
| Men | ||||||||||||||
| 1 × 24HR | 0.27 | 0.17, 0.38 | 0.06 | −0.06, 0.18 | 0.33 | 0.23, 0.42 | 0.24 | 0.18, 0.30 | 0.002 | 0.84 | ||||
| 2 × 24HR | 0.29 | 0.17, 0.41 | 0.07 | −0.09, 0.23 | 0.43 | 0.31, 0.55 | 0.30 | 0.22, 0.38 | 0.002 | 0.84 | ||||
| 3 × 24HR | 0.09 | −0.07, 0.26 | 0.49 | 0.35, 0.62 | 0.33 | 0.23, 0.43 | <0.001 | 0.93 | ||||||
| FFQ | 0.14 | 0.05, 0.24 | −0.01 | −0.18, 0.15 | 0.12 | 0.01, 0.24 | 0.11 | 0.04, 0.18 | 0.30 | 0.18 | ||||
| Women | ||||||||||||||
| 1 × 24HR | 0.11 | 0.00, 0.22 | 0.11 | 0.00, 0.22 | 0.27 | 0.17, 0.36 | 0.01 | −0.15, 0.16 | 0.11 | 0.02, 0.20 | 0.14 | 0.09, 0.19 | 0.03 | 0.62 |
| 2 × 24HR | 0.18 | 0.04, 0.32 | 0.19 | 0.05, 0.32 | 0.32 | 0.21, 0.44 | 0.07 | −0.12, 0.26 | 0.21 | 0.09, 0.33 | 0.22 | 0.15, 0.28 | 0.21 | 0.32 |
| 3 × 24HR | 0.23 | 0.08, 0.38 | 0.42 | 0.29, 0.55 | 0.29 | 0.15, 0.43 | 0.32 | 0.24, 0.40 | 0.14 | 0.49 | ||||
| FFQ | 0.11 | 0.00, 0.23 | −0.05 | −0.17, 0.08 | 0.10 | 0.00, 0.21 | 0.12 | 0.03, 0.20 | 0.08 | 0.00, 0.17 | 0.08 | 0.04, 0.13 | 0.27 | 0.23 |
| Sodium Density | ||||||||||||||
| Men | ||||||||||||||
| 1 × 24HR | 0.25 | 0.12, 0.38 | 0.17 | −0.04, 0.38 | 0.37 | 0.24, 0.50 | 0.29 | 0.20, 0.37 | 0.22 | 0.34 | ||||
| 2 × 24HR | 0.42 | 0.26, 0.58 | 0.05 | −0.29, 0.38 | 0.43 | 0.27, 0.59 | 0.39 | 0.28, 0.49 | 0.11 | 0.54 | ||||
| 3 × 24HR | −0.07 | −0.41, 0.28 | 0.49 | 0.31, 0.66 | 0.38 | 0.22, 0.53 | 0.005 | 0.87 | ||||||
| FFQ | 0.45 | 0.24, 0.65 | 0.45 | −0.02, 0.91 | 0.24 | 0.02, 0.46 | 0.36 | 0.22, 0.50 | 0.38 | 0.00 | ||||
| Women | ||||||||||||||
| 1 × 24HR | 0.12 | −0.03, 0.27 | 0.07 | −0.06, 0.21 | 0.29 | 0.15, 0.43 | 0.15 | −0.05, 0.35 | 0.18 | 0.06, 0.29 | 0.17 | 0.10, 0.23 | 0.25 | 0.26 |
| 2 × 24HR | 0.20 | 0.02, 0.37 | 0.12 | −0.05, 0.30 | 0.35 | 0.17, 0.54 | 0.34 | 0.11, 0.57 | 0.34 | 0.18, 0.51 | 0.27 | 0.19, 0.35 | 0.25 | 0.26 |
| 3 × 24HR | 0.15 | −0.05, 0.35 | 0.56 | 0.35, 0.77 | 0.50 | 0.32, 0.68 | 0.41 | 0.29, 0.52 | 0.008 | 0.79 | ||||
| FFQ | 0.34 | 0.12, 0.56 | −0.03 | −0.36, 0.31 | 0.36 | 0.16, 0.55 | 0.39 | 0.20, 0.57 | 0.42 | 0.21, 0.62 | 0.34 | 0.25, 0.44 | 0.24 | 0.27 |
| Sodium:Potassium Ratio | ||||||||||||||
| Men | ||||||||||||||
| 1 × 24HR | 0.43 | 0.31, 0.55 | 0.30 | 0.12, 0.48 | 0.56 | 0.45, 0.68 | 0.46 | 0.38, 0.55 | 0.041 | 0.69 | ||||
| 2 × 24HR | 0.69 | 0.55, 0.83 | 0.32 | 0.10, 0.54 | 0.72 | 0.58, 0.86 | 0.64 | 0.55, 0.73 | 0.007 | 0.80 | ||||
| 3 × 24HR | 0.35 | 0.10, 0.60 | 0.86 | 0.71, 1.00 | 0.73 | 0.60, 0.85 | <0.001 | 0.91 | ||||||
| FFQ | 0.55 | 0.40, 0.70 | 0.54 | 0.23, 0.84 | 0.66 | 0.48, 0.84 | 0.59 | 0.48, 0.69 | 0.60 | 0.00 | ||||
| Women | ||||||||||||||
| 1 × 24HR | 0.44 | 0.30, 0.58 | 0.25 | 0.15, 0.36 | 0.54 | 0.42, 0.66 | 0.16 | −0.08, 0.41 | 0.37 | 0.26, 0.48 | 0.37 | 0.31, 0.43 | 0.002 | 0.76 |
| 2 × 24HR | 0.68 | 0.52, 0.85 | 0.42 | 0.30, 0.54 | 0.76 | 0.61, 0.92 | 0.39 | 0.16, 0.61 | 0.55 | 0.42, 0.68 | 0.56 | 0.49, 0.63 | 0.003 | 0.75 |
| 3 × 24HR | 0.47 | 0.34, 0.60 | 0.89 | 0.73, 1.05 | 0.71 | 0.57, 0.85 | 0.66 | 0.58, 0.74 | <0.001 | 0.88 | ||||
| FFQ | 0.64 | 0.46, 0.81 | 0.55 | 0.32, 0.79 | 0.63 | 0.46, 0.80 | 0.55 | 0.42, 0.68 | 0.67 | 0.50, 0.84 | 0.61 | 0.53, 0.68 | 0.82 | 0.00 |
Abbreviations: AF, attenuation factor; AMPM, Automated Multiple-Pass Method validation study; CI, confidence interval; FFQ, food frequency questionnaire; 24HR, 24-hour recall; NBS, Nutrition Biomarker Study; NPAAS, Nutrition and Physical Activity Assessment Study; OPEN, Observing Protein and Energy Nutrition.
a For study dates, see Table 1.
b P for heterogeneity across studies.
c Ratio of between-study variance to total variance (a measure of between-study heterogeneity).
d NBS and NPAAS included only women.
e Average weighted by the inverse of the variance.
For potassium density, the FFQ attenuation factors were markedly higher than those for absolute potassium, with an average of over 0.50, and they were higher than those for a single 24HR, which had an average attenuation factor of about 0.40. However, the factor increased to approximately 0.60 for two 24HRs and approximately 0.65 for three 24HRs.
FFQ attenuation factors for sodium were very low (around 0.10), similar to those previously reported for energy (4) (Web Tables 1–8). They were somewhat higher for a single 24HR (around 0.20). For two 24HRs, they increased to approximately 0.25 and to 0.30 for three 24HRs.
FFQ attenuation factors for sodium density were markedly higher than those for absolute sodium, with an average around 0.35 (Table 5), and were higher than those for a single 24HR (average of 0.17 for women and 0.29 for men). With 2 and 3 administrations of a 24HR, the average value increased to about 0.35 and 0.40, respectively.
Attenuation factors for SPR were high. The average value for a single 24HR was approximately 0.40 (0.37 for women and 0.46 for men), rising to about 0.60 for two 24HRs and about 0.70 for three 24HRs. The average attenuation factor for an FFQ was approximately 0.60 (Table 5).
Considerable across-study heterogeneity in attenuation factor values was seen, particularly for the 24HR, with higher values in AMPM and lower values in Energetics (Table 5). Attenuation factors did not differ substantially between men and women (Table 5).
Correlations of reported intake with true intake, averaged over the studies, displayed patterns similar to those of the attenuation factors (Table 6). Correlation coefficients for potassium and potassium density were uniformly higher than those for sodium and sodium density. For FFQs, correlation coefficients for potassium density were higher than those for absolute potassium (average of 0.43 for men and 0.51 for women vs. 0.40 for men and 0.33 for women) and were higher for sodium density than for absolute sodium (average of approximately 0.32 vs. approximately 0.16). For both FFQs and 24HRs, correlations for SPR were higher than those for sodium density and slightly higher than those for potassium density. Correlation coefficients for the individual studies are presented in Web Table 11.
Table 6.
Average Correlation Coefficientsa for Correlations Between Reported and “True” Usual Intakes of Potassium and Sodium, Their Densities, and Their Ratio in 5 Validation Studiesb of Dietary Self-Report Instruments
| Intake Type and Sex | Instrument |
|||
|---|---|---|---|---|
| 1 × 24HR | 2 × 24HR | 3 × 24HR | FFQ | |
| Potassium | ||||
| Male | 0.43 | 0.52 | 0.52 | 0.40 |
| Female | 0.51 | 0.59 | 0.59 | 0.33 |
| Potassium density | ||||
| Male | 0.43 | 0.52 | 0.52 | 0.43 |
| Female | 0.48 | 0.54 | 0.53 | 0.51 |
| Sodium | ||||
| Male | 0.39 | 0.41 | 0.42 | 0.17 |
| Female | 0.24 | 0.28 | 0.39 | 0.15 |
| Sodium density | ||||
| Male | 0.36 | 0.39 | 0.34 | 0.31 |
| Female | 0.26 | 0.34 | 0.41 | 0.33 |
| Sodium:potassium ratio | ||||
| Male | 0.50 | 0.57 | 0.56 | 0.50 |
| Female | 0.43 | 0.55 | 0.64 | 0.49 |
Abbreviations: FFQ, food frequency questionnaire; 24HR, 24-hour recall.
a Adjusted for within-person biomarker variation (see Web Appendix 3 for method). The average is the root mean square of the individual study values.
b The Observing Protein and Energy Nutrition Study (1999–2000), the Energetics Study (2006–2009), the Automated Multiple-Pass Method validation study (2002–2004), the Nutrition Biomarker Study (2004–2005), and the Nutrition and Physical Activity Assessment Study (2007–2009).
Attenuation factors for an FFQ did not appear to be substantially modified by personal characteristics among men (Web Table 12) or for a 24HR in either men or women. However, for FFQs, among women, increased body mass index was associated with lower attenuation factors for potassium and sodium, and black women had lower attenuation factors for potassium than other women. This was similar to our finding regarding modifiers of attenuation for protein and energy (4).
Calibration (prediction) equations for true usual intake
Tables 7 and 8 present details of the calibration models for predicting the logarithm of true usual intakes of potassium and sodium based on either an FFQ report or a single 24HR report and personal characteristics, separately for men and women. The coefficient for the logarithm of self-reported intake is provided for each study. For potassium, race/ethnicity was a strong predictor of intake, and with age and education, it raised the multiple correlation from approximately 0.25–0.35 using a 24HR alone to approximately 0.35–0.40, and from about 0.10–0.20 using an FFQ alone to about 0.20–0.30. Although the information presented in Web Table 12 suggests that certain interaction terms for the interaction of FFQ data with personal characteristics may improve prediction of potassium and sodium intakes, in reality they led to only marginal increases in the average R2—the largest from 0.20 to only 0.22. Therefore, we present results from the simpler calibration model without interactions. Details of the calibration models for potassium density, sodium density, and SPR are presented in Web Tables 13–15. Details of the calibration models for potassium and sodium, when a mean of two 24HRs is used, are presented in Web Table 16.
Table 7.
Calibration Model for Predicting the Logarithm of True Usual Intakes of Potassium and Sodium Among Men in 3 Validation Studies of Dietary Self-Report Instruments
| Instrument and Covariate | Potassium |
Sodium |
||||||
|---|---|---|---|---|---|---|---|---|
| β | P Value | R2a | R2 for Instrumentb | β | P Value | R2a | R2 for Instrumentb | |
| Single 24-Hour Recall | ||||||||
| Studyc | ||||||||
| OPEN Study | 0.307 | 0.26 | 0.21 | 0.214 | 0.40 | 0.16 | ||
| Energetics Study | 0.049 | 0.36 | 0.04 | 0.065 | 0.22 | 0.02 | ||
| AMPM | 0.460 | 0.41 | 0.32 | 0.310 | 0.46 | 0.27 | ||
| P for heterogeneityd | <0.001 | 0.004 | ||||||
| Age group, years (vs. 50–59) | ||||||||
| <40 | −0.088 | 0.01 | 0.011 | 0.24 | ||||
| 40–49 | −0.040 | −0.014 | ||||||
| 60–69 | 0.034 | −0.060 | ||||||
| BMIe (log-transformed) | 0.119 | 0.11 | 0.752 | <0.001 | ||||
| Race/ethnicity (black vs. otherf) | −0.209 | <0.001 | −0.049 | 0.31 | ||||
| Education (vs. college) | ||||||||
| High school | 0.099 | 0.01 | 0.072 | 0.37 | ||||
| Postgraduate | 0.060 | 0.017 | ||||||
| Food Frequency Questionnaire | ||||||||
| Study | ||||||||
| OPEN Study | 0.320 | 0.28 | 0.22 | 0.140 | 0.36 | 0.06 | ||
| Energetics Study | 0.141 | 0.47 | 0.08 | −0.007 | 0.22 | 0.00 | ||
| AMPM | 0.353 | 0.28 | 0.19 | 0.137 | 0.28 | 0.03 | ||
| P for heterogeneity | 0.08 | 0.28 | ||||||
| Age group, years (vs. 50–59) | ||||||||
| <40 | −0.095 | 0.02 | 0.029 | 0.26 | ||||
| 40–49 | −0.058 | −0.020 | ||||||
| 60–69 | 0.014 | −0.055 | ||||||
| BMI (log-transformed) | 0.142 | 0.07 | 0.834 | <0.001 | ||||
| Race/ethnicity (black vs. other) | −0.239 | <0.001 | −0.053 | 0.31 | ||||
| Education (vs. college) | ||||||||
| High school | 0.097 | 0.04 | 0.046 | 0.60 | ||||
| Postgraduate | 0.049 | 0.020 | ||||||
Abbreviations: AMPM, Automated Multiple-Pass Method validation study; BMI, body mass index; OPEN, Observing Protein and Energy Nutrition.
a R2 for model containing the self-report instrument and covariates.
b R2 for model containing the self-report instrument only.
c For study dates, see Table 1.
d P for heterogeneity of adjusted “attenuation coefficient” across studies.
e Weight (kg)/height (m)2.
f “Other” includes non-Hispanic whites.
Table 8.
Calibration Model for Predicting the Logarithm of True Usual Intakes of Potassium and Sodium Among Women in 5 Validation Studies of Dietary Self-Report Instruments
| Instrument and Covariate | Potassium |
Sodium |
||||||
|---|---|---|---|---|---|---|---|---|
| β | P Value | R2a | R2 for Instrumentb | β | P Value | R2a | R2 for Instrumentb | |
| Single 24-Hour Recall | ||||||||
| Studyc | ||||||||
| OPEN Study | 0.293 | 0.30 | 0.26 | 0.077 | 0.23 | 0.03 | ||
| Energetics Study | 0.176 | 0.27 | 0.16 | 0.094 | 0.20 | 0.05 | ||
| AMPM | 0.417 | 0.32 | 0.26 | 0.255 | 0.33 | 0.17 | ||
| NBS | 0.349 | 0.31 | 0.27 | 0.025 | 0.35 | 0.00 | ||
| NPAAS | 0.383 | 0.42 | 0.35 | 0.089 | 0.37 | 0.04 | ||
| P for heterogeneityd | 0.003 | 0.02 | ||||||
| Age group, years (vs. 50–59) | ||||||||
| <40 | −0.103 | 0.11 | 0.090 | <0.001 | ||||
| 40–49 | −0.042 | 0.027 | ||||||
| 60–69 | −0.029 | −0.081 | ||||||
| 70–79 | −0.036 | −0.067 | ||||||
| ≥80 | −0.102 | −0.231 | ||||||
| BMIe (log-transformed) | 0.045 | 0.33 | 0.530 | <0.001 | ||||
| Race/ethnicity (black vs. otherf) | −0.168 | <0.001 | −0.005 | 0.87 | ||||
| Education (vs. college) | ||||||||
| High school | −0.033 | 0.006 | 0.010 | 0.44 | ||||
| Postgraduate | 0.051 | −0.025 | ||||||
| Food Frequency Questionnaire | ||||||||
| Study | ||||||||
| OPEN Study | 0.198 | 0.16 | 0.09 | 0.073 | 0.24 | 0.03 | ||
| Energetics Study | 0.086 | 0.17 | 0.04 | −0.088 | 0.17 | 0.01 | ||
| AMPM | 0.202 | 0.17 | 0.09 | 0.076 | 0.21 | 0.02 | ||
| NBS | 0.295 | 0.26 | 0.19 | 0.116 | 0.36 | 0.04 | ||
| NPAAS | 0.204 | 0.22 | 0.13 | 0.063 | 0.37 | 0.03 | ||
| P for heterogeneity | 0.03 | 0.09 | ||||||
| Age group, years (vs. 50–59) | ||||||||
| <40 | −0.131 | 0.002 | 0.124 | <0.001 | ||||
| 40–49 | −0.034 | 0.034 | ||||||
| 60–69 | −0.002 | −0.062 | ||||||
| 70–79 | −0.024 | −0.081 | ||||||
| ≥80 | −0.115 | −0.196 | ||||||
| BMI (log-transformed) | −0.007 | 0.87 | 0.542 | <0.001 | ||||
| Race/ethnicity (black vs. other) | −0.157 | <0.001 | 0.011 | 0.67 | ||||
| Education (vs. college) | ||||||||
| High school | −0.019 | <0.001 | 0.021 | 0.17 | ||||
| Postgraduate | 0.086 | −0.027 | ||||||
Abbreviations: AMPM, Automated Multiple-Pass Method validation study; BMI, body mass index; NBS, Nutrition Biomarker Study; NPAAS, Nutrition and Physical Activity Assessment Study; OPEN, Observing Protein and Energy Nutrition.
a R2 for model containing the self-report instrument and covariates.
b R2 for model containing the self-report instrument only.
c For study dates, see Table 1.
d P for heterogeneity of adjusted “attenuation coefficient” across studies.
e Weight (kg)/height (m)2.
f “Other” includes non-Hispanic whites.
For sodium, body mass index was the most important predictor, followed by age. The very low multiple correlations due to self-report alone (less than 0.1, except for AMPM) were raised to 0.2–0.4 by including these factors.
DISCUSSION
Dietary self-reporting is a basic component of population surveillance of dietary intake, many dietary intervention studies, and most studies of diet–health outcome relationships. However, reporting error, daily variation in diet, and limitations of food composition databases can affect results, and knowing the measurement properties of self-report instruments is required for their proper interpretation. In this paper, we examined the measurement properties of self-reported intakes of potassium and sodium, their densities, and SPR.
While potassium intake is accepted as a valid target of dietary self-reports, the possibility of capturing sodium intake by self-report is more controversial. Although much of the sodium in the US diet comes from packaged and processed foods, the sodium in these foods is present in highly variable amounts, creating difficulties in compiling accurate food composition databases, especially for FFQs. Furthermore, sodium intakes from salt added during food preparation or at the table are difficult to capture. These difficulties explain the move to use urinary sodium levels (and not self-reports) in some surveillance efforts (29). In the studies in our pooling project, efforts to quantify salt added at the table were limited. However, the majority of US sodium intake is derived from food processed or prepared outside the home (30, 31).
We indeed found differences between the errors of self-reported sodium and potassium intakes. Firstly, unlike potassium, sodium intake was underestimated, by approximately 30% using FFQs and by about 5%–10% using 24HRs (Table 4). Secondly, sodium intake had lower attenuation factors (Table 5) and lower correlations with true intake (Table 6) than potassium intake. Nevertheless, SPR (together with potassium density) had the highest attenuation factors and correlations among the components we investigated. This was true for both 24HRs and FFQs, even though FFQs seriously underestimated SPR. This indicates that while FFQs may be a poor tool for population surveillance of SPR, it nevertheless appears adequate for assessing associations of this ratio with health outcomes. Note that none of the FFQs in our study were specifically designed to assess sodium intake, and with some attention to this issue, improved measurement could likely be achieved.
The higher attenuation factors and correlations with true intake for SPR apparently resulted from the highly correlated reporting errors for sodium and potassium intakes (FFQs, 0.72–0.88; 24HRs, 0.44–0.74). This high correlation caused the reporting errors to partially cancel each other out when the ratio was taken.
Recently, Ioannidis (32) cited Archer et al.'s (33) criticism that National Health and Nutrition Examination Survey data underreport energy intake as the starting point for an attack on observational studies of diet–health outcome relationships. Our finding of serious underestimation of SPR by FFQs, coupled with relatively high correlations between the FFQ-reported ratio and the true ratio, emphasizes that underestimation per se does not invalidate use of a self-report instrument for studying diet–health outcome relationships.
The results presented in this paper were based on the assumptions that doubly labeled water and 24-hour urinary potassium and sodium measurements yield unbiased estimates of their respective intakes and that their errors are not related to true intake or personal characteristics. This assumption is well-founded for doubly labeled water (34) and urinary potassium (35), but it is somewhat controversial for urinary sodium.
Several studies conducted between 1970 and 1990 indicated that urinary sodium was suitable as a recovery biomarker. One feeding study (18) and 4 studies with duplicate food portions (17, 20, 36, 37) compared dietary sodium with corresponding 24-hour urine samples. The data collection periods ranged from 3 days to 28 days over a 12-month period and included 9–43 participants. The average recovery of sodium from 24-hour urine samples ranged from 83% to 95%, similar to the 81% recovery found for protein as estimated from urinary nitrogen (38). In the feeding study, Luft et al. (18) reported a correlation of 0.55 between 24-hour urine sodium and true intake, with improved agreement when multiple repeated 24-hour urine samples were used. More recently, an intervention study (39) found within-person correlations for 24-hour urine assessments separated by several months of only 0.30 for sodium, compared with 0.50 for potassium. A recent experiment in 12 adults, involving constant sodium intake over several months, revealed cyclical rhythms in urinary sodium excretion with phases longer than 24 hours (40). It seems plausible that such variations occur randomly among study subjects and are independent of level of sodium intake and personal characteristics, thus complying with the assumptions for a recovery biomarker.
FFQs assess intake over the moderate- to long-term past, whereas biomarkers and 24HRs assess short-term intake. Therefore, the biomarker assessments in our study were more proximal to the period assessed by the 24HRs than to the period assessed by the FFQs. This could have caused overestimation of 24HR correlations with true long-term intake and underestimation of FFQ correlations. In preliminary investigations using statistical modeling, we found that this does occur, but not to a degree that changes our overall conclusions. Further statistical examination of this issue is needed. Note that both biomarker and self-report assessments cover a much shorter period than may be relevant to dietary influences on health outcomes, suggesting the need for both longitudinal biomarker data and longitudinal self-report data in order to thoroughly study diet–health outcome associations.
As we mentioned previously (4), to avoid excessive attenuation of the estimated relative risk and the requirement of huge sample sizes, one usually needs correlations between reported and true intakes of 0.4 or more. Our finding of correlations of approximately 0.5 for SPR, a potential risk factor for cardiovascular disease, using either an FFQ or 1 or more 24HRs (Table 6) is encouraging. Similarly high correlations were found for potassium density (Table 6).
The ratio of sodium intake to potassium intake appears to be strongly related to personal characteristics, with higher ratios being reported among younger persons, blacks, persons with higher body mass index, and persons with less than a postgraduate education. These factors, together with self-reported intake, allow prediction of true intake with a multiple correlation coefficient of approximately 0.4 (Web Table 15). Huang et al. (41) developed calibration models for potassium and sodium and their ratio, as well as for potassium and sodium density, based on 24HRs, FFQs, and 4-day food records using NBS and NPAAS data; their results substantially agree with ours. These authors have proposed use of their calibration models in cohort studies of postmenopausal women to obtain estimates of relative risks related to SPR. The calibration models derived from the models presented in our tables could serve the same purpose. However, certain caveats must be mentioned.
Firstly, we found heterogeneity of the coefficients for the self-reports across studies, making it sometimes difficult to choose the appropriate coefficient for the study population in question. In situations where the heterogeneity test results (shown in the tables) were not statistically significant, it would be reasonable to use a coefficient that was averaged across the studies. In situations where heterogeneity was observed, investigators would be advised to use the coefficient from the study (or studies) that corresponds most closely to their own in terms of the instrument used and the population studied. Secondly, and more fundamentally, the choice of which variables to use in the prediction equation is complex and is closely tied to the time period targeted for usual intake. More discussion of this point can be found in the article by Prentice et al. (42) but is beyond the scope of this paper. It is clear, however, that the principle of increasing the accuracy of prediction of usual intake is centrally important in nutritional epidemiology.
Other investigators have noted that nutrients are reported with differing levels of error (e.g., see Heitmann et al. (43)). Our results for sodium and potassium (Table 4), as well previous results on energy and protein (4), clearly demonstrate this. Unfortunately, this means that there is no single correction factor that can be applied to adjust estimated population intake distributions for reporting errors.
Overall, our pooling study has clarified strengths and weaknesses of 2 commonly used types of self-report instruments for assessing intakes of potassium and sodium. The modes of administration of the instruments, particularly the 24HR, varied substantially across studies, from the World Wide Web to interviewers. Similarly, the populations differed quite widely in age and racial/ethnic composition. These factors no doubt contributed to between-study heterogeneity in some measures. Despite this heterogeneity, the current study established firmly that 1) the attenuations and correlations with truth for the FFQs studied are much improved when considering potassium or sodium densities in comparison with absolute potassium or sodium; 2) attenuations and correlations of SPR are (together with those for potassium density) the highest among the dietary components we have investigated; 3) multiple 24HRs substantially decrease attenuation and increase correlation compared with a single 24HR; and 4) body mass index strongly predicts underreporting of sodium intake. Overall, our analyses of potassium and sodium, described here, and the results for protein and energy reported previously (4) support the view that levels of dietary reporting error differ across nutrients.
Supplementary Material
ACKNOWLEDGMENTS
Author affiliations: Information Management Systems, Inc., Rockville, Maryland (Laurence S. Freedman, John M. Commins, James E. Moler); Biostatistics Unit, Gertner Institute for Epidemiology and Health Policy Research, Tel Hashomer, Israel (Laurence S. Freedman); Division of General Internal Medicine, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, California (Lenore Arab); Agricultural Research Service, Beltsville Human Nutrition Research Center, US Department of Agriculture, Beltsville, Maryland (Alanna J. Moshfegh, Donna Rhodes); Biometry Research Group, Division of Cancer Prevention, National Cancer Institute, Bethesda, Maryland (Victor Kipnis); Division of Public Health Sciences, Fred Hutchinson Cancer Research Center, Seattle, Washington (Marian L. Neuhouser, Ross L. Prentice, Lesley F. Tinker); Division of Cancer Control and Population Sciences, National Cancer Institute, Bethesda, Maryland (Nancy Potischman, Amy F. Subar); Departments of Epidemiology and Nutrition, Harvard School of Public Health, Boston, Massachusetts (Donna Spiegelman, Walter Willett); Department of Biostatistics, Harvard School of Public Health, Boston, Massachusetts (Donna Spiegelman); Channing Division of Network Medicine, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts (Walter Willett).
The Women's Health Initiative (WHI) was funded by the National Heart, Lung, and Blood Institute through contracts HHSN268201100046C, HHSN268201100001C, HHSN268201100002C, HHSN268201100003C, HHSN268201100004C, and HHSN271201100004C.
The authors thank the following persons for their valuable contributions to this work: Dr. Alfonso Ang (University of California, Los Angeles), Dr. David Baer (US Department of Agriculture), Dr. Sheila Bingham (deceased; Dunn Nutrition Unit, Medical Research Council, Cambridge, United Kingdom), Dr. Kevin Dodd (National Cancer Institute (NCI)), Douglas Midthune (NCI), Dr. Dale Schoeller (University of Wisconsin), and Dr. Richard Troiano (NCI).
The authors thank the WHI investigators and staff for their dedication. A full listing of WHI investigators can be found at http://www.whi.org/researchers/Documents%20%20Write%20a%20Paper/WHI%20Investigator%20Short%20List.pdf.
Dr. Lenore Arab has a proprietary interest in DietDay (Centrax Corporation, Chicago, Illinois), the Web-based 24-hour dietary recall instrument used in the Energetics Study. None of the other authors declared any conflicts of interest.
REFERENCES
- 1.Willett WC. Nutritional Epidemiology. 3rd ed New York, NY: Oxford University Press; 2013. [Google Scholar]
- 2.Kaaks R, Ferrari P, Ciampi A, et al. Uses and limitations of statistical accounting for random error correlations, in the validation of dietary questionnaire assessments. Public Health Nutr. 2002;5(6A):969–976. [DOI] [PubMed] [Google Scholar]
- 3.Subar AF, Kipnis V, Troiano RP, et al. Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN study. Am J Epidemiol. 2003;1581:1–13. [DOI] [PubMed] [Google Scholar]
- 4.Freedman LS, Commins JM, Moler JE, et al. Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for energy and protein intake. Am J Epidemiol. 2014;1802:172–188. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Intersalt Cooperative Research Group. Intersalt: an international study of electrolyte excretion and blood pressure. Results for 24 hour urinary sodium and potassium excretion. BMJ. 1988;2976644:319–328. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Aburto NJ, Ziolkovska A, Hooper L, et al. Effect of lower sodium intake on health: systematic review and meta-analyses. BMJ. 2013;346:f1326. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Whelton PK, He J, Cutler JA, et al. Effects of oral potassium on blood pressure. Meta-analysis of randomized controlled clinical trials. JAMA. 1997;27720:1624–1632. [DOI] [PubMed] [Google Scholar]
- 8.Aburto NJ, Hanson S, Gutierrez H, et al. Effect of increased potassium intake on cardiovascular risk factors and disease: systematic review and meta-analyses. BMJ. 2013;346:f1378. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Arab L, Wesseling-Perry K, Jardack P, et al. Eight self-administered 24-hour dietary recalls using the Internet are feasible in African Americans and whites: the Energetics Study. J Am Diet Assoc. 2010;1106:857–864. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Moshfegh AJ, Rhodes DG, Baer DJ, et al. The US Department of Agriculture Automated Multiple-Pass Method reduces bias in the collection of energy intakes. Am J Clin Nutr. 2008;882:324–332. [DOI] [PubMed] [Google Scholar]
- 11.Neuhouser ML, Tinker L, Shaw PA, et al. Use of recovery biomarkers to calibrate nutrient consumption self-reports in the Women's Health Initiative. Am J Epidemiol. 2008;16710:1247–1259. [DOI] [PubMed] [Google Scholar]
- 12.Prentice RL, Mossavar-Rahmani Y, Huang Y, et al. Evaluation and comparison of food records, recalls, and frequencies for energy and protein assessment by using recovery biomarkers. Am J Epidemiol. 2011;1745:591–603. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Subar AF, Thompson FE, Kipnis V, et al. Comparative validation of the Block, Willett, and National Cancer Institute food frequency questionnaires: the Eating at America's Table Study. Am J Epidemiol. 2001;15412:1089–1099. [DOI] [PubMed] [Google Scholar]
- 14.Willett WC, Sampson L, Stampfer MJ, et al. Reproducibility and validity of a semiquantitative food frequency questionnaire. Am J Epidemiol. 1985;1221:51–65. [DOI] [PubMed] [Google Scholar]
- 15.Patterson RE, Kristal AR, Tinker LF, et al. Measurement characteristics of the Women's Health Initiative food frequency questionnaire. Ann Epidemiol. 1999;93:178–187. [DOI] [PubMed] [Google Scholar]
- 16.Schoeller DA, Hnilicka JM. Reliability of the doubly labeled water method for the measurement of total daily energy expenditure in free-living subjects. J Nutr. 1996;126(1 suppl):348S–354S. [PubMed] [Google Scholar]
- 17.Mickelsen O, Makdani D, Gill JL, et al. Sodium and potassium intakes and excretions of normal men consuming sodium chloride or a 1:1 mixture of sodium and potassium chlorides. Am J Clin Nutr. 1977;3012:2033–2040. [DOI] [PubMed] [Google Scholar]
- 18.Luft FC, Fineberg NS, Sloan RS. Estimating dietary sodium intake in individuals receiving a randomly fluctuating intake. Hypertension. 1982;46:805–808. [DOI] [PubMed] [Google Scholar]
- 19.Freedman LS, Midthune D, Carroll RJ, et al. Adjustments to improve the estimation of usual dietary intake distributions in the population. J Nutr. 2004;1347:1836–1843. [DOI] [PubMed] [Google Scholar]
- 20.Holbrook JT, Patterson KY, Bodner JE, et al. Sodium and potassium intake and balance in adults consuming self-selected diets. Am J Clin Nutr. 1984;404:786–793. [DOI] [PubMed] [Google Scholar]
- 21.Kipnis V, Subar AF, Midthune D, et al. Structure of dietary measurement error: results of the OPEN biomarker study. Am J Epidemiol. 2003;1581:14–21. [DOI] [PubMed] [Google Scholar]
- 22.Cook NR, Obarzanek E, Cutler JA, et al. Joint effects of sodium and potassium intake on subsequent cardiovascular disease: the Trials of Hypertension Prevention follow-up study. Trials of Hypertension Prevention Collaborative Research Group. Arch Intern Med. 2009;1691:32–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Subar AF, Midthune D, Tasevska N, et al. Checking for completeness of 24-h urine collection using para-amino benzoic acid not necessary in the Observing Protein and Energy Nutrition Study. Eur J Clin Nutr. 2013;678:863–867. [DOI] [PubMed] [Google Scholar]
- 24.Molenberghs G, Verbeke G. Linear Mixed Models for Longitudinal Data. New York, NY: Springer-Verlag; 2000. [Google Scholar]
- 25.Kipnis V, Izmirlian G. The impact of categorization of continuous exposure measured with error [abstract] Am J Epidemiol. 2002;155:S28. [Google Scholar]
- 26.Rosner B, Willett WC. Interval estimates for correlation coefficients corrected for within-person variation: implications for study design and hypothesis testing. Am J Epidemiol. 1988;1272:377–386. [DOI] [PubMed] [Google Scholar]
- 27.Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;2111:1539–1558. [DOI] [PubMed] [Google Scholar]
- 28.SAS Institute Inc. Statistical Analysis System (SAS) software, version 9.2. Cary, NC: SAS Institute Inc.; 2009. [Google Scholar]
- 29.American Public Health Association. Implementing Effective Strategies to Reduce Sodium in the Food Supply. (Policy Statement no. 201120) Washington, DC: American Public Health Association; 2011. http://www.apha.org/policies-and-advocacy/public-health-policy-statements/policy-database/2014/07/21/11/36/implementing-effective-strategies-to-reduce-sodium-in-the-food-supply. Accessed October 19, 2014. [Google Scholar]
- 30.Ferguson MH, Kay LA. Sodium concentration in drinking water and other fluids and its significance in restricted sodium intake. Can Med Assoc J. 1953;695:491–493. [PMC free article] [PubMed] [Google Scholar]
- 31.Centers for Disease Control and Prevention. Trends in the prevalence of excess dietary sodium intake—United States, 2003–2010. MMWR Morb Mortal Wkly Rep. 2013;6250:1021–1025. [PMC free article] [PubMed] [Google Scholar]
- 32.Ioannidis JP. Implausible results in human nutrition research. BMJ. 2013;347:f6698. [DOI] [PubMed] [Google Scholar]
- 33.Archer E, Hand GA, Blair SN. Validity of U.S. nutritional surveillance: National Health and Nutrition Examination Survey caloric energy intake data, 1971–2010. PLoS One. 2013;810:e76632. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Schoeller DA. Measurement of energy expenditure in free-living humans by using doubly labeled water. J Nutr. 1988;11811:1278–1289. [DOI] [PubMed] [Google Scholar]
- 35.Tasevska N, Runswick SA, Bingham SA. Urinary potassium is as reliable as urinary nitrogen for use as a recovery biomarker in dietary studies of free living individuals. J Nutr. 2006;1365:1334–1340. [DOI] [PubMed] [Google Scholar]
- 36.Schachter J, Harper PH, Radin ME, et al. Comparison of sodium and potassium intake with excretion. Hypertension. 1980;25:695–699. [DOI] [PubMed] [Google Scholar]
- 37.Clark AJ, Mossholder S. Sodium and potassium intake measurements: dietary methodology problems. Am J Clin Nutr. 1986;433:470–476. [DOI] [PubMed] [Google Scholar]
- 38.Bingham SA, Cummings JH. Urine nitrogen as an independent validatory measure of dietary intake: a study of nitrogen balance in individuals consuming their normal diet. Am J Clin Nutr. 1985;426:1276–1289. [DOI] [PubMed] [Google Scholar]
- 39.Espeland MA, Kumanyika S, Wilson AC, et al. Statistical issues in analyzing 24-hour dietary recall and 24-hour urine collection data for sodium and potassium intakes. Am J Epidemiol. 2001;15310:996–1006. [DOI] [PubMed] [Google Scholar]
- 40.Rakova N, Jüttner K, Dahlmann A, et al. Long-term space flight simulation reveals infradian rhythmicity in human Na+ balance. Cell Metab. 2013;171:125–131. [DOI] [PubMed] [Google Scholar]
- 41.Huang Y, Van Horn L, Tinker LF, et al. Measurement error corrected sodium and potassium intake estimation using 24-hour urinary excretion. Hypertension. 2014;632:238–244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Prentice RL, Pettinger M, Tinker LF, et al. Regression calibration in nutritional epidemiology: example of fat density and total energy in relationship to postmenopausal breast cancer. Am J Epidemiol. 2013;17811:1663–1672. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Heitmann BL, Lissner L, Osler M. Do we eat less fat, or just report so? Int J Obes Relat Metab Disord. 2000;244:435–442. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.