Abstract
Background
Food items on a self-administered food frequency questionnaire (FFQ) may be left blank because the food was not consumed, because of difficulties remembering the frequency or amount of intake, or due to an oversight.
Methods
We explored the predictors and frequency of consumption of omitted food items on an FFQ used in the Nurses’ Health Study II. Of 87,676 women who returned a mailed 147-item FFQ in 1999, 34% completed the entire questionnaire, whereas 66% left at least 1 food item blank. Ten or more foods were omitted by 5% of participants. Foods were more likely omitted by women who were older, more physically active, and had more children. We resurveyed 2876 participants who had left between 1 and 70 food items blank and asked them to fill in the blanks. Overall, 2485 participants provided complete responses.
Results
In the resurvey, 64% of the formerly omitted foods were marked as consumed never or less than once per month, 20% as 1–3 times per month, 8% as once per week, and 9% as more than once per week. Commonly consumed foods and beverages were less likely omitted because they were not consumed than rarely consumed foods. The best estimate for the true intake value of an omitted food was 0.82 times the average population intake.
Conclusions
When calculating nutrient intake, the assumption that items missing represent zero intake is reasonable. However, foods consumed more often in the population at large are less likely than rarely consumed foods to be left blank because they were not consumed.
The food frequency questionnaire (FFQ) is the most commonly used dietary assessment instrument in epidemiologic research.1-6 Data collected with an FFQ are used to calculate nutrient intake and to relate consumption of foods or nutrients to specific disease outcomes.
Most FFQs include between 50 and 200 food items.7 Like any structured self-administered questionnaire with prespecified response options, the FFQ provides the opportunity for nonresponse. Missing values can affect the calculation of nutrient intake and the estimation of the diet-disease relation. Few studies have investigated the distribution of blanks on an FFQ. In the Kaiser Permanente Diet Survey, a 60-item FFQ was administered; 41% of participants completed all questions and 12% left more than 10 items blank.8 In the Iowa Women's Health Study, 11% of the women left 10 or more items blank on a 126-item FFQ. 8,9 On the FFQ used in the New York University Women's Health Study, 3% of participants were missing 10 or more items.10 In the Norwegian Women and Cancer study, a 136-item FFQ had 18% missing values,11 and in the GISSI-Prevenzione study in Italy nonresponse increased from 0.3% in a first application of a diet questionnaire to 10% 12 months later and 20% 36 months after the first application.12
Nonresponse may differentially affect certain foods, and individuals may differ in their likelihood of omitting food items depending on personal characteristics. Food items on an FFQ may be omitted because the food was not consumed or because of difficulties remembering the frequency and amount of intake, especially because the FFQ does not provide a “Don't remember” option. In dietary analyses it is typically assumed that a blank on the questionnaire really means that the food was not consumed, hence intake is set to zero.7 If the assumptions are misspecified, intake will be misclassified.
We used data from the Nurses’ Health Study II (NHS II) to examine patterns and predictors of partial nonresponse on an FFQ. We resurveyed NHS II participants who omitted food items to verify the assumption of zero intake.
Methods
The Nurses’ Health Study II
In 1989, we recruited 116,686 female nurses (25–42 years of age) from 14 U.S. states who were free of diagnosed cancer to address relations between lifestyle factors and chronic disease outcomes. Cohort members have been followed by means of biennial questionnaires. Starting in 1991, the questionnaires have included an FFQ component every 4 years. For the analyses in this paper, we used the 1999 FFQ, which was the most recent questionnaire at the time the study was conducted in 2002. Of our participants, 87,676 provided responses to at least 1 item on the 1999 FFQ; these represent the diet cohort, which was the population base for this analysis.
This study was approved by the Institutional Review Boards of the Brigham and Women's Hospital, Boston, MA, and the Harvard School of Public Health, Boston, MA.
The Semiquantitative Food Frequency Questionnaire
In the NHS II we administer the semiquantitative FFQ developed by Willett et al.1 For each food, a commonly used unit or portion size (eg, 1 egg or 1 slice of bread) is specified, and the women are asked how often on average over the previous year they consumed that amount of each food. There are 9 possible responses, ranging from “never” to “6 or more times per day.” The intake of nutrients is computed by multiplying the frequency of consumption of each unit of food by the nutrient content of the specified portions.
The FFQ administered to the NHS II participants in 1999 included 147 foods and beverages. Among the items were 3 foods with multiple subtypes: 1) Milk: skim or 1% milk; 2% milk; whole milk; soy milk. 2) Cookies: fat-free or reduced fat; other ready-made; home-baked. 3) Sweet roll, coffee cake, or other pastry: fat-free or reduced fat; other ready-made; home-baked. The usual structure of these questions may have affected their missingness pattern. We have therefore given special consideration to them, and excluded them from some statistics.
For nutrient calculations, women are excluded if more than 70 food or beverage items are missing, or if their total caloric intake is below 600 or above 3500 kcal; in 1999, 2554 women were excluded for these reasons.
Resurvey
We resurveyed 2876 NHS II participants who had left blank between 1 and 70 of the 147 food and beverage items of the 1999 FFQ. We restricted our source population from which resurvey participants were selected to 35,947 women who were not enrolled in any other substudy at the time, and who responded to the initial mailing of our biannual questionnaire in 1999 and 2001. We oversampled NHS II participants with a higher proportion of missing values: we selected a random sample of 200 participants each with 1–9 missing foods (n = 1800) and all remaining participants with 10–70 blanks (n = 1076). Table 1 provides a comparison of selected non-dietary and dietary variables across participants of NHS II who completed the 1999 FFQ, among the source population from which we sampled and among the survey sample selected. The sample selected for resurvey was comparable to the source population as well as to the entire NHS II cohort.
Table 1.
Distribution of Non-dietary and Dietary Variables Among Participants of the NHS II Cohort Who Completed the 1999 FFQ, the Source Population from which the Resurveyed Women Were Sampled, and the Resurveyed Women Included in these Analyses
| NHS II Participants with 1999 FFQ | Women in Source Populationa | Resurveyed Womenb | ||||
|---|---|---|---|---|---|---|
| No. of Participants | Mean (SE) | No. of Participants | Mean (SE) | No. of Participants | Mean (SE) | |
| Age (y) | 87,605 | 44.7 (0.02) | 35,935 | 45.2 (0.02) | 2484 | 45.8 (0.09) |
| BMI (kg/m2) | 83,657 | 26.6 (0.02) | 34,579 | 26.1 (0.03) | 2367 | 26.1 (0.12) |
| Activity (METS/wk) | 80,695 | 18.5 (0.08) | 34,769 | 18.8 (0.12) | 2380 | 19.1 (0.46) |
| Smoking (pack-years) | 87,009 | 4.89 (0.03) | 35,747 | 4.76 (0.05) | 2462 | 4.98 (0.19) |
| Caloric intake (kcal/d) | 85,122 | 1822 (1.93) | 35,145 | 1804 (2.94) | 2414 | 1811 (11.5) |
| Carbohydrates (g/d) | 85,122 | 237 (0.30) | 35,145 | 236 (0.45) | 2414 | 234 (1.71) |
| Fat (g/d) | 85,122 | 62.7 (0.08) | 35,145 | 61.5 (0.13) | 2414 | 62.3 (0.51) |
| Alcohol (g/d) | 85,122 | 3.94 (0.02) | 35,145 | 4.12 (0.04) | 2414 | 4.43 (0.16) |
| Fruit (servings/d)c | 78,120 | 1.87 (0.005) | 32,446 | 1.86 (0.008) | 1804 | 1.90 (0.03) |
| Vegetables (servings/d)c | 76,939 | 3.66 (0.008) | 31,972 | 3.64 (0.01) | 1747 | 3.84 (0.06) |
| Orange juice (servings/d) | 83,840 | 0.42 (0.002) | 34,586 | 0.41 (0.003) | 2163 | 0.41 (0.01) |
| Carrots (servings/d) | 85,443 | 0.41 (0.002) | 35,184 | 0.42 (0.003) | 2325 | 0.44 (0.01) |
| Red meat (servings/d) | 79,645 | 1.31 (0.003) | 32,976 | 1.26 (0.004) | 1935 | 1.31 (0.02) |
| Potato chips (servings/d) | 86,854 | 0.15 (0.0007) | 35,719 | 0.15 (0.001) | 2433 | 0.14 (0.004) |
Source Population: Subset of 35,947 NHSII participants from which women were sampled for resurvey.
Resurveyed women: Sample of 2876 women drawn from the source population; a random sample of 200 participants; each with 1–9 missing foods; and all remaining participants with 10–70 blanks.
Fruit and vegetable intakes were calculated among women with no fruit or vegetable item missing
Means are based on reported information; women with missing information for the specific variable were excluded.
SE indicates standard error.
In January 2002, we mailed the participants selected for the resurvey a copy of their 1999 FFQ with the omitted foods and beverages highlighted. Participants were asked to fill in the frequency of consumption of the items they had omitted and, if they did not remember the exact value, to provide their best guess. Of the 2876 women invited, 2485 (86%) provided complete responses for the previously missed foods. The 391 women who did not provide complete information or did not respond to our invitation were excluded from further analyses.
Statistical Analysis
Frequency analyses were used to present distributions. The mean number of foods omitted and respective standard errors were calculated.
Concordance correlation coefficients13 were calculated to compare nutrient intake before and after the resurvey. We plotted the average number of servings per day for each food or beverage reported in the initial survey in 1999 against those initially omitted in 1999 and provided in the resurvey and obtained the slope for the fitted line by using least squares linear regression.
Results
The proportion of omitted food items among the total population of 87,676 Nurses’ Health Study II participants who answered the 1999 FFQ was 2.3%. On average, participants omitted 3.3 food items of the 147-item questionnaire (median: 1 food item). A total of 29,528 women (34%) completed all questions on the FFQ (Table 2). Overall, 95% omitted fewer than 10 foods and 98% omitted 20 or fewer foods.
Table 2.
Distribution of Omitted Food Items on the 1999 FFQ in the Nurses’ Health Study II
| No. of Omitted Food Items | No. of Participants | Proportion of Participants | Cumulative Proportion of Participants |
|---|---|---|---|
| 0 | 29,528 | 33.7 | 33.7 |
| 1 | 16,363 | 18.7 | 52.3 |
| 2 | 9184 | 10.5 | 62.8 |
| 3 | 10,718 | 12.2 | 75.0 |
| 4 | 6986 | 8.0 | 83.0 |
| 5 | 4286 | 4.9 | 87.9 |
| 6 | 2649 | 3.0 | 90.9 |
| 7 | 1859 | 2.1 | 93.0 |
| 8 | 1251 | 1.4 | 94.5 |
| 9 | 873 | 1.0 | 95.5 |
| 10–19 | 2474 | 2.8 | 98.3 |
| 20–29 | 335 | 0.4 | 98.7 |
| 30–39 | 203 | 0.2 | 98.9 |
| 40–49 | 114 | 0.1 | 99.0 |
| 50–59 | 107 | 0.1 | 99.2 |
| 60–69 | 81 | 0.1 | 99.2 |
| 70–79 | 65 | 0.1 | 99.3 |
| 80–89 | 100 | 0.1 | 99.4 |
| 90–99 | 143 | 0.2 | 99.6 |
| 100+ | 357 | 0.4 | 100.0 |
| Total | 87,676 | 100.0 | 100.0 |
The caloric content of a food item was not associated with the likelihood of leaving the food item blank (data not shown). In Table 3 we present data on mean frequency of consumption of selected foods and mean intake of nutrients frequently studied in nutritional epidemiology. The mean frequency of food consumption did not appreciably differ between study participants who omitted 1 or more food items (25.4 servings per day) and women who did not leave any blanks on the FFQ (24.7 servings per day). Frequency of reported consumption of fruits, vegetables, and meat as a main dish did not vary appreciably with up to 40 missing items, but lower intakes were reported by the 1% of the population with 40 or more blank food items. With 21 and more missing food items, there was a declining trend in calculated intake of calories and macronutrients. Macronutrient intake calculated as percentage of energy was not appreciably affected by missing values.
Table 3.
Mean Frequency of Consumption of Selected Foods and Nutrients Among NHS II Participants with and without Missing Food Items on the 1999 FFQ
| Food/Nutrient | No. of Blanks |
||||||
|---|---|---|---|---|---|---|---|
| Total Cohort MFC (SE) (servings/d)a (n = 87,676) | None MFC (SE) (servings/d)a (n = 29,528) | 1–5 MFC (SE) (servings/d)a (n = 47,537) | 6–10 MFC (SE) (servings/d)a (n = 7240) | 11–20 MFC (SE) (servings/d)a (n = 1921) | 21–40 MFC (SE) (servings/d)a (n = 495) | ≥41 MFC (SE) (servings/d)a (n = 955) | |
| Fruit | 1.89 (0.005) | 1.81 (0.008) | 1.92 (0.007) | 2.07 (0.02) | 2.11 (0.04) | 2.03 (0.08) | 1.27 (0.05) |
| Vegetables | 3.68 (0.008) | 3.49 (0.01) | 3.76 (0.01) | 4.05 (0.03) | 4.05 (0.06) | 3.76 (0.13) | 2.01 (0.07) |
| Orange Juice | 0.42 (0.002) | 0.41 (0.004) | 0.43 (0.003) | 0.46 (0.008) | 0.44 (0.01) | 0.42 (0.04) | 0.28 (0.02) |
| Carrots | 0.41 (0.002) | 0.39 (0.003) | 0.42 (0.003) | 0.45 (0.007) | 0.44 (0.01) | 0.37 (0.02) | 0.21 (0.01) |
| Potato chips | 0.15 (0.0007) | 0.15 (0.001) | 0.15 (0.001) | 0.15 (0.003) | 0.13 (0.005) | 0.11 (0.007) | 0.08 (0.006) |
| Meat as a main dish | 1.30 (0.003) | 1.29 (0.005) | 1.31 (0.004) | 1.34 (0.01) | 1.32 (0.02) | 1.35 (0.05) | 0.77 (0.03) |
| Mean Intake/d (SE) (n = 85,122)b | Mean Intake/d (SE) (n = 28,917)b | Mean Intake/d (SE) (n = 46,405)b | Mean Intake/d (SE) (n = 7012)b | Mean Intake/d (SE) (n = 1868)b | Mean Intake/d (SE) (n = 411)b | Mean Intake/d (SE) (n = 509)b | |
|---|---|---|---|---|---|---|---|
| Calories | 1822 (1.9) | 1806 (3.3) | 1830 (2.6) | 1858 (6.7) | 1821 (13.3) | 1733 (28.8) | 1543 (24.9) |
| Carbohydrates (g) | 237 (0.3) | 236 (0.5) | 238 (0.4) | 242 (1.0) | 236 (2.0) | 221 (4.3) | 199 (3.6) |
| Percent of energy from carbohydrates | 51.9 (0.03) | 51.9 (0.05) | 51.9 (0.04) | 51.9 (0.10) | 51.8 (0.22) | 50.6 (0.45) | 51.6 (0.42) |
| Total fat (g) | 62.7 (0.08) | 62.2 (0.14) | 62.9 (0.11) | 64.0 (0.30) | 62.8 (0.60) | 60.4 (1.24) | 53.4 (1.09) |
| Percent of energy from fat | 30.9 (0.02) | 31.0 (0.04) | 30.9 (0.03) | 30.9 (0.08) | 31.0 (0.17) | 31.5 (0.38) | 31.1 (0.34) |
| Alcohol (g) | 3.94 (0.02) | 3.81 (0.04) | 3.98 (0.03) | 4.19 (0.09) | 3.89 (0.16) | 4.15 (0.40) | 3.11 (0.31) |
Foods left blank were set to zero to be consistent with nutrient calculation.
Nutrient intake calculated for women with less than 70 missing foods and more than 600 and fewer than 3500 calories of intake.
MFC indicates mean frequency of consumption; SE indicates standard error.
We also explored whether personal characteristics might predict the probability of leaving food items blank (Table 4). The number of food items omitted on the FFQ increased with age. Similarly, women who had 4 or more children and women who were very physically active had a higher prevalence of missing values than women with no or fewer children and women who were less physically active. Body mass index, smoking status, living arrangements, and alcohol consumption were not associated with the likelihood of leaving blanks. Women who did not report their regular alcohol consumption left substantially more items on the FFQ blank than women who did report their alcohol consumption.
Table 4.
Predictors of the Number of Food Items Omitted on the 1999 FFQ Among 87,676 Participants of the NHS IIa
| Characteristic in 1999b | No. of Women | Mean No. of Foods Omitted (SE) |
|---|---|---|
| Age (y) | ||
| <40 | 15,921 | 2.7 (0.07) |
| 40–44 | 28,327 | 3.1 (0.06) |
| 45–49 | 28,961 | 3.7 (0.06) |
| 50+ | 14,396 | 3.8 (0.09) |
| Missing | 71 | 7.8 (2.6) |
| BMI | ||
| <21 | 11,746 | 3.3 (0.09) |
| 21.0–24. | 30,426 | 3.2 (0.05) |
| 25–29.9 | 22,315 | 3.2 (0.06) |
| 30+ | 19,170 | 3.2 (0.07) |
| Missing | 4019 | 5.2 (0.23) |
| Parity | ||
| Nulliparous | 16,582 | 3.1 (0.07) |
| Parous 1 child | 12,507 | 3.3 (0.09) |
| 2 | 33,391 | 3.4 (0.05) |
| 3 | 16,953 | 3.3 (0.07) |
| 4+ | 5959 | 3.3 (0.12) |
| Missing | 2284 | 5.2 (0.33) |
| Living arrangements | ||
| Married | 20,432 | 3.3 (0.07) |
| Married with child(ren) | 41,033 | 2.9 (0.04) |
| Divorced with child(ren) | 4200 | 3.2 (0.15) |
| Divorced | 2538 | 3.4 (0.20) |
| Widowed with child(ren) | 366 | 3.2 (0.45) |
| Widowed | 217 | 2.5 (0.28) |
| Alone | 3134 | 2.9 (0.16) |
| Other | 452 | 3.7 (0.54) |
| Missing | 15,304 | 4.7 (0.11) |
| Physical Activity 1997 (METS) | ||
| Quintile 1 | 19,489 | 3.3 (0.07) |
| Quintile 2 | 20,669 | 3.2 (0.06) |
| Quintile 3 | 20,540 | 3.0 (0.06) |
| Quintile 4 | 20,211 | 3.3 (0.06) |
| Quintile 5 | 6764 | 5.2 (0.19) |
| Missing | 3 | 2.7 (0.33) |
| Smoking | ||
| Never | 57,206 | 3.3 (0.04) |
| Former | 21,534 | 3.5 (0.07) |
| Current | 8706 | 3.4 (0.11) |
| Missing | 230 | 6.7 (1.2) |
| Alcohol consumption (gm/d) | ||
| None | 41,537 | 2.9 (0.04) |
| <5 | 23,256 | 2.7 (0.04) |
| 5-14 | 15,675 | 2.8 (0.05) |
| 15+ | 4654 | 2.7 (0.08) |
| Missing | 2554 | 20.8 (0.77) |
2554 women were excluded from this analysis because nutrient information was not available.
Mutually adjusted.
SE indicates standard error.
We considered questions with multiple food subtypes separately. Among all participants, 73% (n = 63,714) provided responses for all 4 milk types (skim or 1% milk; 2% milk; whole milk; soy milk); 23% (n = 19,826) of participants provided a response to only 1 milk type and left the other 3 blank; and 0.3% (n = 294) omitted the entire milk question. Among the 63,714 participants who provided complete responses, 67% consumed only 1 type of milk, less than 1% consumed all types, 11% consumed no milk at all, and 22% consumed more than 1 type. No distinct pattern emerged for cookie consumption or for sweet rolls, coffee cake, or other pastries, because consumption of subtypes was less exclusive.
Table 5 shows the frequencies of intake of the food and beverage items initially left blank in 1999 and provided in the resurvey in 2002 by 2485 women. Overall, 64% of the initially omitted foods were never consumed or consumed less than once per month; 20% of initially omitted foods were consumed 1–3 times per month; 6% of foods initially left blank were consumed once per week; 11% were consumed 2–4 times per week or more frequently. Overall, 45% of the initially marked foods were never consumed; hence, a food left blank was 42% more likely not consumed than any marked food.
Table 5.
Frequency Distribution of Foods and Beverages Initially Marked and Initially Omitted Among 2485 NHS II Participants Resurveyed
| Foods and Beverages | ||||||
|---|---|---|---|---|---|---|
| Frequency of Intake | Initially Marked | Initially Omitted and Provided in Resurvey | ||||
| No. | % | Cumulative % | No. | % | Cumulative % | |
| Never or <1/mo | 153,313 | 45 | 45 | 14,696 | 64 | 64 |
| 1–3/mo | 80,809 | 24 | 68 | 4572 | 20 | 84 |
| 1/wk | 50,818 | 15 | 83 | 1739 | 8 | 91 |
| 2–4/wk | 34,677 | 10 | 93 | 1324 | 6 | 97 |
| 5–6/wk | 8067 | 2 | 96 | 276 | 1 | 98 |
| 1/d | 8308 | 2 | 98 | 269 | 1 | 99 |
| 2–3/d | 4500 | 1 | 99+ | 146 | <1 | 99+ |
| 4–5/d | 1064 | <1 | 99+ | 29 | <1 | 99+ |
| 6+/d | 666 | <1 | 100 | 22 | <1 | 100 |
| Total | 342,222 | 23,073 | ||||
The frequency of consumption of the initially omitted food items was closely related to their mean frequency of consumption in the population who completed these items. Commonly consumed food items left blank on the initial survey were reported to be consumed with higher frequency in the resurvey than were less commonly consumed foods (eAppendix, available with the online version of this article). Figure 1 depicts the plot of the mean frequency of consumption of foods and beverages among the responders to the initial FFQ versus the mean frequency of consumption of initially omitted foods and beverages provided in the resurvey of 2485 women. The distribution suggests that the true intake of omitted foods and beverages is not zero and depends on their general frequency of consumption. The regression coefficient for the slope of the fitted line was 0.82 (± 0.021), suggesting that the best estimate for the true intake value of an omitted food is 0.82 times the average intake among participants who provide information on it.
Figure 1.
Relation between mean frequency of consumption of individual foods and beverages provided on the initial FFQ and mean frequency of consumption of initially omitted foods and beverages provided in the resurvey in sample of 2485 resurveyed women; the solid line represents identity; the dashed line is the fitted slope. Plot excludes Water, Coffee, Skim Milk, and Salt.
Total caloric intake and macro- and micronutrient intake were slightly higher after the resurvey; however, carbohydrate and fat intake as a percentage of energy intake remained the same (eTable 1 available in the online version of this article). Concordance correlation coefficients for nutrient intakes before the resurvey (with missing values set to zero) and after the resurvey ranged between 0.97 and 0.98. Among women with 10–19 blanks, absolute macronutrient intakes were underestimated by about 5%–6%; with 20 or more missing values, intakes were underestimated by more than 10%, although calorie-adjusted intake was not appreciably affected (eTable 2 available in the online version of this article). Nutrient intake after filling in the blanks was higher the more items had initially been omitted (eTable 2).
Discussion
In this large cohort of registered nurses, we found that on average participants left few items of the self-administered FFQ blank. Lifestyle factors were associated with the likelihood of providing complete responses on the FFQ. With increasing age women left more items blank, possibly because their memory failed them more often. Our observations correspond to those made in the Norwegian Women and Cancer Study by Parr and colleagues,11 who found higher proportions of missing responses among older participants. In our study, women with more children and who exercised a lot may have been busy and thus had less time to complete the FFQ.
The accuracy of a structured diet questionnaire can be affected by the number of food items left blank. In particular, calculation of nutrient intake may be affected by the number of foods omitted, but the critical number of missing foods affecting accuracy of estimates likely varies by nutrient. It may therefore be prudent to consider whether the food items with the highest number of missing values are important sources of a particular nutrient under study. In the NHS II, nutrient intakes were 4%–6% lower with more than 20 missing foods; however, estimation of energy-adjusted nutrient intake was little affected.
Information on milk consumption was asked as 1 question with 4 subtypes of milk. Participants had a preference for 1 type of milk. Hence if they marked only 1 type of milk, it is likely that the omitted subtypes were not consumed. This suggests that the underlying reasons for the missingness of milk data is not random and has implications for how the missing data should be treated.
Our resurvey of 2485 NHS II participants indicated that a food item left blank on the FFQ is most likely consumed either “never or less than once per month” or “1–3 times per month”. Although 84% of omitted foods were consumed less than once per week, 17% were consumed once per week or more frequently, hence with some regularity. The best estimate for the true value of an omitted food is 0.82 times its average intake in the population. Notably, the general frequency of consumption of a food in the population may predict the frequency with which an omitted food may truly have been consumed: common foods, if omitted, were more likely to have been consumed with higher frequency than less common foods that were omitted.
Our results are consistent with those from some prior studies. Caan et al8 recontacted 123 members of the Kaiser Permanente Medical Care Program who had completed an FFQ and who had omitted 1 or more food items. The follow-up information obtained in a telephone interview disclosed that 52% of food items omitted were never consumed, another 21% were consumed 1–3 times per month, and 27% were consumed at least once per week. Hansson and Galanti14 obtained information on omitted food items of an FFQ administered in a Swedish study and found similar distributions: The omitted answers corresponded to very rare consumption in about 54% of participants. However, similar to our observations, the “true” proportion of null consumption varied greatly by food group and was correlated with the frequency of intake in the overall population. Fraser and colleagues15 recontacted 240 participants in the Adventist Health Study-2 to request information on the initially omitted food items and observed a similar distribution of intake among initially provided and initially missing food items. In a Korean population, 216 men were resurveyed to complete blanks on a previously administered FFQ.16 Average nutrient intake increased significantly after filling in the blanks and was proportional to the number of items originally omitted; however, the proportion of omitted foods was higher in that population than in most of the other studies. In most studies, including ours, nutrient intake after filling in the blanks was higher the more foods had initially been omitted; it is possible that individuals who complete the entire questionnaire on one occasion keep relative consumption of foods in perspective, while a fractioned approach gives more weight to the reporting of individual food items.
Although the common practice of setting missing foods to zero intake is often not correct and does not attempt to impute the best estimate of true intake, it does represent a reasonably good estimate. On average, if foods were left blank, they were more likely not consumed than if they were marked; among foods marked, 45% were marked as never consumed, and of foods initially omitted, 64% were later marked as never consumed. Caan et al8 observed similar differences, with the respective proportions being 30% and 52%; this trend was also observed in the Adventist Health Study-2.15
A more informed imputation of frequency for missing items could take into account how frequently the food or beverage is consumed in the population at large. Prune juice, an infrequently consumed item, was never consumed by 98% of women who left it blank on the initial questionnaire. In contrast, pasta, a food consumed with medium frequency, was never consumed by only 4% percent of women who omitted it, 37% consumed it once per week and another 37% reported eating it 2–4 times per week.
The proportion of omitted foods in this population of educated health professionals was very low. In populations with a high proportion of omitted food items on the FFQ, setting missing values of food consumption to zero may cause substantial misclassification of calculated nutrient intake and affect the ranking of individuals according to their estimated nutrient intake, which may affect classification of individuals in quintiles of intake. In this situation, which should ideally be avoided, an imputation method for blank items may be useful. However, because foods are not omitted at random, special imputation strategies have to be applied. Furthermore, reasons for omitting food items may differ in a less educated population, and thus our results may not be generalizable to all strata of the population.
In our resurvey of 2876 participants in the NHS II, we obtained complete responses from 89% of the sample. Although we assumed that the responses provided in the resurvey were the same as the participants would originally have provided, this assumption may not be correct. The resurvey took place 2.5 years after the initial questionnaire was mailed. Although we asked participants to provide the frequency of food consumption for the missing items during the year prior to June 1999, some responses may have been affected by their consumption in early 2002. We also encouraged participants to provide information on frequency even if they could not remember their exact intake, which may have introduced some misclassification. However, recall of dietary intake in the recent past has been found to be accurate in a number of studies,7 and for most individuals dietary habits are relatively stable.
In conclusion, lifestyle factors including age, body mass index, physical activity, and parity are associated with the number of items left blank. In general when calculating nutrient intake, the assumption that items left missing represent zero intake is reasonable. However, foods consumed with higher frequency in the population at large are less likely left blank because they were not consumed than rarely consumed foods. If no response is provided for more than 20 items on the FFQ, absolute nutrient intake may be underestimated by more than 10% if missing foods are assumed not to have been consumed. However, even with this number of missing responses, nutrient intakes expressed as a percentage of energy are estimated with good accuracy if zero intake is assumed for omitted food items.
Supplementary Material
References
- 1.Willett WC, Sampson L, Stampfer MJ, Rosner B, Bain C, Witschi J, Hennekens CH, Speizer FE. Reproducibility and validity of a semiquantitative food frequency questionnaire. Am J Epidemiol. 1985;122(1):51–65. doi: 10.1093/oxfordjournals.aje.a114086. [DOI] [PubMed] [Google Scholar]
- 2.Munger RG, Folsom AR, Kushi LH, Kaye SA, Sellers TA. Dietary assessment of older Iowa women with a food frequency questionnaire: nutrient intake, reproducibility, and comparison with 24-hour dietary recall interviews. Am J Epidemiol. 1992;136(2):192–200. doi: 10.1093/oxfordjournals.aje.a116485. [DOI] [PubMed] [Google Scholar]
- 3.Cade J, Thompson R, Burley V, Warm D. Development, validation and utilisation of food-frequency questionnaires - a review. Public Health Nutr. 2002;5(4):567–87. doi: 10.1079/PHN2001318. [DOI] [PubMed] [Google Scholar]
- 4.Goldbohm RA, van den Brandt PA, Brants HA, van't Veer P, Al M, Sturmans F, Hermus RJ. Validation of a dietary questionnaire used in a large-scale prospective cohort study on diet and cancer. Eur J Clin Nutr. 1994;48(4):253–65. [PubMed] [Google Scholar]
- 5.Cheng Y, Yan H, Dibley MJ, Shen Y, Li Q, Zeng L. Validity and reproducibility of a semi-quantitative food frequency questionnaire for use among pregnant women in rural China. Asia Pac J Clin Nutr. 2008;17(1):166–77. [PubMed] [Google Scholar]
- 6.Johansson I, Hallmans G, Wikman A, Biessy C, Riboli E, Kaaks R. Validation and calibration of food-frequency questionnaire measurements in the Northern Sweden Health and Disease cohort. Public Health Nutr. 2002;5(3):487–96. doi: 10.1079/phn2001315. [DOI] [PubMed] [Google Scholar]
- 7.Willett W. 2nd edition Oxford University Press; New York: 1998. Nutritional Epidemiology. [Google Scholar]
- 8.Caan B, Hiatt RA, Owen AM. Mailed dietary surveys: response rates, error rates, and the effect of omitted food items on nutrient values. Epidemiology. 1991;2(6):430–6. [PubMed] [Google Scholar]
- 9.Folsom AR, Kaye SA, Prineas RJ, Potter JD, Gapstur SM, Wallace RB. Increased incidence of carcinoma of the breast associated with abdominal adiposity in postmenopausal women. Am J Epidemiol. 1990;131(5):794–803. doi: 10.1093/oxfordjournals.aje.a115570. [DOI] [PubMed] [Google Scholar]
- 10.Riboli E, Toniolo P, Kaaks R, Shore RE, Casagrande C, Pasternack BS. Reproducibility of a food frequency questionnaire used in the New York University Women's Health Study: effect of self-selection by study subjects. Eur J Clin Nutr. 1997;51(7):437–42. doi: 10.1038/sj.ejcn.1600422. [DOI] [PubMed] [Google Scholar]
- 11.Parr CL, Hjartaker A, Scheel I, Lund E, Laake P, Veierod MB. Comparing methods for handling missing values in food-frequency questionnaires and proposing k nearest neighbours imputation: effects on dietary intake in the Norwegian Women and Cancer study (NOWAC). Public Health Nutr. 2008;11(4):361–70. doi: 10.1017/S1368980007000365. [DOI] [PubMed] [Google Scholar]
- 12.Barzi F, Woodward M, Marfisi RM, Tognoni G, Marchioli R. Analysis of the benefits of a Mediterranean diet in the GISSI-Prevenzione study: a case study in imputation of missing values from repeated measurements. Eur J Epidemiol. 2006;21(1):15–24. doi: 10.1007/s10654-005-5086-5. [DOI] [PubMed] [Google Scholar]
- 13.Lin LIK. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989;45:255–68. [PubMed] [Google Scholar]
- 14.Hansson LM, Galanti MR. Diet-associated risks of disease and self-reported food consumption: how shall we treat partial nonresponse in a food frequency questionnaire? Nutr Cancer. 2000;36(1):1–6. doi: 10.1207/S15327914NC3601_1. [DOI] [PubMed] [Google Scholar]
- 15.Fraser G, Henry D, Knutsen S. Missing data in dietary epidemiology: are imputed zeros correct? Am J Epidemiol. 2004;159:S52. abstract. [Google Scholar]
- 16.Ahn Y, Paik HY, Ahn YO. Item non-responses in mailed food frequency questionnaires in a Korean male cancer cohort study. Asia Pac J Clin Nutr. 2006;15(2):170–7. [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.