Validity and reproducibility of a food frequency questionnaire for assessment of fruit and vegetable intake in Iranian adults^*

Abstract

BACKGROUND:

This study's aim was to design and validate a semi-quantitative food frequency questionnaire (FFQ) for assessment of fruits and vegetables (FV) consumption in adults of Isfahan by comparing the FFQ with dietary reference method and blood plasma levels of beta-carotene, vitamin C, and retinol.

METHODS:

This validation study was performed on 123 healthy adults of Isfahan. FV intake was assessed using a 110-item FFQ. Data collection was performed during two different time periods to control for seasonal effects, fall/winter (cold season) and spring/summer (warm season). In each phase a FFQ and 1 day recall, and 2 days of food records as the dietary reference method were completed and plasma vitamin C, beta-carotene and retinol were measured. Data was analyzed by Pearson or Spearman and intraclass correlations.

RESULTS:

Serum Lipids, sex, age, body mass index (BMI) and educational level adjusted Pearson correlation coefficient of FV with plasma vitamin C, beta-carotene and retinol were 0.55, 0.47 and 0.28 in the cold season (p < 0.05) and 0.52, 0.45 and 0.35 in the warm season (p < 0.001), respectively. Energy and fat intake, sex, age, BMI and educational level adjusted Pearson correlation coefficient for FV with dietary reference method in the cold and warm seasons were 0.62 and 0.60, respectively (p < 0.001). Intraclass correlation for reproducibility of FFQ in FV was 0.65 (p<0.001).

CONCLUSIONS:

The designed FFQ had a good criterion validity and reproducibility for assessment of FV intake. Thus, it can serve as a valid tool in epidemiological studies to assess fruit and vegetable intake.

Chronic diseases, including cardiovascular diseases (CVD) and cancers are among the leading causes of mortality worldwide.¹ CVD, accidents and cancers are the top causes of death in Iran respectively.² Previous epidemiologic studies have consistently supported the association between high intake of fruits and vegetables (FV) and lower risk of cancer, heart disease, and stroke.^3–5 Researches have demonstrated that daily intake of 5 servings of FV (at least 400 grams, excluding potatoes) is regarded as an important guideline for prevention of chronic disease.^6–8 Despite these facts, FV intake is inadequate in many developed and developing countries.⁹ In Iran, the per capita net daily consumption of FV has been shown to be below the WHO recommendation⁸ and is estimated to be 142 g for fruits and 229 g for vegetables.¹⁰ Based on such findings, increasing FV intake is an important concern in our society.

Valid and reliable methods for measurement of FV intake is required to evaluate efforts to increase FV consumption in healthy diet such as Dietary Approach to Stop Hypertension (DASH) which recommended 7-8 servings of FV per day.¹¹ A variety of techniques for measurements of habitual dietary intake are available, however, many of them need multiple and repeated recalls which are time consuming and have high recall bias which makes them unsuitable for certain groups or individuals.¹² Thus, development of alternative methods that are inexpensive and easy to perform in the field setting is of importance. Food frequency questionnaires (FFQ) have been proposed as a simple and inexpensive dietary assessment method with low interview and recording error as well as minimal recall bias.¹³ However, FFQs need to be tailored to the food-consumption patterns and culture of the study population.^14–15 Two FFQ validation studies were done in Iran to assess all food groups in “Tehran Lipid and Glucose Study”¹⁶ and “Golestan Cohort Study of Esophageal Cancer”¹⁷ which the second one designed for population with a special dietary pattern. However, they were not particularly designed for estimation of FV as an important component of DASH diet. Validation of FFQ can be done in various ways; comparisons with self reported diet records or recalls have most frequently been reported.^18–19 Another potential method of validation is to compare the results of FFQs with biochemical biomarkers which can provide a quantitative and objective measure of intake. Measurement errors in these methods are essentially independent of the errors associated with FFQ (i.e., systematic overestimation or underestimation of consumption frequency or portion size of certain foods).^20–21 Therefore, correlations between FV consumption determined with a dietary assessment method, such as an FFQ, and biomarkers are not artificially inflated as a result of correlated errors in both methods which can be the case when two similar methods are compared. In the present study, we used biomarkers in addition to multiple 24 hr recalls and food records methods as the reference method. Frequently used biomarkers for FV are levels of carotenoids^20–25 and vitamin C in blood.²³,²⁵ However, correlations between FV intake and concentrations²⁴,²⁵ of the above biomarkers are modest, because biomarkers are also influenced by physiologic factors such as absorption and metabolism. As beta-carotene was shown to have the highest correlation with FV consumption²⁶ and the measurements of it is practical, this carotenoid was chosen.

This study aimed to design and validate a semi-quantitative FFQ for assessment of FV consumption in adults residing in the city of Isfahan, Iran, by comparing results with 24 hr recalls and two self reported diet records as the dietary reference method as well as biomarkers including plasma levels of beta-carotene, vitamin C and retinol.

Methods

Subjects: Subjects were recruited from the participants of an ongoing study, the Isfahan Healthy Heart Program (IHHP)^27–28 who had been selected through cluster random sampling from the Isfahan adult population, and from healthy individuals referred to the Isfahan Cardiovascular Research Center (ICRC) clinic. The study sample included 123 individuals, aged 30-60 years, who were non-diabetic and had no history of CVD, renal, thyroid, hematological, or mental diseases. Smokers, those on special diets, users of vitamin supplements, and pregnant or lactating women were excluded.

Study design: This study was a validation study to design a validated FFQ. The FFQ was completed twice; at the beginning of the study in fall and winter of 2004-2005 (November, December and January) and six months thereafter in spring and summer of 2005 (June, July and August). The total number of samples studied in fall/winter and spring/summer were 123 (64 men and 59 women) and 101 (53 men and 48 women), respectively. Two different seasons were included to allow for seasonal variations in FV intake. The initial FFQ was accompanied by a demographic questionnaire and a 24 hour diet recall which were administered by a trained dietitian. In the same session, the respondents or one of their family members were also trained to complete two self reported diet records in the same week. Additionally, a 10cc fasting blood sample was drawn for determination of biomarkers (beta-carotene, vitamin C and retinol) and for measurement of serum lipids.

The study protocol was approved by the ethical committee of the ICRC, which is a member of the Office for Protection of Human Subjects of the US Department of Health and Human Services and by the Research Council of the National Nutrition and Food Technology Research Institute in Iran.

Development of the food frequency questionnaire: A 110-item FFQ was designed to assess usual fruit and vegetable intake. The initial food list was composed of 105 food items which were fruits and vegetables contributing to the diet of people in Isfahan, based on previous dietary studies in the area as well as the results of National Food Consumption Survey of Iran with specific emphasis on Isfahan province.¹⁰ Face and content validity of the questionnaire were assessed by an expert panel consisting of 3 nutritionists from National Nutrition and Food Technology Research Institute in Iran. They were the nutritionists who were involved in national dietary survey and were familiar with the dietary habits of different societies in Iran.

To evaluate clarity and comprehensiveness, the FFQ was tested in a pilot study among thirty adults who were not entered in the main study participants and had the same characteristics as the study population. In the pilot study we asked an open question about any other kinds of fruits or vegetables that the participants ate. Based on the result of the pilot study, 5 items including persimmon, pomegranate, fig, tomato paste and okra were added to the initial list. No other major changes were made to the questionnaire.

For each item, respondents were asked to indicate the number of times consumed per day, per week, or per month. Never and seldom were also included. They were then asked about the amount consumed each time. To help the respondent's memory, household scales, including glasses and tablespoons, as well as fruits and vegetables displayed in colored photographs were used during the interview. Subjects were asked whether their average portion size was half, the same, double, or three times the servings, or they could choose one out of three pictures, showing small, medium or large portions of different foods or dishes.²⁹ For questions on frequency, all reported numbers were converted to daily frequency and multiplied by the portion size indicated. Seldom and never were calculated as “zero”.

The dietary reference methods: All participants also completed a single 24 hr recall and 2 food records for 3 non-consecutive days, including 2 weekdays and one weekend. We used two dietary assessment methods because it was difficult to fill three 24 hr recall. A single 24 hr recall was completed by interview. Participants were trained for self reporting 2 other dietary records.

To estimate the food portions, a food album which contained all food items in 2 or 3 sizes was used.²⁹ In the case of mixed dishes, to estimate the serving size of each person, the total amount of cooked food as well as the number of persons who consumed it was collected and the amount of the food intake for each person was then calculated. The participants were asked to complete two self reported food records. If he/she was illiterate and was not able to complete the questionnaire, a family member was requested and trained to do it. The participants were followed by phone to verify and complete self reported food records. The mean FV intake from the single 24 hr recall and 2 food records in each season were considered the reference method for each FFQ. Food items were coded based on a nutrition software designed based on food composition tables from the National Nutrition and Food Technology Research Institute.³⁰ The same software was used to determine energy and fat intake to control for confounding.

Food intakes were determined in grams based on the previously established weights of the used measure.³⁰ For data analysis fruits were divided into five categories, including citrus, melons, other fruits, dry fruits, nuts and natural fruit juices. Vegetables were divided into 6 categories: tubers, leafy vegetables, non-leafy vegetables, onion, pickles, and dried vegetables. Potatoes were not included as vegetable.

Anthropometric measurements: A trained nutritionist measured standing height without shoes and recorded to the nearest 0.5 cm at the baseline visit. Body weight was measured with the subjects wearing light clothes without shoes and recorded to the nearest 0.5 kg. Body mass index (BMI) was calculated as body weight (kg) divided by squared height (m²).³¹ It was used as a confounding factor in the statistical analysis.

Biochemical analysis: The participants were requested to come to the study center after fasting over night. Venous blood samples were drawn between 7:00 and 11:00 am to measured plasma vitamin C, beta-carotene, retinol and serum lipids. The sample was transferred into 2 heparinized vacationers and one tube without heparin. The heparinized vacationers were centrifuged, and immediately were covered with aluminum foil and stored in the dark on ice for up to 1 hour until the plasma was separated and then was processed within hours.³² One heparinized aliquot was prepared by using 10 percent metaphosphoric acid to stabilize ascorbic acid. All samples were stored at -70°C . The long-term stability of these nutrients, when stored at -70°C to -80°C, was examined in numerous studies and found to be acceptable.³² Plasma ascorbate was spectrophotomtrically assayed in ICRC laboratory using 2, 4-dinitrophenyl hydrazine as chromogen, which was shown to be highly correlated with high-pressure liquid chromatography (HPLC) methods.³³ The other plasma tubes were transported by air to the nutrition laboratory of the Nutrition and Food Technology Institute in Tehran in cold box within 3 hours, to assay beta-carotene and retinol by HPLC method. Serum was separated from the test tube without heparin in order to measure total cholesterol and triglyceride concentrations by enzymatic methods at the ICRC laboratory.

Statistical analysis: The data were analyzed using SPSS program version 11.5. The distributions of dietary intake values were examined for normality by the Kolmogorov-Smirnov test. The mean age, BMI, and FV intakes in the two seasons were compared through a paired t test and comparison of frequencies of nominal demographic characteristics were analyzed using Pearson's chi-square test. Validation of the FFQ was determined using Pearson correlation coefficients of FV intake as assessed by the FFQ and the dietary reference method and plasma biomarkers. As distributions of dry fruits and nuts, dry vegetable, citrus (spring only) and melon (fall only) were not normal, the Spearman correlation coefficient was used for these variables. Partial correlation was used to adjust confounding factor effects such as age, sex, and BMI. Partial correlation was also used to adjust total cholesterol and triglycerides for correlations between plasma beta-carotene and retinol levels with FV intakes. Finally, partial correlation was used to adjust age, sex, BMI, dietary energy fat intake in correlation between fruits and vegetables assessed by FFQ and the reference method. Participants were divided into 4 groups of fruits and vegetables intake based on FFQs or dietary reference method. Then the frequency of participants in the same quartile, within one quartile and extreme quartile of 2 dietary assessment methods were estimated. Sensitivity, specificity, and predictive values were measured by chi-square test. Intraclass correlation coefficients were used to determine the reproducibility. P-values less than 0.05 were considered statistically significant.

Results

The study sample consisted of 123 subjects (64 males and 59 females) in the first measurement and 101 (53 males and 48 females) subjects in the second measurement. Nineteen subjects did not participate in the second phase due to lack of time and geographic relocation. Table 1, displays demographic characteristics including age, weight, height, BMI and FV intake based on the FFQ and frequency distributions by age group, sex, and educational status in both phases. There were no differences between sample characteristics and mean FV intake in both periods of measurement.

Table 1.

Basic characteristics and fruits and vegetables intake of participants

Validation: Table 2 demonstrates the correlations of FV intake estimated by the FFQ with biomarkers including vitamin C, beta-carotene and retinol adjusted for age, sex, educational level, BMI, as well as triglyceride and total cholesterol for beta-carotene and retinol. The correlations with vitamin C were the highest and with retinol were the lowest. The correlation coefficient of total fruit intake with plasma vitamin C, beta-carotene, and retinol level were 0.50 (p < 0.001), 0.44 (p < 0.001), and 0.29 (p < 0.01) for the winter measurement, and 0.49 (p < 0.001), 0.42 (p < 0.001) and 0.32 (p < 0.01) for the summer measurement, respectively. The correlation coefficients of total vegetable intake with vitamin C, beta-carotene and retinol were 0.4 (p < 0.001), 0.41 (p < 0.001) and 0.27 (p < 0.05) in the winter measurement, and 0.48 (p < 0.001), 0.43 (p < 0.001) and 0.31 (p < 0.01) in the summer measurement, respectively. Correlation between FFQ categories of other fruits, tubers, leafy vegetables and non leafy vegetables with all biomarkers were statistically significant. Correlation coefficients for FFQ categories of fruit juices and onions with vitamin C and beta-carotene were statistically significant. Correlation coefficients of dry fruits and nuts with beta-carotene and retinol were also significant. Citrus fruits and melons had a significant correlation with all biomarkers in the first and second phases, respectively. The correlations of estimated amounts of fruit and vegetable intake by FFQ and dietary reference methods after adjusting for age, sex, BMI, educational level and dietary fat and energy intake are shown in Table 3. Correlation coefficients in all categories of fruit and vegetables ranged from 0.29 (p < 0.01) for melons to 0.60 (p < 0.001) for onions in the winter phase and from 0.31 (p < 0.01) for dry vegetables to 0.59 (p < 0.001) for onions in the summer phase, when the mean of 3 days dietary reference (one 24-hr recall and 2 food records methods in each season) was used as a gold standard. When the two seasons′ data of dietary recalls and self reported dietary records were combined, the mean of all 6 days of the dietary reference method was used to compare it with FFQ in each phase. Correlation coefficients for all FFQ categories of fruits and vegetables were statistically significant except for melons in the winter phase and citrus in the summer phase. The correlation coefficients of FFQ categories with mean of 6 days of the dietary reference method varied from 0.16 (non-significant) in melons to 0.58 for onions (p < 0.001) in the winter phase and from 0.19 (non-significant) for citrus to 0.57 (p < 0.001) for onions in the summer phase. The correlation coefficients of FFQ total fruits with mean of 6 days of the dietary reference method were 0.60 and 0.61 in winter and summer phases, respectively and for total vegetables 0.55 and 0.59, respectively. The death deatenuation increased the correlation coefficients. As shown in Table 4, the percentage of subjects in dietary reference method quartiles matched the same FFQ quartiles, ranged from 42% for vegetables in 1^st FFQ to 51% for vegetables based on 2^nd FFQ. The frequency of population within one quartile of dietary reference method and FFQ were from 72% in vegetable in 1^st FFQ to 91% in total fruits and vegetables in 2^nd FFQ. Gross misclassifications between FFQs and dietary reference methods were rare (0-8 %). Sensitivity, specificity, negative and positive predictive values are shown in table 5. The sensitivities, specificities and positive predictive values of FFQ compared to dietary reference method were 70-86%, 80- 90% and 65-80%, respectively.

Table 2.

Correlation coefficients of fruits and vegetables intake assessed by food frequency questionnaire with plasma biomarkers in two different seasons

Table 3.

Correlation coefficients¹ of fruits and vegetables intake assessed by food frequency questionnaire with dietary reference method in two different seasons

Table 4.

Cross–classification by quartiles of fruits and vegetables intake from dietary reference and the first and second food frequency questionnaire

Table 5.

Sensitivity, specificity and predictive values of food frequency questionnaire for assessing adequacy of intake of fruits and vegetable serving

Reproducibility: Intraclass correlation coefficients (ICC) for total fruits and vegetables were 0.63 and 0.59, respectively (table 6). This varied from 0.16 (non significant) in melons to 0.85 (p < 0.001) for onions. The ICC in the other fruits category was the highest (0.78) among all kind of fruits and onion were the highest (0.85) among all kinds of vegetables.

Table 6.

Intraclass correlation coefficients of fruits and vegetables intake in adults of Isfahan, Iran, 2004-2005

Discussion

We evaluated the reproducibility and relative validity of a semi quantitative FFQ designed to measure fruit and vegetable intake compared to plasma biomarkers and dietary reference methods, including two 24-hr recalls and four food records. The FFQ had higher relative validity and reproducibility in non seasonal fruits and vegetables than seasonal one and also these were higher for total fruit and vegetables than each individual category. There were no differences between the winter and summer phase of fruit and vegetable consumption (two different seasons). Although the types of fruits and vegetables in each season were different, the total amount remained the same. The consumption of citrus fruits and melons was greater in the cold season, while consumption of melons was greater in the warm season. Conversely, consumption of tubers and leafy vegetable consumption was greater in the cold season. An outbreak of the Vibrio Cholerae infection during that warm season³⁴ might cause the people consumed these kinds of vegetables less than usual.

In the present study the correlation of the FFQ in assessing the relative intake of fruits and vegetables compared with plasma vitamin C level was the highest among the plasma biomarkers, and this correlation was higher when compared to total fruits and vegetables intake. Several studies have shown correlations of plasma carotenoids,^20–25 plasma vitamin C,²³,²⁵ and beta-carotene with fruit and vegetable intake. In Block et al. study²⁵ correlations of fruits and vegetables with plasma biomarkers such as vitamin C, carotenoids and beta-carotene were 0.59, 0.40 and 0.38, respectively.

Since the main sources of retinol are animal foods,³⁵ this may explain the lower correlation between plasma retinol and fruits and vegetables than other biomarkers, and the overall weak correlation except for melons and non leafy vegetables in warm season. Moreover, it was found a positive association between plasma retinol level and serum lipids without any relationship to retinol intake.³⁶ Low correlation was observed between the biomarkers and fruit juice intake. This may be due to the lower vitamin C and beta-carotene content of fruit juice as demonstrated by Bogers et al.²³

In Resnicow et al. study²⁶ correlation of plasma beta-carotene and total consumption of fruits and vegetables was 0.46, but this study included both smokers and vitamin supplement users. In the present study the correlation between plasma retinol and beta-carotene with fruit and vegetable intake were similar to previous studies,^17–23 while the FFQ correlation with plasma vitamin C was higher than most studies, except for Block et al. work on relationship of plasma antioxidants and fruit and vegetables intake.²⁵ This may be due to the fact that the FFQ of current study assessed fruit and vegetable consumption over the previous month. The correlation between FFQ and dietary reference methods in assessing total fruits and vegetables intake in each season were relatively good. It was the same when and also when the mean of 6 days of dietary reference method was used as the gold standard. Therefore, the criterion validity of the FFQ for assessing total fruits and total vegetables were acceptable. However, for specific categories of fruits and vegetables, items need to be seasonally driven.

A stronger relationship was observed between FV consumption as measured by the FFQ and the dietary reference method in comparison with plasma biomarkers. Stronger association of vitamin C than carotenoids might be due to differences in storage, metabolism, or the difficulties of measurement. Vitamin C is water soluble, with major stores in muscle tissue, and the rate of utilization depends on numerous factors, including body weight, smoking, vigorous exercise, exposure to stressor and possibly, gender. Carotenoids are lipid soluble, stored in fatty tissues, and utilization also depends on smoking and body weight, although possibly to a lesser extent. It is possible that if carotenoids had been measured in adipose tissue, correlations with fruit and vegetable intake would have been higher.²⁵

Intraclass correlation coefficient (ICC) for total fruits was higher than what it was for total vegetables. ICC for leafy vegetables and tubers were low, probably due to their seasonality and the epidemic of Vibrio Cholerae infection in that warm season. Therefore, people probably consumed these kinds of vegetables less than usual.

Sensitivity of FFQ for assessing fruit and vegetable intake is acceptable, however, the specificity is higher. This means that the ability of this FFQ is higher in identifying persons who consume the recommended amount of fruit rather to than those whose consumption is less than recommendations. Ling et al. reported both higher sensitivity and specificity compared to this study.³⁷ The positive and negative predictive values indicated that this FFQ is able to identify 65-80% of those who really had inadequate fruit and vegetable intake and 84-91% of those with adequate intake based on WHO recommendations.

The highest ICC between two measure of fruits and vegetables were between onions and the other fruits category, respectively. This could be due to the fact that in the Iranian diet, onions are frequently consumed throughout the year and the other fruits category includes many kinds of fruits such as banana and apple which are also consumed throughout the year and are not seasonal. Lower ICC was observed in some kinds of fruits such as citrus and melons which could be due to the seasonality of those items, since the second FFQ was completed after 6 months and in a different season. In other studies that FFQs were completed within 3 or 12 months, the ICC was higher between certain fruits and vegetables. ICC was reported 0.19-0.80 in most studies.^20–25 Bogers et al. reported an ICC between 2 FFQs conducted with one month interval as being 0.81 for total fruits and 0.73 for total vegetables.²³ A follow up that conducted a year later revealed no difference in ICC for total vegetables, but was less for total fruits. In that study, ICC for the other fruits category was the lowest, however in the present study this category had the highest ICC among different categories of fruits. This may be due to the difference of the fruit items in this group between the two studies. In our study this category of fruits included bananas and apples which were consumed in different seasons. Bogers et al. reported that since within variances in fruits consumption was more than vegetables, the ICC of fruits were less than vegetables.²³ ICC for vegetables was lower in the present study. However, higher ICC in both Ling et al.³⁷ and Anderson et al.³⁸ studies might be because of that the second FFQ were completed after 14 days and 6 weeks, respectively.

As a whole, the designed FFQ in this study can be utilized for evaluation of diet modifications such as DASH diet that fruits and vegetables are one of the most important components of it.¹¹ Adherence to DASH diet may have the potential to prevent type 2 diabetes and particularly cardiometabolic risk factors in Iranian adults.^39–42

Limitation: The strength of the present study is the use of several reference criteria, including plasma biomarkers. Performing this study in two different seasons can be considered as both weak and strong aspect. However, since the FFQ assessed FV intake over the previous month, not the previous year, this can also be accounted as a limitation. It might have been better if we had planned the FFQs one year apart.

Conclusions

The criterion validity, reproducibility, sensitivity and specificity of this FFQ were relatively good. The FFQ could be used in community based studies for assessing fruit and vegetable intake. However, the relevance of this FFQ to other populations and other regions of the country need to be established by specific validation studies.

Authors’ Contributions

NM and NO designed the study and wrote the paper. NM, NO and AH contributed in designing the questionnaires. NM carried out the data collection. NM, TN and GN contributed in biochemical analysis. NM and BS analyzed the data. All authors read and approved the paper.