Validation of a food frequency questionnaire for children and adolescents aged 4 to 11 years living in Salvador, Bahia

Abstract

Objective

To assess the validity of a food frequency questionnaire (FFQ) by applying it to children and adolescents living in Salvador, Bahia

Methods

The validity of this FFQ with 98 food items was investigated among 108 children and adolescents who were selected from a sample of 1445 that had been planned for a study on the risk factors for asthma and other allergic diseases. The adults responsible for these children and adolescents gave responses for a 24-hour recall (R24h) and an FFQ. The average energy and nutrient values from the FFQ were compared with those from the R24h by means of the paired t test and Pearson correlation coefficients. The concordance was evaluated using the Bland-Altman method and kappa statistics.

Results

The energy and nutrient intake estimated using the FFQ was significantly higher than what was obtained using the R24h. The correlation coefficients adjusted for energy were statistically significant for protein, fat, vitamin C and zinc. The weighted kappa values ranged from 0.06 for vitamin A (p = 0.24) to 0.34 for energy (p < 0.00). The results from the Bland-Altman plots for lipid, protein and zinc showed the most significant validity parameters, and zinc was found to show the best concordance.

Conclusion

The results suggest that the FFQ showed satisfactory validity for use in studies involving children and adolescents.

Abbreviations

FFQ

Food frequency questionnaire

R24

24-hour recall

SCAALA

Social Changes, Asthma and Allergy in Latin America Programme

TACO

Tabela Brasileira de Composição de Alimentos

ENDEF

Tabela de Composição de Alimentos

Introduction

Epidemiological studies have often used food frequency questionnaires (FFQs) to evaluate the relationship between habitual diet and disease. In many investigations on food consumption among children and adults, FFQs have been shown to be good tools for assessing energy and macronutrient intake and reasonable tools for micronutrient intake.^1,2,3,4 Considering the complexity of individuals’ diets and the substantial variability in food intake over time, measurement errors are inevitable.⁵ Validation of food intake measurement tools makes it possible to increase the accuracy of dietary information and diminish possible sources of bias.⁶ This validation is generally obtained through assessment of concordance in relation to a reference method, i.e. comparison between the method to be tested and another reference method that represents the real food intake of individuals or a population group. The statistical methods most used in validating dietary assessment tools are the following: comparison of mean nutrient measurements; correlation analyses on Pearson’s or Spearman’s correlation coefficient and kappa statistics; and comparative distribution according to quartiles or quintiles of nutrient intake.⁶ In FFQ validation studies, 24-hour recall (R24h) is the method used as the reference point, since this is inexpensive and quickly applied, does not interfere with the family’s dietary behavior and does not require any minimum schooling level, unlike the dietary record method.⁵

In Brazil, there is still a scarcity of FFQ validation studies in relation to children aged 5 to 10 years.^7,8 In this respect, the present study makes it possible to qualitatively and quantitatively assess the diets of children and adolescents. During this stage of life, nutritional needs are greater because of growth and development, and this is also the time at which dietary habits are formed and consolidate. These are recognized to be influenced by cultural, economic, social and environmental factors.

Thus, the aim of this study was to validate an FFQ through applying it to a subsample of children and adolescents aged 4 to 11 years who were living in Salvador, Bahia.

Population and methods

A sample of 108 children and adolescents was randomly selected in order to validate the dietary assessment tool (i.e. the FFQ), using R24h as the reference method. These individuals came from a cohort composed of 1,445 children aged 4 to 11 years that had been set up to study risk factors for occurences of asthma and other allergic diseases (within the SCAALA program: Social Changes, Asthma and Allergy in Latin America). Further details on the methodology of this project can be obtained elsewhere⁹. To estimate the minimum sample size for the present study, the minimum expected correlation between the two methods was taken to be 0.30, the significance level used was 5% and the power was taken to be 80%.¹⁰

Data gathering using the two tools was done simultaneously, over the period from June to December 2006. The interviews were conducted by nutrition students who had been trained and were supervised by nutritionists.

In the R24h, all the foods, drinks and preparations consumed on the previous day were recorded. The FFQ was composed of 98 food items. For both of the food intake assessment tools, the mothers or other adults responsible for the children provided the information on the frequencies and quantities of food and drinks consumed.

For both tools, the portions or home-cooking measurements reported were converted into grams or milliliters. To minimize possible sources of error (memory bias) originating from the food consumption information, an album with drawings of foods and utensils of different dimensions was used, and in addition, standard liquid measurements were presented to the mothers at the time of the interview.^11,12

The nutritional analysis on the food intake quantities recorded using the two tools (FFQ and R24h) was done using the Diet-Pro software,¹³ which was developed at the Federal University of Viçosa, Minas Gerais. Foods that did not form part of the database of this software were added, using information contained in the Brazilian Food Composition Table (TACO);¹⁴ Assessment Table for Food Consumption from Home-Cooking Measurements;¹⁵ ENDEF Food Composition Table¹⁶; and packaged food labels. It should be noted that to obtain the estimated nutritional value of the habitual daily food intake, from the FFQ, the frequency of consumption was transformed according to the number of times per day, by multiplying by the portion size in grams and dividing by the number of days of the month. Both dietary assessment tools provided data on total energy, carbohydrate, lipid, protein, vitamin A, vitamin C and zinc content.

Statistical analysis

The means and standard deviations for the energy and nutrient content estimated from the R24h and FFQ were calculated using the absolute values. The paired t test was used to assess the differences between the means for energy and nutrients derived from the two dietary tools. The data were firstly transformed into log values in order to produce normal distribution, and then the residual nutrient content was calculated, i.e. the portion of the nutrient intake that does not correlate with the total energy intake.¹⁷

To identify consumption correlations, Pearson’s linear correlation coefficient was used, before and after adjustment for total energy intake. To assess the concordance between the results obtained using the two tools, the intake level of each nutrient was classified into quartiles and was tested using weighted kappa statistics. These statistics take into account the magnitude of discordances between the classifications, using the expression 1 – {(i – j)/(k – l)}² for weighting. In this expression, i and j are respectively the lines and columns of the classifications obtained using the two associated dietary tools, and k is the maximum number of possible classifications¹⁸. The values obtained for the kappa coefficient were interpreted in accordance with the cutoff points proposed by Landis and Koch.¹⁸ Thus, kappa values of less than 0.21 were taken to indicate weak concordance; 0.21 to 0.40, fair concordance; 0.41 to 0.60, moderate concordance; 0.61 to 0.80 substantial concordance; and greater than 0.80, almost perfect concordance.

The plots proposed by Bland-Altman¹⁹ were also used to assess the degree of concordance between the values obtained from the two tools. In these, the paired differences in nutrient values obtained from the R24h and FFQ were plotted in relation to their respective means, with the lowest and highest intake limits represented by horizontal lines. The plot shows a trend if the variance of the FFQ tool is greater than the variance of the R24h tool and no trend if the variance of the two tools is the same. All the analyses were done using the STATA software (Stata Corporation, College Station, United States), version 9.0.

The protocol for this study was approved by the ethics committee for human research of the Institute of Public Health, Federal University of Bahia (CEP/ISC protocol no. 003-05). Data were only gathered after signed consent had been received from the children’s guardians.

Results

Among the 108 children who participated in the validation study, 57 (47.2%) were male and 51 (52.8%) were female. They were between 4 and 11 years of age (mean of 6.7 years; SD = 1.67), with a mean weight of 22.8 kg (SD = 6.04) and mean height of 120.8 cm (SD = 10.64).

Table I presents the means and their respective standard deviations for energy and nutrients, derived from the FFQ and R24h. Comparison of mean energy and nutrient intakes estimated using the FFQ and R24h showed that the FFQ overestimated the intake in relation to the result obtained using the R24h. The mean energy, vitamin A and vitamin C intakes were overestimated, respectively, by 315 kcal, 718.2 μg and 207.9 mg. Evaluation of the differences between these means using the paired t test showed statistically significant differences (p < 0.001) for energy and all the nutrients analyzed.

Table I.

Means (with standard deviations) for energy and nutrients obtained from the food frequency questionnaire (FFQ) and 24-hour recall (R24h), and the differences between them, Salvador. Bahia, 2006

Nutrients	FFQ		R24h		Difference between means	p value *
	Mean	SD	Mean	SD
Energy (kcal)	1,989.8	658.6	1,674.0	486.7	315.8	<0.001
Proteins (g)	60.1	23.3	50.3	17.5	9.8	<0.001
Lipids (g)	55.5	22.8	44.6	19.4	10.9	<0.001
Carbohydrates (g)	329.5	114.8	268.6	85.9	60.9	<0.001
Vitamin A (RE)	1,213.7	852.1	495.5	545.7	718.2	<0.001
Vitamin C (mg)	376.6	348.2	168.7	280.3	207.9	<0.001
Zinc (mg)	6.9	4.1	4.6	2.6	2.3	<0.001

RE: Retinol equivalent; *Paired t test: comparison of means between FFQ and R24h.

The crude correlation coefficients obtained ranged from 0.18 for vitamin A to 0.38 for carbohydrates. After adjustment for energy, all the values tended to decrease, and the largest reductions were observed in relation to carbohydrates and vitamin A, for which the adjusted r values came to have the same value (0.14), but did not show statistical significance. All the other adjusted values were lower and were statistically significant (proteins, lipids, vitamin C and zinc). After adjustment for dietary energy, the correlation coefficients between the estimated values from the FFQ and R24h ranged from 0.14 to 0.29 (table II).

Table II.

Pearson’s correlation coefficient (r) and kappa statistics for energy and nutrient intake estimated from the food frequency questionnaire (FFQ) and 24-hour recall (R24h), Salvador, Bahia, 2006

Nutrients	Correlation coefficient				Kappa statistic
	Crude		Adjusted for energy*		Kappa	P
	r	p value	r	p value
Energy (kcal)	0.19	0.047	–	–	0.34	< 0.01
Proteins (g)	0.33	0.001	0.27	0.004	0.24	< 0.01
Lipids (g)	0.33	0.001	0.26	0.006	0.21	< 0.01
Carbohydrates (g)	0.38	0.001	0.14	0.159	0.15	0.05
Vitamin A (RE)	0.18	0.059	0.14	0.154	0.06	0.24
Vitamin C (mg)	0.31	0.001	0.29	0.002	0.21	< 0.01
Zinc (mg)	0.33	0.001	0.21	0.026	0.25	0.04

RE: Retinol equivalent.

The weighted kappa values ranged from 0.06 for vitamin A to 0.34 for energy, and were statistically significant for energy and most of the nutrients: proteins, lipids, carbohydrates, vitamin C and zinc. There was weak concordance (kappa values < 20) for vitamin A and carbohydrates. The nutrients that presented fair concordance (kappa from 0.21 to 0.34) were proteins, lipids, vitamin C and zinc, along with energy (table II).

Bland-Altman plots were used to assess the concordance between the mean energy and nutrient values estimated from the two tools. Figure 1 illustrates the results from the Bland-Altman analyses for lipids, proteins and zinc: these were the nutrients with the best validity parameters, and zinc was the nutrient with the best concordance (p = 0.059).

Fig. 1.

Bland-Altman method, used to evaluate concordance between the food frequency questionnaire (FFQ) and 24-hour recall (R24h) for lipid, protein and zinc intake. Salvador, Bahia, 2006. ¹Upper limit of concordance; ²Difference between means; ³Lower limit of concordance.

Discussion

The present study examined the validity of an FFQ for assessing food intake among children and adolescents aged 4 to 11 years. It was observed that after adjustment for energy, the correlation coefficient decreased for the majority of the nutrients. Similar results were observed by Fumagalli et al.,⁷ who examined the validity of an FFQ for children aged 5 to 10 years in São Paulo; and by Kobayashi et al.,²⁰ who studied the validity of an FFQ for Japanese children aged 3 to 11 years. The results from these studies showed that the intake quantities of energy and several nutrients shown by the FFQ were overestimates. In our study, comparing the differences between the mean values for energy availability and vitamin A and C intake estimated by the FFQ and R24h, it was also observed that the quantities found were overestimates. This was observed when the test for differences in means or the correlation coefficient adjusted for energy consumed was used.

It is possible that these results may be indicating that there was an error in the information on the quantity of nutrient portions consumed. This supposition may be justified by the comment made by Willett,¹⁷ who stated that the adjustment for total calories decreased the correlation coefficient when the variability of nutrient intake was related to systematic overestimation and/or underestimation errors. The explanation for this over-estimation that was identified in relation to the FFQ may be attributable to the number of foods contained in the study tool.

However, it should be noted that other validation studies have presented low values for the correlation, like those in the present study, and that these results may have come from information bias relating to both the FFQ and the R24h.²¹

In this respect, it can be commented that even the method used in the present study as the gold standard has limitations regarding both overestimation and underestimation of the portions consumed. Like in other investigations, it can be understood that one of the main limitations of the present study was that it used the R24h as the reference method, given that like the FFQ, this method also depends on the interviewees’ memories, which may result in either underestimation or overestimation of food intake.

Most of the nutrients studied were shown to present fair concordance according to the kappa coefficient. In this respect, our results are similar to those found in a study conducted on 91 children aged 6 to 10 years who were living in Porto Alegre⁸, or to the study by Fumagalli et al⁷, whose method for recording three days provided kappa coefficient estimates that were classified as weak for the majority of nutrients.

Nonetheless, in our study, the Bland-Altman plots showed that the dispersion between the mean intakes estimated from the two methods presented a good concordance trend for some nutrients like lipids, proteins and zinc. From visual analysis on the plots, it was noted that the points generated were very close to zero and the numbers of points outside of the lower and upper limits of the plot baseline were small, thus indicating that according to this approach, the concordance between the tools was adequate.¹⁹ In a study conducted with of children and adolescents living in southern Spain, the Bland-Altman plot was used to validate the FFQ and R24 h and was found correlation between the instruments for energy and macronutrients. The graphical presentation of our results for proteins and lipids are similar to those found by Spanish authors.²²

It has to be acknowledged that the capacity for comparisons of results between these validation studies is limited, particularly because of the differences in the reference methods used. These reference methods usually consist of the R24h or a food record, and may vary in terms of the number of days of data gathering, the reference periods used and the use made of nutrient intake estimates or the frequency of consuming foods or food groups, in assessing the diet.

Some validation studies conducted among children and adolescents in developed and developing countries have used the food record method as the gold standard^6,23. However, among populations with low schooling levels, which is the reality for a large proportion of the population in many regions of Brazil, it becomes impossible to use this method. Despite all the limitations of the R24h that have been mentioned, it has been indicated to be the most appropriate method for validation studies among populations with low income and low schooling levels, when it is conducted in several steps.²⁴ Use of this tool may minimize memory bias and standardize the interviews.

Even though no method can be considered to be a gold standard for directly assessing the validity of FFQs, the food record and R24h methods have been the ones most used for validations. The R24h method is less demanding than the food record method and is less liable to influence the subjects’ real diet. Its sources of error tend to be more related to errors within the tool, such that responses to the questions asked depend on the interviewee’s memory and motivation and the estimates of food portion sizes.

Thus, the results suggest that the FFQ presented satisfactory validity and can be used in studies for assessing the food consumption of children in the age range of 4 to 11 years. Nevertheless, for it to be used in different geographical areas, procedures for adapting the questionnaire before applying it will be necessary.