Development and validation of a food frequency questionnaire for dietary intake assessment among multi-ethnic primary school-aged children

Abstract

INTRODUCTION

This study aimed to develop and validate a food frequency questionnaire (FFQ) to assess habitual diets of multi-ethnic Malaysian children aged 7–12 years.

METHODS

A total of 236 primary school children participated in the development of the FFQ and 209 subjects participated in the validation study, with a subsample of 30 subjects participating in the reproducibility study. The FFQ, consisting of 94 food items from 12 food groups, was compared with a three-day dietary record (3DR) as the reference method. The reproducibility of the FFQ was assessed through repeat administration (FFQ2), seven days after the first administration (FFQ1).

RESULTS

The results of the validation study demonstrated good acceptance of the FFQ. Mean intake of macronutrients in FFQ1 and 3DR correlated well, although the FFQ intake data tended to be higher. Cross-classification of nutrient intake between the two methods showed that < 7% of subjects were grossly misclassified. Moderate correlations noted between the two methods ranged from r = 0.310 (p < 0.001) for fat to r = 0.497 (p < 0.001) for energy. The reproducibility of the FFQ, as assessed by Cronbach’s alpha, ranged from 0.61 (protein) to 0.70 (energy, carbohydrates and fat). Spearman’s correlations between FFQ1 and FFQ2 ranged from rho = 0.333 (p = 0.072) for protein to rho = 0.479 (p < 0.01) for fat.

CONCLUSION

These findings indicate that the FFQ is valid and reliable for measuring the average intake of energy and macronutrients in a population of multi-ethnic children aged 7–12 years in Malaysia.

INTRODUCTION

As childhood is a major period of growth during which eating habits are formed(1), it is important to make accurate assessments of food intake in children.(2) Such evaluations can determine potential dietary problems and allow corrective action to be taken to change bad eating habits.(3) There is, therefore, an increasing need for reliable intake measurements of foods that are consumed as part of a regular diet. Developing valid, efficient and cost-effective tools to assess the nutritional intake of children is a key research priority.(4) Till this is achieved, accurate assessment of the dietary intake of free-living individuals, especially that of children, remains a challenging process.(5) No single method to assess an individual’s habitual dietary intake is optimal in all situations. Methods should be determined based on the aim of the study, proficiencies of the study population, accuracy and consistency of the data required, and resources available.(5)

The food frequency questionnaire (FFQ) is often used in epidemiological studies to assess long-term eating habits.(6) Although this method may be less accurate than weighed dietary records and 24-hour diet recalls, it is a simple tool to assess the habitual dietary patterns of many subjects and is useful for large-scale surveys.(7) FFQs are also useful to assess dietary intake in a large sample from different settings, for which a quick and straightforward method is needed to estimate the intake of specific nutrients. FFQs have been established as a practical and cost-effective approach for assessing habitual dietary intake over long periods and in large-scale dietary surveys.(5)

Due to a lack of a gold standard for estimating dietary intake, FFQs are typically validated by comparing their results with reference methods such as 24-hour diet recalls or dietary records.(8) According to Bae et al, the validation study is important because the use of an FFQ with low validity might lead to a false relationship between dietary factors and diseases or disease-related markers.(9) It has been suggested that the FFQ should be validated within the target population, as food consumption habits vary according to the ethnicities and social backgrounds of subjects.(10) It is therefore important to make sure that a newly developed FFQ is suitable for a specific group, especially in terms of the types of foods that children consume. It is also essential to determine whether the FFQ can provide reproducible results. The reproducibility of an FFQ can be assessed by administering it twice to the same group during the review period.(11)

Based on our literature search, no FFQ has been developed and validated for assessing the dietary intake of the primary school population in Malaysia. We therefore aimed to develop and validate an FFQ to assess regular dietary intake among multi-ethnic Malaysian children aged 7–12 years. This study describes the process of developing the FFQ and establishing its validity and reproducibility.

METHODS

This study is part of the Nutrition Survey of Malaysian Children, which is among a series of surveys known as the South East Asian Nutrition Surveys (SEANUTS).(12) SEANUTS is a multi-centre study carried out among 16,744 children aged 0.5–12 years, across four countries.(13) The FFQ study was conducted prior to data collection for SEANUTS and was divided into three phases: development study (Phase 1), validation study (Phase 2) and the reproducibility study (Phase 3). Ethical approval was granted by the Medical Research and Ethics Committee of Universiti Kebangsaan Malaysia and the study protocol was registered in the Dutch Trial Registry (NTR2462). Written informed consent was obtained from the parents and verbal assent was obtained from the children prior to data collection.

Subjects in this study were not participants of the main Nutrition Survey of Malaysian Children. A total of 236 subjects (105 boys, 131 girls) were recruited in Phase 1 of the study and another 209 subjects (91 boys, 118 girls) participated in Phase 2. Subjects were recruited from primary school students aged 7–12 years. They comprised the main ethnicities living in Malaysia — namely Malays, Chinese, Indians, and Sabah and Sarawak bumiputras (natives) — and were selected using purposive sampling to match the proportions of ethnicities in the population and for the relevant age group.(14) The reproducibility study (Phase 3) was assessed in a subsample of 30 subjects who participated in Phase 2, through repeat administration of the FFQ (FFQ2) seven days after the first administration of the FFQ (FFQ1). The inclusion criteria were: children who were (a) Malaysian; (b) aged 7–12 years; and (c) mentally and physically healthy. Children who were not enrolled in school and those who had genetic problems or physical disabilities were excluded. For subjects aged 7–9 years, questionnaires were completed by their parents or caregivers as proxies. Subjects aged 10–12 years answered the questionnaires by themselves under parental guidance. Height, weight, waist and hip circumference measurements were taken using calibrated tools after the subjects had completed the questionnaires.

Development of the FFQ was divided into three major steps: (a) the creation of a food list after a pilot 24-hour diet recall study; (b) prioritisation and categorisation of food items; and (c) assignment of food frequency intake and portion size. A pilot study using 24-hour diet recalls was conducted among 236 primary school children from various states in Malaysia. The group of children who participated in the development study did not participate in the validation study. Information on food intake during weekdays and weekends was obtained through interviews with the children’s parents or caregivers, who acted as their proxies. A food photo album was used as a guide during the interview to determine sizes of food portions.(15) A total of 318 food items, from food intake on weekdays and weekends, were derived from the 24-hour diet recalls. Similar foods and beverages were combined into single food items. The mean intake from each food item was calculated and percentages of energy and nutrients were derived using the Block equation.(16) We were especially interested in total energy, protein, fat and carbohydrate (CHO) intake. The nutrients contributed by each food item were determined and the cumulative percentage of energy and macronutrients was calculated.

A total of 94 food items were determined from the analysis and organised into 12 main food groups: (a) cereals and cereal products; (b) meat and meat products; (c) fish and seafood; (d) eggs; (e) legumes and legume products; (f) milk and dairy products; (g) vegetables; (h) fruits; (i) confectionary; (j) beverages; (k) spreads; and (l) seasonings and flavourings. For easier administration, items within each food group were further grouped into smaller categories based on similar food types. Individual food items that could not be incorporated into larger food categories remained as is. Following Norimah and Margetts’ method,(17) food items accounting for up to 90% of the total energy and macronutrient intake and those that appeared more than 20 times in 24-hour diet recalls were selected to be included in the FFQ.(18) The food item at the 90% mark was also included.

Frequency of intake was evaluated based on habitual intake over the previous month. Food frequency consumption of each item was evaluated using eight categories: (a) never; (b) 1–3 times per month; (c) once a week; (d) 2–4 times per week; (e) 5–6 times per week; (f) once a day; (g) 2–3 times per day; and (h) ≥ 4 times per day. Each food item in the FFQ was assigned a portion size using local household units such as plate, bowl, tablespoon, etc, according to the Nutrient Composition of Malaysian Foods(19) and the Atlas of Food Exchanges and Portion Sizes.(15)

For Phase 2, a team of nutritionists and dietitians reviewed the newly developed FFQ to confirm its content validity. Data was collected through the administration of written questionnaires. The two steps in the Phase 2 study were FFQ administration and the collection of a three-day dietary record (3DR). The FFQ was completed by the adult or proxy who spent the most time with the child (usually the mother). Instructions on how to complete the FFQ were provided. In the FFQ forms, subjects were first asked if they had eaten a particular food item in the previous month. If they had consumed the food item, they were asked how often it was taken and the serving size that was usually consumed at each time. The 3DRs for children aged 7–9 years old were completed by the same adult or proxy who completed the FFQ. Subjects aged 10–12 years filled in the forms by themselves under parental guidance. All foods that had been eaten for three consecutive days (two weekdays and one weekend), including school days, were recorded. Subjects were asked to give detailed descriptions of each food item eaten whenever possible, including portion size, preparation method, recipe, brand and time of food and beverage consumption. Clear instructions on how to fill in the 3DR form and food portion photos were provided to assist parents in completing the form. Subjects were also instructed to follow their usual diet. The 3DR forms were collected approximately one week after the completion of the FFQs. Reproducibility was measured by repeat administration of the FFQ (FFQ2) to 30 subsamples of subjects who had completed their first FFQ (FFQ1). FFQ2 was administered seven days after FFQ1.

To analyse nutrient intake, food items recorded in the 24-hour diet recall forms were converted to grams and then to nutrient intake based on the Nutrient Composition of Malaysian Foods(19) and Atlas of Food Exchanges & Portion Sizes(15), using the Nutritionist Pro™ software (Axxya Systems LLC, Stafford, TX, USA). Dietary data from the 3DRs was analysed using the same software to obtain energy and nutrient values for each subject. The amount of daily food intake was calculated from the FFQ according to the following formula: frequency of intake (conversion factor) × serving size × total number of servings × weight of food in one serving. The conversion factor used to estimate food intake was based on the frequency of intake, as shown in Table I. Total energy and nutrient intake were calculated using our own validated Excel-based platform. For food items that were not available in the Nutrient Composition of Malaysian Foods,(19) such as pork and some cooked foods, The Composition of 200 Foods Commonly Eaten in Singapore(20) and nutritional food labels were consulted.

Table I.

Conversion factors for frequencies of intake reported in the food frequency questionnaires.

The mean ± standard deviation (SD) intake was calculated for energy and each macronutrient (i.e. CHO, protein and fat). Frequency statistics were computed for other descriptive characteristics of the subjects. The normality of the data in the validation study was tested using the Kolmogorov-Smirnov test. For the reproducibility study, we used the Shapiro-Wilk test. The mean intake values derived from the FFQ and 3DR were compared using paired t-test. To determine whether the intake derived from the FFQ was related to the intake derived from the 3DR, Pearson correlation coefficients were calculated. Spearman’s correlation coefficients were computed to determine the strength of the relation between FFQ1 and FFQ2. We utilised the Bland-Altman plot to assess the agreement between the FFQ and 3DR across a range of nutrient intake. A cross-classification analysis was used to determine whether there was good agreement between the FFQ and 3DR, and whether participants had been misclassified into incorrect categories based on intake levels. To perform the cross-classification analysis, intake for energy, CHO, protein and fat derived from the FFQ and 3DR were divided into quartiles. Subjects were classified as belonging to the same quartiles, belonging to the same and adjacent (± 1) quartiles, or entirely misclassified according to the FFQ and 3DR assessments. The reproducibility of the FFQ was assessed using Cronbach’s alpha. A p-value < 0.05 was considered statistically significant.

RESULTS

Among the foods that were selected for inclusion in the FFQ, 18 food items contributed up to 90% of total energy intake; 13, 17 and 16 food items contributed up to 90% of protein, CHO and fat intake, respectively (Table II). Of the 233 primary school children recruited for Phase 2, only 209 subjects successfully completed both the FFQ and 3DR. Nine subjects were excluded because they were unable to complete both the FFQ and 3DR, and another 15 subjects were excluded because their FFQs and 3DRs were unlikely to represent habitual intake. The physical characteristics of the subjects are shown in Table III.

Table II.

Food items that contributed up to 90% of total energy, carbohydrate, protein and fat intake.

Table III.

Physical characteristics of the subjects in the validation study (n = 209).

Table IV shows a comparison of the energy and macronutrients in the FFQ and 3DR and the correlations between the two methods. The FFQ produced a significantly higher intake of energy and macronutrients (average relative difference of 20.4%) than the 3DR. Compared with the 3DR, the most marked overestimation of nutrients observed in the FFQ was for CHO (30.8%). On average, the Pearson correlation of absolute nutrient intake between the two dietary assessments was r = 0.416, ranging from r = 0.310 (fat) to r = 0.497 (energy). Table V shows the mean daily intake of energy and macronutrients and the correlation between the two FFQs. The mean intake of energy and macronutrients in FFQ1 was higher than FFQ2, except for fat intake. Spearman’s correlation coefficients showed moderate correlations between FFQ1 and FFQ2, ranging from rho = 0.333 (protein) to rho = 0.479 (fat). There were significant differences in the intake of energy, CHO and fat.

Table IV.

Comparison of means and correlations for intake of energy and macronutrients in the first administration of the food frequency questionnaire (FFQ1) and three-day dietary record (3DR) (n = 209).

Table V.

Comparison of means and correlations for intake of energy and macronutrients between the first administration (FFQ1) and the second administration of the food frequency questionnaire (FFQ2) in a subsample (n = 30).

The intake of energy and macronutrients from both methods were distributed into quartiles of intake and cross-classified. Table VI presents the percentage of subjects who were correctly classified into the same quartiles and adjacent quartiles, and subjects who were grossly misclassified by both methods. Subjects were categorised as grossly misclassified if they were ranked at the extremes. An average of 76% of children, ranging from 70.3% (fat) to 84.2% (energy), were classified into the correct quartile and less than 7% of them were grossly misclassified when ranked by the FFQ and 3DR. Cronbach’s alpha values for the intake of energy and macronutrients based on FFQ1 and FFQ2 ranged from about 0.6 to 0.7 (Table VI).

Table VI.

Relative validity and internal reliability of the food frequency questionnaire (FFQ).

The Bland-Altman method was used to evaluate the agreement between the FFQ and 3DR results by plotting the individual differences between the two administrations of the FFQ against the mean of the measurements.(21) The energy and macronutrients showed a similar pattern (Fig. 1); differences in absolute intake between the two methods were equally distributed. All four scatter plots were predominantly distributed within the 95% limits of agreement. The positive and negative values in the differences between the assessments imply over- and underreporting, respectively, of the FFQ compared to the 3DR.(22)

Fig. 1.

Bland-Altman plots show agreement between the first administration of the food frequency questionnaire (FFQ1) and three-day dietary records (3DR) for (a) energy; (b) carbohydrate; (c) protein; and (d) fat intake. The thicker line represents the mean difference in absolute intake between the two methods and the thinner lines represent the limits of agreement (± 2 standard deviation [SD]).

DISCUSSION

In many epidemiological studies, FFQs originally developed for adults are used to assess habitual food intake in children.(23,24) In contrast, the food list in our FFQ was developed specifically for children, using a data-based approach. Such an FFQ is more likely to provide representative dietary data for Malaysian school-aged children. Children aged 7–12 years were included in the FFQ developmental process in order to compile a list of the 94 most relevant food items in the diets of Malaysian children. Food items that contributed up to 90% of the total energy and macronutrient intake were defined, with white rice, chicken and fried rice being the main contributors to total energy intake (Table II). White rice contributed the most to energy consumption as it is the staple food for Malaysians.(25) Rice, cereals and cereal-based products (i.e. white rice, fried rice, white bread and noodles) composed 62.6% of the total CHO intake. Overall, chicken, fish and seafood were the major contributors to total fat intake (32.3%) and contributed 39.5% of the total protein intake in the diet of the Malaysian children in the study.

We chose to develop a semi-quantitative FFQ that estimated food intake using categories rather than exact frequencies, as it has been shown that loss of information is minimal when food intake categories are used to estimate nutrient intakes.(26) This study also asked the respondents about their intake of pre-specified portion sizes rather than estimates of their regular portion sizes, which made the FFQ easier to administer and more reliable.

Percentages of mean differences were used to determine the agreement between the FFQ and 3DR methods. According to Norimah and Margetts, good agreement between two methods is achieved if the percentages of the mean differences are less than 10%.(17) In this study, the mean differences of energy and macronutrients were greater than 10% between each method, showing a moderate agreement between the methods. However, a study by Fumagalli et al also showed mean differences of more than 10% for energy (19.9%) and fat (64.5%).(23) The length of the food list in the FFQ may be a factor in this problem. Subjects tend to choose more food items when a lengthy food list is used. Conversely, choices are restricted when a shorter food list is used.(10) The number of food items on our FFQ is 94, which we consider to be the optimal number of items that can provide an accurate picture of dietary habits without having a negative impact on the study participants. Thompson and Byers stated that a good FFQ should comprise between 90 and 100 food items in order to give an accurate representation of energy and nutrient intake, as FFQs with food lists that are too long or too short tend to overestimate and underestimate intake, respectively.(27) The highest overestimation in this study, which was observed in CHO intake (30.8%), indicates that some respondents may have had difficulty in estimating portion sizes of food. This could be due to a dependence on third parties (caregivers or parents) to report the intake of the children.(28)

In this study, overall correlations for energy and macronutrients between the FFQ and 3DR were considered fair. Hankin et al suggested that correlation coefficients between dietary methods should be considered poor if correlation coefficients in most dietary validations are less than 0.30, fair if they are between 0.30 and 0.49, and good if they are over 0.50.(29) The Pearson correlation coefficients obtained in this study showed a moderate correlation range of between 0.310 (fat) and 0.497 (energy). A study by Kobayashi et al also showed moderate to high correlations, in the range of 0.33 (energy) to 0.66 (protein).(7) However, the performance of an FFQ cannot be determined by correlation alone, as there may be high day-to-day variability in food consumption. Moreover, correlations cannot assess the agreement between two methods.(10) Hence, cross-classification and Bland-Altman plots were applied to assess the relative validity of the FFQ against the 3DR.(30) The two methods showed a satisfactory level of agreement in terms of dietary assessments. Compared to correlation coefficients, cross-classification can give a much clearer and undistorted picture of how well an FFQ performs.(10) It can also assess the ability of the FFQ and 3DR methods to classify the same individuals in the same categories of nutrient intake.(31) Cross-classification analysis was performed to classify nutrient intake among subjects, in order to determine the agreement between the FFQ and reference methods.(32) The degrees of misclassification in this study were small for both energy and macronutrient intake. The proportion of subjects correctly classified by the FFQ into the same or adjacent quartiles as the 3DR ranged from 70.3% (fat) to 84.2% (energy); few subjects were grossly misclassified, ranging between 2.4% (energy and CHO) and 6.7% (protein and fat).

The Bland-Altman plot was used to obtain further information on the agreement between the FFQ and dietary records through graphical presentation. Interpretation of the Bland-Altman plots was based on the categories defined by Tang,(33) in which ‘good agreement’ occurs when the difference between the two measurements is approximately equal to one SD of the average nutrient intake in the reference method; ‘fairly good agreement’ occurs when the difference between the two measurements is approximately equal to two SDs of the average nutrient intake in the reference method; and ‘poor agreement’ occurs when the difference between the two measurements is approximately equal to three SDs of the average nutrient intake in the reference method.(33)

An important factor when determining FFQ reliability is the time interval between the administrations of the FFQs. If the time interval is short, the subject may be able to remember the first FFQ and reliability is overestimated. However, if the time interval is too long, dietary patterns may change, affecting the study’s reliability.(34) The energy, CHO and protein intake reported were higher in FFQ1 compared to FFQ2. In general, food intake during the first administration of an FFQ is higher than that during the second.(35) Moreover, children tended to report a higher frequency of intake in the first administration compared with that in subsequent FFQs.(35) This study showed a moderate correlation between both FFQs. Cronbach’s alpha value demonstrated good agreement between FFQ1 and FFQ2. According to Santos, a value for Cronbach’s alpha greater than 0.70 shows good agreement between two measurements.(36)

However, some limitations associated with this study must be considered. Firstly, no objective measurement was used to validate the FFQ, which centres on energy and macronutrient intake. The 3DR is based on subjective measurement and was chosen as the reference method because of its high level of accuracy when validated for assessing dietary intake in infants and children.(37) Moreover, the measurement errors of the 3DR and the FFQ are highly independent, because unlike the FFQ method, the 3DR method does not depend on memory, is open-ended and involves direct estimation of portion size.(26) In addition, assessment of energy and macronutrient intake in children is further complicated by various factors, including weight status, dependence on third parties such as caregivers, parents or teachers (as proxies to report the intakes of children), the cognitive abilities of children, and motivation and bias in reporting food intake as influenced by growth or weight status.(28) Another reasonable explanation for why children have difficulty completing their reports is that they have not yet attained the ability to think abstractly.(38)

In conclusion, a self-administered, semi-quantitative FFQ consisting of 94 food items and 12 food groups was developed for Malaysian school-aged children. It was found to be valid and reliable for measuring the average intake of energy and macronutrients among children aged 7–12 years. We are confident that this FFQ can serve as a basic tool for further dietary assessments in a multi-ethnic children’s population in Malaysia.