Relative validity and reproducibility of a parent-administered semi-quantitative FFQ for assessing food intake in Danish children aged 3–9 years

Abstract

Objective

To assess the relative validity and reproducibility of the semi-quantitative FFQ (SFFQ) applied in the evaluation of a community intervention study, SoL-Bornholm, for estimating food intakes.

Design

The reference measure was a 4 d estimated food record. The SFFQ was completed two times separated by a 1-month period in order to test reproducibility.

Setting

The Capital Region and the Regional Municipality of Bornholm, Denmark.

Subjects

A total of fifty-four children aged 3–9 years were enrolled in the study.

Results

In terms of validity, the SFFQ generally overestimated intakes compared with the food records, especially for vegetables. For most intakes, the mean difference increased with increasing intake. Gross misclassification was on average higher for energy and nutrients (17 %) than for foods (8 %). Spearman correlation coefficients were significant for twelve out of fourteen intakes, ranging from 0·29 to 0·63 for foods and from 0·12 to 0·48 for energy and nutrients. Comparing the repeated SFFQ administrations, the intakes of the first SFFQ were slightly higher than those of the second SFFQ. Gross misclassification was low for most intakes; on average 6 % for foods and 8 % for energy and nutrients. Intra-class correlations were significant for all intakes, ranging from 0·30 to 0·82 for foods and from 0·46 to 0·81 for energy and nutrients.

Conclusions

The results indicate that the SFFQ gives reproducible estimates. The relative validity of the SFFQ was low to moderate for most intakes but comparable to other studies among children.

An adequate diet is essential for optimal well-being and development during childhood and for the prevention of disease^{( 1 , 2 )}. Therefore, accurate assessment of children’s dietary intake has become important for monitoring the nutritional status of the population, as well as for conducting interventional and epidemiological research on the link between diet and health among children.

Dietary assessment in young children is particularly challenging, and it is important to develop suitable methods for this age group. Below the age of 8 years, children are not capable of reporting their own dietary intake due to limited cognitive and literacy levels. Therefore, a proxy reporter, typically the parent, needs to be involved, which poses additional challenges^{( 3 , 4 )}.

The estimated food record (FR) has been recommended for dietary assessment among young children^{( 5 )}. The use of FR is labour intensive, but enables the use of several proxy persons to provide accurate collection of dietary information, for example parents and day-care staff. In large studies the FFQ is the preferred method because administration is easy and costs are low. However, FFQ should be validated in the target population^{( 6 )}. The validity of FFQ is likely to be lower in younger children but the evidence is scarce^{( 7 )}. Only a handful of studies have examined the relative validity of parent-administered FFQ among children below the age of 8 years^{( 8 – 16 )} and very few have examined food group intakes^{( 11 , 13 )}. Furthermore, few studies include a reproducibility test^{( 10 , 12 , 13 , 17 )}. Reproducibility studies are difficult to design but are equally important^{( 6 )}.

As a part of the Danish community intervention study, ‘Health and Local Community’ (SoL-Bornholm), a questionnaire to assess food intake in 3- to 8-year-old children was developed. Currently, Danish FFQ for adults exist but no (published) Danish FFQ has been developed for young children. This is therefore, to our knowledge, the first Danish FFQ to be validated for this age group.

The purpose of the present study was to assess the relative validity and reproducibility of the semi-quantitative FFQ (SFFQ) for its application in the evaluation of the community intervention study, SoL-Bornholm, in Denmark. The assessment gave priority to food groups relevant to the SoL intervention study (listed below).

Methods

Participants and design

In September 2012 the community-based health promotion intervention study, SoL-Bornholm, was initiated. The main purpose of the intervention study was to promote healthy eating and physical activity among children. Important success criteria were increased intakes of vegetables, fruits, fish and whole grains, and decreased intakes of soft drinks and candy. In order to assess the food intake among participating children, a parent-administered FFQ was developed for the purpose and tested during the year 2013.

A convenience sample was recruited through postings on school intranets, in day-care centres in the SoL-Bornholm communities and in neighbourhoods in the Capital Region of Denmark. Recruitment from the Capital was necessary since very few families from the intervention setting were able to be recruited in this way. The majority of the final sample originated from the Capital. The sample was, however, comparable to the actual final users with regard to age and food culture. Children aged 3–9 years were included. Before the start of the study, parents were invited to a group meeting where they were given detailed instruction by a trained nutritionist on how to fill out the FR. The instruction used examples of child meals and how to estimate portion sizes. Throughout the study, the parents received reminders by email or text message, and telephone support if necessary. The study was carried out at the Research Centre for Prevention and Health.

The FR and SFFQ were entered by a nutritionist, following standardized procedures. All records and questionnaires were screened for completeness and parents were contacted to provide clarifications in the case of missing data or misunderstandings.

Semi-quantitative FFQ

A parent-administered SFFQ was developed based on an SFFQ previously used and validated in a nationwide survey among children in Norway^{( 11 )}. The SFFQ was translated and modified to the Danish language. It was also adapted to Danish food culture as well as to the target age group using data from national Danish food intake surveys among children^{( 18 )}.

The nineteen-page questionnaire included twenty-two questions covering 183 food items grouped together according to the Danish meal pattern (Western breakfast, open sandwich for lunch, hot meal for dinner). The recall period was the past 4 weeks, and the frequency alternatives varied from ‘never/less than once per month’ to ‘five or more times a day’. A photographic booklet including sixteen series of colour photographs with four different sizes of portions of meals or food items, ranging from small (A) to large (D), was used by the parents when reporting the amounts of food eaten by the child. The photographic booklet originally included six differently sized portions (A–F) and was developed to assist food reporting among children and adults aged 4–75 years^{( 19 )}. The photographs have been validated among 622 adults and 109 children aged 8–12 years^{( 19 )}. When no photograph was available for a food item, household units were used, e.g. slices, pieces and spoons. The questionnaire consisted of seven sections: (i) introductory questions (height, weight, birth date); (ii) breakfast; (iii) bread (including toppings on bread); (iv) fruits; (v) hot meals (including vegetables); (vi) snacks/sweets; and (vii) beverages. Summary questions about entire food groups were added to the questionnaire to prevent misreporting. A summary question was, for example, ‘How often and how many pieces of fruit does your child eat?’

A one-page information letter was mailed to parents together with the questionnaire. This letter contained brief, but detailed information on how to fill out the questionnaire. It also encouraged parents to seek information on the child’s intake of foods and beverages when meals were eaten outside the home.

Approximately one month after completion of the SFFQ, the parents were asked to fill out the SFFQ once again in order to test its reproducibility. This interval was chosen to avoid seasonal variation. For convenience, the first administration of the SFFQ is called SFFQ1 and the second administration is referred to as SFFQ2.

Estimated food record (reference method)

The estimated FR was chosen as the reference method. This type of FR is characterized by being less time-consuming than the weighed FR because participants are asked to use household units or photographs instead of weighing all foods and meals. The FR covered four consecutive days, including three weekdays and one day during the weekend (Wednesday to Saturday). The parents were asked to register in detail all foods and beverages consumed by their child using photographs of portion sizes (the same picture booklet as described in the preceding section) or household units. Besides food amounts, the FR contained sections for registering time and location of meals, the food preparation method, fat percentage and food brand. The FR were filled out by parents shortly after the first administration of the SFFQ.

Nutrient calculations

Daily individual consumption was calculated from the SFFQ and FR. For the SFFQ, monthly frequencies were divided by 30·4375, and weekly frequencies by 7, to calculate the daily intakes in g/person per d. For the FR, daily intakes were calculated as the mean of the recorded days. Intake in grams was calculated by use of standard weights of food items and portion sizes^{( 19 , 20 )}. Estimation of the individual daily energy, nutrient and food intakes was calculated using the Danish Food Composition Databank version 7^{( 21 )} and the software program FoodCalc version 1·3^{( 22 )}. Total energy intake was calculated according to the Nordic Nutrition Recommendations^{( 23 )} with fat contributing 37 kJ/g, available carbohydrates and protein contributing 17 kJ/g and dietary fibre contributing 8 kJ/g.

Furthermore, the intakes of specific foods and food groups were calculated. Eight food groups were developed specifically to be used for the evaluation of the SoL-Bornholm intervention study. The grouping of foods into food groups was based on the standards developed by the National Food Institute, Technical University of Denmark^{( 21 )}. These food groups were then modified and supplemented to be more comparable with the Danish dietary guidelines^{( 24 )} and to suit the SoL-Bornholm intervention study. The adaptations were as follows: (i) for the fruit group, removing olives, jams, juices, dried fruit and nuts; (ii) for the vegetable group, removing juices, ketchup, potatoes, soya drinks and mushrooms; (iii) creating the following new food groups relevant for the SoL-Bornholm intervention study: coarse vegetables, canned fish, sweet drinks, candy and whole grains. The vegetable group served as a measure of total vegetable intake, whereas the group named coarse vegetables included only vegetables with high fibre content (above 2 g/100 g) such as cabbage, root vegetables and onions. Potatoes were excluded from the vegetable group because they do not count as a vegetable in the Danish dietary guidelines. Canned fish included all fish sold in cans or similar packaging, and was included because these fish products were a main intervention focus for supermarkets. The whole grains group included rye bread and oats as a marker of whole grain intake. For convenience, this food group is called whole grains in the following sections. Sweet drinks included all non-alcoholic drinks with natural or added sugars (e.g. juice, chocolate milk and lemonade), not including drinks with artificial sweeteners. The candy group included chocolates, liquorice, fruit gums and similar.

A weighting factor was calculated for each of the individual fifty-four children using SFFQ data and the summary questions^{( 25 )} for the following food categories: cold meals (open sandwich); fruit; vegetables; and hot meals. The answers from the summary questions were divided by the sum of frequencies or amounts from the FFQ, for each food group, as shown below:

For example, the weighting factor was 0·92 for one of the children’s fruit intake. The calculation was based on the calculated intake from the summary question (2 portions of fruit, 6–7 times/week), which was 185·7 g, and the sum of the reported fruit intake from question 15 about intake of different types of fruit, which was 201·55 g. When applying the weighting factor the adjusted fruit intake in grams per day was therefore 10 % smaller.

Previous studies have shown that applying weighting factors often leads to better estimates of nutrient and food intakes^{( 25 , 26 )}.

Statistical analyses

All statistical analyses were performed using the SAS statistical software package version 9·4. Descriptive statistics were used to describe baseline characteristics and average intakes of foods, energy and nutrients. Most of the intake data were non-normally distributed both before and after log-transformation, and therefore the mean was presented along with the median and the 5th and 95th percentile.

The Bland and Altman limits of agreement (LOA) method was used to assess agreement (or bias) of the two administrations of the SFFQ and between the two methods^{( 27 )}. This method was used to visually inspect agreement using plots of the mean difference and the 95 % LOA. Mean difference was calculated as the mean of all individual differences between methods (or repeated measures); that is, mean (SFFQ1 – SFFQ2)/n or mean (SFFQ – FR)/n, where n=2. The 95 % LOA was calculated for each food or nutrient intake as the mean difference ±1·96 sd of differences. This measure represents a range of values within which 95 % of all differences between methods are expected to fall. Bland–Altman plots were drawn using the differences between the two measurements plotted against the means of the measurements^{( 27 )}. The middle line of the plot represents the mean difference and the two outer lines represent the 95 % LOA.

A cross-classification analysis was undertaken to calculate the percentage of observations classified into the same or opposite quartile of intake (referred to as gross misclassification in the following)^{( 28 )}. This was done for testing the relative validity as well as reproducibility. The linear association between the two dietary methods and between the first and second administration of the SFFQ was described using Spearman’s rank correlation coefficients^{( 28 )}. In addition, intra-class correlation coefficients were calculated to assess reproducibility. For the assessment of the relative validity, correlations were corrected for the day-to-day within-person variation using the de-attenuation method^{( 7 )}. The corrected correlation, r _c, was calculated as follows:

where r _o is the observed correlation, is the ratio of the within- and between-person variances and n is the number of replicates per person for the given variable. Within-person variation and between-person variation were calculated from replicated FR using the Proc Mixed model in SAS. Additionally, Spearman’s rank correlation coefficients were adjusted for energy and person-specific factors. The residual model was used for energy adjustment of correlations between the FR and the first administration of the SFFQ^{( 29 )}. Person-specific factors were defined as the child’s age category (pre-school or school) and sex. These covariates were included in a model, and Spearman’s partial rank-order correlation coefficients were used for analyses of relative validity and reproducibility.

Results

Fifty-four children were enrolled in the study. Fifty-four completed SFFQ1 (100 %), forty-eight completed SFFQ2 (89 %) and fifty-three completed FR (98 %) were available. One participant had only three fully recorded days in the FR, but of satisfactory quality, and was therefore included in the analysis. The rest were of satisfactory quality and none were considered to have recorded implausible intakes. None of the children had energy intakes below or above the thresholds suggested for quality check by Willett and Stampfer^{( 29 )}. The characteristics of the children are presented in Table 1.

Table 1.

Descriptive characteristics of the children included in the study (n 54), Capital Region and the Regional Municipality of Bornholm, Denmark, 2013

Characteristic	n	%
Girls	24	44
Boys	30	56
Age group
Pre-school children (3–6 years)	26	48
School children (6–9 years)	28	52
Parent reporter, mother	46	85
Parent reporter, father	8	15
	Mean	sd
Weight (kg)*	23·12	5·97
Height (cm)*	119·74	13·52

^*Self-reported by parents.

Relative validity of the semi-quantitative FFQ

Food, energy and nutrient intakes from the SFFQ and the FR are shown in Table 2. The mean difference between methods varied among intakes, but in general the intakes were overestimated when compared with the reference (Table 2). Vegetable intakes were particularly overestimated by the SFFQ (151 g for total intake (118 %) and 73 g for coarse vegetables (189 %)). A few intakes were underestimated, particularly the sugar energy percentage. The lowest degree of over- or underestimation was observed for intake of whole grain foods (−3 g (3 %)). When using Bland–Altman plots to examine agreement between methods, it was observed that the mean difference increased with increasing intake for most intakes. For most intakes, the differences were both positive and negative, implying that the participants both under- and over-reported at high intakes. This was, however, not the case for vegetables and energy, where high intakes were mostly related to over-reporting (Fig. 1). When inspecting the 95 % LOA (the outer lines in the plot) we observed that the 95 % LOA generally were wide, indicating that agreement was better at the group level than at the individual level.

Table 2.

Relative validity of daily intakes of foods, energy and nutrients estimated by the first administration of the parent-administered semi-quantitative FFQ (SFFQ1) and the average of the 4 d estimated food records (FR), and correlation between SFFQ1 and FR. A total of fifty-three Danish children aged 3–9 years, from the Capital Region and the Regional Municipality of Bornholm, were included in all analyses

	SFFQ1			FR			Mean difference (SFFQ – FR)		Correlation between methods
	Mean	Median	P5, P95	Mean	Median	P5, P95	Mean†	%	Spearman’s r	95 % CL	Variance ratio‡	Corrected Spearman’s r ‡
Vegetables and vegetable products (g/d)	278	265	147, 502	128	110	12, 227	151	118	0·46***	0·22, 0·65	3·17	0·62
Coarse vegetables (g/d)	112	99	46, 232	39	30	1, 102	73	189	0·40**	0·14, 0·60	4·43	0·58
Fruit and fruit products (g/d)	206	195	6, 384	166	172	4, 329	40	24	0·45***	0·20, 0·64	3·11	0·60
Fish and fish products (g/d)	32	25	1, 92	26	18	0, 80	5	18	0·63***	0·43, 0·77	3·24	0·85
Canned fish (g/d)	12	7	0, 44	14	3	0, 54	−2	−14	0·63***	0·42, 0·76	2·72	0·82
Rye bread and oats, ‘whole grains’ (g/d)§	88	92	30, 145	90	84	31, 173	−3	−3	0·58***	0·36, 0·73	1·83	0·70
Sweet drinks (g/d)||	125	71	10, 457	105	83	0, 255	21	20	0·29*	0·02, 0·52	1·46	0·34
Candy and chocolate (g/d)	17	15	5, 39	16	14	0, 43	1	9	0·31*	0·05, 0·54	9·61	0·57
Energy (kJ/d)	8783	8103	5443, 13 694	6487	6509	4642, 8486	2296	35	0·12	−0·16, 0·37	4·79	0·18
Dietary fibre (g/d)	26	26	17, 41	20	19	11, 29	7	35	0·25	−0·03, 0·49	3·41	0·34
Protein energy percentage (%)¶	15	15	11, 19	15	16	12, 19	−0·3	−2	0·30*	0·03, 0·53	5·54	0·46
Fat energy percentage (%)¶	34	34	25, 46	30	31	24, 37	4	13	0·48***	0·23, 0·66	2·92	0·63
Carbohydrate energy percentage (%)¶	53	53	44, 63	57	58	49, 64	−4	−6	0·43**	0·17, 0·62	3·55	0·59
Sugar energy percentage (%)¶	5	5	2, 10	7	7	1, 13	−2	−26	0·28*	0·01, 0·51	5·69	0·44

P5, 5th percentile; P95, 95th percentile; 95 % CL, 95 % confidence limits.

*P<0·05, **P<0·01, ***P<0·001.

^†{[Mean (SFFQ−FR)]/[mean (FR)]} ×100.

^‡Variance ratio is the ratio between within- and between-person variances. Spearman correlation coefficient was adjusted for within-person variance using the de-attenuation method.

^§Including rye bread and oats as a marker of whole grain intake.

^||Including all non-alcoholic drinks with natural or added sugars, not including drinks with artificial sweeteners.

^¶Intakes of macronutrients expressed as a percentage of energy: (macronutrient absolute value × energy conversion factor) ×100.

Fig. 1.

Bland–Altman plot assessing the relative validity of the parent-administered semi-quantitative FFQ (SFFQ) for assessing daily vegetable intake in Danish children aged 3–9 years (n 53), Capital Region and the Regional Municipality of Bornholm, Denmark, 2013. The difference in intake between the first administration of the SFFQ (SFFQ1) and the average of the 4 d estimated food records (FR) is plotted v. the mean intake from the two methods; —— represents the mean difference and – – – – – represent the 95 % limits of agreement corresponding to ±1·96 sd

The proportion of children correctly classified into the same quartile ranged from 26 % (sweet drinks) to 49 % (coarse vegetables; Table 3). Gross misclassification into the opposite quartile was lowest for foods, on average 8 %. In general, the cross-classification analyses on energy intake and macronutrient energy percentages revealed slightly lower percentages for classification into the same quartile and higher percentages for gross misclassification (Table 3).

Table 3.

Percentage of participants classified by the first administration of the parent-administered semi-quantitative FFQ (SFFQ1) into the same or opposite quartile of consumption as measured by the average of the 4 d estimated food records (FR) or the second administration of the SFFQ (SFFQ2). A total of fifty-three and forty-eight Danish children aged 3–9 years, from the Capital Region and the Regional Municipality of Bornholm, were included in analyses of SFFQ1 v. FR and SFFQ1 v. SFFQ2, respectively

	SFFQ1 v. FR		SFFQ1 v. SFFQ2
	Same quartile (%)	Opposite quartile (%)	Same quartile (%)	Opposite quartile (%)
Vegetables and vegetable products (g/d)	43	7	38	8
Coarse vegetables (g/d)	49	15	40	13
Fruit and fruit products (g/d)	47	11	54	8
Fish and fish products (g/d)	44	0	46	0
Canned fish (g/d)	41	6	54	8
Rye bread and oats, ‘whole grains’ (g/d)†	36	4	69	0
Sweet drinks (g/d)‡	26	7	52	4
Candy and chocolate (g/d)	36	12	33	8
Energy (kJ/d)	36	26	56	0
Dietary fibre (g/d)	34	18	42	17
Protein energy percentage (%)§	38	17	50	13
Fat energy percentage (%)§	43	14	58	0
Carbohydrate energy percentage (%)§	42	11	60	8
Sugar energy percentage (%)§	38	18	48	8

^†Including rye bread and oats as a marker of whole grain intake.

^‡Including all non-alcoholic drinks with natural or added sugars, not including drinks with artificial sweeteners.

^§Intakes of macronutrients expressed as a percentage of energy: (macronutrient absolute value × energy conversion factor) ×100.

All but two correlations were significant (Table 2). The highest degree of linear association was observed for intakes of whole grains and fish products (r=0·58–0·63), there was a less strong association for vegetables and fruit (r=0·40–0·46), and the lowest degree of association was observed for sweet drinks and candy (r=0·29–0·31). There was a tendency towards weaker correlations for energy and macronutrients and the correlation for energy was especially low (r=0·12). The ratio of within- and between-person variance measured from the 4 d FR was between 1·46 and 9·61, and the de-attenuated (corrected) correlation coefficients were therefore substantially higher than the crude correlations (Table 2). Adjustment for energy did not change the correlation coefficients considerably. The most important and largest change was observed for candy (crude value r=0·31, energy-adjusted value r=0·43). When correlations were adjusted for age category or sex, a small change was observed for correlation of energy intake (r=0·06 when adjusted for age category and r=0·08 when adjusted for sex), but otherwise no considerable changes were observed.

Reproducibility of the semi-quantitative FFQ

Food, energy and macronutrient intakes for the repeated SFFQ are shown in Table 4. Overall, intakes from SFFQ1 were slightly higher than those from SFFQ2 (Table 4). The intake of sweet drinks was especially higher for the first SFFQ (35 g (27 %)). Through visual inspection of the Bland–Altman plots, a tendency towards poorer agreement with high intakes was observed. This, however, was not the case for vegetables and whole grains. We also observed that the 95 % LOA (the outer lines in the plot) were wide, indicating that agreement was better at the group level than at the individual level.

Table 4.

Reproducibility of daily intakes of foods, energy and nutrients estimated by the first and second administration of the parent-administered semi-quantitative FFQ (SFFQ1 and SFFQ2, respectively), and correlation between SFFQ1 and SFFQ2. A total of forty-eight Danish children aged 3–9 years, from the Capital Region and the Regional Municipality of Bornholm, were included in all analyses

	SFFQ1			SFFQ2			Mean difference (SFFQ1 – SFFQ2)		Correlation between repeated SFFQ
	Mean	Median	P5, P95	Mean	Median	P5, P95	Mean†	%	Spearman’s r	95 % CL	ICC	95 % CL
Vegetables and vegetable products (g/d)	268	256	144, 502	233	228	83, 435	35	15	0·43**	0·17, 0·64	0·38**	0·12, 0·59
Coarse vegetables (g/d)	110	99	46, 232	91	89	23, 177	19	17	0·56***	0·32, 0·72	0·44***	0·19, 0·64
Fruit and fruit products (g/d)	195	192	7, 384	200	192	2, 373	−4	2	0·55***	0·32, 0·72	0·55***	0·32, 0·72
Fish and fish products (g/d)	30	22	1, 92	26	16	5, 79	5	18	0·72***	0·55, 0·83	0·81***	0·69, 0·89
Canned fish (g/d)	12	5	0, 44	10	4	0, 39	1	7	0·66***	0·45, 0·79	0·75***	0·60, 0·85
Rye bread and oats, ‘whole grains’ (g/d)‡	83	85	30, 139	79	87	25, 135	4	5	0·84***	0·73, 0·91	0·82***	0·70, 0·89
Sweet drinks (g/d)§	130	71	10, 457	95	55	13, 297	35	27	0·71***	0·53, 0·83	0·60***	0·39, 0·75
Candy and chocolate (g/d)	18	15	6, 39	16	13	3, 53	1	8	0·30*	0·02, 0·54	0·30*	0·03, 0·53
Energy (kJ/d)	8508	7990	5443, 13 694	7749	7581	4716, 12 146	759	10	0·71***	0·53, 0·82	0·66***	0·47, 0·79
Dietary fibre (g/d)	25	25	17, 37	23	23	14, 34	2	9	0·46***	0·20, 0·66	0·46***	0·21, 0·65
Protein energy percentage (%)||	15	15	12, 19	16	15	13, 21	−0·5	−3	0·51***	0·26, 0·70	0·53***	0·30, 0·70
Fat energy percentage (%)||	34	35	25, 46	34	34	24, 42	−0·1	−0·2	0·81***	0·68, 0·89	0·81***	0·69, 0·89
Carbohydrate energy percentage (%)||	54	54	44, 63	53	52	46, 63	0·6	1	0·74***	0·58, 0·85	0·78***	0·64, 0·89
Sugar energy percentage (%)||	6	5	2, 10	5	4	2, 9	0·7	13	0·57***	0·34, 0·73	0·62***	0·41, 0·77

P5, 5th percentile; P95, 95th percentile; 95 % CL, 95 % confidence limits; ICC, intra-class correlation coefficient.

*P<0·05, **P<0·01, ***P<0·001.

^†{[Mean (SFFQ1−SFFQ2)]/[mean (SFFQ1)]} ×100.

^‡Including rye bread and oats as a marker of whole grain intake.

^§Including all non-alcoholic drinks with natural or added sugars, not including drinks with artificial sweeteners.

^||Intakes of macronutrients expressed as a percentage of energy: (macronutrient absolute value × energy conversion factor) ×100.

The cross-classification analysis indicated a low rate of gross misclassification into the opposite quartile for most intakes, on average 6 % for foods and 8 % for energy and nutrients (Table 3). The proportion of children correctly classified into the same quartile of intake was highest for intakes of whole grains (69 %) and lowest for candy and vegetables (33 % and 38 %, respectively; Table 3).

Intra-class correlation coefficients between the first and second administration of the SFFQ were on average 0·58 for foods and 0·64 for energy and nutrients, and all were statistically significant (Table 4). Spearman’s correlation coefficients were generally similar to the intra-class correlations (Table 4). The food group candy and chocolate deviated from the rest with a markedly lower correlation (r=0·30). Correlations for vegetable intakes were also somewhat lower than the majority (intra-class correlation coefficient=0·38; Spearman correlation coefficient=0·43). The strongest correlations for food groups were observed for fish and whole grains (intra-class correlation coefficient=0·81–0·82). When Spearman’s correlation coefficients were adjusted for age category or sex, no considerable changes were observed.

Discussion

The relative validity of the SFFQ, when compared with a 4 d estimated FR, varied across the different food groups and nutrients (Tables 2 and 3). Most intakes were overestimated, which is a similar finding to the majority of previous studies. Only two of the similar studies identified found an overall underestimation of intakes^{( 12 , 13 )}, whereas seven observed an overestimation^{( 8 – 10 , 14 – 16 , 30 )}. Likewise, the validation study performed on the original Norwegian version of the SFFQ reported overestimation of intakes; however, these overestimations were not as large as those observed in the present study^{( 11 )}. A review reported that FFQ overestimated total energy intake by 2–59 % when compared with the doubly labelled water method^{( 31 )}. Conversely, dietary assessment using FR is typically related to underestimation of total energy intakes (19–41 %)^{( 31 )}. When testing an FFQ using FR as the reference, it is therefore not surprising that we observed an overestimation of most intakes.

The results of the present validation study point towards an overestimation of the child’s vegetable intake in particular. This can perhaps be explained by the difficulty parents face when attempting to remember the child’s intake of vegetables or when estimating portion sizes. It has previously been observed that there is a low correlation between parental reports of children’s fruit and vegetable intake and corresponding serum micronutrients (r=0·14–0·29)^{( 32 )}. It is also possible that some parents have reported vegetables that were also a part of mixed dishes, causing double reporting.

When assessing validity with the use of cross-classification into quartiles, the results pointed towards highest agreement among food group intakes (Table 3). Coarse vegetables, fruit, candy, energy, dietary fibre and macronutrient energy percentages were related to a high degree of gross misclassification into the opposite quartile (>10 %), which were higher than or comparable to other studies using quartiles^{( 13 , 14 )}. Despite this, the percentages for gross misclassification of vegetables, fish, whole grains and sweet drinks were comparable to (or lower than) a similar study among children^{( 13 )} and other studies performed among adolescents^{( 33 – 35 )}, all of which used quartile classification.

The use of correlation coefficients to assess validity is under debate but there is a general agreement that correlations above 0·5 are acceptable or good, and that correlations below 0·3–0·4 indicate a low degree of linear association^{( 6 , 28 )}. In the view of this, the majority of food group intakes can therefore be considered to have an acceptable validity for assessing intakes on a group level and for ranking of children according to food intakes. There are, however, concerns about the measurement of sweet drinks and candy. Few studies have reported results on correlation coefficients of compared food intakes. When compared with the original Norwegian version of the SFFQ, the results of the present study were not considerably different (r=0·26–0·69)^{( 11 )}. The Norwegian study, however, is not directly comparable since the children were 2 years old. One other study, performed in a similar age group, reported correlation coefficients for foods in the range of 0·23 to 0·62, comparable to the present SFFQ^{( 13 )}. Not surprisingly, the correlation coefficients improved after correction for within-person variance. A few other similar studies have reported de-attenuated correlations^{( 9 , 13 – 15 )} and among these only one study examined food intakes^{( 13 )}. The latter study observed a large variation in variance ratios (−16·25 to 9·87), but likewise higher correlations after de-attenuation.

The correlation coefficients for energy intake and macronutrient energy percentages were generally similar to or lower than those observed in other studies^{( 9 , 12 – 14 , 16 , 17 , 30 )}. All correlations increased after de-attenuation, which is comparable to what is observed in other studies^{( 9 , 14 , 15 )}. The correlation for energy intake improved to 0·23 when excluding observations from two children who exceeded the 95 % LOA in the Bland–Altman plot, but the association was still low compared with other studies. The low validity of the energy intake can be explained by a number of different factors. First, dietary assessment in young children is notoriously challenging. As mentioned above, a proxy reporter, typically the parent, needs to be involved, which poses additional challenges. The major challenges include: (i) recall bias; (ii) difficulties in estimating the child’s portion size; and (iii) limited supervision of meals when the child is out of the home^{( 3 , 4 )}. Recall bias is a common problem within retrospective methods and even though parents have been recognized as reliable proxy reporters, remembering the child’s diet places high demands on the parent^{( 4 , 31 )}. Recall bias poses a particular problem when the child spends much of his/her time away from home, which is the standard for most Danish children. A study among 3-year-old children found that the linear association between the FFQ and a food diary decreased markedly for energy intake when the number of meals eaten outside the home increased (r=0·38 for no meals outside the home to r=0·31 for four or more meals outside the home)^{( 16 )}. The problem is perhaps minimized when children bring their food from home, which is the case in the present study, but limited supervision of meals will remain a challenge. Estimating children’s portion sizes seems to be a problem across dietary assessment methods^{( 3 , 4 )} and this could also be a possible explanation for the low validity observed for energy intake. A study indicated that the dietary report is more reliable when both parents participate^{( 4 )}. In the present study, 85 % of the reporters were the mother. According to the additional analyses of person-specific factors, the relative validity of energy intake was affected by age and sex. The sample size was too small to analyse the validity of energy intake in subgroups, but informal tests indicated that the validity was better for children from day-care centres and for boys. If the validity is better in day-care children, this supports the hypothesis that limited supervision of meals is a problem when reporting children’s diet. It is likely that schoolchildren consume more meals and drinks out of the home, when visiting friends and family.

Another explanation, unrelated to the reporter, could be the duration of the recording period in the FR. For pragmatic reasons the FR was filled out for 4 d, but optimally a longer recording period would be needed to reflect a reliable intake^{( 36 )}. A recent study, with a large sample size, did observe a small insignificant difference in energy intake between 3 d and 7 d periods^{( 37 )}. The study was conducted in adults, so a longer period would probably be needed for children because of the higher within- to between-subject variation ratio. The problem of high day-to-day variability related to FR can be taken into account by using the de-attenuation method. Finally, a low validity of major food group intakes may have influenced the energy intake. As reported, intakes of sugary foods, sweet drinks and candy had a rather low validity. In addition, it was observed that the reported intake of meat and meat products had a low validity (r=0·21; not reported).

The reproducibility of the SFFQ was tested by comparing two administrations of the SFFQ with a 1-month interval. Mean difference was on average 9 %, indicating a small degree of overestimation by the first SFFQ and a good reproducibility at the group level (Table 2). There was a more moderate agreement at the individual level, as reflected in the wide 95 % LOA. The tendency of overestimation is also reported in other studies, and a previous review stated that the first administration tends to result in greater frequencies of consumption than subsequent administrations^{( 32 )}. This can probably be explained by the learning effect and the increased awareness of eating habits from filling out the FR.

The proportion of children correctly classified into the same quartile was of a similar magnitude to that observed in a reproducibility study in adolescents that also used quartile classification^{( 35 )}. In the present study, gross misclassification into the opposite quartile was below 10 % for most intakes, which is even less than in the adolescent study^{( 35 )}. All correlation coefficients were significant and comparable to those in other reproducibility studies performed among children^{( 10 , 12 , 13 )}. Only the intake of candy was associated with poor correlation. Reproducibility of this food group was not reported by the aforementioned studies among children, but was assessed in adolescents as ‘sugar, sweets and snacks’ (r=0·58)^{( 35 )}. Furthermore, it is possible that the Christian holiday, Ascension Day, during the first administration of the SFFQ might have influenced the present results. Two children had a much lower intake of candy and chocolate at the second SFFQ (lower than the 95 % LOA), which was possibly caused by increased consumption during the holiday.

It has previously been suggested that the sample size of a validation or reproducibility study should be at least fifty for the LOA method and a minimum of 100 subjects for assessing correlation coefficients^{( 6 )}. The sample size of the present study is therefore a considerable limitation of the study and most likely influences the results causing a higher degree of variation, for example larger sd and 95 % LOA. In the cross-classification analysis, gross misclassification of a few children can make a large difference to the percentages. It can therefore be hypothesized that a larger sample size would have led to a higher level of reproducibility and relative validity. A small sample size can also be related to sampling bias, for example when responders are better than non-responders when answering FFQ. An analysis of sampling bias was not within the scope of the study. The sample size of the present study was comparable to that in other studies testing validity or reproducibility of FFQ. Participation in these kinds of study is challenging since it is related to a high subject burden.

In the current study, the estimated FR was chosen as the reference instrument. Unlike the FFQ, the FR method is prospective, does not depend on memory, is open-ended and has direct estimation of portion sizes using household units. Therefore, the measurement errors of FR and FFQ are fairly independent, and we considered the estimated FR to be the best alternative when choosing between different dietary assessment methods^{( 6 )}. The FR method has other limitations however, for example a likely change of dietary habits^{( 38 )}. Under optimal research conditions, inclusion of an objective measure (e.g. a recovery biomarker) in addition to the self-reported intake data (the dietary assessment method) would be ideal. Unlike self-reported data, biomarkers are not prone to reporting bias and the measurement errors are therefore less correlated with the FFQ. The inclusion of expensive biomarkers was unfortunately not possible in the SoL-Bornholm study.

A limitation of the reproducibility study could be a possible memory effect during the completion of SFFQ2 as parents could possibly still remember what they filled in 4 weeks ago^{( 6 )}. It is, however, a challenge to conduct a reproducibility study on FFQ assessing food intakes covering a month, and an optimal approach is unlikely to exist. Having repeated the SFFQ2 a year after and during the same month would also have been an option, but when conducted in children this may not be the best approach considering the likely increase in energy and nutrient intakes^{( 23 )}. In addition, it would probably have led to lower response rates. Still, reproducibility studies are important and often neglected. A method that lacks reproducibility cannot be valid. When used in the context of a controlled intervention study with repeated measurements, it is of great importance that the study has an acceptable reproducibility in addition to validity^{( 6 , 39 )}.

The SFFQ tested in the present study had a low level of relative validity for some intakes, which needs to be taken into consideration when analysing results obtained using the SFFQ. The intake of vegetables had a high level of overestimation, intakes of sweet drinks and candy had the lowest correlation among the food groups, and the results for energy intake in particular indicated a low level of validity across statistical methods when compared with other studies. Concerning the reproducibility, an acceptable level was observed for all intakes, but for candy and chocolate. When considering that this food group also obtained a low validity score, the results for this food group must therefore be interpreted with caution when using the questionnaire for testing intervention effects in the SoL-Bornholm study. For all other food group intakes, the study results indicated that the SFFQ is a useful tool when assessing children’s food intakes on a group level in the SoL-Bornholm study. The SoL-Bornholm study was designed to test post-intervention differences between the intervention and control group and therefore reproducibility was more important than the ability to measure absolute intakes. The effectiveness analyses should be adjusted for energy intakes, age and sex to reduce potential variance from these variables.

For future use of the SFFQ, it is recommended that the food lists, frequency options and portion sizes are further developed. It is possible that some of the overestimation could have been avoided if the possibility of double reporting was reduced. If the low level of validity for energy intakes was caused by difficulties related to portion size estimation, it is possible that conversion to a non-quantitative FFQ would have increased the validity. This would, however, require that standard portions were developed for this age group in a representative sample.

In summary, the relative validity of the SFFQ was generally highest for foods, but overall low to moderate for most intakes, but comparable to findings of other studies among children. The reproducibility of the SFFQ was high or moderate for almost all intakes across statistical methods. The SFFQ can be used for ranking of children according to food intakes and the authors found that the SFFQ is a useful tool for evaluating the intervention effects of the SoL-Bornholm community intervention study. The use of the SFFQ to evaluate intakes of candy, energy and some of the macronutrient intakes is problematic and these intakes need to be interpreted with caution.