Test–retest reliability and validity of a web-based food-frequency questionnaire for adolescents aged 13–14 to be used in the Norwegian Mother and Child Cohort Study (MoBa)

Abstract

Background

The assessment of food intake is challenging and prone to errors; it is therefore important to consider the reliability and validity of the assessment methods.

Objective

The aim of this study was to analyze the reproducibility and validity of a developed food-frequency questionnaire (FFQ) for use among adolescents.

Design

In total, 58 students (aged 13–14) from four different schools in the southern part of Norway participated in the reproducibility study of filling out the FFQ 4 weeks apart. In addition, 93 students participated in the relative validity study where the FFQ was compared to 2×24-hour dietary recalls, while 92 students participated in the absolute validity study where the intakes of fatty acids and vitamin D from the FFQ were compared to fatty acids and 25-hydroxy-vitamin D₃ in whole blood.

Results

The median Spearman correlation coefficient for all nutrients in the test–retest reliability study was 0.57. The median Spearman correlation for all nutrients in the relative validity study was 0.26, while the correlations coefficients were low in the absolute validity study with n-3 fatty acid coefficients ranging from 0.05 to 0.25, and absent for vitamin D (r=0.000).

Conclusion

The test–retest reproducibility was considered good, the relative validity was considered poor to good, and the absolute validity was considered poor. However, the results are comparable to other studies among adolescents.

The diet of children and adolescents is recognized to be important for both current growth and development and future health (1, 2). In general, many of today's health problems, that is, overweight and obesity, development of metabolic syndrome, diabetes, and a number of cancers, are related to what we eat (3). Evidence supports that the nutrition-related risk factors for chronic diseases start in early childhood and increases in adolescence (4). Diet is a modifiable risk factor which makes it important to investigate, and it makes both children and adolescents important groups of study. The diet of Norwegian adolescents has frequently been described as insufficient, especially with regard to sugar and fiber intake and fruits and vegetables (5, 6); however, recent detailed studies are lacking.

The Norwegian Mother and Child Cohort Study (MoBa), a longitudinal study comprising 114,500 children and 95,200 mothers (7, 8), can provide a good basis for both general knowledge of dietary intake in adolescents as well as linking the diet to future health outcomes. In the forthcoming years, MoBa will include about 60,000 adolescents, and for this, a reliable and validated dietary assessment tool needs to be developed.

The assessment of food intake is challenging and prone to errors, especially among children and adolescents (9, 10), and incorrect information may lead to false associations between dietary factors and disease (11, 12). Further, adolescence is a crucial period of life, and implies multiple physiological and psychological changes that affect nutritional needs and habits (13–15). To overcome such hurdles, it is critical to choose the best dietary intake method available for estimating nutrient intake (11). However, to conduct a dietary survey among MoBa participants, methods such as 24-HDRs and weighed records are not apt options as they may be burdensome for participants and project staff. Food-frequency questionnaire (FFQ) is an acceptable and useful tool to assess nutrient intake for a larger population – and an FFQ reflects long-term usual intake (10, 11, 16, 17). Furthermore, it is generally easy to administer and involves lesser time than other dietary assessment methods, which makes it a practical and affordable method in studying diets of adolescents (17, 18).

It is crucial to consider the degree to which a questionnaire can measure true dietary intake. Therefore, the aim of this study was to analyze the reproducibility and validity with regard to food and nutrient intake of a developed FFQ for use among adolescents. Both relative validity, where intake measured by the FFQ is compared with 2×24-hour dietary recalls (24-HDR), and absolute validity for intake of fatty acids and vitamin D correlated with levels of fatty acids and 25 hydroxy vitamin D₃, respectively, in serum are presented.

Methods

Study design and recruitment

Data were collected over a period from October to November 2011. Four 9th-grade classes were informed about the study, and an information/invitation letter and consent form for both parents and students were handed out. After consenting to participate, the students received the first FFQ by e-mail. The e-mail contained short information about the web-based FFQ and a link to the FFQ. The participants had 1 week to finish the FFQ. A reminder message was sent after 4 days to those who had not filled in the FFQ. The first 24-HDR took place 1 week after the participant had finished the FFQ, and the second, 4 weeks after the first 24-HDR. All participants were called on a random day. Those who had not filled in the FFQ were not called.

Two weeks after the students had filled in the FFQ, blood samples were taken at school.

For the test–retest reliability study, a new link to the FFQ was sent to those who had filled in the FFQ for the first time. This was done 4 weeks after the first time FFQ. Figure 1 gives an overview of the design and recruitment.

Fig. 1.

Outline of study design and numbers participating in each phase (participation rate).

Study population

A total of 285 students aged 13–14 from four different schools in Vest-Agder County were invited to participate in the validity and reliability study. Of which 120 students agreed to participate, and 100 students filled in the first FFQ. The same 100 students were called on a random day to take part in the first 24-HDR. Two students were missed due to a mistake, resulting in 98 students participating in the first 24-HDR. Ninety-three students participated in the second and last 24-HDR. In the results from the relative validity, only those who completed the first FFQ and both 24-HDR were included (n=93). In the test–retest study, 98 students filled in the first FFQ. Of these students 58 filled in the questionnaire the second time, which was the number included in the test–retest analyses. The absolute validation study included 92 participants who filled in the FFQ the first time and also had a blood sample taken.

The food-frequency questionnaire

The FFQ is self-administered and is based on a Danish FFQ for adolescents (19).

The FFQ consists of 156 questions. The first part includes questions about gender, age, weight and height (self-reported), and the level of activity (i.e. how many hours physical activity at school and how many hours spent in front of a computer to chat, play, surf, and do homework on a daily basis). The activity questions are validated questions from the HEVAS study (20). Furthermore, the FFQ contains questions on the average consumption of 131 food and beverage items during the past 4 weeks. These food items are divided into different sections in the FFQ (i.e. beverage, dairy products, bread and cereals, spread/cold cuts, fish and meat, fruit and vegetables, cakes/desserts/snacks). Of relevance to the validity study, the questionnaire posed questions on the intake of fatty fish, egg, milk, butter, margarine, cod liver oil, and vitamin D supplements. The frequency intervals vary according to food and beverage items. For beverages, the intervals range were never, 1–3 glasses per month, 1 glass per week, 2–6 glasses per week, 1 glass per day, 2–3 glasses per day, more than 3 glasses per day. For frequently consumed food items, the intervals range were ‘never’, 1–3 times per month, once a week, 2–3 times per week and more, 4–6 times per week to ‘once or more times a day’.

The respondents indicate their answers in ‘units per day’, ‘units per week’, and ‘units per month’. For most items, a unit is a standard portion size (i.e. a glass of juice, a bowl of cereals, an apple). For some items, extra information is provided (i.e. for soda: 1 bottle = 0.5 l).

All questions are close ended, except four which are related to dietary supplements, food-allergies, if there was anything they avoided eating, and if the last 4 weeks had been average in terms of eating habits. These questions were placed at the end of the questionnaire.

When calculating the nutrient intake, the standard portion included in the FFQ (e.g. glass of juice), which was coded according to the Norwegian standards (21) in grams, was multiplied by the frequency ticked off in the FFQ. Furthermore, each food item had a food-code linkage corresponding to the numbers in the Norwegian food composition table. For dishes, the codes adjusted for nutrient loss during cooking were chosen. Using FoodCalc and the Norwegian food composition table (22–24), the nutrient intake was calculated.

The repeated 24-HDR

The 24-HDRs were conducted on two non-consecutive days and were based on interviews done by three master's degree students and one bachelor degree student, using the ‘triple-pass’ method from King's College London (22). The four interviewers were all given the same information on how to perform the interviews and they followed a detailed description according to the triple-pass method. They practiced in advance and all the four interviewed each other afterwards to check if they were harmonized according to the triple-pass method.

All week days were randomly assigned to the participants, aiming to cover all days of the week equally. However, due to difficulties in making contact with the participants on given days, the distribution of days for the two recalls was skewed. For recall 1 and 2, respectively, the distribution was as follows: Monday (14%, 31%), Tuesday (23%, 16%), Wednesday (25%, 14%), Thursday (12%, 10%), Friday (8%, 5%), Saturday (4%, 5%), Sunday (15%, 18%).

To help quantify the food intake for the participants, a validated food picture booklet (23) with pictures of different dishes and portion sizes was handed out at school before the first interview. The booklet consisted of 16 pictures of different household measurements and portion sizes. The 16 food pictures were categorized into four different portion sizes named A–D, where A was the smallest portion and D was the largest.

The interviewer called the participant while they were at home on their home phone/cell phone. The participant was free to get help from his or her parents while taking part in the interview and could also check food packages and brand name at home. At the beginning of the interview, the interviewer introduced the procedure, explained the ways the picture booklet was to be used, and emphasized that the participant could at any time ask questions to the interviewer if anything was unclear.

The interview was standardized according to the triple-pass method (22), which consists of three phases to assess food intake. First, the participants were asked to give a brief report of what they have eaten and drunk the previous day, without the interviewer interrupting them. Second, the interviewer collects detailed information concerning the items in ‘the quick list’ (a list of additional questions about what the participants answered in the first part). During this phase, details about composite dishes and all food items eaten were recorded. Recipes were asked for, and if they knew or the parent knew, the recipes were noted. In the third, and last, part the respondents are given the opportunity to provide additional information about the food they have eaten and liquids they have drunk over the last 4 weeks. In this part, the interviewer also had the opportunity to ask for information about foods and drinks not mentioned. Use of dietary supplements/vitamins and questions about allergies were asked for at the end of the interview.

All the information gathered by the interviewer was written down on record sheets while the interview took place. At the end of the data collection all notes were handled systematically and transferred to Excel. A food-code linkage was provided, using FoodCalc and the Norwegian food composition table (24–26) to calculate the nutrient intake. The four interviewers standardized their coding by having meetings at several stages during the coding phase. Composite dishes and recipes were coded by ingredients if given during the interview or by using a similar dish and recipe if no details were available. Food items that were not in the food composition table were discussed and similar foods were selected and the same code was used by all interviewers. Fats used in cooking were coded when informed of. For standardized portion sizes that were not in the photographic booklet, the Norwegian standards were used (21). Nutrients presented in the relative validation and reliability study are calculated without dietary supplements. However, for the absolute validation dietary supplements were included.

Blood samples

For the absolute validity study, we selected fatty acids and vitamin D concentration in whole blood to correlate with the calculated nutrient intake from the FFQ. Several health outcomes have been related to these nutrients and are therefore interesting (27, 28). As for vitamin D, it is found naturally in a few food items like fatty fish and fish liver (28), and is a critical and essential nutrient for human health; however, the relevance for cancer and cardiovascular diseases are currently discussed (28). The fatty acids and vitamin D concentrations were measured in capillary non-fasting whole blood using dried blood spot (DBS). The fatty acids analyzed were alpha-linolenic acid (ALA), eicosapentaenoic acid (EPA), docosapentaenoic acid (DPA), docosahexaenoic acid (DHA), and linoleic acid (LA); and for vitamin D, 25 hydroxy vitamin D₃ (25(OH)D₃) was analyzed. According to the laboratory at VitAs, all assays on DBS are compared with both internal and external reference methods on plasma or serum. The chosen laboratory function is a reference lab for fat-soluble vitamins for the Nordic External Quality assurance Program, and qualify and participates and is compliant in the largest vitamin D External Quality Assessment, called DEQAS. As found by others, both 25(OH)D₃ and fatty acid analysis perform equally well on DBS as in ordinary serum or whole blood samples (29). Drops of blood from each participant were transferred on to a special filter paper after a pinprick on fingertip. Three nurses were responsible for this procedure, and the students were taken out of their class into another classroom for the blood test. Samples were air dried for at least 2 h as recommended, packed separately in the supplied airtight Bioindex DBS transport bags containing desiccant, deep frozen at −20°C, and sent to the laboratory (Vitas AS) (30) for analyses of levels of the biomarkers 25(OH)D₃ and full fatty acid profile. The technique used to analyze fatty acids was gas chromatography with flame ionization detection (30), and the analysis technique used on 25(OH)D₃ was liquid chromatography with mass spectrometry detection. Fatty acids in blood were expressed as g/100 g FAME and 25(OH)D₃ as nmol/L.

Ethics

Our study has been performed in accordance with the guidelines laid down in the Declaration of Helsinki, and all procedures involving human subjects were approved by the Regional Ethics Committee of southern Norway. Written informed consent was obtained from all participants and their parents.

Statistical analysis

Descriptive analyses were used to evaluate the characteristics of the participants (mean age, gender, height, weight, BMI). The total intake of energy and selected nutrients from FFQs and 24-HDR were not normally distributed and are therefore presented as median with 25th and 75th percentile.

For the relative validity study, the mean intake from the two 24-HDRs was calculated and used in the correlation analysis. Median daily intakes of the selected nutrients and E% were calculated separately based on the FFQs and the 24-HDRs. Median food intakes reported from FFQs and 24-HDRs are presented in these food groups: dairy products; eggs; meat and poultry; fish; bread and cereals; potatoes, fruit, and vegetables; sugar, sweets, and snacks; butter, margarine, and oils; beverages (water, juice, sodas, tea, and coffee).

Correlations between the food, nutrient, and energy intakes estimated by the two FFQs (reproducibility study) and between FFQ and the mean of two 24-HDR were done with the use of Spearman's correlation coefficient (r). To adjust for the within-person variation in the measurements, corrections of correlation coefficients were performed according to Willett (16). The corrected correlation, R ₁, was calculated as follows: $R_1=R_0\sqrt{1\frac{s_w^2}{s_b^2}/n}$, where $s_w^2/s_b^2$ is the ratio of the within- and between-person variances and n is the number of replicates per person for the given variable. The within- and between-person variances were obtained from an ANOVA model. These correlations are presented in Table 4.

Table 4.

Relative validity of the FFQ compared to 2×24-HDRs

		FFQ		24-HDR
Nutrients	n	Median	(P25, P75)	Median	(P25, P75)	Spearman r	p	Corrected, R a	CC%	GM%
Energy (MJ)	93	7.1	(5.1, 8.9)	7.3	(5.9, 9.3)	0.33	0.001	0.34	40	29
Protein, g	93	73.2	(52.0, 95.1)	70.7	(56.2, 92.5)	0.42	0.000	0.43	38	24
Total fat, g	93	56.9	(43.7, 71.4)	66.2	(46.1, 82.3)	0.21	0.041	0.22	28	26
Saturated fat, g	93	25.5	(18.9, 32.7)	26.7	(20.2, 34.2)	0.13	0.226	0.14	26	32
Monounsaturated fat, g	93	17.6	(13.6, 22.8)	20.2	(14.5, 25.7)	0.19	0.074	0.21	28	29
Polyunsaturated fat, g	93	7.1	(5.9, 9.3)	9.5	(6.8, 13.6)	0.25	0.014	0.28	31	24
Trans-fat, g	93	1.7	(1.2, 2.1)	1.4	(0.8, 1.8)	0.24	0.023	0.24	33	28
Cholesterol, mg	93	199.1	(145.4, 289.1)	191.4	(132.0, 260.2)	0.19	0.065	0.51	25	34
Carbohydrate, g	93	217.0	(159.9, 278.4)	217.1	(167.6, 276.8)	0.41	0.000	0.42	37	25
Fiber, g	93	17.4	(12.4, 22.6)	14.6	(11.7, 21.2)	0.34	0.001	0.34	35	24
Sugar, g	93	48.1	(34.0, 69.6)	44.9	(29.5, 74.5)	0.21	0.042	0.22	32	32
Retinol, mg	93	436.5	(261.0, 717.3)	419.3	(285.0, 592.2)	0.42	0.000	0.43	34	22
Vitamin D, µg	93	2.1	(1.3, 3.9)	1.8	(0.7, 3.6)	0.22	0.033	0.26	32	32
Vitamin E, mg	93	5.1	(4.2, 6.6)	5.7	(3.9, 7.2)	0.28	0.006	0.36	26	24
Thiamin, mg	93	1.1	(0.7, 1.5)	1.0	(0.8, 1.4)	0.46	0.000	0.46	36	21
Riboflavine, mg	93	1.8	(1.1, 2.5)	1.2	(0.9, 1.6)	0.50	0.000	0.50	35	20
Vitamin C, mg	93	82.8	(56.0, 119.3)	51.8	(29.5, 77.2)	0.50	0.000	0.50	42	16
Calcium, mg	93	980.1	(603.8, 1345.3)	782.4	(540.6, 1041.1)	0.55	0.000	0.56	39	17
Iron, mg	93	7.4	(5.5, 9.7)	7.4	(5.8, 8.6)	0.30	0.003	0.56	36	28
Percentage energy intake (E%)
Protein, E%	93	17.3	(15.7, 19.2)	17.0	(14.3, 18.6)	0.24	0.019	0.25	32	30
Fat, E%	93	30.3	(27.5, 34.5)	31.3	(25.5, 36.8)	0.17	0.098	0.17	28	28
Saturated fat, E%	93	13.4	(12.2, 15.3)	13.2	(10.8, 15.4)	0.15	0.163	0.16	30	30
Monounsaturated fat, E%	93	9.3	(8.5, 10.5)	9.8	(7.6, 11.7)	0.18	0.083	0.19	22	32
Polyunsaturated fat, E%	93	4.0	(3.4, 4.5)	4.7	(3.8, 6.3)	−0.07	0.495	−0.08	24	47
Trans-fat, E%	93	0.8	(0.7, 1.1)	0.6	(0.5, 0.9)	0.40	0.000	0.40	35	25
Carbohydrate, E%	93	51.0	(48.7, 55.2)	51.4	(45.0, 58.2)	0.28	0.006	0.29	35	23
Sugar, E%	93	12.3	(9.0, 16.0)	11.2	(6.9, 17.1)	0.26	0.013	0.30	35	29

Median intake of nutrients and E% estimated by the food-frequency questionnaire (FFQ) and the 24-hour dietary recalls (24-HDRs) (median, 25th and 75th percentile, Spearman correlation coefficient, percent correctly classified (CC) and grossly misclassified (GM) into quartiles of nutrient intake).

^aCorrelation coefficient adjusted for within-person variance $R=R_0\sqrt{1+s_w^2/s_b^2/n}$ where $s_w^2/s_b^2$ is the ratio of the within- and between-person variances and n is the number of 24-HDRs available. The within- and between-person variances were obtained from an ANOVA model.

To test the FFQ's ability to rank individuals according to their nutrient and energy intake, cross classifications between the FFQ and 24HDRs, and FFQ and fatty acids in blood and FFQ and second FFQ were carried out. The nutrient and energy variables were divided into quartiles, and results are presented as participants classified into the same quartile [correctly classified (CC)] and those who were grossly misclassified (GM) to a non-adjacent quartile.

Self-reported weight and height were used to calculate body mass index (BMI=kg/m²). The significance level was set to 5%, and all statistical analysis were carried out using the computer program SPSS version 18 (SPSS Inc. Chicago, IL, USA).

Results

Sample

The characteristics of the 93 students who completed the FFQ and the two 24-HDRs are presented in Table 1. There was an equal distribution of girls and boys in the study. Mean age of participants was 13.8 years while mean BMI was 19.7 kg/m².

Table 1.

Characteristics of the participants in the validation study (n=93)

	Study population	Girls	Boys
Number of participants, n (%)	93 (100)	53 (57)	40 (43)
Mean age, y	13.8	13.8	13.8
Mean weight, kg	54.9	52.7	57.8a
Mean height, cm	166.8	164.1	170.4a
Mean BMI	19.7	19.5	19.9
Overweight, n (%)b	5	1 (3)	4 (13)
Obese, n (%)b	0	0 (0)	0 (0)

^aValue missing for one boy

^bage- and gender-specific classification of normal weight, overweight, and obese according to Cole et al.'s definition.

Test–retest reliability

There were 58 students who participated in the reliability study, 22 boys and 36 girls. The median Spearman correlation coefficient for all nutrients in the test–retest study was 0.57 (Table 2). The correlations ranged from r=0.280 for trans-fat to r=0.680 for cholesterol. All correlations were significant. For most nutrients, more than 50% of the participants were correctly classified into the same quartile of intake. For trans-fat, only 29% participants were correctly classified. For all but three nutrients (trans-fat, monounsaturated fat, and saturated fat), less than 20% of the participants were GM into a non-adjacent quartile. The median Spearman coefficient for all food groups was 0.52 (Table 3), with a range from r=0.35 for bread and cereals to r=0.67 for beverages. For all but the food group ‘bread and cereals’, less than 20% of the participants were GM, that is, classified into a non-adjacent quartile.

Table 2.

Test–retest reliability of a food-frequency questionnaire for adolescents

	FFQ1		FFQ2
	Median	(P25, P75)	Median	(P25, P75)	Spearman r	p	CC%	GM%
Energy (MJ)	7.1	(5.3, 8.9)	6.2	(4.5, 7.9)	0.604	<0.001	45	5
Protein, g	75.0	(50.8, 95.0)	64.6	(43.1, 80.9)	0.624	<0.001	34	10
Total fat, g	54.8	(44.6, 69.6)	46.8	(37.8, 63.8)	0.573	<0.001	43	14
Saturated fat, g	25.2	(18.8, 31.8)	21.9	(17.1, 30.1)	0.594	<0.001	48	14
Monounsaturated fat, g	17.1	(14.1, 21.6)	15.2	(12.0, 20.1)	0.553	<0.001	47	14
Polyunsaturated fat, g	7.0	(5.9, 9.3)	6.3	(4.8, 8.6)	0.551	<0.001	59	16
Trans-fat, g	1.7	(1.1, 2.3)	1.6	(1.2, 2.2)	0.480	<0.001	41	17
Cholesterol, mg	198	(145, 267)	182	(136, 233)	0.680	<0.001	50	10
Carbohydrate, g	221	(171, 273)	187	(137, 245)	0.605	<0.001	40	10
Fiber, g	16.4	(12.7, 22.7)	15.9	(11.4, 19.0)	0.478	<0.001	43	19
Sugar, g	47.3	(23.44, 71.0)	44.8	(28.0, 62.7)	0.660	<0.001	47	16
Retinol, mg	424	(250, 787)	431	(290, 750)	0.504	<0.001	50	10
Vitamin D, µg	2.2	(1.4, 3,9)	2.2	(1.2, 3.8)	0.590	<0.001	52	16
Vitamin E, mg	4.9	(4.2, 6.3)	4.8	(3.6, 5.9)	0.638	<0.001	52	16
Thiamin, mg	1.1	(0.8, 1,5)	1.0	(0.7, 1.4)	0.571	<0.001	50	12
Riboflavine, mg	1.9	(1.2, 2.5)	1.5	(0.9, 2.3)	0.605	<0.001	47	16
Vitamin C, mg	79	(50, 120)	66	(50, 105)	0.663	<0.001	43	9
Calcium, mg	1,012	(594, 1,291)	814	(439, 1, 152)	0.589	<0.001	53	14
Iron, mg	7.5	(5.7, 9.9)	6.8	(4.8, 8.7)	0.533	<0.001	34	19
Percentage energy intake (E%)
Protein, E%	17.7	16.0, 19.3	17.2	15.7, 18.6	0.674	<0.001	33	10
Fat, E%	30.3	27.2, 32.8	31.3	26.8, 33.9	0.422	0.001	43	19
Saturated fat, E%	13.4	12.2, 14.5	11.9	6.5, 14.8	0.328	0.012	26	22
Monounsaturated fat, E%	9.3	8.3, 10.0	8.3	6.7, 10.3	0.375	0.004	33	22
Polyunsaturated fat, E%	4.0	3.4, 4.6	3.9	2.9, 4.5	0.549	<0.001	40	12
Trans-fat, E%	0.8	0.7, 1.1	0.8	0.6, 1.2	0.280	0.033	29	24
Carbohydrate, E%	52.7	49.3, 55.3	51.2	48.6, 56.1	0.531	<0.001	38	19
Sugar, E%	12.0	7.7, 16.0	11.4	8.8, 17.1	0.638	<0.001	48	10

Median, 25th and 75th percentile, Spearman correlation coefficient, percent correctly classified (CC) and grossly misclassified (GM) into quartiles of nutrient intake (n=58).

Table 3.

Test–retest reliability of a food-frequency questionnaire for adolescents

	FFQ1		FFQ2
	Median	(P25, P75)	Median	(P25, P75)	Spearman r	p	CC%	GM%
Dairy products (milk, yoghurt)	624	(274, 812)	323	(181, 673)	0.522	<0.001	55	14
Eggs	12	(5, 19)	12	(5, 18)	0.421	0.001	47	17
Meat and poultry	63	(50, 88)	56	(44, 78)	0.411	0.001	47	19
Fish	22	(14, 32)	15	(10, 25)	0.494	<0.001	38	17
Bread and cereals	153	(110, 220)	152	(80, 193)	0.347	0.008	36	24
Potatoes, fruits, and vegetables	198	(128, 270)	158	(112, 237)	0.667	<0.001	43	7
Sugar, sweets, and snacks	84	(65, 106)	71	(51, 92)	0.581	<0.001	53	14
Butter, margarine, and oils	7	(4, 16)	7	(3, 15)	0.569	<0.001	48	16
Beverages (water, juice, sodas, tea, and coffee)	926	(769, 1,129)	825	(513, 1,009)	0.669	<0.001	52	9

Food intake (g/day) presented for main food groups (median, 25th and 75th percentile, Spearman correlation coefficient, percent correctly classified (CC) and grossly misclassified (GM) into quartiles of food intake) (n=58).

Relative validity

The median Spearman correlation coefficient for all nutrients in the relative validity study was r=0.26, ranging from r=−0.07 for energy percent from polyunsaturated fat to r=0.55 for calcium (Table 4). Significant correlations were found for all nutrients, except intake of saturated fat, monounsaturated fat, cholesterol, energy percent from fat, and polyunsaturated fat. To correct for within-person variation between the two 24-HDRs, corrected correlation coefficients are presented in Table 4. Most correlations were improved, especially correlations for polyunsaturated fat, cholesterol, vitamin D, vitamin E, and energy percent from sugar.

The correct classifications of participants into same quartiles were also low for these nutrients, ranging from 22% for energy percent from monounsaturated fat to 30% for E% from saturated fat. Best classifications were found for energy intake (40%) and vitamin C intake (42%).

The median Spearman correlation coefficient for all food groups was r=0.19 (Table 5), ranging from r=0.01 for sugar, sweets, and cakes to r=0.627 for dairy products. There was a low percentage of the participants being correctly classified into quartiles of food intakes (22–42%) (Table 5).

Table 5.

Relative validity of the FFQ compared to 2×24-HDRs

		FFQ		24-HDR
Nutrients	n	Median	(P25, P75)	Median	(P25, P75)	Spearman r	p	CC%	GM%
Dairy products (milk, yoghurt)	93	573	(225, 869)	255	(141, 504)	0.627	<0.001	42	10
Eggs	93	13	(5, 21)	0	(0, 0)	0.173	0.098	24	0
Meat and poultry	93	75	(50, 84)	75	(30, 134)	0.162	0.121	31	23
Fish	93	23	(10, 32)	0	(0, 9)	0.188	0.070	22	20
Bread and cereals	93	156	(85, 216)	178	(113, 248)	0.385	<0.001	33	23
Potatoes, fruits, and vegetables	93	194	(122, 275)	195	(106, 301)	0.246	0.017	30	28
Sugar, sweets, cakes, and snacks	93	82	(63, 108)	65	(25, 107)	0.010	0.925	23	29
Butter, margarine, and oils	93	7	(4, 16)	15	(4, 31)	0.152	0.145	31	30
Beverages (water, juice, sodas, tea, and coffee)	93	906	(775, 1112)	760	(475, 1062)	0.276	0.010	33	32

Median food intake in selected food groups and E% estimated by the food-frequency questionnaire (FFQ) and the 24-hour dietary recalls (24-HDRs) (median, 25th and 75th percentile, Spearman correlation coefficient, percent correctly classified (CC) and grossly misclassified (GM) into quartiles of food intake).

Absolute validity

Significant correlations were found for the n-3 fatty acids, EPA, DPA, and DHA; however, not for the fatty acids ALA and LA, total n-3 intake and vitamin D from FFQ compared to whole blood analyses (Table 6). The correlation coefficients for n-3 fatty acids were low ranging from r=0.05 to 0.25, and absent for vitamin D (r=0.000). The correct classifications into quartiles were low for all fatty acids and vitamin D.

Table 6.

Correlation between intake of fatty acids and vitamin D (including supplements) (FFQ) and fatty acids and 25(OH)D₃ measured in blood

	FFQa	Blood samplesb	Spearman r	p	% CC	% GM
C 18:3, n-3, ALA	0.71 (0.57, 1.06)	0.29 (0.24, 0.36)	0.046	0.666	25	38
C 20:5, n-3, EPA	0.11 (0.06, 0.15)	0.40 (0.31, 0.49)	0.252	0.015	29	29
C 22:5, n-3, DPA	0.04 (0.03, 0.06)	0.95 (0.81, 1.05)	0.250	0.016	28	25
C 22:6, n-3, DHA	0.18 (0.10, 2.77)	2.06 (1.77, 2.38)	0.232	0.026	26	25
Total n-3	1.03 (0.82, 1.53)	3.83 (3.34, 4.25)	0.175	0.096	29	33
C 18:2, n-6, LA	5.92 (4.63, 8.15)	15.63 (14.66, 16.77)	0.004	0.972	23	36
Vitamin D/25(OH)D3	2.3 (1.4, 4.0)	33.8 (27.9, 40.4)	0.000	0.998	23	36

Median, 25th and 75th percentile, Spearman correlation coefficient, percent correctly classified (CC) and grossly misclassified (GM) into quartiles of nutrient intake. (n=92).

^aFatty acid: g/day, vitamin D: µg/day

^bfatty acids: g/100 g FAME, 25(OH)D₃: nmol/l.

Discussion

An FFQ was developed for use among adolescents in the Norwegian Mother and Child Cohort. In the present study, the test–retest reliability of this FFQ was explored by comparing two administrations of the FFQ over a 4-week period. Further, the relative validity of the FFQ was explored against two 24-HDRs. Finally, the absolute validity with regard to selected fatty acids and vitamin D were explored by comparing the FFQ intake with fatty acid and 25(OH)D₃ levels in whole blood.

Test–retest reliability

The general test–retest reliability of the questionnaire was good with median correlations of 0.53 for nutrients and 0.52 for food groups, which fell within the range considered good for an FFQ (12, 16). Low correlations were only found for trans-fat (r=0.28). This might be because intake of trans-fat generally is low in the Norwegian public (31) and further that foods, such as cookies and biscuits, containing trans-fat are eaten seldom, making that intake more difficult to measure. In our sample, median intake of sweet biscuits was 2 g/day (Q1: 2 g/day, Q4: 2 g/day), where of course few of these include trans-fat (data not shown).

Our results are comparable to other studies involving adolescents (32, 33). Kobayashi et al. tested the reproducibility of two newly developed questionnaires for adolescents and found correlations between 0.67 and 0.84 for nutrients in the first questionnaire and 0.39 to 0.83 for the other. In general, these correlations are somewhat higher than those found in the present study. However, the present correlations are higher than what was reported by Preston et al. (mean r=0.43) among adolescents. In their general review, Cade et al. found that correlations between 0.5 and 0.7 were most common in reproducibility studies (12). The strength of our study is the time lapse between the two FFQ administrations, which was 4 weeks, and which according to Cade et al. is in line with approximately 34% of all the test–retest studies (12). Cade et al. state that it is not wise to administer a questionnaire at a very short interval as respondents may remember their answers; however, true changes in diet may occur if the interval is too long (12). A limitation with this study is the low sample size (n=58), as it is lower than the ideal sample size of 100–200 (16). The participants in this study were involved in many assessments, and the second FFQ was the last thing they did, and we assume that that is an explanation for the low participation.

Relative validity

Comparisons of nutrient and energy intakes estimated from the FFQ with the reference method, 2×24-HDRs, showed limited agreement. The correlation coefficients between the FFQ and the 24-HDRs varied from −0.07 to 0.55. In most validation studies, correlation coefficients are considered to be poor if r<0.49 and good if r>0.5 (12, 34). Overall, poor to good associations were observed for 10 of the nutrients and E% from trans-fat. In this study, three of the nutrients (riboflavine, vitamin C, and calcium) had a correlation coefficient like Willett recommend (r>0.5–0.7), and in total eight of the nutrients were over the threshold that Cade suggests (r>0.4) (12, 16). The correlation was considered good for dairy products but poor for most other food groups. Although comparison to other studies is difficult due to differences in methodology for reference methods, comparing our results with six relevant studies representing FFQs for adolescents (10, 34–38) show that our results are comparable for several nutrients like fiber and calcium. A similar study also found lower correlations for food intake compared to nutrient intake (37). Few participants were classified into the right quartile based on the two assessment methods. Overall, based on the percentage of subjects classified into the same quartile, the ranking ability of the questionnaire was evaluated to be limited. The lack of agreement between the two methods may be due to limitations of the FFQ, the reference measure (the 24-HDR), or how the study was conducted. One challenge in the validation of an FFQ is the selection of a proper reference method. When validating an FFQ, dietary records are likely to have the least correlated errors according to Willett (16). Multiple 24-HDRs are the primary alternative to dietary records (16). However, it is a suboptimal alternative since both rely on memory and perception of serving sizes. Willett argue that in situations, as when subjects are less than highly motivated, 24-HDRs are good alternatives to dietary records (16). It is important to be aware of the fact that FFQ is designed to measure usual intake while the 24-HDR is designed to measure short-term intake. However, two 24-HDRs are sufficient to capture usual food intake if first, random within-person error is removed by statistical modelling, as done in Table 4 in our study. Several correlations were strengthened after this correction, probably because by using the arithmetic mean of both interviews in the first correlation, the proportions in the upper and lower tails of the intake distribution were overestimated. Then for individual-level intake assessment, the correlations between the two methods were probably somewhat underestimated due to regression dilution bias. Second, two 24-HDRs are sufficient to capture usual food intake if random between-person error is eliminated by including sufficient observations. The numbers included were not optimal; however, compared to others, we believe that the 24-HDRs to some extent serve as a superior reference method for the relative validity assessment.

A limitation with the 24-HDR is that subjects knowing that their diet is under investigation, may change their dietary intake and this creates a distortion of their actual intake (12, 39). The students knew what week they would be called, but not what day of the week. Being aware of the week they would be called may have resulted in more healthy eating habits in that particular week. This is often subconscious and has been shown for adolescents, especially girls (40, 41). One should further notice the particular days the recall took place. A majority of the days were weekdays, and it is well known that diet changes during the weekends, especially Friday and Saturday (42). This is a limitation with the study and could have led to underreporting of certain nutrients like different fats and sugar and foods like sweets and cakes. There is of course a general limitation with underreporting to be aware of, which is a particular problem among girls (43). Another consideration is the number of days for recall; the low number of days (n=2) may not be enough to assess the usual intake for adolescents (37). Several other validation studies have more than two, often 3–4 days of recall (35–37). Willett also suggests that 3–10 days of recall would be appropriate if the population investigated is small (16).

There are several additional limitations to be considered in our validation study. One limitation is the low response rate and the large dropout rate from those who handed in the written consent to those who finished the first FFQ, even though a reminder message was sent after the first e-mail invitation to the FFQ to increase the response rate (44). Low response rate is also seen in other validation studies (45). Another limitation refers to the problem of estimating portion size and how detailed the information had to be. Estimating food with photographs, as done in the 24-HDRs, is a complex task (23). Since only a short booklet was used in this study, there were no pictures of all the different kinds of food items or portion sizes. Therefore, the interviewers could end up in a situation where they had to help the participants to remember and suggest estimates of the food. In the FFQ, household measurements and specific portion sizes were used. A large number of the questions were also grouped (more than one food item in one question), and the students did not have the opportunity to answer as precisely as in the 24-HDRs (i.e. if the pasta was whole grain). This can explain some of the differences observed.

Absolute validity

The evaluation of this FFQ's absolute validity showed poor correlations in EPA, DPA, and DHA compared to the same nutrients in blood. The correlation coefficients between FFQ and blood in this study ranged from r=0.000 to 0.291. These are low values especially when correlation is considered to be poor if r <0.49 (3). None of the nutrients in this study reached a correlation level above this. In addition, the classification of participants into the right quartile calculated by these two assessment methods was low, on average 26.1%, meaning that the results allow for misclassification (23).

There are few studies comparing self-reported dietary intake with biomarkers of fat from whole blood as reviewed by many (46, 47), particularly data in children are scarce. Two studies in children were identified: Burrows et al. (48) and Okuda et al. (49). Burrows et al. (48) found about the same correlation in EPA (r=0.24), lower correlation in LA (r=0.01), ALA (r=0.03), DPA (r=0.03), and DHA (r=0.11) compared to the present study. Okuda et al. (49) found roughly similar results as our study for EPA (r=0.281) and DHA (r=0.233), lower correlation for DPA (r=0.093). All values were unadjusted (49). The correlation coefficients between vitamin D and 25(OH)D₃ were lower than what is observed by others (50, 51). One could speculate if the FFQ did not include enough detailed questions regarding sources of fatty acid and vitamin D. We could have included more questions on oils with regard to fatty acids. As for vitamin D, most food sources were included in the FFQ. One could argue that since some vitamin D stems from sunlight a good correlation would be difficult to show, especially since the samples were taken in October where adolescents have spent less time outdoors. However, according to Pedersen (52) sunlight supplies ignorable amounts of vitamin D at our latitude in October–April. Brantsæter et al. found among pregnant women that the correlation between dietary intake measured by FFQ and plasma concentrations of vitamin D increased during winter compared to the whole year (53). This supports ignorable amounts from sunlight during winter (51). Although it is a reference method for vitamin D measurements, the method of DBS sampling is still a critical method and our analyses were done on non-standardized samples, that is, the blood were not drawn while the subjects had an overnight fast.

Conclusion

The FFQ was developed as a self-administered web-based tool for assessing food habits for adolescents and is to be used in follow-up studies of the adolescents in the Norwegian Mother and Child Cohort. The test–retest reproducibility was considered good, the relative validity was considered poor to good, and the absolute validity was considered low for fatty acids and absent for vitamin D. The ranking ability was considered to be low based on validity. However, the results are comparable to other studies among adolescents. These findings should be incorporated in the evaluation of results when using this FFQ.

Conflict of interest and funding

The authors declare no conflict of interest and funding.