Development and evaluation of a diet quality screener to assess adherence to the Dutch food-based dietary guidelines

Abstract

The Eetscore FFQ was developed to score the Dutch Healthy Diet index 2015 (DHD2015-index) representing the Dutch food-based dietary guidelines of 2015. This paper describes the development of the Eetscore FFQ, a short screener assessing diet quality, examines associations between diet quality and participants’ characteristics, and evaluates the relative validity and reproducibility of the Eetscore FFQ in a cross-sectional study with Dutch adults. The study sample consisted of 751 participants, aged 19–91 years, recruited from the EetMeetWeet research panel. The mean DHD2015-index score based on the Eetscore FFQ of the total sample was 111 (sd 17·5) out of a maximum score of 160 points and was significantly higher in women than in men, positively associated with age and education level, and inversely associated with BMI. The Kendall’s tau-b coefficient of the DHD2015-index between the Eetscore FFQ and the full-length FFQ (on average 1·7-month interval, n 565) was 0·51 (95 % CI 0·47, 0·55), indicating an acceptable ranking ability. The intraclass correlation coefficient between DHD2015-index scores derived from two repeated Eetscore FFQ (on average 3·8-month interval, n 343) was 0·91 (95 % CI 0·89, 0·93) suggesting a very good reproducibility. In conclusion, the Eetscore FFQ was considered acceptable in ranking participants according to their diet quality compared with the full-length FFQ and showed good to excellent reproducibility.

Adherence to dietary guidelines is often evaluated by quantifying overall diet quality, which on its turn can be used to study the potential impact of overall diet on health and disease^(1,2,3). Diet quality is also frequently used to adjust for overall diet in epidemiological studies exploring associations between specific dietary factors, such as alcohol consumption, and health outcomes^(4,5,6). One of the approaches to assess overall diet quality is by calculating an index score^(2,7). One of the commonly used Dutch indexes used to assess diet quality is the Dutch Healthy Diet 2015-index (DHD2015-index)^(8,9,10).

The DHD2015-index has been developed to assess adherence to the Dutch food-based dietary guidelines of 2015^(8,11,12). The DHD2015-index can be calculated using data from multiple 24-h dietary recalls or a FFQ designed to estimate daily energy intake, macronutrients⁽¹³⁾ and relevant food groups. Calculating the DHD2015-index is particularly relevant to quantify diet quality for use in epidemiological analyses as either an exposure factor or covariate. However, administering multiple recalls or a general FFQ is rather time-consuming and burdensome for participants, researchers and interviewers, and in case diet quality is the sole parameter of interest, aforementioned dietary assessment methods may be unnecessarily time-consuming^(14,15).

Therefore, we developed the Eetscore FFQ to assess intake for estimating the DHD2015-index score for monitoring and ranking individuals based on their diet quality. The Eetscore FFQ could be used for research purposes but also for use in clinical settings to assess adherence to the Dutch food-based dietary guidelines and to monitor dietary changes^(16,17,18). In the present paper, we describe the development of the Eetscore FFQ and examine associations between the DHD2015-index scores, derived from the Eetscore FFQ, and participants’ characteristics. Moreover, we evaluated the relative validity and reproducibility of the Eetscore FFQ.

Methods

Participants

Participants were recruited from the EetMeetWeet research panel, a longitudinal observational study on diet and health in Dutch adults. This research panel consisted of inhabitants living in or near five cities located in the central part of the Netherlands (i.e. Wageningen, Renkum, Ede, Arnhem and Veenendaal)⁽¹⁹⁾. The present observational study was conducted between February 2017 and July 2017. All participants of the EetMeetWeet research panel (n 4936) were invited and 1055 participants were willing to participate in this study. Participants with data of at least one Eetscore FFQ (n 760) were included in the study (online Supplemental Fig. 1). We excluded seven participants who were pregnant and two participants with a BMI of above 45 kg/m². The final study sample included 751 participants to examine associations between DHD2015-index scores and participants’ characteristics, 565 participants to evaluate the relative validity of the Eetscore FFQ, and 343 participants to evaluate the reproducibility of the Eetscore FFQ.

This study was conducted according to the guidelines laid down in the Declaration of Helsinki. As human participants were not subjected to procedures or were required to follow rules of behaviour and the study did not concern medical scientific research, the study was not subjected to the Medical Research Involving Human Subjects Act (WMO) and did not need approval of the Medical Ethical Committee of Wageningen University. All participants registered themselves and gave written informed consent.

Study design

Participants were asked to complete various questionnaires on their dietary intake as well as some general demographic and lifestyle characteristics. Dietary intake data were collected from the Eetscore FFQ and a full-length FFQ (online Supplemental Fig. 1). FFQ were administered online in a random order separated by at least 1 month, where 47 % of the participants first completed the Eetscore FFQ and subsequently the full-length FFQ. In order to evaluate the reproducibility of the Eetscore FFQ, all participants were asked to complete the Eetscore FFQ for a second time.

Dutch Healthy Diet index 2015

The DHD2015-index consists of fifteen food components representing the Dutch dietary guidelines: vegetables, fruit, whole-grain products, legumes, nuts, dairy products, fish, tea, fats and oils, coffee, red meat, processed meat, sweetened beverages and fruit juices, alcohol, and salt. The scoring for the DHD2015-index has been described in detail elsewhere⁽⁹⁾ and is summarised in Table 1 including cut-off and threshold values. For each component, a maximum of 10 points can be allotted which means complete adherence to the Dutch dietary guidelines. Assessment of all scores together result in a total score ranging between 0 and 150 points. The DHD2015-index score gives an indication of diet quality and is positively associated with nutrient density⁽⁹⁾ and negatively associated with all-cause mortality⁽²⁰⁾.

Table 1.

Cut-off and threshold values for the calculation of the DHD2015-index component scores and the additional component ‘unhealthy choices’

	Component	Component type	Dutch dietary guidelines 2015	Minimum score ( = 0 points)	Maximum score ( = 10 points)
1	Vegetables	A	Eat at least 200 g of vegetables daily	0 g/d	≥ 200 g/d
2	Fruit	A	Eat at least 200 g of fruit daily	0 g/d	≥ 200 g/d
3	Whole-grain products	A	Eat at least 90 g of whole-grain products daily	0 g/d	≥ 90 g/d
		R	Replace refined cereal products by whole-grain products	No consumption of whole-grain products or ratio of whole grains to refined grains ≤0·7	No consumption of refined products or ratio of whole grains to refined grains ≥11
4	Legumes	A	Eat legumes daily	0 g/d	≥ 10 g/d
5	Nuts	A	Eat at least 15 g of unsalted nuts daily	0 g/d	≥ 15 g/d
6	Dairy products*	O	Eat a few portions of dairy products daily, including milk or yogurt	0 g/d or ≥ 750 g/d	300–450 g/d
7	Fish†	A	Eat one serving of fish weekly, preferably oily fish	0 g/d	≥ 15 g/d
8	Tea	A	Drink three cups of black or green tea daily	0 g/d	≥ 450 ml/d
9	Fats and oils	R	Replace butter, hard margarines and cooking fats by soft margarines, liquid cooking fats and vegetable oils	No consumption of soft margarines, liquid cooking fats and vegetable oils or ratio of liquid cooking fats to solid cooking fats ≤0·6	No consumption of butter, hard margarines and cooking fats or ratio of liquid cooking fats to solid cooking fats ≥13
10	Coffee	Q	Replace unfiltered coffee by filtered coffee	Any consumption of unfiltered coffee	Consumption of only filtered coffee OR No coffee consumption
11	Red meat	M	Limit consumption of red meat	≥ 100 g/d	≤ 45 g/d
12	Processed meat	M	Limit consumption of processed meat	≥ 50 g/d	0 g/d
13	Sweetened beverages and fruit juices	M	Limit consumption of sweetened beverages and fruit juices	≥ 250 g/d	0 g/d
14	Alcohol	M	If alcohol is consumed at all, intake should be limited to one Dutch unit (10 g ethanol) daily	Women: ≥ 20 g ethanol/d Men: ≥ 30 g ethanol/d	Women: ≤ 10 g ethanol/d Men: ≤ 10 g ethanol/d
15	Salt	M	Limit consumption of table salt to 6 g daily	≥ 3·8 g Na/d	≤ 1·9 g Na/d
16	Unhealthy choices	M	Limit consumption of unhealthy day and week choices	> 7 week choices/week	≤ 3 week choices/week

A, adequacy component (consume an adequate amount); R, ratio component (replace less healthy products by more healthy alternatives); O, optimum component (optimal consumption range); Q, qualitative component (choose healthier option); M, moderation component (limit consumption).

^*Maximum of 40 g cheese could be included.

^†Maximum of 4 g lean fish could be included.

Development Eetscore FFQ

The Eetscore FFQ was specifically developed to assess the DHD2015-index⁽⁹⁾ using data of the Dutch National Food Consumption Survey (DNFCS) 2007–2010⁽²¹⁾ as a reference. Foods were selected that were part of a component in the Dutch dietary guidelines^{(22,23,24,25)}. Thereafter, foods were aggregated into food items based on their food group, portion size and eating time. For example, ‘fruit’ was considered as one food item whereas ‘cheese’ was divided into ‘cheese on bread’ and ‘cheese with dinner’. Furthermore, Na intake was separated into two parts: Na intake from foods and discretionary salt. Two items on discretionary salt were included to estimate the frequency of salt or Na-rich products (i.e. soya sauce and soup flavouring) added during cooking and at the table.

In addition to the fifteen components of the DHD2015-index, the Eetscore FFQ was also developed to score one additional component on unhealthy foods. This so-called unhealthy choices component was based on a guideline of the Netherlands Nutrition Centre (NNC)^(8,12) aiming to get insight in dietary intake beyond the Dutch dietary guidelines. Foods that are high in energy, saturated fat and sugar have been categorised as unhealthy by the NNC. Therefore, food items that contributed most to total energy, saturated fat and mono- and disaccharide intake of the adult population in the DNFCS 2007–2010⁽²¹⁾ were selected to be included in the unhealthy choices component, unless they were already included in one of the other DHD2015-index components. For example, although orange juice was categorised as an unhealthy choice, it was not included in the unhealthy choices component as it was already included in the component sugar-containing beverages. Food items included in the unhealthy foods component were sweet spreads, cakes, cookies, chips or pretzels, chocolate, savoury snacks, sauces, and use of sugar in coffee or tea. Together these unhealthy food items contributed for at least 80 % to total energy, saturated fat and mono- and disaccharide intake⁽²⁶⁾.

The aggregation of foods resulted in a list of fifty-five food items, which together accounted for 85 % of energy intake from the adult population of the DNFCS 2007–2010 (online Supplemental Table 1). These fifty-five food items were interrogated in forty questions covering the intakes of all components over the previous month. The six answer categories for frequency questions ranged from ‘never’ to ‘every day’ for regularly consumed foods and from ‘not this month’ to ‘4 times a month’ for episodically consumed foods. Portion sizes were assessed in standard portions and commonly used household measures⁽²⁷⁾. Average daily intakes of food items were calculated by multiplying frequency of consumption by portion size in grams. The Na content of a food item was calculated by multiplying the weighted frequency of use by the Na content of each food in that food item^(21,28). The Eetscore FFQ was administered via the open-source survey tool LimeSurvey^TM (LimeSurvey Project Team/Carsten Schmitz, Germany, 2012).

Scoring Dutch Healthy Diet index 2015 from Eetscore FFQ

The food components vegetables, fruit, whole-grain products, legumes, nuts, dairy products, fish, tea, fats and oils, coffee, red meat, processed meat and sweetened beverages and fruit juices were scored according to the scoring of the original DHD2015-index (Table 1). The scoring for the components alcohol and Na deviates from the scoring of the original DHD2015-index. The Eetscore FFQ distinguishes between alcohol consumption during week and weekend days to account for binge drinking⁽²⁹⁾. For both subcomponents, a maximum score of 5 points can be allotted. The intake of discretionary salt contributed at maximum 2 points out of 10 based on the assumption that about 20 % of total Na intake from the Dutch population is from added salt^{(21,30,31,32)}. Na intake from foods contributed to the remaining 8 points. The unhealthy choices component was scored as a moderation component based on the guideline to limit consumption of unhealthy foods. A score of 0 was assigned when more than seven unhealthy foods per week were consumed. The maximum score of 10 points was assigned when three or less unhealthy foods per week were consumed (Table 1).

Full-length FFQ and its scoring with the Dutch Healthy Diet index 2015

A 166-item semi-quantitative FFQ was used to assess habitual dietary intake. Food items for this version were selected based on the DNFCS 2007–2010. This FFQ was evaluated for energy intake, macronutrients, dietary fibre and selected vitamins^(13,33). The reference period was the previous month. This full-length FFQ has previously been evaluated against two 24-h dietary recalls to assess the DHD2015-index (tau-b: 0·56; 95 % CI 0·52, 0·61)⁽⁹⁾. Answer categories for frequency questions ranged from ‘not this month’ to ‘7 d/week’, and portion sizes were estimated using standard portions and commonly used household measures⁽²⁷⁾. Average daily intake (in grams) of food items were calculated by multiplying frequency of consumption by portion size. Average daily Na intake was calculated by multiplying frequency of consumption by portion size and energy and nutrient content per gram using the 2010 Dutch Food Composition Table ⁽³⁴⁾. The full-length FFQ was administrated via the Dutch FFQTOOL⁽³⁵⁾.

Because of a technical problem with administering the full-length FFQ (results of questions on fish intake and use of cooking fats were not saved correctly), we were not able to calculate all component scores of the DHD2015-index. Moreover, the full-length FFQ does not distinguish between types of coffee (filtered v. unfiltered). Therefore, the component scores for fish, fats and oils and coffee are not presented.

Covariates

Age, body weight, height, educational level (low: primary school, vocational or lower general secondary education; moderate: higher secondary education or intermediate vocational training; and high: higher vocational education or university), smoking status (current, former and never) and drug use were self-reported when completing the Eetscore FFQ. Body weight and height were used to calculate the BMI.

Statistical analysis

Baseline characteristics are presented as mean and standard deviation scores of the DHD2015-index and its components, calculated from the Eetscore FFQ and the full-length FFQ, separately for men and women. Differences in participant characteristics between men and women were quantified by means of Mann–Whitney U tests and χ ² tests. ANOVA and ANCOVA, adjusting for age and BMI, were performed to compare the total DHD2015-index score derived from the Eetscore FFQ between men and women, while Mann–Whitney U tests and χ ² tests were performed to compare the subcomponents of the DHD2015-index. Linear trends in participants’ characteristics across sex-specific quartiles of the DHD2015-index scores were examined using general linear models.

A Bland–Altman plot was used to examine the agreement between the Eetscore FFQ and the full-length FFQ⁽³⁶⁾. Kendall’s tau-b correlation coefficients were calculated between scores derived from the Eetscore FFQ and the full-length FFQ to examine ranking of participants according to diet quality. For the component dairy products, a Kendall’s tau-b correlation coefficient was calculated between grams of dairy product intake as well. Spearman’s correlation coefficients were calculated, to allow comparison with previous studies. Correlation coefficients of ρ > 0·4 were considered to indicate an acceptable association and ρ = 0·5–0·7 were considered to indicate a reasonably good association^(37,38). CI were calculated using Fisher’s z-transformation. The proportion of participants assigned to the same or adjacent DHD2015-index score quartiles was calculated to evaluate the agreement between the two methods. Kappa (κ) coefficients were calculated between the quartile scores to further evaluate the level of agreement between the two methods. κ coefficients between 0·21–0·40 indicate a fair level of agreement, κ = 0·41–0·60 a moderate and κ = 0·61–0·80 a substantial level of agreement⁽³⁹⁾. Correlation coefficients were calculated with and without under and over reporters in energy intake as identified by the Goldberg cut-off method⁽⁴⁰⁾. However, we decided to include all participants in our analysis, because excluding under and over reporters (n 55) did not affect the ranking of participants in their diet quality (online Supplemental Table 2).

Reproducibility of the Eetscore FFQ was examined by intraclass correlation coefficients (ICC), where an ICC > 0·4 was considered fair to good and an ICC ≥ 0·75 excellent^(37,38). Moreover, the minimal detectable change (MDC) at the 95 % confidence levels for the total DHD2015-index score was calculated. The MDC₉₅ provides the minimal change in DHD2015-index score that indicates a true change, with 95 % certainty, which is not due to variation in performance or measurement error^(41,42). The MDC was calculated with the following formula: MDC₉₅ = 1·96 × pooled sd × √2(1-ICC).

All data were analysed using SAS statistical software version 9.4 (SAS Institute Inc.) and SPSS statistics 25 (IBM), and a P value of <0·05 was considered to be statistically significant.

Results

Participant characteristics

Sixty-eight per cent of the study sample were women and mean age of the study sample was 56·9 (sd 15·8) years (Table 2). Mean BMI was 24·2 (sd 3·8 kg/m²) and 65 % of the study sample had a high educational level. Thirty-five per cent of the study sample was classified as being overweight or obese. Men were older (mean 63·4 (sd 12·4) years) than women (mean 53·9 (sd 16·2) years) and had a significantly higher BMI (mean 25·4 (sd 3·6) kg/m²) than women (mean 23·7 (sd 3·8) kg/m²). Men also used more lipid-modifying, anti-diabetic and anti-hypertensive medication than women (P < 0·001).

Table 2.

General characteristics, mean scores and standard deviations of the total score of the Dutch Healthy Diet 2015 index (DHD2015-index) and its components based on the Eetscore FFQ in 541 women and 237 men

(Mean values and standard deviations)

	Women (n 514)				Men (n 237)				P *
	Mean	sd	n	(%)	Mean	sd	n	(%)
Age (years)	53·9	16·2			63·4	12·4			<0·001
BMI (kg/m2)	23·7	3·84			25·4	3·60			<0·001
Body weight (kg)	67·7	11·3			83·2	13·5			<0·001
Educational level†									0·08
Low			70	14			33	14
Middle			113	22			47	20
High			331	64			154	65
Smoking status									<0·001
Current			26	5			20	8
Former			177	34			134	57
Never			311	61			83	35
Medication use
Lipid-modifying			32	6			58	25	<0·001
Anti-diabetic			15	3			22	9	<0·001
Anti-hypertensive			74	14			62	26	<0·001
DHD2015-index (score)‡	114·4	16·0			102·8	18·0			<0·001
1. Vegetables (score)	7·2	2·8			5·7	2·9			<0·001
2. Fruit (score)	7·9	2·8			6·7	3·3			<0·001
3. Whole-grain products (score)	7·7	2·3			7·4	2·5			0·15
4. Legumes (score)	8·0	3·4			8·3	3·2			0·09
5. Nuts (score)	5·6	3·7			5·3	3·9			0·15
6. Dairy products (score)	5·9	3·3			6·6	3·3			0·009
7. Fish (score)	6·8	3·4			6·5	3·5			0·29
8. Tea (score)	6·5	4·1			4·4	4·1			<0·001
9. Fat and oils (score)	6·0	4·6			6·5	4·5			0·33
10. Coffee (score)	7·9	2·7			7·8	2·8			0·86
11. Red meat (score)	9·6	1·4			9·1	2·3			<0·001
12. Processed meat (score)	6·3	3·3			4·1	3·6			<0·001
13. Sugar-containing beverages (score)	8·2	2·6			7·1	3·3			<0·001
14. Alcohol (score)	8·2	3·2			7·3	3·2			<0·001
15. Na (score)	8·5	1·6			7·1	2·8			<0·001
16. Unhealthy choices (score)	3·9	4·1			2·9	4·0			0·004

^*Mann–Whitney U test and χ ² test were used to compare general characteristic values and the component scores between women and men and an ANOVA was used to compare the total DHD2015-index score.

^†Low education = primary school, vocational and lower general secondary education; moderate = higher secondary education and intermediate vocational training; high = higher vocational education and university.

^‡The total score ranges between 0 and 160 points.

The mean DHD2015-index score for the total study sample, based on the Eetscore FFQ, was 111 (sd 17·5) out of a possible maximum total score of 160 points (Table 2). Mean DHD2015-index scores were significantly lower for men than for women (103 (sd 18·0) points v. 114 (sd 16·0) points), and this difference increased after adjustment for age and BMI (104 (se 1·08) points v. 115 (se 0·72) points). Women scored significantly higher on the components vegetables, fruit, ratio whole to refined grains, tea, red meat, processed meat, sugar-containing beverages, alcohol, salt and unhealthy choices. Men scored significantly higher on the component dairy products.

Age was positively associated with the DHD2015-index score as derived from the Eetscore FFQ (P < 0·001, using sex-specific quartiles). Mean BMI was lower in the higher DHD-15 index quartiles (P < 0·001), while educational level was higher (P = 0·008) (Table 3). Smoking and medication use were not significantly associated with the DHD2015-index.

Table 3.

Participant characteristics across sex-specific quartiles of the DHD2015-index based on the Eetscore FFQ1 in 751 participants

(Mean values and standard deviations; numbers and percentages)

	Quartiles DHD2015-index derived from Eetscore FFQ																P for trend
	Q1 (n 190)				Q2 (n 197)				Q3 (n 182)				Q4 (n 182)
	Mean	sd *	n	%	Mean	sd	n	%	Mean	sd	n	%	Mean	sd	n	%
DHD2015-index score	89	11·1			107	6·9			117	6			131	7
Age (years)	53·7	16·2			56·2	15·5			57·6	16·4			60·4	14·2			<0·001
Body weight (kg)	75·9	16·1			72·7	13·9			71·8	13·9			69·8	11·5			<0·001
BMI (kg/m2)	25·1	4·2			24·4	3·8			24·1	3·9			23·3	3·2			<0·001
Smoking																	0·23
Never			93	48·9			104	52·8			101	55·5			96	52·7
Former			78	41·1			81	41·1			73	40·1			79	43·4
Current			19	10			12	6·1			8	4·4			7	3·8
Education																	0·008
Low			35	18·4			33	16·8			24	13·2			11	6
Intermediate			41	21·6			38	19·3			47	25·8			34	18·7
High			114	60			124	62·9			111	61			136	74·7
Medication use
Lipid-modifying drugs			26	13·7			25	12·7			21	11·5			18	9·9	0·70
Diabetic drugs			13	6·8			6	3			7	3·8			11	6	0·27
Anti-hypertensive drugs			35	18·4			32	16·2			28	15·4			41	22·5	0·29

^*The total score ranges between 0 and 160 points.

Relative validity of Eetscore FFQ compared with full-length FFQ

Mean time difference between filling in the Eetscore FFQ and the full-length FFQ was 1·7 (sd 1·15) month. The mean DHD2015-index score based on thirteen components and calculated from the Eetscore FFQ was 90·5 (sd 15·5) out of a possible total score of 130 points (Table 4), which was significantly higher than the total score calculated from the full-length FFQ (82·2 (sd 15·1)). Absolute agreement and limits of agreement were studied using a Bland–Altman plot (Fig. 1). The DHD2015-index score calculated from the Eetscore FFQ was 8·3 (sd 11·6) points higher than the score of the full length FFQ. The limits of agreement were −15·0 and 31·5 points. The Kendall’s tau-b coefficient between the two scores was 0·51 (95 % CI 0·47, 0·55) (Table 4), indicating a reasonably good ranking ability.

Table 4.

Total score and 13 component scores of the Eetscore FFQ and the full-length FFQ in 565 participants

(Mean values and standard deviations, Kendall’s tau-b coefficients, Spearman’s correlations and 95 % confidence intervals)

		Eetscore FFQ		Full-length FFQ		Tau-b	95 % CI	R	95 % CI
		Mean	SD	Mean	SD
1.	Vegetables	6·9	2·9	7·1	3·0	0·41	0·36, 0·46	0·55	0·48, 0·61
2.	Fruit	7·6	3·1	7·8	3·2	0·63	0·58, 0·68	0·72	0·67, 0·77
3.	Whole-grain products	7·7	2·3	7·0	3·1	0·22	0·15, 0·28	0·29	0·20, 0·37
4.	Legumes	8·0	3·4	8·2	3·4	0·43	0·35, 0·50	0·47	0·38, 0·55
5.	Nuts	5·6	3·8	5·0	4·0	0·48	0·43, 0·54	0·59	0·52, 0·65
6.	Dairy products	6·1	3·3	6·3	3·3	0·32	0·26, 0·38	0·43	0·35, 0·50
7.	Fish	6·7	3·4	–	–
8.	Tea	5·9	4·1	7·1	3·6	0·63	0·58, 0·68	0·72	0·66, 0·77
9.	Fat and oils	6·2	4·6	–	–
10.	Coffee	7·9	2·7	–	–
11.	Red meat	9·4	1·8	9·2	2·0	0·30	0·22, 0·38	0·32	0·24, 0·41
12.	Processed meat	5·7	3·5	6·0	3·5	0·55	0·51, 0·60	0·71	0·66, 0·76
13.	Sugar-containing beverages	7·9	2·9	6·9	3·5	0·50	0·45, 0·56	0·61	0·55, 0·68
14.	Alcohol	7·9	3·3	2·8	4·4	0·41	0·37, 0·45	0·45	0·40, 0·50
15.	Salt	8·1	2·2	7·7	2·7	0·36	0·31, 0·42	0·48	0·41, 0·55
16.	Unhealthy choices	3·6	4·1	1·0	2·6	0·39	0·33, 0·45	0·45	0·38, 0·52
	DHD2015-index 13 components*	90·5	15·5	82·2	15·1	0·51	0·47, 0·55	0·70	0·66, 0·75
	DHD2015-index all components†	111·3	17·3	82·2	15·1	0·47	0·43, 0·51	0·65	0·60, 0·70

^*The total score ranges between 0 and 130 points. The scores of components ‘7. Fish’, ‘9. Fats and oils’ and ’10. Coffee‘ were not available for full-length FFQ.

^†The total score ranges between 0 and 160 points.

Fig. 1.

Bland–Altman plot of total score based on thirteen components of the Dutch Healthy Diet 2015 index (DHD2015-index) based on the Eetscore FFQ and the full-length FFQ in 565 participants.

When comparing the mean DHD2015-index component scores between the Eetscore FFQ and the full-length FFQ, the largest absolute difference was observed for the component alcohol (mean difference 5·1 (sd 4·4) points) and the smallest absolute difference was observed for the component dairy products (mean difference 0·1 (sd 3·5) points). The lowest tau-b coefficient was seen for the component whole grain (0·22; (95 % CI 0·15, 0·28)). Tau-b correlations ranged between 0·2 and 0·4 for the components dairy products, red meat, salt and unhealthy choices. For the components vegetables, legumes, nuts, processed meat, sugar-containing beverages and alcohol, tau-b correlations ranged between 0·4 and 0·6. Tau-b correlations of 0·6 or higher were seen for the components fruit and tea. The Kendall’s tau-b coefficient between the grams of dairy product intake was 0·57 (95 % CI 0·53, 0·61). Based on the DHD2015-index score quartiles, 47 % of the participant were placed in the same quartiles, 43 % in the adjacent quartiles and 1 % in the extreme quartiles. The κ coefficient was 0·29 (95 % CI 0·24, 0·35), indicating a fair level of agreement between the two methods.

Reproducibility of the Eetscore FFQ

On average, participants completed the second Eetscore FFQ 3·8 (sd 0·82) months after the first Eetscore FFQ. Mean total DHD2015-index scores were 112·6 (sd 17·0) for the first Eetscore FFQ and 111·9 (sd 17·2) for the second Eetscore FFQ (Table 5), with a mean difference of 0·8 (sd 10·0) points (P = 0·16). The ICC between both Eetscore FFQ was 0·91 (95 % CI 0·89, 0·93). The largest difference in component scores was observed for tea (mean difference 0·4 (sd 2·4) points); the smallest difference was observed for coffee (mean difference 0·0 (sd 1·6) points). The lowest ICC were observed for the components red meat (0·71; (95 % CI 0·64, 0·77)) and legumes (0·72; (95 % CI 0·65, 0·77)), indicating a good reproducibility. The components fruit, tea and alcohol had an ICC of 0·9 or higher, indicating excellent reproducibility. All other components had an ICC between 0·8 and 0·9. The MDC for the total DHD2015-index score was 14·5 points.

Table 5.

Total score and 16 component scores of both Eetscore FFQ in 343 participants (intraclass correlation coefficient (ICC) and 95 % confidence intervals between both Eetscore FFQ)

(Mean values and standard deviations).

		Eetscore FFQ-1		Eetscore FFQ-2		ICC	95 % CI
		Mean	SD	Mean	SD
1.	Vegetables	6·9	3·0	6·7	2·9	0·82	0·77, 0·85
2.	Fruit	7·8	3·0	7·7	3·0	0·91	0·89, 0·93
3.	Whole-grain products	7·7	2·4	7·4	2·6	0·84	0·80, 0·87
4.	Legumes	8·1	3·4	7·9	3·5	0·72	0·65, 0·77
5.	Nuts	5·6	3·7	5·5	3·8	0·82	0·78, 0·85
6.	Dairy products	6·3	3·3	6·4	3·2	0·84	0·80, 0·87
7.	Fish	6·8	3·4	6·7	3·4	0·88	0·85, 0·90
8.	Tea	6·2	4·1	5·9	4·1	0·90	0·88, 0·92
9.	Fat and oils	6·1	4·6	6·1	4·6	0·80	0·75, 0·84
10.	Coffee	8·3	2·4	8·3	2·5	0·87	0·84, 0·90
11.	Red meat	9·5	1·7	9·6	1·4	0·71	0·64, 0·77
12.	Processed meat	5·8	3·5	5·7	3·6	0·87	0·84, 0·89
13.	Sugar-containing beverages	7·8	3·0	7·9	2·8	0·87	0·84, 0·90
14.	Alcohol	8·0	3·2	7·9	3·3	0·93	0·91, 0·94
15.	Na	8·1	2·1	8·4	2·0	0·80	0·76, 0·84
16.	Unhealthy choices	3·6	4·0	3·8	4·0	0·86	0·83, 0·89
	DHD2015-index*	112·5	17·2	111·9	17·4	0·91	0·89, 0·93

^*The total score ranges between 0 and 160 points.

Discussion

The Eetscore FFQ is a screener designed to assess diet quality based on adherence to the Dutch dietary guidelines by calculating DHD2015-index scores. We showed that DHD2015-index scores differed significantly between men and women. The DHD2015-index score was inversely associated with BMI, while it was positively associated with age and education. The Eetscore FFQ showed a moderate Kendall’s tau-b correlation with the full-length FFQ indicating that the Eetscore FFQ can be used for ranking of participants according to their diet quality. Furthermore, the ICC between both Eetscore FFQ was 0·91 showing good reproducibility.

Associations with individual characteristics

DHD2015-index scores derived from the Eetscore FFQ differed significantly between men and women. This difference became even larger after adjusting for age and BMI. Women scored significantly higher on the total DHD2015-index score, which was explained by higher intakes of fruit, vegetables and tea and lower intakes of red and processed meat, sugar-containing beverages and alcohol. Worldwide, studies have shown that women have a better diet quality than men⁽⁴³⁾. In general, women are probably more health-oriented and have better knowledge of nutrition than men^(43,44,45).

On average, older people had higher diet quality scores than younger people, which is also comparable with the results of other studies^(43,46). Furthermore, the DHD2015-index derived from the Eetscore FFQ was inversely associated with BMI. The DHD2015-index scores derived from a full-length FFQ or two 24-h dietary recalls previously showed similar associations with sex, age, BMI and education level in other general Dutch populations (EPIC-NL, DNFCS 2007–2010, NQ-Plus)^{(9,10,15,20,47,48)}. However, the DHD2015-index was previously also inversely associated with smoking, which was not seen in the present study^(9,20,47). This could be explained by the high percentage of highly educated participants and the low percentage of current smokers in the present study.

Relative validity

Using Bland–Altman analysis, absolute agreement showed an overestimation of the DHD2015-index score derived from the Eetscore FFQ compared with the DHD2015-index score derived from the full-length FFQ, based on thirteen components (mean difference 8·3 (sd 11·6) points). Furthermore, the Bland–Altman plot showed relatively wide 95 % limits of agreement.

The DHD2015-index score calculated from the Eetscore FFQ showed a reasonably good Kendall’s tau-b correlation (0·51; (95 % CI 0·47, 0·55)) with the DHD2015-index score calculated from the full-length FFQ. This Kendall’s tau-b correlation was slightly higher than the correlation between the full-length FFQ and the DHD-FFQ (0·40; (95 % CI 0·37, 0·43))⁽¹⁵⁾. The DHD-FFQ was developed to assess the DHD-index, the precursor of the DHD2015-index^(10,15). The observed correlation in the present study was comparable with that of Whitton et al.⁽⁴⁹⁾ who observed a Spearman’s correlation coefficient of 0·51 comparing the ‘Diet Screener’ and a 163-item FFQ in Singapore residents. A study of Rifas-Shiman et al. ⁽⁵⁰⁾ found a Spearman’s correlation coefficient of 0·61 comparing ‘PrimeScreen’ with a 131-item FFQ in an American population. Two studies by Schröder et al., both performed in a Spanish population, observed comparable Pearson’s correlation coefficients for the ‘short Diet Quality Screener’ (r = 0·61), the ‘Mediterranean Diet Adherence Screener’ (r = 0·52) and the ‘brief Mediterranean Diet Screener’ (r = 0·40)^{(18, 51)}. Furthermore, a correlation of 0·38 was observed for the ‘Diet Quality Score’, developed in the UK, derived from the short-form FFQ compared with a 217-item FFQ⁽¹⁶⁾. Correlation coefficients ranging between 0·5 and 0·7 are common in validation studies of dietary assessment methods^(37,38). The κ coefficient showed a fair level of agreement (0·29; 95 % CI 0·24, 0·35) between the DHD2015-index calculated from the Eetscore FFQ and with the DHD2015-index score calculated from the full-length FFQ. This κ coefficient was lower than coefficients found for comparable screeners that were κ ranges between 0·38 to 0·58^{(16,49, 52)}. Taking all above-mentioned correlation coefficients together, the correlation coefficient for DHD2015-index scores between the two methods was considered acceptable, even though the full DHD2015-index score could not be assessed for the full-length FFQ.

For the components whole grains, dairy products, red meat and unhealthy choices, the correlations between the DHD2015-index score based on the Eetscore FFQ and the full-length FFQ were lower than the expected value of 0·4. The low correlation for the component whole grains could be explained by the difference in food items for this component between the Eetscore FFQ and the full-length FFQ. For example, the Eetscore FFQ distinguishes between whole grain and white rice and pasta, whereas the full-length FFQ was not able to make this distinction. Second, the dairy product component was scored as an optimum component. For example, a score of 5 points is allotted when someone consumes one portion (150 g) of dairy products, but also when someone consumes four portions (600 g) of dairy products. The correlation between dairy product intake in grams was therefore higher than the correlation between component scores. Not surprisingly, also red meat showed a very low correlation between the two instruments. This could be explained by the grouping of the food items of the full-length FFQ. Some food items of the full-length FFQ consisted of both red meat and processed meat, while the Eetscore FFQ made a clear distinction between these two food items. The assumptions we made regarding the percentages of foods in the full-length FFQ that can be classified as red meat or processed meat may therefore deviate from the amounts actually consumed and therefore result in a different score. The low correlation for the component unhealthy choices could be explained by the smaller number of foods included in the Eetscore FFQ compared with the full-length FFQ (8 v. 15 items, respectively); for example, the consumption of candies, pancakes and pizza was not covered in the Eetscore FFQ, whereas these items were available from the full-length FFQ.

Although the correlation was acceptable, the largest mean difference in scores between the Eetscore FFQ and the full-length FFQ was seen for the component alcohol, which could be explained by the difference in questions. The Eetscore FFQ takes into account binge drinking, whereas it was not possible to assess binge drinking with the full-length FFQ. Additionally, the full-length FFQ used in the present study was not able to distinguish between types of coffee (filtered or unfiltered); therefore, the component scores for coffee could not be evaluated in the present study.

The Eetscore FFQ and the full-length FFQ showed a low correlation between Na intake (0·36; (95 % CI 0·31, 0·42)). This could partly be explained by the availability of information on salt added during cooking and at the dinner table in the Eetscore FFQ, whereas this information was lacking in the full-length FFQ. In general, FFQ are not suitable to assess Na intake, which usually underestimates true Na intake. The results on the salt component should therefore be interpreted with caution. Ideally, Na intake is estimated based on 24-h urinary Na, which is considered the gold standard⁽⁵³⁾.

Taking all the above suggested explanations for differences between the two FFQ together, the Eetscore FFQ may be better to distinguish between food items regarding the components whole grains, red meat, alcohol and coffee than the full-length FFQ and therefore may be better able to assess diet quality. This raises the question whether the full-length FFQ used in this study was the most appropriate FFQ to evaluate the Eetscore FFQ. In future studies, a method able to distinguish between types of coffee, to assess binge drinking and to distinguish between food items of whole grains and red meat, should be used to evaluate these components.

Reproducibility

The reproducibility of the Eetscore FFQ was assessed over an interval period of approximately 4 months. The observed ICC of 0·91 for the total DHD2015-index score and ICC ranging from 0·71 to 0·93 for the component scores indicate good to excellent reproducibility of habitual diet quality. The Eetscore FFQ also showed good reproducibility in comparison with other studies. For instance, an ICC of 0·69 was observed after a 4-month interval for the ‘Diet Screener’ assessing the Alternative Healthy Eating Index-2010 (AHEI-2010) in an Asian population⁽⁴⁹⁾. Furthermore, an ICC of 0·69 was observed for the reproducibility after a 1-month interval period of the ‘Mediterranean Diet Adherence Screener’ assessing adherence to the Mediterranean diet⁽⁵⁴⁾. Correlation coefficients between 0·5 and 0·7 are common in reproducibility studies of FFQ^(37,38). Thus, our findings indicate that the Eetscore FFQ is a reliable instrument for assessing adherence to the Dutch dietary guidelines over time.

Strengths

Both the Eetscore FFQ and the full-length FFQ were administered online, which is assumed to be less burdensome for the interviewee and expected to be less biased by social desirable answering⁽¹⁴⁾. The order in which participants received the questionnaires was at random. Therefore, it is unlikely that the order could have influenced the results. Furthermore, additional adjustments for the order of questionnaires did not alter our correlation coefficients notably (online Supplemental Table 3).

Limitations

The Eetscore FFQ was designed to capture the quality of the dietary pattern of the general Dutch population. In the present study, we evaluated the Eetscore FFQ in a population that might not be representative for the general Dutch population because of the large proportion of highly educated participants and the interest in participating in a study on nutrition and health. Furthermore, fewer people were overweight (BMI ≥ 25 kg/m²; 27 %) or obese (BMI ≥ 30 kg/m²; 8 %) in our study population compared with the general Dutch population (35 % and 15 %, respectively) indicating that our study population is a health-conscious population. We expect that the use of the Eetscore FFQ in a more representative general population will result in lower DHD2015-index scores, but in similar correlations between diet quality and health outcomes^(55,56). The Eetscore FFQ was initially developed for the adult population (18–69 years), based on data of the DNFCS 2007–2010⁽²¹⁾. However, also participants over 70 years of age were recruited from the EetMeetWeet study (n 147). Excluding them did not affect the ranking of participants in their diet quality; therefore, we decided to include data of all participants in our analysis (online Supplemental Table 4).

A full-length FFQ is no ‘golden standard’ reference method; therefore, we can only determine relative validity. Furthermore, the Eetscore FFQ and the full-length FFQ were designed similarly and may have correlated errors. This may cause an overestimation of relative validity^(37,57). Therefore, it is suggested to also assess relative validity between the Eetscore FFQ and multiple 24-h dietary recalls or multiple food records in the future. It is also suggested to evaluate the Eetscore FFQ against biological markers for dietary intake in future studies, since these are considered more objective than self-reported dietary assessment methods^(14,58).

Because of a technical problem with administering the full-length FFQ, that is, questions on fish intake and use of cooking fats were not saved, we were not able to calculate all components of the DHD2015-index. It was therefore also not possible to calculate associations between the DHD2015-index scores derived from the Eetscore FFQ and energy, and macro- and micronutrient intakes derived from the full-length FFQ. Others found significant positive correlations between diet quality scores and energy, and macro- and micronutrient intakes^{(15,18,51,54,59)}. However, the DHD2015-index was previously evaluated against the same full-length FFQ, as used in this study, and 24-h dietary recalls. This previous study showed significant associations for energy and several macro- and micronutrient intakes across quintiles of the DHD2015-index scores⁽⁹⁾. We therefore expect the associations between the Eetscore FFQ and energy and macro- and micronutrient intakes to be in the same direction.

The Eetscore FFQ was specifically designed to assess the DHD2015-index and is therefore not designed to estimate daily energy and macro- and micronutrients.

Conclusion

In conclusion, the Eetscore FFQ is a screener of diet quality assessing adherence to the Dutch dietary guidelines by calculating DHD2015-index scores. The results showed that DHD2015-index score differs between men and women. The score is also positively associated with age and educational level and inversely associated with BMI. The Eetscore FFQ was considered an acceptable screener to rank participants according to their diet quality, but relatively poor for assessing diet quality on the individual level. Further validation for this purpose is necessary. Moreover, the Eetscore FFQ showed good to excellent reproducibility and is therefore able to monitor diet quality of individuals.