Adaptation, validation and reproducibility of a short FFQ to assess food group intake in the population resident in the Basque country (Spain)

Abstract

Objective:

To adapt a short FFQ (SFFQ) and evaluate its relative validity and reproducibility to assess food group intake in a population resident in the Basque Country. Moreover, the possible influence of associated variables (such as education level) on its validity and reproducibility was determined.

Design:

Nine-day 24-h recalls (24HR) were used as a reference to explore validity over the course of 1 year. The degree of misclassification in the SFFQ was evaluated by a contingency table of quartiles and by Bland–Altman plots comparing SFFQ2 and 24HR. SFFQ was administered twice to explore reproducibility at 1 year.

Setting:

Basque Autonomous Community.

Participants:

Adults aged ≥21 years (n 99). The sample was randomly selected and representative of the target population.

Results:

For validity, statistically significant correlations were observed for more than half of the food groups, with the lowest correlations (r or ρ) for fat (–0·008) and the highest for other foods (0·963). The mean percentage of the subjects’ food intake that was classified into the same or adjacent quartile in both methods was 75·2 %. Reproducibility was explored by the correlation coefficient and ranged from 0·201 to 0·809, and 82·6 % of participants were in the same or adjacent quartile in both SFFQ. The associated variables did not seem to influence the validity and reproducibility of the SFFQ.

Conclusions:

An adapted SFFQ presented good reproducibility and validity for measuring most food groups in the target population, and these results did not seem to be influenced by the associated variables.

Cancer and circulatory system diseases are the principal causes of mortality in the Basque Autonomous Community⁽¹⁾ and Spain⁽²⁾, and these data are in line with the mortality rates of the European Region according to a WHO report⁽³⁾. Lifestyle, including diet, has an important role in the treatment and prevention of non-transmissible chronic diseases, as previously mentioned^(4–6). In this context, nutrition assessment is the first step to dietary modification in community-based interventional programmes⁽⁷⁾; however, it is widely recognised that a cause of uncertainty in the knowledge of the role of diet in the development of diseases is an intrinsic lack of precision in the methods used to assess dietary intakes⁽⁸⁾, and this inaccuracy may be impeding our ability to understand the impact of dietary factors on these diseases⁽⁹⁾.

Some dietary assessment methods, such as dietary records and dietary histories, are expensive and time-consuming, and they are not suitable for epidemiological studies⁽⁹⁾. Thus, FFQ, among other methods used to assess dietary intakes, has been and will probably continue to be the main tool used in epidemiological studies^(10,11) because they are easy to administer, they can assess dietary intake for a long period of time, and they are cheaper⁽¹²⁾. In this method, selected food items are listed and the frequency of intake and usual portion or serving sizes (average quantity per intake) are noted. To calculate the consumption of a food item, the portion size is multiplied by its intake frequency⁽¹³⁾.

The lesser number of food items found in an FFQ compared with open methods makes it easier and cheaper to collect, process and analyse data⁽¹⁴⁾. Moreover, when using a short FFQ (SFFQ), the time for data processing is reduced, and these questionnaires are less burdensome for the participants to complete⁽¹⁵⁾ than are other dietary evaluation methods. SFFQ has been previously developed and validated for different population groups^(16,17), but because of the variety of dietary habits that depend on ethnic, social and cultural backgrounds and the sensitivity of FFQ to the culture and ethnic background of the study population, FFQ must be tailored to target populations^(13,18).

Although some studies have validated different versions of FFQ for various objectives in the Spanish population, to the best of our knowledge no SFFQ has been validated for use in the residents of the Basque Autonomous Community. Therefore, we required an SFFQ to assess the dietary patterns and nutritional improvement in this population⁽¹⁹⁾. The current study was performed to adapt an SFFQ, evaluate its relative validity and reproducibility to assess food group intake in a population resident in the Basque Country, and determine the possible influence of associated variables (such as education level or weight status) on its validity and reproducibility. The original SFFQ used in the current study was validated in the general population in other regions of Spain⁽²⁰⁾.

Methods

Subjects and study design

The target population consisted of a sample of adults living in the Basque Autonomous Community (a region in northern Spain). The sampling design was polietapic and stratified by sex and age among individuals drawn randomly from the Basque Country population with the aim of obtaining a representative sample⁽²¹⁾. The sample size was chosen based on the findings of Cade et al. ⁽¹⁴⁾, which established that a sample size of at least 50–100 individuals is sufficient for testing FFQ validity.

Eligibility criteria for inclusion were established to recruit a healthy population and are as follows: (1) adults aged ≥18 years, (2) residents who had lived in the Basque Autonomous Community for >5 years, (3) free-living people without serious diseases requiring a special diet, (4) subjects not on a weight reduction diet and (5) subjects not related to other participants.

Subjects included in the validation study were recruited from February 2013 to February 2014. In total, 135 subjects were invited to participate in the current study, and the response rate was 73·3 %, that is, a total of ninety-nine subjects consented to join the current study and completed the first SFFQ (SFFQ1) and the first series of 24-h recalls (24HR). Among these subjects, eighty-two finished three series of 24HR and a second SFFQ (SFFQ2) and were included in the final analyses (56·1 % were females). Thus, the final participation rate was 60·7 %. Each of the series of 24HR included three phone interviews (on non-consecutive days) in which participants were asked to describe the foods and amounts consumed over the previous day. Information was collected for 3 days – 2 weekdays and 1 weekend day (1 day’s information in each of the interviews). A series of 24HR interviews were carried out at 4-month intervals during the study period.

The mean age was 47·3 years (range 21·0–88·0, sd 17·8); the mean BMI was 25·2 kg/m² (range 17·0–36·5, sd 4·2); and no significant differences were observed by sex for age and BMI (P > 0·05). The current study covered a time period of 1 year. The study design is shown in Fig. 1. At the beginning and end of this period, the participants completed the same SFFQ (SFFQ1 and SFFQ2, respectively) through a face-to-face interviews conducted by well-trained interviewers. Although the evaluation study was conducted via face-to-face interviews using an SFFQ, in later studies it was self-reporting of diet with instructions on how to complete the SFFQ with staff members providing assistance when needed. In any case, both face-to-face interviews and self-reported SFFQ were subsequently checked by a dietician.

Fig. 1.

Flow chart of the validation study. SFFQ, short FFQ; 24HR, 24-h recall

Short FFQ

After comparing the original SFFQ⁽²⁰⁾ with a quantitative study on food consumption in the Basque Autonomous Community⁽²²⁾, we identified twenty-two new items that corresponded to frequently consumed foods in this population. The food items that we added to the original SFFQ, which consisted of forty-five items, were whole grains (muesli, whole wheat flakes…); cookies with fibre; butter; margarine; sugar (added to milk, yogurt, coffee); honey; garlic and onions; fried foods or foods with added oil; whole bread; fried tomato sauce; mayonnaise; spicy (pepper, paprika, hot chili…); salt; sausages, foie-gras and pates; bacon; raisins, prunes, dried figs and dates; coffee or tea; cider; and aperitifs with alcohol (vermouth…) (food items are ordered in the same way as they were presented in the SFFQ).

Moreover, in the new questionnaire, items corresponding to milk and dairy products were itemised as: whole milk, semi-skimmed milk and skimmed milk; and full-fat yogurt and low-fat yogurt. These introduced three additional items because in the original there was only an item for each food. In the end, the SFFQ included sixty-seven food items. The distribution of new food items by food groups was as follows: processed meat (two new items), milk (two new items), dairy products (one new item), cereals (two new items), fruits (one new item), vegetables (one new item), sweets and sugar (three new items), other foods (three new items), other alcoholic drinks (two new items), fat (two new items), coffee or tea (one new item), added salt (one new item) and spicy foods (one new item). The distribution of the rest of food groups remained the same. Additionally, an open question was added with the objective of identifying foods that were not included in the SFFQ, such as soy products. When a food was written in the open questionnaire, we also asked how often they consumed it.

For each food item, participants were first asked whether they consumed that specific item. If the participant affirmed consumption, they were asked about the usual frequency of consumption (weekly or monthly) of one standard serving⁽²³⁾. With respect to vegetable oil(s) consumption, due to the difficulty in estimating it, we followed the same rule as the original authors⁽²⁰⁾; in all cases, we used the median quantity consumed by the study population; for the Basque Country population, this was 29·89 g/d⁽²²⁾. Additionally, we asked about the consumption frequency of fried or stewed foods and foods with oil added (e.g., salads), and about the type of vegetable oil(s) usually consumed with these foods (olive, sunflower, corn, soya or other); the combination of both data allowed us to estimate the proportion of daily consumption of each type of oil.

Answers about weekly or monthly intake frequency for each food item were transformed into daily servings. The daily intake of each food item (g/day) was calculated by multiplying the reported number of daily servings by the standard serving size of each food item in Spain⁽²³⁾. For items that included several foods, each food’s contribution was estimated with weighting coefficients obtained from usual consumption data⁽²²⁾. All food items consumed were entered into DIAL 2.12⁽²⁴⁾, a dietary assessment programme to estimate the total energy intake. The results of food group intake are expressed as daily consumption per 1000 kcal.

24-h recalls

To minimise errors in portion size, we used the protocol described in the Innovative Dietary Assessment Methods in Epidemiological Studies and Public Health – Standard Operating Procedure (IDAMES-SOP) for 24HR telephone interviews⁽²⁵⁾. Based on this protocol, a trained dietician collected the 24HR, and all completed recalls were checked by a research supervisor for accuracy. To avoid errors, participants were not informed about the day they would be contacted. Once they were contacted for interview via telephone, a short explanation about the procedure was given to each participant. They were asked about everything they had eaten or drunk the last 24 h and they then had to describe in detail all of the items, such as the type of food and its characteristics (full-fat dairy, fresh fish, etc.), preparation method and the dressings (type of oil used) or condiments used.

Simultaneously, the interviewer collected all the data in a note file, and once the interview was finished, the interviewer reviewed it. These data were registered into DIAL 2.12⁽²⁴⁾, and in case of mixed dishes for which the participants did not know the ingredients, a search in the database of standardised recipes^(26,27) was done.

Covariates

Information on sociodemographic and economic characteristics (sex, age, marital status and children, birthplace, place of residence, household members and number of rooms used for sleeping, educational attainment, economic activity and last work) and lifestyle (alcohol consumption, smoking status, duration of sleep and physical activity (PA)) were registered at the same time as SFFQ1 via a face-to-face interview based on questions employed by the Spanish Health Survey⁽²⁸⁾.

We estimated the crowding index using the household members and the number of rooms used for sleeping. This index is considered a good indicator of social status⁽²⁹⁾. The following variables were re-categorised for easier data analysis: age was dichotomised taking account of the median (younger subjects <46 years and older ones ≥46 years); marital status was separated into the following categories: not part of a couple (single, separated or divorced, widowed) and part of a couple (married and others); birthplace was dichotomised as Basque Country and others (any region apart from Basque Country); economic activity was separated into working, unemployed, retired and others (looking after home or family or others); and last work was categorised as steady salaried employee (cooperative members included), temporary salaried employee and others (employer, businessmen/women or others). Finally, BMI classification, alcohol consumption and smoking status were dichotomised as non-overweight and overweight/obese; non-current (those who do not drink or those who drank only in the past) and current drinkers (those who currently drink); and non-smokers and smokers, respectively.

Additionally, we obtained the following data about weight status: self-reported weight and height, desired weight and healthy weight. These data were obtained by asking the participants the following questions: ‘What do you usually weigh?’, ‘How tall are you?’, ‘What would you say is your ideal weight?’ and ‘What would your doctor say is your ideal weight according to standard height/weight charts?’)^(30,31). BMI was estimated from self-reported height and weight and classified according to the WHO criteria⁽³²⁾.

Apart from BMI, using these data the following variables were calculated: for body weight dissatisfaction (BWD), the desired weight minus the self-reported weight was divided by the self-reported weight and multiplied by 100⁽³³⁾; and for healthy weight perception (HWP), the healthy weight minus the self-reported weight was divided by the self-reported weight and multiplied by 100. The ‘weight difference percentage’ was considered a meaningful difference if it was ≥5 %. A 5 % cut-off was used in a previous study⁽³⁴⁾ to assess the desire to weigh less using the ideal–actual weight discrepancy. This method was preferred for its simplicity and utility⁽³⁵⁾.

Three other categories were created for BWD: desired weight less than self-reported weight (dissatisfaction by excess: relative difference ≤–5 %); desired weight greater than self-reported weight (dissatisfaction by defect: relative difference ≥5 %); and desired weight same as self-reported weight (satisfaction: –5 % < relative difference < 5 %). Finally, three categories were generated for HWP: healthy weight less than self-reported weight (relative difference ≤–5 %); healthy weight greater than self-reported weight (relative difference ≥5 %); and healthy weight same as self-reported weight (–5 % < relative difference < 5 %).

Statistical analysis

The data were analysed using SPSS for Windows (version 22.0, SPSS Inc.) and STATA13.0 (Stata Corp LP). The results are expressed as mean (standard deviation) and median daily intake as well as grams per 1000 kcal per day to adjust for energy density. The distribution of values was examined for normality by the Kolmogorov–Smirnov–Lilliefors test. Data collected through SFFQ and 9-day 24HR were non-normally distributed, with the exception of cereals, legumes and fat intake assessed by SFFQ1 and SFFQ2. Cereals, legumes and fat intakes were analysed using parametric methods. For non-normally distributed data, that is, for the rest of food items, non-parametric tests were performed. In the case of crude correlations, Spearman’s correlation coefficients were used, except for the three food groups whose distribution was normal for which Pearson’s correlation coefficients were used.

The validity of SFFQ was evaluated by comparing the average 9-day 24HR with the data obtained in the SFFQ. Differences in each comparison are presented as a percentage of consumption for the 9-day 24HR. To estimate these percentages of consumption relative to the 9-day 24HR, non-intakers’ data for the 9-day 24HR were not taken into account, since the division of a number by zero would result in infinity (SFFQ value × 100/0 = ∞). The differences in food group intake between two SFFQ and between SFFQ and the average of 9-day 24HR were assessed using paired t-tests (Student’s t or Wilcoxon signed-rank test). To determine if crude and age- and sex-adjusted correlations existed between the two methods, 9-day 24HR and SFFQ, and between two SFFQ, Pearson or Spearman correlations were conducted. To correct within-person errors in the measurement, age- and sex-adjusted correlations were multiplied by a de-attenuation factor, (1 + γ/n)^1/2, where γ is the ratio of within- and between-person variances and n is the number of repeats (n 11)⁽³⁶⁾. One-way ANOVA was used to calculate this ratio. We also estimated de-attenuation correlations associated with P-values⁽³⁷⁾.

Additionally, to measure the degree of agreement, subjects were classified into quartiles based on food group intakes obtained by the two methods (SFFQ2 and 24HR), and the percentages of agreement (same and adjacent quartiles) and complete disagreement (distant quartiles) are presented. Additionally, the agreement between two SFFQ was also calculated. Reproducibility and reliability were analysed using Cohen’s κ statistic for qualitative variables and intraclass correlation coefficients (ICC) for quantitative variables⁽³⁸⁾. The cut-off points for Cohen’s κ were: >0·8, excellent; >0·6, good/moderate; and >0·4, acceptable, while those for ICC were: <0·4, poor; 0·40–0·75, good or moderate; and >0·75, excellent⁽³⁸⁾.

Bland–Altman⁽³⁹⁾ plots on energy-adjusted values were also used to graphically check the agreement between the two methods; these plots would show the differences in intake between the two methods (SFFQ2 v. 9-day 24HR) against the mean intake of both measures ((SFFQ2 + 9-day 24HR)/2). The upper and lowest acceptable limits with 95 % CI were calculated with this formula: mean ± (1·96 × sd). For Bland–Altman plots, the same food groups as used in Fernández-Ballart et al. ⁽⁴⁰⁾ were selected for the current study.

Finally, the possible influences of associated variables on validity and reproducibility were assessed by χ ² test and Fisher’s exact test. In the current study, the results are shown for the same food groups as Bland–Altman plots. The associated variables examined in the current study were marital status, children, education level, economic activity, BMI, BWD and HWP classifications, PA level and smoking status. For these analyses, dichotomic categories of associated variables were crossed with the quartiles (same/adjacent v. distant) based on the food group intakes obtained using the two methods (SFFQ and 24HR). Variables were re-categorised in the following way to analyse the influence of covariates: marital status (without partner v. with partner), children (yes v. no), education level (primary education or without studies v. higher than primary education), economic activity (working v. others), BMI (under/normal weight v. overweight/obesity), BWD (dissatisfied v. satisfied), HWP (dissatisfied v. satisfied), physical exercise during free time (yes v. no) and smoking status (yes v. no). All tests were two-sided, and P-values <0·05 were considered statistically significant.

Results

The general characteristics of participants are shown in Table 1. The mean age was 47·3 years, and the majority of subjects were an active worker, residing in an urban area, without partners and had children and at least a secondary education. Regarding the economic activity, more males than females were working; in the case of men, there were more steady salaried employees, while among women, there were more temporary salaried employees. With regard to lifestyle, most of the participants had a normal weight, were current drinkers, were non-smokers and did physical exercise during free time; however, the analysis of these variables by sex showed that males smoked a greater number of cigarettes and had a higher PA level than found for women. Additionally, with respect to weight status, there were no significant differences by sex in BMI, BWD and HWP classifications.

Table 1.

Characteristics of the study sample: adults living in the Basque Autonomous Community

Variables	Total (n 82)		Males (n 36)		Females (n 46)		P *
	Mean	sd	Mean	sd	Mean	sd
Sociodemographic characteristics
Age (years)	47·3	17·8	45·2	16·6	48·9	18·8	0·346
Marital status (%)
Without partner	65·9		61·1		69·6
With partner	34·1		38·9		30·4		0·423
Children
Yes (%)	52·4		47·2		56·5		0·403
No. of children	1·0	1·2	0·8	1·0	1·2	1·4	0·172
Birthplace (%)
Basque Country	81·3		88·9		75·0
Other†	18·8		11·1		25·0		0·113
Place of residence (%)
Rural	23·1		15·2		28·6
Urban	76·9		84·8		71·4		0·093
Household crowding index	1·0	0·4	1·2	0·5	1·0	0·4	0·076
Education level (%)
Primary education or without studies	20·7		19·4		21·7
Secondary education/professional training	42·7		52·8		34·8
University degree	36·6		27·8		43·5		0·229
Economic activity (multiple answers) (%)
Working	47·6		55·6		41·3
Unemployed	18·3		22·2		15·2
Retired	20·7		22·2		19·6
Other‡	13·4		–		23·9		0·018
Last work (%)
Steady salaried employee	45·6		47·2		44·2
Temporary salaried employee	35·4		22·2		46·5
Other§	19·0		30·6		9·3		0·019
Weight status
BMI classification
Overweight/obesity	45·6		55·9		37·8		0·110
BWD (%)
Dissatisfaction by excess	49·3		40·0		55·8
Dissatisfaction by defect	6·8		10·0		4·7
Satisfaction	43·8		50·0		39·5		0·354
HWP (%)
HW less than self-reported weight‖	48·6		41·4		53·3
HW greater than self-reported weight¶	5·4		6·9		4·4
HW same as self-reported weight**	45·9		51·7		42·2		0·588
Lifestyle factors
Alcohol consumption (%)
Current drinkers	79·3		83·3		76·1		0·412
Smoking status
Smoker (%)	25·6		33·3		19·6		0·156
No. of cigarettes	2·9	6·7	4·5	7·8	1·7	5·5	0·046
Sleep
No. of hours per day	7·1	1·2	7·1	1·3	7·2	1·2	0·439
Main physical activity (%)
Sitting	37·8		25·0		47·8
Standing	34·1		30·6		37·0
Walking	19·5		27·8		13·0
High physical effort	8·5		16·7		2·2		0·017
Physical exercise during free time (%)
Yes	57·3		66·7		50·0		0·130

BWD, body weight dissatisfaction; HWP, healthy weight perception.

^*Significant P-values are highlighted in bold.

^†Any region apart from Basque Country.

^‡Looking after home or family and others.

^§Employer, businessman/women or others.

^‖Relative difference ≤–5 %.

^¶Relative difference ≥5 %.

^**5 % < relative difference < 5 %.

Validation and reproducibility analysis is shown in Tables 2 and 3; see online supplementary material, Supplementary Tables 1 and 2, and Fig. 2. In addition, some of these tables (in particular Table 2 and see online supplementary material, Supplementary Table 1) include data on reproducibility from the SFFQ. The mean daily intake of food groups (g/1000 kcal) (assessed by 9-day 24HR, SFFQ1 and SFFQ2) is presented in Table 2. Higher intake values were obtained with the SFFQ (SFFQ1 and SFFQ2) than with the 9-day 24HR for the following foods and beverages (sequenced in order of appearance in Table 2): fatty fish, other foods, light drinks, other alcoholic drinks, fat, broth and soups, added salt and spicy foods (P < 0·05). Among these products, the highest mean differences were observed for light drinks, fat, and broth and soups. The intake value of the following products was lower for the SFFQ1 and SFFQ2 than for the 9-day 24HR (sequenced in order of appearance in Table 2): total meat, processed meat, milk and cereals (P < 0·05). The highest mean differences, in favour of 9-day 24HR, were observed for total meat and milk. However, no significant differences were found between mean intakes obtained in the SFFQ1 and SFFQ2. For some foods, such as milk, the SFFQ mean intake was lower than the 9-day 24HR mean intake; however, the percentage reported in the 9-day 24HR was >100 %, at 230·3 %; this result could be due to a wide standard deviation.

Table 2.

Mean daily intake of food groups estimated from 9-day 24HR and two SFFQ

Food intake (g per 1000 kcal)	9-day 24HR			SFFQ1					SFFQ2
	Mean	sd	Median	Mean	sd	P * †	Median	% of 9-day 24HR	Mean	sd	P * ‡	Median	% of 9-day 24HR	P * §
Total meat	97·0	64·0	86·8	60·0	25·1	<0·001	60·0	112·3	65·5	47·1	<0·001	53·7	112·3	0·636
White meat	35·3	41·9	20·1	23·6	17·4	0·184	22·2	111·7	25·9	20·0	0·256	19·5	133·9	0·342
Red meat	32·9	30·5	30·3	24·7	16·7	0·031	22·5	118·6	26·9	23·4	0·140	20·1	138·0	0·487
Processed meat	28·8	27·4	21·9	11·7	8·6	<0·001	10·6	72·3	12·7	12·2	<0·001	10·3	79·7	0·969
Eggs	23·1	22·7	17·8	10·4	5·9	<0·001	8·9	79·1	11·5	9·1	<0·001	9·3	89·9	0·604
Total fish	37·2	38·7	26·8	37·6	25·9	0·369	32·7	154·7	38·8	25·8	0·401	35·5	135·8	0·603
Lean fish	19·7	25·4	8·2	18·8	16·2	0·467	15·3	128·1	18·8	15·9	0·409	15·8	90·0	0·885
Fatty fish	10·2	21·2	0·0	15·0	14·5	0·002	10·6	243·3	16·6	14·1	<0·001	12·0	267·3	0·237
Seafood	7·3	19·8	0·0	3·7	5·0	0·151	3·0	63·2	3·4	3·6	0·549	2·4	15·7	0·963
Milk	128·0	95·4	104·0	91·1	83·2	0·004	95·8	230·2	95·8	88·7	0·017	107·9	252·0	0·551
Dairy products	55·3	49·6	52·7	67·1	49·8	0·056	62·9	256·2	61·7	48·1	0·135	47·2	258·1	0·388
Cereals	87·5	49·6	78·2	67·4	23·2	0·007	64·7	113·8	65·2	30·9	0·001	62·1	105·4	0·555
Fruits	154·7	145·8	114·0	164·1	103·7	0·152	166·7	150·2	184·2	134·6	0·023	162·5	186·5	0·324
Vegetables	108·2	78·5	91·3	101·5	65·2	0·537	90·6	138·9	95·3	57·5	0·225	84·4	130·0	0·507
Legumes	10·3	12·7	4·4	9·4	5·2	0·807	8·9	122·5	9·3	5·7	0·752	8·3	127·9	0·924
Potatoes	42·3	42·6	31·5	30·8	23·3	0·064	27·8	125·7	31·2	24·3	0·106	27·0	108·2	0·758
Sweets and sugar	12·0	11·7	8·6	14·6	12·1	0·030	11·2	181·5	15·1	13·8	0·079	12·3	203·0	0·912
Other foods	0·6	2·8	0·0	5·7	8·2	<0·001	3·2	258·4	6·6	11·1	<0·001	3·4	186·3	0·585
Non-alcoholic drinks	93·0	107·9	68·4	65·3	83·2	0·017	40·2	181·2	76·7	96·1	0·090	42·2	383·4	0·279
Sugary drinks	42·9	84·4	0·0	46·4	61·7	0·184	21·3	91·4	49·9	66·4	0·198	25·4	130·5	0·819
Light drinks	6·8	23·8	0·0	19·0	45·2	0·007	0·0	220·8	26·8	69·9	0·004	0·0	421·6	0·224
Alcoholic drinks	57·0	95·7	14·4	34·2	54·6	0·059	16·3	115·7	46·5	96·7	0·593	21·1	245·7	0·095
Wine	18·2	37·9	0·0	13·8	28·5	0·697	0·0	130·3	16·4	33·8	0·437	2·9	156·2	0·382
Beer	34·1	78·8	0·0	14·3	34·7	0·008	0·0	49·3	24·4	89·6	0·188	0·0	92·8	0·073
Other alcoholic drinks	5·2	21·8	0·0	6·1	13·4	0·007	0·6	190·5	5·8	14·5	0·015	0·0	261·6	0·992
Fat	25·9	14·3	23·3	45·6	5·7	<0·001	45·3	233·3	45·1	6·9	<0·001	44·9	234·2	0·605
Coffee or tea	39·7	44·8	19·8	38·2	35·6	0·818	32·5	769·1	44·2	47·8	0·402	35·2	750·3	0·164
Broth and soups	3·6	9·9	0·0	19·2	17·0	<0·001	15·0	105·2	21·6	19·9	<0·001	16·4	122·3	0·311
Added salt	1·3	0·8	1·1	1·9	1·1	<0·001	1·9	195·6	1·5	0·9	0·185	1·5	153·7	0·005
Spicy foods	0·0	0·2	0·0	0·1	0·3	0·002	0·0	30·8	0·1	0·2	0·001	0·0	66·3	0·823

24HR, 24-h dietary recalls; SFFQ, short FFQ.

^*Significant P-values are highlighted in bold.

^†Differences in intakes of food groups between 24HR and SFFQ1.

^‡Differences in intakes of food groups between 24HR and SFFQ2.

^§Differences in intakes of food groups between SFFQ1 and SFFQ2.

Table 3.

Correlation coefficients for the estimation of energy density in food groups by 9-day 24HR and two SFFQ^*

Food groups (g per 1000 kcal)	Crude				Age- and sex-adjusted correlations
	9-day 24HR				9-day 24HR
	SFFQ1	P †	SFFQ2	P †	SFFQ1	P †	SFFQ2	P †
Total meat	0·016	0·889	0·286	0·009	−0·040	0·727	0·130	0·250
White meat	0·005	0·962	0·383	<0·001	−0·003	0·981	0·203	0·071
Red meat	0·272	0·014	0·379	<0·001	0·249	0·026	0·213	0·058
Processed meat	0·347	0·001	0·254	0·021	0·216	0·054	0·042	0·714
Eggs	0·216	0·052	0·287	0·009	0·169	0·134	0·085	0·451
Total fish	−0·020	0·859	0·203	0·067	0·019	0·870	0·185	0·100
Lean fish	0·115	0·304	0·154	0·168	0·087	0·441	0·210	0·061
Fatty fish	−0·157	0·158	0·038	0·733	−0·006	0·961	0·061	0·591
Seafood	0·024	0·832	−0·092	0·412	−0·054	0·637	0·083	0·466
Milk	0·266	0·016	0·112	0·319	0·346	0·002	0·045	0·694
Dairy products	0·447	<0·001	0·524	<0·001	0·437	<0·001	0·595	<0·001
Cereals	0·006	0·960	0·116	0·301	0·007	0·953	0·173	0·125
Fruits	0·520	<0·001	0·496	<0·001	0·416	<0·001	0·382	<0·001
Vegetables	0·299	0·006	0·366	0·001	0·264	0·018	0·350	0·001
Legumes	0·308	0·005	0·164	0·141	0·214	0·056	0·076	0·503
Potatoes	0·129	0·249	0·362	0·001	0·076	0·504	0·383	<0·001
Sweets and sugar	0·473	<0·001	0·288	0·009	0·345	0·002	0·189	0·093
Other foods	0·233	0·035	0·176	0·114	0·172	0·127	0·649	<0·001
Non-alcoholic drinks	0·311	0·004	0·232	0·036	0·270	0·015	0·179	0·113
Sugary drinks	0·393	<0·001	0·459	<0·001	0·489	<0·001	0·263	0·018
Light drinks	0·318	0·004	0·258	0·019	0·351	0·001	0·290	0·009
Alcoholic drinks	0·632	<0·001	0·705	<0·001	0·489	<0·001	0·609	<0·001
Wine	0·610	<0·001	0·578	<0·001	0·551	<0·001	0·712	<0·001
Beer	0·599	<0·001	0·647	<0·001	0·619	<0·001	0·656	<0·001
Other alcoholic drinks	0·332	0·002	0·428	<0·001	0·029	0·799	0·056	0·623
Fat	0·283	0·010	−0·008	0·941	0·204	0·070	0·092	0·419
Coffee or tea	0·583	<0·001	0·536	<0·001	0·476	<0·001	0·315	0·004
Broth and soups	0·229	0·038	0·210	0·058	0·261	0·019	0·201	0·074
Added salt	0·156	0·163	0·265	0·016	0·134	0·237	0·205	0·068
Spicy foods	0·001	0·993	0·109	0·331	−0·064	0·575	−0·011	0·925

24HR, 24-h dietary recalls; SFFQ, short FFQ.

^*Higher and lower correlation coefficients in each column are depicted as bold numbers.

^†Significant P-values are highlighted in bold.

Fig. 2.

Bland–Altman plot for agreement between SFFQ2 and the mean of 9-d 24HR for intake of vegetables (a), meat and meat products (b), potatoes (c) and legumes (d). SFFQ, short FFQ; 24HR, 24-h dietary recall

Table 3 presents the crude and adjusted coefficients of the associations between the SFFQ and 9-day 24HR. The crude correlation coefficients varied from –0·008 (fat) to 0·705 (alcoholic drinks). The means of correlation coefficients for all food groups were 0·277 and 0·304 for SFFQ1 and SFFQ2, respectively. The adjusted coefficients ranged between –0·011 (spicy foods) and 0·712 (wine). The mean age- and sex-adjusted coefficient for SFFQ1 was 0·231, while that obtained for SFFQ2 was 0·254. The de-attenuated correlations and ICC between food intakes estimated from the SFFQ and 9-day 24HR are shown in online supplementary material, Supplementary Table 1. The mean de-attenuated correlation coefficients were 0·286 (for SFFQ1) and 0·301 (for SFFQ2) and ranged from –0·013 (spicy foods) to 0·963 (other foods). According to ICC, good/moderate or excellent reproducibility was observed for both SFFQ for the following foods/beverages: dairy products, fruits, vegetables, sugary drinks, alcoholic beverages (especially wine and beer), and coffee or tea (sequenced in order of appearance in online supplementary material, Supplementary Table 1). Moreover, Bland–Altman plots showed that for all the analysed food groups, <10 % of participants were out of the limits of agreement (Fig. 2), indicating fairly good agreement between the SFFQ2 and 9-day 24HR. This analysis also showed a tendency for the differences to increase as the magnitude of measurement increased for all food groups except vegetables.

Reliability statistics for the SFFQ are presented in Table 4. Crude Pearson’s correlations ranged from 0·201 (fat) to 0·809 (alcoholic drinks), and age- and sex-adjusted coefficients ranged from 0·219 (spicy foods) to 0·823 (wine). ICC showed excellent or good reproducibility for the majority of analysed foods: processed meat, eggs, lean fish, milk, dairy products, fruits, vegetables, sweets and sugar, other foods, non-alcoholic drinks, sugary drinks, light drinks, alcoholic drinks, wine, beer, other alcoholic drinks, and coffee or tea.

Table 4.

Correlation coefficients between SFFQ1 and SFFQ2 and intraclass correlation coefficients (ICC)

Food groups (g per 1000 kcal)	Crude		Age- and sex-adjusteda		ICC	P *
	Correlation coefficients	P *	Correlation coefficients	P*
Total meat	0·406	<0·001	0·294	0·008	0·277	0·006
White meat	0·372	0·001	0·254	0·023	0·243	0·013
Red meat	0·417	<0·001	0·288	0·010	0·308	0·002
Processed meat	0·600	<0·001	0·401	<0·001	0·427	<0·001
Eggs	0·440	<0·001	0·507	<0·001	0·466	<0·001
Total fish	0·488	<0·001	0·416	<0·001	0·381	<0·001
Lean fish	0·433	<0·001	0·407	<0·001	0·427	<0·001
Fatty fish	0·347	0·001	0·216	0·055	0·158	0·077
Seafood	0·434	<0·001	0·284	0·011	0·377	0·017
Milk	0·577	<0·001	0·455	<0·001	0·492	<0·001
Dairy products	0·555	<0·001	0·436	<0·001	0·468	<0·001
Cereals	0·260†	0·018	0·286	0·010	0·250	0·011
Fruits	0·492	<0·001	0·244	0·029	0·451	0·004
Vegetables	0·500	<0·001	0·577	<0·001	0·604	<0·001
Legumes	0·361†	0·001	0·319	0·004	0·359	<0·001
Potatoes	0·343	0·002	0·285	0·010	0·297	0·003
Sweets and sugar	0·533	<0·001	0·530	<0·001	0·581	<0·001
Other foods	0·661	<0·001	0·408	<0·001	0·431	<0·001
Non-alcoholic drinks	0·702	<0·001	0·514	<0·001	0·585	<0·001
Sugary drinks	0·714	<0·001	0·497	<0·001	0·625	<0·001
Light drinks	0·611	<0·001	0·653	<0·001	0·738	<0·001
Alcoholic drinks	0·809	<0·001	0·624	<0·001	0·542	<0·001
Wine	0·755	<0·001	0·823	<0·001	0·822	<0·001
Beer	0·754	<0·001	0·767	<0·001	0·527	<0·001
Other alcoholic drinks	0·745	<0·001	0·735	<0·001	0·736	<0·001
Fat	0·201 †	0·070	0·228	0·042	0·197	0·037
Coffee or tea	0·626	<0·001	0·641	<0·001	0·611	<0·001
Broth and soups	0·455	<0·001	0·312	0·005	0·349	0·001
Added salt	0·293	0·007	0·251	0·025	0·230	0·018
Spicy foods	0·390	<0·001	0·219	0·051	0·237	0·016

SFFQ, short FFQ.

^*Significant P-values are highlighted in bold.

^†Pearson correlation coefficients; higher and lower correlation coefficients in each column are depicted as bold numbers.

The classification of SFFQ1 and SFFQ2 intakes into quartiles showed that the minimum percentage of subjects placed in the same or adjacent quartile was 72·0 % (this percentage was obtained for the food group ‘fats’). For the rest of the food groups, the percentage of agreement between SFFQ1 and SFFQ2 was ≥75·6 % (see online supplementary material, Supplementary Table 3). The mean percentage of participants placed in the same or adjacent quartiles was 82·6 %. Cohen’s κ indicated good, moderate or acceptable agreement for the following foods/beverages (ordered from higher to lower κ values): sugary drinks, processed meat, light drinks, total fish and alcoholic drinks.

However, 75·2 % of participants’ food intakes derived from the SFFQ2 and 9-day 24HR were classified into the same or adjacent quartiles (Table 5). Cohen’s κ ranged from –0·045 (fat) to 0·632 (alcoholic drinks), with an average value of 0·257, indicating acceptable to good agreement for the following products (ordered from higher to lower κ values): alcoholic drinks, beer, coffee or tea, wine, dairy products, fruits and sugary drinks. Regarding the influence of associated variables (marital status, having children, education level, economic activity, BMI, BWD and HWP classifications, PA level and smoking status) on validity and reproducibility, no association was found (P > 0·05). For example, the results for vegetables (SFFQ1 v. SFFQ2 and SFFQ2 v. 9-day 24HR) by marital status were χ ² = 0·457 (P > 0·05) and 1·553 (P > 0·05), respectively; the results for meat and meat products (SFFQ1 v. SFFQ2 and SFFQ2 v. 9-day 24HR) by having or not having children were χ ² = 3·579 (P > 0·05) and 0·824 (P > 0·05), respectively; the results for potatoes (SFFQ1 v. SFFQ2 and SFFQ2 v. 9-day 24HR) by education level were P = 0·526 and 0·175, respectively (Fisher’s exact test); and the results for legumes (SFFQ1 v. SFFQ2 and SFFQ2 v. 9-day 24HR) by economic activity were χ ² = 1·059 (P > 0·05) and 0·010 (P > 0·05), respectively.

Table 5.

Misclassification and weighted κ between SFFQ2 and 9-day 24HR

Food groups	Percentage of agreement		P *
	SFFQ2/9-day 24HR
	Same or adjacent (%)	Weighted κ
Total meat	69·5	0·194	0·040
White meat	72·0	0·400	<0·001
Red meat	73·2	0·304	0·003
Processed meat	70·7	0·204	0·032
Eggs	78·0	0·264	0·009
Total fish	75·6	0·154	0·081
Lean fish	65·9	0·121	0·132
Fatty fish	72·0	−0·043	0·662
Seafood	65·9	−0·098	0·821
Milk	64·6	0·114	0·150
Dairy products	81·7	0·433	<0·001
Cereals	70·7	0·184	0·048
Fruits	80·5	0·433	<0·001
Vegetables	69·5	0·264	0·009
Legumes	61·0	0·053	0·314
Potatoes	79·3	0·353	<0·001
Sweets and sugar	73·2	0·304	0·003
Other foods	78·0	0·130	0·048
Non-alcoholic drinks	69·5	0·204	0·032
Sugary drinks	82·9	0·430	<0·001
Light drinks	76·8	0·205	0·017
Alcoholic drinks	86·6	0·632	<0·001
Wine	87·8	0·466	<0·001
Beer	92·7	0·605	<0·001
Other alcoholic drinks	82·9	0·374	<0·001
Fat	62·2	−0·045	0·657
Coffee or tea	86·6	0·532	<0·001
Broth and soups	79·3	0·199	0·016
Added salt	73·2	0·234	0·017
Spicy foods	75·6	0·092	0·132

SFFQ, short FFQ; 24HR, 24-h dietary recalls.

^*Significant P-values are highlighted in bold.

Discussion

In the current study, we adapted and assessed the validity and reproducibility of an SFFQ in a population resident in the Basque Country; additionally, we analysed the influence of some variables (such as education level or weight status) on validity and reproducibility. To our knowledge, no previous study has investigated these aspects in this population. The general characteristics of the sample, with regard to having children, place of residence, education level and economic activity, were similar to those of the general Basque population⁽⁴¹⁾. Relative to weight status, the percentage of overweight/obesity in our sample was similar to the latest health survey carried out in the Basque Autonomous Community⁽⁴²⁾, and in both cases, the percentage of men with overweight/obesity was higher than that of women.

The results indicate that the SFFQ is a reasonably good method for dietary assessment in relation to 24HR for many food groups. In addition, the number of items in this FFQ could be considered optimal, because it matched the median number used in other studies⁽¹⁴⁾. It is important to emphasise that there is no fixed number of items for FFQ, and each item should be designed to provide information for which it was intended⁽⁴³⁾.

The design of the current study was the same as that used in other validation and reproducibility studies^(44,45). In our case, SFFQ was administered at the beginning and the end of study, and 24HR was performed during the year at 4-month intervals, in agreement with the theory that considers a sufficient number of days to be necessary during a term (typically 1 year) to represent average dietary intakes⁽¹⁴⁾.

In relation to the validity of SFFQ and in agreement with other authors, we also found that food group intakes assessed by SFFQ were, in general, higher than those estimated by the 9-day 24HR^(44,46,47). A possible explanation for this could be that participants might overestimate the consumption of ingredients of mixed dishes or meals and they might consider these quantities as a full portion even though they are smaller⁽⁴⁴⁾. Foods with the highest mean differences between SFFQ and 9-day 24HR (in favour of SFFQ) were light drinks, fat, and broth and soups. In case of drinks (light drinks and broth and soups), the overestimation could be because the consumption of these products is not frequent, which means estimations are less accurate⁽⁴⁸⁾. In relation to fat intake, it should be noted that for the SFFQ, in all cases we used the median of quantity consumed by the Basque Country population⁽²²⁾, and this quantity was higher than those registered by the 24HR. It should be noted that individual fat intake is difficult to estimate because it is usually added in the cooking of main courses for the whole family⁽⁸⁾.

The correlations between SFFQ and 9-day 24HR were significant for more than half of the studied food groups, with the lowest correlations being for fat and spicy foods and the highest correlations being for alcohol drinks and other foods. The low correlation for spicy foods could be related to the low accuracy of estimation of foods consumed in small quantities⁽⁴⁸⁾. With regard to fat, as mentioned above, we used the median of quantity consumed by the study population in the FFQ. In this sense, the data recorded for fat by 9-day 24HR showed high intra- and inter-individual variability, and this is why the de-attenuated correlation coefficient was higher than the crude correlation for this food. In addition, in some food groups, such as meat or fish, the correlation coefficients were <0·4, and this might reflect difficulties in recognising and classifying these categories⁽⁴⁹⁾ even though there were some examples of each item in our questionnaire. On the other hand, the high correlation found for alcohol drinks and other foods could be due to using beverage-specific questions in the SFFQ⁽⁵⁰⁾ and may indicate that the intake of these products is associated with particular situations, such as weekends; thus, it may be easier for participants to remember the quantity more precisely. Relative to de-attenuated correlations, they were in almost all cases similar to adjusted correlations, and as Fernández-Ballart et al. ⁽⁴⁰⁾ described, this small difference could be due to relatively low within-individual variation with regard to the between-individual variations and/or because of the large number of days recorded (9 d).

In general, correlations between SFFQ2 and 9-day 24HR were slightly higher than those found between SFFQ1 and 9-day 24HR. This has also been reported in other studies and could be the result of changes that occur throughout the duration of participation in the study⁽⁴⁵⁾ or because of a learning effect⁽⁴⁴⁾ by which participants were able to observe their diets more conscientiously during the second SFFQ⁽⁵¹⁾.

The results of both Bland–Altman plots and classification by quartiles confirmed that the agreement between SFFQ2 and 9-day 24HR was fairly good, where the percentage of participants being out of the limits of agreement was <10 %, and the mean percentage of participants’ food intakes being classified into the same or adjacent quartiles was 75·2 %. Although meat and meat products showed worse agreement when the intake of these foods was higher, other authors have also observed this phenomenon and considered it a measurement error proportional to mean intake⁽⁴⁰⁾.

With regard to the reproducibility of the adapted SFFQ, the intakes of more than half of food groups were higher in SFFQ2 than in SFFQ1. The same result occurred in the Shanghai Women’s Health Study⁽¹⁸⁾ and in the Tehran Lipid and Glucose Study⁽⁵²⁾. This could be because the participants, after the first SFFQ1 and ninth 24HR, became more aware of their intakes; however, other researchers have suggested a contrary effect, that is, intake was lower in the second FFQ than in the first^(44,53). Moreover, it should be noted that, in our case, the interval between both FFQ was 1 year; thus, the differences observed could have been due to changes in participants’ diets and variations in their responses⁽⁴⁷⁾.

Crude and age- and sex-adjusted correlation coefficients for reproducibility in the current study were similar to those reported by Ogawa et al. ⁽⁵⁴⁾ and higher than those reported by others^(44,55). Regarding the significant level of correlations between SFFQ, we found that all the coefficients were significant except for fat in crude correlations, and for fatty fish and spicy foods in adjusted correlations. These results are similar to previously reported data⁽⁵⁶⁾. In addition, the results of classification by quartiles confirmed that the agreement between both SFFQ was good, as the mean percentage of food intakes that was classified into the same or adjacent quartiles was 82·6 %, similar to other studies^(44,45). Finally, associated variables such as education level, having children, weight status or lifestyle did not seem to influence the validity and reproducibility of SFFQ. In any case, a larger sample size would be needed to confirm the results observed in the current study. Although limited data are available regarding the influence of variables on the validity and reproducibility of FFQ, other authors⁽⁵⁷⁾ also could not find a relationship between education level and reproducibility of an FFQ. To our knowledge, no studies reported to date have evaluated the possible association between education level and validity of an FFQ.

Strengths and limitations

A possible limitation of the current study, as has been reported for other validation and reproducibility studies, is that these results may not be generalisable to other populations⁽⁵⁸⁾. Study strengths include that the results are discussed in detail, which show the challenges associated with the registration of different foods. Food items were not regrouped because single items are usually better than grouping, since similar foods are easy to be differentiated and grouped items could be confusing to participants⁽¹⁴⁾. In addition, this SFFQ can be very useful to estimate the intake of some specific foods, such as different alcoholic beverages (with different alcoholic graduation and bioactive compounds), as well as foods that are usually consumed in small quantities (e.g. spicy foods). In any case, in view of the results of the present research and in order to improve the accuracy of self-reported data, the final version of this SFFQ included, in addition to quantities of food servings (grams or millilitres)⁽²³⁾, its equivalent in household measures. Other strengths of the current study are the design and the fact that we analysed the independent effect of associated variables on reproducibility and validity.

Conclusions

The adapted SFFQ presents good reproducibility and validity for measuring most food groups in a population resident in the Basque Country, and it did not seem to be influenced by associated variables such as education level or weight status. However, in future studies, the intakes of food groups estimated by this SFFQ that showed a low validity should be used and interpreted with caution.