The Cardio-Med survey tool: development and pilot validation of a FFQ in a multicultural cardiology cohort

Abstract

Objective:

(i) Describe the development of a multipurpose Cardio-Med survey tool (CMST) comprising a semi-quantitative FFQ designed to measure dietary intake in multicultural patients with or at high risk of CVD and (ii) report pilot evaluation of test–retest reliability and validity of the FFQ in measuring energy and nutrient intakes.

Design:

The CMST was developed to identify CVD risk factors and assess diet quality over 1 year using an FFQ. Design of the ninety-three-item FFQ involved developing food portion photographs, and a list of foods appropriate for the Australian multicultural population allowing the capture of adherence to a Mediterranean diet pattern. The FFQ was administered twice, 2 weeks apart to assess test–retest reliability, whilst validity was assessed by comparison of the FFQ with a 3-d food record (3DFR).

Setting:

The Northern Hospital and St Vincent’s Hospital, Melbourne, Australia.

Participants:

Thirty-eight participants aged 34–81 years with CVD or at high risk.

Results:

Test–retest reliability of the FFQ was good: intraclass correlation coefficient (ICC) ranged from 0·52 (Na) to 0·88 (alcohol) (mean 0·79), with energy and 70 % of measured nutrients being above 0·75. Validity was moderate: ICC ranged from 0·08 (Na) to 0·94 (alcohol) (mean 0·59), with energy and 85 % of measured nutrients being above 0·5. Bland–Altman plots demonstrated good levels of agreement between the FFQ and 3DFR for carbohydrates, protein, alcohol, vitamin D and Na.

Conclusions:

The CMST FFQ demonstrated good test–retest reliability and moderate validity for measuring dietary energy and nutrients in a multicultural Australian cardiology population.

In Australia, CVD is the second largest contributor to death despite advancements made in cardiovascular health and mortality reduction over the last few decades⁽¹⁾. The prevention of CVD at both primary and secondary level is strongly influenced through modification of lifestyle-based risk factors including smoking, physical activity and diet. In relation to diet, adherence to the Mediterranean dietary pattern (MDP) has been inversely associated with CVD risk factors⁽²⁾ and mortality^(3–5). The MDP is primarily plant based, characterised by a high intake of fruits, vegetables, legumes, wholegrains, nuts and extra virgin olive oil. Dairy products and fish are consumed in moderate amounts as is alcohol, predominantly red wine consumed with meals. Meat and meat products are consumed in small portions and less frequently⁽⁶⁾. Plant-based traditional diets from other cultures, including Asian populations⁽⁷⁾, have many similarities to the MDP and have been associated with low CVD risk^(8,9).

Accurate measurement of habitual diets in individuals and populations is inherently complicated due to recall bias and the impact of measurement on modifying intake. Epidemiological studies frequently employ the use of FFQ as their preferred dietary assessment method because of their enhanced ability to estimate habitual intake^(10,11), ease of administration and low-cost burden compared with other methods, such as weighed food records and 24-h diet recalls^(11,12). Numerous FFQ exist in Australia; however, they have a limited capacity to capture culturally specific dietary behaviours including food preparation methods that may influence adherence to the MDP. Australia is a multicultural population; with nearly half of Australians being overseas born or have at least one parent born overseas⁽¹³⁾ comes with a broad range of food cultures and differing food consumption patterns compared with other populations⁽¹⁴⁾. This then limits the applicability for international FFQ to be utilised in Australia due to being culture specific with the inclusion of unfamiliar foods.

The Cardio-Med survey tool (CMST) was specifically designed in response to the limitations of currently available FFQ coupled with the knowledge that CVD poses a significant health burden in Australia, and the MDP is associated with a reduction in CVD. It was important that the CMST had multiple utilities including the ability to capture CVD risk factors, alongside accurate assessment of dietary intake and being culturally sensitive to the Australian population through the inclusion of culturally unique food preferences and consideration of cuisine and diverse dietary practices via a purpose-built FFQ^(14,15). Additionally, the FFQ has the ability to assess adherence to traditional dietary patterns such as the MDP. Our intentions for the CMST will be for use in assessing CVD risk, dietary intake and adherence to the MDP in an Australian cardiology population and within an Australian Mediterranean diet intervention for secondary prevention of CHD⁽¹⁶⁾. Furthermore, the CMST has the potential for utility in other countries with a multicultural population similar to Australia.

As far as we are aware, no such tool to date has been developed. The aim of the current paper was to (i) report the development of the multipurpose CMST comprising a semi-quantitative FFQ and (ii) evaluate the test–retest reliability of the FFQ and its validity against a 3-d food record (3DFR), in measuring energy and nutrient intakes amongst patients with CVD.

Methods

Development of the Cardio-Med survey tool

General overview

The CMST (see online supplementary material) was designed to be a self-administered multipurpose questionnaire for use in a population at high risk of CVD to enable identification of CVD risk factors and to assess diet quality and adherence to the MDP.

The CMST was developed using pre-validated questions from several existing tools with additional questions tailored to the Australian cardiology population context. The tool was field tested with health professionals and a target population group for question clarity prior to this validation study with numerous revisions occurring. The final version of the CMST survey consists of 133 questions and contains (i) a health and lifestyle section comprising thirty-six questions and (ii) a dietary intake section comprising ninety-seven questions further broken down to; (i) a forty-item FFQ, (ii) fourteen portion size questions, (iii) twenty-nine additional food questions and (iv) fourteen food habit questions. Food habit questions were not within the scope of assessing validity of the FFQ and were excluded. Thus, eighty-three dietary intake questions (collectively referred to as ‘FFQ’ in the current paper) remained for validation purposes in the current study.

Health and lifestyle

This section focused on lifestyle and cardiovascular risk and contained thirty-six questions which probed on the domains of ethnicity, income, education, employment, medical history, medication and supplement use, smoking and physical activity. Questions were derived from validated health and lifestyle questionnaires and national surveillance survey data to ensure optimal comprehension and validity. Modifications were made to the wording and/or response options of select questions to ensure appropriateness for our study population. The operationalisation of cultural groups was constructed from questions pertaining to country of birth, mother/father’s country of birth, language spoken at home and whether they were Indigenous Australians, a methodology analogous to that employed by Australian government agencies⁽¹⁷⁾.

Dietary intake: forty-item FFQ

This section within the CMST was developed to assess habitual diet over the preceding 12 months, in addition to capturing elements of different cuisines within a multicultural setting with an ability to measure adherence to the MDP. Development of the FFQ was modelled on two Australian FFQ: (i) the Dietary Questionnaire for Epidemiological Studies Version 2 (DQESv2), designed by the Cancer Council of Australia over 20 years ago for 40- to 69-year-old Australian residents born in Australia, Greece or Italy⁽¹⁸⁾ and (ii) the MEDiterranean ISlands-Australia (MEDIS-Australia) FFQ⁽¹⁹⁾, a modified version of the validated MEDIS FFQ⁽²⁰⁾, designed to explore adherence to a Greek MDP in elderly Greek islander-born migrant Australians. Utilisation of the DQESv2 related to the content of foods contained relevant to the Australian population, with utilisation of the MEDIS-Australia FFQ for identification of dietary and lifestyle practices relevant to a traditional MDP.

Food items for inclusion were based on (i) the principles of the traditional Greek MDP⁽³⁾, (ii) the fourteen-point Mediterranean Diet Adherence Screener from the Spanish PREvencion con DIeta MEDiterranea (PREDIMED) study⁽²¹⁾, (iii) food items included in previously validated FFQ^(12,18,19), (iv) Australian dietary guideline recommendations⁽²²⁾ and (v) National Nutrition Survey data⁽²³⁾. These surveys were utilised to ensure a selection of diverse foods was included; key food and beverage groups consumed by the Australian population were covered and dietary practices relevant to the MDP were identified. Components (foods and preferences) correlating with the MDP from the Mediterranean Diet Adherence Screener questionnaire were important to include due to the notable reduction in CVD risk observed in the PREDIMED study⁽⁴⁾.

The forty food items included were grouped into eight sections: (i) vegetables, (ii) fruit, (iii) meat and fish, (iv) cereal products, (v) dairy, (vi) other (including nuts, snack foods, chocolate and take away foods), (vii) herbs and spices and (viii) condiments. A novel/unique feature was the inclusion of culinary herbs and spices, not typically included in FFQ but are common in traditional diets such as the MDP, and many of which are associated with the modulation of chronic disease risk, including CVD risk reduction through their anti-inflammatory effects and reduction of blood pressure^(24,25). Identifying consumption of herbs and spices was deemed important due to their use for flavouring foods to improve palatability, especially vegetables, which may enhance their consumption⁽²⁴⁾. It was important that the FFQ had the ability to capture food items, consumed within different traditional diets in a multicultural Australian population, including those which are consistent with a MDP. Over the past decade, there has been an increase in the number of migrants in Australia from Asia⁽¹³⁾. Being cognisant of this and the diversity of cultures in Australia, it was important that the FFQ had the ability to capture food items consumed within different traditional diets in a multicultural Australian population, including those which are consistent with a MDP. For example, non-soya legumes are part of the MDP; however, soya-based foods consumed by Australians of Asian cultural background would be used instead to determine adherence to a MDP. Ten response categories were included to measure frequency of consumption of foods within the forty-item FFQ: ‘never’, ‘once per month’, ‘two to three time per month’, ‘once a week’, ‘twice a week’, ‘3–4 times per week’, ‘5–6 times per week’, ‘once a day,’ ‘twice a day’ and ‘three times a day’.

Dietary intake: portion size

Portion size estimates were obtained using either food portion photographs (fourteen commonly consumed foods in the FFQ) or from a portion size database developed for any food which did not have an associated food portion photograph. Pre-defined portion sizes estimated from standard recipes, product information, natural units, Australian dietary guidelines, Australian National Nutrition Survey data or Australian food composition tables were used to compile the portion size database. Food portion photographs, another unique feature of the FFQ (more commonly utilised within online FFQ)^(26,27), were created by the research team specifically as a means of enhancing accuracy in perceiving serve sizes and consequently accuracy of measuring food intake. Majority of foods were photographed in a standardised setting, that is, same plate, cutlery and lighting. Three photographs were provided for each food representing either 1, 2 or 3 standard serves according to Australian dietary guidelines⁽²²⁾. To accommodate for a variability in portion size, options were available to select above and below each portion photograph (see online supplementary material).

Dietary intake: additional food questions and food habit questions

Additional food and beverages not contained in the forty-item FFQ, along with food habit questions, were included in a separate section. There were twenty-nine additional food and beverage questions relating to the consumption of fats, oils, dressings, sugar, milk, bread, eggs, fruit, alcohol, non-alcoholic beverages and types of food/beverages including cheese, yoghurt, nuts, seeds, chocolate, snacks, takeaway food, milk, tea and coffee. Fourteen questions pertaining to diet-related behaviours (food habits) were included, questions related to salt use, fat-trimming habits, cooking/shopping, adherence to specialised diets, food change desires, cultivation of home gardens and cooking methods to identify cuisine (e.g., frying and slow-cooked (wet/moist) casserole dishes).

Participant recruitment

This pilot validation study was part of a cross-sectional study aimed at evaluating diet quality and the association between diet quality, specifically adherence to a MDP, and severity of CVD in an Australian cohort of patients with established CVD or at high risk of developing CVD. Sixty-one participants were recruited as part of the cross-sectional study from two major hospitals in Melbourne, Australia: The Northern Hospital and St Vincent’s Hospital. Recruitment occurred through referral of patients to the research team by cardiologists or nurses, or opportunistically by researchers from cardiology outpatient clinics, cardiac rehabilitation groups, cardiac catheter laboratory or from the inpatient coronary care ward. Inclusion criteria were age ≥18 years and adequate English comprehension for both reading and writing. Of the sixty-one participants recruited, forty-three participants participated in the validation study. Non-compliance and study withdrawal were due to change of mind, medical reasons, insufficient time, lost information in the mail and participant’s being non-contactable.

Study design

On the day of recruitment on site at the hospital, participants had anthropometric and blood pressure measurements taken and completed the first CMST. Participants were instructed to complete a second CMST 2 weeks later at their home and a 3DFR within 1 week of completing the initial CMST. Participants were provided with prepaid envelopes to return all completed documents. Medical history and medication use were collected via self-report data from the CMST and from the participant’s medical records. Sociodemographic and lifestyle information was collected from self-report data in the CMST. Self-report physical activity data were collected using the International Physical Activity Questionnaire Short Form included within the CMST⁽²⁸⁾.

Measurements

Participant’s blood pressure and anthropometric measures including height, weight, waist, hip and neck circumference were carried out by one of the four trained research dietitians using standard techniques. Standard equipment was used including digital blood pressure monitor (OMRON, Tp9, IntelliSense), electric scales (TANITA, BC-545), retractable tape measure and portable stadiometer (CHARDER, HM-200P).

Validity assessment

The FFQ was compared with the 3DFR to assess validity by means of comparing energy and nutrient intake data^{(11,12,20,29)}. This record was to cover three typical days of the participant’s diet: two weekdays (non-consecutive) and one weekend day. Many studies have demonstrated reasonable validity using food records between 2 and 5 d without significant participant burden^(20,30). Detailed written instructions on how to complete the 3DFR were provided; participants were instructed to include food type, brand, method of preparation and recipes. Participants were advised to record all quantities of food and beverages at the time of consumption and to record measurements by weighing items. A shape chart, ruler and images were provided to assist participants to measure foods that could not be weighed or not quantifiable using household measures.

Test–retest reliability assessment

Participants were instructed to self-complete two CMST at an interval of 2 weeks apart to assess test–retest reliability of the FFQ. A repeat administration time of 2 weeks was selected based on similar validation studies^(29–31) and from a review highlighting that correlations were higher when repeat administrations of FFQ were 1 month or less apart, compared with 6–12 months⁽¹¹⁾.

Returned 3DFR and CMST were revised, and information requiring clarification was performed by a research dietitian via telephone, with a final review by one research dietitian to ensure standardisation.

Nutritional analysis

3DFR were analysed for nutritional composition using a food and nutrient composition software package (FoodWorks version 7, Xyris Software). For the FFQ, total nutrient intake was calculated via a three-step method. First, grams of food per day were calculated by multiplying daily frequency of food consumption by the appropriate portion size in grams. Second, food intake (g/d) was multiplied by the corresponding nutrient per gram of that food (from a constructed nutrient database utilising the FoodWorks software). Third, total daily nutrient intake was calculated by tallying the daily individual nutrient intake of all food items.

Missing data in the FFQ were treated using commonly employed methods of applying a value of zero (no food consumed)⁽¹¹⁾, and this should create minimal bias if <10 % is missing⁽³²⁾.

Exclusion of participants

Participants missing more than 25 % of FFQ data were excluded from the analysis⁽³³⁾. In addition, participants were excluded if they failed to complete all validation study requirements or if their FFQ or 3DFR dietary data had implausible total daily energy intakes. The cut-off values employed were <2500 kJ or >18 000 kJ^(34,35). Five participants were excluded from analyses due to missing data (n 2) or not fulfilling study requirements (n 3), and thus thirty-eight were included in analysis.

Participant evaluation questionnaire

Participants were encouraged to complete an evaluation questionnaire comprising of nine questions covering a range of items including completion time, comprehension, adequate answer choice, formatting, questions which evoked feelings of uncomfortableness, additional information they desired to share and suggested improvements. Respondent answers will be utilised to revise the CMST.

Statistical analysis

Descriptive statistics for baseline characteristics were presented as mean ± sd for continuous variables and frequencies (percentage) for categorical variables. The normality of the distribution of continuous variables was assessed using the Kolmogorov–Smirnov test.

Means ± sd for the dietary intake of energy and macro/micronutrients were calculated for FFQ1, FFQ2 and the 3DFR. ANOVA was performed to determine whether any statistically significant differences existed in the means of energy and intake of macro/micronutrients among FFQ1, FFQ2 and the 3FDR, using the Bonferroni rule to correct for the inflation of type I error due to post-hoc multiple comparisons. In case of non-normally distributed continuous variables, the non-parametric Kruskal–Wallis test was used instead of ANOVA.

Two-way random intraclass correlation coefficient (ICC) and Pearson’s correlation coefficient (r) were used as measures of correlation to assess the test–retest reliability between FFQ1 and FFQ2 and the validity of FFQ1 against the 3DFR, for the dietary intake of energy and macro/micronutrients. In case of non-normally distributed continuous variables, the Spearman’s correlation coefficient (ρ) was used instead of r. Energy adjustment of nutrients was performed using the residual method described by Willet et al.⁽³⁶⁾.

One sample t test was also performed to assess the statistical significance of the mean difference of intake between the FFQ and the 3DFR compared with zero. Bland–Altman plots were used to further evaluate the correlation between FFQ1 and 3DFR and graphically present the levels of dietary intake with the lowest and highest level of agreement between the two dietary intake assessment methods, only for those nutrients that the t test showed statistical significant results.

All statistical analyses were performed using the statistical software package SPSS (version 24.0). All reported P values were two-tailed, and the level of statistical significance was set at P < 0·05.

Results

Baseline characteristics of participants having full data in all examined variables are presented in Table 1. The mean age of participants was 66 ± 9 years, with more males participating in the study compared with females (60·5 v. 39·5 %). Regarding other characteristics, 55 % of the study participants had CHD/established CVD (angina and/or acute myocardial infarction) with the remainder at high risk of CVD, 29 % had type 2 diabetes, 71 % had dyslipidaemia, 50 % were hypertensive, 24 % of participants were smokers and 34 % reported low levels of physical activity. Furthermore, study participants had a mean BMI of 29 ± 7 kg/m² and mean waist circumference of 101 ± 17 cm. Cultural diversity of Australia was evident in our population with 60·5 % being classified as Australian, 15·8 % British, 7·9 % Southern European, 5·3 % Western European, 7·9 % South Asian and 2·6 % South-East Asian (data not presented). The percentage of participants born overseas (42·1 %) was considerably higher compared with the overseas-born population in Australia (26 %)⁽¹³⁾.

Table 1.

Participant characteristics (n 38)

	Mean	%	sd
Age (years)	65·6		9·3
Sex
Male		60·5
Country of birth
Australia		57·9
Overseas		42·1
Education level
Primary		7·9
Secondary/Tafe		71·0
University		21·1
Weight (kg)	80·0		22·2
Height (cm)	166·0		9·8
BMI (kg/m2)	28·8		6·7
Waist circumference (cm)	100·5		17·0
Current smoker		23·7
Physical activity*		65·8
Blood pressure
Systolic (mmHg)	134·8		21·9
Diastolic (mmHg)	77·6		11·1
Diabetes		28·9
Dyslipidaemia		71·1
Hypertension		50·0
CHD		55·3

^*Assessed as category 2 or 3 from the International Physical Activity Questionnaire in Cardio-Med survey tool.

Table 2 summarises the comparisons of mean dietary intake of energy and macro/micronutrients, as these were recorded by the FFQ (FFQ1 and FFQ2) and the 3DFR. The mean dietary intake of energy and majority of macro/micronutrients as estimated by the FFQ were almost consistently higher when compared with the 3DFR. More specifically, the FFQ1 consistently reported higher dietary intakes of energy (P = 0·001), protein (g) (P = 0·002), sugar (P = 0·004), total fat (g and %E) (P ≤ 0·007), monounsaturated and polyunsaturated fat (including linoleic and α-linolenic fatty acids (P < 0·05), fibre (P = 0·032), alcohol (g and %E) (P = 0·001), vitamins (such as vitamins C, E and folate) (P < 0·05), as well as minerals and trace elements (such as K, Mg, Ca and Fe) (P ≤ 0·002), compared with the 3DFR. Regarding differences between FFQ1 and FFQ2, dietary fibre was the only nutrient for which intake was differentiating between the two time points of the FFQ administration to the study participants (31·4 (sd 13·6) v. 27·9 (sd 11·0), P < 0·05). No other statistically significant differences were observed.

Table 2.

Differences in the mean daily dietary intakes of energy and macro/micronutrients estimated by the FFQ (FFQ1 and FFQ2) and the 3-d food record (3DFR) (n 38)

	Daily dietary intake						P *
	FFQ1		FFQ2		3DFR
	Mean	sd	Mean	sd	Mean	sd
Energy (kJ)	8923·0a	1917·1	8441·7b	2961·6	7133·0a,b	2509·5	0·001
Macronutrients
Protein (g)	104·1a	34·9	97·9b	37·1	81·3a,b	30·0	0·002
Protein (%E)	19·8	3·5	19·9	3·3	19·8	4·2	0·932
Carbohydrates (g)	209·8	69·8	204·5	68·5	187·0	64·0	0·127
Carbohydrates (%E)	40·3	9·0	41·9	8·2	43·3	7·3	0·092
Sugar (g)	108·7a	43·8	110·0b	49·1	83·4a,b	44·9	0·004
Total fat (g)	87·4a	32·5	78·9b	32·6	62·1a,b	28·8	<0·001
Fat (%E)	36·0a	8·0	34·3b	6·5	31·6a,b	6·8	0·007
Saturated fat (g)	26·1	12·1	24·8	12·1	22·0	12·1	0·066
Monounsaturated fat (g)	36·7a	14·9	33·3b	15·4	24·1a,b	12·9	<0·001
Polyunsaturated fat (g)	17·4a	11·1	14·4b	7·7	11·2a,b	8·5	0·005
Linoleic (g)	15·2a	10·7	12·3b	6·8	9·1a,b	7·1	0·004
α-Linolenic (g)	1·6a	0·7	1·6	0·7	1·3a	1·0	0·041
Fibre (g)	31·4a,c	13·5	27·9c	10·9	25·4a	14·5	0·032
Alcohol (g)	4·6a	8·6	7·0b	21·1	3·8a,b	9·3	0·001†
Alcohol (%E)	1·4a	2·4	1·6b	3·7	1·1a,b	2·3	0·001†
Micronutrients
Vitamin C (mg)	221·3a	116·2	199·2b	113·3	113·6a,b	93·2	<0·001
Vitamin D (mcg)	3·8	2·8	3·6	2·5	3·2	3·2	0·265†
Vitamin E (mg)	13·7a	6·1	12·4b	5·3	8·3a,b	4·7	<0·001
Folate (mcg)	450·7a	138·8	407·2	147·2	358·6a	192·0	0·031
Na (mg)	2247·0	872·1	2102·7	883·8	2184·5	1554·5	0·568
K (mg)	4020·2a	1244·4	3720·2b	1259·0	3050·7a,b	1325·3	<0·001
Mg (mg)	422·3a	143·2	383·4b	129·4	306·8a,b	103·3	<0·001
Ca (mg)	1006·7a	465·6	968·4b	470·5	729·0a,b	400·5	0·002
Fe (mg)	14·5a	5·6	13·0b	4·8	10·7a,b	4·8	0·001

^a,b,c,d,eMean values within a row sharing the same superscript letter were significantly different (P < 0·05).

^* P-values were derived from the ANOVA, using the Bonferroni rule to correct for the inflation of type I error due to multiple comparisons.

^† P-values were derived from the Kruskal–Wallis test, using the Mann–Whitney U tests for pairwise comparisons.

Table 3 presents the ICC and r values indicating the associations and as such the level of agreement between FFQ1 and FFQ2 (test–retest reliability) and between FFQ1 and 3DFR (validity). As far as the test–retest reliability is concerned, the ICC values ranged from 0·52 (linoleic acid) to 0·88 (Na and alcohol (%E)) with a mean of 0·79 (P < 0·05). In addition, the r values ranged from 0·4 (linoleic acid) to 0·93 (alcohol (%E)) with a mean of 0·69 (P < 0·05). For energy and majority of nutrients, the ICC values demonstrated good agreement (ICC > 0·75) between the two time points when the FFQ was administered with the exception of protein (%E), total fat (both g and %E), polyunsaturated fat, linoleic acid, alcohol (g) and folate, which all demonstrated moderate agreement (ICC 0·5–0·74). Regarding validity, the ICC values ranged from 0·08 (Na) to 0·94 (alcohol (g)) with a mean of 0·59; all nutrients excluding Na had P > 0·05. For majority of nutrients, ICC values ranged between 0·5 and 0·75, thus demonstrating a moderate agreement, with the exception of alcohol which demonstrated excellent agreement (ICC > 0·9) and protein (%E), folate, Na and Mg which demonstrated poor validity (ICC < 0·5). The energy-adjusted ICC values ranged from –0·02 (Na) to 0·84 (alcohol (g)) with a mean of 0·47. After energy adjustment, ICC values were mostly comparable with the exception of protein (g) (ICC_adjusted = 0·29 v. ICC_crude = 0·53), fat (g) (ICC_adjusted = 0·29 v. ICC_crude = 0·57), alcohol (%E) (ICC_adjusted = 0·67 v. ICC_crude = 0·92), vitamin D (ICC_adjusted = 0·12 v. ICC_crude = 0·62), vitamin E (ICC_adjusted = 0·17 v. ICC_crude = 0·52) and Fe (ICC_adjusted = 0·22 v. ICC_crude = 0·55). The r values ranged from 0·05 (Na) to 0·6 (fibre and alcohol (%E)) with a mean of 0·48; all values excluding vitamin D and Na had P > 0·05. Given the pattern of r values was comparable with the validity and test–retest reliability ICC values, it will not be further discussed; however, their inclusion enables comparison by other published validation studies which report r values.

Table 3.

Correlation coefficients indicating the level of agreement between FFQ1 and FFQ2 (test–retest reliability) (n 38) and between FFQ1 and 3-d food record (validity) (n 38)

	Test–retest reliability				Validity
	ICCa	ICCb	r a	r b	ICCa	ICCb	r a	r b
Energy (kJ)	0·83		0·73		0·58		0·51
Macronutrients
Protein (g)	0·84	0·60	0·73	0·44	0·53	0·29	0·45	0·32*
Protein (%E )	0·72	0·62	0·56	0·56	0·49	0·49	0·33	0·36
Carbohydrate (g)	0·85	0·77	0·74	0·63	0·57	0·57	0·42	0·46
Carbohydrates (%E )	0·81	0·73	0·68	0·70	0·67	0·63	0·54	0·56
Sugars (g)	0·86	0·87	0·76	0·78	0·62	0·72	0·51	0·69
Fat (g)	0·73	0·56	0·59	0·49	0·57	0·29	0·53	0·50
Fat (%E )	0·73	0·74	0·60	0·60	0·63	0·72	0·54	0·53
Saturated fat (g)	0·87	0·72	0·78	0·84	0·71	0·75	0·57	0·70
Monounsaturated fat (g)	0·75	0·74	0·61	0·59	0·53	0·35	0·50	0·38
Polyunsaturated fat (g)	0·56	0·57	0·43	0·46	0·65	0·65	0·59	0·57
Linoleic (g)	0·52	0·35	0·40	0·44	0·60	0·42	0·57	0·57
α-Linolenic (g)	0·87	0·80	0·77	0·81	0·56	0·53	0·42	0·38
Fibre (g)	0·85	0·86	0·78	0·85	0·71	0·77	0·60	0·68
Alcohol (g)	0·69	0·67	0·93†	0·52†	0·94	0·84	0·60†	0·51†
Alcohol (%E )	0·88	0·47	0·92†	0·60†	0·92	0·67	0·56†	0·53†
Micronutrients
Vitamin C (mg)	0·84	0·87	0·74	0·79	0·56	0·57	0·60	0·62
Vitamin D (mcg)	0·75	0·68	0·50†	0·50†	0·62	0·12	0·28* †	0·18* †
Vitamin E (mg)	0·78	0·43	0·66	0·72	0·52	0·17	0·52	0·41
Folate (mcg)	0·69	0·74	0·55	0·62	0·46	0·50	0·35	0·45
Na (mg)	0·88	0·77	0·80	0·65	0·08*	–0·02*	0·05*	–0·02*
K (mg)	0·85	0·88	0·75	0·83	0·63	0·65	0·58	0·71
Mg (mg)	0·79	0·82	0·67	0·76	0·44	0·32	0·42	0·39
Ca (mg)	0·87	0·88	0·77	0·79	0·66	0·69	0·60	0·66
Fe (mg)	0·84	0·44	0·76	0·66	0·55	0·22	0·48	0·59
Mean	0·79	0·69	0·69	0·66	0·59	0·47	0·48	0·50

ICC, intraclass correlation coefficient; r, Pearson’s correlation coefficient.

^aCrude values; ^benergy adjusted values.

^*Non-statistically significant correlation coefficient, P > 0·05.

^†Spearman’s correlation coefficient.

Figure 1 illustrates the Bland–Altman plots that were further used to assess the validity of the FFQ against the 3DFR for carbohydrate (g), alcohol (g and %E), Na, protein (%E) and vitamin D (μg). Upon visual inspection, a good level of agreement was observed. In addition, a tendency for larger differences (poorer agreement) in nutrients’ intake was observed between FFQ1 and 3DFR when the mean estimates were becoming higher (nutrient intakes increased).

Fig. 1.

Bland–Altman plots depicting the level of agreement between FFQ1 and 3-d food record (3DFR) for (a) carbohydrate (g), (b) alcohol (g), (c) alcohol (%E), (d) Na (mg), (e) protein (g) and (f) vitamin D (μg)

Discussion

The aim of the current study was to describe the development of the CMST and evaluate the test–retest reliability and validity in measuring energy and nutrient intakes of the FFQ (within the CMST) in a sample of individuals with CVD or at high risk of developing CVD.

The FFQ in the CMST was developed to measure diet quality in a culturally diverse Australian CVD population. The FFQ has the additional ability to capture adherence to traditional dietary patterns such as the MDP. This is possible due to the breadth and variety of foods captured consumed in various cultures (Mediterranean and non-Mediterranean) that represent the cardio-protective MDP.

Our results highlight that the FFQ demonstrates good test–retest reliability for energy and most macro/micronutrients, with 14/20 displaying ICC values above 0·75, excluding protein (%E), fat (g and %E), polyunsaturated fat, alcohol (g and %E), linoleic acid and folate. Correlation coefficients in reproducibility studies generally range between 0·4 and 0·7^(10,11), which are consistent with our findings (ICC ranged from 0·52 to 0·88) and the range of correlation values obtained in other similar studies. An Australian study demonstrated ICC values between 0·66 and 0·9⁽³¹⁾, a Canadian study demonstrated ICC values between 0·38 and 0·91⁽²⁹⁾, a Spanish study reported ICC values of 0·52–0·94⁽³⁷⁾ and a Greek study reported ICC values of 0·38–0·78⁽³⁸⁾. In comparison with other literature, our group mean ICC value (0·79) was consistent^(31,39) or slightly higher^{(12,21,29,38)}. This was not surprising, given a higher mean value may be attributed to learned responses due to shorter time periods between repeat FFQ administrations⁽¹¹⁾. Our study had an interval of 2 weeks between administration of FFQ1 and FFQ2. This was shorter than the comparison literature which ranged between 4 weeks and 1 year, and thus may be perceived as a limitation. Nevertheless a shorter time period was chosen due to time restriction, to minimise participant loss and reduce substantial dietary changes that may occur with longer times between administrations⁽⁴⁰⁾.

The observed validity of ICC values ranged from 0·08 to 0·94 (mean 0·59) with energy and majority of nutrients (17/20) being above 0·5, except for Mg, folate and protein (%E), thus suggesting a moderate validity of the FFQ. We likewise observed a range and mean comparable with other similar validation studies. Cantin et al.⁽²⁹⁾ reported ICC values ranging from 0·26 to 0·84 (mean 0·56), and Khalesi et al.⁽³⁴⁾ observed ICC ranges between 0 and 0·8 (mean 0·52), whilst Marcedo-Ojeda et al.⁽³⁹⁾ reported ICC ranges of 0·19–0·73 (mean 0·51). Our observed ICC values ranged widely with Na demonstrating the lowest correlation, despite mean Na intake not being significantly different between the two assessment methods, a result similarly observed in Khalesi et al.⁽³⁴⁾. It has been suggested that dietary assessment methods often underreport Na intake due to the difficulties in measuring Na content of foods and the amount added to foods and recipes⁽³⁴⁾. Furthermore, linear regression analysis suggested a proportional bias for dietary Na intake (data not shown), which combined with a low ICC value suggests that our FFQ is not a valid measure of Na intake. Alcohol (g and %E) demonstrated the highest correlation, a result comparable with observations in other validation studies^(29,34,39). A reported moderately low alcohol consumption was not surprising, given the nature of the study population (cardiology patients). This combined with reported alcohol intake in the FFQ being based on consumption over a week (a timeframe more comparable with the 3DFR reference method), rather than a year, may have produced higher correlation values. There is debate whether the crude or energy-adjusted correlation coefficients should be considered during the validation of FFQ⁽⁴¹⁾. In the current study, adjustment for energy tended to produce similar values, with an observed trend towards a reduction in ICC values for the majority of nutrients. The reduction of correlation coefficients may be explained by a reduction in between-person variance^(10,42) and an increase in correlated measurement error (systematic errors of over/under estimation) resulting when adjustment for energy occurs⁽⁴³⁾. Furthermore, the results may demonstrate that the variability of the nutrients in this population was not related to energy intake, which when they are related, improvements in correlations are usually observed⁽¹⁰⁾. The effect of energy adjustment in other validation studies has been varied, with correlation coefficients reported to either improve^(12,44,45), reduce^{(40,42,43,46)}, or largely remain the same^(47,48).

When comparing FFQ with alternative diet reference methods, it is desirable to have a reference method which assesses dietary intake over the same time period as the test method⁽¹¹⁾. Our study had a discrepancy in time periods covered by the FFQ and 3DFR, being 12 months and 3 d, respectively. The fact that the current study was a pilot, alongside time constraints, hindered our ability to employ a longer reference method. Increasing the length of the reference method (e.g., 3DFR collected every 3 months for 1 year) would allow better capture of seasonal effects and provide a more comparable level of habitual intake to dietary information collected in the FFQ.

All dietary assessment methods are subject to bias, with FFQ commonly exhibiting an overestimation of energy and nutrient intakes. We found a general overestimation of mean energy and majority of nutrients measured by our FFQ in comparison with the 3DFR; findings which are comparable with other validation studies comparing FFQ with food records^(29,49). Such biases are likely to arise from an overestimation resulting from assigning mean serving sizes when portion selection is unavailable⁽⁵⁰⁾, or when individual foods are condensed to a single food group⁽⁴⁰⁾. A proportion of foods (65 %) in the FFQ were assigned an average portion size. Additionally, our FFQ examined fruits as a food group rather than consumption of individual fruits; both of these factors may have led to an overestimation of intake. It must be noted that despite food records being the gold standard for measuring dietary intake, they are not exempt from bias, with underestimation being a common characteristic⁽⁴⁰⁾. Furthermore, food records may be subjected to bias through misinterpretation of the reported foods consumed by participants by the dietitian analysing the records. To minimise this effect, the same dietitian cross-checked all records to ensure standardisation⁽²⁹⁾.

Additional limitations for consideration include the low response rate with the 3DFR, which may implicate potential respondent bias. Food records are known to be burdensome; thus, participants may alter their eating habits or fabricate records to alleviate burden⁽²⁶⁾. Furthermore, our sample size of thirty-eight was relatively low suggesting that we were potentially underpowered for specific nutrients. A minimum desirable advised sample size for validation studies is in the vicinity between 50 and 100 participants⁽¹¹⁾. Despite the small sample size, an advantage of this pilot study is the capacity to assess the interim functioning of the FFQ and to identify areas for improved validity.

Despite evidence of cultural diversity in the study population, an under representation was observed in participants classified as North-East Asian (in particular, migrants from China). The current study did not comprise participants born in China or culturally classified as North-East Asian; however, compared with Australian population census data, North-East Asia is the third most common region of birth (12·8 %), and China is the third most common birthplace (8·3 %)⁽¹³⁾ for overseas-born residents. Despite this under representation, we believe any impact on the integrity and utility of the FFQ to be minimal. Development of the FFQ was considered, with many culturally unique food preferences, cuisine and diverse dietary practices accounted for, including those from China, thus the FFQ remains relevant to this population group, albeit not currently field tested within. The lack of evaluation received regarding certain foods consumed in this population group that were not included in the FFQ may impact if future revisions of the CMST are made.

Due to our FFQ’s ability to measure dietary intake with moderate accuracy, we believe it may be of utility in other multicultural non-Mediterranean populations. Our FFQ is unique in the sense that it contains features often missing from commonly utilised FFQ, including portion size photographs, herbs and spices and cooking methods. Although reporting of food group consumption and MDP is beyond the scope of the current paper, a strength of our FFQ is its ability to capture elements of cuisine and assess adherence to a healthy dietary pattern like the MDP in multicultural cohorts. This is attributable to the incorporation of the detailed description of cuisine factors and breadth of food variety included; a level of detail not normally incorporated in FFQ. Whilst the FFQ will not replace dietary assessments performed by dietitians or weighed food records which are considered gold standard for measuring dietary intake⁽¹⁰⁾, the CMST has a dual ability to measure dietary intake with moderate accuracy and capture CVD risk factors (many of which overlap with other chronic diseases). Although designed specifically for a cardiology population, we believe that the overlap of CVD risk factors with other chronic conditions will broaden its applicability beyond this population and allow use in a variety of clinical settings and interventions. Moreover, the tool has the potential for revisions based on participant evaluation feedback and the addition of foods, that is, those reported missing by participants and expansion of fruit choices. This is likely to enhance the validity of measurement of nutrients in addition to conducting further validation within a larger population and validation against dietary biomarkers for selected nutrients, as markers tend to be highly correlated with dietary intake levels and free from some of the limitations associated with usual dietary intake estimates.

In conclusion, the multipurpose CMST was developed to assess CVD risk and dietary intake in a multicultural Australian cardiology population with the capability to assess adherence to a MDP. Furthermore, the semi-quantitative FFQ demonstrated good test–retest reliability and moderate validity in measuring energy and nutrient intakes.