Assessing factorial and convergent validity and reliability of a food behaviour checklist for Spanish-speaking participants in US Department of Agriculture nutrition education programmes

Abstract

Objective

To assess convergent validity, factorial validity, test–retest reliability and internal consistency of a diet quality food behaviour checklist (FBC) for low-literate, low-income Spanish speakers.

Design

Participants (n 90) completed three dietary recalls, the Spanish-language version of the US Department of Agriculture (USDA) Household Food Security Survey Module (HFSSM) and the Spanish-language FBC. Factor structure was examined using principal component analysis. Spearman correlation coefficients between FBC item responses and nutrient intakes from 24 h recalls were used to estimate convergent validity. Correlation coefficients were also calculated between FBC item responses at two time points in another group of participants (n 71) to examine test–retest reliability. Cronbach's α coefficient was determined for items within each sub-scale.

Setting

Non-profit community agencies serving low-income clients, migrant farm worker camps and low-income housing sites in four California counties.

Subjects

Spanish-speaking women (n 161) who met income eligibility for the SNAP-Ed (Supplemental Nutrition Assistance Program–Education).

Results

Factor analysis resulted in six sub-scales. Responses to nineteen food behaviour items were significantly correlated with hypothesized 24 h recall data (with a maximum correlation of 0·44 for drinking milk and calcium) or the USDA HFSSM (0·42 with the food security item). Coefficients for test–retest reliability ranged from 0·35 to 0·79. Cronbach's α ranged from 0·49 for the diet quality sub-scale to 0·80 for the fruit and vegetable sub-scale.

Conclusions

The twenty-two-item FBC and instruction guide will be used to evaluate USDA community nutrition education interventions with low-literate Spanish speakers. This research contributes to the body of knowledge about this at-risk population in California.

Latinos comprise over one-third of the population in California⁽¹⁾. Due to higher rates of hypertension, obesity, diabetes and metabolic syndrome in Latina women, this group is at higher risk for CVD than non-Latino whites⁽²⁾. Nutrition education interventions promoting dietary change have been shown to have positive effects on behaviour, thereby reducing the chronic disease burden⁽³⁾. Evaluation tools to assess these dietary behaviour changes are essential.

Several short dietary or behavioural assessment tools have previously been developed and reported in the nutrition literature. Their purposes include population monitoring, survey assessment and evaluation of nutrition education interventions. These tools include the National Cancer Institute's 5-a-Day for better health fruit and vegetable screener⁽⁴⁾, Kristal et al.'s Food Behavior Checklist at the University of Washington⁽⁵⁾, Connor et al.'s Diet Habits Survey at Oregon Health Sciences University⁽⁶⁾, Wakimoto et al.'s brief dietary screeners at the University of California, Berkeley⁽⁷⁾ and Townsend et al.'s Food Behavior Checklist at the University of California, Davis^(8–10). This literature review found no rigorously validated Spanish-language food behaviour evaluation tools to assess diet quality. Tools for Spanish-speaking low-literacy populations are especially lacking, with only one study describing validation of a set of short FFQ in a low-literacy Spanish-speaking sample⁽⁷⁾. There is a need for tools for US Department of Agriculture (USDA) food assistance and education programmes with a low respondent burden that can be administered in a group setting to this audience^(8,9,11,12). Tools that have exhibited adequate validity and reliability in a particular population need to be re-evaluated for use in another population that differs in terms of education, income, cultural background, country of origin, language or literacy^(11,13). An ideal evaluation instrument for these federal programmes should exhibit adequate validity and reliability in the target population^(11,14), be sufficiently brief to avoid detracting from the education portion of the intervention, and include behaviours presented in the education sessions^(9–11).

The current study sought to test a food behaviour checklist in a Spanish-speaking, low-income population in California. Face validity was previously assessed and reported⁽¹⁵⁾. This assessment involved rigorous methods including a comparison of five translated versions and cognitive testing interviews with low-income clients to determine their preferred word choices, resulting in a tool with low reading difficulty. Developed using visual information processing theories, this tool consisted of sixteen behavioural items with each composed of simplified text and visual. Based on our previously reported findings with an English-speaking audience^(8,16), the representative visuals were used as effective substitutes for text and/or as extralinguistic information to add clarity and facilitate understanding and hence learning for a low-literate audience. Also reported was that clients preferred colour photographs with realism, shape and colour cues compared with greyscale photographs with realism, shape cues or black/white line drawings, or abstract visuals with minimal realism^(8,15). The present paper examines the Spanish-language food behaviour checklist (FBC) in terms of factorial and convergent validity, internal consistency and test–retest reliability with a low-literate Spanish-speaking audience.

Methods

Sample

The study was conducted at the University of California in four California counties. Selected participants (n 161) were female, over the age of 18 years, spoke Spanish as a first language, met income eligibility for SNAP-Ed (Supplemental Nutrition Assistance Program–Education; formerly known as Food Stamp Nutrition Education), and had at least one child/youth under the age of 19 years living at home. Interviews were obtained from clients at non-profit community agencies serving low-income clients, trailer parks, migrant farm worker camps, government-funded day-care centres serving low-income clients and low-income housing sites.

Design and protocol

The protocol involved three studies. In the first, the two samples were combined for analysis of factorial structure and internal consistency. In the second, test–retest reliability was assessed⁽¹⁴⁾, and the participants received $US 10 for completing the FBC on two occasions three weeks apart with no planned intervention during the interim. In the third, convergent validity of the items and sub-scales was examined with a sample of women different from those in the reliability sub-study. In the initial meeting, demographic information including acculturation was collected and one 24 h recall was conducted in Spanish. At two subsequent meetings, two additional recalls, the Spanish-language FBC, the USDA eighteen-item food security scale and anthropometric data were collected. All interviews took place in person. Participants received a total of $US 40 in gift cards to major chain stores for the convergent validity study. The protocol was approved by the Institutional Review Board of the University of California.

Staff training

Two staff persons were responsible for recruitment of eligible participants from the four counties. These staff members were female, familiar with each respective community and spoke Spanish as a first language. Staff travelled to a central site for two-day intensive training in recruitment and data collection procedures (agenda and training materials available from the first author). A Spanish-speaking project coordinator supervised the staff to ensure consistency of data collection procedures.

Data collection

Family record, acculturation and anthropometric data

Standard demographic data were collected. Acculturation level was determined by the Bidimensional Acculturation Scale for Hispanics (BAS)⁽¹⁷⁾. Measured height, weight and waist circumference were collected using standardized anthropometric equipment and procedures (training protocol available from the first author)⁽¹⁸⁾.

Food behaviour checklist items

Food behaviour items (i.e. text and visuals) mirrored those included in the sixteen-item English-language version of the questionnaire^(8–10), with an additional four fat/cholesterol and three fruit and vegetable items. Face validity was established, with details described elsewhere⁽¹⁵⁾. To provide consistency in administration of the tool and reduce random error, a 22-page instruction guide was developed and reviewed by eight professional and two paraprofessional staff⁽¹⁹⁾. Items were worded so that the desirable food behaviours did not always elicit the same type of response. Responses were re-coded during analyses so that a higher score indicated more favourable behaviour.

Dietary recalls

To assess convergent validity of the food behaviour items, the USDA five-pass method for 24 h recalls was used⁽²⁰⁾. All interviews were conducted in person using standardized probes and models to aid in estimation of portion size. After collection of dietary information, foods were entered into the Food Processor SQL software package version 10.3 (ESHA Research, Salem, OR, USA). Specific recipes and ingredients for Mexican foods consumed by clients were added for the present study.

Food security

Validity of the FBC food security item was determined by Harrison et al.'s Spanish-language version of the USDA Household Food Security Survey Module (HFSSM) using the past 12 months as the time frame^(21,22).

Data analysis

Analyses were performed using the SAS for Windows statistical software package release 5.1.2600 (SAS Institute, Cary, NC, USA).

Item analysis

Mean responses and standard deviations were calculated for each item to determine capacity for change as a result of an intervention.

Factor validity

The factor structure of the FBC was examined in three stages. Principal component analysis with Varimax rotation was the data reduction technique of choice as our purpose was variable reduction or replacement of the original FBC items with sub-scale scores summarizing the data parsimoniously⁽²³⁾. Factors with eigenvalues >1·0 were included. Any item with a factor loading of >0·50 was considered to load on the given factor, in conjunction with review of the content of the individual items.

Internal consistency

Cronbach's α coefficients⁽¹⁴⁾ were calculated for sub-scales with three or more items to determine the consistency of responses to the final sub-scales. In our population of low-literate Spanish speakers, we considered α = 0·60 as acceptable. Spearman's correlation was determined for sub-scales with two items.

Test–retest reliability

Test–retest reliability of individual items on the FBC was indicated by the Spearman rank-order correlation between the scores for a given item at the two time points, as well as the intraclass correlation coefficient^(14,24), two methods that are commonly used together in assessing test–retest reliability^(25,26). Reliability of the sub-scales and total FBC was indicated by the Spearman rank-order correlation and intraclass correlation coefficient between the scores for a given sub-scale or the total FBC at the two time points. Items asking about ‘yesterday’ or ‘during the past week’ were excluded from this analysis.

Convergent validity

Using hypothesized relationships of nutrient intakes and food behaviours, convergent validity was examined using the mean of three 24 h dietary recalls. Given that many of the variables were not normally distributed, Spearman correlation coefficients were calculated to evaluate associations of FBC items and sub-scales with hypothesized dietary recall variables. Correlation coefficients were considered statistically significant if a relationship was hypothesized (i.e. milk consumption and calcium intake) and the P value was less than 0·05. For dichotomous items (yes/no responses), t tests were used. In addition, t tests were also used to compare the means of items expected to yield the same results, in the same units, in lieu of the method used by Bland and Altman⁽²⁷⁾.

Results

Sample and characteristics

A group of seventy-one participants provided data for the reliability study. Of the ninety women recruited for the validation study, eight did not complete all three days of dietary data collection or did not supply all of the necessary data, generating a final sample of eighty-two. Data for 153 participants were included in the factor analysis.

On average, the women in the validation study were 36 years old and had spent 12 years in the USA. Administration of the BAS⁽¹⁷⁾ yielded a mean score of 15·2 on the Hispanic scale and 4·7 on the non-Hispanic, with a score of ≥12 indicating a high level of adherence to the cultural domain. Average household size was 4·5 (sd 1·3) members. Participants had a mean BMI of 31·1 (sd 6·7) kg/m² and an average waist circumference of 93·6 (sd 16·8) cm. Participants recruited for the reliability study were not asked to provide demographic information. Because they were recruited at the same sites as participants in the validation study, we have no reason to suspect they possess different demographic characteristics.

Table 1 shows the mean consumption of food groups or nutrients from the three 24 h recalls for these low-income Spanish-speaking women.

Table 1.

Mean consumption of food groups and nutrients from 24 h recalls for low-income Spanish-speaking women in California (n 82)

Food group or nutrient	Mean intake or mean % of RDA/AI	sd
Total daily energy (kJ)	7787	2382
Total daily energy (kcal)	1860	569
Percentage of energy from fat	31·3	5·5
MyPyramid grain (ounce-equivalents)	6·8	2·9
MyPyramid vegetable (cups)	1·5	1·0
MyPyramid fruit (cups)	1·8	1·1
MyPyramid dairy (cups)	1·7	0·9
MyPyramid meat (ounce-equivalents)	6·2	3·2
Folate (% of RDA)	75	N/A
Fe (% of RDA)	75	N/A
Ca (% of AI)	78	N/A
Vitamin K (% of AI)	35	N/A

AI, Adequate Intake; N/A, not applicable.

Food security

Food security of household adults was classified into one of four ranges on the continuum of food security using the USDA HFSSM⁽²¹⁾. The mean result was 3·0, indicating the presence of marginal food security. Approximately 55% of participants were classified as having high food security, 35% as having marginal food security, 9% as having low food security and 1% as having very low food security⁽²⁸⁾.

Food behaviour checklist

Item analysis

The mean and standard deviation of the responses for each item on the FBC are shown in Table 2.

Table 2.

Factor validity and item analysis for twenty-three items in the food behaviour checklist for low-income Spanish-speaking women in California (n 154)

Food behaviour item	Factor loading	Mean response (n 82)†	sd
Fruit and Vegetable sub-scale	Factor 1
1. ¿Come frutas o verduras entre comidas?/Fruit or veg as snacks‡,§	0·63	2·9	0·8
2. ¿Cuántas porciones de fruta come cada día?/Svgs of fruit each day∥	0·76	3·4	0·8
3. Durante el día ¿come diferentes frutas?/More than one kind of fruit each day¶	0·76	2·8	0·9
4. ¿Cuánta fruta come cada día?/Cups of fruit each day††	0·63	2·3	0·6
5. ¿Come diferentes verduras cada día?/More than one kind of veg each day¶	0·56	2·9	0·8
6. ¿Cuántas porciones de verduras come cada día?/Svgs of veg each day∥	0·67	3·4	0·8
7. ¿Come más de dos porciones de verduras en su comida principal?/More than 2 svgs of veg at main meal§	0·51	2·6	0·9
8. ¿Qué cantidad de verduras come cada día?/Cups of veg each day††	0·55	2·2	0·6
9. ¿Come dos verduras o más en su comida principal?/Two or more veg at main meal§,‡‡	0·66	2·8	0·8
Does not load on any factor at >0·50:
10. La semana pasada, ¿comió frutas cítricas como naranja, mandarina o toronja o tomó jugo de esas frutas?/Citrus fruits or citrus juice during past week§§	0·24	2·7	0·7
Milk/Dairy sub-scale
11. ¿Toma leche?/Drink milk§	N/A	3·1	0·9
12. ¿La semana pasada, tomó leche o puso leche en su cereal?/Drink milk or use milk on cereal in past week§§	N/A	2·8	0·5
Food Security item
13. ¿Se le acaba la comida antes del fin del mes?/Run out of food before end of month∥∥	N/A	2·5	1·1
Diet Quality sub-scale	Factor 2
14.La semana pasada, ¿comió pescado?/Fish during past week§§	0·58	1·8	1·0
15.¿Quita la piel del pollo?/Take skin off chicken¶	0·72	3·2	1·0
16. ¿Considera la información nutritiva de la etiqueta al momento de seleccionar los alimentos que comprará?/Use label when food shopping¶	0·61	2·4	1·0
17. ¿Cómo cree que son sus hábitos de alimentación?/Rate eating habits¶¶	0·55	2·7	0·6
Fast Food sub-scale	Factor 3
18. ¿Comió frituras o botanas fritas ayer?/Fried snacks yesterday†††	0·75	2·4	0·9
19. ¿Comió alimentos fritos ayer?/Fried food yesterday†††	0·69	2·2	1·0
20. ¿Comió comida rápida ayer?/Fast food yesterday†††	0·62	2·8	0·6
Sweetened Beverages sub-scale	Factor 4
21. ¿Toma bebidas de frutas, bebidas deportivas o ponches?/Drink fruit drinks, sport drinks or punch‡‡‡	0·51	2·5	1·0
22. ¿Toma refrescos que no son de dieta?/Drink regular soda‡‡‡	0·82	3·0	0·9
Does not load on any factor at >0·50:
23. ¿Comió carne o cerdo ayer?/Red meat or pork yesterday†††	0·07	2·1	1·0

veg, vegetables; svgs, servings; N/A, not applicable.

^†All items were re-coded on a 4-point scale, with a higher score indicating more favourable behaviour. Dichotomous items with mean >2·6 and all other items with mean >3 were considered to have little potential to reflect behaviour change.

^‡English translation not included in questionnaire; provided here for the reader's convenience.

^§Every day = 4, often = 3, sometimes = 2, no = 1.

^∥Open-ended question: 0 servings = 1, 1 serving = 2, 2 servings = 3, >2 servings = 4.

^¶Almost always = 4, often = 3, sometimes = 2, no = 1.

^††3 cups or more = 4, 2·5 cups = 3·5, 2 cups = 3, 1·5 cups = 2·5, 1 cup = 2, 0·5 cup = 1·5, none = 1.

^‡‡Administered to a subset of the sample only (n 76).

^§§Yes = 3, no = 1.

^∥∥Almost always = 1, often = 2, sometimes = 3, no = 4.

^¶¶Excellent = 4, poor = 1.

^†††Yes = 1, no = 3.

^‡‡‡Every day = 1, often = 2, sometimes = 3, no = 4.

Factor validity

For use in the community, our goal was to have sub-scales that made logical sense to the health educator administering the FBC and the low-income client participating in the nutrition education programmes. For example, the fruit and vegetable sub-scale should contain only fruit and vegetable items, even though fat-related items performed well as surrogates of fruit and vegetable behaviours in our earlier study with low-literate English speakers^(9,10). Correspondingly, the fat sub-scales should only contain fat items. Our initial step was a factor analysis of the ten fruit and vegetable items. Nine of the ten items met the criteria for loading on two factors. Six items loaded on the first factor and included both fruit and vegetable items. Three vegetable items load on the second factor. The citrus item was retained by itself, as it did not load on either factor at >0·50. As this interpretation would be confusing for USDA programme participants and the educators, we repeated the analysis with one factor. Nine items loaded on the one factor (Table 2). Again, the citrus item did not meet the loading criterion. Conceptually, the latter analysis made more sense and was preferred by educators and clients.

Second, we segregated the two milk/dairy items (milk/dairy sub-scale) and the one food security item (food security sub-scale). Third, the remaining ten items were factor analysed. The principal component analysis produced three factors. Four items loaded on the first factor, labelled the ‘Diet Quality’ sub-scale. Three items loaded on the second factor, labelled ‘Fast Food’. Two items loaded on the third factor, labelled ‘Sweetened Beverages’. One item about eating red meat did not meet the criterion of >0·50 for any of the three factors. The red meat item was retained separately (Table 2).

Internal consistency

The internal consistency of the fruit and vegetable sub-scale was excellent (α = 0·80), while that for the diet quality sub-scale was understandably lower (α = 0·49) given the few items in the sub-scale (Table 3). It is important to note that, in general, the coefficient increases as the number of items in the scale increases. The optimal number of items for a scale is often ten to fifteen; the dietary quality sub-scale includes only four, as the tool is intended to be brief for use in the community. For the two-item dairy sub-scale, the Spearman's correlation coefficient was 0·42. While this is a relatively low correlation in comparison to α for the fruit and vegetable sub-scale, this finding is desirable given that there are only two questions assessing milk/dairy behaviours. If we were to discover a high correlation between the two items, we would assume measurement of the same behaviour associated with the construct and would eliminate one item, as our goal is a parsimonious tool. The moderate correlation indicates that the two items reflect different behaviours within the dairy domain construct, which is desirable in terms of capturing important behaviours related to dairy consumption. A correlation of 0·26 was found for the two sweetened beverage items (Table 3).

Table 3.

Internal consistency, test–retest reliability and convergent validity for food behaviour checklist sub-scales for a sample of low-income Spanish-speaking women in California

Food behaviour sub-scale†	Internal consistency (n 154)‡	Test–retest reliability§ (n 71)			Recall nutrient/other (n 82)
		r	95% CI	ICC	r	95% CI
Fruit and vegetables∥	0·80	0·60****	0·42, 0·73	0·63	0·20 (NS) (MyPyramid cups of fruit)	–0·02, 0·40
Dairy/calcium¶	0·42	0·78****	0·67, 0·86	0·81	0·26* (protein, g)	0·04, 0·45
					0·25* (vitamin A, RE)	0·03, 0·45
					0·38*** (riboflavin, mg)	0·17, 0·55
					0·42*** (vitamin D, μg)	0·22, 0·59
					0·43**** (Ca, mg)	0·23, 0·59
					0·33** (MyPyramid cups of dairy)	0·12, 0·51
Food security††	N/A	0·48****	0·27, 0·64	0·46	0·42*** (USDA food security scale)	0·22, 0·59
Diet quality‡‡	0·49	0·69****	0·71, 0·88	0·71	–0·22* (fat, % total kJ/kcal)	–0·42, 0·00
					–0·33** (trans fat, % total kJ/kcal)	–0·51, –0·12
					–0·23* (MyPyramid grains, oz-eq)	–0·43, –0·01
					–0·26* (MyPyramid cups of dairy)	–0·45, –0·04
					–0·25* (total sugars, g)	–0·45, –0·03
					–0·24* (net CHO, g)	–0·44, –0·02
Fast food§§	N/A	N/A	N/A	N/A	0·23* (vitamin A, RE)	0·01, 0·43
					0·23* (vitamin B12, μg)	0·01, 0·43
Sweetened beverages∥∥	0·26	0·55****	0·36, 0·70	0·55	–0·33** (fat, % total kJ/kcal)	–0·51, –0·12
					–0·37*** (sat fat, % total kJ/kcal)	–0·55, –0·16
					–0·41*** (total sugars, g)	–0·58, –0·21
					–0·44**** (net CHO, g)	–0·60, –0·24
Total scale¶¶	0·75	0·71****	0·51, 0·78	0·74	N/A	N/A

r, Spearman's correlation coefficient; ICC, intraclass correlation coefficient; N/A, not applicable; RE, retinol equivalents; USDA, US Department of Agriculture; oz-eq, ounce-equivalents; CHO, carbohydrate; sat, saturated.

^*P < 0·05

^**P < 0·01

^***P < 0·001

^****P < 0·0001.

NS: P > 0·05.

^†Questions re-coded on 4-point scale with higher scores reflecting a healthier diet.

^‡Cronbach's α for sub-scales with three or more items; Spearman's r for sub-scales with two items. Validity and reliability databases combined for a total sample size of 154.

^§Test–retest reliability analysis excludes items that refer to the past week or yesterday.

^∥Scale excludes citrus item.

^¶Test–retest reliability includes only one item: drink milk.

^††Food security item validated using the USDA Household Food Security Survey Module.

^‡‡Scale includes the following items: eat fish in the past week, take skin off chicken, use food label when shopping and rate eating habits.

^§§Scale includes the following items: fried snacks yesterday, fried food yesterday and fast food yesterday.

^∥∥Scale includes the following items: drink fruit drinks, sport drinks or punch and drink regular soda.

^¶¶Total scale excludes citrus item.

Test–retest reliability

All fifteen items tested in the reliability portion of the study met the criterion (P < 0·05) for acceptability using Spearman's correlation coefficients (Table 4). Values ranged from 0·35 to 0·79 (mean 0·56). Intraclass correlation coefficients are also shown, and ranged from 0·34 to 0·81 (mean 0·55). The seven items that referred to ‘yesterday’ or ‘the past week’ were excluded from the analysis.

Table 4.

Test–retest reliability, convergent validity coefficients and hypothesized relationships for food behaviour checklist items and sub-scales with dietary recall variables for a sample of low-income Spanish-speaking women in California

	Test–retest reliability (n 71)			Recall nutrient (n 82)
Food behaviour checklist item or sub-scale	r	95% CI	ICC	r †	95% CI
Fruit and Vegetable sub-scale items: Expect positive correlations with vitamins A (RE) and C (mg), β-carotene (μg), folate (DFE μg), K (mg), pantothenic acid (mg), niacin equivalents (mg), dietary fibre (g), cups of fruits and vegetables; and negative correlation with fat (% total kJ/kcal), sat fat (% total kJ/kcal), trans fat (% total kJ/kcal)
1. Fruit or veg as snacks‡,§	0·47****	0·26, 0·64	0·50	0·24* (vitamin A)	0·02, 0·44
				0·24* (MyPyramid cups of fruit)	0·02, 0·44
				–0·32** (fat)	–0·50, –0·11
2. Svgs of fruit each day∥	0·37**	0·15, 0·56	0·34	0·28* (MyPyramid cups of fruit)	0·06, 0·47
3. More than one kind of fruit each day¶	0·46****	0·25, 0·63	0·46	0·23* (MyPyramid cups of fruit)	0·01, 0·43
4. More than one kind of veg each day¶	0·45****	0·24, 0·62	0·47
5. Svgs of veg each day∥	0·57****	0·38, 0·71	0·53
6. More than 2 svgs of veg at main meal‡	0·35**	0·12, 0·54	0·34	0·21 (P = 0·551) (K)	–0·01, 0·41
				0·23* (pantothenic acid)	0·01, 0·43
				0·30** (niacin)	0·09, 0·49
7. Cups of fruit each day††	0·70****	0·55, 0·80	0·72	0·23* (MyPyramid cups of fruit)	0.01, 0·43
8. Cups of veg each day††	0·73****	0·60, 0·82	0·68
9. Two or more veg at main meal‡	Not tested	N/A	N/A	0·22 (P = 0·0529) (MyPyramid cups of veg)	0·00, 0·42
Milk/Dairy sub-scale items: Expect positive correlations with vitamin A (RE) and D (μg), riboflavin (mg), Ca (mg), cups of dairy
10. Drink milk‡	0·78****	0·67, 0·86	0·81	0·43*** (vitamin D)	0·23, 0·59
				0·34** (MyPyramid cups of dairy)	0·13, 0·52
				0·44*** (Ca)	0·24, 0·60
				0·40** (riboflavin)	0·20, 0·57
				0·27* (vitamin A)	0·05, 0·46
				0·42*** (lactose)	0·22, 0·59
11. Drink milk or use milk on cereal in past week‡‡	N/A	N/A	N/A	0·14**,‡‡‡ (vitamin A)	–0·08, 0·35
				1(n 7): 257·3 (148·0), 3 (n 75): 502·8 (508·5)
Food Security item: Expect positive correlations with USDA HFSSM, cups of fruits and vegetables and negative correlations with fat (% total kJ/kcal), since questions have been re-coded to reflect food security
12. Run out of food before end of month§§	0·48****	0·27, 0·64	0·46	0·42*** (USDA HFSSM)	0·22, 0·59
Diet Quality sub-scale items: Expect positive correlations with vitamin and mineral intake; and negative correlations with energy (kJ/kcal), fat (% total kJ/kcal), sat fat (% total kJ/kcal), cholesterol (mg), MyPyramid meat oz-eq
13. Fish during the past week‡‡	N/A	N/A	N/A	0·26*,‡‡‡ (vitamin K)	0·04, 0·45
				3(n 29): 44·5 (45·9), 1 (n 53): 24·7 (15·7)
14. Take skin off chicken¶	0·51****	0·31, 0·67	0·58
15. Use label when food shopping¶	0·74****	0·61, 0·83	0·72	–0·35** (trans fat)	–0·52, –0·14
				–0·33** (sat fat)	–0·51, –0·12
				–0·35** (fat)	–0·52, –0·14
				–0·31** (total sugars)	–0·50, –0·10
16. Rate eating habits∥∥	0·79****	0·68, 0·87	0·80	0·22* (Ca)	0·00, 0·42
				0·26* (pantothenic acid)	0·04, 0·45
				0·27* (vitamin A)	0·05, 0·46
Fast Food sub-scale items: Expect positive (due to re-coding) correlations with vitamin and mineral intake, fibre (g), cups of fruit and vegetables; and negative correlations with fat (% total kJ/kcal), sat fat (% total kJ/kcal) and trans fat (% total kJ/kcal)
17. Fried snacks yesterday¶¶	N/A	N/A	N/A	0·25*,‡‡‡ (vitamin A)	0·03, 0·45
				1 (n 31): 751·6 (699·8), 3 (n 51): 1248·9 (2345·8)
18. Fried food yesterday¶¶	N/A	N/A	N/A	0·26*,‡‡‡ (vitamin K)	0·04, 0·45
				1 (n 37): 28·1 (30·4), 3 (n 45): 34.6 (32·0)
19. Fast food yesterday¶¶	N/A	N/A	N/A	0·24*,‡‡‡ (vitamin A)	0·02, 0·44
				1 (n 9): 349·2 (141·8), 3 (n 73): 809·6 (1957·6)
Sweetened Beverages sub-scale items: Expect negative correlations (due to re-coding) with total sugars (g), net CHO (g), fat (% total kJ/kcal), sat fat (% total kJ/kcal)
20. Drink fruit drinks, sport drinks or punch†††	0·48****	0·27, 0·64	0·46	–0·24* (sat fat)	–0·44, –0·02
				–0·28** (total sugars)	–0·47, –0·06
				–0·34** (net CHO)	–0·52, –0·13
21. Drink regular soda†††	0·48****	0·27, 0·64	0·43	–0·26* (sat fat)	–0·45, –0·04
				–0·24* (fat)	–0·44, –0·02
				–0·33** (total sugars)	–0·51, –0·12
				–0·31** (net CHO)	–0·50, –0·10
Meat item: Expect negative (due to re-coding) correlations with fat (% total kJ/kcal), sat fat (% total kJ/kcal), cholesterol (mg), MyPyramid meat oz-eq
22. Red meat or pork yesterday¶¶	N/A	N/A	N/A	–0·35**,‡‡‡ (MyPyramid meat)	–0·53, –0·14
				1 (n 40): 7·2 (3·5), 3 (n 42): 5·2 (2·6)
				–0·21* (cholesterol)	–0·41, 0·01
				1: 300·8 (142·9), 3: 241·9 (112·3)
				–0·27** (saturated fat)	–0·46, –0·05
				1: 24·4 (9·3)· 3: 19·1 (7·1)
				–0·27* (fat)	–0·46, –0·05
				1: 72·6 (26·3), 3: 59·3 (22·9)
Citrus item: Expect positive correlation with vitamin C (mg)
23. Citrus fruits or citrus juice in past week‡‡	N/A	N/A	N/A	0·20*,‡‡‡ (folate)	–0·02, 0·40
				1 (n 19): 234·2 (141.4), 3 (n 63): 319·1 (190.7)

r, Spearman's correlation coefficient; ICC, intraclass correlation coefficient; RE, retinol equivalents; DFE, dietary folate equivalents; sat, saturated; veg, vegetables; svgs, servings; USDA, US Department of Agriculture; HFSSM, Household Food Security Survey Module; oz-eq, ounce-equivalents; CHO, carbohydrate; N/A, not applicable.

All questions re-coded on a 4-point scale.

^*P < 0·05

^**P < 0·01

^***P < 0·001

^****P < 0·0001.

^†Correlations with dichotomous variables are t tests.

^‡Every day = 4, often = 3, sometimes = 2, no = 1.

^§English translation not included in questionnaire; provided here for the reader's convenience.

^∥Open-ended question: 0 servings = 1, 1 serving = 2, 2 servings = 3, >2 servings = 4.

^¶Almost always = 4, often = 3, sometimes = 2, no = 1.

†3 cups or more = 4, 2·5 cups = 3·5, 2 cups = 3, 1·5 cups = 2·5, 1 cup = 2, 0·5 cup = 1·5, none = 1.

^‡‡Yes = 3, no = 1.

^§§Almost always = 1, often = 2, sometimes = 3, no = 4.

^∥∥Excellent = 4, poor = 1.

^¶¶Yes = 1, no = 3.

^†††Every day = 1, often = 2, sometimes = 3, no = 4.

^‡‡‡Correlation marked as significant to reflect t test results. Values indicate mean intake (sd).

Convergent validity

A list of hypothesized correlations with dietary recall variables for each category or sub-scale of food behaviour items is shown in Table 4, with nineteen FBC items showing statistically significant correlations with hypothesized nutrient intake. For seven dichotomous items, t tests provided a comparison of means.

All items were re-coded so that higher scores reflect a healthier diet. Item 21 in Table 4, regular soda behaviour, may be used as an example of how to interpret table results. For this item, a negative correlation is indicated between item response and saturated fat. With re-coding, a higher score represents lower intake of regular soda; thus, those who consumed more regular soda had greater intakes of saturated fat (r= −0·26, 95% CI −0·45, −0·04) and more total sugars (r= −0·33, 95% CI −0·51, −0·12).

For the fruit, vegetable and dairy items, associations were positive for several relevant nutrients. People reporting more desirable behaviours related to these foods also reported diets of higher quality. Of these FBC items, six were significantly correlated with the corresponding MyPyramid cups from 24 h recalls. For the two items measured in the same units using both the FBC and recall, cups of fruit and cups of vegetables, we conducted paired t tests to determine similarity between the means⁽²⁷⁾. In both cases, the means from the recalls were significantly higher than the means resulting from the FBC (not shown). One fat and cholesterol item was significantly correlated with fat or cholesterol intake and MyPyramid ounce-equivalents of meat from the recalls (Table 4).

Use of the food label to choose foods was negatively associated with intakes of total fat (r= −0·35, 95% CI −0·52, −0·14), saturated fat (r= −0·33, 95% CI −0·51, −0·12) and trans fat (r= −0·35, 95% CI −0·52, −0·14). Conversely, those who consumed fruit drinks, sport drinks or punch had higher intakes of saturated fat (r= −0·24, 95% CI −0·44, −0·02), total sugars (r= −0·28, 95% CI −0·47, −0·06) and net carbohydrate (r= −0·34, 95% CI −0·52, −0·13), with re-coding so that a higher score represents lower intake. The food security item was significantly correlated with responses on the USDA eighteen-item HFSSM (r= 0·42, 95% CI 0·22, 0·59; Table 4).

Several of the items examined did not meet the minimum requirement for significance as indicated by the confidence intervals. These items included eating two or more vegetables at the main meal, drinking milk, and having citrus fruit or juice in the past week (Table 4).

Total scores for each sub-scale were correlated with specific nutrient intakes from 24 h recalls, revealing a significant relationship between sub-scale scores and several relevant nutrients (Table 3).

Item reduction

Using factor analysis, correlation coefficients for test–retest reliability, correlation coefficients with nutrient intake from 24 h recalls and item analysis results, items were considered for deletion. One item, ‘citrus fruit or citrus juice during the past week’, was selected for deletion. This item showed little potential to reflect behaviour change based on its high mean response value, did not load with the other fruit and vegetable items in the factor analysis, and did not correlate with nutrient intake from 24 h recalls. Other marginal items were retained for further study. The final checklist contains twenty-two items.

Discussion

In the present study we examined factorial and convergent validity and reliability of a Spanish-language FBC. Our purpose was to assess the validity of items and sub-scales and to use those results to select well-performing items for the checklist.

The study explored ten food behaviours related to fruit and vegetable intake. Of these, nine items loaded on one factor and seven were significantly correlated with recall nutrient intake (Table 4). While the remaining three – ‘more than one kind of vegetable’, ‘servings of vegetables’ (i.e. from the Food Guide Pyramid guidelines) and ‘cups of vegetables’ (i.e. from the MyPyramid guidelines) – did not show hypothesized correlations with nutrient intake, they are retained for further study because they loaded with other items on the fruit/vegetable construct. As previously mentioned the citrus item was selected for deletion, as it did not load with other items in the factor analysis and did not show the hypothesized correlation with vitamin C intake. Although correlating weakly with folate intake and containing some folate, this was not the main hypothesized relationship and was not considered justification for item retention.

In addition to the Spearman's correlation coefficients calculated for the fruit/vegetable items, we also performed paired t tests for two items measured in the same units in the FBC and recalls, cups of fruit and cups of vegetables. This analysis was performed as an alternative to the Bland and Altman approach⁽²⁷⁾, as their method is most appropriate with continuous variables and these two items are able to take on a limited number of values. In both cases, values were significantly higher using the recalls. This result may indicate a need for further testing and development of appropriate visuals for these two items. The current items ask clients to estimate their consumption using visuals of liquid measuring cups filled with fruits and vegetables, while clients used actual dry measuring cups during the recall. Further testing may reveal a more appropriate way to represent the concept of ‘cups’ in the questionnaire.

Several FBC items were associated with fruit and vegetable intake, serving as surrogates of consumption of these foods. Three fast food items – ‘fried snacks yesterday’, ‘fried food yesterday’ and ‘fast food yesterday’ – and one diet quality item – ‘fish in the past week’ – showed significant correlations with vitamins A and K (Table 4), two micronutrients found predominantly in fruit and vegetables. It is not surprising that women choosing lowfat protein sources such as fish would also consume a diet rich in fruit and vegetables; conversely, those who regularly consume fried and fast food would presumably consume fewer fruit and vegetables. It is important to note that similar results were found in our FBC study with low-income English-speaking clients^(9,10).

A factor analysis identifies those items correlated with a latent variable representing a construct⁽¹⁴⁾. Such an analysis revealed that while fat-related items such as ‘fast food yesterday’ and ‘fried food yesterday’ showed strong loading on the same latent variable or factor, ‘fruit drinks, sport drinks or punch’ and ‘regular soda’ did not load with them. This result may seem surprising given that the two beverage items were correlated with fat intake. One explanation is that the amount of soda the participant drank was independent of the amount of fast food and fried food she consumed. Because many people drink soda and other sweetened beverages at home or work apart from at mealtimes, this result seems plausible.

This Spanish-language checklist⁽²⁹⁾ contained four items related to fat/cholesterol intake not found in the English version⁽³⁰⁾. Previous studies have reported differences in consumption of fat intake across ethnic groups^(31,32). Murtaugh et al. reported that, compared with non-Hispanic whites, Hispanic women living in the south-west USA consumed a greater proportion of energy from fat⁽³¹⁾. Kristal et al. found that Hispanics consumed more fat from fried vegetables and also consumed more fat from meat than whites⁽³²⁾. The additional questions were intended to capture differences between English and Spanish speakers, expand the sub-scale and identify behaviours that best reflected fat intake.

One of six original fat and cholesterol items, ‘eating red meat or pork yesterday’, showed significant correlations with fat, saturated fat and cholesterol intake, as well as MyPyramid ounce-equivalents of meat from 24 h recalls (selected correlations shown in Table 4). Surprisingly, ‘using the label when food shopping’, with a visual of a Latina reading a nutrition food label on a box of cereal in a supermarket, performed well in terms of reliability (r= 0·74) and validity against multiple measures of fat and sugar (Table 4). Likewise, a mediator of dietary behaviours, ‘rating your eating habits’, also performed well in terms of reliability (r= 0·79) and validity (i.e. calcium, pantothenic acid, vitamin A). Overall, however, the original fat and cholesterol items performed better as surrogates of fruit and vegetable behaviours, with inverse correlations with vitamins A and K, than as items for fat behaviours (Table 4). Several previous validation studies have also reported relatively low correlations between fat-related items and fat and cholesterol intake⁽³³⁾. Researchers hypothesized that this may have resulted from omission of key fat-related behaviours, namely use of vegetable oil and lard in cooking, refried beans and tortillas. While our FBC measured fat behaviours in terms of red meat and fried sources, some other relevant behaviours may be lacking. An alternative explanation is that client perceptions of high-fat foods and high/low-fat diets may be inaccurate. Further research is recommended.

One particular finding to note is the correlation between the sweetened beverage sub-scale and net carbohydrate (Table 3) as well as each sweetened beverage item and net carbohydrate (Table 4). As hypothesized, this relationship indicates an increase in carbohydrate with an increase in consumption of regular soda and fruit drinks, sport drinks and punch. Given that the majority of carbohydrate in the diet likely comes from simple sugars in sweetened beverages in this population, this correlation is logical.

As hypothesized, there was a significant correlation between responses to the food security item and the food security level derived from the 18-item USDA scale. Contrary to our expectations, the food security item did not show any associations with nutrient intake, unlike the English version of the FBC^(9,10).

The present study examined the validity and reliability of a Spanish-language FBC that was based on an English-language version found previously to have adequate psychometric properties in an English-speaking population^(9,10). Using visual information processing theories the readability of the checklist was improved, increasing its ability to accurately capture existing changes in dietary behaviour^(8,16). Readability of the text of the final twenty-two-item FBC was estimated to be 71 using the Fernández-Huerta formula⁽³⁴⁾, the equivalent of the Flesch Reading Ease for English text⁽³⁵⁾. This score indicates a ‘fairly easy’ reading level. No formula calculates readability of text with visuals⁽¹⁵⁾.

Members of the Network for a Healthy California, as well as nutrition education programmes in other states, are currently using the Spanish-language FBC⁽¹⁵⁾ and instruction guide⁽¹⁹⁾ with Spanish speakers and the English-language FBC^(8–10,30) and instruction guide⁽³⁶⁾ with English speakers. These instruments have a low respondent burden, are easy to administer in a group setting and assess eating behaviours that have known associations with risk of chronic disease contained within sub-scales^(8,9,11,15).

As a result of the study reported herein, the current version of the Spanish tool contains twenty-two items and seven sub-scales composed of nine items on fruit and vegetables, four on diet quality, three on fast food, two on dairy/calcium, two on sweetened beverages, one on meat and one food on security.

Limitations

Use of a convenience sample reduced the external validity of these findings by limiting generalizability to other Spanish-speaking audiences⁽³⁷⁾. While collecting three recalls instead of one provides a more accurate representation of usual diet, this self-report method remains imperfect⁽³⁸⁾ and is subject to variations in respondents’ cognitive ability and other forms of bias⁽³⁹⁾.

Applications

The new versions of the FBC (available from the second author) and instruction guide (available from the second author) will be used to evaluate nutrition education interventions among low-income Spanish speakers in community settings in California⁽²⁹⁾. Nutrition educators seeking to improve food behaviours of participants in the EFNEP (Expanded Food and Nutrition Education Program), WIC (Special Supplemental Nutrition Program for Women, Infants, and Children) and SNAP-Ed may be able to use the valid items and sub-scales when designing evaluation instruments for low-literate Spanish speakers. The FBC offers advantages over the 24 h recall as an evaluation tool as it is less time-intensive, focuses on specific behaviours presented in the intervention, can be administered to small and large groups, and does not require an interviewer trained in administering diet recalls. Evaluation of these nutrition education programmes will ensure programme integrity and continued funding. This research contributes to the growing body of knowledge regarding food behaviours among Spanish speakers in the USA and may be incorporated into future endeavours to educate at-risk populations.