Measuring Nutrition Literacy in Spanish-Speaking Latinos: An Exploratory Validation Study

Abstract

Background

Nutrition is important for preventing and treating chronic diseases highly prevalent among Latinos, yet no tool exists for measuring nutrition literacy among Spanish speakers. This study aimed to adapt the validated Nutrition Literacy Assessment Instrument for Spanish-speaking Latinos.

Methods

This study was developed in two phases: adaptation and validity testing. Adaptation included translation, expert item content review, and interviews with Spanish speakers. For validity testing, 51 participants completed the Short Assessment of Health Literacy-Spanish (SAHL-S), the Nutrition Literacy Assessment Instrument in Spanish (NLit-S), and socio-demographic questionnaire. Validity and reliability statistics were analyzed.

Results

Content validity was confirmed with a Scale Content Validity Index of 0.96. Validity testing demonstrated NLit-S scores were strongly correlated with SAHL-S scores (r= .52, p< .001). Entire reliability was substantial at 0.994 (CI 0.992–0.996) and internal consistency was excellent (Cronbach’s α= .92).

Discussion

The NLit-S demonstrates validity and reliability for measuring nutrition literacy among Spanish-speakers.

INTRODUCTION

Diet quality, or the overall pattern of food and nutrient intake, is a construct increasingly used to characterize and evaluate population diets, food environments, food offerings in public food assistance programs, and nutrition interventions. Efforts to improve diet quality are important because high diet quality decreases the risk of mortality in general, as well as mortality from cardiovascular diseases and cancer among Americans¹. In the United States, Latino immigrants disproportionately experience factors leading to barriers for healthy behavior, such as healthy diets. These factors include poor educational attainment, poverty, food insecurity, lack of health care access and acculturation². Specifically, acculturation is associated with an increased intake of fast food, snacks and added fats, and decreased intake of fruits, vegetables, rice, and beans^3,4.

Of any racial/ethnic group, Latino adults in the U.S. demonstrate the lowest health literacy, and some data suggest health literacy is an important mediator of health disparities experienced by Latinos ^5–7. More specifically pinpointing health literacy within a nutrition context, nutrition literacy may be a root cause for poor dietary quality among Latinos.

In a case-control study of 201 Latinas, many were unfamiliar with the U.S. Department of Agriculture’s serving sizes (ranged by food groups from 44.2%–93.8% incorrect) and only 39.8% were aware of saturated fat. Nutrition knowledge was associated with two times greater likelihood of using food labels to make healthful choices (p=0.007), increased consumption of fruits and vegetables and decreased consumption of salty snacks (p<0.05)⁸. In a cross-sectional study of low income, obese adult Latinos, community-based nutrition education was associated with 1.2 more daily fruit and vegetable servings (p=0.03) and 0.3 fewer sugary beverage servings (p=0.01)⁹. The noted poorer diet quality among more acculturated Latinos⁴, suggests that improving nutrition literacy is a critical target for improving overall health for the Latino population.

Current tools for assessing health literacy in the Spanish-speaking Latino population^10–13 are inadequate for measuring nutrition literacy. However, in our previous work, we have validated the Nutrition Literacy Assessment Instrument (NLit) for measuring nutrition literacy among breast cancer survivors¹⁴, parents¹⁵, and adult patients with nutrition-related chronic disease¹⁶. The NLit measures nutrition knowledge and nutrition skills to follow a healthy diet for English audiences. The instrument has six domains, with 64 items: ‘Nutrition & Health’ measures reading comprehension of the summarized Dietary Guidelines for Americans; ‘Energy Sources in Food’ measures knowledge of the macronutrient sources in food; ‘Household Food Measurement’ measures identification of recommended portions; ‘Food Label and Numeracy’ measures ability to apply information obtained from the nutrition facts panel; ‘Food Groups’ measures ability to classify foods by nutrition category; and ‘Consumer Skills’ measures ability to navigate food products to make healthy food choices.

Thus, the objectives of this study were to: 1) adapt the existing NLit tool for measuring nutrition literacy for use in a Midwestern U.S. Spanish-speaking Latino population, and 2) estimate the tool’s validity in a community sample of Spanish-speaking Latinos.

METHODS

Study design

This observational, cross-sectional study was conducted in an urban metropolitan area in the Midwest. All data was collected between November 2014 to December 2015. The study was divided into two phases: 1) Cultural and linguistic adaptation, a process which involved translation, face validity, content validity, and cognitive interviewing; and 2) Instrument validity testing.

Participant selection

In this study, three audiences were engaged. For adaptation (Phase 1) we involved three experts for content review. Two experts were credentialed dietitians who practice in clinic-based nutrition education within the Latino community, and one expert had extensive experience in public health nutrition and in developing adaptive measures for learning among Latinos. The recruitment of the panel was based upon their knowledge of nutrition and experience with Latino’s specific dietary practices. A convenience sample of three Latino community representatives was recruited based upon meeting the inclusion criteria of the target sample and participated in cognitive interviews. Experts were compensated for their time, and community participants were compensated $25 in gift cards. For validity testing (Phase 2) we recruited Latino primary care patients using a variety of approaches including by phone via a patient registry; by flyer and/or invitation in waiting rooms of two University-affiliated safety net clinics; and by campus broadcast email. Eligible participants were over 18 years of age and identified Spanish as their primary language. Subjects with overt psychiatric illness, cognitive impairment, or visual acuity insufficient to read the testing instrument were excluded. Participants of validity testing were given $10 in gift cards for completing the study visit.

Ethics

The University’s Institutional Review Board approved the study, all subjects provided written informed consent, and all procedures were in accordance with the ethical standards described in the Declaration of Helsinki.

Measurements

The Nutrition Literacy Assessment Instrument (NLit)

The NLit was validated in a sample of 429 adults with nutrition-related chronic disease (hyperlipidemia, hypertension, diabetes, and/or overweight/obesity)¹⁶. Confirmatory factor analysis demonstrated an excellent scale validity and entire reliability of 0.97 (95% CI=0.96–0.98), while test-retest reliability of 0.88 (95% CI = 0.85–0.90) was also excellent. Convergent validity was measured with diet quality as the reference measurement. In a step-wise multi-variate regression, a significant relationship was seen between NLit score and diet quality (Healthy Eating Index-2010) (R² 0.10, B estimate=42.68, p<0.0001), with NLit contributing most significantly to the model (B estimate=0.30, p=0.003). Scoring interpretation is ≤44 indicates a “likelihood of poor nutrition literacy,” 45–57 indicates a “possibility of poor nutrition literacy,” and ≥58 indicates “likelihood of good nutrition literacy.”

The Short Assessment of Health Literacy-Spanish

The Short Assessment of Health Literacy-Spanish (SAHL-S) is an 18 item health literacy instrument that is reliable (α =0.80), was validated against the Spanish TOFHLA (r=0.62, p<0.05), and takes an estimated 2–3 minutes to complete¹⁰. A score between 0 and 14 suggests inadequate health literacy.

Spanish General Language Scale

General Spanish language usage was measured by 5 questions of the language scale and the short acculturation scale. The instrument had high internal consistency (α=0.95), and was validated against the Acculturation Rating Scale for Mexican Americans (ARSMA)¹⁷ (r=0.88, p<0.001)¹⁸.

Phase 1: Cultural and Linguistic Adaptation

Face Validity and Translation to Spanish

The NLit was reviewed by the research team to check relevance of food items to the local Mexican majority Spanish-speaking Latino community. The goal of this adaptation was to ensure the foods presented in the translated instrument are familiar to the target population, consequently increasing the face validity of the instrument to Latinos. The criteria used to do the cultural adaptation of the foods were based on foods that are widely recognized by the community, and not necessarily based in the consumption of those foods. For example, “pear,” a distractor answer (that is, an incorrect answer option) in the English NLit version was replaced with “mango” for the Spanish version. The context of the question concerns the presence of fat in food, and pears and mangos have comparable fat content (<1 gram per medium piece of fruit). However, mangos are better recognized by the target population. No changes made to food items resulted in changes to the nutritional context of the original question. However, in some cases, different replacements were chosen based on the nutritional context of the question. For example, the NLit includes salmon in two domains, but because salmon is not commonly recognized by the Latino population of interest, it was replaced. In the Energy Sources in Food domain, salmon was replaced with atún (tuna) as the correct answer for a protein source high in healthy fats. A more commonly consumed fish is tilapia (not high in healthy fats), so in the Household Food Measurement section a question concerning the recommended portion size of salmon was instead replaced with tilapia.

The translation of the instrument into Spanish was by Consensus Translation^19–23. Two native Spanish speakers independently performed translations. After completion, a committee of three native Spanish speakers convened to review and revise the translations and decided on the most appropriate adaptation to use.

Content Validity

Three credentialed nutrition professionals who provide nutrition education directly with Spanish speakers in the target population were selected to review the translated instrument using a survey adapted from Polit and Beck²⁴. Experts ranked items for relevance on a 4-point scale ranging from 1–4 with ‘4’ being the most relevant, ‘1–2’ rankings were assigned a score of ‘0’ =’not relevant’, and ‘3–4’ rankings were assigned a score of or ‘1’ = ‘relevant’. Scale Content Validity Index (S-CVI), or “the average proportion of items given a rating of 3 or 4” was calculated for each domain and for the instrument overall by averaging the Item-CVIs. Acceptable S-CVI was set at ≥ 0.90²⁴ Expert rankings of item clarity and additional comments made by all experts were used together to identify the need for item modifications.

Cognitive Interviews

After modifications were made resulting from expert review. cognitive interviews were conducted among three native Spanish speakers. Cognitive interviews involve an open dialogue about the thoughts and interpretations made by participants as they complete the instrument²⁵. The interviews were specifically structured around the questions that arose from the expert review, mostly concerning either clarity of language or familiarity with food items included. An interview guide we previously used with English-speakers when developing the NLit²⁶ was translated to Spanish using Consensus Translation, and interviews for the present study were performed by a native Spanish-speaking research assistant. Comments made during the interviews helped guide final adjustments made to the NLit-S. Prior to instrument testing for validity and reliability, the finalized NLit-S was assessed by the Fernandez-Huerta Readability test to determine the grade-level reading. A score of 70 or higher is considered appropriate for a general adult population²⁷.

Phase 2: Instrument Validity and Reliability Testing

Construct Validity and Reliability

Construct validity has many definitions, but the term generally refers to an instrument’s ability to measure the intended concepts²⁸, nutrition literacy in this case. In one visit, participants completed the NLit-S, the SAHL-S¹⁰, and a demographic questionnaire. Answers to both NLit-S and SAHL-S were scored as correct or incorrect. Construct validity was determined two ways, by: 1) Bayesian confirmatory factor analysis (CFA) and 2) convergent validity²⁹, measured by the extent to which the NLit-S correlated with the SAHL-S using Spearman’s rho. The relationship of constructs via subscales of NLit and its respective items were analyzed by IRT via binary CFA. Binary CFA is a generalization of Rasch models. The binary CFA analysis was conducted using CBID software³⁰. The model fit was evaluated by two statistical fit indexes: Comparative Fit Index (CFI>.90) and Root Mean Square Error of Approximation (RMSEA<.08). The composite reliability was estimated with the output obtained by binary confirmatory factor analysis. Instrument reliability, or predictability and consistency, was measured via CFA and Cronbach-α. The interpretation of reliability was determined according to Shrout’s adjectives, which is: 0.00–0.10 as virtually none, 0.11–0.40 as slight, 0.41–0.60 as fair, 0.61–0.80 as moderate, and 0.81–1.0 as substantial reliability³¹.

RESULTS

Cognitive interviewing was conducted with three native Spanish speakers, while a total of 51 participated in validity and reliability testing. For the latter, most participants were female (76%) with a mean age of 35 years (range 18–63, SD 12.85). Approximately half of the participants (51%) had an annual income < $20,000. Mexico was the most prevalent country of origin (74.5%), and the primary language of the participants was Spanish, with 58.9% reporting they prefer to read more in Spanish than English or only Spanish, also 74.5% reporting speaking only Spanish or more Spanish than English at home. Characteristics of the test sample are shown in Table 1.

Table I.

Characteristics of the Sample of Spanish-Speaking Latinos who Participated in Validity Testing of the Nutrition Literacy Assessment Instrument in Spanish (n=51)

Characteristic	Mean ± SD or n (%)
Age, years	35 ± 12.8
Gender
Female	39 (76%)
Country of Origin
Mexico	38 (75%)
USA	4 (8%)
Other Spanish-Speaking Country	9 (17%)
Race
White	19 (37%)
Mixed Races	20 (39%)
Unknown	12 (22%)
Annual Household Income
<$10,000	13 (26%)
$10,000 to 19,999	13 (26%)
$20,000 to 29,999	9 (18%)
≥$30,000	8 (15%)
Education
< High school	10 (20%)
High school/GED or less	11 (22%)
Some college/associate’s degree	17 (33%)
Bachelor’s degree or higher	12 (24%)
Language Spoken at Home
Only Spanish	21 (41%)
More Spanish than English	17 (33%)
English and Spanish Equally	9 (18%)
Time Living in the Community
≤ 1 year	13 (26%)
>1 year to ≤ 10 years	12 (24%)
> 10 years	25 (49%)
SAHL-S Score	16.9 ± 1.83
NLit-S Score	46.6 ± 10.4

Phase 1: Cultural and Linguistic Adaptation

After content review, the combined S-CVI for all domains was 0.96, while S-CVI ranged from 0.90 (Energy Sources in Food) – 1.0 (Household Food Measurement and Food Label and Numeracy) among domains. The relevance of each item was strongly supported by content experts, and thus only adjustments were made to items, no items were deleted. Cognitive interviews further affirmed the relevance and clarity of the NLit-S and helped to guide final adjustments to the phrasing of questions and responses and to the food items presented throughout. CVIs and adjustments made to food items through content review and cognitive interviewing are found in Table 2. The Fernandez-Huerta Readability score was calculated at 77, and considered an appropriate reading level for the target population.

Table 2.

Results of Content Review and Changes to Food Items to Improve Cultural Relevance

DOMAIN	S-CVI*	CHANGES TO FOOD ITEMS
		Original Food	Final Food
NUTRITION & HEALTH	0.97	Spaghetti with meat sauce	Arroz con pollo
		Garlic bread	Una rebanada de pan tostado
		Green beans	Habas hervidas
		Chocolate pudding	El flan
		Lemonade	Horchata
ENERGY SOURCES IN FOOD	0.90	Pear	Mango
		Margarine	Manteca
		Toast with strawberry jam	Pan dulce
		Bacon	½ taza de frijoles
		Salmon	Atún
HOUSEHOLD FOOD MEASUREMENT	1.0	Salmon	Tilapia
FOOD LABEL & NUMERACY	1.0	Macaroni and cheese	Pozole
FOOD GROUPS	0.94	Lemonade	Horchata
		Fruit Punch	Tang de limón
CONSUMER SKILLS	0.97	Canned green beans	Elotes enlatados
		Frozen green beans	Elotes congelados
		Kale	La col rizada
		Blueberries	Piña
		Berry juice	Jugo de piña

^*Scale Content Validity Index: Experts ranked items from each domain for relevance and individual item-content validity index scores were calculated. S-CVIs were calculated by averaging the item-content validity index scores for each domain. An S-CVI score of 0.90 or higher for each domain is desired.

Phase 2: Instrument Validity and Reliability Testing

While 96% (n=49) of participants achieved an adequate health literacy score (SAHL-S score >14), only 14% (n=7) achieved a good nutrition literacy score (NLit-S ≥ 58) and 37% (n=19) scored in the lowest nutrition literacy category (NLit-S ≤ 44). The NLit-S and SAHL-S scores were positively correlated overall (r=.521, p<.001) and in each domain except Household Food Measurement. CFI and RMSEA both achieved acceptable model fit overall and in each domain. Entire reliability for the instrument was substantial (0.81–1.0) both overall and within each domain (0.918; 0.849–0.966, 95% Confidence Interval). Cronbach-α for the entire NLit-S was excellent at 0.92, with individual domains ranging 0.63 (Household Food Measurement and Consumer Skills) to 0.86 (Food Label and Numeracy). Validity and reliability statistics are presented in Table 3.

Table 3.

Validity and Reliability Statistics for the Nutrition Literacy Assessment Instrument in Spanish

NLit-Sa Subscale	Correlation with SAHL-Sb (Spearman’s r)	Comparative Fit Index (CFI)c	Root Mean Square Error of Approximation (RMSEA)d	Entire Reliabilitye (95% Confidence Interval, CI)	Internal Consistencyf (Cronbach’s alpha)
Nutrition & Health	0.412*	1.0*	0.000**	0.918 (0.849 – 0.966)###	0.667##
Energy Sources in Food	0.492**	0.970*	0.064**	0.933 (0.881 – 0.965)###	0.730##
Food Label & Numeracy	0.518**	1.0*	0.000**	0.974 (0.957 – 0.985)###	0.865###
Household Food Measurement	0.111	1.0*	0.000**	0.929 (0.863 – 0.967)###	0.630##
Food Groups	0.451**	1.0*	0.000**	0.968 (0.945 – 0.982)###	0.836###
Consumer Skills	0.284*	1.0*	0.000**	0.888 (0.787 – 0.954)###	0.608##
Entire NLit-S	0.521**	0.986*	0.031**	0.994 (0.992 – 0.996)###	0.914###

^aNLit-S=Nutrition Literacy Assessment Instrument-Spanish

^bSAHL-S=Short Assessment of Health Literacy-Spanish; p=<0.05*; p≤0.001**

^cCFI=Comparative Fit Index ≥0.90 indicate acceptable model fit*

^dRMSEA=Root Mean Square of Approximation ≤ 0.06 indicate acceptable model fit**

^eEntire reliability is the reliability of the entire domain.

^efWe classified reliability as follows: 0.61–0.80 is moderate reliability^##, 0.81–1.0 is substantial reliability^###

DISCUSSION

The Nutrition Literacy Assessment Instrument-Spanish (NLit-S) is the first comprehensive nutrition literacy measure available for Spanish speaking Latinos in the United States, the fastest growing minority in the country. Results of this study demonstrate the NLit-S is a reliable tool to measure six different domains of nutrition literacy (nutrition and health, energy sources in foods, food label and numeracy, household food measurement, food groups and consumer skills). The instrument presented excellent Cronbach-α and substantial entire reliability (0.81 – 1.00) for the overall questionnaire and for each of the six domains. Moreover, the instrument CFI and RMSEA were acceptable for the entire measure and for each domain of nutrition literacy. The NLit-S was strongly correlated (r=.521, p<.001) with SAHL-S, which is a validated measure of health literacy for Latinos in Spanish¹⁰.

The NLit-S was not only linguisticly adapted to Spanish, but it was also culturally adapted through the participation of Latino nutrition experts and community members to include culturally relevant foods before testing the instrument for validity and reliability. Importantly, the study sample was comparable to US Census data for Latinos in age (sample mean age was 35 years, US Census mean age was 34.4 years) and heritage (sample was comprised of 75% Mexican American, US Census was 64% Mexican American)³².

The moderately strong correlation between SAHL-S and NLit-S demonstrates that both instruments are measuring a global concept of literacy, yet it also represents the specificity of each measurement. SAHL-S is specific for concepts of health literacy of medical and healthcare terms, which are different than concepts of nutrition literacy and include health terms as well as nutrition knowledge and skill. These differences can be clearly observed in the difference we found in the mean scores of SAHL-S and NLit-S. Whereas mean health literacy scores were adequate, adequacy was not replicated in nutrition literacy scores, where the mean score of 46.6 indicates the possibility of poor nutrition literacy. The difference between health and nutrition literacy scores emphasizes that nutrition literacy requires knowledge and skills that are not measured by medical and healthcare terms.

One of these differing skill sets could be Household Food Measurement, which did not show significant correlation with SAHL-S scores. Questions in this section provide two cues for identifying the portion amount: 1) images of food items on plates and 2) a description of the amount of food in cups or ounces. Respondents are asked to select whether the portion depicted is “about right,” “too much,” or “too little” compared to what they perceive are recommended portions. It is possible that participants were unfamiliar with ounces and cups, although this concern was specifically raised by the research team during expert review and cognitive interviewing, and both groups felt the terms were familiar and did not require change. Additionally, in cognitive interviewing to refine the English NLit, several participants referenced the images and not the amounts to answer the questions¹⁴, so amounts may not be necessary cues. In the English NLit validation, Household Food Measurement scores were significantly related to diet quality¹⁶. Thus, the lack of correlation suggests that further investigation into this domain for Spanish-speaking Latinos is needed.

While adequate scores on the SAHL-S indicate reading levels were adequate for comprehending the NLit-S, lower scores on the NLit-S do raise the question of appropriate targets for reading levels of educational materials created for general Spanish-speaking audiences. The NLit-S requires reading skills to answer questions, and it was developed for reading levels of 77, considered adequate for an adult population³³. However, there is some evidence that targeting a lower reading level may be necessary for this population. The Program for International Student Assessment (PISA) is a global evaluation that aims to assess the quality of education in different countries, in their native languages, through questions that require reading, science and mathematics skills. Per the data from PISA 2015, among eight Latin American countries including Mexico, all scored below average in the three evaluated areas³⁴. Townsend et al also emphasized that Latinos who use English as a Second Language (ESL) and have low-incomes may also experience difficulties reading in Spanish. They suggest using a systematic 5-step process, including visual cues combined with minimal text information, significantly improves readability for this population³⁵.

While nutrition knowledge and food label use among Latinos are important determinants for consumption of fruits and vegetables^36,37, sugar-sweetened beverages^36,37, fast food³⁷, energy intake³⁷ and weight management³⁸, Latinos still face important barriers that prevent them from being empowered to make healthy food choices. Our findings concur with Bohel, who noted that Latinos in the US face not only linguistic barriers related to English proficiency, but also comprehension of nutrition information that surpasses overall literacy³⁹. Similarly, Perez-Escamilla et al, reported that in a sample of mainly Spanish-speaking Latinos, positive attitudes and general knowledge of nutrition was present, however there was a lack of nutrition knowledge, especially for understanding nutrition terms, such as saturated fat; understanding of the relationship between nutrition and health outcomes; and food sources of vitamins and minerals⁴⁰. Likewise, Sharif et al found that while Latinos frequently checked food labels for nutrition information, they did not correctly comprehend information from the labels⁴¹.

There are important limitations to this study. First, dietary intake was not calculated for this sample, and as a result the association between nutrition literacy and diet quality in this sample specifically is unknown. Second, participants did not complete a test-retest to demonstrate reproducibility of test results. However, both diet quality and test-retest were collected in the validation of the English version of the instrument, which included a relevant percentage of English speaking Latinos. Third, contrary to the sample completing the English NLit, neither existence of chronic disease nor health education due to chronic disease was measured in this sample. Moreover, while general acculturation was measured in this group, dietary acculturation was not. The study was also subjected to common survey limitations such as self-report and social desirability bias. Finally, it is important to emphasize cautions for generalizations for other Latino subgroups in the US, since this sample was composed of dominantly Mexican-Americans who were recent immigrants (living less than 10 years) in the Midwest.

Clearly, nutrition literacy is an understudied area among Latinos, despite their well-reported disparity of health literacy. The NLit-S helps to move efforts forward for a new area of research on nutrition and health disparities for Spanish-speaking Latinos. Adapting and creating nutrition education materials and interventions to target improvements in known nutrition literacy levels of Latinos could help in the prevention and treatment of nutrition-related chronic diseases thereby reducing long-term morbidity and mortality in this minority group. Additionally, the processes used to translate, adapt and validate the tool for Spanish-speakers can serve as a model for adaption of the NLit or other nutrition measures to different populations with other languages and/or cultural backgrounds in the US and the world. This model is essential, because it is well-reported that only back-translation, without cultural adaptations, to other languages can have a significant impact in the accuracy of instrument psychometrics⁴².

While the NLit-S demonstrates validity and reliability for assessing nutrition literacy levels of Spanish-speaking Latinos, more research is needed to understand the relationship between nutrition literacy and dietary behavior among this minority, aiming to enhance efficacy of nutrition education and intervention programs to increase diet quality and health outcomes in this population.