Cognitive Interviews Conducted in Spanish: A Method for Enhancing a Hispanic Infant Feeding Questionnaire

Abstract

Research indicates that using cognitive interview techniques, that are qualitative in nature, can enhance the quality of health survey instruments. This study was conducted during the formative phase of the Un Bebe Saludable: Un Regalo Muy Especial (A Healthy Baby: A Very Special Gift) program. Cognitive interviews were conducted in Spanish in an effort to improve the quality of a Hispanic infant feeding behavior survey designed to evaluate a peer education program for Latina mothers participating in the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC). Results indicate that a small sample of cognitive interviews can improve the level of question comprehension and the ease at which respondents can process the information from specific survey items. Our experience suggests that adding cognitive interviewing to conventional field pre-testing and back-translation techniques during the development of Hispanic health surveys may enhance the quality of the health behavior data that are generated.

There is a growing need for culturally competent programs focusing on health promotion and disease prevention among Hispanic subgroups. Despite this urgent need, there continue to be relatively few culturally appropriate survey instruments available for measuring health behavior changes among specific Hispanic subpopulations (U.S. Department of Health and Human Services, 1993). Developing a survey instrument that is appropriate for Latinos in the United States frequently requires both modifying the English version of the instrument as well as subjecting the Spanish-language instruments to rigorous testing which includes both cognitive testing and pretesting (Weidmer, Brown, & Garcia, 1999).

Despite nearly two decades, one of the most comprehensive Hispanic health research studies to date is the Hispanic Health and Nutrition Examination Survey (HHANES) conducted from 1982 until 1984. The Hispanic subpopulations that were studied included Mexicans from the Southwest; Cubans from Dade County, Florida; and Puerto Ricans from. New York. The surveys that were used received forward and backward translations. The study was further enhanced by the use of bilingual and bicultural interviewers. Limitations exist in the application of HHANES research data because the data should not be extrapolated and applied to the same subpopulation living in other areas of the United States, nor can HHANES data be used to develop appropriate public health agendas for Central and South American immigrants and refugees (Carter-Pokras, 1994; Kuczmarski, Kuczmarski, & Najjar, 1995).

Developing survey instruments that are both linguistically and culturally valid is challenging, yet imperative. The challenge is not only that the instruments be culturally and linguistically relevant, but that they be comprehensible, maintaining the original meaning of the items used (Sperber, Devellis, & Boehlecke, 1994).

One of the most widely utilized methods for validating the translation accuracy of research instruments is known as back-translation (Brislin, 1986; Marin & Marin, 1991). Back-translation is a method where an original questionnaire is translated into another language and then translated back into the source language by a blinded, independent translator. For example, the Spanish translation of an English instrument is given to an independent bilingual translator, who then translates it back into English without having seen the original English version. The source-language versions are compared and can then be revised as needed (Sperber et al., 1994). Although using back-translation techniques to translate Spanish instruments back to English may improve linguistic accuracy, this translation technique cannot provide insight into a respondent's interpretation of the question. By adding cognitive testing to the conventional use of back-translation in Spanish survey development, researchers can minimize some of the sources of measurement error by identifying questions that may be difficult to comprehend, questions that are misinterpreted by the respondents, and response options that fail to reflect the respondent's experience (Jobe & Mingay, 1991).

Background

Over the past two decades, collaboration between cognitive scientists and survey researchers has enhanced the quality of health behavior surveys through the application of cognitive testing techniques (Fienberg, Loftus, & Tanur, 1985; Jobe & Mingay, 1991). During a 7-year period between 1978 and 1984, a collaborative group of cognitive psychologists and survey methodologists began to apply methods and theories of cognitive science to the study of respondents' answers to autobiographical and attitude surveys (Jobe & Mingay, 1991). Government agencies in the United States, West Germany and the United Kingdom were instrumental in promoting most of the scholarly conferences and research that followed in the collaborative effort. In the US, the National Center for Health Statistics (NCHS) established the Questionnaire Design Research Laboratory (QDRL), the first permanent laboratory for conducting cognitive interviews as a standard part of questionnaire development (Warnecke et al., 1997).

Some of the general features of the cognitive interviewing technique include:

1. focusing on the questionnaire instrument rather than the entire survey process;

2. detecting both overt and covert problems in the survey by paying particular attention to the mental processes that the respondents are using to answer the questions; and

3. testing survey questions at various points in the design process. (Willis, 1994)

The basic approach to cognitive interviewing includes both “think aloud” and “probing” techniques. Thinking aloud refers to when the respondent is asked to “think aloud” as they answer the survey questions. The interviewer reads each question and records the processes the respondent uses to answer each question. The interviewer will interject “tell me what you're thinking” when the respondent pauses (Willis, 1994). Probing is a technique whereby the interviewer asks the survey questions and then follows the respondent's answer with requests for additional information based on the specific content of the respondent's answer. Probing that is conducted at the time the questionnaire is being administered is known as concurrent probing. Probes can be spontaneous or they can be prepared prior to the cognitive interview (Foddy, 1998; Willis, 1994; Willis, Royston, & Bercini, 1991). Most cognitive interviews include both spontaneous and prepared probes.

Cognitive interviewing, in addition to back-translation, can assist investigators in exploring respondents' abilities to interpret questions, their techniques used for the retrieval of information from memory, their judgment formation on specific questions, and their editing responses (Sudman, Bradburn, & Schwarz, 1996; Willis et al., 1991). Such insight is especially relevant for cross-cultural research, such as the “Un Bebé Saludable: Un Regalo Muy Especial” (A Healthy Baby: A Very Special Gift) project, involving women from Central and South America.

This study discusses the translation and cognitive interviewing techniques used to enhance the quality of the feeding survey instrument that was used for the evaluation of the Hispanic peer education program, “Un Bebé Saludable: Un Regalo Muy Especial.”

Six cognitive interviews were conducted in Spanish with a sample of women from Central and South America, in an effort to improve the quality of a Hispanic infant feeding behavior survey that was designed to evaluate a peer education program for Latina mothers participating in the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC). Survey questions were designed to provide descriptive sociodemographic data and to explore changes in eleven key health behaviors during the infants' first year of life (see Table 1).

TABLE 1. Key Messages for “Un Bebe Saludable: Un Regalo Muy Especial”.

“Un Bebe Saludable: Un Regalo Muy Especial” Key Messages

Prolong breastfeeding. Respond to baby's signals of hunger and fullness. Feed only breast milk, formula, or plain water in your baby's bottle. Avoid using sweeteners in the baby's bottle. Avoid putting the baby to bed with a bottle. Avoid the use of sweeteners on the baby's pacifier. Introduce solid foods (including cereal) when your baby is four to six months of age Feed breast milk or formula rather than cow's milk until the baby is 12 months of age Discard any breast milk or formula left in your baby's bottle after feeding. Avoid adding sweeteners, fats, and salt to your baby's food. Follow your health care provider's advice on immunizing your baby.

The Process for Conducting the Cognitive Interviews

Research indicates a need for culturally appropriate education and support for new Hispanic mothers concerning the feeding of their newborns. This need has been reinforced by Healthy People 2010, which includes specific objectives directed at prolonging breastfeeding for Hispanic infants under the age of one year (U.S. Department of Health and Human Services, 2000). “Un Bebé Saludable: Un Regalo Muy Especial” (A Healthy Baby: A Very Special Gift) was a peer education program designed in response to these issues.

Funded by the Extension Service, United States Department of Agriculture (USDA), the “Un Bebé Saludable” project was a 14 month intervention evaluation study, which began prenatally, to promote breastfeeding and other appropriate infant feeding behaviors for Hispanic mothers and their infants enrolled in the Special Supplemental Nutrition Program for Women Infants and Children (WIC). The intervention was pilot-tested in a treatment/control group design in Prince George's and Montgomery Counties in Maryland. Trained Hispanic peer educators, or “promotoras” (promoters), delivered the intervention through individual home visits, using specially designed materials that were pre-tested with members of the target population. Evaluation included pre- and post-intervention infant feeding behavior surveys conducted in Hispanic mothers' homes by trained bilingual interviewers.

Several steps were involved in the design of the pre-and post-intervention survey instrument. First, formative research was conducted through interviews with 52 Central and South American mothers recruited at local pregnancy and pediatric clinics, to identify key infant behaviors that should be targeted in the “Un Bebé Saludable” interventions (Ellis, Feldman, Anliker, & Bernstein, 1994). Second, a program evaluation survey was developed in English to assess changes in each of these behaviors, as well as health behavior theory constructs including behavioral intent and social support.

The English version of the survey was then translated into Spanish by three bilingual project committee members. The project promotoras were also consulted for language usage, such as specific Latin American terms for breastfeeding, bottle feeding, and pacifiers. It is also interesting to note that during the initial instrument development, some questions were formulated simultaneously in both English and Spanish, to anticipate potential translation problems. This process was similar to that of “decentering,” which refers to approaching the initial English version of the questionnaire as subject to improvement and modification based on the input received during cross-cultural review (Cella et al., 1998; Sperber et al., 1994). Decentering allowed the modification of the source wording when translations by the bilingual committee members revealed ambiguity and lack of clarity in the original wording.

Next, an outside bilingual consultant from Peru back-translated the Spanish translation of the instrument into English. Since the survey respondents would be predominantly from Central and South America, it was important that the back-translation be conducted by someone with a similar cultural and linguistic background.

Cognitive interviews began as soon as the Spanish version of the infant feeding behavior survey was verified as being linguistically appropriate, and the University Institutional Review Board granted approval for the study. Two bilingual project committee members conducted the cognitive interviews with women attending a local Maryland pregnancy center. Women were recruited in a convenience sample on two clinic days designated by the center's director. To be eligible, respondents had to be Hispanic immigrants to the United States and be pregnant.

The questionnaire was first piloted with two women to determine the length of time needed to complete each questionnaire using conventional interviewing techniques. Then six additional interviews were conducted using the cognitive interviewing methods. Of the eight respondents, three were from El Salvador, two from Mexico, and one each from Guatemala, Peru, and the Dominican Republic. Each woman was given a $10 gift certificate to a local grocery store for her efforts.

The bilingual project researchers developed a Spanish adaptation of the think-aloud and probing techniques described in the National Center for Health Statistics (NCHS) Working Paper Series entitled, “Cognitive Interviewing and Questionnaire Design: A Training Manual” (Willis, 1994). One bilingual researcher conducted the cognitive interviews in Spanish while a second bilingual researcher observed the interviews and transcribed responses.

The following introduction was given in Spanish to each respondent:

We have designed a Hispanic infant feeding program for new mothers. We have developed a questionnaire for the project and would like your help in improving it. There are no right or wrong answers, but we would like to know what you are thinking as you answer each question.

A practice think-aloud exercise was then given to the respondents before beginning the cognitive interviews. Each respondent was asked to do the following:

While we are going through the questionnaire, I'm going to ask you to think aloud so that I can understand if there are problems with the questionnaire. By “think aloud,” I mean repeating all the questions aloud and telling me what you are thinking as you hear the questions and as you pick the answers. Visualize the place where you live and think about how many windows there are in that place. When you are counting the windows tell me what you are seeing and thinking.

In addition to the think-aloud method, both spontaneous and prepared verbal probes were used to explore the basis for various responses. Prepared verbal probes were developed prior to the interviews for questions where the research committee anticipated potential problems. Research has shown that the effectiveness of a probe is directly related to its specificity (Foddy, 1998). The spontaneous probes were developed by the interviewer during the course of the interview, depending on the respondent's answer to the question (Williset al., 1991). The following are examples of prepared verbal probes that were used:

1. What does the term _____________ mean to you?

2. What things do you need to remember to answer each question? Do you recall events, or are you approximating to be able to answer?

3. Is it easy or difficult to remember what actually happened?

4. In your own words, what does the question ask?

Results

The length of the six cognitive interviews ranged from 36 to 55 minutes. Conventional interviews using the same instrument were 25 to 30 minutes in duration. Although time consuming, cognitive testing improved the quality of the original survey instrument by identifying specific items as either confusing or difficult for the respondents to process. Respondents were able to tell the interviewer that certain items did not make sense to them unless they were modified in specific ways. For example, the following item evaluating the key behavior “Avoid putting the baby to bed with a bottle,” was difficult for respondents to interpret, process, and respond to:

How strongly do you agree with the following statement?

“If a baby is old enough to hold a bottle, it is acceptable for them to go to bed with it.”

1. strongly agree

2. agree

3. not sure

4. disagree

5. strongly disagree

Due to the risks of baby-bottle tooth decay and choking, baby-bottle propping is not appropriate at any age; however it was important to note how the respondents were interpreting this question when it was asked. Two respondents answered that they were “not sure.” When probed, the respondents said they were “not sure” because it depended on the age of the child. One respondent further elaborated that it would not be acceptable for an infant to hold a bottle in bed because they could choke, but for a 3 or 4 year old who could place the bottle on the table when they were finished, it would be acceptable. Based on these responses the survey item was revised as follows:

Do you think that if a baby is old enough to hold a bottle, it is okay to put the baby to bed with the bottle?

• a. yes

• b. no

• If yes, at what age? _______________

Since research indicates that acculturation plays an important role in the modification of Hispanic values, norms, attitudes, and behaviors, a five-item acculturation scale was included at the end of the survey (Balcazar, Castro, & Krull, 1995; Garza & Gallegos, 1995; Marin, Sabogal, Marin, Otero-Sabogal, & Perez-Stable, 1987). The five acculturation items were based on language usage (Wallen, Feldman, & Anliker, 2002). Respondents were asked to answer on a five-point scale, indicating:

1. if they spoke Spanish 100% of the time;

2. if they spoke Spanish 75% of the time;

3. if they spoke Spanish 50% of the time;

4. if they spoke Spanish 25% of the time; or

5. if they spoke English 100% of the time.

Cognitive testing revealed that one of the acculturation items needed revision. The original question was as follows:

In which language do you prefer to speak?

One of the respondents laughed and answered, “I would prefer to speak English, but I can't!” The item was revised as follows:

In which language do you feel most comfortable speaking?

In addition to improving the accuracy and comprehension of specific items, cognitive interviews allowed respondents to evaluate redundancy of question content. The final survey version of the survey has 132 items, compared to 143 in the original questionnaire.

Discussion

This report discusses the types of problems that can be detected in an infant feeding questionnaire by using cognitive interviewing during the pilot testing of the instrument. The cognitive methods used in this study effectively identified cognitive sources of response error that would not have been evident when using the typical field pretesting of a questionnaire.

The cognitive methodology used in the “Un Bebé Saludable” questionnaire development is unique in several ways. First, while most cognitive interviews have been conducted in laboratory facilities such as universities, our cognitive interviews were conducted in the field at a local pregnancy center. Second, our interviews were conducted in Spanish using bilingual and bicultural interviewers. Research has shown that the language chosen for survey data collection may be significant (Bond & Yang, 1982; Feldman, 1975), yet there have been a limited number of studies describing the benefits of using cognitive interviews conducted in Spanish with Hispanic immigrants to the United States as the respondents (Weidmer et al., 1999).

Implications for Research and Practice

Although our study involved only a small sample of cognitive interviews, it is important to recognize the contributions that these interviews provided. The modifications that we made to the infant feeding questionnaire as a result of these interviews enhanced the quality of the instrument, and enabled us to more effectively and accurately evaluate the “Un Bebé Saludable: Un Regalo Muy Especial” program.

In order for data generated by health behavior surveys to be useful and relevant for shaping health policy and care delivery for a given population, it is crucial to assess how respondents understand and answer questions related to their infant feeding practices (Fienberg et al., 1985). Our cognitive testing results are limited because of the small sample size that was used, however the results add to a growing body of knowledge that demonstrates the effectiveness of cognitive methodology in improving the quality of data generated by health behavior surveys (Bailey & Marsden, 1999; Jobe et al., 1990; Warnecke et al., 1997; Weidmer et al., 1999).

Despite the vast need for expedited research involving specific Hispanic subpopulations, it would be irresponsible to conduct health behavior surveys that have not been rigorously tested for both linguistic and cultural accuracy. Although standard pretesting may provide the researcher with information about the time required to complete a survey and the ease with which the questions are understood, it does not provide insight into specific question interpretation, memory retrieval, nor judgment formation. The quality of our survey instrument was clearly enhanced by using the cognitive techniques described.

Research using cognitive methodology to improve health behavior survey administration continues to grow. Opportunities exist for collaboration between cognitive psychologists and survey researchers in field survey interviews such as those used in “Un Bebé Saludable: Un Regalo Muy Especial” (A Healthy Baby: A Very Special Gift).