Understanding Spanish-language Response in a National Health Communication Survey: Implications for Health Communication Research

Abstract

Spanish-speaking Latinos account for 13% of the US population yet are chronically under-represented in national surveys; additionally, response quality suffers from low literacy rates and translation challenges. These are the same issues that health communicators face when understanding how best to communicate important health information to Latinos. The Health Information National Trends Survey (HINTS) offers a unique opportunity to understand the health communication landscape and information needs of the U.S. population. We describe the challenges in recruiting Spanish-speaking HINTS respondents and strategies used to improve rates and quality of responses among Spanish-speaking Latinos. Cognitive interviewing techniques helped to better understand how Spanish-speaking Latinos were interpreting the survey questions, and the extent to which these interpretations matched English-speaking respondents’ interpretations. Some Spanish-speaking respondents had difficulty with the questions because of a lack of access to health care. Additionally, Spanish-speaking respondents had a particularly hard time answering questions that were presented in a grid format. We describe the cognitive interview process, and consider the impact of format changes on Spanish-speaking people’s responses and response quality. We discuss challenges that remain in understanding health information needs of non-English-speakers.

Introduction

Numbering 56.6 million, Latinos in the United States account for one in four youth under age 17 and one in six of all US residents (U.S. Census Bureau, 2016). Latino ethnicity is defined as having ancestry from a Spanish-speaking country; however, Spanish-language proficiency is an important source of diversity within this population. Of the one in six U.S. residents who is Latino, 38% are Spanish monolingual, another 36% are bilingual Spanish/English, and the remaining 25% are English monolingual (Krogstad & Gonzalez-Barrera, 2015). National surveys that aim for representativeness of the population should include Spanish-speaking Latinos to achieve that goal. However, obtaining adequate samples of Spanish-speakers in general national population surveys is time-consuming and requires significant resources. Ensuring that those sampled provide complete and comparable responses is additionally challenging and represents a unique health communication issue.

This study reports on efforts to understand barriers to Spanish-speakers’ responses and response quality in a national mail survey on health communication issues administered by the National Cancer Institute (NCI). Beyond concerns about representativeness important for all survey researchers, the survey examined herein is especially relevant to communication researchers because many of the questions have to do with communication processes, sources and information preferences. Challenges identified in this study are important for practitioners to consider in the design of health communication materials for Latinos.

Obtaining adequate samples of Spanish-speakers to national surveys is difficult due to a variety of challenges. Operationally it is challenging to contact respondents that do not speak English: For interviewer-administered surveys, this is a labor-intensive process that typically requires multiple contacts with the household. For a telephone survey, the initial call into the household is typically made by an English-speaker who identifies the residents as non-English-speakers and the language spoken. A new interviewer, who speaks the language, is then assigned to the household. A similar process occurs for in-person surveys. However, this process cannot be easily replicated on a survey that contacts households via U.S. postal mail. There are not tailored opportunities for a survey representative to identify non-English speaking households and provide a questionnaire translated into the appropriate language. Current practice is to pre-identify households that are likely to have Spanish-speakers and send two questionnaires, one in English and one in Spanish; at least one study has shown that it may be better to send two questionnaires to every household (Brick, Montaquila, Han, & Williams, 2012).

The challenges to Latino participation in surveys are common to many mail surveys, and not unique to HINTS (Dutwin & Lopez, 2014). However, the nature of the questions asked in HINTS may amplify the potential impact of low response rates, because the questions deal specifically with information preferences, exposures, and communication challenges that are inextricably tied to the literacy and language challenges faced in survey development and translation. Spanish-dominant Latinos in the U.S. are less-educated, have lower incomes, and have less access to health care, compared with either English-dominant Latinos or the non-Hispanic White population (Hakimzadeh & Cohn, 2007). The low levels of health care access among Spanish-speakers may render surveys on this topic less salient and relevant. As such, low response rates among Spanish-speakers may be somewhat attributable to the focus of these questions. If this is true, it introduces a unique type of bias that may affect understanding of this population’s health information needs and preferences and influence the design of appropriate interventions.

Additionally, ensuring that Spanish-language responses are concordant with English-language responses to the same questions is challenging, again due to literacy barriers that transcend culture or language, and also, uniquely related to culture, relevance and culturally-patterned response styles. Previous studies have demonstrated culturally-patterned differences in response styles across countries (Johnson et al., 1997), and other studies have examined within-ethnicity differences and similarities among ethnic minority groups in the United States (Cho, Holbrook, & Johnson, 2013; Davis, Resnicow, & Couper, 2011; Marin, Gamba, & Marin, 1992). A prominent theory of survey response describes how culture, operationalized as race/ethnicity, affects four cognitive tasks required to answer survey questions: interpretation, information retrieval from memory, judgment formation, and response editing (Warnecke et al., 1997). The strongest evidence for the effects of culture on survey question response is the result of differences in response editing, in that an acquiescent response style may be a result of stronger cultural pressure to provide socially desirable responses. Among Latinos, simpatía – a personal quality that implies conflict avoidance and emphasizes positive behaviors while deemphasizing negative behaviors – may be a particularly relevant cultural script (Triandis, Marin, Lisansky, & Betancourt, 1984). If culture has some influence over the relevance of items beyond language translation, we should see that difference for English-speaking as well as Spanish-speaking Latinos. Previous research suggests that Latinos respond to survey questions differently than the mainstream, non-Hispanic population, although these differences attenuate with acculturation (Marin et al., 1992). For example, Davis and colleagues (2013) found a strong preference for response scale endpoints (extreme response style) among 70% of a sample of 288 Mexican-Americans; however, less-acculturated Latinos (Spanish-dominant) were more likely than more-acculturated Latinos to provide extreme responses. Consistent with this finding is Cho and colleagues’ finding that Latino respondents who were born in Mexico or Puerto Rico and preferred to communicate in Spanish experienced greater difficulty in understanding survey questions (Cho et al., 2013). This pattern – ethnic differences in response style, attenuating by acculturation status – is also supported outside of the United States context (Morren, Gelissen, & Vermunt, 2012).

We use the design of the largest national health communication survey as a unique exemplar of health communication challenges faced by those trying to reach Spanish-speaking Latinos in the United States. The objective was to examine the challenges in recruiting Spanish-speaking Latino respondents to HINTS, and to describe the methodological strategies used to improve response rates and data quality, defined as completeness and comparability to English-speakers, among this population. Specifically, we sought to understand how Spanish-speaking respondents understood the questions and response options, how Spanish-speakers formulated their responses, and how these factors differed between English-and Spanish-speaking respondents.

The Health Information National Trends Survey (HINTS)

The National Cancer Institute’s (NCI) HINTS was developed over a decade ago to monitor changes in health communication, information technology, and related health behavior and knowledge (Nelson et al., 2004). Each wave has been fielded in both English and Spanish, with the exception of HINTS Puerto Rico, which was fielded only in Spanish. As of February 2017, the publicly-available data set had been used for analyses in over 300 publications (http://hints.cancer.gov/research.aspx). HINTS was first fielded as a Random Digit Dial (RDD) survey in 2002-2003, and again in 2005. In 2008, in response to falling response rates, a split mailed survey and RDD sampling frame was employed as an embedded experiment to evaluate impact on response rates (n=7,674).

The fourth iteration of HINTS included five distinct data collection cycles; this manuscript reports on the first three. Instruments for each data collection cycle of HINTS 4 include a core set of items common to each data collection cycle as well as rotating and special emphasis modules. Data were collected for the first cycle from October 2011 through February 2012 (n=3,959), data for the second cycle were collected from October 2012 through January 2013 (n=3,630), and data for the third cycle were collected from September 2013 through December 2013 (n=3,185). Cognitive testing of survey questions was conducted prior to the full launch of each cycle and was iterative: Results informed the redesign of questions that were then retested. Cognitive testing for the pilot and first cycle included tests of multiple versions of core survey items; for subsequent cycles, cognitive testing aimed to assess new questions to be included in rotating and special emphasis modules and where time allowed, also included testing of core items.

Strategies to Improve Response Rates

HINTS 4 Cycle 1 was conducted by mailed questionnaire following a protocol similar to that employed for the 2008 data collection. The sample for HINTS 2008 was drawn from a split frame and administered in dual modes wherein approximately half of the respondents received the survey by telephone interview; half by mailed questionnaire (Cantor et al., 2009). Overall, response rates for HINTS 2008 were higher for the mailed survey (31%) than telephone survey (24%). Among Hispanic respondents to the 2008 survey, a higher percentage responded to the mailed survey (9.3%) compared to the telephone survey (7.9%); in 2008, the mailed instrument was only provided in English with instructions in Spanish for Spanish-speakers who were interested in participating to call a toll-free number to complete the survey by telephone.

To encourage survey response, the HINTS program moved to use of mailed surveys and several enhancements were made to the 2008 protocol for HINTS 4. HINTS 4 implemented two approaches to within-household sample selection wherein either all adults within a sampled household were asked to complete the survey (all-adult approach) or the adult with the next birthday was asked to complete the survey (next birthday approach). A total of four mailings (initial mailing, a reminder postcard, three follow-up mailings) were sent to sampled households following a modified Dillman approach (Dillman, Smyth, & Christian, 2014). For the all-adult approach, rigorous follow-up procedures were developed wherein partially-complete households were sent additional mailings targeting the non-responding adults in the household and a short version of the survey instrument was developed to target households that did not respond to early invitations to participate. To improve participation among Hispanic respondents, all of the survey materials, including letters, FAQ’s, and questionnaires, were translated into Spanish following the TRAP method (Harkness et al., 2010). Households were flagged as potentially Spanish-speaking using the following methods: 1) Households located in Census tracts with relatively high proportions of linguistically-isolated Spanish households were flagged; 2) Households identified as having a Hispanic surname were flagged; and 3) Households that requested Spanish-language materials were flagged. All households flagged as potentially Spanish-speaking were sent cover letters and questionnaires in English and Spanish for the second, third, and fourth mailing. An additional mailing, sent only to households flagged as potentially Spanish-speaking, was added to the protocol to further encourage the return of Spanish questionnaires. Further details on the mailing protocol are described in the HINTS 4, Cycle 1 methods report; a summary of all HINTS responses is in Table 1.

Table 1.

The Health Information National Trends Survey (H INTS), Survey Administration Modes and Response Rates by Ethnicity and Language.

	HINTS 1 (2003)	HINTS 2 (2005)	HINTS 3 (2007)	HINTS Puerto Rico (2009)	HINTS 4 (2011-2014)
					Cycle 1	Cycle 2	Cycle 3
	Phone	Phone	Mail & Phone	Phone	Mail	Mail	Mail
N, Full sample	6369	5586	7674	639	3959	3630	3,185
Response Rate1	33.1%	20.8%	RDD = 24.2%	58.2%	36.7%	40.0%	35.2%
			Mail = 31.0%
N, Latino, English	428	230	426	0	382	397	343
N, Latino, Spanish2	336	266	196	639	79	121	168

¹Response rates were calculated using the RR2 formula of the American Association of Public Opinion Research (AAPOR); additional methodological details can be found in the technical documentation at: www.hints.cancer.gov.

²Assumes that respondent is Latino if survey response is in Spanish and race/ethnicity is missing.

In the remainder of this paper we discuss the problems Spanish-speakers had in responding to the HINTS questionnaire and their implications for health communication design and research. Results are based on observations made during the pre-testing process for each of three separate cycles of HINTS 4, as well as review of the data that was returned by respondents during the main data collection.

Cognitive Interview Methods

Design and instrumentation

Survey questions were developed by an interdisciplinary team of researchers and tested using cognitive interviewing. Cognitive interviewing developed collaboratively between cognitive psychologists and survey methodologists (Jabine, Straf, Tanur, & Tourangeau, 1984; Jobe & Mingay, 1991; Kerwin & Willis, 2010; Willis, 2005). For purposes of pretesting and evaluating survey questionnaires, individuals who are representative of the population to be surveyed are recruited to participate in one-on-one interviews with trained interviewers, who instruct participants to verbalize their thought process as they read and answer survey questions. Interviewers observe survey respondents’ cognitive processes by eliciting narratives about how respondents answer questions by administering verbal probe questions (e.g., “What does it sound like that question is asking you?”). Cognitive interviewing provides information about how respondents comprehend a question, retrieve relevant information, form summary statements, and select appropriate response options; and also identifies situations in which the premises and logic of the evaluated survey question fail to match the respondent’s life situation.

Procedures

Interviews were conducted by expert, native-language facilitators employed by either the primary contractor responsible for the survey (Westat; English interviews) or a market research company that specializes in Latino audiences (EurekaFacts; Spanish interviews). Project staff trained interviewers in the specific protocols. The interviews were conducted in conference rooms equipped with two-way mirrors and were audio recorded. Project staff observed the interviews in progress; before ending the interview, interviewers checked in with the observers to add questions for clarification as needed. Interviews lasted 90 minutes.

Recruitment

Participants were recruited from suburban Washington, D.C. to represent two groups: cancer patients/survivors and the general population. Cancer patients/survivors were recruited from a list maintained by Westat. To recruit from the English-speaking general population, Westat placed advertisements around the region and in online classifieds. To recruit from the Spanish-speaking general population, EurekaFacts placed advertisements around the region and in online classifieds, conducted in-person recruitment efforts with community organizations that serve the Latino population, and recruited from the EurekaFacts participant database. Participants received $50 as an incentive to participate in the study.

Participant Characteristics

Spanish-speaking respondents’ countries of origin included: Puerto Rico, El Salvador, Colombia, Bolivia, Honduras, Guatemala, Mexico, and Peru. Overall, Spanish-speaking respondents had lower levels of education compared with the English-speaking respondents. Nearly half of the English-speaking participants were non-Latino White. Table 2 provides the total number of interviews conducted, over the three cycles of interviewing, and broken out by the language in which the interview was conducted. The results discussed below draw from interviews conducted over these three waves.

Table 2.

Cognitive Interview Participant C haracteristics

	Cycle 1		Cycle 2		Cycle 3
	English	Spanish	English	Spanish	English	Spanish
Group
Cancer	10	5	13	7	10	5
General population	23	12	9	8	10	5
Age
18-29	5	5	5	3	5	1
30-54	15	9	10	9	6	5
55+	13	3	7	3	9	4
Sex
Male	13	8	8	7	8	4
Female	20	9	14	8	12	6
Ethnicity
White	10	0	14	0	10	0
Non-white, non-Latino	11	0	7	0	9	0
Latino/Hispanic	2	17	1	15	1	10
Education
Less than H.S.	0	5	0	3	0	2
High school	7	5	5	3	9	6
Some college	17	6	10	4	7	0
College degree	9	1	7	5	4	2

Analytic Approach

Two researchers, who also participated in the English-language interviewing, conducted a qualitative analysis of the cognitive interview data. The analysis focused on describing the meaning of the answers to each question, identifying and describing any problems that arose during the interview and offering recommendations on the use of the questions (Willis, 2010). Analysis was conducted on two levels, according to the Successive Aggregation cognitive analysis model described by Willis (2015). First, interviewers summarized the findings for each tested question from each interview that they conducted, in the original language of the interview. These interviewer summaries generally consist of descriptions of how participants answered the questions and any problems that occurred during the question-answer process. The Spanish summaries were translated into English. Systematic question-by-question analyses were conducted to identify themes or patterns that occurred across interviews. The interviewer summaries were consolidated. In addition, audio recordings of the interviews were used to review or supplement the information from the summaries. This step was iterative and conducted at each of the data collection cycles, allowing for changes to be made to instruments prior to the release of the survey for the main data collection. Finally, a bilingual investigator uninvolved in the original analyses but who observed English and Spanish cognitive interviews compared the English and Spanish cognitive interview reports and detailed question-by-question summaries to understand the extent to which question interpretations matched or failed to match. This step was conducted after the third data collection cycle and served to inform the fourth cycle and also to glean lessons for health communication design and research.

Cognitive Interview Results

We organize key findings into two themes: formatting concerns and content relevance.

Formatting Concerns

Many of the questions tested featured scaled response options (Fig. 1). Spanish-speaking respondents tended to dichotomize responses (i.e., demonstrated extreme response style). The “somewhat” response options did not appear to resonate with Spanish-speakers (e.g., H4, Fig. 1), who generally selected either endpoint when presented with a scale. English-speakers, on the other hand, exhibited more variation in response styles, although they also dichotomized some questions. English-speakers exhibited extreme response to questions that focused on a specific decision or interaction with the medical establishment (e.g., D2, Fig. 1), but were more comfortable exhibiting a range of responses to questions about their personal beliefs or character (e.g., H5, Fig. 1).

Fig. 1.

Examples of grids with 4-point scales.

When probed, Spanish-speaking respondents indicated a challenge in forcing their thought process to conform to the survey, and indicated difficulty in applying the statements to themselves, as they do not tend to think in the manner presented in the scale. For example, some Spanish-speaking participants answered these questions by a two-step process: First, they would determine whether their response would be yes or no (agree or disagree), and then think about which item from the scale to select (strongly vs. somewhat).

Additionally, Spanish-speaking respondents tended to select “strongly disagree” to convey their confusion when they were particularly challenged by a question. This was particularly true with respect to grids, which were consistently confusing to Spanish-speakers. Specifically, interviewees noted that it was unclear that response options in grids applied for each sub-item (row) of the question, and most Spanish-speaking respondents did not answer all items in a grid format. It is worth noting that although Spanish-speakers had an exceptionally difficult time with these questions, English-speakers struggled as well, and the strongly agree/strongly disagree question format was the most difficult for all respondents.

As a response to the challenges identified with the grid format, the survey was reformatted as a single column, and this appeared to be less complex to answer in subsequent tests of the question. However, despite the issues observed with respect to the multiple response options (that is, strongly agree/strongly disagree), those were not changed, and the observed challenges remained in subsequent iterations.

Content (Ir)Relevance

In addition to concerns about the survey formatting, the subject matter of the questions was problematic for Spanish-speakers. Spanish-speaking respondents were more likely than English-speakers to be unable to provide accurate and/or complete answers on the questionnaire. This difficulty appeared regardless of the respondents’ educational level. Some issues relate to the challenges of translating newer technologies: For example, “app” (a mobile application) was not translated but left in English, and was misunderstood by several Spanish-speaking respondents upon probing. There was no alternative proposed, and this word was left as-is. This example is particularly interesting in light of research that suggests that Latinos are eager adopters of mobile technologies (Lopez, Gonzalez-Barrera, & Patten, 2013).

Another issue for many respondents – English-speakers as well as Spanish-speakers – was difficulty in understanding the high-level vocabulary used in the questions; even after reading the questions twice, many of the respondents were unable to understand some of the questions. For example, one question sought to understand the extent to which respondents perceived conditions and diseases as a function of individual behaviors versus genetic makeup (the question was phrased differently and tested in several different formats but the results pertaining to misunderstanding of genetics were similar). Many Spanish-speakers responded to this question based on their own (personal or a family member’s) experience with each specific condition or disease; where they did not have direct experience, they left the question blank, assuming it was not relevant.

Importantly, Spanish-speaking respondents did not skip the questions they misunderstood but answered them despite not understanding. When probed about what they thought the question meant, respondents had incorrect understandings and confessed to having not understood the question but feeling that they had to answer anyway. Other research has shown that Latinos are more likely to feel pressured to respond to questions that they may not fully understand or perceive as relevant, in an effort to be helpful, part of a cultural script known as simpatía (Triandis et al., 1984).

Technology and interaction with providers

Some questions were not understood because they were technical medically or scientifically (e.g., the genetics example above), or because of language translation challenges (e.g., “app”). Of potentially more consequence for health communication scientists and practitioners, however, are those questions that were misunderstood because they assumed a familiarity with the medical establishment and high-tech communications technologies. For example, Spanish-speakers responded to questions about their level of comfort in sending information electronically to doctors (Fig. 2), but when prompted by the interviewer, revealed that they didn’t understand the point of such messages. English-speakers, on the other hand, revealed concern about information privacy and security of the messages. The questions were designed to get at the latter constructs. In their current form, however, they may be tapping into different concerns between English and Spanish-speakers. This is underscored by some Spanish-speaking participants’ confusion about the questions: For example, one participant understood “obtaining information electronically” as information that could be shared through email, and another noted that although he was unable to view his medical information online, his doctor stores the records. Another participant described his answer as a response to the practical implications of the question: He did not have access to the Internet or a computer, and therefore could not access information electronically.

Fig. 2.

Comfort communicating with provider via mobile phone or tablet.

Access to care

Some questions required a connection to the medical establishment that some Spanish-speakers in the cognitive interview samples did not have. Without health insurance or access to regular primary care, some questions do not make sense. For example, questions about preferences for medical decision-making or communicating with physicians presume regular contact with a health care provider.

Provider-patient communication and informed decision-making

Respondents also demonstrated confusion in answering questions designed to assess medical decision-making and provider-patient communication. There were two issues: comprehension and conceptual. With respect to comprehension, some of the challenge in understanding what is a medical decision was due to the content of the question and did not differ between the English and Spanish-speakers: Original versions of the questions asked (female participants only) about when they discussed their Pap test with their medical providers. Since this test begins when women are young, is conducted as part of a regular annual exam, and rarely requires follow up, there is not much need for discussion for most people. Thus, the question was confusing to participants: Should they answer based on what the doctor told them the first time they got the test, which may have been years ago, or on their results (which many do not receive if results are normal), or something else? Based on early cognitive testing with both English-and Spanish-speakers, the question was modified to ask about mammograms, and to be specific about the content of the conversation and decision-making context: “Has a doctor ever told you that you could choose whether or not to have a mammogram?”. This specific question eliminated the comprehension issues in subsequent cognitive testing; however, among some Spanish-speakers, comprehension and conceptual problems with decision-making remained for other questions. For example, one question, intended to be a screener that would lead to additional questions about the decision-making process, elicited confusion for several Spanish-speakers: “When was the last time you made a medical decision?”. Examples of interpretations by Spanish-speakers of “medical decision” included: the last time they had an operation; the decision about which day to come in for an appointment; and an order given by a doctor. Half of the Spanish-speaking respondents could not answer the question in the way it was intended by the survey designers because they believed medical decisions should be made only by doctors and followed by patients. No similar pattern was detected among English-speakers.

Participation in Research

Some Spanish-speaking Latino participants appeared to lack understanding about clinical research and clinical trials in particular. For example, one series of questions asked respondents to indicate their knowledge about clinical trials (translated as “ensayos clínicos”). All respondents answered this question apparently without problem; however, when probed, none of the Spanish-speaking respondents were able to correctly define “clinical trials”; for example, one respondent defined “clinical trials” as a way of testing vaccines on people (“it is an experiment” to see how they react). Others described their knowledge and preferences about “laboratory tests” when answering questions about clinical trials. This is not a simple translation issue, since other translations (“pruebas clínicas,” tested in some questions in Cycle 2 and “estudio clinico,” tested in Quinn and colleagues’ 2013 study) produced similar misunderstandings, but rather indicates the general lack of awareness about and access to medical research in this population (Quinn et al., 2013). Moreover, some Spanish-speaking participants continued to express incorrect understanding even after the interviewer attempted to explain clinical research. Other Spanish-speaking participants appeared to understand after explanation, but felt unqualified to participate in such research, although they may not have clearly understood. For example, one participant believed that people should let the experts do their job, indicating misunderstanding of the question’s intent.

Discussion

This study examined challenges in designing an instrument that is appropriate to a particular population – Spanish-speakers – for a national survey of health information needs and sources and described the differences in their response styles. Through a combination of strategic sampling decisions and reformatting of the survey instruments to address cognitive differences in survey response, we were able to improve Spanish-speaking participation in this survey. The findings from the cognitive interviewing study have significant implications for survey researchers, communication researchers, and health communication practitioners.

With regard to formatting concerns, we found that Spanish-speaking respondents demonstrated extreme response style, particularly with respect to 4-point strongly agree/strongly disagree response options. These findings are consistent with the published literature on Latinos and response styles (Cerda & Basar, 2010; Davis et al., 2011; Marin et al., 1992). Additionally, a novel finding from this study was that Spanish-speaking respondents tended to select “strongly disagree” to convey confusion. In terms of recommendations, an important challenge in survey question design is striking the right balance between providing response options that maximize variation but reflect meaningful distinctions; this is made more challenging in surveys that seek to include diverse populations when that diversity may affect the ways in which respondents answer questions.

This study identified cultural differences such as a concern for privacy that may also affect interpretation and perception of relevance for some items. Results suggest that some Spanish-speakers may be interpreting questions somewhat differently than English-speakers. The different interpretations suggest that questions about specific issues such as technology in health care may require taking emic perspective, wherein it is understood that the question will draw on culturally specific constructs, rather than an etic perspective that suggests that the constructs are universally understood in a common manner, which could create problems in interpreting data across groups that may have interpreted the question differently (Warnecke et al., 1997) .

Additionally, results suggest that questions about patient-clinician relationships, interactions with clinical infrastructure, and technology preferences are most relevant to the population that is integrated within the health care system. Although integration within the health care system is not a cultural factor but rather reflects socioeconomic position, the implications for comprehension of survey questions and the potential for data analysts to fall prey to the category fallacy – drawing conclusions from the data based on an assumption that respondents had similar understanding of the questions – is similar to the culturally-based distinctions described above. For example, questions about preferences for medical decision-making or communicating with physicians are not as relevant to those who do not have a primary care provider or otherwise have regular access to health care, as is common among many Latinos in the general population (Hakimzadeh & Cohn, 2007). Obtaining valid, comparable responses to such questions requires either more precise targeting of respondents who may be eligible, or additional clarification to set up the questions (i.e., extended definitions or complex skip patterns). In developing future surveys about patient-provider communication, or to inform the development of health communication campaigns, improving response rates and data quality for Spanish-speakers may require tailoring questions to fit their integration within the health care system. This may also be true for other low-income, uninsured populations, and future research is warranted to further examine cognitive processing differences for these types of questions.

Limitations

We acknowledge that these results are subject to several limitations. Demographic differences in the English-and Spanish-speaking samples precluded our ability to disentangle language barriers from education, income and lived experience (i.e., the relevance of specific survey questions) (Gibbons, Zellner, & Rudek, 1999). As such, the challenges to complete, correct survey item response identified in this study may also be related to literacy, education, and health literacy. For example, in Cycle 2, 8% of English-speakers had less than a high school education. In contrast, 38% of Spanish-speakers did.

These results are based on a small number of qualitative interviews with Spanish-speaking volunteers drawn from suburban Washington, DC. Further research, with a larger number of Spanish-speakers from a variety of U.S. geographies, is needed to better understand the measurement issues associated with this population.

Conclusion

Challenges in improving response rates and quality in written surveys with Spanish-speakers are similar for understanding communication inequalities and effects of health communication. Health communicators also face the issues of education, literacy, and translation when they think about how best to communicate important information to Latinos. This study thus suggests that survey researchers, just as communicators, consider investing in design and outreach separately, because responses from English-speakers only may be insufficient to generalize to a U.S. population that is increasingly linguistically diverse (Colby & Ortman, 2015).

Furthermore, challenges remain in understanding the health information needs of non-English-speakers. There seem to be more interpretation problems by Spanish-speakers, and this is partly related to asking about health care communication issues: By design, HINTS asked questions about cutting-edge health communication technologies, but these may not be relevant for the U.S. non-English-speaking population, which is also less-educated and lower-income than the English-speaking population. It may be worth reconsidering the need for nationally representative data on emerging technologies; another solution may be improving skip patterns using non-threatening questions to assess familiarity with certain technologies prior to asking more detailed questions. However, skip patterns must be used with caution in mail surveys, since they are hard to follow during self-administration. Mixed-mode administration (that is, interviewer-and self-administered, as with previous iterations of HINTS) allows for more complex skip patterns than a purely self-administered collection approach.

Finally, this study suggests that future research to understand the health information needs, sources, and access to health information in the U.S. non-English-speaking population could benefit from a mixed-methods approach. Mixed-methods approaches to research – combining qualitative and quantitative methods to answer a set of research questions – allows for the limitations of each approach to be addressed by the other (Curry & Nunez-Smith, 2015). The findings reported in this manuscript suggest that beginning with a qualitative approach to understand health information needs, sources, and access may be useful for developing relevant questions and question formats to use in closed-ended surveys to collect nationally representative data.