Customizing Survey Instruments and Data Collection to Reach Hispanic/Latino Adults in Border Communities in Texas

Abstract

Objectives. We sought to modify an instrument and to use it to collect information on smoking knowledge, attitudes, beliefs, and behaviors among Hispanics/Latinos, and to adapt survey methods to obtain high participation levels.

Methods. Promotoras (outreach workers) conducted face-to-face interviews with 1485 Hispanic adults (July 2007–April 2008). The project team used GeoFrame field enumeration methods to develop a sampling frame from households in randomly selected colonias (residential areas along the Texas–Mexico border that may lack some basic necessities (e.g. portable water), in El Paso, Texas.

Results. The revised questionnaire included 36 unchanged items from the State Adult Tobacco Survey, 7 modified items, and 17 new items focusing on possible culturally specific quitting methods, secondhand smoke issues, and attitudes and knowledge about tobacco use that might be unique for Hispanic/Latino groups. The eligibility rate was 90.2%, and the conservative combined completed screener and interview response rate was 80.0%.

Conclusions. Strategic, targeted, carefully designed methods and surveys can achieve high reach and response rates in hard-to-reach populations. Similar procedures could be used to obtain cooperation of groups who may not be accessible with traditional methods.

The size of the Hispanic population in the United States has changed dramatically over the past 35 years. In 1970, 9.6 million Hispanic adults and children resided in the United States (4.7% of the total population); by July 2006, the number had more than quadrupled, reaching 44.3 million Hispanic adults and children, 14.8% of the total adult population.¹ The growth rate of the Hispanic population was more than 3 times the growth rate of the entire US population between 2000 and 2006. In 2004, the Census Bureau projected that the number of Hispanics living in the United States will reach 102.6 million by the year 2050 and will likely compose about 24% of the population.²

There is reason to believe that this population may not be readily accessible for data collection. Documentation of some of the potential barriers to participation can be found in ethnographic research carried out during the 2000 Census.³ Ethnographic studies in colonias (residential areas along the Texas–Mexico border that may lack some basic necessities), one of which was in Texas, documented barriers to census enumeration: irregular housing, little or no knowledge of English, limited formal education, concerns regarding confidentiality, and complex and fluid households. Translations, enumeration strategies, advertisements, and reassurance about confidentiality were employed, and resulting evidence reveals success in addressing the concerns. The importance of community collaboration for improving participation rates has been documented.^4,5

One health-related behavior of interest for all populations is tobacco use. Hispanic and Latino populations in the United States face unique challenges that may put them at higher risk for tobacco use and for exposure to secondhand smoke for several reasons: (1) racial/ethnic communities continue to be targeted by tobacco industry sponsorship efforts, including Mexican rodeos, Cinco de Mayo (May 5th) festivities, and activities associated with Hispanic heritage month⁶; (2) more Hispanic and Latino workers may be exposed to secondhand smoke in the workplace than are other groups, despite the fact that there is a high percentage of smoke-free homes and strong support for smoke-free policies in workplaces among Hispanics and Latinos⁷; (3) preliminary findings suggest that as Hispanic and Latino women become more acculturated into mainstream US society their smoking rates may increase, a pattern that does not seem to be occurring among men, where the rate is already elevated⁸; and (4) overall prevalence rates of current smoking (smoking every day or some days) among Hispanic adults remains low at 13.3%⁹ and among Hispanic high-school students (smoking at least once in the past 30 days) is 16.7%.¹⁰ However, there may be marked differences in prevalence rates among subgroups of Hispanics and Latinos. For example, a recent analysis of 2002–2005 data from the National Survey on Drug Use and Health revealed that for Mexican respondents, the prevalence of past-30-day cigarette smoking was 23.8%, whereas for those self-identified as Puerto Rican the prevalence was 31.5%.¹¹

Some research has revealed that the attitudes, behaviors, knowledge, and experience of Hispanic and Latino persons residing in the United States regarding tobacco use may differ from those of persons in non-Hispanic groups, which may warrant customized approaches to smoking prevention and cessation programs.^12,13 Although Hispanic young adults acknowledge the adverse health effects associated with tobacco use, smoking among Hispanic youths continues to be a social activity. In Hispanic cultures, smoking among females is actively discouraged. Young adults also appear to be unwilling to access resources for cessation partly because family and family relationships are considered to be an integral part of smoking prevention and cessation. Targeted, sustained interventions in the Spanish language may also be limited, in part because of the perception that tobacco use is not a problem among Hispanic and Latino populations.¹⁴

A strategy that may help identify and address tobacco-related issues among Hispanic populations in the United States is the collection of specific data measuring knowledge, attitudes, beliefs, and behaviors that is obtained in culturally sensitive and relevant ways. To this end, M. O., L. L. P., S. L. T., and R. S. C., who were affiliated with the Centers for Disease Control and Prevention's (CDC's) Office on Smoking and Health, examined available survey methods, tobacco-related instruments, and their utility for obtaining information from Hispanic populations. An additional objective was to achieve acceptable levels of participation and response rates from a population that does not always respond to standard telephone or mail survey methods.³ There were 2 phases to the project; the first focused on designing a Hispanic/Latino Adult Tobacco Survey (Hispanic/Latino ATS) to collect information from this population. The second phase of the project involved developing data collection methods and sampling techniques that would reach the target population (Hispanic and Latino adults living in the United States).

METHODS

In April 2002, a meeting on Effective Tobacco Control in Hispanic/Latino communities was held by the CDC and was attended by state tobacco control partners and CDC staff. (State participants were selected on the basis of their work with specific Hispanic and Latino populations in the areas of surveillance and evaluation, program, policy, communication, and community advocacy.) The discussion covered a broad range of topics, including suggestions of how to categorize different population subgroups, protective factors (against tobacco use) that could be operating in Hispanic and Latino populations, the cultural context for cigarette use and use of noncigarette tobacco products such as smokeless tobacco, as well as the role of acculturation in the health behaviors of this population. There was also discussion of the methods used for data collection and sampling. The conclusions from the meeting provided the basis for revisions made to the State Adult Tobacco Survey (State ATS) to create the Hispanic/Latino ATS.

Hispanic/Latino Adult Tobacco Survey

The modifications to the State ATS were made keeping in mind the monitoring and evaluation needs of individual states and comparability across states. The modifications were designed to incorporate measures and language to capture attitudes, behaviors, knowledge, and experience of these specific subpopulations and to be sensitive to the varied and unique cultural and social experiences of Hispanic and Latino persons. Since July 2009, 25 states have completed at least 1 State ATS supported by the Office on Smoking and Health at CDC (S. Thorne; Health Scientist, Office on Smoking and Health, CDC; written communication; August 2009). The objective was to prepare a battery of questions that measured Hispanic and Latino smokers' and nonsmokers' (1) smoking behavior and experiences and (2) knowledge, attitudes, beliefs, and behaviors concerning smoking.

Five of the 10 participants in the surveillance and evaluation group of the 2002 expert advisory panel were selected to participate in the revision of the instrument on the basis of their expertise in tobacco control and their research experience with Hispanic and Latino subpopulations (e.g., Mexican American, Puerto Rican, Cuban, Central American, and South American). Following the revisions, 2 teams of professional translators translated and cognitively tested the instrument in English and Spanish.^15–18 They reconciled discrepancies and reached agreement on a translation that best achieved equivalence of measurement between English and Spanish and between different varieties and dialects of Spanish. In 2004 and 2005, the questionnaire was cognitively tested in Chicago, Illinois; Los Angeles, California; Miami, Florida; New York, New York; San Antonio, Texas; El Paso, Texas; and Washington, DC.

Sixty-eight interviews were conducted with Hispanic and Latino respondents (16 in English and 52 in Spanish).¹⁹ Several subpopulations were included in the cognitive testing of the questionnaire (e.g., respondents from Mexico, Puerto Rico, Cuba, Guatemala, El Salvador, Columbia, Dominican Republic, Peru, Ecuador, and Honduras).

Sample Design and Data Collection

From July 2007 to April 2008, the Hispanic/Latino ATS was administered in the southeast colonias in El Paso. (Research Triangle Institute International was contracted by the CDC to complete the field portion of these interviews. Research Triangle Institute International contracted with Texas A&M University to complete the data collection for the study.) According to the Texas Secretary of State Web site, a colonia is “a residential area along the Texas–Mexico border that may lack some of the most basic living necessities, such as potable water and sewer systems, electricity, paved roads, and safe and sanitary housing.”^20,21 Over the past 15 years, Texas has adopted several laws to address infrastructure problems associated with colonias, thereby ensuring water and sewer services and limiting utility connections in substandard developments.²² The sample frame of colonias was grouped into 2 categories (green and not-green) depending on the level of public services available. “Green” colonias received the highest level of services, and “not-green” colonias were those with lower or unknown levels of services.

Because of the experiences in the 2000 US Census and their past experience in working with residents of the colonias, individuals from the Texas A&M University Colonias Program administered the survey. The interviewers met the program's criteria for promotoras (outreach workers who assist in the development of community programs in the colonias). There were 9 female interviewers and 1 male interviewer, who ranged in age from early 20s to late 40s. An intense 3-day training program was conducted in Spanish and covered the following topics: background of the study, interviewer responsibilities, interviewing skills and techniques, locating and sampling participants, and questionnaire administration. Role playing was used to ensure that the interviewers were proficient in the required skills.

Mailing lists could not be used as the sampling frame because post office boxes were widely used in the area. Post office boxes are not suitable for in-person surveys because they are not locatable on the ground. Instead, the project team used the GeoFrame (RTI International, Research Triangle Park, NC) field enumeration process; this involved the use of digital photography and geospatial technology to produce a low-cost household enumeration of all of the colonias.^23–25 With this method, the project enumeration team photographed each dwelling unit with a digital camera, and identified global positioning system coordinates.²⁶ As a result, the project team created the sampling frame from a complete enumeration of the 8990 apparent dwelling units in the defined area, and dwelling units were randomly selected from this enumeration. The project team provided the interviewers with the geospatial information and a photograph of the selected unit.

For the purposes of this survey, the project team defined a dwelling unit as a single structure containing 1 or more rooms or a trailer, converted bus, or other domicile where people could sleep. It was not unusual in the colonias to find multiple dwelling units on a single property that housed various branches of a single family or to find more than 1 household occupying a single dwelling unit. A screening interview was used to determine the number of Hispanic adults (aged 18 years or older) who lived in the household and to select the adult with the latest birthday as the respondent.²⁷ If no one answered the door, a note in English and Spanish describing the study, the purpose of the visit, and a telephone number for scheduling an appointment was left.

RESULTS

The survey is made up of 6 core sections that include questions about cigarette smoking, cessation, secondhand smoke, risk perception, and social influences, demographic items, and supplemental questions to assess specific program needs. Optional modules can be added to the core to obtain additional information on use of noncigarette tobacco products, cessation, secondhand smoke exposure, health and social influences, policy issues, parental involvement, media exposure, and additional demographic items. The revised questionnaire included 36 unchanged items from the State ATS, 7 items modified with word choice changes or ordering, and 17 new items that focused on possible culturally specific methods used for quitting (e.g., medicine man, spiritist, or religious leader), secondhand smoke issues, and attitudes and knowledge about tobacco use that might be unique for Hispanic and Latino groups. (The resulting instruments in English and Spanish are available at http://www.cdc.gov/tobacco/data_statistics/surveys/hispanic_latino_ats_guide.)

The Sample

The project team selected the sample in 2 stages (see Table 1); the first stage consisted of a random sample of 2100 apparent dwelling units in each of the 2 strata (green or not-green) of the colonias. The sample of units was distributed as evenly as possible between the 2 strata of green or not-green colonias. The project team based eligibility on the selected unit being a dwelling unit, and self-report of Hispanic or Latino residents, with at least 1 resident who was aged 18 years or older. During data collection, the interviewers determined that 203 units of the selected sample were ineligible because they were not dwelling units, thus leaving 1897 units. Interviewers were able to screen household members in 1713 of these. In the second stage, the interviewers selected 1 adult from each of the eligible dwelling units with the “last birthday”method.²⁷ There was no cap placed on the number of visits to a specific household. Instead, in the daily review of eligible households, the interviewers and project team identified the households that were difficult to contact in-person and attempts were made at different times of day and different days of the week. After 15 in-person visits, the data collection manager attempted to make phone contact for 10 households where a phone number was obtained.

TABLE 1.

Distribution of the Sample: Colonias in El Paso, Texas, 2007–2008

	Stage of Sampling
First stage: enumerated dwelling units (DUs)
Sampling frame	All DUs enumerated in target population of colonias
Stratification	Green and not-green colonias
Type of selection	Systematic samples
Sample sizes	Selected sample: 2100 DUs; Assumed eligiblea: 1897 DUs; Screened eligible: 1713 DUs
Second stage: eligible persons
Sampling frame	Roster of eligible persons
Stratification	None
Type of selection	Last birthday method
Sample sizes	Selected sample: 1713 eligible persons (1 per address) Participants: 1485 respondents

^aWe confirmed 203 dwelling units were not eligible because of vacancy, being a business, or being demolished. Of the remaining 1897 cases, we were unable to contact a household member for 184 of them. The interviewers visited each of these 184 dwelling units. Because they could not confirm they were not dwelling units, we assumed they were eligible dwelling units.

Data Collection

The survey was conducted from July 2007 through April 2008; Office of Management and Budget clearance was obtained by CDC. Both Research Triangle Institute International and Texas A&M University's institutional review boards approved the study. The screening interview took an average of 5 minutes to complete; the main interview took an average of 21 minutes (ranging from 8 to 60 minutes). In 29 cases, when face-to-face interviews could not be scheduled, the interview was conducted by phone. Participants who completed the main interview received a $15 Wal-Mart gift card for their time and participation. The project team maintained quality control in the field by (1) having the study coordinator review all questionnaires so that immediate feedback could be given to the interviewers; (2) having the study coordinator visit 5% to 9% of each team's cases to verify that the interview had occurred and to collect information on the team's professionalism (there were no instances of false reporting by the teams); and (3) having the data collection manager periodically visit to observe interviews and monitor procedures.

Response Rates

Table 2 displays the final status of the 2100 cases. The calculation of the response rate with the method of the American Association for Public Opinion Research can be found in Table 3. American Association for Public Opinion Research response rate 2²⁸ provides a conservative calculation, because it assumes all of the unknown cases were eligible households. The eligibility rate was 90.2%, the conservative combined screener and interview response rate was 80.0%, and the hit rate (the percentage of all cases released that resulted in a completed interview) was 70.7%. Costs, which included enumeration of dwelling units, random selection of the units and the respondent, completion of the interview, and entry of the responses, were estimated to be less than $300 per completed interview. Cost information for instrument development was not available.

TABLE 2.

Disposition of the Sample: Colonias in El Paso, Texas, 2007–2008

Description	No. of Cases
Final screener codes
Total	2100
Completed screener	1713
Unknown eligibility—no contacts, unable to locate	184
Not eligible—vacant, business, demolished units	203
Final interview codes
Total	1713
Completed interviews	1485
Eligible, noninterview—refusal, unable to contact	187
Not eligible—not Hispanic, younger than 18 years, incapable	41

TABLE 3.

Response Rates for the Hispanic/Latino Adult Tobacco Survey in Colonias in El Paso, Texas, 2007–2008

	Response Rate 4a		Response Rate 2b
Rate	Calculation	Value	Calculation	Value
Eligibility rate	Percentage of cases released that are eligible	90.2%	Percentage of cases released that are eligible	90.2%
	(I+P+e[UH+UO]) / (I+P+R+NC+O+e[UH+UO] + NE)		(I+P+e[UH+UO]) / (I+P+R+NC+O+e[UH+UO] + NE)
Screener response rate	Percentage of eligible households that completed a screener	91.3%	Percentage of eligible households that completed a screener	90.0%
	(I+P) / (I+P+R+NC+O+e[UH+UO])		(I+P) / (I+P+R+NC+O+UH+UO)
Interview response rate	Percentage of eligible respondents that completed an interview	88.8%	Percentage of eligible respondents that completed an interview	88.8%
	(I+P) / (I+P+R+NC+O+e[UH+UO])		(I+P) / (I+P+R+NC+O+UH+UO)
Combined screener and interview response rate	(Screener response rate) × (interview response rate)	81.1%	(Screener response rate) × (interview response rate)	80.0%
Hit rate	Percentage of all cases released that resulted in a completed interview (I+P) / (I+P+R+NC+O+UH+UO+NE)	70.7%	Percentage of all cases released that resulted in a completed interview (I+P) / (I+P+R+NC+O+UH+UO+NE)	70.7%

Notes. I = no. of completed interviews; P = no. of partial interviews (this is included to match the American Association for Public Opinion Research definition; we had no partial interviews in the Hispanic/Latino Adult Tobacco Survey); R = no. of refusals and break-offs; NC = no. of noncontacts; O = no. of others; e = estimated proportion of cases of unknown eligibility that are eligible (for the screener response rate calculation, e = (I+P) / (I+P+NE); for the interview response rate calculation, e = 1); UH = no. of unknown if occupied dwelling units; UO = no. of unknown, other; NE = no. not eligible.

^aAmerican Association for Public Opinion Research response rate 4 is used to estimate how many of the cases with unknown status were likely to have been eligible. The equation is: (I+P) / (I+P+R+NC+O+e[UH+UO]).

^bAmerican Association for Public Opinion Research response rate 2 is the most conservative response rate calculation, in that it assumes that all of the unknown cases were eligible. The equation is: (I+P) / (I+P+R+NC+O+UH+UO).

DISCUSSION

The results support the conclusion that culturally sensitive modifications to survey procedures used to locate and contact specific population groups can result in response rates that far exceed those common in survey work today.^26,29,30 The 80% combined screener and interview response rate in the current survey is not typical of response rates in other survey work. For example, the 2007 Behavioral Risk Factor Surveillance System, which used telephone and not personal interviews, reported response rates from 26.90% to 65.36%.³¹ Direct comparisons of the response rates and information obtained with these methods need to be made with caution because of the many differences in methodologies.

Developing instruments for specific population subgroups requires consideration of culture and language, cognitive demands, and potential response errors.³² Collecting information from specific subpopulations requires community knowledge and specialized training. The interviewers were trained to collect the information in a manner that was culturally sensitive and appropriate. Some of these interview methods focused on, but were not limited to, conducting the survey in Spanish, cognizance of demeanor during the interview, tone, and probing techniques. There was a considerable amount of monitoring of the methods and interaction with interviewers by the project staff from the Texas A&M Colonias Program. This approach is ideal for collecting information from specific subpopulation groups. What has been demonstrated is that a strategic, targeted, carefully designed survey can result in high response rates. Whether this rate can be achieved in other locations is not known until the findings from the present study are replicated.

The solutions to the numerous data collection challenges encountered on the Hispanic/Latino ATS may be applicable to surveys of other population subgroups. GeoFrame was developed to assist in the creation of an area probability sample in which there was little or no landline telephone coverage and a lack of postal listings. A question may arise as to whether it is a cost-effective and timely solution to create a sample similar to the one used in this study. For many areas in the United States that do not have postal listings or landline telephone coverage, such an approach may be considered. Responses to this question may vary depending on population size and density. The utility of the approach is not limited to the United States; it may be useful in international surveys where there are not already existing sampling frames or may extend to surveys conducted in the aftermath of war or natural disasters, when up-to-date residential listings are unavailable.

To overcome barriers to clear and concise communication, a great deal of preliminary work was done in the development and modification of the questionnaire. Numerous field tests of the survey were conducted, along with input from stakeholders and target population, before the actual implementation of the survey. Although this may not be possible for all surveys, at a minimum close attention should be paid to the development and translation of the questionnaire, taking into account cultural considerations as well as verbal or written comprehension by the target population.

When we began preparations for this survey, we assumed that telephone interviews were not appropriate for the target population. We designed an interview scenario in which respondents would feel comfortable by hiring native Spanish–speaking interviewers who were familiar with the interview area. In addition, training methods were used to allow interviewers to gain experience in interviewing procedures that were used in collecting survey data. Finally, measures were taken to assist interviewers in overcoming potential objections by respondents. Involvement of the Texas A&M Colonias Program proved vital to the success of the study, as they had insight and understanding of the colonias and their residents that proved to be essential in ensuring cooperation. The use of uniforms with logos from the Texas A&M and Texas A&M Colonias Program and other Colonias Program materials may have helped to ease concerns for respondents about strangers in their community.

Strengths and Limitations

There are several important strengths in this study; first is the use of a survey that was designed to be culturally appropriate for Hispanic and Latino persons; second, the use of promotoras who were Spanish-speaking and familiar with the interview area; and, third, the use of GeoFrame to create a probability sample in which traditional methods could not be used because of inconsistent telephone coverage and postal listings. The approach to sampling can be applied in other geographic areas where sampling frames are not readily available such as in geographic regions that have been devastated by hurricanes or in isolated rural regions in countries after other catastrophic events such as war.

There were also some limitations: the use of personal interviews may have resulted in interviewer bias if respondents were uncomfortable communicating sensitive personal information. However, the opportunity for establishing rapport that exists in personal interviews³³ may have added to the relatively high participation rates, and the opportunity to incorporate probes and clarification of items can help to add to more complete responding. In addition, because of the focus of the sampling, it is not possible to determine how generalizable the findings are to other adult Hispanic or Latino persons who reside in areas other than the selected colonias.

Conclusions

The results of this survey will be used by the Texas State Health Department to improve and develop programs for the residents of the colonias. In addition, the methods developed may be useful in the evaluation of interventions. For example, Sabemos: Por respecto—Aquí no se fuma (“Now that we know: out of respect, do not smoke here”) is a community outreach toolkit distributed by the CDC for parenting tobacco-free Hispanic and Latino youths (aged 14 years and younger).³⁴ The Hispanic/Latino ATS methodology could be used to evaluate the Sabemos toolkit in individual communities by administering the survey before and after implementation so as to assess changes in tobacco use knowledge, attitudes, and behavior.

In summary, this work was carried out to tailor a survey to gather population-specific information and increase survey participation from hard-to-reach populations. It is not possible to determine which components of the survey modifications and methods used to collect the data had the greatest impact on response rates in this study, nor whether these efforts could have worked as well in isolation from each other. However, the comprehensive approach used here provided response rates that exceeded those occurring in other surveys of this population and may be useful in other hard-to-reach communities.