Developing a Culturally-Informed Survey Instrument to Assess Biomedical Research Participation among Latinos on the U.S.-Mexico Border

Abstract

Racial/ethnic minorities, rural populations, and those with low socioeconomic status income are underrepresented in research. Assessing preferences for recruitment, participation, and the role of beliefs about biomedical research in specific and unique underserved communities represents a potentially critical step in reducing barriers to biomedical research participation. We developed culturally-informed survey to measure factors related to participation, knowledge, expectations, and barriers to biomedical research participation among Latinos living in a United States (U.S.) border community. We employed a multi-disciplinary team approach to a three-phase qualitative study that included interviews (n=35), focus group (n=24) s, and “think-aloud” cognitive interviews (n=5). Our study demonstrates the value of multiple qualitative approaches to inform a culturally-relevant quantitative survey incorporating words and constructs relevant to the population of interest. The study contributes to qualitative method research paradigms by developing a research protocol that integrates the expertise and perspectives of researchers and community members from multiple disciplines and institutions.

Background

Underserved groups, including racial/ethnic minorities, rural populations, and those with limited education and low household income, are underrepresented in biomedical research (Ford et al., 2008; Murthy, Krumholz, & Gross, 2004; Sam S Oh et al., 2015; Sateren et al., 2002). These groups also suffer disproportionately from chronic and infectious diseases such as diabetes, cardiovascular disease, cancer, and HIV (Chow, Foster, Gonzalez, & McIver, 2012; Montoya & Whitsett, 2003; Roger et al., 2011; Smedley & Haynes, 1999; Smedley, Stith, & Nelson, 2003). Projections estimate sharp increases in incidence of these diseases, with increasing disparities experienced by racial and ethnic minorities, (Boyle, Thompson, Gregg, Barker, & Williamson, 2010; Rowley & Bezold, 2012; Smith, Smith, Hurria, Hortobagyi, & Buchholz, 2009; Yanovski & Yanovski, 2011) who are expected to comprise the majority of the U.S. population by 2040 (Ortman & Guarneri, 2009).

Evidence generated from biomedical research can inform policies and practice that address both the social determinants of health as well as health care access and quality. Agencies such as the National Institutes of Health (NIH) have attempted to bolster recruitment through the NIH Revitalization Act of 1993, which mandates inclusion of minorities in NIH-funded research (Chen, Lara, Dang, Paterniti, & Kelly, 2014). However, minority participation in biomedical research remains low, and little progress toward improving this underrepresentation in research has occurred (Chen et al., 2014; Sam S Oh et al., 2015). African Americans and Latinos, for example, comprise 30% of the U.S. population but make up only 6% of research participants in clinical and biomedical research (S. S. Oh et al., 2015).

Noting Latinos are the largest and fastest growing minority population in the United States (Ortman & Guarneri, 2009; Passel, Cohn, & Lopez, 2011), such underrepresentation has significant implications in terms of prevention, treatment, and post-treatment care of acute and chronic illness. Extensive qualitative research, has revealed factors that may inhibit participation—such as fear, mistrust of scientists and the government, and lack of understanding—or promote participation—such as such as altruism, race or cultural concordance between researcher and participants, and perception of the importance of the topic—in biomedical research among racial/ethnic minorities (Bakken, Lantigua, Busacca, & Bigger, 2009; Brewer et al., 2014; Calderón et al., 2006; Corbie-Smith, Thomas, Williams, & Moody-Ayers, 1999; Friedman, Foster, Bergeron, Tanner, & Kim, 2015; Fryer et al., 2015; Hohl, Gonzalez, Carosso, Ibarra, & Thompson, 2014; van der Velde, Williamson, & Ogilvie, 2009). Katz and colleagues developed a quantitative survey for general use among racial/ethnic minorities (Katz et al., 2006). However, no quantitative survey instrument that we are aware of has been culturally tailored to assess preferences for recruitment, participation, and the role of beliefs about biomedical research in specific and unique underserved communities. Quantifying these factors for distinct underrepresented populations represents a potentially critical step in reducing barriers to biomedical research participation.

The goal of this project was to develop a comprehensive set of survey items for a culturally-informed survey to measure factors related to participation, knowledge, expectations, and barriers to biomedical research participation among Latinos living in a United States (U.S.) border community. In this article, we describe our multi-disciplinary team approach to a three-phase qualitative study that included one-on-one interviews, focus groups, and “think-aloud” cognitive interviews to develop survey items. We also discuss lessons learned to inform future efforts to develop measures for use with other unique underserved populations.

Methods.

Study Setting

This study took place in a southeastern New Mexico border community in the Paso del Norte region. The U.S-Mexico border region hosts a large, diverse Latino population, including Spanish and English speakers; first, second, third and fourth generation immigrants; and families with Mexican, Central, and South American ancestry (Housing Assistance Council, 2013). The region contains many colonias, “U.S.-Mexico border communities with marginal conditions of housing and infrastructure” (U.S. Department of Housing and Urban Development, 2015). Colonia residents are prime examples of medically underserved individuals under-represented in the field of biomedical research, thus contributing to the growing health disparities in the U.S.

Rural Latino immigrant communities are increasingly being sought as venues for biomedical research (Hohl et al., 2014; James, Yu, Henrikson, Bowen, & Fullerton, 2008). In New Mexico, border communities such as this study site tend to be rural and largely Latino with high poverty rates and low educational achievement levels (Ennis, Rios-Vargas, & Albert, 2011; U.S. Department of Health and Human Services, 1999), factors that characterize both high burden of disease and low rates of participation in biomedical research.

Study Design and Rationale

In recent years, researchers and groups such as the National Cancer Institute (NCI) and the NIH have prioritized the development of innovative methods to address complex research questions in public health (Creswell, Klassen, Plano Clark, & Smith, 2011; National Institutes of Health, 2001; Plano Clark, 2010). Studies that involve multiple qualitative approaches offer opportunities for complementary techniques to address research questions that may have cultural relevance and potentially impact health disparities (Ivankova, Creswell, & Stick, 2006; Morse, 2010).

The purpose of this project was to develop items for a quantitative survey tool that could comprehensively assess the beliefs, preferences, and willingness of Latinos to participate in biomedical research. The project followed a qualitative sequential design of three phases that occurred from March to June 2012 (Table 1). A qualitative approach was employed in order to develop survey items that reflected the language, values, and understanding of the Latino border population (J. Corbin & A. Strauss, 2007). Furthermore, the flexibility of multiple qualitative methods enabled us to comprehensively address the complexity of our research problem (Morse, 1999). In Phase 1, face-to-face elicitation interviews enabled us to ascertain this population’s unique understanding and attitudes as well as preferences for recruitment and participation in biomedical research. By gathering in-depth one-on-one perspectives, we aimed to determine constructs that are important and unique to this population. In Phase 2, focus groups served to confirm emergent themes and clarify ambiguities revealed in elicitation interviews. Together, the interviews and focus groups allowed for investigation of concepts previously identified in the literature and the particular cultural and social constructs that influence border population participation in biomedical research. Data from interviews and focus groups informed the development of a draft quantitative survey instrument. In Phase 3, the use of “think-aloud” cognitive interviews added depth to interview and focus group data by assessing face validity and clarity of survey items. We applied findings from the cognitive interviews to refine and finalize the survey instrument. The Institutional Review Boards of the Fred Hutchinson Cancer Research Center and New Mexico State University approved all study procedures.

Table 1.

Study Phases and Purposes

Phase	Purpose	Method	n
1	Ascertain understanding, attitudes, involvement, and interest in biomedical research Explore preferences for recruitment and participation	Face-to-face elicitation interviews (March 2012)	35
2	Confirm emergent themes from elicitation interviews Clarify ambiguities revealed in elicitation interviews	Focus groups (March-April 2012)	24
3	Assess face validity and clarity of survey items	Think-aloud cognitive interviews (June 2012)	5

Project Team

A comprehensive approach to developing a survey of this nature required team-based work, multiple sets of expertise, and consistent attention to the connections between varying aspects of the priorities of community members and researchers. The collaboration involved a research university, an NCI-designated comprehensive cancer center, and a community health clinic. Our team consisted of cross-institutional collaborators: two junior faculty (an anthropologist and health disparities researcher), a medical doctor, a senior behavioral health researcher, and two promotores (community health workers). Promotores are bilingual/bicultural Latino community members who are recognized members of local health care teams (U.S. Congress, 2009; U.S. Department of Health and Human Services, 2007). The promotores who contributed to this project were well-integrated members of the local community through their work as health advocates employed at a community health clinic, and had worked with members of the study team on previous projects. Promotores have a familiarity with community members often unattainable by researchers or medical personnel (Office of Minority Health, 2015). Promotores assisted with recruitment, conducted all one-on-one interviews, and were present during focus groups.

Recruitment

Participants were recruited for elicitation interviews, focus groups, and cognitive interviews via tear-off flyers posted in frequently visited community locations (e.g., grocery stores, laundromats, gas stations, local churches), at community health fairs attended by the promotores, and through invitations from promotores at the local community health center. Interested participants were instructed to call the research office where bilingual/bicultural staff explained study procedures and goals, answered questions and screened potential participants for eligibility. Participants were eligible if they identified as Latino, were Spanish-speaking residents of the Paso del Norte region and were at least 18 years old. Promotores obtained written informed consent of all participants at each phase of the study.

Phase 1: Elicitation interviews

To inform the survey design by eliciting perspectives of the community members where the instrument would be administered, the team collected qualitative data through face-to-face, one-on-one elicitation interviews. The research team developed a 17-question semi-structured interview guide to elicit knowledge of biomedical research, perspectives about participating in biomedical research, and cultural constructs that may facilitate or inhibit participation. Questions were based on a literature review of common barriers to participation among minority groups, particularly Latinos, those with low socio-economic status and those living in rural areas. Questions were designed to ascertain participants’ knowledge, attitudes, involvement, and interest in biomedical research and preferences for recruitment and participation. The senior researchers trained promotores in qualitative interviewing techniques, including role plays in which promotores practiced asking follow-up questions and probes to elicit participant perspectives about all aspects of the topics under investigation (Bernard, 2011; J. Corbin & A. L. Strauss, 2007). Promotores conducted 35 elicitation interviews (28 women and 7 men) through a purposive sampling design. Interviews were audio recorded and varied in duration from 25–60 minutes.

Elicitation interview data were transcribed and translated. The data were then analyzed in both English and Spanish using a modified grounded theory perspective, both inductively allowing categories of analysis to emerge from data and identifying other categories based on the interview questions (Glaser & Strauss, 2009). The team consulted regularly to refine codes and agree upon emergent themes.

Phase 2: Focus groups

To verify the accuracy of common responses in the elicitation interviews and to gather additional data in areas needing further investigation, the study team conducted three focus groups (n=24). Focus groups were conducted in Spanish and the majority of participants were women (n=20). The focus groups allowed researchers to confirm salient barriers and facilitators to participation in biomedical research that were especially pertinent for this group, such as understanding of informed consent and understanding of the term “biomedical research.”

In addition to repeating questions from the elicitation interviews, the team added questions to further explore ambiguities in those interviews. For example, it was unclear whether participants recognized the difference between providing biological samples for clinical lab work and providing biological samples for biomedical research. We asked focus group participants: “Is participating in biomedical research different than providing a sample to your doctor so that it can be examined in a laboratory?” If anyone in the group answered in the affirmative, we asked how it was different. Further, although most participants in the interviews did not describe potential harms they believed could come of participation in biomedical research, several participants hinted at possible negative consequences. To better understand participants’ thoughts about these potential consequences, we added the question: “Have you heard or known about someone who had a bad experience due to participating in biomedical research? If yes, can you tell us about it?”

Additionally, we restructured some focus group questions in successive groups after discussion with the research team. We determined that the language used in some of the questions framed the focus group as a test of knowledge rather than an open discussion of participants’ knowledge and experience. For example, in the first focus group, we asked “What does biomedical research mean?” In subsequent focus groups, we instead asked, “What do you understand about the term ‘biomedical research’?” The latter phrasing elicited more extensive discussions of participants’ experiences and perspectives and appeared to reduce social desirability.

The focus group data for the first two focus groups were transcribed and translated. The data for the third focus group remained in audio format. Transcriptions and audio data were uploaded into Atlas.ti for analysis (Friese, 2011). The elicitation interview codebook was modified for use with the focus group data. Additional themes that arose in the focus group data were also noted for inclusion in the quantitative survey (Ceballos et al., 2014).

The data from the elicitation interviews were triangulated with data from focus groups to develop an initial set of survey items that comprised seven conceptual areas of interest: 1) participation and willingness; 2) knowledge and misperceptions; 3) expectations; 4) barriers; 5) informed consent; 6) trust; and 7) genetics. With permission from the authors, approximately one quarter of the survey items were adapted from the Tuskegee Legacy Project (TLP) Questionnaire (Katz et al., 2006). For example, Question 8 of the TLP asks “Most new medical treatments, medicines and drugs are tested in a study on people before they are allowed to be used for the general population. Have you ever been a participant in a study of that kind (Yes/ No/Don’t know/Refuse)”. Because participants often conflated biomedical research with clinical practice, such as providing blood at a primary care visit, we adapted the question to ask: “Have you ever taken part in a biomedical research study in which a researcher—not your regular doctor or health care provider—took a biological sample as part of a biomedical study? (Yes, I was asked and I participated/Yes, I was asked and I decided not to participate/No, I have never been asked)”. In addition to the adaptations of the TLP, themes that emerged from interviews and focus groups were incorporated into unique survey items. For example, participants described that they perceived discrimination against Hispanics as a barrier to participation in research. Thus, we devised questions to ascertain barriers specifically related to biases: “What are some of the reasons you may not participate in biomedical studies? (Concern that the results may be used against Hispanics [Yes/No]; Concern that the researchers may be biased against Hispanics [Yes/No])”.

As we constructed survey items, we consulted with experts to discuss and agree on appropriate scale anchors that both reflected findings from the interviews and focus groups and would be useful for analysis. For example, both interview and focus group participants described scenarios that applied to “other Hispanics” but not to themselves. Thus, we created response options for some items that included “true for me/not true for me; true for most Hispanics/not true for most Hispanics.”

Phase 3: Cognitive interviews

After completing a final draft survey, the research team developed a cognitive interview guide to test the survey questions among the study population.

The cognitive interview is designed to improve the accuracy of survey results through developing more culturally, socially, and linguistically appropriate questions. It also provides insight into structuring the range of options that will best capture the thoughts and experiences of survey participants. (Desimone & Le Floch, 2004; Nápoles-Springer, Santoyo-Olsson, O’Brien, & Stewart, 2006). Moreover, it can identify issues regarding the items’ accuracy by helping to ensure face validity (Hauge et al., 2015). During this study phase, interviewers asked survey-takers to “think aloud” as they read and responded to survey questions.

Two team members conducted five cognitive interviews using the draft survey instrument. One of the cognitive interviews was conducted with a bilingual participant who examined and commented on both the English and Spanish versions of the draft survey. All comments made by research participants were recorded by hand; the interviewers focused in particular on areas of potential confusion as identified in the analysis of qualitative data described above. Table 2 illustrates the major categories and sample survey questions as well as corresponding cognitive interview questions from the interview guide.

Table 2.

Categories and examples of survey items and cognitive interview probes

	Draft survey items		Cognitive interviews
Survey Category	Example question	Response categories	Example probe	Probe purpose
Knowledge and Misperceptions	Biomedical research involves taking pills or injections.	True/False	Please tell me in your own words what this question is asking.	Test comprehension
Participation and Willingness	Have you ever taken part in a biomedical study in which a researcher—not your regular doctor or health care provider—took a biological sample as part of a biomedical study?	Yes/No	Are there any words that you don’t understand in this question?	Test question clarity
Genetics	I have heard that genetic studies can tell me about my risk for developing a disease.	Yes/No	You said [answer]. How did you come up with this answer?	Elicit participant thought process
Trust	What are some of the reasons you may not participate in biomedical studies? Concern that researchers were not trustworthy	Yes/No	I noticed you hesitated before your response, what were your thoughts?	Identify source of potential confusion
Recruitment preferences	If you were invited to participate in a biomedical study, who would you like to invite you? Priest/Pastor	Yes/No	Can you please discuss why you would like a priest or pastor to invite you to a study?	Elicit participant thought process
Barriers	What are some of the reasons you may not participate in biomedical studies? No transportation	Yes/No	What does “no transportation” mean to you?	Verify interpretation of phrase/ address face validity

All team members reviewed cognitive interview notes to identify specific survey items are areas that required revision. We determined that the survey needed to be shortened to avoid fatigue for survey participants, and that some clarification of questions and responses was necessary. For example, cognitive interviewees had difficulty responding to questions that offered only “yes” or “no” responses. We had originally designed the survey that way to obtain what we perceived to be simple and clear responses. However, cognitive interviewees suggested such a forced choice would not accurately reflect their thoughts, instead pushing them to make arbitrary decisions. In response to this issue, we added a “maybe” response category to four questions.

We also removed several questions that cognitive interviewees suggested were repetitive, vague, and/or irrelevant to the experiences of participants. For example, the original survey contained two similar items in a section asking survey participants to state whether specific perspectives were true for them and whether they were true for the majority of Latinos. One item was “closed to new ideas” and the second was “I don’t like to try new things.” Cognitive interviewees expressed confusion about the difference between those two items and stated that they would answer both of them in the same way. Subsequently, we removed “I don’t like to try new things.”

Discussion

In this study, we drew on the advantages of multiple qualitative data collection and analysis methods to develop a culturally, socially, and linguistically sensitive instrument to collect more accurate data about factors that may influence participation in biomedical research among border Latino populations. The study benefitted from many strengths, such as the involvement of promotores, a high functioning inter-institutional and interdisciplinary team, and participation of members of an underrepresented minority population. Some aspects of these strengths also represented challenges to the process, such as ensuring adequate training of promotores, grappling with competing research paradigms among team members, and addressing issues common to the Latino border population, such as social desirability. The strengths and limitations of these factors based on our team’s experience are discussed as lessons learned below.

Lessons Learned

Involving promotores

The value of involving promotores in research to facilitate engagement among Latino populations is well-established (Deitrick et al., 2010; Schwingel et al., 2015; Swider, 2002). In this study, we hired three promotores to conduct the elicitation interviews. The community connections they had were essential to recruiting participants. The promotores were often members of the community themselves and had a facility with language and cultural norms that facilitated rapport with research participants. As others have demonstrated, inclusion of “insiders”, i.e., promotores, facilitated the access to valuable perspectives of the border community members and contributed to both the credibility of our findings as well as validity of the survey items (Gair, 2011; Sixsmith, Boneham, & Goldring, 2003; Thomas, Blacksmith, & Reno, 2000). Promotores provided feedback to the research team about appropriate methods of recruitment in this particular community. Because they had a history of working in the community, they had developed a level of trust that would have been impossible for the “outsider” researchers to achieve quickly.

Promotores as interviewers.

Although the promotores were trained in interviewing techniques, they had little practice prior to conducting the interviews for the study. Competent interviewers require significant time for practice to hone interviewing and facilitation skills, particularly in understanding how to probe and appropriately clarify participant responses (Bernard, 2011; Krueger & Casey, 2008). This lack of practice was evident in the interview data. When participants did not volunteer in-depth responses, probes were regularly not utilized. However, the focus groups provided additional data that could fill in some of the gaps left in the elicitation interview data. We would not recommend eliminating promotores as interviewers and research assistants because their deep knowledge of local community dynamics is essential to establishing connections and rapport in border communities. However, future studies of this nature may consider pairing promotores with more experienced interviewers.

Interdisciplinary and inter-institutional team

Successfully forming and maintaining the team.

Creswell and colleagues have developed several criteria for the process of developing a research study that are applicable to our work, including inclusion of diverse disciplines; the intentional design to collect, integrate, and triangulate both multiple types of data (e.g., interviews and focus group transcripts, cognitive interviews); and the application of one set of data in the next phase of data collection (Creswell et al., 2011). Our team met common challenges of interdisciplinary team-based teams, such as transcending disciplinary comfort zones, reflecting critically, and creating synergy across disciplines (Hesse-Biber, 2016). We ultimately were successful in working across disciplinary and institutional boundaries to design and implement this study. In order to develop the survey, we first needed to conduct exploratory research with the border Latino population since existing instruments do not focus on this population. Thus qualitative research was a necessary and valuable step in the research process. It provided the methodological tools we needed to gain a broad understanding of the issues, perspectives, and experiences of this particular population in relation to biomedical research as well as to develop a culturally, socially, and linguistically sensitive survey instrument that could accurately capture this complex phenomenon.

Research team language proficiency.

Our research team had some non-bilingual members. To bring together a team with the necessary expertise for the study meant that we could not create a completely bilingual team. Qualitative data analysis presented a particular challenge as we wanted all team members to participate in the analysis process. The need to translate qualitative data from Spanish to English added additional time to the study. Because our focus was on generating nuanced themes not previously captured in quantitative data collection, our translations had to be equally nuanced.

Competing research paradigms.

As Sandelowski (2000) argues, while methods can be mixed, paradigms cannot. The paradigm of researchers is revealed through how the researchers mix methods. If researchers base their inquiry in two fundamentally different paradigms (for example positivist and constructivist) difficulties can arise in research design, analysis and data interpretation. During the process of analysis it became clear that the research team members did not share a collective paradigm. While the qualitative researchers approached these data from a constructivist/interpretive paradigm rooted in ethnographic data analysis methods, others approached data from a more positivist paradigm rooted in quantitative and epidemiological analysis methods. Still others on the team were situated at various points along this continuum. For example, in our initial coding process, some used modified grounded theory approach explained above to identify themes. These members highlighted specific passages within the text and focused on interpretation grounded in interview and focus group data. Other members of the team found it important to count the number of instances of a particular theme in order to determine its importance to the interpretation and analysis –a more positivist approach. Significant unanticipated additional time was required for team members to collectively engage in productive discussions that ultimately led to a shared understanding of the purpose and interpretation of the project with the goal of identifying the willingness to participate, and beliefs and preferences for recruitment in biomedical research. Although our team was able to reach a common paradigm that allowed for the successful completion of the project, we believe addressing this issue at the outset of a project would facilitate more fluid movement from one study phase to the next.

Challenges of engaging a diverse Latino border community

Diversity of Latino population.

The Latino population living near the U.S.-Mexico border is a diverse community. Moreover, variations among Latino subgroups (e.g., country of origin or socioeconomic status) add complexity to the task of developing a single, culturally-relevant survey tool. Our study faced two issues in this regard. First, we were only able to conduct research in one colonia community, limiting our findings to the dynamics of that particular community. Thus future studies should test the survey in multiple communities. Second, language arose as a challenge, particularly in conducting focus groups. Although all participants met the inclusion criteria of being Spanish speakers, during the focus groups, some participants felt more comfortable speaking in English than in Spanish and others moved fluently from English to Spanish throughout the conversation. Although Spanish is the dominant language in the region, this border region is characterized by distinct linguistic patterns that often incorporate both English and Spanish (Vigil & Bills, 2014). Consequently, we found conducting research only in Spanish left some out of the conversation. Future studies may benefit from conducting separate focus groups in both English and Spanish or implementing a brief screening tool to determine participants’ comfort in the language of the focus group.

Social desirability.

Social desirability is a common issue in research of this type (Hewitt, 2007; King & Bruner, 2000). It is possible that some of the data we collected reflects participants’ desire to give the “right” answer rather than the most honest one. The use of promotores may have heightened the effects of social desirability because of prior relationships with research participants, who perceived them as health care professionals. However, our use of multiple methods to gather data mitigates this limitation.

Conclusion

This study exemplifies a three-phase, sequential approach in which interview and focus group findings informed a draft quantitative survey and subsequent cognitive interview findings facilitated revision and a final survey instrument. This work culminated in a culturally-tailored survey instrument to measure perceptions of biomedical research participation among Latino border residents. Our study demonstrates the value of multiple qualitative approaches to inform a culturally-tailored quantitative survey that includes words and constructs relevant to the population of interest. Additionally, the study contributes to qualitative method research paradigms by developing a research protocol that integrates the expertise and perspectives of researchers and community members from multiple disciplines and institutions. Finally, this study outlines lessons learned that allow for effective modification of this research model for future projects. As others have also shown, such studies hold promise for more accurate, sensitive, and comprehensive research programs that seek to understand and evaluate complex health issues in diverse populations.