Abstract
An important step in using culture to increase colorectal cancer screening is the development and use of a reliable and valid measure. Measurement items that work well are defined as those that use clear and simple language, do not result in significant missing data, do not yield unexpected frequencies or patterns of association, and capture an important component of the underlying construct. The authors’ work to develop such a measure includes cognitive response testing. This article describes 41 African American participants’ reactions to and processing of items that have been used in the public health literature to assess cultural attitudes believed to be relevant to colorectal cancer screening. Participants were asked to verbalize thoughts, feelings, interpretations, and ideas that came to mind while examining or responding to 10 to 11 survey items. The results of cognitive response testing suggest negative reactions to items addressing the fatalism construct, concerns about appearing racist when responding to discrimination and mistrust items, and resistance to phrasing or terminology that conveys negative attitudes or frames of reference. When items were framed in a positive way, participants reported less frustration, confusion, and concern for how they would be perceived by others. The responses of older African Americans in this sample were consistent with research previously completed by Pasick et al.; participants questioned the relevance of items related to cultural constructs to health and cancer preventive behaviors. Recommendations for the assessment and use of cultural constructs and items assessing constructs are provided.
Keywords: Black/African American, minority health, cultural competence, cancer prevention and control, surveys, program planning and evaluation
The importance of cultural constructs as influences on health, health promotion, and health care has been clearly established (Brach & Fraser, 2000; Lewis-Fernandez & Diaz, 2002; Office of Behavior and Social Science Research, 2004; Smedley, Stith, & Nelson, 2003). The theory of reasoned action (TRA) emphasizes a central role for social cognitions in the form of subjective norms and provides an important model that places the individual within the social context and suggests a role for culture through beliefs and values (Ajzen, 1992; Fishbein & Azjen, 1975). According to TRA, colorectal cancer screening (CRCS) rates among African Americans are likely to be higher if individuals believe that screening will prolong their lives and improve their health (attitude toward the behavior) and that important people in their lives believe that screening is good and that they should screen (Bandura, Adams, & Byer, 1977). Cultural theorizing suggests that identification (ethnic identity) with other African Americans dictates the importance of the subjective norms of this group. Concern for community health and colorectal cancer (CRC) mortality (collectivism) further influences the pressure to screen. Additionally, attitudes and subjective norms are affected by other group and cultural constraints, such as trust of the medical system, fatalism, norms around masculinity, privacy, and so forth (Deshpande, Sanders Thompson, Vaughn, & Kreuter, 2009; Smedley et al., 2003).
An important step in using culture to increase CRCS is the development and use of a reliable and valid measure. Measurement items that work well are defined as those that use clear and simple language, are not rated as difficult by interviewers or research participants, do not result in significant missing data, do not yield unexpected frequencies or patterns of association, and capture an important component of the underlying construct (Pasick, Stewart, Bird, & D’Onofrio, 2001). Studies that have addressed item and measurement quality note variations in item interpretation based on participation in the activity under study and variations in recall based on the frequency of participation in the activity/activities. Participation in a particular activity may be influenced by culture, which may lead to these differences in item interpretation and recall by members of diverse groups (Warnecke, Johnson, et al., 1997; Warnecke, Sudman, et al., 1997). For example, culture may influence the perception of what is healthy eating or what medical procedures are acceptable, affecting both recall and willingness to report a behavior.
Our preliminary work toward the development of a clearly defined, reliable, and valid measure of culture in relationship to cancer screening includes cognitive response testing to explore participants’ reactions and thought processes when exposed to items measuring cultural constructs as well as items assessing cancer screening, knowledge, attitudes, and beliefs. Cognitive response testing has been used as a technique to improve the quality of survey items and data (Alaimo, Olson, & Frongillo, 1999) and has been found to be useful in work with culturally diverse groups (Lutz & Swasy, 1977; Sudman, Bradbum, & Schwartz, 1996). In cognitive response testing, participants are asked to verbalize thoughts, feelings, interpretations, and ideas that come to mind while examining or responding to a survey item (Lutz & Swasy, 1977). This article adds to the literature on the measurement of cultural constructs by describing African American participant reactions to and processing of items that have been used in the public health literature to assess cultural attitudes believed to be relevant to CRCS and to recommend item revisions.
METHOD
Participant Population
A purposive sample of older African American men and women was recruited to complete one-on-one cognitive response interviews. Eligibility criteria for the cognitive response testing included being self-identified African American, no history of CRC, and having a mailing address and working telephone number. Efforts were made to recruit an equal number of men and women.
Item Selection
Research assistants completed a search of computerized databases including Ovid Healthstar (1990-2006), PsycINFO (1990-2008), Medline (1990-2008), Sociofile (1990-2008), and Social Sciences Citation Index (1990-2008). The search was structured to capture findings that could be considered relevant to current cancer screening activities. Key words that included sociocultural constructs, cultural constructs, social constructs, medical mistrust, fatalism, religiosity, spirituality, collectivism, communalism, racial and ethnic identity, and privacy were used to search titles, abstracts, and subject headings in all databases. A tentative “best” set of items was drafted based on these findings.
A Scientific Advisory Committee (SAC) and a Community Advisory Committee (CAC) received an annotated version of the measures and were asked to identify redundancies in how questions were asked across existing measures and surveys. Each committee suggested items to exclude or reword and suggested edits. The revised set of items was reviewed by a graduate student in comparative literature for grammatical correctness. The process resulted in a set of 42 measurement items.
Procedures
The Washington University in St. Louis Institutional Review Board approved this study and the consent procedures used. Participants were recruited through three sources: community venues (e.g., churches, health centers, housing facilities, barber shops, beauty salons), FReDa (Formative Research Database), and referral. FReDa was established by the Washington University Health Communication Research Laboratory (HCRL) in 1999 and contains the contact information of more than 900 individuals (95% African American) who expressed interest in participating in formative research.
Forty-one eligible participants completed the oneon- one interviews. Each participant responded to one of four sets of items: each containing 10 to 11 items developed using an odd/even split. All 90- to 120-minute interviews were digitally recorded and each session’s recording was professionally transcribed. Each individual received a $50 gift card for participation. The senior investigator and research assistants, all African American women, conducted interviews.
Data coding and analysis were completed based on digitally recorded and professionally transcribed interviews. After reviewing the project goals, the content of the interviews, and the existing literature, the senior investigator developed a defined coding guide that prescribed rules and categories for identifying and recording content. Behavioral coding included questions asking the following: request for clarifications, provision of answer clarifications, and answer type—adequate, qualified, don’t know, and refusal. In addition, all coders read and coded the interview transcripts individually, identifying text units that described relevance to health and decisions related to health, lack of understanding, or discomfort with items. The coders then met to reach a consensus on the definitions and examples used to code interview text. On completion of coding, the coders reconvened to formulate core ideas and general themes that emerged from each interview. An interview summary, with examples, was developed for each construct.
RESULTS
Comprehension
Although most participants did not readily report difficulties with the comprehension or definition of words or phrases, there were a few exceptions during the initial testing phase. Although not a concern among the majority of participants, several reported that the wording associated with fatalism items, such as “If I get cancer, it was meant to be,” was complicated and difficult to understand. Alternative wording that proved successful in Phase 2 testing included “I believe that there are things that I can do to find colon (colorectal), breast, and prostate cancers,” which used a positive frame to query the presence of fatalistic beliefs. Queries of diet were sometimes confused with issues of dieting rather than a reference to food intake: “Q: Does a belief in fate influence your decisions about diet or what you eat? R: Umhm, yeah. Q: How so? R: Cause a diet, right? Like going on a diet?”
Definitional issues also emerged. The term screening was viewed as too general. Several participants referenced annual physical exams in response to items that queried screening attitudes. Even when more aware of the use of the term as a prevention technique, participants stated a preference for items that queried a specific screening test. Alternative wording, such as “My decisions to get tested for cancer will help find cancer,” was better understood. However, this wording confused at least one participant who was unsure if the referent to finding cancer was to a group or her personally. Similarly, participants had difficulty with general terms, such as maintaining one’s health.
Participants understood prevention as the inhibition of the disease process and screening as early detection. For instance, when defining screening one respondent replied, “I don’t think it’ll prevent it. It will let you know if you have it, but I mean it maybe head it off, but if you have cancer you have cancer.” Thus, items that put forward an association between screening and prevention received negative responses, regardless of a positive attitude toward screening. It was suggested that when discussing screening, the term detect be substituted for prevention. In addition, associations between screening and longer life were rejected. “You can get screened and still catch something or get something. I don’t believe that screening can make me live a longer life …”
Items that referenced religion and spirituality also created confusion among participants. Most respondents associated religion with church affiliation. Spirituality was generally perceived as a strong belief in God without church affiliation. Individuals with strong beliefs in God but no church affiliation had greater difficulty responding to items that seemed to imply a religious affiliation. In addition, participants generally had greater difficulty responding to spirituality items compared with religious items, with one participant commenting, “I thought these were going to be easy questions.” For example, participants found it difficult to respond to “I understand my place in the world or I have a sense of purpose”:
Q: So tell me, in your own words, what that statement is saying, I understand my place in the world.
R: I really don’t. I really don’t know because I really don’t understand my place in the world.
Q: Because you said that you disagree with the statement, and then would your response to this item influence your health decisions?
R: No.
Q: No. Why would you say that?
R: Because whatever it is that I do understand or don’t understand about my place in the world, it would not affect my health decisions because this is my life, my body. I want to be healthy. I want to take care of me and do the right thing, whatever it is far as me being healthy. So that wouldn’t affect my decision on my health.
Another participant was concerned that the question was demeaning, and another believed that it was racially motivated,
Well, let me think about it. It sounds so racial. It puts me at a defensive—in a defensive mode to understand my place. And it makes me feel—I feel like saying yeah, I understand my place. My place is higher than whatever you’re expecting it to be. And I’m not trying to—I’m trying not to sound so prejudiced.
Participants reported greater confidence in their responses to items that specifically queried the role of God in their life and health decisions. For instance,
Q: We’re still going to use that same scale, where 5 is strongly agree and 1 is strongly disagree. Looking at number two, “When I have decisions to make in my life, I often ask what God wants me to do.”
R: Strongly agree.
Q: And tell me, in your own words, what this statement is saying.
R: I mean, that’s just the way I live my life. I don’t understand how you can make a major decision in your life, not every decision but major decisions, without consulting God. I mean, I don’t have people that I really confide in, not in major decisions.
The challenges associated with differentiating spiritual and religious constructs, as well as the utility of the constructs, were reflected in preference for terms used to identify the central force for religious or spiritual guidance. When asked about the terms God and Higher Power, God was the more preferred choice.
Items that queried discrimination were problematic as issues of income and insurance were discussed as factors informing responses to items that specifically queried racial discrimination. Few participants were responsive to attempts to address the inclusion of socioeconomic status in racial discrimination items. Items that were designed to address collectivism (responses based on perceived group interests) such as “When I hear information about the health of the black community, I am more likely to get screened” were described as difficult to answer, eliciting responses such as “I don’t know how to answer that. I don’t want to … don’t know how to answer that.” Several participants commented on the need to provide specific data and sources instead of general statements.
It seemed like they singling out—I understand that Blacks are higher risk in certain categories than other people, but when they say, “When I hear information,” okay. I want to know where they got the information from … just give me the data of the information, what percentage are you talking about …
Offensive
There were three categories of questions that participants viewed as offensive. Some participants viewed items that addressed racial identity, mistrust of the medical profession, and racial discrimination as offensive, describing them as “too racial” and creating discomfort among respondents.
Seemed to me—I get the impression that it seems like on racial background, racial theme was what this is asking. Of course I hope that’s not what it is. But by the question itself seems that’s what it’s relating to.
Participants stated that they were worried that items such as “It is important that my doctor respect me as an African American” made them appear prejudiced. Discrimination items that asked whether African Americans received different treatment from members of other racial groups were termed as potentially racist by several participants. The participants with this complaint suggested asking about African American treatment in health care settings. For example, items such as “I have personally been treated poorly or unfairly by doctors or health care workers because of my race” were included in the survey. Finally, some participants termed items that queried mistrust as stupid.
Ethnic identity items that queried respect as African Americans created some concerns among cognitive response testing participants and were frequently difficult to answer. Participants commented that doctors should respect everyone regardless of race, and issues of racism were raised again. Some participants suggested that “respect” be changed to “takes into account.” The rationale was that doctors who acknowledged African ancestry or racial background might be more knowledgeable of the more prevalent diseases or health issues in the priority community, and this could be important in the delivery of care.
Items that addressed the fatalism construct were also viewed negatively. Participants stated that statements such as “diseases like cancer cannot be cured” were “bad” to make in studies focused on health. For example, one respondent noted,
Because you got people that see that and say, “Diseases like cancer cannot be cured” and it would stick in their minds. It would stay on. “Oh my goodness, if I ever get cancer, they’re never going to be able to cure it. Well, they cured Sister Bowman’s or they cured Sister Jean’s. But then they died later on anyway.” I think it brings up too much negativity. There’s nothing in that statement that gives any kind of hope.
Items such as “I believe that if I get tested for cancer, I will live a better life,” however, were less problematic.
Participants also reported that the inclusion of items querying community health beliefs and attitudes related to cancer, such as “Cutting cancer will make it spread” or “colorectal cancer is a disease that White people get” might not be responsible. Although participants acknowledged that they had heard these and similar statements in the past, they believed that their appearance in a health survey or questionnaire might lead some to accept these beliefs or attitudes. Finally, when discussing privacy items, participants noted some reluctance to discuss issues related to behaviors that they associated with the transmission of sexual diseases.
Relevance for Health
Two constructs proved particularly difficult for participants to relate to health: collectivism and spirituality. Participants were unclear how their personal screening decisions or health behaviors affected other African Americans and, likewise, how the behavior of other African Americans might influence their health or health behaviors. Furthermore, participants did not understand how spirituality items, which focused on relationships, the meaning of life, and place in the world could affect their health decisions.
DISCUSSION
The results of cognitive response testing provide insights that may inform the development of a single, reliable, and valid measure of multiple constructs related to cancer screening. Consistent with expectations based on TRA, participants understood and could discuss the relevance of cultural constructs that reflected African American social norms that are believed to influence health behavior. The strongest of these constructs were religiosity, privacy, and medical mistrust. However, testing revealed that there are general issues, such as negative reactions to items addressing the fatalism construct, concerns about appearing racist when responding to discrimination, and mistrust items, as well as definitional issues that are of concern in item selection and wording.
The responses of African Americans in this sample were consistent with research previously completed by Pasick et al. (2001); participants questioned the relevance of items that related some constructs to health and cancer preventive behaviors. Although discussed in the culture and cancer literature (Deshpande et al., 2009), fatalism, collectivism, and spirituality were difficult cultural constructs for participants to relate to cancer screening behaviors. In this sample, participants did not view fatalism as a normative response and did not wish to see items that might be perceived as supporting fatalistic views. Although cultural theorizing would suggest a role for collectivism and spirituality, participants were more responsive to items addressing religiosity than spirituality and only related to collectivist items that addressed the use of group statistics to motivate health behaviors.
Consistent with recent findings in communication research by Nicholson et al. (2008), most participants were resistant to phrasing or terminology that conveyed negative attitudes or frames of reference. These issues were noted most frequently when items addressed fatalism, cancer beliefs, discrimination, and privacy. Participants were particularly concerned about appearing prejudiced or racist when queried about mistrust and discrimination.
There are concepts that met with resistance but have relevance to screening behavior. Consistent with previous research, the term prevention represents such a construct (Sanders Thompson, Cavazos-Rehg, Jupka, Caito, & Gratzke, 2008). In the limited time available for assessments of cultural and screening attitudes among African American participants, it is not possible to define the multiple dimensions of prevention. Therefore, items might better address screening benefits without using prevention language. Comprehension of items was also related to participant experiences in broader social contexts. For example, discrimination is a complicated social issue that participants found difficult to disentangle. To understand participant experiences and responses, items addressing discrimination should address multiple bases for discrimination, separately and in conjunction to one another.
Consistent with prior observation (Pasick et al., 2001), specificity is an important component in generating the sense that items are simple and direct. Participants recommended changes that identified a specific test, such as mammography, or procedure, such as colonoscopy, rather than general queries related to screening, which some participants confused with obtaining a physical exam. Similarly, participants preferred concrete items related to religious behavior, such as prayer and seeking God’s guidance, over more abstract items used to assess spirituality, such as place in the world and purpose in life. Although there were concerns about the appropriateness of discussing issues such as sexually transmitted disease, there was a general willingness to disclose to health care professionals.
Limitations
Although we made an effort to include African Americans of diverse background (within the target age group), the participants were not representative of a national or local sample of African Americans. As noted in TRA, context matters, and it is likely that age, literacy, and health status affect the attitudes and preferences expressed. This sample was composed of volunteers, and it is impossible to know how individuals who chose to participate differ from those who did not. In addition, the provision of a monetary incentive may have influenced some individuals to participate in the study more than others. However, these findings provide a basis for considering the use of terminology, phrasing, and framing of cultural items assessed in relationship to cancer screening.
Implications
To enhance the use of cultural constructs, attempts to define and identify the most relevant constructs and select the most effective items to measure these constructs are essential. As in previous efforts to use cognitive response testing to improve the quality of survey items and data (Alaimo et al., 1999) with diverse groups (Lutz & Swasy, 1977; Sudman et al., 1996;), the data gained from this formative research effort suggest issues that may be considered in selecting constructs and compiling items to develop a reliable and valid measure of cultural constructs relevant to cancer screening.
Collectivism and spirituality were difficult to relate to CRCS, suggesting that research should consider which cultural constructs are useful in promoting specific cancer-preventive behaviors.
Items should query relevant constructs using a positive frame.
Concrete items should be used to query cultural and cancer screening behaviors (religiosity items vs. spirituality items, medical discrimination items vs. social discrimination) and should reference specific tests (CRCSs, mammography vs. screening) and behaviors.
Constructs that have scientific and lay meanings should be addressed using examples and without reference to the terminology that meets with resistance (prevention).
Recommendations for Future Research
While cognitive response methods can yield information on confusion related to terminology, phrasing, and negative emotional responses, quantitative data are necessary to determine the reliability and validity of final survey items. Future research can build on this cognitive response data by examining the relationships among items and scales representing cultural constructs and cancer preventive behaviors, including screening. The performance of items and scales developed addressing the issues outlined above should be compared with measures of cultural constructs currently in use. Because context matters, researchers should determine whether there are differences in how younger and older African Americans respond to items and scales, in addition to exploring geographical and ethnic variations in responses and preferences.
References
- Ajzen I. Attitudes, personality and behavior. Chicago, IL: Dorsey Press; 1992. [Google Scholar]
- Alaimo K, Olson C, Frongillo E. Importance of cognitive testing for survey items: An example from food security questionnaires. Journal of Nutrition Education. 1999;21:269–275. [Google Scholar]
- Bandura A, Adams N, Byer J. Cognitive processes mediating behavioral change. Journal of Personality and Social Psychology. 1977;35:125–139. doi: 10.1037//0022-3514.35.3.125. [DOI] [PubMed] [Google Scholar]
- Brach C, Fraser I. Can Cultural competency reduce racial and ethnic health disparities? A review and conceptual model. Medical Care Research and Review. 2000;57:181–217. doi: 10.1177/1077558700057001S09. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Deshpande A, Sanders Thompson V, Vaughn K, Kreuter M. Use of socio-cultural constructs in cancer research among African Americans: A review. Cancer Control. 2009;16:256–265. doi: 10.1177/107327480901600308. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fishbein M, Azjen I. Beliefs, attitudes, intention, and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley; 1975. [Google Scholar]
- Lewis-Fernandez R, Diaz N. The cultural formulation: A method for assessing cultural factors affecting the clinical encounter. Psychiatric Quarterly. 2002;73:271–295. doi: 10.1023/a:1020412000183. [DOI] [PubMed] [Google Scholar]
- Lutz RJ, Swasy JL. Integrating cognitive structure and cognitive response approaches to monitoring communications effects. Advances in Consumer Research. 1977;4:363–371. [Google Scholar]
- Nicholson RA, Kreuter MW, Lapka C, Wellborn R, Clark EM, Sanders-Thompson V, Casey C. Unintended effects of emphasizing disparities in cancer communication to African Americans. Cancer Epidemiology, Biomarkers, & Prevention. 2008;17:2946–2953. doi: 10.1158/1055-9965.EPI-08-0101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Office of Behavior and Social Science Research. Progress and promise in research on social and cultural dimensions of health: A research agenda. Washington, DC: National Institutes of Health; 2004. [Google Scholar]
- Pasick RJ, Stewart SL, Bird JA, D’Onofrio CN. Quality of data in multiethnic health surveys. Public Health Reports. 2001;116(Suppl. 1):223–243. doi: 10.1093/phr/116.S1.223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sanders Thompson V, Cavazos-Rehg P, Jupka K, Caito N, Gratzke J. Evidential preferences: Cultural appropriateness strategies in health communications. Health Education Research. 2008;23:549–559. doi: 10.1093/her/cym029. [DOI] [PubMed] [Google Scholar]
- Smedley B, Stith A, Nelson A, editors. Unequal treatment: Confronting racial and ethnic disparities in healthcare. Washington, DC: National Academies Press; 2003. [PubMed] [Google Scholar]
- Sudman S, Bradbum N, Schwartz N. Thinking about answers: The application of cognitive processes to survey methodology. San Francisco, CA: Jossey-Bass; 1996. [Google Scholar]
- Warnecke RB, Johnson TP, Chávez N, Sudman S, O’Rourke DP, Lacey L, Horm J. Improving question wording in surveys of culturally diverse populations. Annals of Epidemiology. 1997;7:334–342. doi: 10.1016/s1047-2797(97)00030-6. [DOI] [PubMed] [Google Scholar]
- Warnecke RB, Sudman S, Johnson TP, O’Rourke D, Davis AM, Jobe JB. Cognitive aspects of recalling and reporting health-related events: Papanicolaou smears, clinical breast examinations, and mammograms. American Journal of Epidemiology. 1997;146:982–992. doi: 10.1093/oxfordjournals.aje.a009226. [DOI] [PubMed] [Google Scholar]