Issues in Cross Cultural Comparative Research

Abstract

It is the purpose of this article to describe the methodological issues when designing qualitative cultural comparative studies, which may be used to address health disparities. Overall, two broad types of comparative studies were found: primary and secondary comparative studies. Methodological issues to consider when designing primary studies were reviewed. The main areas discussed in this paper relate to sampling and study purpose, sample boundaries, theoretical context, concept development, analogous comparisons and systematic comparisons. Nurses are in a prime position to pose the qualitative research questions needed to address health disparities within their clinical settings. It is suggested that awareness of the method types and issues might inspire further qualitative comparative work.

According to the National Institutes of Health, Institute of Nursing Research (USDHHS, 2007), studies explaining health disparities among many of our nations subpopulations are needed. Although advances have been made in many areas, reasons for health disparities in areas such as disability outcomes remain unexplained (Harrison, 2009; Tripp-Reimer, Coi, Kelley, & Enslein, 2001). Qualitative research may provide the methodological means for developing the foundational knowledgebase needed, but to develop theories of health disparities, the qualitative methods used for group comparisons need careful consideration.

When quantitative researchers compare quantitative data the assumption is that the data is representative, somewhat like a microcosm, of the groups from which it was sampled. When qualitative researchers make comparisons of data it is from non-randomized samples. The evidence used for comparisons are not generalizable—but they may be transferable (Lincoln & Guba, 1985). The qualitative data compared should be carefully crafted to complete a picture of a phenomenon of interest within each group in order to thoroughly understand the phenomenon prior to making comparison (Alasuutari, 1995; Morse 2003). If this is not considered carefully a critical advisor might pose the question: What was the point of making comparisons of two obviously different non-representative samples? When constructed well, qualitative comparative data may be highly useful for public health intervention because it may describe the context as well as the range of experiences leading to health disparities. The purpose of this article is to discuss the issues researchers might consider when designing studies using qualitative comparisons. In keeping with the purpose of this article, the types of qualitative comparisons found in the extant literature are briefly described. Next, issues researchers might consider prior to designing comparative studies are discussed.

Types of Qualitative Comparisons

Based on a review of the extant literature in Pub Med (n=1015), CINAHL(n=16) and Anthropology Plus (n=3) over the past 5 years using the terms “qualitative method” and “Comparison”, a total of 20 manuscripts met the criteria of being English language, research based, narrative comparisons between two distinct samples and were available for review. From this review, it is suggested that two broad types of methodological comparisons were published, secondary and primary comparative studies. These categories were considered beneficial groupings because they inspired consideration of different issues for designing qualitative comparative studies, which will be reviewed after a brief discussion of the comparative types. They may also provide researchers with ideas for future comparative studies using qualitative methods.

Secondary Comparative Studies

The first type, secondary comparative studies (n=3 out of 20), were comparisons of studies where data collection was completed for a different purpose in at least one of the two samples. There were two types of secondary comparative studies. The first was a comparison of data collected for two studies completed for different purposes, such as the comparison of coping processes of nurses to that of chaplains (Ekedahl & Wengstrom, 2008). Authors combined their data from previous studies to compare coping processes between groups. They published the results of this third analysis by creating themes that focused on the differences between processes. The second type of secondary comparative studies was the comparison of evidence from multiple studies to another group currently being studied. For instance, Bush, Collings, Tamasese, and Waldegrave (2005) compiled what was known about perception of self among the Samoan people of New Zealand and compared it to eight psychiatrists' views of self during the course of three focus groups. Both types of secondary comparative studies relied on analysis of previously collected data and analyses.

Primary Comparative Studies

Another type of comparative study was the primary comparative study (n=17). These studies were designed to compare an attribute or phenomenon among participants from different cultural groups. Frequently, the comparisons were made based on ethnic group affiliation (Collins, Decker, & Esquibel, 2006; Im et al., 2009), but comparisons were also made based on organizational culture, such as intensive care units (Baggs, Norton, Schmitt, Dombeck, Sellers & Quinn, 2007). In primary comparative research, some researchers compared individual cases, such as in a study of perceptions of age and social integration among three women with varying degrees of disability (Harrison & Kahn, 2004), while other researchers compared individual cases to larger groups, such as in a study of the communication patterns of one child with mental retardation and autism to the communication patterns of a group of 19 other children without these conditions (Hendenbro & Tjus, 2007). Further, data from large samples compiled based on common cultural attributes, such as community membership (Guest, 2007) and/or ethnic identity (Mahoney, Cloutterbuck, Neary, & Zhan, 2005) were used for comparisons. For instance, Guest (2007) compared self-esteem and social comparisons during middle childhood in Chicago public housing and Angola camps.

Both types of aforementioned comparisons, secondary and primary, were used by researchers in recent years. Regardless of the type, however, few researchers detailed the design challenges they faced when making the comparisons. Secondary comparisons of qualitative data have unique challenges because most design issues were decided when data was collected initially. The use of secondary data must be carefully considered. Only the methodological issues when designing a primary comparative study will be discussed in the next section.

Comparative Design Issues in Qualitative Research

In the comparative research reviewed, design decisions were made with attention to the creation of a trustworthy understanding and representation of cases, groups, or phenomenon compared. The main areas suggested for consideration relate to sampling and study purpose, sample boundaries, theoretical context, concept development, analogous comparisons and systematic comparisons.

Sampling and Purpose

The first issue to be discussed in relation to this is related to the sampling and the purpose of the study. Qualitative research is not generalizable to the larger population, which could leave some to wonder what benefit qualitative methods provide in the area of health disparities research. The qualitative comparison of two hapless, convenient samples may not establish the theoretical knowledge needed to explain health disparities. The knowledge needed regarding health disparities is most likely created when the purpose of the study is to explore reasons for health disparities between two groups and the samples are purposefully gathered to construct a complete picture of each group. Thus the evidence would provide rich descriptive evidence on the topic. Qualitative sampling in particular “is about establishing the range of phenomenon, not establishing the proportion of traits in a population at-large” (Werner & Bernard, 1994, p. 8). Thus, if a researcher proposes to build theoretical understanding based on experiences, then qualitative methods can work if the sampling techniques were used appropriately. Sampling can build upon the epidemiological literature pointing out the demographic variations, which can serve as a basis for sampling. However, if the only purpose is to determine the range of varying traits in two groups that have disparities in outcomes, then qualitative comparisons may be meaningless.

Clear Group Boundaries

Finally, before conceptual boundaries within groups can be created, the boundaries between the groups themselves should be understood. This may be difficult to ascertain when geographic boundaries blur, which could be the case when studying health disparities. Ethnographic exploration of communities with clear geographic boundaries may be amenable to what is considered ‘traditional’ comparative fieldwork. When very clear geographic boundaries exist, it is easier to segregate evidence from distinct groups.

It is more difficult when the different groups under investigation interact on a daily basis blurring the lines that separate them. Commonalities in culture and history may exist across multiple ethnic groups set within unclear geographic boundaries, and these shared meanings often deserve exploration in order to understand health disparities. In other words, to what extent do the groups share their cultures? This situation may be more challenging for qualitative researchers to address. It must be remembered that indiscriminate acceptance of generalizations about a people that ignores the differences among and between groups could limit the meaningfulness of research. Townsend (2003) states, “…broadening the scope of inquiry beyond that which is considered typical from the dominant culture's perspective can only enhance the researchers' senses to differences that may exist among and between cultural groups” (p. 274).

Two main challenges due to blurred geographic boundaries might be considered in the design of a study. The first is when the different groups studied are intertwined but within a circumscribed area. The second is when the groups studied are intertwined but dispersed across various geographic locations. In research investigating life course trajectories, it was reported that ethnic commonalities existed among Mexican American women despite their dispersion across vary different geographic locales in Texas. Indeed, authors reported that “we agree with Barth (1969/1996) that an ethnic boundary might exist despite proximity to and interaction with various other ethnic groups, and that ethnic unity and culture might be maintained despite scattering across settings” (Harrison, Angel, & Mann, 2008, p. 770). Geographic locale influenced resource acquisition and human agency in the aforementioned study. This meant that commonalities in cultural units existed within ethnicity across geographic locales but differences in resource distribution influenced how those cultural units influenced behavior. The meaning behind group identity may unite a people and subsequently transcend geographic boundaries in the context of a multi-ethnic grouping of people. Hence, ethnic identity may be a reasonable way to study groups of people within unclear geographic boundaries. The researcher, however, may need considerable fieldwork in order to understand group boundaries and the influence of ethnic identity.

Theoretical Context Providing Flexibility

The second issue to consider when designing a primary comparative study surrounds the ascription of influence on health disparities. Even though similarities and differences could be detected between groups, it may be a challenge to scientifically illustrate that the reason for the difference may be attributed to the presence or absence of certain variables. The classic problem in cross-cultural research, ‘Galton's problem’, was due to the comparison of cases that overlapped in ways that made the trustworthiness of the comparisons suspect (Legesse, 1973). For instance, it could be difficult to attribute reasons for differences between groups to cultural variables when geographic locale, history, and language overlapped to the point that groups shared cultural meanings, difficult to untangle. This requires that the context of both groups be understood in order to assert an explanation for the influences on differences. Past researchers have been foiled when they made comparisons, attributed their findings to an attribute of both groups, only to find out the differences were due to a third unconsidered variable. Qualitative researchers doing health disparities- cultural comparisons--may be just as easily foiled if they do not develop thorough understandings prior to analyses.

Qualitative researchers have an advantage when considering Galton's problem because qualitative research builds contextual understandings. Regardless, context changes and concepts may be unstable artifacts of a group (Gatewood, 2000). Hence, to address this problem, anthropologists suggested that cross-cultural comparisons be based on a definition of culture that allows for thorough examination and development of comparable cultural units. In other words, the design of the study, prior to entering the field, should bring with it a flexible understanding of the theory behind culture. Brumann (1999) wrote that culture is “the clusters of common concepts, emotions, and practices that arise when people interact regularly”. Brumann stated that this definition of culture called for flexibility when entering the field and promoted understandings of groups that would overcome Galton's problem.

Regardless of the exact definition or theory of culture used, importance rests on being flexible and open while in the field collecting data. This way, when that third unknown variable arises, the researcher is able to see it. According to de Munck and Korotayev (2000), Brumann's definition of culture allowed for flexibility when examining the various ways in which phenomenon within a culture may come to explain outcomes. de Munck and Korotayev suggested this because in various scenarios the mechanism driving the cultural differences was difficult or impossible to know before spending time interacting within that culture. If a researcher created a fixed definition for the cultural units under investigation before understanding the detailed, evolving nature of that culture, results could be indistinct or unclear. Flexibility within the definition of culture inherent in the study design accompanied by time spent in the field should facilitate a thorough understanding of the nuances of group differences.

Independent Concept Development

In addition to flexibility and time a third issues involves the development of concepts studied. These concepts should be thoroughly developed before group comparisons. According to Weller and Baer (2002), a firm understanding of within-group concepts must precede cultural comparisons. The process of identifying within-group concepts could be thought of as creating finely crafted pieces of a puzzle from raw material belonging to two separate but similar puzzles, comparing the pieces of the puzzles to each other, and then describing the comparisons of the pieces to understand the abstract concepts that specify how different pieces produce different puzzles. The reason for doing this is simple. Phenomenon should be understood as products of the group under investigation, not as products of both groups simultaneously.

This deserves further explanation, however, because researchers might be tempted to collect and analyze data from two distinct groups simultaneously—comparing as the groups as the process proceeds. In other words, researchers might develop a conceptual understanding based on a comparative process that defines concepts based on a lack of or abundance of some aspect of the concept when compared to evidence from the other group. When the researcher draws from both groups simultaneously the concept created is based on evidence from both groups, which can change the way the concept develops independently. Evidence for the meaning of the concept should be constrained to the boundaries of the group studied.

Analogous Comparisons

The fourth suggestion is that comparative concepts should be conceptually analogous (Alasuutari, 1995). The researcher should compare apples to apples. Further, judgments about concepts should be kept to a minimum until researchers are certain that they have chosen comparable concepts that have been fully developed. Indeed, misunderstandings may occur when researchers assume groups share potential for similar conceptual properties when in reality the concepts compared were different concepts.

Systematic Comparisons of Similarities and Differences

Fifth, comparisons should be made by explicating similarities and differences between concepts within groups prior to providing theoretical reasons for health disparities. This may seem obvious, but authors may be tempted to create lengthy narratives about reasons for health disparities without conducting the analytical work necessary to understand where the actual similarities and differences reside. Just stating that resources were different in one group when compared to another may not fully attend to the reasons for the health disparities. To attend to health disparities, careful explication of the similarities and differences along with the logical context for their influence on health disparities may be needed prior to theorizing the mechanisms.

In conclusion, research is needed on reasons for health disparities. Given the complexity of the problem, it is recommended that methodological decisions be carefully considered. To assist with the methodological decisions researchers face, details were provided on the types of cultural comparative studies currently being under taken, as well as a discussion of the types of issues researchers may wish to consider when designing a cross-cultural comparative study. It is not suggested that there is only one way of designing a comparative study. The beauty of qualitative research to these authors is its openness to creativity and exploration. It is suggested that analytic decisions be crafted to meet the needs of the study's purpose while carefully attending to issues that might influence a study's comparative trustworthiness.