Research with American Indian and Alaska Native Populations: Measurement Matters

Abstract

Research is an important tool in addressing myriad American Indian and Alaska Native (AIAN) health disparities; however, tensions exist between common empirical measurement approaches that facilitate cross-cultural comparisons and measurement specificity that may be more valid locally and/or culturally appropriate. The tremendous diversity of AIAN communities, small population sizes of distinct AIAN cultural groups, and varying cultural contexts and worldviews should influence measurement decisions in health research. We provide a framework for guiding measurement in collaboration with AIAN communities using examples from substance abuse research for illustration. Our goal is to build upon ongoing efforts to advance measurement validity for AIAN research by engaging community-researcher partnerships and critical thinking in the selection, adaptation, creation and implementation of measures.

Research on American Indian and Alaska Native (AIAN) health is critical to addressing disparities that are documented across the physical and mental health spectrum in these communities (Indian Health Service [IHS], 2014). Yet researchers often encounter difficulties ensuring that measurement strategies are both scientifically and culturally sound to produce reliable, valid data. Studies that employ measures borrowed from research with non-Native populations may not account for unique AIAN worldviews and values associated with health and wellbeing (Westermeyer, 2004; Kagawa-Singer, et al., 2014). Such limitations are increasingly recognized with attempts to advance measurement validity for AIAN contexts (e.g., Mohatt, Fok, Burket, Henry & Allen, 2011; Whitesell, et al., 2015; Whitbeck, Chen, Hoyt & Adams, 2004; Mullany, Barlow, Neault, Billy, Hastings, et al., 2013). This paper builds on these efforts and proposes a systematic approach to engaging community-researcher partnerships in applying critical thinking to the selection, implementation, adaptation, and/or creation of measurement tools and strategies with AIAN communities.

We begin with socio-historical, cultural and modern-day contextual background about AIANs essential to understanding how and why common measures may not perform as expected with AIAN populations. Then, we present a framework for approaching measurement of health outcomes and risk and protective factors in AIAN settings. Next, we provide examples from studies of substance use to illustrate the framework in public health research. We selected substance use research for this purpose because AIANs share unique substance use-related disparities and risk and protective factors compared to other racial and ethnic groups in the United States (US). We conclude by recommending strategies for improving measurement processes in collaboration with AIAN communities.

American Indian and Alaska Native Communities

The contemporary settings in which AIANs live are extremely diverse. As of this writing, there are 567 federally recognized tribes in the US representing hundreds of distinct cultures. Many more tribes do not have federal recognition: some are recognized by states while others lack political recognition at all (Bureau of Indian Affairs, 2015; Miller, 2004; National Conference of State Legislatures, 2015). Tribal populations range from Alaska Native villages of a few hundred people to the Navajo and Cherokee Nations, each with hundreds of thousands of members. Some AIAN peoples reside on tribal lands and have strong connections to their ancestral geographies. Tribal spaces may be small and dense (e.g., Pueblo communities), or situated in rural, remote areas with great distances between community members (e.g., rural reservations or Alaska Native villages). A majority of AIAN people live in semi-rural, suburban or urban areas (U.S. Census, 2012).

The 5.2 million people in the US who identify as AIAN (US Census, 2012) occupy a unique socio-historical context relative to other members of American society (Whitbeck, Hartshorn, & Walls, 2014). AIANs have endured colonization and resulting inter-generational trauma, including genocide, ethnocide, forfeiture of homelands, loss of languages, forced displacement, and imposed assimilation. One lingering effect is the implementation of procedures for tribal membership (e.g., meeting a minimum “blood quantum”) that impact sense of belonging and identity (Doerfler, 2015). The experience of AIAN group marginalization and discrimination (Walters, Simoni, & Evans-Campbell, 2002) includes large-scale misappropriation of cultures and identity, such as race-based mascots in major league sports, or stereotyped graphics of Native people in marketing. AIAN communities are more likely than other US groups to experience over-crowded households, lower educational attainment, poverty, unemployment, chronic stress, and trauma (Sarche & Spicer, 2008, U.S. Department of Justice, 2004). These social and historical determinants of health have resulted in significant health disparities for AIANs including disproportionately high rates of type 2 diabetes, substance abuse, suicide, tuberculosis, pneumonia, and psychological distress (NCHS, 2012; IHS, 2014).

Though significant, these challenges do not define AIAN communities or individuals. Federally recognized tribes exist as sovereign nations within the US. AIAN activism propelled a shift in official federal emphasis from Indian termination to self-determination beginning in the 1970’s and continuing to present day (Wilkinson, 2005). As policy, sovereignty grants tribes the authority to govern themselves as domestic dependent nations within the U.S. A deeper interpretation emphasizes AIAN survivance (Vizenor, 1999) as active survival, denouncement of victim status, and political and intellectual autonomy (see also Doerfler, 2015). As in all cultures, AIAN customs and norms are diverse and evolving (Lakes, Lopez, & Garro, 2006), yet there are rich cultural heritages rooted for many tribes in collectivism, extended kinship systems, deep respect for elders, and traditional spiritual ways. This interplay of identity, politics, culture, and colonization all make conceptualizing and operationalizing “culturally-specific” constructs and interpreting related research findings complex in AIAN settings.

Framework for Measurement

AIAN diversity and the small population size within given AIAN groups pose a challenge for effective measurement. On the one hand, there is a need for measures tailored for both AIANs in general and for specific tribal groups; yet, on the other, common measures developed, validated, and widely used in non-Native populations are necessary for determining if predictors of general population health and substance use are relevant for AIANs. Common measurement may also permit useful cross-population comparisons (Beals, et al., 2003) and potentially facilitate pooling/harmonization of the data across samples for enhanced statistical power. Thus, there are compelling reasons to consider both tailored and common measures.

The tension between tailored versus common measurement approaches can be overwhelming and may stall health research with AIAN populations at the outset. In response, we offer a framework for thinking strategically about these issues during selection, adaptation, creation, and implementation of measures in research. This framework draws upon 1) the general literature on measurement development processes (Carmines & Zeller, 1979; Nunnally, 1967; Fowler, 1995; Babbie, 2015), and 2) the authors’ and others’ experience adapting and creating measures for specific AIAN contexts (Beals, et al. 2003; Mullany, Barlow, Neault, Pan, Billy, et al., 2012; Gonzalez & Trickett, 2014; Whitbeck, et al., 2004).

The framework is illustrated in Figure 1 and depicts a dynamic, interconnected set of processes.

Figure 1.

Framework for guiding measurement with AIAN populations.

There are three basic components: The outermost Measurement Development Cycle, the innermost Wheel of AIAN Specificity, and the Community-Research Partnership layer embedded between the circles that drives their alignment. The terms cycle, wheel, and partnership emphasize the dynamic nature of the Framework. Cycle represents the iterative process of continuous measure development. The inner Wheel rotates depending on measurement needs relative to research goals, AIAN context, and phase of measurement development. The Partnership layer represents the critical and driving relationship between AIAN community members and researchers, the expertise each brings to the research endeavor, and their collaborative role in aligning each phase of the measurement cycle with the most harmonious level of specificity.

Measurement development cycle.

The Measurement Development Cycle includes common social and behavioral sciences terminology to describe the measurement process (Babbie, 2015 ). The cycle begins with Conceptualization (the process of identifying the ideas or constructs for eventual measurement), followed by Operationalization (concretely defining indicators of the constructs, ascertaining whether existing measures will work, writing items, developing metrics), Implementation (employing scientific methods for gathering data), and Interpretation (framing results and articulating the meaning of the data). Again, this cycle is ongoing and iterative, with Interpretation looping back to inform Conceptualization of constructs for future research.

Wheel of AIAN specificity.

At the center of the framework is a rotating wheel designed to reflect degrees of cultural specificity potentially required across phases of the Measurement Development Cycle. The static alignment of this two-dimensional figure should not veil the fact that any quadrant on the specificity wheel can be the entry point for measure development. In its current presentation, the upper right quadrant represents what can be borrowed from extant literature, largely from non-Native populations. We have labeled this quadrant “common” to refer to measures frequently used and cited. “Common” implies some degree of shared meaning and approach to measurement. The remaining three quadrants in the Wheel of AIAN Specificity reflect increasing levels of tailoring and cultural specificity: (1) Tailored for the AIAN Population in general (i.e., broadly inclusive of diverse AIAN tribal land-based, rural, semi-rural and urban populations in the U.S.); (2) Tailored for a Specific Tribe or for a group of culturally similar tribal communities (e.g. Navajo, Pueblo, Lakota); or, (3) Tailored for a Specific AIAN Community, such as a specific AIAN municipality or village within the broader tribal group (e.g. Isleta versus San Felipe Pueblos, Oglala versus Sicangu Lakota, etc.). Decisions about common approaches or tailoring may be needed at Conceptualization, when constructs specific to AIAN contexts are not reflected in existing measures, or at the level of Operationalization, when common constructs have different manifestations in AIAN cultures. Or, community-research partnerships may need to tailor Implementation protocols or Interpretation of data.

In theory, more specificity across the Measurement Development Cycle should translate to greater validity. This is particularly true when there is evidence that more common measures fail to capture important constructs in ways that are reliable and valid in specific AIAN cultural contexts. In reality, however, highly tailored novel measures may introduce unknown errors of reliability, dimensionality, and other psychometric indices important to quantitative inquiry. Further, when common measures do have sufficient reliability and validity in AIAN populations to provide useful, trusthworthy information, they bring the added benefit of facilitating comparisons to other populations. The balance between common and tailored approaches should always be carefully considered.

Community-Research Partnership.

Set between the outer Measurement Development Cycle and the inner Wheel of AIAN Specificity is the critical engagement of a collaborative Community-Research Partnership. This is the force that determines the alignment of the measurement cycle and specificity wheel. An increasing literature on community-based participatory research (CBPR; e.g., Cochran, et al., 2008; Horn, McCraken, Dino, & Brayboy, 2008; Chavez, Duran, Baker, Avila, & Wallerstein, 2008) and, more specifically, tribally-based participatory research (TBPR; Fisher & Ball, 2003; Mariella, Brown, Carter, & Verri, 2009) undergirds this component of the framework. CBPR and TBPR promote genuine researcher and community trust relationships and collaboration throughout the research process and have emerged in response to past wrongs, including reductionistic or stigmatizing studies that ignored or dismissed tribal knowledge and/or that solely served outside research interests (Baldwin, Johnson & Benally, 2009). TBPR honors tribal soverignty and the inherent rights and duties of tribal governments to promote and preserve community health, including the protection of cutlural resources and knowledge (Harding, et al., 2012). In Figure 1, a given partnership would work to identify important concepts, translate them into operational definitions and measures, guide implementation efforts to ensure meaningful data collection, and bring cultural and contextual knowledge to bear on the interpretation of findings (see also Beauvais, 1995).

Process of alignment.

Assumptions should be evaluated at each phase of the Measurement Development Cycle. One consideration is whether the construct of interest for AIAN populations maps appropriately onto a corresponding common measure. If yes, Conceptualization can align with common on the specificity wheel (as depicted in Figure 1); if not, the wheel should be rotated to align with the optimal level of tailoring. Another appraisal relates to whether or not the existing Operationalization of a given construct is adequate for AIAN studies. Using an example that we elaborate later, a construct like ‘substance use disorder’ may be relevant for a study of AIAN substance abuse in terms of conceptualization, but at the operationalization level, one may wish to capture distinct patterns of use (e.g., more binge use, less daily drinking). The specific alignment of the cycle and wheel could reflect considerations such as access to alcohol in a local context (e.g., dry reservation vs. urban setting) and cultural values around alcohol use (e.g., prioritizing sobriety vs. accepting moderate drinking). Research partners should again consider tailoring needs at the level of Implementation. Common methodologies may be adequate, or they may need to be adapted. For example, many CBPR projects employ community members as research assistants and interviewers; thus, confidentiality considerations in small, close-knit tribal communities may be especially important, particularly surrounding sensitive topics (Katz, Martinez, & Paul, 2011). Finally, research partners need to consider the extent to which data Interpretation needs to be tailored to a specific community. Does substance use disorder have the same meaning and implications for health in an AIAN community as in other populations, or, between AIAN communities? Gathering valid data ultimately depends on optimizing project needs with alignment at each phase of the measure development cycle.

Counterbalancing considerations for tailoring to cultural and community contexts are the challenges and implications of doing so. Effective measurement requires considerable time and community and research resources, both of which can be in short supply. For instance, Alegria and colleagues (2004) describe a five-step process for attaining cultural relevance of measures for diverse Latino samples. Multiple, complex tasks appear in each step of the framework including extensive reviews of existing measures, creation of novel measurement items, translation and back translation, qualitative measurement assessment, pretesting, and psychometric analysis of measurement properties. Measure development research is an expensive undertaking, requiring advanced methodological expertise, and iterative processes of measure refinement; budgets for research with AIAN populations rarely afford this. For some AIAN communities, there is a pressing need to collect data rapidly to inform and address urgent concerns (e.g., cluster suicides; Wissow, Walkup, Barlow, Reid, & Kane, 2001), with no time for tailoring. In addition, small AIAN community samples present difficulties in establishing rigorous psychometric evidence of the integrity of tailored measures. Research partners often have to rely on common measures while relying on the community-research partnership for the appropriate interpretation of findings, including acknowledgement of limitations.

Applying the Framework: Examples from Substance Use Research

In this section, we use examples from one area of research with AIAN populations that has received a great deal of attention: substance use and abuse. This illustration is particularly salient because surveillance studies spanning several decades consistently report a greater frequency of substance use and associated problems for AIAN adults and youth compared to other racial and ethnic groups in the U.S. (e.g., Beauvais, Oetting, & Edwards, 1985; Beauvais, Jumper-Thurman, Plested & Helm, 2004; Stanley, Harness, Swaim, & Beauvais, 2014). The stark public health impacts of substance use and abuse in AIAN communities are evident: Driving under the influence and alcohol-related violent offenses and deaths are significantly more common in AIAN populations than in non-Native communities (West & Naumann, 2011; IHS, 2014). The potential influences of these disparities have deep historical roots. As one example, alcohol had a specific role in colonization, with intoxication used by European settlers and later agents of the federal government as tools to coerce tribal entry into unfair treaties and trade agreements (Frank, Moore & Ames, 2000; Barlow, et al, 2012). From these origins as a means of political, economic, and social subjugation, alcohol and substance use problems have emerged at higher rates among AIANs, along with modern day stereotypes of the “drunken Indian” (i.e., May 1986; Barlow et al., 2012; Duran & Duran, 1995). However, AIAN communities are also characterized by high rates of abstention (Spicer et al., 2003; SAMHSA, 2014), reflecting the complexities of substance use and the need for measurement approaches that tend to these complexities.

We begin our illustration of the Measurement Development Cycle and Wheel of AIAN Specificity by focusing on outcomes – here, substance use and abuse – that are generally assumed to be directly observable and thus more readily measured. Next, we examine risk and protective factors – constructs typically reflecting constellations of perceptions, attitudes, or ideas that cannot always be directly observed; these include mental health concerns, family dynamics, and AIAN cultural factors.

Measuring outcomes: Substance use and abuse.

Common measures of substance use for documenting epidemiologic patterns are cataloged across numerous measurement compendiums, websites, and textbooks. Measurement approaches for substance use disorders are generally structured screeners or highly standardized computer-assisted interviews (e.g., Composite International Diagnostic Interview (CIDI): Kessler, et al., 2005) that mirror psychiatric diagnostic criteria from the Diagnostic and Statistical Manual of Mental Disorders (APA, 2013). Academicians have engaged in lengthy and rigorous psychometric evaluation and validation studies to develop and evaluate measures of substance use and disorders, including large and diverse samples to document broad applicability (e.g., NIH PhenX). Despite these rigorous standardization efforts, there may be risks when common substance use measures are implemented without careful scrutiny of their specific relevance in AIAN cultural contexts.

The following examples illustrate the intricacies of adequately measuring alcohol and other substance use in AIAN contexts. Diverse cultural norms for alcohol use among different AIAN populations suggest varied Conceptualization of problematic use. There is sociocultural variability in trajectories and attitudes about drinking and drug use within and between AIAN individuals and cultural groups (Greene, Eitle, & Eitle, 2014). In some AIAN contexts or communities, social drinking is acceptable and only excessive or binge drinking is of concern, while for others, any amount of alcohol use is viewed as problematic. In this instance, while common measures may be appropriate for enumerating patterns of use, the Interpretation phase of the measurement cycle would require turning the Wheel of AIAN Specificity to a tailored quadrant particular to a tribe or cultural group, to ensure that observed patterns of use are interpreted within the local context, values, and norms. Consider also those reservation communities where alcohol use is illegal; the most prevalent pattern of problematic use in such contexts may be intermittent binge drinking amidst periods of abstinence (Spicer et al., 2003). Common measures capturing past week or past month frequency could miss significant binge episodes, their severity and consequences, and the local meaning and impact of such episodes (e.g., individual functioning or status within the community). Thus, Operationalization tailored to this community context is required.

Another concern with current models at the level of Conceptualization reflects emphasis on abuse and disorder, generally to the exclusion of strengths. The existing literature tends to focus on high rates of problematic use while ignoring the aforementioned high rates of abstinence among AIANs compared to most other racial/ethnic groups. These patterns and the conceptual significance of sobriety narratives are underexplored in the current literature (exceptions include Mohatt, et al., 2004; Bezdek & Spicer, 2006; Spicer, 2001). Shifting focus during Conceptualization could help reverse the stigmatizing stereotypes of substance-abusing AIAN communities perpetuated in the past (see May, 1986) and move towards a strengths-based health promotion narrative.

In some cases, the Conceptualization and Operationalization phases of the Measurement Development Cycle appear to align well with common approaches, but practical issues may arise in Implementation. Community-research partnerships should consider whether tailoring should occur with respect to administration of measures, including who gathers data and and in what form. As an example, the American Indian Services Utilization, Psychiatric Epidemiology, Risk and Protective Factors Project employed local interviewers trained to collect data about mental health and substance use disorders (Beals, et al., 2003). In other contexts, however, community interviewers might result in biased data due to research participant concerns about confidentiality. For instance, Mullany and colleagues found that using audio computer-assisted self interview technologies versus local Native interviewers or self-report yielded more complete and valid substance abuse data from a sample of AI youth (Mullany, et al., 2013). Ultimately, as Gonzalez and Trickett (2014) assert, “Measurement development efforts must be responsive to cultural practices” (p. 114). They further highlight implementation factors like the importance of local norms for training interviewers and proper use of linguistic tools (e.g. avoiding non-transferable metaphors, local word choices, grammar, syntax, and notions of time).

Accurate Interpretation of findings almost always requires alignment with a tailored quadrant of the Wheel of AIAN Specificity to facilitate understanding through a culturally-grounded and locally-meanigful lens. Research on initiation of substance use among adolescents in a Northern Plains tribe (Whitesell et al, 2014; Whitesell, Beals, Big Crow, Mitchell, & Novins, 2012) provides a telling example. In this study, lifetime and past month use of tobacco, alcohol, and marijuana and age of onset were assessed using common measures. Measures were minimally adapted (e.g., examples of alcoholic beverages that were common locally), because community partners found the measures appropriate for their tribe as they were. Resulting data indicated that youth were initiating marijuana use almost concurrently with tobacco and considerably before alcohol, a finding at odds with general population studies where alcohol use has been show to precede marijuana use. Community advisors suggested that earlier use of marijuana was consistent with cultural values around “natural” substances, while alcohol was viewed as a substance external to tribal ways and a symbol of colonization. They also noted that marijuana, which could be grown locally, was as easy to access as alcohol on their dry reservation. While researchers and community advisors alike were concerned about the early initiation of marijuana use, this cultural lens was critical to the interpretation of data and identification of prevention strategies.

Another tailored interpretation example involves a Southwestern tribal community, where the incidence of binge alcohol and drug use was high among AIAN youth (Mullany, et al., 2009; Barlow, et al., 2012; Tingey, et al., 2012). The research team used qualitative methods to gain interpretative insights directly from youth who revealed that binge use was often done in isolation and served to block the pain and memory of past traumas. Reflecting on these findings, community advisors suggested that youth binge substance use was a form of self-injury and suicide. Insights gained from this collaborative, community-specific interpretative process led to a body of work on the intersection of binge substance use and suicide risk (Tingey, et al., 2012; Cwik, et al., 2015) and spawned prevention programs targeting substance use and suicide simultaneously.

Three examples of risk and protective factors.

Accurately measuring substance use patterns in AIAN populations is necessary but not sufficient for understanding, preventing, or treating problematic use. Accurate measurement of substance use risk and protective factors is also critical. Such factors are often conceptualized as latent constructs – in other words, phenomena that are not directly observable, such as attitudes, beliefs, and perceptions. The measurement challenges therefore become even more complex. We provide examples of three commonly researched risk and protective factors to illustrate the framework: 1) depression, 2) parenting; and 3) AIAN culture.

Depression.

Major depressive disorder is one of the most commonly occurring mental disorders in the U.S. and is a known risk factor for substance use disorders (Kessler, Nelson, Mcgonagle, et al., 1996). Depression has culturally distinct meanings, correlates, and expressions across sociocultural groups (Nichter, 1981; Nichter, 2010), including AIAN communities (Kaufman, Beals, Croy, Jiang, & Novins, 2013; Manson, 2003), thus calling into question the conceptual and operational “fit” of common measures of depression for AIANs. For instance, ethnographic research with a Flathead community in Montana showed that “depression” was articulated in terms of multidimensional loneliness (O’Nell, 1998) and was deeply connected to Flathead ways of being that emphasize belonging. Standard diagnostic criteria were misaligned with local expressions; that is, depression was understood differently at the level of Conceptualization. Depression in contexts like these might therefore be Operationalized to include distinct features such as loneliness and belonging. As another example of Operationalization challenges, Stevens and colleagues (1999) found that negatively worded scale items (common to many depression scales) displayed poor psychometric properties compared to positively phrased items in an AI sample. They speculated that local cultural norms were such that tribal members avoided speaking or thinking in negative, undesirable ways. This might mean that questions about negative feelings would not elicit the information that researchers are trying to obtain with certain AIAN individuals/groups.

Structured diagnostic interviews for depression yield AIAN rates that are on par with, and in some cases, lower than the general population (Beals, et al., 2005a,b; Whitbeck, et al., 2006). Alternatively, non-diagnostic screeners generally reveal higher rates of psychological distress among AIANs (Barnes, Adams & Powell-Griner, 2010). One explanation for the discrepancy relates to retrospective timing. Diagnostic criteria frequently include symptoms during past 30-days, past year, and lifetime intervals and ratings of the impact on psycho-social functioning. Screeners typically have shorter retrospective windows (i.e., 2 weeks - 30 days) and may be especially sensitive to normative stress responses to the high rates of death, trauma, and negative life events in many AIAN communities (K. Walters, personal communication, April, 2014). “Standard” screener cutoff scores may also overestimate depression for AIANs (Manson, Ackerson, Dick, Baron, & Fleming, 1990). Ultimately, common measures, conceptualizations and operationalization of depression may poorly capture the patterns and expressions of depressive experiences and their impact on substance use in AIAN communities.

Parenting.

Family factors are associated with substance use among adolescents. Research with non-Native samples consistently demonstrates that higher parental warmth and monitoring is associated with fewer youth substance use and related problems (Hawkins, Catalano, & Miller, 1992; Ary, Duncan, Duncan, & Hops, 1999; Chassin, Curran, Hussong, & Colder, 1996). While there is some empirical evidence for the salience of parenting practices and family sanctions against drug use for AIAN youth development (Walls, et al., 2007; Swaim, Oetting, Thurman, Beauvais, & Edwards, 1993), unique family structures and patterns of influence need be considered for valid conceptualization and operationalization of parenting in AIAN contexts. Given the interconnected extended family networks and kinship roles common within AIAN communities, measures of parenting should include all adults who play a parenting role for youth (Whitbeck, Hartshorn, & Walls, 2014). For instance, while parents may not know their adolescent’s whereabouts after school, other family members may be fulfilling that monitoring role. Similarly, youth who have poor relationships with their parents may have warm, supportive relationships with grandparents, aunts, or uncles that are protective. Expansion of the parenting construct to include extended family-child relationships means that Operationalization must be tailored to include meaningful protective factors for youth substance use in AIAN communities (Whitbeck, Hartshorn, Walls 2014).

AIAN cultural factors.

Depression and parenting have been associated with substance use in the broader research literature and we have discussed how measures of these constructs may require tailoring for AIAN populations. Other potential risk and protective factors may be AIAN-specific and require the use of measures developed expressly for this population. For instance, there is increasing research on cultural factors that may be protective, including ethnic identity, traditional cultural involvement, and language revitalization (Institute of Medicine, 2013). Tribal communities have long embraced concepts of “culture as prevention” or “culture as treatment” (Gone & Calf Looking, 2011; Walters, Simoni & Evans-Campbell, 2002); yet operationalizing culture is a difficult task. “Culture” is frequently used synonymously with traditional/pre-colonial identity, practices, values or beliefs. Yet AIAN cultures are ever evolving, with extensive heterogeneity in postcolonial expressions of AIAN culture within and across communities. Many questions remain for community members and researchers alike. Should ”culture” be operationalized by participation in traditional cultural practices, engaging in spiritual ceremonies, speaking the tribal language, living by traditional values, or all of the above? Is “culture” reflected more in individual practice or identity, or does it have broader community-level meaning? How are traditional and contemporary cultures intertwined? Are multicultural identities qualitatively distinct or simply different points on a quantitative continuum? Questions such as these remain unanswered, and a single, clear-cut Conceptualization of “culture” is perhaps an inappropriate goal. Nonetheless, these are questions to be grappled with if we are to understand how culture impacts substance use and how culture can be employed to enhance the effectiveness of prevention, intervention, and treatment efforts.

Not surprisingly, operationalization of AIAN cultural factors is remarkably diverse, making it difficult to interpret overall effects of culture on health. Measures include indicators of multicultural or orthogonal identity (Oetting, & Beauvais, 1991; Cokley, 2007), racial self-actualization (Chae & Walters, 2009), identity attitudes (Walters, 1999), spirituality (Garroutte et al., 2003), worldviews like interrelatedness and value systems (Lowe & Struthers, 2001; Mohatt, Fok, Burket, Henry, & Allen, 2011), multi-dimensional enculturation including traditional activities, language, spirituality, and identity (Whitbeck et al., 2004), and factors impacting modern cultural expressions, such as historical trauma (e.g., boarding school lineage: Bombay, Matheson, & Anisman, 2013), and historical cultural loss (Whitbeck, et al., 2004). Community-level indicators of culture also exist in constructs like social capital (Mignone, Elias & Hall, 2011) and cultural continuity (Chandler & Lalonde, 2008). While some studies suggest that variables like identity (Wolsko, Lardon, & Mohatt, 2007) and spirituality (Stone, Whitbeck, Chen, Johnson & Olson, 2006; Kulis, Hodge, Ayers, Brown & Marsiglia, 2012) are protective against alcohol or drug use and related behaviors, others report null effects (Whitesell, et al., 2014; Whitesell, et al., 2009). Such unevenness in findings may reflect unevenness in the reliability and validity of cultural measures: some have not undergone psychometric analysis to document that they are trustworthy, and the interplay of multiple culturally relevant risks with protective factors may skew interpretation of results (Walls, Armenta & Whitbeck, 2016).

Another challenge straddles the line between Operationalization and Implementation and highlights the significance of tailoring. Consider the construct “cultural engagement” to reflect the extent to which individuals participate in the traditional culture practices of their tribe. Operationalization of cultural engagement might include questions about the frequency of cultural practices like engagement in spiritual ceremonies; speaking or learning the tribal language; eating traditional foods; participating in pow-wows, drumming groups, or tribal dancing; and practicing traditional arts. Such a measure could serve as an index of cultural engagement. The specific items to be included in an index, however, would need to be tailored extensively to tribal context because the specifics of cultural practices and even the existence of some exemplars vary widely. For example, in some communities, use of tribal language is pervasive, with many Native speakers engaged in everyday conversation; in other tribes, language has been nearly lost and is limited to songs, prayers, and ceremonies. In the latter communities, people may be very engaged and identify strongly with their culture. To “count” language use equally in these two settings would not provide evenly reliable or valid data on cultural engagement. The index approach might also exclude individuals who have strong cultural identities or beliefs, but whose engagement might be constrained due to practical, geographical, or other access barriers. Another approach might therefore be to assess cultural identification or beliefs independent of behavioral practices. At the Interpretation phase, local knowledge will further guide meaning and explanations for relationships between predictors and outcomes.

Community/research teams are making tremendous efforts despite significant conceptual, operationalization, implementation and interpretation challenges like those described above. We offered but three illustrations of the complex web of risk and protective factors that might impact substance use. The ongoing measurement work involving these and other constructs continues to bring us closer to a fuller understanding of important social determinants of health for AIAN communities.

Conclusion and the Way Forward

Getting measurement right is essential. This is true for all research, but particularly so for AIAN communities where rigorous measurement tools may not work as intended or have not been evaluated for cultural fit. Research cannot empirically identify health disparities (e.g., substance use/abuse) or risk and protective factors (e.g., depression, parenting, and culture) unless we have measures and methods that produce reliable, valid and relevant data. The framework proposed here offers a heuristic for approaching measurement in tribal research, emphasizing the importance of careful review and alignment of required specificity at each phase of the measure development cycle driven by an authentic community-research partnership.

There are important practical issues to be considered for this measurement framework to be used effectively. Community-engaged research, fundamental to this approach, requires considerable start-up time to promote authentic teamwork, build trust, and generate co-learning. These elements are essential for community and research team members to arrive at shared understanding of culturally and scientifically sound approaches to answer research questions. Competing demands such as project deadlines, responsibilities and priorities beyond the specific research project, and limited funding for measurement development impede this process. Partnerships should proactively budget time and resources for measurement development when planning research. The application of this measurement framework requires time, energy, and resources if it is to succeed. Increasing researchers’ and communities’ collective commitment to improving the quality of measurement in culturally diverse communities will ultimately strengthen research striving to reduce health disparities.

We conclude with four key tasks that can be undertaken now to improve overall quality of measurement in AIAN populations:

1. Catalogue and Assess Existing Measures.

We propose initiatives to catalogue what measures have been used with AIAN populations and describe degrees of tailoring for each. In this regard, the National Institute on Drug Abuse is funding an effort to produce a compendium of measures that have been used with AIAN populations in studies of substance use and abuse and related risk and protective factors. It may also be useful to document where specificity misalignment across the Measurement Cycle from Conceptualization to Implementation has hampered research efforts, or, where optimization has supported successes.

2. Invest in Measurement.

High quality measure development work is needed. The comprehensive review of measures noted above has already identified noticeable gaps in data regarding the psychometric soundness and predictive validity of measures currently used in substance use research. Resources are needed to support studies of the reliability and validity of existing measures for use with AIAN populations and for creating and validating novel measures targeting culturally salient constructs.

3. Create Effective Measure Development Strategies for AIAN Research.

Researchers and communities will benefit from frameworks like the one proposed here for deciding when and how to adapt versus create measures and in striking an optimal specificity balance. Clear guidelines for how to make these decisions can support the field (see, for example, Gonzales & Trickett, 2014), especially for researchers who have not worked with AIAN populations and for community partners who are not experienced in measure development.

4. Support Research Capacity within AIAN Communities.

Finally, authentic community-research partnerships are essential for improving measurement and research in tribal communities. The effectiveness of these partnerships is dependent on supporting research capacity within AIAN communities. This will include training researchers who are themselves members of both tribal and academic communities and thus uniquely suited to foster community-engaged, scientifically rigorous measure development work. Strong scientific methodology coupled with strong community involvement will be critical to distinguishing the next 30 years from the last, enabling researchers and communities to work together to find real solutions to health disparities that weigh too heavily on AIAN communities.