Advancing Scientific Methods in Community and Cultural Context to Promote Health Equity: Lessons From Intervention Outcomes Research With American Indian and Alaska Native Communities

Abstract

Evidence-based interventions hold promise for reducing gaps in health equity across diverse populations, but evidence about effectiveness within these populations lags behind the mainstream, often leaving opportunities to fulfill this promise unrealized. Mismatch between standard intervention outcomes research methods and the cultural and community contexts of populations at greatest risk presents additional challenges in designing and implementing rigorous studies; these challenges too often impede efforts to generate needed evidence. We draw on experiences with American Indian and Alaska Native (AIAN) communities to illustrate how consideration of culture and context can constructively shape intervention research and improve the quality of evidence produced. Case examples from a partnership with one AI community highlight opportunities for increasing alignment in intervention development, research design, and study implementation to maximize both validity and feasibility. We suggest that responsively tailoring intervention outcome research to cultural and community contexts is fundamental to supporting health equity.

The emphasis on using evidence-based programs and practices to improve health outcomes requires the kinds of methodological innovations in intervention research catalogued in this special issue of the American Journal of Evaluation. The populations at greatest risk for health inequities, including American Indian and Alaska Native (AIAN) populations, particularly require research methods that are both rigorous and flexible as the demand for trustworthy evidence from funders and communities alike grows. Standard intervention research designs are often impossible or impractical to implement within AIAN communities, due to small population sizes affording small sample sizes, distant and often remote locations, substantial cultural and linguistic diversity, community distrust of research, or tensions between scientific and cultural or community protocols (Beals, Manson, Mitchell, Spicer, & The AI-SUPERPFP Team, 2003). Consequently, credible evidence lags and communities are left without the benefit of rigorous science to inform their efforts to improve health equity. Drawing on our work evaluating preventive interventions with AI youth, we share lessons about partnering with communities to build evidence about programs and practices that, in turn, address health inequity.

Throughout the paper, we use the term intervention outcomes research (IOR) as both a special case of the larger intervention research enterprise and a special case of the larger evaluation enterprise. IOR unfolds at the intersection of the two. IOR examines the impact of interventions or programs on participant outcomes, typically first through efficacy studies and then through effectiveness studies; it is part of the larger intervention research field which also includes intervention development, process evaluation, and dissemination and implementation research. Our case examples focus on behavioral outcomes related to risk and resilience. While IOR draws on epidemiological research, and integrates some process evaluation methods in its design, it is primarily concerned with establishing a causal relationship between an intervention and an outcome (e.g. behavior), under either highly controlled conditions (efficacy) or when implemented in real-world settings (effectiveness).

This definition is central to our discussion: by establishing causality between intervention and outcome, IOR documents that a program can make a difference, a prerequisite to dissemination and implementation beyond the original setting. Disseminating and implementing evidence-based programs then becomes a strategy for promoting health equity. While these statements appear simple, they contain a number of assumptions that may result in increased disparities, especially among the most vulnerable.

In the following sections, we offer reflections on our intervention work with AI communities to “unpack” some of these assumptions. We begin by reviewing the broader context of IOR in AIAN communities, including a review of tribal community cultural and contextual considerations, and an overview of tribal experiences with research. We then review the implications of this context for identifying the critical elements of IOR with tribal communities. Next, we provide two case examples of tribal community-based intervention studies to illustrate specific ways in which tribal community contexts shape interventions, research design, and implementation. While these examples in many ways reflect the specific context from which they emerge, they also illustrate important considerations relevant in other AIAN cultures and contexts, as well as in other settings where researchers and communities are working in partnership to understand the impact of interventions within particular populations. In our concluding remarks, we consider the lessons we have learned and how they might inform and advance IOR with communities most in need.

The Context of Research in AIAN Communities

Cultural & Contextual Considerations.

AIAN communities are geographically, culturally, and linguistically diverse. In the 2010 Census, there were 5.2 million people in the United States who identified as AIAN, representing 1.7% of the population (U.S. Census Bureau, 2012). Tribal communities exist in rural and urban areas across the country, including reservations, tribal trust lands, tribal jurisdiction statistical areas, tribal designated statistical areas, and Alaska Native regional corporations and village statistical areas (U.S. Census Bureau, 1994). Their geographies encompass oceans, mountains, prairies, lakes, rivers, woodlands, and deserts which are closely tied to their traditional spiritual and cultural practices (Lane, 2001).

There are 566 federally-recognized tribes in the United States, but that number is dynamic (Federal Register, 2015). Tribes who existed long before the United States continue to seek “federal recognition” which acknowledges sovereign status and the benefits to be gained from federal trust obligations not afforded to tribes without this federal recognition (McCarthy, 2011). Such trust obligations include funding for education, healthcare, housing, and other services and supports, even while the adequacy of those funds remains woefully insufficient (Gone & Trimble, 2012; Jones, 2006; U.S. Commission on Civil Rights, 2003).

Despite the diversity among tribes, similarities exist. Nearly all tribal communities are deeply embedded in traditional family and kinship networks in which relational bonds extend well beyond those of the nuclear family (Red Horse, 1997). Tribal family and kinship networks provide a sense of place, belonging, and connection to the past as well as future generations (Sarche & Spicer, 2008). AIAN communities often look to their traditional cultural teachings and practices as a guide to health, wellness and balance (Hodge, Limb, & Cross, 2009). And nearly all find that their tribal language holds the key to their culture, and therefore, must be nurtured and, in many cases, revitalized in the face of its loss (McCarty, Romero, & Zepeda, 2006).

Family, kinship, culture, and language have buoyed tribal communities for the last 500 years against numerous, orchestrated, and massive assaults to their integrity (Cross & Blackstock, 2012). Suffering disease, warfare, and outright massacres in the face of European efforts to colonize their land, tribal populations were drastically reduced between the time of first European contact and 1890, which, with the massacre at Wounded Knee, marked the end of the “Indian wars.” U.S. efforts to address the “Indian problem” then turned to attempts at the annihilation of AIANs as a culturally-distinct population (Hays, 1997; Thornton, 1987). These efforts have been well-documented and ranged from grand and obvious to more subtle and insidious. Among the grand and obvious efforts was the forced removal of AIAN children to government and religiously run boarding schools which were directed to “kill the Indian and save the man” by severing children’s ties to family, community, and culture (Adams, 1995; Bess, 2000). These efforts also included actions that eroded tribal land bases through efforts like the Dawes, or General Allotment Act, which divided collectively-held tribal lands into individual allotments, leaving “surplus” land open to settlement by non-natives (Otis, 1973). Among the more subtle and insidious have been efforts to withhold from tribes or tribal citizens what they are due through treaty rights and obligations, such as efforts to limit or prevent entirely the exercise of traditional treaty rights, curtailing religious freedoms, federal mismanagement of funds and resources held in trust for tribal nations and citizens, or environmental degradation of tribal lands by outside interests (Harjo, 2004; Merjian, 2010; Whaley & Bresette, 1994).

The collective impact of these attempts at physical, spiritual, and cultural annihilation has been termed “historical trauma” – described by Yellow Horse Brave Heart as the “cumulative emotional and psychological wounding over the lifespan and across generations, emanating from massive group trauma experiences,” (Yellow Horse Brave Heart, 2002). Historical trauma is considered at the root of the tremendous disparities in health and well-being experienced by tribal communities and citizens today who, by almost any indicator, fare worse than White Americans, and often worse than other groups that also suffer health and socioeconomic disadvantage (Jones, 2006; Yellow Horse Brave Heart, Chase, Elkins, & Altschul, 2011). Suicide, substance use, exposure to violence, adverse childhood experiences, and mental health concerns such as post-traumatic stress disorder have all been documented at higher rates in tribal populations relative to others (Beals et al., 2013; Brockie, Dana-Sacco, Wallen, Wilcox, & Campbell, 2015; Jiang, Mitran, Minino, & Ni, 2015; Rosay, 2016; Spicer et al., 2003).

AIAN communities experience contemporary structural disadvantages that further drive the breadth and depth of these health disparities. Health, education, and legal systems serving tribal communities are vastly under-funded relative to need (Indian Law & Order Commission, 2013; Jones, 2006), and the quality of food, housing, and outdoor spaces to exercise and play are often inadequate (Bauer et al., 2012; Edmunds et al., 2013; Zephier, Himes, Story, & Zhou, 2006). With such basic structures falling short, it is not surprising that AIANs experience among the greatest rates of poverty and lowest levels of education compared to other groups in the United States (National Center for Education Statistics, 2008, 2014).

Tribal Experiences with Research.

Interest in AIAN culture and a desire to contribute to an understanding and amelioration of the physical and mental health disparities experienced by tribal communities has led Native and non-Native researchers from the social and behavioral sciences to carry out studies that rely on the methods of their fields (Walls, Whitesell, Barlow, & Sarche, 2017). Willingness to trust in the researcher(s) and believe in the potential of social and behavioral science methodologies to address the pressing concerns of their communities has led tribal communities to enter into research partnerships.

In their research partnerships, tribes and tribal entities hold researchers to practice standards that demand accountability to the direct and often immediate interests of the community and its citizens (Norton & Manson, 1996). Exercising their inherent rights to self-determination and governance as embodied in US Treaty law and legislation, these tribal standards for research accountability reflect efforts to protect against the kinds of cultural, community, and individual research harm that have resulted in the past, as well as to promote clear and lasting benefit (Holkup, Tripp-Reimer, Salois, & Weinert, 2004; Norton & Manson, 1996; Schnarch, 2004).

Tribes engage in government-to-government relationships with the US federal government, and as such, are in a strong position to advocate for national research priorities and policies that reflect their interests. Major federal agencies such as the National Institutes of Health, the Centers for Disease Control and Prevention, the Substance Abuse and Mental Health Services Administration, and the Administration for Children and Families all engage Tribal Advisory Committees (TACs). TAC members (often tribal government leaders) work to inform federal agencies of tribal health needs and priorities, and to ensure that federal tribal health policies and funding priorities align (U.S. Department of Health & Human Services, 2010).

Tribes also directly oversee research that happens within their communities (Harding, Harper, Stone, & al., 2012; Sarche, Novins, & Belcourt-Dittloff, 2010). Oversight takes many forms, including tribally-run Institutional Review Boards that conform to all Department of Health and Human Services Code of Federal Regulations guidelines as well as non-IRB tribal research review boards. When tribes do not maintain an IRB or tribal research review board themselves, they may partner with other entities, including the Indian Health Service national or area IRBs or tribal college IRBs. They may also rely on a tribal health board or tribal council to provide tribal review and approval of research.

Tribal approval of research generally includes a provision for tribal ownership of data, which may include physical ownership , or, more often, intellectual ownership (Harding et al., 2012). In practice, this means that researchers are entrusted as stewards of the data who, among other things, must work closely with their tribal partners to ensure that study findings are reported in ways that are sensitive to tribal concerns such as maintaining community confidentiality, identifying strengths and solutions in addition to problems, and situating findings within an appropriate cultural frame (LaFrance & Nichols, 2009).

Tribal control and ownership of the research process and its products have put tribes at the forefront of community-engaged and community-driven research (Tribal Evaluation Workgroup, 2013). In addition to helping set broad national priorities for research and regulating research within their communities, tribes have also pushed the substantive focus of research by calling attention to the importance of culture and context in research. This has required researchers to revisit research methods, including measurement strategies (Walls et al., 2017). Research measures have rarely been developed with tribal communities in mind. Existing measures may be adapted based on community input, or, much less frequently, new measures may be created from the ground up based on community-identified constructs of relevance. However, such measures are then subject to scrutiny of reliability and validity as well as limited comparability. Similarly, methods that are grounded in academic research approaches (e.g. randomized assignment to treatment groups) may not be culturally acceptable (e.g. seen as withholding potentially helpful intervention in communities in critical need) and must be adapted.

Community-based participatory research (CBPR) approaches represent best practices and are widely accepted as the sine qua non of research with tribal communities (Fisher & Ball, 2003; Israel et al., 2003). CBPR principles provide guidance for researchers seeking to partner with tribes to conduct research (intervention and otherwise) that seeks to promote the health and well-being of tribal communities and citizens while acknowledging the historical roots of current disparities (Wallerstein & Duran, 2006). While the full realization of CBPR methods is rare in research with AIAN communities, particularly given the constraints of funding priorities and grant timelines, it represents an ideal that guides research in partnership between universities and communities seeking to improve health equity for AIANs.

How Context Shapes Intervention Outcome Research in AIAN Communities: Overarching Themes

This historical, cultural, and contextual fabric of AIAN communities shapes the way IOR can and should be conducted. In this section, we discuss approaches to tailoring academic research methods to increase their feasibility, fit, meaning, and, ultimately, rigor for tribal communities. As the above discussion suggests, the process of tailoring must unfold in close collaboration with tribal community partners in accordance with the principles of CBPR.

Indigenous traditions of evaluation.

Scientific evidence, for most who employ academic research approaches, is produced as a result of specific research methodologies, including RCTs, interrupted time series, or pragmatic trials. Such methodologies aim to maximize internal and external validity and demonstrate the efficacy and effectiveness of a program or intervention. Producing evidence of effectiveness is, of course, vital: In the context of scarce resources, there is an imperative to invest in programs that work. Indeed, numerous journal articles, websites, and federal efforts, including other articles in this special issue, delineate levels and strength of evidence used to assess effectiveness of medicines, interventions, or programs intended to improve health (Gottfredson et al., 2015; Substance Abuse and Mental Health Services Administration, 2017; The Cochrane Collaboration, 2017). By using these criteria to locate research findings along a continuum of weak to strong evidence, communities are able to make informed decisions about the plausible impact of a given program. Using a program demonstrated to have an impact is paramount for communities struggling to meet even basic needs of their people. However, within the historical context of AIAN traditions of observation and experiential knowledge generation – sometimes accrued over centuries – considering only evidence derived from and judged by academic research approaches imposes a singular and limited view.

Generations of evidence about the dynamics of human behavior over the life course within kinship groups, across communities, and over time have formed the foundation of cognitive, behavioral, and spiritual instruction within AIAN communities (LaFrance & Nichols, 2009). Examples include the Seven Teachings of the Ojibway, comprised of the universal concepts of love, honesty, respect, kindness, humility, bravery, and wisdom, which provide guidance for good living and conduct towards others (Bouchard D, 2009). The Medicine Wheel, a symbol common across many tribal Nations, is a circle divided into four equal parts, representing spiritual, emotional, mental, and physical wellness as essential aspects of health and well-being; harmony and balance among the four parts are believed to promote holistic health and personal, kin, and community well-being (Roberts, Harper, Bull, & Heidemann-Provost, 1998). These cultural frameworks have been built on evidence that is internally and externally valid, derived from indigenous methodologies of observation, experimentation, evaluation, and replication (Gone & Trimble, 2012). Despite their different roots, these frameworks overlap with many academic research approaches. Used in combination with methods from the academic paradigm, they hold promise to strengthen the quality of data and the validity of conclusions drawn from IOR. Identifying the overlap between academic and traditional indigenous evidence, and triangulating diverse forms of research evidence is challenging, rewarding, and absolutely necessary.

Creative research designs to support rigor and feasibility in tribal contexts.

AIAN communities widely embrace the imperative to generate rigorous evidence to support – or refute – the effectiveness of programs. However, a employing purely academic research and evaluation designs and methods often limits the generation of evidence for communities that are most in need. Research is disproportionately conducted with mainstream and urban communities, where many aspects of academic research infrastructure are supported. Demonstrated effectiveness under circumstances typical among mainstream populations, however, may not be useful to lower-resourced or culturally distinct communities. Over time, preference for involving resourced communities in research, although perhaps more expedient than intentional, is likely to widen health equity gaps.

Programs whose evaluations meet strict academic criteria for evidence of effectiveness are included on established lists of evidence-based interventions maintained by the Department of Health and Human Services, such as SAMHSA’s National Registry of Evidence-based Programs and Practices (NREPP) and the Administration for Children and Families Home Visiting Evidence of Effectiveness review (HomeVEE). A primary goal of reviews for lists such as NREPP and HomeVEE is to make information about evidence-based practices available to communities and to promote the use of these strategies to improve health outcomes. Programs included on these lists, however, often lack evidence of effectiveness within the most vulnerable communities. Additionally, communities without the resources to adapt programs from the list or to evaluate the effectiveness of a program within their population may be disinclined to use any program on the list and thus unlikely to reap any benefit from the accumulated evidence. Evidence-based programs for AIANs, in particular, are scarce; AIANs are not typically included in research in sufficient numbers to support subgroup analyses and specific studies of interventions within AIAN communities are few (Mohatt, Fok, Henry, People Awakening Team, & Allen, 2014; Walkup et al., 2009).

Exacerbating the lack of evidence specific to disadvantaged communities are the very criteria used to add programs to these lists. While federal review of evidence and the criteria used to judge evaluation methods provide important standards to ensure scientific rigor, the standards may inadvertently yet systematically exclude completely relevant and appropriate evaluation findings. Studies drawing on research designs that are responsive to the contexts of under-resourced communities may be excluded since they do not meet very specific “inclusion” criteria. For example, the Office of Adolescent Health evidence review excludes results of subgroup analysis of age and race/ethnicity categories. This exclusion arises out of concern for inflated error related to multiple comparisons, but may have the effect of dismissing relevant and rigorous evidence that can be critical to understanding the impact of interventions across the developmental spectrum (e.g., enhanced impact of prevention efforts earlier in adolescence) or within specific ethnic/cultural populations, such as AIAN communities.

Evidence review criteria codify specific methods and often provide guidance for strong IOR, but can also have unintended consequences in some situations. To ensure eligibility for evidence review, researchers designing studies may shy away from innovative approaches, or, perhaps more importantly, from methods that are responsive to particular cultural contexts. The result may be an academic research design force-fit into a context where it cannot, ultimately, produce rigorous evidence. That is, the evaluation may meet requirements for inclusion of review, but the findings produced within strict regulatory framework may no longer be relevant to the setting. For example, interventions implemented with fidelity to a parent model, un-adapted or only superficially adapted for a particular community, may be awarded high marks for scientific rigor, but lack of authenticity will compromise the cultural rigor and therefore diminish the relevance of study findings. Another example occurs when research incentives inconsistent with community values are used to support enrollment or retention; they may effectively help meet review criteria (e.g., adequate retention) while undermining validity. This is a loss to advancing IOR in diverse settings across distinct cultural populations and undercuts the community-research pact CBPR approaches embody. As researchers committed to honoring community perspectives in research, narrow definitions of what “counts” as evidence constrain our ability to engage in genuine partnerships and to design studies that build credible evidence.

Standards of evidence for documenting intervention efficacy and effectiveness lean heavily on RCT designs (Gottfredson et al., 2015). The scientific peer review process for both funding and publication often prioritizes RCT designs, equating them with rigor without careful consideration of whether or not they are the best approach to ensuring it. Despite this pressure toward RCT designs, they are not necessarily the best choice for evaluating the effectiveness of an intervention. Indeed, across disciplines, from medicine to education, scientists are increasingly questioning the elevated status of RCTs as the “gold standard” for evidence of intervention efficacy and effectiveness and the value of a narrow focus on internal validity (Bothwell, Greene, Podolsky, & Jones, 2016; Thomas, 2016).

A critical consideration in the selection of an appropriate research design should be the context in which it will be implemented and the ability or adaptability of given methods to produce high quality evidence within that setting (Kirkhart, 2010; LaFrance, Nichols, & Kirkhart, 2012). High quality evidence is best built through thoughtful consideration of an array of study designs, weighing the advantages and disadvantages of each within the context and constraints of the study population and setting. Rigorous science must occur at the highest level in consideration of community context, rather than in mechanical response to a checklist of elements required for standard academic research methods.

How Context Shapes Intervention Research in AIAN Communities: Case Examples

To illustrate how rigor can be achieved in IOR with AIAN communities, despite the challenges, and thus support efforts to promote health equity, we offer specific examples from two intervention studies we conducted with AI youth and families in Northern Plains’ reservation communities. These examples illustrate how culture and context shape IOR from intervention development to research design, and finally to implementation. Respecting AI culture and context in this way in turn supports the generation of valid data and trustworthy conclusions about intervention effectiveness. We provide some context specific to these communities and a brief overview of each study, and then draw specific examples from each case to illustrate how the tribal culture and context shaped the intervention, research design, and implementation.

The reservation covers a large land area, characterized by small towns and villages separated by great distances; many people live outside even these small population centers (out of respect for confidentiality for the community, we do not name the specific reservation involved). Distance to services such as schools, shopping, and health care are vast for many people, and lack of transportation is a significant barrier. Poverty and unemployment are high. Health disparities are significant. Historical and current trauma shape everyday experiences. In short, significant challenges compromise the health and well-being of reservation residents.

Amid these challenges, the strengths of the community are many. This reservation is characterized by a strong culture and a long history of tribal values and traditions. These have enabled the culture to survive the systematic attempts at cultural eradication described above; strength and resilience are evident in the very existence of the tribe through this long and disturbing history. They are also evident in ongoing efforts to uphold tribal sovereignty, revitalize tribal language, sustain cultural practices, promote traditional values, and adapt and grow with the current environment.

The Wiconi Teca Waste project.

The Wiconi Teca Waste project, examined the effectiveness of Circle of Life (COL), a school-based sexual risk reduction (STD/HIV prevention) program developed specifically for AI middle-school youth. All thirteen middle schools on the reservation participated in a three-year group-randomized wait-list comparison study. In the first year of the study, two waves of baseline data were collected from 6^th and 7^th graders in all schools. In the second year, seven randomly selected schools received the COL intervention; the remaining six schools received the intervention in the third year. Trained community members delivered COL to groups of students in their classrooms during the regular school day. Participating youth were surveyed semi-annually throughout the study, for a total of eight waves of data collection. Results demonstrated significant delays in onset of sexual initiation among youth who received COL, with the strongest effects seen among the youngest participants (ages 10, 11, 12) (Kaufman, 2010; Kaufman, 2014).

The Thiwáhe Gluwáš’akapi project.

The second study, Thiwáhe Gluwáš’akapi, emerged out of community responses to findings from the Wiconi Teca Waste study on early adolescent substance use. Of particular concern were data showing early initiation of substance use among youth. The goal of the Thiwáhe Gluwáš’akapi study, now in its fifth year, was to build a tailored substance use prevention program for middle school youth through a rigorous process of cultural adaptation, implementation and evaluation of the relative effectiveness of program components. Thiwáhe Gluwáš’akapi is an adaptation of the Iowa Strengthening Families Program for Parents and Youth 10–14 (SFP 10–14). It incorporates core components from SFP 10–14 with components reflecting the Tribe’s strong cultural kinship systems. Thiwáhe Gluwáš’akapi is delivered through a seven-week series of sessions held with groups of families (typically 6–10 families). Each session begins with a meal shared with all families, followed by concurrent youth and adult sessions. Each night ends with a family session that allows youth and adults to share what they have learned and practice skills together. Thiwáhe Gluwáš’akapi is being implemented within communities across the reservation, rotating across communities to implement three to five groups of families in geographically dispersed areas each fall and spring.

Both of these studies, like all of our work in partnership with AIAN communities, have highlighted the importance of partnering with the community in both the design and implementation of IOR. They have also demonstrated the importance of involving community research staff in all aspects of the study, from design through intervention implementation, data collection, and dissemination. Both Wiconi Teca Waste and Thiwáhe Gluwáš’akapi employed research staff in a university field office on the reservation who were responsible for the implementation of both intervention and data collection. The intervention adaptation and program implementation of Thiwáhe Gluwáš’akapi was led by the second author, a doctoral-level researcher from the community who is a university faculty member working out of the field office on the reservation.

In the sections that follow, we illustrate how partnerships with community members have played integral roles in shaping intervention development, research design, study implementation and ultimately the quality of data collected.

Shaping intervention development.

Community partnership plays an essential role in the creation of interventions to be tested with AIAN populations. Ideally, interventions are created and evaluated within the community they are intended to serve. The development of COL, for example, embraced this approach. This culturally-grounded curriculum, intended for use across diverse AIAN communities, was the result of years of input from tribal leaders, community members, youth, and education and health specialists across the nation prior to evaluation (Kaufman, 2012). This approach, however, is costly and time-consuming. As a result, few evidence-based programs designed specifically for AIAN people exist; extant evidence-based interventions must be adapted to align with the local culture, teachings, and ways of learning.

The incorporation of local teachings must remain true to the key components of an evidence-based program and true to the meaning and message of the local teachings. This requires time, open communication, and flexibility. In the adaptation of SFP 10–14 to create the Thiwáhe Gluwáš’akapi program, we utilized a systematic process to develop a culturally focused intervention program that included deep-structure cultural adaptation (Okamoto, Kulis, Marsiglia, Steiker, & Dustman, 2014). In this process, we engaged in extensive consultation with community representatives (Community Advisory Board (CAB) members, focus groups with parents and youth, survey of broader community) to ensure fit with community diversity, and worked with the original program developers to ensure fidelity to core intervention components. Specifically, this community, like most AIAN communities, was not homogenous with respect to culture. It included members with diverse cultural experiences: Those able to speak tribal languages fluently and those who understand just a few words; and those whose spirituality is guided by traditions, by organized religion, or a combination of both. Over the course of many conversations with the CAB, we arrived at an approach they believed would benefit families across these different perspectives: Include a strong emphasis on traditional kinship systems as fundamental to strengthening family connections and youth resilience, and incorporate tribal language. The developers affirmed that the addition of these elements did not compromise the core components. Our intensive, reciprocal, dynamic process of fitting intervention to community required time and resources not often accommodated in common funding mechanisms. Often, adaptation consumes considerable time up front, leaving less time for study implementation, data collection, and analysis.

Shaping research design.

Community perspectives are vital to shaping research design to ensure reliability and validity of data and to support sound inferences about intervention effectiveness. In the case of the Wiconi Teca Waste study, we planned a group-randomized wait-list control trial of COL. As in most RCT designs, youth recruited at baseline were intended to be the study sample. However, the Tribe requested that we continue to enroll youth in the study during the two intervention years. This adjustment recognized that youth in their community sometimes do not attend school for a period of time for various reasons, including family mobility on and off the reservation or caring for sick or disabled relatives. Additionally, the Tribe recognized that youth with transient school attendance may be in particular need of a preventive intervention. We agreed. We allowed youth to enroll in both experimental and comparison groups at any time prior to completion of the intervention, a period of about 3 years (or 6 waves of data collection). To adapt to the Tribe’s request, we had to relax standards of random assignment and inclusion, and accept that baseline results were likely impacted by the fluid nature of the sample. However, we also recognized that school attendance was unlikely related to assignment arm of the study (i.e., immediate intervention or wait list). We were able to capitalize on advances in missing data procedures and thus accommodate the Tribe’s request without compromising scientific rigor.

Another example of the community shaping research design comes from the Thiwáhe Gluwáš’akapi project. This project emerged from consultation with a CAB convened to help us understand epidemiological analyses of pre-intervention data from the Wiconi Teca Waste study that documented early adolescent risk for substance use initiation (Whitesell et al., 2014; Whitesell et al., 2012). Consistent with principles of CBPR, we shared findings with the CAB and asked for their insights in interpreting them. The CAB was quick to urge us to move from documenting troubling patterns of early substance use to identifying solutions. We worked closely with these advisors for more than a year to review interventions with documented effectiveness in other populations, identify an intervention they believed would be a promising fit for youth and families in their community (SFP 10–14), and plan a study to culturally adapt the intervention for this community and evaluate its effectiveness. In that process, we considered and rejected a number of potential study designs. Withholding the intervention from youth and families in a control group, a basic feature of RCTs, was a sticking point. Wait-list designs were not considered good alternatives because wait-listed youth would not receive the intervention in time to prevent early onset of substance use, a stated focus of intervention by the CAB; delayed intervention would be too late. The CAB saw wait-listing as particularly problematic given the strong evidence base for SFP 10–14 in other populations (Spoth, Redmond, Mason, Schainker, & Borduin, 2015). Thus, it was difficult to justify withholding the intervention from a control group on ethical grounds. Concomitantly, we were making substantial cultural revisions and recognized the need to carefully examine the effectiveness of new components being added as well as of existing core intervention components being considered for removal. Since SFP was evidence-based for another population, we agreed that our primary goal was to generate data to inform the development of a parsimonious and optimally effective and culturally tailored intervention for AI youth. We thus chose a study design best suited for these questions, the Multiphase Optimization Strategy design (MOST) (Collins et al., 2011). Using the MOST approach and a fractional factorial design, we delivered the intervention to all participating youth and families but randomly assigned intervention groups to receive different versions of the intervention, allowing examination of the impact of particular components. In the end, barriers to standard effectiveness study designs in this community led us to consider an alternative design that was more appropriate to the core study questions.

Is some cases, community preferences reach beyond study design to include requirements for study protocols. In Wiconi Teca Waste, for example, the Tribe requested that parental consent be obtained in person by trained research staff, rather than through consent forms sent home from school to parents. This request came from the Tribe’s recognition that research consents can be difficult to understand, and that the intervention focused on a sensitive topic, sexual risk reduction. They wanted to ensure that parents heard about the project in terms they could understand and had the opportunity to ask questions. We accommodated this request; the Tribe was correct in ensuring that participants were truly providing informed consent. It was daunting and expensive to accomplish this – field research staff met with more than 800 parents and guardians dispersed across nearly 3500 square miles. The process, however, proved uniquely valuable. By obtaining parental consent in a transparent and personal way, we built relationships that instilled confidence within the community about the project, which resulted in higher rates of participation in the study and higher quality data.

Shaping study implementation.

Community contexts impact both the implementation of the intervention itself and the implementation of study protocols around data collection. Here we highlight three examples where context shaped implementation: Local research workforce, university and funder policies, and lack of transportation.

Regarding the local workforce, the contribution of well-trained staff working within AIAN communities cannot be understated. In many rural communities, the lack of educational and employment opportunities supporting research make it difficult to recruit trained personnel. Investment in job training is critical for program delivery and data collection efforts, but also enables community members to contribute in key ways to the development of interventions and data collection systems. Local research staff provide valued input and perspectives on all aspects of the work. They play pivotal roles in everything from identifying priority research questions, to translating language, to connecting and working with local residents. Once hired and trained, retaining staff who are well-versed in research methods, local culture, and community resources can be difficult because of reliance on soft money, with grant-supported projects and funding cycles that often result in funding gaps. Data analytic work – key to the dissemination of results – can become compressed at the end of the project due to the time required at study startup for time-intensive adaptation and staff research training. The necessity of designs that incorporate rolling enrollment to accommodate small populations further makes it difficult to have evidence ready in time to apply for the next funding cycle to secure uninterrupted funding. When funding gaps occur, well-trained staff are often lost to other employment. When new funding is obtained, the process of hiring and training staff must often begin again. Longer project periods (perhaps six or seven years rather than the standard five) should be considered for community-based IOR.

Trained and continuously-supported local staff contribute not only to research protocol integrity and adherence to human subject protections, but also to the continuity of identified community needs. Staff who become familiar with granting systems are able to map the federal research priority areas to community goals, helping to bridge research and community priorities. In addition, local staff are in the best position to make referrals for participants in the limited and ever-changing landscape of reservation services. Supporting the long-term employment of experienced research staff strengthens the foundation of community-based research, sustainable community intervention, and feasible health equity.

While discussions of the challenges of community-based work often focus on relatively weak community research infrastructure, inflexible university and funder policy also contribute to difficulties with executing strong research protocols in communities. For example, tribes often have the ability to pay consultants, such as members of community advisory boards, immediately; university procedures for payment can take in excess of several weeks. In communities where poverty is prevalent and financial resources are generously shared, waiting for payment can be an impediment to participating in the process. We have lost valuable community consultants over frustration with these payment systems. Similarly, fiscal policies which require cumbersome tracking procedures and inflexible payment systems for participant compensation may compromise sample retention. These procedural mismatches can create tension, and require collaborators (funders, researchers, and AIAN communities) to continue to adapt and think in innovative ways to enable IOR in AIAN communities.

Lack of transportation is a very real challenge in many AIAN communities. The rural nature of reservation life requires residents to travel long-distances for work, school, shopping and even social connection. This reality often shapes the nature of interventions that can be implemented and the representativeness of the samples that can be recruited. In urban environments, projects can provide bus tokens to enable families to attend programs. This is not feasible in a rural setting. For Thiwáhe Gluwáš’akapi, for example, we explored the possibility of providing bus vouchers to families to attend sessions. There is a limited public transportation system on the reservation, but service did not run late enough in the evenings to accommodate program sessions. One school partnered with us to provide transportation to participating families in concert with other evening activities at the school, and we realigned session schedules to take advantage of that transportation option. Alternatives such as having staff drive families to and from sessions were not allowed by university liability policies. Purchasing a van that would accommodate the number of participants who need rides is an expensive and often not allowable option under grant funding. The result, unfortunately, was that we were unable to accommodate families who wanted to participate but did not have the resources for transportation.

Implications

The studies we described above illustrate how the context of IOR and, in particular, the demands of both academic and community research protocols play a pivotal role in shaping research. They reflect lessons learned in partnership with AIAN communities, but they matter for other communities as well. It takes time and resources to thoughtfully and meaningfully adapt culturally consonant interventions and additional time and resources to build the body of evidence on effectiveness in underserved communities. Relevant research design must be responsive to cultural and contextual considerations. These considerations both give rise to innovation and ultimately support both scientific and cultural rigor, the foundations of high quality IOR. Commitment to rigorous methods helps ensure that the evidence produced is reliable and valid, that it provides trustworthy answers to questions of intervention effectiveness. Over-reliance on particular rigorous methods, however, can actually undermine the value of data when these methods are applied within contexts for which they are a poor fit. Utilization of a design in a community where it is either unethical or impractical will likely result in biased sampling, faulty data, and something that is very far afield from rigorous science.

Community-based IOR occurs within systems at funder, university, and local levels. The alignment of priorities, resources, and policies across these players will support high quality research and evidence but is often overlooked or avoided in IOR methods. Ethical concerns about evidence review processes and the implications of this process for minority and low-resourced communities present clear challenges for achieving health equity. These challenges, however, can be addressed. Community engagement with scientific methods can produce credible evidence. Evidence reviews can accommodate community-based methods in their processes. These efforts will strengthen our ability to advance scientifically rigorous IOR and improve health outcomes for the most vulnerable, and advance important principles of scientifically rigorous IOR.