Integrating Community-engaged Adaptation and a Multiphase Optimization Strategy Approach: Designing Substance Use Prevention for American Indian Families

Abstract

Initiation of substance use often occurs earlier among American Indian (AI) youth than among other youth in the United States, bringing increased risk for a variety of poor health and developmental outcomes. Effective prevention strategies are needed, but the evidence base remains thin for this population. Research makes clear that prevention strategies need to be culturally coherent; programs with an evidence base in one population cannot be assumed to be effective in another. However, guidance on effective adaptation is lacking. This paper reports on cultural adaptation of an evidence-based program utilizing the multiphase optimization strategy (MOST) approach embedded within a community-engaged process to evaluate intervention components. The Strengthening Families Program for Parents and Youth 10-14 was adapted to become the Thiwáhe Gluwáš’akapi Program for American Indian youth and families. Three program components were evaluated for their effectiveness with regard to outcomes (youth substance use, theoretical mediators of program effects on substance use, and program attendance) in a sample of 98 families (122 youth and 137 adults). Consistent with the MOST approach, the value of components were also evaluated with regard to efficiency, economy, and scalability. Expanding on the MOST approach for cultural adaptation, we also considered the results of the MOST findings regarding the acceptability of each component from the perspectives of community members and participants. The promise of a strategic component-based approach to adapting evidence-based interventions is discussed, including the benefits of engaging community to ensure relevance and considering both cultural and scientific rationale for each component to enhance impact.

Research has documented disproportionate rates of substance use disorder among American Indians (AIs) compared to national samples (Grant et al., 2016; Whitbeck, Walls, & Hartshorn, 2014). Early initiation of substance use is a significant risk factor for later problematic use and disorder (Kunitz, 2008; Whitesell, Beals, Mitchell, Manson, & Turner, 2009) and use starts earlier among AIs compared to others in the U.S. (Whitesell, Beals, Mitchell, Novins, et al., 2007). In the Northern Plains tribe where this study was conducted, more than half of adolescents had used marijuana and more than a third had used alcohol by age 13 (Whitesell et al., 2012).

Early substance use is also associated with a long list of concurrent problems that threaten successful development and, in the extreme, put adolescents at risk of not surviving to adulthood. These problems include suicide, risky sexual behavior, car accidents, alterations in brain morphology and activity, psychopathology, antisocial behavior, and school failure (De Ravello, Everett Jones, Tulloch, Taylor, & Doshi, 2014; Lisdahl, Gilbart, Wright, & Shollenbarger, 2013; Moss, Chen, & Yi, 2014; Volkow, Baler, Compton, & Weiss, 2014). AI youth face disproportionate rates of many of these problems as well, including suicide and sexual risk (Centers for Disease Control and Prevention, 2010; Kaufman, et al., 2010). Thus, preventing substance use during early adolescence has the potential to impact a host of health and developmental disparities, both concurrent and prospective. In response to this evidence for the need for early intervention, efforts to prevent substance use and related adolescent risk behaviors are ongoing around the country in both urban and reservation AI communities. Unfortunately, evidence regarding effective prevention strategies is lacking for this population (Whitesell, Mousseau, Keane, Sarche, & Kaufman, 2017). Communities thus must create their own untested programs or rely on programs documented to be effective in other populations. However, program implementation without adaptations or with superficial adaptation are likely to be less effective (Barrera, Castro, Stryker, & Toobert, 2013). Deeper alignment with culture is often necessary to engage effectiveness (Okamoto, Kulis, Marsiglia, Steiker, & Dustman, 2014). It is not always clear, however, how to best adapt programs to meet the needs of particular cultural communities. Rigorous methods are needed to systematize adaptation–a “science of adaptation”–to ensure both intervention integrity and cultural fit (Castro & Yasui, 2017).

We utilized two complementary methods that support effective adaptation. First, we engaged community in the initial review and revision of an evidence-based program (EBP), to align it with local culture and context while maintaining fidelity to the EBP. Second, we conducted an optimization trial, using an expanded multiphase optimization strategy (MOST) approach to gather data about potential adaptation components; this step was intended to help inform decisions about components to include in the final program. We provide an overview of the community-engaged context of our work below, followed by a description of the how we applied the MOST approach within this context in the optimization trial.

Community-Engaged Process

The preventive intervention work described here grew directly out of epidemiological and etiological work with a Northern Plains reservation community, demonstrating early initiation of substance use among adolescents (Whitesell et al., 2014; Whitesell, Beals, Mitchell, Novins, et al., 2007; Whitesell et al., 2012). In that research, we worked closely with a Community Advisory Board (CAB) that reviewed project findings, helped researchers interpret the data (Whitesell et al., 2012), and urged us to move beyond describing the problems to finding solutions to benefit the community. The research team identified evidence-based interventions for consideration, presented them to the CAB for discussion, gathered additional questions, and reconvened with additional information or options as needed; this iterative review process took about a year. The Iowa Strengthening Families Program for Parents and Youth 10-14 (SFP 10-14) was ultimately selected as the best option, although adaptations were deemed necessary. SFP 10-14 is a family-based early substance use prevention program, delivered to youth and their parents in seven weekly meetings consisting of youth and parent sessions followed by a family session. The emphasis is on building skills in both parents (e.g., communication, nurturing, limit-setting), and in youth (e.g., goal setting, peer resistance) (Molgaard, Spoth, and Redmond, 2000). Facilitators lead sessions with both parents and youth using a combination of videos, interactive discussions, activities, and games designed to teach skills and provides opportunities for practice.

Multiphase optimization strategy approach (MOST)

The MOST approach specifies three phases for the effective and efficient development of interventions (Collins, 2018). In the initial preparation phase, intervention components are created. In the optimization phase, candidate components are experimentally tested to gather data to inform decisions about which should be included in a final, optimized intervention. The final evaluation phase is the test of the optimized intervention, using a randomized controlled trial or other rigorous research design. The MOST approach has generally been articulated with regard to the development of interventions, but we found it useful in guiding the adaptation of the SFP 10-14 intervention and suggest it may hold promise for guiding a more rigorous science of adaptation. This report focuses on the optimization phase of this process, touching briefly on the preparation phase (reported in detail in Ivanich, Mousseau, Walls, Whitbeck, & Whitesell, 2018). The evaluation phase will be a randomized controlled trial of the optimized intervention, funded by the National Institutes of Health to begin in 2019.

Preparation phase.

In addition to the community-engaged process described above that led us to the adaptation of SFP 10-14 for this community, the preparation included an intensive process of community-engaged adaptation (Ivanich, et al., 2018). Adaptations were made cautiously, balancing scientific evidence and cultural understanding, with two primary goals: (1) retain core effective components of SFP 10-14, appropriately translated for local culture and context; (2) add cultural and contextual components to complement and reinforce core components. Program developers at Iowa State University were consulted regularly to ensure fidelity to the original program. The adapted program, named Thiwáhe Gluwáš’akapi (TG), translates as “sacred home in which family is made strong.” Some adaptations were embedded throughout the curriculum in ways that could not be isolated or tested independently of the full intervention. These were changes made to align the overall program with cultural values (e.g., integrating a cultural kinship system to expand “family,” including extended family in SFP sessions, and moving a session on listening skills earlier into the curriculum to honor cultural oral traditions and ways of learning). The advisors on our CAB felt strongly that these adaptations were essential to the value of the TG program for families in the community and fundamental to the implementation of SFP 10-14 in this community. These adaptations were not tested in the optimization phase.

Optimization phase.

The optimization phase includes trying out candidate intervention components to assess their potential contribution to the intervention, based on selected decision criteria. Three potential adaptations to SFP 10-14, suggested in discussions with community advisors, were selected as candidate components for the optimization trial, to be tested before making decisions about their place in a final TG program. Two were additions to TG (i.e., they were not part of SFP 10-14): using tribal language for kinship terms and using Facebook groups to connect families outside of sessions. These components were proposed to increase family engagement and reinforce program messages. The third candidate component was substance-use-specific content in SFP 10-14; in contrast to tribal language and Facebook, this component involved removal of content from SFP 10-14, to test effectiveness of TG without it.

Traditional cultural component: Including tribal language.

The importance of language in transmitting culture is clear; yet in this AI tribe, as in many others, the local language was largely stripped away through colonization. Language revitalization efforts are underway, but many within this tribal community have had at best limited knowledge of their language. We did not want families with limited tribal language knowledge to feel excluded, so we introduced tribal language only for kinship relationship terms that were relevant to the fundamental cultural adaptations made to the TG program. This approach included language in a limited way for participants who had no experience with it and connected with existing knowledge for participants with language experience; in both cases, reinforcing core program messages and traditional ways of knowing and transmitting kinship roles and responsibilities. We created two versions of the TG program: one used English words for kinship relationships and the second, tribal language words for those relationships.

Contemporary cultural component: Using social media.

Cultural adaptations to interventions for AI populations often focus on incorporating traditional cultural elements, as exemplified in our incorporation of the cultural kinship model and tribal language, but we also recognized the importance of contemporary cultural adaptation. One such adaptation used social media to reinforce program messages and build community among families participating within each group. Social media has become a common tool for information dissemination and communication in this community, as around the country; Facebook use was particularly prevalent on this reservation at the time this study began. Thus, we explored the added value of Facebook groups as a platform for reinforcing program content and enhancing social connection among parents. Adult participants received posts about session content, had opportunities to respond to questions about lessons, and were invited to post their own reflections.

Substance-use-content component.

Finally, we explored whether or not substance-use-specific content was critical to positive outcomes that have been documented with SFP 10-14 (including reduced substance use risk). The focus on substance use within tribal communities has often been stigmatizing, and offering a program focused on substance use prevention is somewhat off-putting to many families that might otherwise benefit from this type of program. Teaching substance use resistance skills is only one of many theoretical components incorporated in SFP 10-14 to protect youth from substance use. We evaluated its inclusion as a core component by comparing a version of TG with the substance-use-specific content to a version with an alternative session on healthy eating and exercise (attention control).

Applying the MOST approach to adaptation.

Consistent with the MOST approach, we utilized a fractional factorial design to compare the relative effectiveness of these adaptation components on quantifiable outcomes (Collins, 2018). We tested whether each candidate component had a demonstrated effect on targeted outcomes (i.e., positive, negative, or no effect). In evaluating these findings, we also considered how each candidate component mapped onto the core components of SFP 10-14 and the theoretical and empirical foundations of the program.

Consistent with the MOST approach, we also considered other optimization criteria. Collins (2018) highlights three criteria in addition to effectiveness: efficiency, economy and scalability. Collins defines efficiency as producing “a good outcome without wasting money, time, or any other valuable resource” (Collins, 2018, p. 14) and economy as being affordable, within available resources. These two criteria are related, but efficiency pertains to getting the most out of an intervention and economy to whether or not resources will support it. Scalability concerns the extent to which an intervention can be implemented in real-world settings without the extensive adjustment. These considerations were particularly important in the reservation context where this work took place (described below), where resources are often scarce and planning realistically for sustainability and scalability to similar AIAN communities was a priority. We used process data and both qualitative and quantitative evaluation feedback from facilitators and participating families to provide information regarding these criteria.

We also recognized the importance of an optimization criterion that not articulated by Collins but particularly salient in the context of intervention adaptation: Cultural acceptability. Optimizing SFP 10-14 for AI youth and families to create TG required us to ensure that youth and families would connect to the program, that it would feel comfortable and familiar to them, that it would engage them. While the fractional factorial results could be useful to some extent here (i.e., indicating differences in engagement as reflected in program attendance), participant perspectives on program evaluations and qualitative interviews were particularly informative regarding how components were related to cultural acceptability.

Decision-making.

The culmination of the optimization trial was the decision-making step (Collins, 2018), during which we weighed the evidence and selected which candidate components to retain in the optimized TG program. This paper reports those findings.¹

Method

Community

This project was carried out on a Northern Plains reservation. The tribal Research Review Board and the university Institutional Review Board reviewed and approved this study. Out of respect for community confidentiality, we do not name the specific community but include broad descriptive information to provide context. This reservation is located in a rural area, distant from major population centers. Access to many services and to employment opportunities is limited. Like many AI communities, this tribe has endured federal policies designed for eradication, spanning Indian wars of the 1800s to forced boarding school and relocation policies of the 1900s. The tribe has persevered in the face of these challenges, but the intergenerational effects of this history remain evident in the lives of children and families today.

Twelve of 13 middle schools on the reservation partnered with us in this study. Field research faculty and staff led recruitment of participants through school events (e.g., parent-teacher conference nights, family nights, holiday events) and booths at community events (e.g., powwows, health fairs, basketball tournaments). Interested families were invited to an informational session that included a meal. For those who chose to participate, we first obtained consent from the adults, parental consent for the youth, and then youth assent. Participating schools provided meeting spaces for informational sessions and subsequent program sessions.

Participants

Middle-school youth aged 10-14, residing on the reservation, and adults who parented them were eligible to participate. All youth in a participating family between 10 and 14 years old and one to three parenting adults could participate. Parenting adults were defined broadly to reflect cultural kinship patterns, including parents, grandparents, and extended family members.

A total of 136 families enrolled in the program, including 197 adults and 169 youth. The sample comprised 20 groups of families across five cohorts (3-12 families per group, 3-5 groups per cohort), from the fall of 2015 through the fall of 2017.² Each group of families was randomly assigned to one of the four MOST conditions shown in Table 1; thus each version of the TG program was delivered to five groups of families. Table 1 shows the number of participants enrolled by MOST condition, and the number and percentage of those enrolled who attended at least one program session and completed a survey at the end of the seven-week TG program and were, thus, counted as participants for the purposes of the analyses reported here. Analyses (not reported here) showed no significant differences between these participants and those who only completed the baseline survey on any of the baseline measures.

Table 1.

MOST conditions, enrollment, and participation

	Component			Number Enrolled			Number (%) Participating
Condition	1	2	3	Families	Adults	Youth	Families	Adults	Youth
A	English kinship terms	No Facebook	Substance Use	30	41	37	21 (70)	26 (63)	24 (65)
B	English kinship terms	Facebook	Nutrition	41	60	48	31 (76)	45 (75)	36 (75)
C	Tribal kinship terms	No Facebook	Nutrition	31	45	41	26 (84)	37 (82)	36 (88)
D	Tribal kinship terms	Facebook	Substance Use	34	51	43	20 (59)	29 (57)	26 (60)
Total				136	197	169	98 (72)	137 (70)	122 (72)

Note: The TG intervention was delivered to a total of 20 groups of families; 5 groups were randomly assigned to each MOST condition.

A sample of youth, parent and family sessions in each cohort were observed using fidelity checklists that incorporated adapted session content and MOST conditions. High fidelity to core program content was observed (>90% across youth, adult, and family sessions). Corrective feedback was provided to facilitators as needed in ongoing supervision.

Data Collection

Participants completed up to three surveys: (a) within two weeks prior to participation (baseline); (b) within two months after program completion, and (c) approximately six months after the program (for all but the last cohort). Both youth and adults responded to questions about themselves and family members participating in the program with them. For example, if a youth participated with both her mother and father, she would complete a set of questions about her father and the same set of questions about her mother. If a father participated with two children, he would likewise answer a set of questions about each of those children. Because the addition of each participating family member significantly increased the length the survey, respondent burden was managed by surveying each participant about no more than two family members, even if more than two family members participated in the TG program with that participant. Priority was given to the younger children and to their primary caregivers.

Outcome measures were selected or, as needed, created by a multidisciplinary research team that included academic and community experts in local tribal culture, health disparities, adolescent substance use, family-based interventions, and measurement in American Indian and Alaska Native (AIAN) populations. When available, measures with evidence of reliability and validity with AIAN populations were used (e.g., Kaufman et al., 2010). Validated measures of theoretical mediators targeted by the intervention were drawn from earlier studies of SFP 10-14 (e.g., Spoth, Guyll, Trudeau, & Goldberg-Lillehoj, 1996) and adapted as needed for this population. Items were reviewed by community partners and edited for culturally relevant content and language. Surveys were revised based on analysis of pilot test results and cognitive interviews. Confirmatory factor analyses (not reported here) demonstrated construct validity; Cronbach’s alpha scores (reported for each measure below) reflect adequate reliability.

Outcome Measures

The primary outcome of interest was youth substance use. Other outcomes assessed were the theoretical mediators of the effects of the TG program on youth substance use, proximal outcomes such as youth substance use attitudes, and measures of parent-child relationships and parenting practices (e.g., parental monitoring, discipline) reported by both youth and adults. Family cultural practices (youth report) and parents’ cultural socialization practices (adult report) were also included because the cultural components included in the adapted program were intended to enhance parents’ intentional cultural socialization efforts and were expected to be reflected in family cultural practices. Cultural experts and community advisors suggested these would be protective against early substance use. Mental health outcomes (both youth and adult) and social network connections will be explored in future analyses.

Youth Substance Use.

Youth Substance Use behavior was measured with questions adapted from previous surveys of AIAN youth (Kaufman et al., 2010); in this report, we focus on past-month use of alcohol, marijuana, and cigarettes.

Theoretical Mediators.

Youth Substance Use Attitudes and Norms.

Three items adapted from measures used in SFP 10-14 outcome studies (Spoth, Guyll, Trudeau, & Goldberg-Lillehoj, 1996) assessed substance use attitudes. Youth were asked how wrong they believed it was for someone their age to use alcohol, marijuana, and tobacco (Cronbach’s alpha=.94). Substance Use Norms were assessed with three items from the Monitoring the Future survey (Bachman et al., 2011) asking how many students youth thought used alcohol, marijuana, and tobacco (Cronbach’s alpha=.92).

Youth Peer Resistance Skills.

Peer resistance skills were assessed with 16 items adapted from an SFP 10-14 measure (Spoth, Redmond, Hockaday, & Yoo, 1996). Youth were asked to imagine that a friend asked them to steal something from a store, or offered them alcohol, tobacco, or marijuana. For each scenario, youth were asked how likely they would be to (1) do what their friend asked; (2) say no and walk away; (3) say why it would be wrong; and (4) suggest something else to do. Cronbach’s alphas ranged from .87 to .94.

Parenting Related to Substance Use.

The first measure of parenting related to substance use, Communication about Alcohol and Drugs, was adapted from Miller-Day and Kam (2010). Both adults and youth responded to five items about the extent to which they discussed values and rules around alcohol and drug use (Cronbach’s alpha=.89 for adults; .92 for youth). Rules about Substance Use were reported by adults, with three items adapted from SFP 10-14 measures (Cronbach’s alpha=.90).

Family Risk and Protective Factors.

Parent-child communication, consistent discipline, standard setting, and parental monitoring were each reported by both youth and adults. Youth reports of parent-child communication were based on six items from a previous study of AI youth (Kaufman et al., 2010). Questions asked how often youth talked to their parents about friends, the future, parties, personal problems, schoolwork, and grades (Cronbach’s alpha=.77). Adult reports of parent-child communication were assessed using SFP 10-14 evaluation items, with questions about discussing youth’s goals and dreams, youth’s problems, family’s values, and problems at home (Cronbach’s alpha=.88).

SFP 10-14 items were used to measure consistent discipline for both youth and adults; for example, how often adults disciplined the youth for something at one time but not another (4 items each; Cronbach’s alpha=.55 [youth] and .65 [adults]). Standard setting was assessed from both youth and adult perspectives with four questions from the SFP 10-14 measures that assessed adults’ giving reasons for decisions, soliciting youths’ opinions, explaining rules, and invoking empathy (Cronbach’s alpha=.73 [youth] and .75 [adults]).

Parental monitoring was also assessed with parallel youth and adult items adapted from SFP 10-14, assessing how often adults knew where youth were, who they were with, and what they were doing, with one additional item added for adults that asked how often they had help from family members in monitoring their child. (Cronbach’s alpha=.84 [youth] and .74 [adults]).

Positive and negative parental affect were assessed from both adult and youth perspectives with adapted SFP 10-14 items. Three positive affect items (i.e., conveyed caring, appreciation, acted loving; Cronbach’s alpha=.78 [youth]; .77 [adults]) and four negative affect (i.e., got angry, shouted, swore, lost temper; Cronbach’s alphas=.84 [youth]; .82 [adults]) were included. Two adult-report anger management items, also adapted from SFP 10-14 items, were whether adults took time to calm down before disciplining and recognized minor misbehaviors that warranted small consequences (Cronbach’s alpha =.65). One additional adult measure was family meetings, assessed with five SFP 10-14 items about past-month use of meetings to discuss family fun, problems, responsibilities, problems involving the children, and strategies for responding to offers of substance use (Cronbach’s alpha =.87).

Peer Risk and Protective Factors.

We also assessed key peer factors that have been shown to be predictive of substance use in adolescence. Seven items assessed Association with Deviant Peers (e.g., peers that encouraged disobeying parents, got in fights; Cronbach’s alpha =.91). Six Association with Prosocial Peers items asked about how who encouraged positive behavior (e.g., participated in cultural activities, extracurricular school activities; Cronbach’s alpha =.84). Both measures were adapted from Kaufman et al. (2010) and completed by youth.

Cultural Protective Factors.

Youth reports of Family cultural practices were assessed with six items adapted from Kaufman et al. (2010) that surveyed engagement in a variety of traditional cultural practices (Cronbach’s alpha =.79). A parent-report measure of Cultural socialization was created this study, drawing heavily on measures of cultural socialization for other ethnic groups (see Hughes, et al, 2006, for a review). It included 27 items across six subscales: Traditional Cultural Practices (e.g., I took my children to [tribal] cultural events, like powwows or ceremonies); Cultural Kinship Practices (e.g., I made sure my children spent time with family members, like grandmas, grandpas, aunts, uncles, and cousins); Cultural Language (e.g., I made sure my children got to hear [tribal] language spoken by others); Spiritual Practices (e.g., I talked to my children about how spiritual practices can help them live a good life); Cultural Pride (e.g., I encouraged my children to be proud of their [tribal] heritage,); Integration into Mainstream Western Culture (e.g., I talked to my children about the history of Indians in this country and how that affects them today). Cronbach’s alpha for the overall scale was .95.

Program Attendance.

We tracked attendance to explore the extent to which variations in program components might be related to retention in the TG program over time. It is important to note, however, that the substance use adaptation component was not expected to impact attendance because this manipulation (i.e., replacement of the substance use session with the alternative nutrition session) occurred in the fifth week of the seven-week program.

Analytic Design

The fractional factorial design utilized to compare the relative effectiveness of the three experimental adaptation components (tribal language, social media, and substance use content) was determined using SAS PROC FACTEX, (SAS Institute Inc., 2008) as recommended in Collins (2018), specifying three factors and testing for main effects only. The procedure resulted in four conditions for the fractional factorial design, with the three components crossed as shown in Table 1 and each component delivered to half of the participants in the study. After confirming baseline equivalence in evaluation outcomes across the four MOST conditions, the Mixed procedure in SPSS for Windows v. 25 (SPSS, Inc., 2010) was used to assess differences in post-program outcomes for both youth and adults. In each model, we included exposure to one of the three adaptation components as a fixed effect and specified a random intercept to account for participant clustering within families.

Results

Table 2 presents the demographics for the 137 adult and 122 youth participants. Most identified as AI, and approximately equal numbers of boys and girls participated. About 75% of adults were women; 52% were mothers of the participating youth. The average age of youth was 11.5 years (range 10-14). The average age of adults was 41.5, with a range from 19 to 78 years, reflecting the inclusion of grandparents and other relatives serving as caregivers for youth.

Table 2.

Demographic characteristics of TG participants

	Adults (N=137)	Youth (N=122)
Female (%)	75.2	46.7
Age (Mean, SD)	41.5 (10.9)	11.5 (1.28)
AIAN (%)	98.5	91.0
Educational Level (%)
Less than High School	19.7
High School/GED	24.8
Some College	37.2
College Degree or Post Graduate	18.2
Marital Status
Married or Cohabitating	59.4
Separated, Divorced or Widowed	16.5
Single, never married	24.1
Relationship to Youth Participant (%)
Mother	51.8
Father	16.8
Other	31.4
Grade in School (%)
4th grade		14.8
5th grade		28.7
6th grade		24.6
7th grade		14.8
8th grade		14.8
9th grade		2.5
School Performance*
Mostly As		43.5
Mostly Bs		38.9
Mostly Cs		14.8
Mostly Ds		2.8

^*11.5% missing

Analyses began with a comparison of outcomes at baseline across MOST conditions, to check that random assignment of groups to conditions achieved the desired goal of creating equivalent groups, thus supporting the comparison of post-program outcomes. No significant differences were found across the four MOST conditions delineated in Table 1 on any of the key outcome measures. In the interest of space, these analyses are not reported here.

Effectiveness

Results of analyses evaluating the effectiveness of the candidate components – tribal language, social media, and substance use content – on post-program outcomes are shown in Table 3. No effects approached significance at the adjusted alpha of .002 (alpha=.05, adjusted for multiple comparisons). Thus, these findings were not informative with regard to the inclusion of the candidate components in the optimized intervention; we did not find evidence that including or not including any of these three would alter the effectiveness of the intervention, at least in the short term. Decisions for which, if any, components to include thus rested on other criteria.

Table 3.

Program outcomes by MOST intervention component exposure.¹

		Component
		Kinship Terms			Facebook			Substance Use Content
		English	Tribal language	p	No	Yes	p	Included	Not included2	p
Primary Outcome: Past month substance use (%)
Alcohol	youth	5.2%	3.4%	.63	3.4%	5.1%	.66	4.4%	4.1%	.93
Marijuana	youth	10.3%	8.3%	.71	6.8%	11.9%	.34	8.7%	10.2%	.78
Cigarettes	youth	6.9%	5.1%	.68	1.7%	10.2%	.05	7.4%	4.1%	.46
Theoretical Mediators
Substance use attitudes and norms (0-4)
Attitudes	youth	3.2	3.5	.29	3.2	3.5	.23	3.3	3.3	.84
Norms	youth	0.9	0.8	.71	0.9	0.8	.78	0.8	0.9	.77
Peer Resistance skills (0-4)
Use	youth	0.4	0.4	.95	0.5	0.4	.80	0.4	0.5	.87
Say no	youth	3.0	3.0	.82	3.1	2.9	.25	2.9	3.1	.44
Say why it would be wrong	youth	3.0	2.6	.16	2.8	2.8	.99	2.7	3.0	.32
Suggest something else	youth	2.9	2.9	.92	2.9	2.8	.63	2.8	3.0	.31
Parenting related to substance use (1-5)
Rules about Substance Use	adult	4.5	4.6	.38	4.6	4.6	.99	4.6	4.6	.76
Communication about alcohol and drugs	youth	4.1	4.4	.17	4.1	4.4	.17	4.3	4.2	.48
	adult	4.4	4.5	.63	4.4	4.5	.71	4.4	4.5	.49
Family risk and protective factors (1-5)
Parent-child communication (1-6)	youth	3.2	3.4	.19	3.2	3.4	.25	3.4	3.2	.32
	adult	4.4	4.4	.93	4.4	4.5	.48	4.4	4.5	.46
Consistent discipline	youth	3.9	3.9	.88	3.9	3.8	.37	3.9	3.8	.23
	adult	4.1	4.2	.37	4.2	4.0	.11	4.2	4.0	.19
Standard setting	youth	3.2	3.5	.14	3.3	3.4	.75	3.4	3.2	.33
	adult	3.8	3.8	.88	3.8	3.8	.82	4.0	3.6	.03
Parental monitoring	youth	4.0	4.2	.28	4.0	4.2	.23	4.1	4.1	.74
	adult	4.6	4.6	.82	4.7	4.6	.29	4.6	4.6	.65
Positive parental affect	youth	4.3	4.5	.24	4.3	4.5	.30	4.4	4.3	.50
	adult	4.6	4.5	.72	4.5	4.6	.46	4.6	4.5	.50
Negative parental affect	youth	2.0	1.9	.75	1.8	2.1	.12	1.9	2.0	.53
	adult	1.7	1.6	.83	1.6	1.7	.23	1.6	1.7	.21
Anger management	adult	3.8	4.2	.03	4.0	4.0	.98	4.1	3.9	.34
Family meetings (0-4)	adult	2.7	2.7	.88	2.7	2.8	.54	2.8	2.6	.14
Peer risk and protective factors (0-4)
Deviant peers	youth	0.5	0.4	.46	0.4	0.5	.66	0.5	0.4	.81
Prosocial peers	youth	2.1	2.4	.26	2.2	2.4	.26	2.4	2.1	.10
Cultural protective factors
Family Cult. Practices (0-2)	youth	1.4	1.4	.79	1.4	1.5	.19	1.4	1.5	.30
Cultural socialization (1-5)	adult	3.6	3.7	.62	3.7	3.6	.63	3.7	3.6	.59
Attendance (1-7)3	family	4.2	4.5	.38	4.3	4.4	.86	4.3	4.3	.92

Note: Higher scores on all measures reflect more of the outcome being assessed. The range for each outcome is noted in parentheses. Measures relevant to dyadic relationships are reported for the first referent only.

¹8 participating adults and 4 participating youth did not complete post-program surveys.

²A healthy eating and exercise session replaced substance use content for participants in this condition.

³Includes all participants, N=137 adults, N=122 youth.

Efficiency, economy, and scalability

Inclusion of tribal language was not related to the efficiency, economy, or scalability of the TG program. The tribal language component did not require resources, after the initial curriculum development. Implementation required minimal, if any, additional facilitator time (perhaps some time to learn words for novice language speakers) and no additional cost.

Efficiency, economy, and scalability were relevant consideration for the social media component. Including this component required additional facilitator time to monitor the Facebook page, post messages, respond to posts between program sessions, and receive training in online security protocols. In practice, this was the most difficult component to implement, and the one where fidelity to the intended protocol was lowest. These considerations weighed against including this component.

Including substance-use-specific content was related to efficiency and economy, in that eliminating this content could shorten the program. This content was part of the original SFP 10-14 program, so including it would not be an added cost, but removing it could be a cost savings.

Acceptability

Input from community advisors and feedback from TG participants both strongly supported the use of the traditional language for the kinship terms. Participants expresses appreciation for the use of tribal language in the program, both in program evaluations an in feedback directly to facilitators, suggesting that incorporating tribal language enhanced connection to TG. The specific inclusion of traditional language words for family relationships directly reinforced core SFP 10-14 messages in building family connections.

We originally heard from community advisors that social media would likely be a good addition to the program. However, facilitators found it difficult to monitor and provide timely content and families engaged only marginally with this online component (i.e., few signed up for the Facebook group, and discussion questions posted by the facilitators did not generate many responses). While implementation and level of engagement were likely related, the increased complexity and low participation considerably weakened enthusiasm for this component.

Process data from TG program facilitators, including evaluation forms they completed after delivering program sessions and debrief interviews at the end of the study, emphasized the positive responses they received from families to substance use sessions. We were somewhat surprised by this, given that community advisors had prompted to us to consider dropping this content because of concern that stigma around substance use might keep families from participating. We found, instead, that families welcomed support and guidance for talking with their children about substance use. In addition, in interviews conducted with families who did not attend sessions, barriers discussed focused on transportation, childcare, and scheduling; stigma around substance use content was not mentioned.

Optimized TG program

Weighing the criteria above, we included traditional language and substance-use-specific content in the optimized TG intervention, and did not include the social media component. The decision to include the use of traditional language kinship terms rested largely on the criterion of cultural acceptability. While we did not see evidence of increased program effectiveness related to this component, acceptability was high and resource cost low. Participants connected well to this content and little to no additional resources were required. The decision to retain the substance-use-specific content, which we had considered eliminating, was more complex. This component also showed no impact on effectiveness, one way or the other, so it could arguably be dropped – to reduce stigma and increase efficiency. However, the criteria of acceptability again weighed heavily. Participants liked this content and responded well to it. In addition, we were hesitant to eliminate this content, given the substantial body of evidence regarding long-term effects of SFP 10-14 on youth substance use outcomes, effects that may become evident only over time as these youth age into the riskiest period for substance use initiation. Weighing these considerations, we chose to retain the substance use content in the final TG program. The decision to drop the social media component was straightforward. It did not add value in terms of effectiveness, it required considerable resources and was difficult to implement, and feedback from participants and facilitators did not suggest that it supported acceptability.

Discussion

This study emerged out of a long partnership between university researchers and an AI reservation community that had documented early initiation of substance use and high rates of problematic use. Community partners urged action to prevent use among youth. University partners identified existing prevention programs with strong evidence of effectiveness in other populations but no prior studies directly relevant to this community. Thus, these partners set out to find a program that was a near fit to the culture and context of this community and then adapt it to be a good fit. The challenge was to identify a process that would maximize the effectiveness of the adapted program for families in this community.

Two primary strategies were used – an intensive community-engaged approach to identify key adaptations seen as essential to aligning SFP 10-14 with the cultural context of this community and an innovative MOST approach to evaluate three candidate components before determining whether or not to include them in an optimized intervention. This process reflected the importance of community engagement in identifying intervention goals, selecting an evidence-based program, adapting that program, designing an evaluation approach, creating measures, and collecting data. Each of these steps needed not only input from community members but also leadership from within the community. Study co-investigators and staff lived and worked in the community, leading the project from within. Community voices were critical to optimizing the relevance of the intervention for local families. As we worked with community partners, we built a frame for considering adaptations to the SFP 10-14 program that prioritized retaining core components while incorporating important cultural teachings about family. We established an iterative process of gathering input from these partners and from model developers as we considered adaptations in line with community priorities (Ivanich, Mousseau, Walls, Whitbeck, & Whitesell, 2018). At each step, we paid attention to both the cultural and scientific rationale for each component and to the theory of change associated with each element, with an eye to maximizing the potential impact of each element added to the curriculum (or considered for removal). This rigorous approach resulted in a program that honored both the local community and culture and the scientific evidence, adding or removing elements only on the basis of sound theory, research, and cultural guidance. We believe this systematic approach has positioned TG to be both acceptable and effective.

We took the additional step of using a MOST approach to evaluate three candidate components being considered for inclusion in the TG program, to gather data on which components to include in an effective, efficient, acceptable optimized intervention. Implementing the MOST approach – varying components included in this complex program delivered to families over the course of several weeks in a fractional factorial design – posed challenges. Numerous implementation strategies were put in place to support facilitators in delivering the TG program with fidelity to the version of the program randomized for a particular group (e.g., tribal kinship terms instead of English). This design was not easy to implement.

Despite the challenges, we believe the MOST approach served us well, guiding a systematic process in line with recent calls for developing a science of intervention adaptation (Castro & Yasui, 2017).

Limitations

The fractional factorial design did not allow the examination of interaction effects; testing interactions (e.g., full factorial design) would have required a larger sample size than feasible in this community. This design offered the most pragmatic combination of scientific rigor, practicality, and strategic allocation of resources. Analyses examined only immediate post-program outcomes; differential effects of the adaptation components that might have become apparent only over a longer time period could not be detected. Few program implementation measures were collected, limiting conclusions that can be drawn about engagement outcomes.

This study took place on one reservation and included only families from that community who voluntarily enrolled their youth in the TG program. The sample was not representative of that tribal community nor of diverse AI communities more broadly. Our analysis included all families who participated in at least one session of the TG program and completed the post-program survey; the average number of sessions attended was just over 4 out of 7 (see Table 3), so outcomes findings were diluted by lack of exposure to the full program.

Future Directions – Evaluation Phase

This paper details a community-engaged, culturally and scientifically rigorous process for adapting an evidence-based substance use prevention program and testing the relative value of candidate components (the optimization phase of the MOST process). The optimized intervention is now ready for the evaluation phase, a randomized controlled trial, comparing both immediate and long-term outcomes for families.