The Efficacy of a Multicultural Prevention Intervention among Urban American Indian Youth in the Southwest U.S.

Abstract

This study explored how a non-targeted group of ethnic minority youth might or might not benefit from a prevention intervention focused on other cultural groups. The study specifically evaluated the effects of an evidence-based drug prevention curriculum with a sample of urban American Indian youth in the southwest U.S., most of whom self-reported multi-ethnic heritages. Using growth curve modeling, this research examined the developmental trajectory of drug use for these youth, and compared it with the trajectory of youth from other racial/ethnic groups at pre-intervention, post-intervention, and two follow up time periods. Results indicate that alcohol and marijuana use increased from pre-intervention across subsequent time periods for all youth. The drug use of the American Indian youth in the treatment group increased on some measures. Specifically, they reported a steeper trajectory in the amount and frequency of alcohol and marijuana use compared to the youths in the treatment groups with other racial/ethnic identifications. The implications of these findings for the development of culturally grounded prevention programs for multiethnic, urban American Indian youth are discussed. Editors' Strategic Implications: This research provides a specific example, but also makes a strong global argument, for the need to develop and evaluate prevention programs that are culturally grounded in the worldview of the target group.

Introduction

Culturally specific, school-based drug prevention curricula have been receiving increased attention in recent years. These programs are based on the premise that infusing youths' culture into the content and format of the prevention message will reduce adolescent drug use (Kandel 1995). One such program, keepin' it R.E.A.L., has achieved this by creating the curriculum from the developmental and cultural realities of Mexican American youth from the southwest United States (Gosin et al. 2003). To date, the program has been developed for and evaluated with primarily Mexican American youth (Kulis et al. 2005). Research has assessed the differences in program effectiveness by acculturation level among Mexican/Mexican Americans (Marsiglia et al. 2005) but has not closely examined the residual effects of this program among other minority youth populations (i.e., the program's effects on non-targeted minority youth populations). In what ways might this program impact the drug use of minority youth who are not Mexican American? How might these effects compare with the treatment effects of the program for Mexican American and European American youth?

Using growth curve modeling, this study highlights the developmental trajectory of drug use among a group of urban, mostly multi-ethnic American Indian youth that have participated in the keepin' it R.E.A.L. drug prevention program. This study examines the potential for an evidence-based curriculum to achieve positive residual effects related to drug and alcohol use for this population of youth. The findings from this study have implications for the development of new culturally grounded prevention curricula for American Indian youth and other minority youth populations. Specifically, it elucidates the level of cultural specificity that may be necessary to achieve positive drug prevention effects with certain American Indian youth.

Epidemiology and Correlates to Drug Use for American Indian Youth

Of all racial/ethnic youth populations in the U.S., none seem to be more at risk than American Indians (Herring 1994; LaFromboise et al. 1990; National Institute on Alcohol Abuse and Alcoholism [NIAAA] 2002). Numerous studies have reported higher rates, earlier onset, and more severe consequences of drug use with American Indian youth compared to their non-Native peers (Moncher et al. 1990; Schinke et al. 2000; Spicer et al. 2003). Research indicates that by 11 years of age, nearly one-third of all American Indians have tried alcohol (Mail 1995; NIAAA 2002), and that 78% of American Indians aged 12–17 from tribes throughout the U.S. have tried alcohol, compared with 53% of the general population (Cabape and Howley 1992; Spicer et al. 2003). Once they enter adolescence, American Indian youth seem particularly prone to using alcohol and other drugs in combination with one another (U.S. Congress Office of Technology Assessment [OTA] 1990); marijuana and tobacco are reportedly the next most popular drugs after alcohol. Among American Indian youth, research has illustrated high-frequency use of marijuana (Novins and Mitchell 1998) and tobacco (Moncher et al. 1990). It has been estimated that about half (51–62%) of American Indian youths have tried marijuana, as compared with fewer than half (28–50%) of other racial/ethnic youths, although usage varies tribally (Indian Health Service [IHS] 2005; NIAAA 2002).

The consequences of drug use among American Indian youth are severe. Moncher et al. (1990) found that the amount of drug and alcohol use by these youth has been associated with academic failure, delinquency, unemployment, and violent criminal behavior. When compared with other racial/ethnic groups, American Indian youth experience more severe health, social, and economic consequences related to drug use (Schinke et al. 2000; Spicer et al. 2003), including depression, conduct disorder, posttraumatic stress disorder, and suicidality (Stiffman et al. 2003). With the evidence of the early onset of drug use initiation and apparent negative consequences of drug use for American Indian youth, the need for understanding what these youth require for effective drug use prevention is crucial.

Although specific reasons behind the prevalent drug use among American Indian youth are not consistently understood, multiple factors have been suggested as contributors to drug use, including: peer group and family offerers (Kulis et al. 2006); laissez-faire child rearing practices (Herring 1997); lack of stability of the home and family (Garrett and Herring 2001); reservation American Indian youths' excessive amounts of free time (Herring 1997); urban American Indians youths' experiences in foster care and their disconnection with mainstream society, resulting in high rates of depression, anxiety, suicide, and dual axis diagnosis (Flynn et al. 1998); conflicts between cultural ideals and behavioral realities (Garrett and Herring 2001; Herring 1997); parental and community attitudes about drug use and concomitant adult-use models (Office of Applied Studies [OAS] 2004); adolescent social adjustment factors and peer pressures (NIAAA 2002); and socioeconomic stressors (Clarke 2002; OAS 2004). In addition, generational stress related to forced assimilation, colonization, and acculturation have been cited as contributors to social problems of American Indians, including the drug use of American Indian youth (Beauvais 1998; Frank et al. 2000; OTA 1990). When facing acculturation concerns, LaFromboise et al. (1990) and Schinke et al. (1988) suggest that racial/ethnic youth may use drugs as an attempt to cope with increased pressure in their attempt to fit within both their dominant and minority cultures. The historical experiences of American Indians suggest the importance of emphasizing the unique sociohistorical/cultural contexts that may contribute to drug prevention efforts with these youth.

Etiology of Drug Use for American Indian Youth

Theoretically, it has been suggested that the cultural context plays a role in the development of substance using behaviors of indigenous youth populations (Oetting et al. 1998; Szapocznik and Coatsworth 1999), although research in this area demonstrates mixed findings (Trimble 1995). Oetting et al. (1998) noted that the relationship between culture and drug use is complex, incorporating the influence of both cultural and subcultural norms for drug use. In effect, they state that it is possible for youth to have both high cultural identification and high levels of drug use, because they are influenced by different subcultures, such as the family or peer group. One subculture may affect cultural identification (e.g., the family), while another might influence drug use (e.g., peer groups). The complexity of this relationship may account for research findings that have not found a direct relationship between ethnic identity and drug use (e.g., Bates et al. 1997), and suggests the need to examine this relationship more closely. Nonetheless, some research has suggested that there are unique sociocultural contributors to the drug use of Native youth, including the influence of extended family (Hurdle et al. 2003; Trotter et al. 1997). In fact, some research suggests that family influence has a direct effect on the drug use of Native youth (Oetting et al. 1988; Oetting et al. 1989). This is contrasted with Euro American youth, whose familial influences on drug use are mediated through the peer group. Oetting et al. (1989) hypothesized that strong cultural values coupled with isolation from potential peer groups lead to family members (e.g., siblings, cousins) serving as a “surrogate” peer group for Native youth. Overall, research guided by contextual theory highlights the complexity of the etiology of American Indian youths' substance using behavior from an ecological and relational perspective. This research also suggests that there may be challenges for understanding culturally specific drug use prevention principles for this population.

Drug Prevention Programming for American Indian Youth

There have been substantial efforts in recent years toward the development of drug prevention programs focused on American Indian youth. For example, Marlatt et al. (2003) described the development of the Journeys of the Circle Project, which is a culturally congruent life skills course targeted toward Northwestern American Indian youth. Similarly, Schinke et al. (2000) implemented and evaluated a culturally tailored life skills intervention with American Indian youth from 10 reservations in North and South Dakota, Idaho, Montana, and Oklahoma. Aside from the research conducted by Schinke and colleagues, however, the majority of prevention efforts with American Indian youth have not been rigorously evaluated for efficacy (Beauvais and Trimble 2003; Hawkins et al. 2004). Beauvais and Trimble stated that most of the prevention research with American Indian youth has focused largely on “commentary and recommendations and not on the science of prevention” (p. 397).

However, while there are relatively few evaluation studies focused specifically on drug use and American Indian youth, there have been ongoing efforts related to culturally specific drug prevention programs for minority youth in general. Related to this research is the debate as to whether prevention programs need to be culturally “grounded,” that is, developed from the cultural values and variability inherent within each culture, or if they can be culturally “adapted,” that is, modified from universal prevention programs originally developed for non-Hispanic European American populations (Hecht et al. 2003). Each of these approaches has implications toward the fidelity and fit of prevention interventions (Castro et al. 2004). Further, culturally focused prevention curricula range in ethnic specificity; some are highly specific (e.g., Hecht et al. 2003) while others are more broadly targeted for “minority” youth (e.g., Botvin et al. 1997). In sum, there is much left to learn about the degree of cultural specificity required for positive prevention effects with minority youth.

The keepin' it R.E.A.L. Curriculum

keepin' it R.E.A.L. is a culturally grounded, video-enhanced prevention intervention that was developed and normed from the narratives of Latino, African American, and Euro American youth (Gosin et al. 2003), and validated with teacher and student feedback (Gosin et al. 2003.; Harthun et al. 2002). It has been identified as a “Model Program” by Substance Abuse and Mental Health Services Administration (SAMHSA; U.S. Department of Health and Human Services [DHHS] n.d.), which is a designation given by the organization to effective programs that have been evaluated using “rigorous” standards of research. The 10-lesson curriculum was designed for use with middle school students (6th, 7th, and 8th graders), and draws from a variety of theoretical frameworks, such as communication competence theory (Spitzberg and Cupach 1984) and ecological risk and resiliency theory (Bogenschneider 1996). The curriculum situates these frameworks within the unique geographic and cultural contexts of Latino, African American, and European American youth in the southwest United States. The primary focus of the program is on teaching drug resistance skills using four strategies: refuse, explain, avoid, and leave (Hecht et al. 2003). “Refuse” consists of statements of saying “no” to substance use offers, while “explain” consists of more elaborate reasons for refusing these offers. “Avoid” refers to avoiding situations where drugs and alcohol might be present, and “leave” refers to leaving the environment once the youth encounters substance use (Hecht et al. 2003).

The cultural specificity of the program components is based on prior research, which found ethnic differences in common communication styles, competencies, and norms (Hecht and Ribeau 1984; Hecht et al. 1990) and drug use contexts (Gosin et al. 2003). Based on this research, three versions of the curriculum have been developed: 1) A Latino version, which primarily reflects Mexican American and Mexican values (e.g., familismo, or family orientation), 2) A Non-Latino version, grounded primarily in European American and African American values, and 3) A Multicultural version, which combines half of the lessons from the Latino version and half of the lessons from the non-Latino version (Hecht et al. 2003; Kulis et al. 2005). An example of an objective from the Latino version of the curriculum is for the student to “recognize what he/she does affects his/her community, group, and family,” while an objective from the Non-Latino version is for the student to “recognize what he/she does may have favorable or unfavorable consequences on his/her own future goals” (Gosin et al. 2003, p. 128). The in-class curriculum was supplemented with a media campaign, consisting of television, radio, and billboard advertisements that reinforced the four strategies (refuse, explain, avoid, and leave) and follow-up booster activities such as school assemblies, poster projects, murals, and essay contests (Kulis et al. 2005).

Research evaluating the curriculum found that the use of alcohol, cigarettes, and marijuana increased over time for students who received any version of the intervention and students in the control group; however, the increase was significantly less for students in the intervention group (Hecht et al. 2003). This effect was particularly salient for alcohol use (Gosin et al. 2003; Hecht et al. 2003). Further, the intervention students perceived significantly smaller increases in their peers' substance use compared with those in the control group (DHHS n.d.; Hecht et al. 2003).

Past research suggests differential effects of each version of the program, with the Latino and Multicultural versions of the intervention providing more benefits to Mexican/Mexican American students (Gosin et al. 2003; Kulis et al. 2005) and the overall sample (Hecht et al. 2003). However, while versions of the curriculum that incorporated aspects of culture (the Latino and Multicultural Versions) were found to impact a wider array of substance use and attitudinal outcomes in the desirable direction than did the Non-Latino version, tests of strict cultural matching of program content with the students' racial/ethnic backgrounds did not produce statistically significant differences (Hecht et al. 2003; Kulis et al. 2005). In other words, these studies did not find that Mexican/Mexican American students receiving the Latino version of the curriculum demonstrated better overall outcomes than those receiving other versions of the curriculum. This finding is significant, because it provides support for prevention programs that broadly target “minority” youth, rather than those that are ethnic-specific. Subsequently, the strength of culturally grounded prevention programs may lie in their ability to reflect regionally-specific multicultural environments, rather than specific ethnic groups. In order to examine this hypothesis, we chose to examine the program's efficacy with another predominant minority cultural group in the southwest U.S.

To date, no studies of keepin' it R.E.A.L. have examined the residual effects of the program among youth who were not Latino, African American, or Euro American. Perhaps this program's strength lies in its ability to integrate multiculturalism into drug prevention, defined as the inclusion of cultural values from all groups participating in the prevention program (Green 1999), rather than its cultural specificity related to drug use prevention. Therefore, the purpose of this study was to examine the impact of the curriculum on urban American Indian youth of the southwest in order to examine this hypothesis more closely. The study presented here involved a reanalysis of data that was previously reported (Gosin et al. 2003; Hecht et al. 2003; Kulis, et al. 2005; Marsiglia et al. 2005). However, this study differs from previous studies because it tests the generalizability of the culturally enhanced versions of the curriculum with American Indian youth. This is accomplished through the use of growth curve modeling to examine the developmental trajectory of drug use among youth who participated in keepin' it R.E.A.L.

Method

Design and Procedures

In 1998, the keepin' it R.E.A.L. youth substance use prevention study was initiated in 35 middle schools in a metropolitan, southwestern city, encompassing more than 75% of all middle schools within the city boundaries. While most of the schools were in lower income Hispanic neighborhoods, the sample also included several schools in wealthier, non-Hispanic Euro American areas. The study used an experimental design, with three intervention conditions (Latino version, Non-Latino version, and Multicultural version) and one comparison condition. The assignment of schools to the treatment or comparison conditions was accomplished through block randomization that controlled for the size and ethnic composition of schools. Specifically, the 35 participating schools were stratified by size of enrollment and Latino ethnicity. This process yielded 8–10 schools within each condition, and each condition had a comparable mix of large and small schools, and schools with high proportions of Latino students and lower proportions of Latino students. Although the design was not stratified according to American Indian status, the percent of assignment to a treatment condition, as opposed to control, was similar for American Indians and non-American Indians: 76% and 71 %, respectively. Furthermore, chi-square tests showed no significant associations between wave 1 substance use (alcohol, marijuana, and cigarettes) and treatment status, suggesting that substance users were no more or less likely to have been assigned to a treatment rather than a control condition.

In the study schools, all students in the seventh grade participated after student assent and passive parental consent were obtained for the survey component of the study in accordance with school district and university human subjects protections. In fall 1998 (Wave 1), prior to the implementation of the program, students in all the schools completed a pre-test survey instrument that measured the adolescents' experiences with substance use, norms towards substance use, and family and individual background characteristics. Eighty seven percent of the seventh grade students completed the survey.

Following the pre-test survey, the keepin' it R.E.A.L. drug prevention program was initiated in 25 of the 35 schools. In the late spring of 1999 (Wave 2), a follow-up survey was administered to all seventh grade students approximately 2 months after delivery of the prevention curriculum had been completed in treatment schools. This survey replicated measures in the pre-test surveys so that potential treatment effects could be reliably measured. Further follow-up questionnaires were administered after an initial set of school booster sessions were delivered in treatment schools in fall 1999 (Wave 3) and after all booster sessions were completed in spring 2000 (Wave 4; See Hecht et al. 2003, for further details of research design).

Participants

Table 1 displays descriptive statistics for all students in the study. The total number of participants was 4,222, but the statistics are displayed separately for two groups: students who claimed any American Indian ethnicity (N = 685) and students who did not claim this ethnicity (N = 3537). The American Indian youth in the study primarily attended schools outside of their reservation communities. In this way, they were considered “urban” or “semi-urban” in nature. The American Indian youths' tribal affiliation is not identified due to prohibitions to divulge this information without specific and expressed permission from each tribe represented in the study. However, this southwestern metropolitan area is home to the largest proportion of American Indians in the U.S., representing numerous tribal groups (Forquera 2001; U.S. Census 2000).

Table 1. Characteristics of students within the study (N = 4,222).

	American Indian N = 685 Percent	Non-American Indian N = 3,537 Percent
In treatment program	76%	71%
Male	47%	52%
Received free or reduced lunch	79%	84%
	Median	Median
Academic grades	6 (“B's and C's”)	7 (“Mostly B's”)

About three-fourths of the students in both groups were in schools that were randomly assigned to the treatment program (76% of American Indian youth, 71% of other students). The American Indian group had slightly fewer male students (47% vs. 52%), and lower rates of receiving free or reduced lunches (79% vs. 84%). Both groups' receipt of lunches, however, is higher than the national rate of participation in the federal school lunch program, which was 57.8% in 1998 (U.S. Department of Agriculture Food and Nutrition Service n.d.). The median self-reported grades of American Indians were lower: “B's and C's” compared to “Mostly B's” for other students.

Measures

Substance Use

This study focused on the use of gateway drugs (cigarettes, alcohol, and marijuana), as there was relatively little use of other substances on pre-test evaluation. For example, only five percent of pre-test respondents reported any lifetime use of cocaine, crack, LSD, PCP, heroin, downers, speed or crystal methamphetamine. Changes in the use of gateway drugs were fundamental metrics to assess program efficacy. In addition, other measures related to substance use were collected in the survey, including substance use norms, intentions, and refusal strategies.

In each of the four survey waves, two questions were asked about cigarettes, alcohol, and marijuana use. For example, the first alcohol question asked, “How many drinks of alcohol have you had in the past 30 days?” Responses were ordinal categories on a 9-level scale, ranging from “none” to “more than 30 drinks.” The second question concerning alcohol use asked, “How many days in the past 30 days have you had alcohol to drink (not counting religious services)?” Responses to this question were in six ordinal categories, ranging from “none” to “16–30 days.” To form a single scale, the responses to the two questions were standardized to mean 0 and standard deviation 1 and were summed. This formed a single outcome scale. Similar pairs of questions were asked about cigarette use and marijuana use, and a single standardized scale was created for each.

Treatment Indicator and Individual-Level Measures

A dichotomous variable was used to indicate if the student received any version of the keepin' it R.E.A.L. program—“1” if the student was in the program and “0” otherwise. We also measured several individual-level characteristics of the students. Our models include predictors for gender, academic performance, and free or reduced lunch status (a proxy for socioeconomic status). Gender is coded “1” if male and “0” if female. Academic performance was the students' self-reported grades, which was measured on a scale from “1” (mostly F's) to “9” (mostly A's). Because our aim was to investigate the efficacy of keepin' it R.E.A.L. for urban American Indian youth versus other youth, we coded race/ethnicity to be “1” if the student claimed any American Indian heritage at any of the four waves and “0” otherwise. In the surveys, students were allowed to identify with multiple races and ethnicities and we utilized a broad definition of American Indian ethnicity. In other words, we considered a student to have American Indian heritage if he or she marked “American Indian” or any combination of races or ethnicities involving American Indian at any one of the four study waves. We investigated the consistency of American Indian identification across the four waves, and found that 41% of the students we coded as American Indian selected the American Indian label at each survey they completed. Conversely, this means that 59% of American Indian students did not choose an American Indian label in at least one survey wave. These apparent inconsistencies, however, may reflect the dynamic nature and true uncertainties of ethnic identity. Ethnic identity is fluid for adolescents (Spencer et al. 2000), and even more so for multiracial adolescents (Harris and Sim 2002). Harris and Sim examined data from adolescents who were surveyed twice about their racial identity, and they found that among children of parents with different races, only 48.4% gave consistent racial identities in repeated surveys. Ultimately, in our data we are unable to determine if these shifting identities are the result of measurement error or real changes in the way students define themselves. We believe, however, that if observed changes in identity were measurement errors, then the effect of these errors would be to diminish the estimated effects, thereby making it less likely to detect significant associations.

Analyses

Our aim was to examine how changes in substance use over time varied across treatment and racial/ethnic groups. Because the etiological processes often differ (Scheier 2001), we estimated a separate model for each indicator of substance use: alcohol, marijuana, and cigarettes. Although the use scales of these substances are correlated (alcohol and cigarettes r = .53, alcohol and marijuana r = .60, and cigarettes and marijuana r = .53), only 29% of students using substances are users of all three substances; most users (71%) are using two substances or less. Therefore, we conceptualize the use of these substances as separate outcomes, and do not apply corrections to our significance tests. In addition to our theoretical reasons to not conceptualize these outcomes as multiple indicators of a process, there is substantial debate in the medical and public health community as to whether adjustment for multiple outcomes is necessary, or even valid (Altman 2000; Perneger 1998; Rothman 1998)

Because we examined repeated measures of substance use across four survey waves, we selected growth curve modeling to determine the developmental trajectory of drug use of American Indian and other youth participating in the keepin' it R.E.A.L. program. Growth curve models are forms of hierarchical linear models in which multiple measurements are nested within a single individual (Raudenbush and Bryk 2002). In addition to measurements within individuals, we also account for the clustering of students within schools, and thus our models have three levels. Measurement waves are nested in students, and students are nested in schools. Because our initial model showed a linear trend, we employed linear growth models as opposed to models with quadratic and higher order terms. Our growth models are estimated with HLM 6.02, and they take the following form:

$${\text{Y}}_{\mathit{\text{tij}}}={\pi }_{0ij}+{\pi }_{1ij}{\text{a}}_{\mathit{\text{tij}}}+{\text{e}}_{\mathit{\text{tij}}}$$ (1)

where Y_tij is the substance use outcome at time t for student i in school j, π_0ij is the initial level of substance use for student i in school j, π_1ij is the growth in the level of substance use for student i in school j, and e_tij are normally distributed errors. a_tij represents values for time, which increases linearly (Raudenbush and Bryk 2002). Equation (1) is the level 1 model. Two additional equations specify the level 2 models of initial status and linear growth. In our models, π_0ij and π_{1 ij} are predicted with a program indicator, American Indian status, and an interaction between the program indicator and American Indian status, gender, grades, and free or reduced lunch status:

$${\pi }_{0ij}={\beta }_{00j}+{\beta }_{01}(\text{TX})+{\beta }_{02}(\text{AMIND})+{\beta }_{03}(\text{TX})(\text{AMIND})+{\beta }_{04}(\text{MALE})+{\beta }_{05}(\text{GPA})+{\beta }_{06}(\text{LUNCH})+r_{0ij}$$ (2)

$${\pi }_{1ij}={\beta }_{10j}+{\beta }_{11}(\text{TX})+{\beta }_{12}(\text{AMIND})+{\beta }_{13}(\text{TX})(\text{AMIND})+{\beta }_{14}(\text{MALE})+{\beta }_{15}(\text{GPA})+{\beta }_{16}(\text{LUNCH})+r_{1ij}$$ (3)

Lastly, the clustering at the school level is addressed with random coefficients for school-level variation in initial use and growth:

$${\beta }_{00j}={\gamma }_{000}+{\text{u}}_{00j}$$ (4)

$${\beta }_{10j}={\gamma }_{010}+{\text{u}}_{10j}$$ (5)

Attrition decreased the sample size across waves from the initial 4,222 students at pre-treatment. Student transfers (students transferring from one school to another) was the main source of attrition: transfer rates averaged 16% and 19% during grades 7 and 8, respectively. Attrition was also due to student absences on the days in which the follow-up questionnaires were administered or the inability to link later student responses to pre-treatment interviews (Hecht et al. 2003). The inability to link these responses is due to using combinations of initials, gender, and birthdates as identifiers, instead of using complete first and last names. For example, some students (10%) did not provide complete demographic data, which made linking impossible. This linking system was necessary for the execution of the study because a number of school administrators objected to having full identifying information on the questionnaires. The system linked 24% of participants across all four waves, 22 % across three waves, and 19% across only two waves. Hecht et al. (2003) provides a detailed listing of the linking procedure and students lost at each wave. The inability to link a student, however, was not significantly associated with ethnic group (Kulis et al. 2005). Thus it is unlikely that the linking procedures introduced selective attrition bias.

If there were no missing data, the 4,222 students across four waves would have yielded 16,888 observations. However, the study contains approximately 11,600 observations across four waves, with a missing data rate of about 31%. This is not unusually high, as the literature reports typical attrition rates of adolescent school-based studies to be between 18% and 56% (Aneshensel et al. 1989; Josephson and Rosen 1978). Using multivariate logistic regression, we conducted an attrition analysis to examine which student characteristics predicted them to be missing at any of the four study waves. Students with better academic performance were significantly less likely to miss a survey. Significantly more attrition was found among students receiving free or reduced lunches, and students with higher marijuana and cigarette consumption. Gender and alcohol use did not significantly predict attrition, nor did American Indian status. This latter finding is important because it suggests that there were not selectively higher rates of missing information by American Indian status.

Ad-hoc and simple methods of dealing with missing data, such as listwise deletion or mean imputation, often lead to biased hypothesis tests (Little and Rubin 2002). We employ a more appropriate method, multiple imputation (Little and Rubin 2002), which has been used successfully in studies of program efficacy (Graham et al. 2002). The critical assumption is that, conditional on other non-missing attributes, the data are missing at random (MAR). Although this assumption is un-testable, the assumption can be strengthened by including relevant predictors in an imputation model. Using the PROC MI procedure in SAS 9.1, we created 10 complete datasets, and then analyzed them with complete-data methods. The results of these complete-data analyses are combined to arrive at a single estimate that properly incorporates the uncertainty in the imputed values. Hierarchical linear models, such as growth curve modeling, are compatible with multiple imputation procedures (Raudenbush and Bryk 2002).

Results

Before discussing the statistical models, we examine several figures that illustrate the observed trajectories of substance use for different groups in our sample: American Indian and non-American Indian students who were participants in the treatment program or control condition. Figure 1 displays the alcohol use for these groups over the four survey waves. One trend that stands out is the increase in use for all groups. This is not surprising, as middle school is a time of experimentation and increasing substance use. The important question was whether the treatment program slows the growth in use compared to the control condition. For non-American Indian students, it appears that the treatment was successful. The dotted line (treatment) has a lower slope than the solid line (control). For American Indian students (the lines marked with squares), however, the treatment program is not effective. The dotted line (treatment) has a steeper slope than the solid line (control).

Fig. 1. Observed alcohol use trajectories for entire sample (N = 4,222).

Figure 2 illustrates the usage in marijuana across the four waves. Similar to the figure for alcohol, the program appears to be effective for non-American Indian students (growth is slower for treatment condition than control), but ineffective for the American Indian students.

Fig. 2. Observed marijuana use trajectories for entire sample (N = 4,222).

Figure 3 illustrates the observed means for cigarette use. Cigarette usage appears unaffected by the treatment for both American Indian and non-American Indian students.

Fig. 3. Observed cigarette use trajectories for entire sample (N = 4,222).

These figures, however, are not statistical, and they do not account for other covariates or the uncertainty in the estimates. Although they point to differences in treatment effects between American Indian and non-American Indian students, growth curve models were needed to formally test if the trends were significant.

Growth Curve Models

Table 2 presents the results of three conditional growth curve models. Each model has the same predictors, but a different substance use outcome. Coefficients are separated into two groups: 1) coefficients predicting initial substance use levels at the pre-test wave and 2) coefficients predicting growth in substance use across the four survey waves. Although our main concern is with the growth in substance use over time, we briefly describe the coefficients for initial use. In the model for alcohol (model 1 in Table 2), significant predictors of initial use were grades (β₀₅ = −.15, P < .001) and free or reduced lunch status (β₀₆ = −31, P < .01). Controlling for other predictors, the interpretation of these coefficients means that, at the initial pre-treatment survey, each additional point on the self-reported grades scale was associated with .15 points less on the alcohol scale, and students receiving free or reduced lunches scored .31 points less on the alcohol scale. Because the keepin' it R.E.A.L. curriculum was randomly assigned to schools, we had no reason to expect that the curriculum would be associated with initial, pre-treatment substance use. We included a predictor for treatment (β₀₁), as well as an interaction between treatment and American Indian identification (β₀₃), to verify that there was no association between treatment assignment and initial use. The lack of significance for these coefficients gives us confidence that, although there is variation in initial use, these variations are not statistically different according to treatment status.

Table 2. Conditional linear growth models of substance use.

Model number	1	2	3
	Alcohol scale	Marijuana scale	Cigarette scale
Fixed effects
Initial use π0ij
Intercept γ000	.85** (.23)	1.06*** (.25)	.96*** (.21)
Treatment β01	.04 (.12)	−.06 (.09)	.02 (.08)
American Indian β02	.26 (.18)	.06 (.12)	.25 (.16)
Treatment * American Indian β03	−.28 (.21)	.12 (.16)	−.16 (.19)
Male β04	.05 (.06)	.23*** (.06)	.06 (.06)
Grades β05	−.15*** (.03)	−.18*** (.03)	−.15*** (.03)
Free/Reduced lunch β06	−.31** (.11)	−.31*** (.12)	−.28** (.11)
Growth π1ij
Intercept γ010	.33** (.08)	.23** (.07)	.14 (.11)
Treatment β11	−08* (.03)	−.03 (.03)	−02 (.03)
American Indian β12	−.14 (.09)	−03 (.04)	−.07 (.08)
Treatment * American Indian β13	.19* (.10)	.11* (.05)	.11 (.09)
Male β14	.01 (.03)	−.02 (.02)	.05* (.02)
Grades β15	−.01 (.01)	−.01 (.01)	−.01 (.01)
Free/Reduced lunch β16	.00 (.04)	.05 (.03)	.03 (.05)
Random effects
Level 1
Temporal variation etij	1.24	1.07
Level 2
Initial status r0ij	1.22***	1.42***
Growth r1ij	.40***	.43***
Level 3
Initial status u00j	.24***	.21***
Growth u01j	.04*	.07***
N	16,888	16,888	16,888

Note: Coefficients displayed are parameter estimates; standard errors are in parentheses

^*P < .05,

^**P < .01,

^***P < .001, two-tailed tests

The coefficients for growth indicate how predictors affect the increase in substance use over time. In the growth coefficients in model 1, the coefficient for the intercept (γ₀₁₀) indicates that, controlling for gender, grades, and lunch status, the alcohol scale increased on average by .33 points per wave for students who were not American Indian and who were not in the treatment group. However, for non-American Indian youth who were in the treatment group, alcohol use increased by only .25 points per wave (γ₀₁₀ + β₁₁ = .33 − .08 = .25), which was a significantly lower growth in use (P < .01). Further, the significant positive coefficient for the Treatment * American Indian interaction (β₁₃) indicates that the program was significantly less effective for American Indian students (P < .05). In other words, for American Indian students in keepin' it R.E.A.L., their alcohol use increased by .30 points per wave (γ₀₁₀ + β₁₁ + β₁₂ + β₁₃ = .33 − .08 − .14 + .19 = .30). American Indian students not in the program had their alcohol use increase by only .19 points per wave (γ₀₁₀ + β₁₂ = .33 − .14 = .19).

Model 2 in Table 2 examines the growth in marijuana use across the four waves. The interpretation of coefficients in these models is identical to previous models. The significant and positive coefficient for Treatment * American Indian in model 2 (β₁₃ = .11, P < .05) indicates that keepin' it R.E.A.L. was less effective for urban American Indians than for other students in reducing the growth in marijuana used in the last month. As in the model of alcohol, these models suggest that the keepin' it R.E.A.L. curriculum does not slow the use of marijuana among urban American Indian students as it does for other populations. Model 3 in Table 2 focused on cigarette use. There was not a significant interaction between Treatment and American Indian ethnicity (β₁₃).

Discussion

This study utilized growth curve modeling to investigate the developmental trajectory of drug use among a sample of mostly urban and largely multi-ethnic American Indian youth that participated in the keepin' it R.E.A.L. drug prevention program. We investigated the potential for the curriculum, originally created for use with Mexican American youth, to achieve positive residual effects related to gateway drug use for a sample of urban youth with partial or full American Indian heritage. Prior to the current study, no evaluation studies of keepin' it R.E.A.L. examined the program's effect on urban American Indian youth. Our findings illustrated steeper increases in the drug use of American Indian students participating in the program across four waves of data compared with other youth. These findings with the urban American Indian participants contrast with previous studies that demonstrated that the program slowed the increase over time in substance use for students from all racial/ethnic backgrounds (Hecht et al. 2003) as well as students of Mexican heritage (Kulis et al. 2005). In addition, based on our findings, it appears that their usage of cigarettes was not curbed after participating in the keepin' it R.E.A.L. program. However, it should be noted that in the overall sample of students, cigarette use was less common than either alcohol or marijuana use, and desired treatment effects were thus less pronounced for cigarettes than for other substances (Hecht et al. 2003). Overall, the findings are particularly striking when considering the efficacy of the keepin' it R.E.A.L. program with other ethnic/minority youth. Collectively, our findings indicate that this program was not effective for reducing or curbing the substance use in this sample of urban, largely multiethnic, southwest American Indian youth. However, our findings are consistent with prior research that illustrates chronically high rates of alcohol and marijuana use for American Indian youth (Moncher et al. 1990; NIAAA 2002; Novins and Mitchell 1998; OTA 1990).

The findings related to alcohol and marijuana use for American Indian youth in this study are particularly notable, as prior evaluation research on the program found its strongest effects on the use of these substances with youth from other racial/ethnic backgrounds (Hecht et al. 2003; Kulis et al. 2005). In other words, participation in keepin' it R.E.A.L. appears to curb the use of marijuana and alcohol for certain racial/ethnic youth groups with the exception of certain American Indian youth. These findings are important because they lend evidence to the limited generalizability of culturally grounded drug prevention programs for certain youth ethnic groups, and even suggest the possibility that an inappropriate match between program and participant characteristics might actually lead to an increase in the problem behavior. As such, these findings inform the growing literature on iatrogenic effects in drug prevention programs (Dishion et al. 1999; Werch and Owen 2002), which have rarely addressed issues of ethnicity or culture in their analyses. Dishion et al. describe the process of “deviancy training” for high-risk youth populations participating in prevention programs, which refers to the positive peer reinforcement for antisocial behaviors (e.g., drug use, delinquency) that these youth receive by participating in these programs. It is unlikely that this process is pervasive with keepin' it R.E.A.L., since the program has demonstrated positive prevention effects with Mexican, Mexican-American, African American, and Caucasian youth. It is more likely that the iatrogenic effects related to the program for American Indian youth is a result of program implementation or theory error (Werch and Owen 2002).

Specifically, our findings suggest that the theoretical principles for keepin' it R.E.A.L. derived from prior research may not be culturally inclusive of American Indian youth. Evaluation findings from keepin' it R.E.A.L. supported the hypothesis that prevention programs reflecting the broader, regionally specific, multicultural environment of minority youth would result in better outcomes for them, compared with programs that do not reflect this environment (Kulis et al. 2005). While previous research suggests that broader, multicultural versions of drug prevention curricula may be more appropriate for minority youth (Botvin et al. 1997; Hecht et al. 2003; Kulis et al. 2006), these findings might not be applicable to American Indian youth based on our findings. Urban American Indian youth share a unique socio-historical context (e.g., reservation lifestyle, extended family structure, complex ethnic and cultural identities) that may not be reflected in multicultural versions of drug prevention curricula. Further, Native-specific theory indicates that adverse health outcomes, such as drug use, may be related to current and past trauma of Indigenous peoples such as colonization, traumatic life events, and discrimination (Walters et al. 2002). These unique social stressors, and the program's inability to address them, may also have affected the overall efficacy of the program on the population. American Indian-specific curricula may need to address this context more specifically in order to achieve positive prevention effects. Further, it may be important for future American Indian-specific curricula to take into account the level of racial/ethnic identity development of prospective student-participants.

Overall, our findings inform the question concerning the level of cultural specificity necessary for multicultural prevention programs to achieve positive effects on youth behaviors. Specifically, they add new information to the debate regarding whether drug prevention curricula need to be culturally “matched” to the program participants to achieve maximum benefit. While the findings from this study are consistent with the cultural matching hypothesis, it is important to note that they do not provide definitive support that drug prevention programs need to be matched to the cultural worldviews of urban American Indian youth. Future research focused on Native-specific drug prevention programs may need to examine the level and type of cultural matching necessary to achieve positive prevention effects with urban American Indian youth.

On a broader level, the findings from this study have implications for the identification and dissemination of evidence-based interventions for minority youth populations. While evidence-based practices for children and adolescents have received much attention over the past decade (e.g., Brestan and Eyberg 1998; Chorpita et al. 1998), they have been critiqued for lack of attention to their real world applicability or generalizability (Okamoto and LeCroy 2004; Raw 1998). The findings from this study illustrate how cultural variation can impact both the efficacy and effectiveness of empirically based interventions. They also point to the necessity of more research focused on culturally specific prevention interventions in order to ameliorate problem behaviors of minority youth populations.

Study Limitations

When interpreting the study's findings, several limitations should be noted related to the study design, contamination, attrition issues, ethnic heterogeneity, and generalizability. Block randomization was utilized in the assignment of schools to one of three versions of the intervention or to the control condition; however, it is possible that the groups of participants may have been influenced by historical events within the schools' environment or student and teacher interactions in the varying schools and in the different intervention/control conditions. Further, the sample size of American Indian youth in this study was too small to examine hypotheses by specific version of the curriculum. The small sample size of American Indian youth also prevented us from conducting meaningful sub-group analyses based on tribal affiliation and the complex set of ethnic identities represented among those with American Indian origins, which would have elucidated within-group heterogeneity.

Less than a third of the sample claiming an American Indian background chose that description exclusively. Our definition of American Indian was broad, and many students claimed it in one or two of the study waves but not in other waves. In addition, there were much larger numbers claiming both Mexican and American Indian heritage. Because many of our participants claimed American Indian heritage at one of the four waves and not at others, it is likely that this group of middle school students may have ethnic identities that are still forming or shifting. At times their American Indian ethnicity may be more salient for them than at other times. Thus, our results may indicate that the keepin' it R.E.A.L. program is less effective for urban youth with complex multi-ethnic identities or those with still unformed or shifting ethnic identities. Further explorations of the salience of urban American Indian identity and ethnic identity development for middle school students, and resulting subgroups in the context of the urban southwest, may be necessary to understand how the findings inform efforts to prevent youth substance use.

Although methods for handling missing data compensated for possible attrition effects to some degree, it is important to note that there was substantial participant attrition across the data collection waves. The generalizability of the study findings are limited, as the sample included American Indian youth in public middle schools with non-Indian majorities in a large southwestern metropolitan area, and included urban youth who mostly lived off tribal reservations. Currently there are 562 federally recognized tribes across the U.S. (Bureau of Indian Affairs 2003) and many non-recognized tribes, with significant cultural and regional differences such as living situations, degree of Indian ancestry or blood quantum, and cultural affiliation, identity, and participation (Hawkins et al. 2004). The findings from the current study may not reflect these differences.

Implications for Future Research

Future research needs to examine the efficacy and effectiveness of multicultural and Native-specific prevention programs in order to determine the degree of cultural matching necessary for urban American Indian youth. These programs may need to be culturally “grounded” in the worldviews of American Indian youth in order to achieve maximum drug prevention benefits for these youth. Future research should also investigate the influence of complex multiethnic identities of urban American Indian youth, including the impact of ethnic identity development for them. The perceptions of Native youth, tribal elders, and school practitioners working with the population may need to be incorporated in the content and delivery of the prevention program. This process is similar to the development of prevention programs for other ethnic youth populations (Harthun et al. 2002), and may enhance the psychometric, developmental, and cultural validity of the program for American Indian youth.

Conclusions

While previous evaluations of the keepin' it R.E.A.L. curriculum support its efficacy with specific groups of racial/ethnic adolescents (Hecht et al. 2003; Kulis et al. 2005; Marsiglia et al. 2005), the current study suggests that the program may have limited effectiveness in curbing the drug use of American Indian youth. Despite the current study's limitations, our findings have implications for the development of culturally grounded prevention programs in schools, reservation, and non-reservation communities for American Indian youth and for other minority youth populations. Our findings suggest that American Indian youth may require drug prevention curricula that are specific to their developmental and cultural realities. In order to effectively address drug use among American Indian youth, prevention researchers and specialists may need to focus on creating and/or modifying drug prevention programs in order to address the daily traditions, cultures, and values of American Indian youth.