Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Apr 1.
Published in final edited form as: Drug Alcohol Depend. 2016 Feb 10;161:214–221. doi: 10.1016/j.drugalcdep.2016.02.007

Use of Evidence-Based Treatments in Substance Abuse Treatment Programs Serving American Indian and Alaska Native Communities

Douglas K Novins 1,*, Calvin D Croy 1, Laurie A Moore 1, Traci Rieckmann 2
PMCID: PMC4817996  NIHMSID: NIHMS759381  PMID: 26898185

Abstract

Background

Research and health surveillance activities continue to document the substantial disparities in the impacts of substance abuse on the health of American Indian and Alaska Native (AI/AN) people. While Evidence-Based Treatments (EBTs) hold substantial promise for improving treatment for AI/ANs with substance use problems (as they do for non-AI/ANs), anecdotal reports suggest that their use is limited. In this study, we examine the awareness of, attitudes towards, and use of EBTs in substance abuse treatment programs serving AI/AN communities.

Methods

Data are drawn from the first national survey of tribal substance abuse treatment programs. Clinicians or clinical administrators from 192 programs completed the survey. Participants were queried about their awareness of, attitudes towards, and use of 9 psychosocial and 3 medication EBTs.

Results

Cognitive Behavioral Therapy (82.2%), Motivational Interviewing (68.6%), and Relapse Prevention Therapy (66.8%) were the most commonly implemented psychosocial EBTs; medications for psychiatric comorbidity was the most commonly implemented medication treatment (43.2%). Greater EBT knowledge and use were associated with both program (e.g., funding) and staff (e.g., educational attainment) characteristics. Only two of the commonly implemented psychosocial EBTs (Motivational Interviewing and Relapse Prevention Therapy) were endorsed as culturally appropriate by a majority of programs that had implemented them (55.9% and 58.1%, respectively).

Conclusions

EBT knowledge and use is higher in substance abuse treatment programs serving AI/AN communities than has been previously estimated. However, many users of these EBTs continue to have concerns about their cultural appropriateness, which likely limits their further dissemination.

Keywords: Indians, North American, Substance Abuse Treatment Centers, Diffusion of Innovation

1. INTRODUCTION

The dissemination and implementation of evidence-based treatments (EBTs) by substance abuse treatment programs remains one of the greatest challenges we face in improving the quality of such services (Institute of Medicine, 2006). In no part of American society is the need for quality substance abuse services greater than in American Indian and Alaska Native (AI/AN) communities, where the rates of substance use problems are higher than in the rest of the United States and access to care remains limited (Beals et al., 2006, 2005; O'Connell et al., 2005; Whitesell et al., 2012). While EBTs have the potential to improve substance abuse treatment services for AI/ANs, as they do for non-AI/AN populations, there have been a number of concerns raised by experts in this area regarding efforts to increase EBT use (Gone and Looking, 2011; Novins et al., 2011). These include longstanding concerns regarding the cultural appropriateness of many EBTs as well as a lack of guidance on how to adapt interventions for AI/AN populations while maintaining their effectiveness (Novins et al., 2011). Furthermore, the imposition of policy mandates by federal and state authorities to use EBTs in order to receive funding may inadvertently make them even more controversial by placing their use in opposition to tribes’ continued efforts to maintain their sovereign status (Novins et al., 2011). Despite these long-standing concerns, engagement with EBTs (i.e., awareness, attitudes towards, and actual use) by substance abuse treatment programs serving AI/AN communities has not been studied systematically, leaving the above concerns in the realm of expert opinion and limiting our ability to improve the process of disseminating and implementing EBTs in programs serving AI/AN communities.

In contrast with research on substance abuse programs serving AI/AN communities, there is a large and growing literature on the use of EBTs in substance abuse treatment programs more generally, enough to support at least two systematic reviews (Garner, 2009; Walters et al., 2005). Organizational factors associated with greater EBT engagement include larger program size (Guerrero et al., 2013), organizations that are younger (Lundgren et al., 2012), having better internet technology (Lundgren et al., 2011b), lower levels of organizational stress (Lundgren et al., 2012), accepting private insurance (Guerrero et al., 2013), the use of total quality management techniques (Fields and Roman, 2010), and supervisor expectations regarding EBT use (Guerrero et al., 2013).

Similarly, workforce factors associated with greater EBT engagement include higher levels of clinician education (Lundgren et al., 2011b) and clinical experience (Bride et al., 2010; Ducharme et al., 2010), positive attitudes to science-based treatments (Bride et al., 2010) as well as training in (Bride et al., 2010) and experience with specific EBTs (Bride et al., 2010; Lundgren et al., 2012).

There is also evidence of variation in the factors supporting the implementation of different EBTs, particularly between psychosocial and medication EBTs (Oliva et al., 2011). For example, McGovern, Fox et al. (2004) reported that while clinicians who labeled themselves as either using a 12-step model or cognitive behavioral model for treatment reported comparable interests in psychosocial EBTs such as Relapse Prevention Therapy and Motivational Interviewing, those therapists using a cognitive behavioral treatment model were more open to using medication EBTs. Rieckmann et al. (2011) reported similar findings regarding use of buprenorphine, with less emphasis on 12-step services and a greater percentage of clients with opiate use disorders being associated with a greater likelihood to offer buprenorphine treatment. Among the medication EBTs, Knudsen, Abraham, and Roman’s (2011b) work suggests that use of medications for the treatment of comorbid psychiatric conditions was more common than the use of medications for relapse prevention. Organizational factors associated with use of medication EBTs include access to medical staff, for-profit institutional structure, larger program size, placement in a hospital setting, accreditation, and greater access to trainings and to web-based materials), and program participation in research (Abraham et al., 2009, 2013, 2011, 2010; Ducharme and Roman, 2009; Knudsen et al., 2011a; Krull et al., 2011; Roman et al., 2011; Savage et al., 2012).

Research also suggests that modifications to EBTs are often made in substance abuse treatment settings, but that these modifications vary substantially across settings (Lundgren et al., 2011a). Furthermore, many programs that use EBTs do not provide training and ongoing support for high quality implementation (Olmstead et al., 2012).

Drawing on data from the first national study of substance abuse treatment programs serving AI/AN communities, the goal of this paper is to examine the depth of engagement with EBTs in these programs.

2. METHODS

Data for these analyses come from the Centers for American Indian and Alaska Native Health’s Evidence-Based Practices and Substance Abuse Treatment for Native Americans project. This project focused on how substance abuse treatment programs serving AI/AN communities use and perceive EBTs. An advisory board of administrators, service providers, evaluators from the AI/AN substance abuse treatment community, and researchers with expertise in AI/AN substance abuse treatment and dissemination research supports this project.

This project consisted of three phases: 1) convening an advisory board to identify key issues in the dissemination and implementation process and to develop study measures and methods (Novins et al., 2011), 2) completion of qualitative case studies of 18 substance abuse treatment programs serving AI/AN communities (Legha et al., 2014; Legha and Novins, 2012; Moore et al., 2015), and 3) conducting a national survey of AI/AN substance abuse treatment programs to explore their use of EBTs (Novins et al., 2012). This paper draws on the data collected during this final phase.

2.1. Participants and Study Procedures

As described in detail elsewhere (Novins et al., 2012), data collection was conducted using a stratified sampling approach, dividing these programs into the following five strata: 1) the 20 largest AI/AN tribes, 2) urban AI/AN health clinics; 3) substance abuse services operated by the AN Health Corporations; 4) other tribes (federally recognized minus the 20 largest); and 5) other local and regional programs (independent nonprofit or for profit).

Using existing tribal, organizational and substance abuse program listings, consultation with Indian Health Service and state substance abuse treatment administrative staff, and the analysis of publicly-available information on the Worldwide Web, we identified specific treatment programs that had the potential to provide substance abuse services to AI/AN communities. We then contacted each identified program and determined whether it provided substance abuse treatment services to AI/AN communities. If the program confirmed providing such services, we described the project and asked whether there was a clinical administrator or other senior clinical staff whom we could ask to complete the survey (Novins et al., 2012). Once this staff member was identified and agreed to participate in the study, the staff member was given the choice of completing the survey online or over the telephone. Only two participants chose the telephone interview. The others were emailed a link to the survey for completion. Given the contingent question structure of the survey (with more questions asked when respondents endorsed greater experience with specific EBTs), completion time varied from 20–60 minutes. Once data collection was completed, all identifying information was deleted from the project databases, rendering these data anonymous. A total of 192 surveys were completed, yielding an overall participation rate of 63%, consistent with meta-analyses of participation rates in telephone and internet surveys (Cook et al., 2000; Van Horn et al., 2009).

Key sample characteristics are summarized in the left-hand columns of Table 1. The majority of programs were located in rural areas (74.0%) and were operated by a tribe or tribal consortium (63.0%); only 24.5% were accredited. The average number of front-line clinical staff was 5.6 with 83.3% reporting having at least one staff member who identified as AI/AN. The majority of programs reported that they collected data on treatment outcomes (64.2%) and consider EBTs in their strategic planning (58.3%). Study procedures were approved by the Colorado Multiple Institutional Review Board, who classified the study as exempt; and the Oregon Health and Science University’s Institutional Review Board, who classified the study as expedited. The Indian Health Service Institutional Review Board classified the study as not human subjects research.

Table 1.

Sample Characteristics and Crude (Univariate) Associations with Evidence-Based Treatment Engagement

Sample
Characteristics		 Engagement with
Psychosocial
EBTs		 Engagement with
Medication EBTs
%/mean SE B SE B SE
Program Characteristics
    Program location is rural 74.0 3.17 0.001 0.14 0.04 0.26
    Program is located at a tribal location 81.3 2.82 0.14 0.15 0.32 0.29
    Type of program1
        tribal 53.1 3.60 −0.03 0.15 −0.42 0.28
        tribal consortium 9.9 2.16 0.13 0.21 −0.18 0.39
        IHS or federal 51.0 3.61 0.16 0.13 −0.07 0.25
        independent 25.0 3.13 −0.10 0.16 −0.32 0.31
    Funding1
        Direct, 638 compact, block grant, or tribal 65.6 3.43 0.53 0.13 * 0.31 0.25
        HRSA or 638 contract 57.3 3.57 0.23 0.13 0.001 0.24
        Grants (SAMHSA or private foundation) 39.6 3.53 0.23 0.12 0.33 0.23
        Medicaid or fee for service 65.1 3.44 0.23 0.12 0.72 0.24 *
    Program accredited 24.5 3.10 0.07 0.14 0.37 0.26
    Serves adolescents 84.4 2.62 −0.19 0.17 0.65 0.31
    Levels of care provided1
        ASAM Level 0.5, early intervention 64.6 3.45 −0.31 0.13 0.06 0.25
        ASAM Level I, outpatient treatment 89.1 2.25 0.10 0.22 0.17 0.43
        ASAM Level II, intensive outpatient/partial
hospitalization		 34.9 3.44 0.41 0.13 * 0.02 0.25
        ASAM Level III, residential/inpatient treatment 19.8 2.88 −0.04 0.16 −0.07 0.31
        ASAM Level IV, medically-managed intensive
inpatient treatment		 2.1 1.03 0.24 0.41 1.52 0.80
    Program has an electronic health record system 55.1 3.64 0.28 0.12 0.56 0.22 *
Staff Characteristics
    Number of front line clinical staff 5.6 0.55 0.02 0.01* 0.02 0.01
    Number of disciplines among clinical staff 3.1 0.10 0.13 0.04* 0.28 0.08 *
    Average years of staff education 16.4 0.11 0.07 0.04 0.20 0.07 *
    Average years of employment in program 5.6 0.26 −0.01 0.02 0.00 0.03
    Staff that are certified addiction counselors
        none 10.4 2.20 --A --A --A --A
        1–50% 46.4 3.60 0.61 0.20 * 1.05 0.38 *
        more than 50% 43.2 3.58 0.77 0.20* 0.92 0.38
    Staff in recovery from alcoholism
        none 14.6 2.55 --A --A --A --A
        1–50% 53.1 3.60 0.31 0.17 0.80 0.33
        more than 50% 29.7 3.36 0.39 0.19 0.60 0.36
    Percent of clinical staff that are American Indian or
Alaska Native
        none 16.7 2.69 --A --A --A --A
        1–50% 44.8 3.59 0.22 0.17 0.40 0.32
        more than 50% 38.5 3.51 0.16 0.18 −0.08 0.33
    Program struggles to recruit and retain clinical staff 51.6 3.61 0.19 0.12 0.21 0.22
    Program has open clinical staff positions 38.0 3.50 0.14 0.12 0.34 0.23
    Program has continuing education requirement 81.3 2.82 0.47 0.15* 0.56 0.29
Clinical Assessment and Program Evaluation
    Structured assessment of client's substance abuse
at intake		 76.6 3.06 0.21 0.14 −0.22 0.26
    Formal method to assess client's motivation for
change at intake		 27.5 3.46 0.44 0.14* 0.59 0.27
    Program evaluates effectiveness of services
provided		 65.8 3.47 0.29 0.12 0.17 0.24
    Program collects data on treatment outcomes 64.2 3.51 0.30 0.12* 0.02 0.23
    Program has outside evaluator examine the quality
of services		 40.6 3.59 0.19 0.12 −0.12 0.23
    Program has participated in a research or program
evaluation		 31.6 3.40 0.35 0.13* 0.31 0.24
Evidence-Based Treatment Experience and Attitudes
    Feels pressure to use Evidence-Based Treatments 36.5 3.47 0.11 0.12 0.06 0.23
    Program requires clinical staff to use Evidence-
Based Treatments		 43.8 3.58 0.43 0.12* 0.30 0.23
    Believes working with an expert in a particular
treatment would help improve services		 95.7 1.48 −0.12 0.30 0.72 0.55
    Evidence-Based Treatments are considered in
strategic planning		 58.3 3.61 0.48 0.12* 0.59 0.22
    EBPAS Openness Subscale 2.3 0.06 0.30 0.08* 0.16 0.15
    EBPAS Divergence Subscale 1.5 0.05 0.07 0.08 −0.12 0.16
Open in a new tab

Notes.

1

Percentages sum to more than 100 as participants were permitted to endorse more than one response.

%/Mean=percent for categorical variables, mean for continuous variables

SE=standard error

EBT=Evidence-Based Treatment

*

p≤.01

A=reference group

EBPAS=Evidence-Based Practices Attitudes Scale

2.2. Measures

The survey was designed by the Advisory Board drawing on examples of other surveys of substance abuse treatment programs, including the National Drug Abuse Treatment System Survey (Andrews et al., 2014), the Comprehensive Community Mental Health Services for Children and Their Families Program Evaluation (Center for Mental Health Services, 2005), the University of Georgia National Treatment Center Study (Knudsen et al., 2011b), and the Assessment of the National Drug Abuse Treatment Clinical Trials Network Suvey (McCarty et al., 2008) as well as the results of the qualitative interviews and focus groups conducted in the second phase of the project. The survey consisted of 17 sections: background information about the respondent and program, program workforce, assessment process, quality improvement and training procedures, and individual sections for the 12 EBTs described below. The survey may be accessed online at www.ucdenver.edu/caianh/ebp.

2.2.1. Dependent Variables: EBT Engagement

Our measures of EBT engagement were drawn from the section of the survey which examined awareness of, attitudes towards, and use of 9 psychosocial EBTs (Behavioral Couples Therapy, Cognitive Behavioral Therapy, Community Reinforcement and Family Training, Contingency Management, Matrix Model, Motivational Interviewing, Multisystemic Therapy, Relapse Prevention Therapy, Twelve Step Facilitation) and 3 psychopharmacologic EBTs (Medications for Comorbid Psychiatric Conditions, Medications for Relapse Prevention, Medications for Withdrawal). A brief description was provided for each EBT, then participants were asked to rate their program’s experience with that EBT on the following scale: 0= Unfamiliar with the EBT; 1=not interested in the EBT; 2=considered the EBT, but “see many pros and cons”; 3=planning on using the EBT, but have not used it yet; 4=using the EBT, but not a permanent part of the program; 5=made EBT a permanent part of the program; and 6=used the EBT in the past, but don’t use it currently. In addition to examining responses to these questions for each EBT individually, we created two summary scales of EBT engagement, one averaging the participant’s responses to the 9 psychosocial EBTs (Psychosocial EBT Engagement; Cronbach’s α=0.62); the other averaging responses to the 3 psychopharmacologic EBTs (Psychopharmacologic EBT Engagement (α=0.69). Confirmatory Factor Analyses (CFAs) supported the two-scale structure for these variables (see this manuscript’s supplementary materials for details1). For these scales (Psychosocial and Medication EBT engagement) we combined the two highest levels of the scale (i.e., responses of 6 were recoded to 5).

2.2.2. Independent Variables

Potential Predictors of EBT Engagement were drawn from the sections of the survey focused on program and staff characteristics, assessment and evaluation procedures, and attitudes towards EBTs. These variables were originally chosen by the project’s Advisory Board based on the literature regarding the dissemination and implementation of EBTs to substance abuse treatment programs or identified as being potentially important for programs serving AI/AN communities. Please see Table 1 and the online supplement2 for further details regarding these variables.

2.2.3. EBT Implementation Follow-up Questions

Among participants that reported implementing specific psychosocial EBTs, we examined participant responses to two key implementation questions, how the EBT is used in the program (following the manual exactly, using the parts of the manual perceived as most helpful, rewriting the manual to make it fit better with the program and/or more culturally appropriate, not using the manual but drawing on key concepts of the EBT), and its perceived cultural appropriateness (i.e., endorsing an asset of the EBT that it is culturally appropriate).

2.3. Analytic Plan

First we calculated percentages for the different levels of engagement for each of the EBTs. , Next we developed two multiple regression models, one predicting the level of psychosocial EBT engagement and another predicting the level of psychopharmacologic EBT engagement. We selected variables for the multiple regressions that had univariate associations with the engagement scale scores with p≤0.25 (Hosmer et al., 2013). Then, we used backward elimination to remove variables from both models until all remaining variables were either themselves significant at p≤0.05 or belonged to a set of variables in which at least one was significant. Because of the multiple comparisons involved in conducting the initial univariate analyses, we discuss only differences significant at the level of p≤0.01; for the multivariate analyses we discuss associations significant at the p≤0.05 level (Sokal and Rohlf, 1969; Weiss, 1999). Finally, we calculated the percentage of programs utilizing the different methods of implementation and the percentage that endorsed specific EBTs as culturally appropriate. We restricted these final analyses to those EBTs where at least 20% of the sample reported implementation to assure an adequate sample size.

3. RESULTS

3.1. EBT Engagement

3.1.1. Psychosocial Treatments

Levels of engagement with the 9 psychosocial treatments queried in the survey are summarized in the top of Table 2. Among the 5 most commonly implemented psychosocial treatments, Matrix Model had the highest proportion of implementers (i.e., programs scoring ≥4 on the engagement scale for that EBT) to report that they had not committed to permanent use of the intervention (32.9% compared to an average of 26.0% for all 5 of these treatments) and the highest proportion who reported discontinuing its use (17.3% compared to an average of 5.8%). Overall, 95.8% of programs reported implementing at least one of these nine psychosocial treatments, but the average score on the global psychosocial treatment scale was 2.2, reflecting that four of these treatments were unfamiliar to most programs (Contingency Management, Behavioral Couples Therapy, Community Reinforcement and Family Training, and Multisystemic Therapy).

Table 2.

Percentages of Participants Having Different Levels of Treatment Use

Raw Ratings
 Descriptive Ratings
Not
Familiar		 Not
interested
in		 See
pros
and
cons		 Planning
on using		 Using but
not
permanent		 Permanent
use		 Used
in
past
0 1 2 3 4 5 6 Percent With
Rating ≥ 4		 Mean
Rating		 SD of
Ratings
Psychosocial Treatments
    Cognitive Behavioral Therapy 4.2 1.1 8.9 3.7 24.1 56.4 1.6 82.2 4.2 1.3
    Relapse Prevention Therapy 17.0 3.2 6.4 4.8 14.9 52.6 1.1 68.6 3.6 1.9
    Motivational Interviewing 11.1 2.6 10.5 9.0 19.5 45.7 1.6 66.8 3.7 1.7
    Twelve-Step Facilitation 24.1 7.0 10.7 4.3 9.1 41.6 3.2 54.0 3.0 2.1
    Matrix Model 25.9 10.6 19.1 7.4 12.2 18.5 6.4 37.0 2.4 1.9
    Contingency Management 55.3 7.4 13.7 5.8 6.3 9.5 2.1 17.9 1.4 1.8
    Behavioral Couples Therapy 69.3 3.1 16.2 4.2 4.2 2.6 0.5 7.3 0.8 1.4
    Community Reinforcement & Family
Training		 80.0 5.8 4.2 7.9 2.6 2.1 0.5 5.2 0.6 1.3
    Multisystemic Therapy 81.5 5.8 7.9 1.1 1.1 1.6 1.1 3.7 0.4 1.1
    All Psychosocial Treatment Ratings 95.8 2.2 2.1
Medication Treatments
    Meds for Comorbidity 37.9 4.2 11.1 3.7 7.4 35.3 0.5 43.2 2.5 2.2
    Meds for Relapse Prevention 26.5 14.8 24.9 5.8 9.5 16.4 2.1 28.0 2.1 1.8
    Meds for Withdrawal 32.6 25.3 15.8 2.6 6.8 15.3 1.6 23.7 1.8 1.8
    All Medication Treatment Ratings 54.2 2.1 2.0
Open in a new tab

3.1.2. Medication Treatments

Levels of engagement with the 3 medication treatments queried in the survey are summarized in the bottom of Table 2. Just 54.2% of the programs reported implementing at least one of these three medication treatments, reflecting the considerable unfamiliarity with these treatments and either a lack of interest or ambivalence (i.e., seeing pros and cons) in use of medications for relapse prevention (39.7%) and withdrawal (41.1%).

3.2. Factors Associated with EBT Engagement

Twelve of 37 variables had crude (univariate) associations with engagement with psychosocial treatments (Table 1 middle columns); 6 in the final multivariate model (Table 3). In the final multivariate model, engagement with the psychosocial EBTs was associated with receiving direct IHS funding for services, greater mean years of staff education, having staff that are certified addiction counselors, requiring clinical use of EBTs, considering EBTs in strategic planning, and higher scores on the EBPAS Openness Scale. Five of 37 variables had crude (univariate) associations with EBT engagement for medication treatments (Table 1); 4 in the final multivariate model (Table 3). In the final multivariate model, engagement with medication EBTs was associated with receiving Medicaid or fee for service reimbursements, serving adolescents,

Table 3.

Adjusted (Multivariate) Associations with Evidence-Based Treatment Engagement from Final Regression Models

Engagement with
Psychosocial EBT
 Engagement with
Medication EBTs
B SE B SE
Program Characteristics
    Funding
        Direct, 638 compact, block grant, or tribal 0.32 0.13* 0.13 0.26
        HRSA or 638 contract 0.17 0.12 −0.28 0.25
        Grants (SAMHSA or private foundation) 0.11 0.11 0.33 0.23
        Medicaid or fee for service 0.03 0.12 0.61 0.24*
    Serves adolescents 0.61 0.30*
Staff Characteristics
    Average years of staff education 0.07 0.03* 0.22 0.08*
    Staff in recovery from alcoholism
        none --A --A
        1–50% 0.77 0.33*
        more than 50% 0.80 0.39*
    Staff that are certified addiction counselors
        none --A --A
        1–50% 0.41 0.19*
        more than 50% 0.48 0.20*
Evidence-Based Treatment Experience and Attitudes
    Program requires clinical staff to use Evidence-Based Treatments 0.23 0.11*
    Evidence-Based Treatments are considered in strategic planning 0.32 0.11*
    EBPAS Openness Scale 0.22 0.07*
Variance Explained by Final Regression Models (R2) 0.27 0.17
Open in a new tab

Notes.

EBT=Evidence-Based Treatment

*

p≤.05

A=reference group

3.3. Implementation Strategies and Perceived Cultural Appropriateness

Implementation strategies for the five psychosocial EBTs that were most commonly used by these programs are summarized in Table 4. Following the manual exactly was the third most common strategy utilized across these EBTs, ranging from 7.8% (Relapse Prevention Therapy) to 27.1% (Matrix). Among the programs that had implemented the five most commonly utilized psychosocial treatments, only Relapse Prevention Therapy and Motivational Interviewing were noted to be culturally appropriate by the majority of programs.

Table 4.

EBT Implementation Methods and Perceived Cultural Appropriateness*

Implementation Methods
Following
manual
exactly		 Using
parts of
manual		 Rewriting
manual		 Using
EBT key
concepts		 EBT is
culturally
appropriate
n % % % % %
Cognitive Behavioral Therapy 157 10.8% 31.2% 5.1% 52.9% 44.6%
Motivational Interviewing 126 19.1% 43.7% 0.8% 36.5% 55.9%
Relapse Prevention Therapy 128 7.8% 53.9% 5.5% 32.8% 58.1%
Twelve-Step Facilitation 101 23.8% 37.6% 3.0% 35.6% 37.6%
Matrix Model 70 27.1% 47.1% 11.4% 14.3% 42.9%
Open in a new tab

Notes.

*

Analyses performed on those EBTs implemented by 20% of more of programs, focusing on programs that had actually implemented that EBT.

4. DISCUSSION

Given the controversy that has surrounded efforts to increase the use of EBTs in substance abuse treatment programs serving AI/AN communities (Gone and Looking, 2011; Novins et al., 2011), the fact that the use of at least one psychosocial EBT was almost universal in these programs is an unexpected finding. This may reflect the extensive efforts at the policy level to encourage and support EBT exploration and implementation for substance abuse treatment programs more generally (Addiction Technology Transfer Center Network, 2014; Oregon Health Authority, 2015; Rieckmann et al., 2009; Substance Abuse and Mental Health Services Administration, 2015a, b) as well as efforts focused specifically on programs serving AI/AN communities (Addiction Technology Transfer Center Network, 2014; Oregon Health Authority, 2015). Certainly awareness and use of EBTs in these programs is far broader than suggested by thought leaders working in this area. However, consistent with these expert assessments, these results also suggest that depth of engagement is quite shallow. For example, that actual implementation of psychosocial treatments was concentrated in four EBTs (Cognitive Behavioral Therapy, Motivational Interviewing, Relapse Prevention Therapy, Twelve-Step Facilitation) while another four EBTs were unfamiliar to the majority of respondents (Contingency Management, Behavioral Couples Therapy, Community Reinforcement and Family Training, Multisystemic Therapy) suggests that programs serving AI/AN programs are only accessing a limited subset of psychosocial EBTs. The breadth of dissemination efforts should be examined to clarify how we can enhance these programs’ awareness of a broader array of treatments and their potential roles in service provision. This is particularly critical as the passive diffusion process within the AI/AN substance abuse treatment community is likely to reinforce the use of those EBTs that are well known among this network of programs and providers, making it difficult for other psychosocial treatments to garner the attention they deserve (Greenhalgh et al., 2004). That the majority of actual users of three of the five most commonly implemented EBTs did not rate them as culturally appropriate further underscores the need for better guidance on how to implement these treatments in culturally appropriate ways, including guidance on acceptable approaches to adaptation of treatment manuals. That most of the commonly utilized psychosocial EBTs are not typically implemented with fidelity is consistent with studies of substance abuse treatment programs serving non-AI/AN communities (Friedmann et al., 2007; Riley et al., 2008), but also underscores the need to develop new ways to support the implementation of these EBTs (Martino, 2010). Models of implementation that emphasize adapting interventions to fit better with program resources and client and community needs may be particularly useful in addressing these issues (Aarons et al., 2012).

In contrast, access to medication treatments is far from universal, especially since the medication EBTs focus on distinct therapeutic indications (far more distinct than the psychosocial EBTs, which have considerable overlap with one another in terms of their therapeutic targets). Use of medications for relapse prevention and withdrawal is particularly uncommon, a problem that has been described in substance abuse treatment programs more broadly (Knudsen et al., 2011b). Further exploration of these data will be needed to determine how much of this is due to lack of infrastructure, limited awareness and knowledge, and conflicts with overall treatment philosophy (Knudsen et al., 2011a).

EBT Engagement was found to increase with the occurrence of particular factors, similar to EBT implementation in studies of US substance abuse programs more broadly. For psychosocial EBTs, having base funding available to support services (direct funding from the Indian Health Service, Tribe, or from federal block grants) suggests that financial support not tied to specific grants or fee for service expectations provides greater flexibility to explore/implement these EBTs (Greenhalgh et al., 2004; Institute of Medicine, 2006). That engagement with the medication EBTs was higher among programs receiving Medicaid or fee for service funding is likely due to the importance of having the fiscal infrastructure to bill in a fee-for-service manner to support these services. Interestingly, we were unable to identify a relationship between grant funding and EBT engagement, a factor that had been hypothesized to be a key impetus towards EBT use in these programs (Novins et al., 2011), suggesting that factors other than grant funding are driving EBT use in these programs. That higher staff education was associated with greater engagement in both classes of EBTs, and that engagement with psychosocial EBTs in particular was higher when at least some staff were certified addiction counselors, underscores the importance of solid educational and training backgrounds for exploring and implementing EBTs. Similarly, that administrative procedures (program requirement to use EBTs, considering EBTs in strategic planning) and attitudes (openness to new treatment approaches) were associated with greater psychosocial EBT engagement show the importance of administrative support and organizational culture in facilitating the exploration and use of EBTs.

It was interesting that presence of staff in recovery from alcoholism was positively associated with the level of medication EBT engagement. This covariation seems to run counter to the findings from more general studies of US programs (McGovern et al., 2004; Oliva et al., 2011). Because the 12-step movement in tribal communities developed in unique ways that involved the blending of 12-step philosophies with AI/AN traditional beliefs, perhaps there has been less of an emphasis on avoiding medications than in mainstream 12-step programs (Novins et al., 2011).

Several key limitations of this study should be noted. First, this study relied on the reports of program staff members. While this is a standard approach for studies such as this one (Knudsen et al., 2011a), it does raise the risk of inflation of the rates of EBT engagement as this may have been viewed by participants as socially desirable (Manuel et al., 2011). However, given the low rates of reported engagement for many of the EBTs we queried and the low rates of reported implementation with fidelity to treatment manuals, it does not appear that social desirability had a pervasive impact on survey responses. Second, the focus on program-level engagement of EBTs means we were unable to examine the individual clinician-level factors that may be particularly important for EBT implementation. Third, the cross sectional non-experimental nature of these data mean that while we can identify factors associated with EBT engagement, we can describe neither the temporal process of exploration and implementation nor what factors advance this process. Fourth, by combining EBTs into two groups, psychosocial and medication treatments, the regression analyses conducted here focus on global indicators of EBT engagement. While this approach is informative, it is likely that programs often consider EBTs on a case by case basis. Further analyses regarding the implementation of specific EBTs will help elucidate these more targeted aspects of implementation. Finally, the participation rate in this survey was 63%. While this participation rate is consistent with other surveys using comparable methodologies (Cook et al., 2000; Van Horn et al., 2009), and is impressive given the long-standing reluctance of AI/AN communities to participate in research (Burhansstipanov et al., 2005; Novins et al., 2012), it still raises the risk of unmeasured bias in the programs that participated in the study versus those that did not. In particular it is possible that programs that have not implemented EBTs may have been less likely to participate in the survey, resulting in an overestimation of EBT engagement. However, the substantial variation in key program characteristics and in the rates of engagement with EBTs suggests that we successfully recruited a heterogeneous group of substance abuse treatment programs.

This first national study of substance abuse treatment programs serving AI/AN communities suggests that the degree of engagement with both psychosocial and medication EBTs is considerably greater than anecdotal analyses have suggested. However, this engagement is with a relatively limited number of EBTs that are often implemented without strong fidelity. Given this, along with the low endorsement of these EBTs as culturally appropriate, considerable additional work is needed to make the benefits of these treatments more available to AI/ANs with substance use problems.

Supplementary Material

1

Highlights.

  • We examined evidence-based treatment (EBT) use in tribal substance abuse treatment.

  • 96% of programs implemented a psychosocial treatment; 54% a medication treatment.

  • Greater EBT engagement was associated with program characteristics (e.g., funding) and staff characteristics (e.g., educational attainment).

Acknowledgments

The authors wish to acknowledge the Centers for American Indian and Alaska Native Health Research’s Substance Abuse Treatment Advisory Board, which includes the following members: Annie Belcourt (University of Montana); Gordon Belcourt (Montana-Wyoming Tribal Leaders Counsel, deceased); Kathleen Masis (Montana-Wyoming Tribal Leaders Counsel, Emeritus); Sarah G. Conti (Pagosa Springs, Colorado); Dennis Dahlke (Peaceful Spirit Alcohol Recovery Center); Raymond Daw (Yukon-Kuskokwim Healthcare Corporation, Alaska); Darren Dry (Jack Brown Center); Candace Fleming (Centers for American Indian and Alaska Native Health); John Gastorf (Cherokee Nation Behavioral Health, Emeritus); Craig Love (Westat); and Paul Spicer (University of Oklahoma).

Funding Source

This research was supported in part by the National Institute on Drug Abuse (R01-DA022239, Douglas Novins, Principal Investigator)

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

1

Supplementary material can be found by accessing the online version of this paper at http://dx.doi.org and by entering doi:…

Contributors

DN is responsible for the conception and design of the study. DN, LM, TR were involved in the acquisition of the study data. All authors were involved in the analysis and interpretation of study data, drafting of this manuscript, and its critical revision. All authors have approved the final article.

Conflict of Interest

No conflicts declared

References

  1. Aarons GA, Green AE, Palinkas LA, Self-Brown S, Whitaker DJ, Lutzker JR, Silovsky JF, Hecht DB, Chaffin MJ. Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implement Sci. 2012;7:32. doi: 10.1186/1748-5908-7-32. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Abraham AJ, Ducharme LJ, Roman PM. Counselor attitudes toward pharmacotherapies for alcohol dependence. J. Stud. Alcohol Drugs. 2009;70:628–635. doi: 10.15288/jsad.2009.70.628. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Abraham AJ, Knudsen HK, Rieckmann T, Roman PM. Disparities in access to physicians and medications for the treatment of substance use disorders between publicly and privately funded treatment programs in the United States. J. Stud. Alcohol Drugs. 2013;74:258–265. doi: 10.15288/jsad.2013.74.258. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Abraham AJ, Knudsen HK, Roman PM. A longitudinal examination of alcohol pharmacotherapy adoption in substance use disorder treatment programs: patterns of sustainability and discontinuation. J. Stud. Alcohol Drugs. 2011;72:669–677. doi: 10.15288/jsad.2011.72.669. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Abraham AJ, Knudsen HK, Rothrauff TC, Roman PM. The adoption of alcohol pharmacotherapies in the Clinical Trials Network: the influence of research network participation. J. Subst. Abuse Treat. 2010;38:275–283. doi: 10.1016/j.jsat.2010.01.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Addiction Technology Transfer Center Network. [Accessed on January 31,2016];National American Indian and Alaska Native Addiction Technology Transfer Center. 2014 http://attcnetwork.org/national-focus-areas/?rc=americanindian.
  7. Andrews CM, D'Aunno TA, Pollack HA, Friedmann PD. Adoption of evidence-based clinical innovations: The case of buprenorphine use by opioid treatment programs. Med. CareRes. Rev. 2014;71:43–60. doi: 10.1177/1077558713503188. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Beals J, Novins DK, Spicer P, Whitesell NR, Mitchell CM, Manson SM. Help seeking for substance use problems in two American Indian reservation populations. Psychiatr. Serv. 2006;57:512–520. doi: 10.1176/ps.2006.57.4.512. [DOI] [PubMed] [Google Scholar]
  9. Beals J, Novins DK, Whitesell NR, Spicer P, Mitchell CM, Manson SM. Prevalence of mental disorders and utilization of mental health services in two American Indian reservation populations: mental health disparities in a national context. Am. J. Psychiatry. 2005;162:1723–1732. doi: 10.1176/appi.ajp.162.9.1723. [DOI] [PubMed] [Google Scholar]
  10. Bride BE, Abraham AJ, Roman PM. Diffusion of contingency management and attitudes regarding its effectiveness and acceptability. Subst. Abuse. 2010;31:127–135. doi: 10.1080/08897077.2010.495310. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Burhansstipanov L, Christopher S, Schumacher SA. Lessons learned from community-based participatory research in indian country. Cancer Control. 2005;12:70–76. doi: 10.1177/1073274805012004s10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Center for Mental Health Services. The Comprehensive Community Mental Health Services for Children and Their Families Program Evaluation Findings: Annual Report to Congress 2005. Rockville, MD: 2005. [Google Scholar]
  13. Cook C, Heath F, Thompson RL. A meta-analysis of response rates in web- or internet-based surveys. Educ Psychol. Meas. 2000;60:821–836. [Google Scholar]
  14. Ducharme LJ, Knudsen HK, Abraham AJ, Roman PM. Counselor attitudes toward the use of motivational incentives in addiction treatment. Am. J. Addict. 2010;19:496–503. doi: 10.1111/j.1521-0391.2010.00081.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Ducharme LJ, Roman PM. Opioid treatment programs in the Clinical Trials Network: representativeness and buprenorphine adoption. J. Subst. Abuse Treat. 2009;37:90–94. doi: 10.1016/j.jsat.2008.09.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Fields D, Roman PM. Total quality management and performance in substance abuse treatment centers. Health Serv. Res. 2010;45:1630–1649. doi: 10.1111/j.1475-6773.2010.01152.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Friedmann PD, Taxman FS, Henderson CE. Evidence-based treatment practices for drug-involved adults in the criminal justice system. J. Subst. Abuse Treat. 2007;32:267–277. doi: 10.1016/j.jsat.2006.12.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Garner BR. Research on the diffusion of evidence-based treatments within substance abuse treatment: a systematic review. J. Subst. Abuse Treat. 2009;36:376–399. doi: 10.1016/j.jsat.2008.08.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Gone JP, Looking PE. American Indian culture as substance abuse treatment: pursuing evidence for a local intervention. J. Psychoactive Drugs. 2011;43:291–296. doi: 10.1080/02791072.2011.628915. [DOI] [PubMed] [Google Scholar]
  20. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82:581–629. doi: 10.1111/j.0887-378X.2004.00325.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Guerrero EG, He A, Kim A, Aarons GA. Organizational implementation of evidence-based substance abuse treatment in racial and ethnic minority communities. Adm. Policy Ment. Health. 2013;41:737–749. doi: 10.1007/s10488-013-0515-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Hosmer DW, Lemeshow S, Sturdivant RX. Applied Logistic Regression. Hoboken, N. J: Wiley; 2013. [Google Scholar]
  23. Institute of Medicine. Improving the Quality of Health Care for Mental and Substance-Use Conditions: Quality Chasm Series. Washington, D.C: National Academies Press; 2006. [PubMed] [Google Scholar]
  24. Knudsen HK, Abraham AJ, Oser CB. Barriers to the implementation of medication-assisted treatment for substance use disorders: the importance of funding policies and medical infrastructure. Eval. Program Plann. 2011a;34:375–381. doi: 10.1016/j.evalprogplan.2011.02.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Knudsen HK, Abraham AJ, Roman PM. Adoption and implementation of medications in addiction treatment programs. J. Addict. Med. 2011b;5:21–27. doi: 10.1097/ADM.0b013e3181d41ddb. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Krull I, Lundgren L, Zerden Lde S. Attitudes toward evidence-based pharmacological treatments among community-based addiction treatment programs targeting vulnerable patient groups. J. Addict. Dis. 2011;30:323–333. doi: 10.1080/10550887.2011.609808. [DOI] [PubMed] [Google Scholar]
  27. Legha R, Raleigh-Cohn A, Fickenscher A, Novins D. Challenges to providing quality substance abuse treatment services for American Indian and Alaska Native communities: perspectives of staff from 18 treatment centers. BMC Psychiatry. 2014;14:181. doi: 10.1186/1471-244X-14-181. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Legha RK, Novins D. The role of culture in substance abuse treatment programs for American Indian and Alaska Native communities. Psychiatr. Serv. 2012;63:686–692. doi: 10.1176/appi.ps.201100399. [DOI] [PubMed] [Google Scholar]
  29. Lundgren L, Amodeo M, Cohen A, Chassler D, Horowitz A. Modifications of evidence-based practices in community-based addiction treatment organizations: a qualitative research study. Addict. Behav. 2011a;36:630–635. doi: 10.1016/j.addbeh.2011.01.003. [DOI] [PubMed] [Google Scholar]
  30. Lundgren L, Chassler D, Amodeo M, D'Ippolito M, Sullivan L. Barriers to implementation of evidence-based addiction treatment: a national study. J. Subst. Abuse Treat. 2012;42:231–238. doi: 10.1016/j.jsat.2011.08.003. [DOI] [PubMed] [Google Scholar]
  31. Lundgren L, Krull I, Zerden Lde S, McCarty D. Community-based addiction treatment staff attitudes about the usefulness of evidence-based addiction treatment and CBO organizational linkages to research institutions. Eval. Program Plann. 2011b;34:356–365. doi: 10.1016/j.evalprogplan.2011.02.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Manuel JK, Hagedorn HJ, Finney JW. Implementing evidence-based psychosocial treatment in specialty substance use disorder care. Psychol. Addict. Behav. 2011;25:225–237. doi: 10.1037/a0022398. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Martino S. Strategies for training counselors in evidence-based treatments. Addict. Sci. Clin. Pract. 2010;5:30–39. [PMC free article] [PubMed] [Google Scholar]
  34. McCarty D, Fuller B, Kaskutas LA, Wendt WW, Nunes EV, Miller M, Forman R, Magruder KM, Arfken C, Copersino M, Floyd A, Sindelar J, Edmundson E. Treatment programs in the National Drug Abuse Treatment Clinical Trials Network. Drug Alcohol Depend. 2008;92:200–207. doi: 10.1016/j.drugalcdep.2007.08.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. McGovern MP, Fox TS, Xie H, Drake RE. A survey of clinical practices and readiness to adopt evidence-based practices: dissemination research in an addiction treatment system. J. Subst. Abuse Treat. 2004;26:305–312. doi: 10.1016/j.jsat.2004.03.003. [DOI] [PubMed] [Google Scholar]
  36. Moore LA, Aarons GA, Davis JH, Novins DK. How do providers serving American Indians and Alaska Natives with substance abuse problems define evidence-based treatment? Psychol. Serv. 2015;12:92–100. doi: 10.1037/ser0000022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Novins DK, Aarons GA, Conti SG, Dahlke D, Daw R, Fickenscher A, Fleming C, Love C, Masis K, Spicer P. Use of the evidence base in substance abuse treatment programs for American Indians and Alaska Natives: pursuing quality in the crucible of practice and policy. Implement. Sci. 2011;6:63. doi: 10.1186/1748-5908-6-63. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Novins DK, Moore LA, Beals J, Aarons GA, Rieckmann T, Kaufman CE. A framework for conducting a national study of substance abuse treatment programs serving American Indian and Alaska native communities. Am. J. Drug Alcohol Abuse. 2012;38:518–522. doi: 10.3109/00952990.2012.694529. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. O'Connell JM, Novins DK, Beals J, Spicer P, Team A-S. Disparities in patterns of alcohol use among reservation-based and geographically dispersed American Indian populations. Alcohol. Clin. Exp. Res. 2005;29:107–116. doi: 10.1097/01.alc.0000153789.59228.fc. [DOI] [PubMed] [Google Scholar]
  40. Oliva EM, Maisel NC, Gordon AJ, Harris AHS. Barriers to use of pharmacotherapy for addiction disorders and how to overcome them. Curr. Psychiatry Rep. 2011;13:374–381. doi: 10.1007/s11920-011-0222-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Olmstead TA, Abraham AJ, Martino S, Roman PM. Counselor training in several evidence-based psychosocial addiction treatments in private US substance abuse treatment centers. Drug Alcohol Depend. 2012;120:149–154. doi: 10.1016/j.drugalcdep.2011.07.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Oregon Health Authority. [Accessed on January 31, 2016];Additions and mental health services: Evidence-based practices. 2015 http://www.oregon.gov/oha/amh/Pages/ebp.aspx.
  43. Rieckmann TR, Kovas AE, Fussell HE, Stettler NM. Implementation of evidence-based practices for treatment of alcohol and drug disorders: the role of the state authority. J. Behav. Health Serv. Res. 2009;36:407–419. doi: 10.1007/s11414-008-9122-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Rieckmann TR, Kovas AE, McFarland BH, Abraham AJ. A multi-level analysis of counselor attitudes toward the use of buprenorphine in substance abuse treatment. J. Subst. Abuse Treat. 2011;41:374–385. doi: 10.1016/j.jsat.2011.05.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Riley KJ, Rieckmann T, McCarty D. Implementation of MET/CBT 5 for Adolescents. J. Behav. Health Serv. Res. 2008;35:304–314. doi: 10.1007/s11414-008-9111-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Roman PM, Abraham AJ, Knudsen HK. Using medication-assisted treatment for substance use disorders: evidence of barriers and facilitators of implementation. Addict. Behav. 2011;36:584–589. doi: 10.1016/j.addbeh.2011.01.032. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Savage SA, Abraham AJ, Knudsen HK, Rothrauff TC, Roman PM. Timing of buprenorphine adoption by privately funded substance abuse treatment programs: the role of institutional and resource-based interorganizational linkages. J. Subst. Abuse Treat. 2012;42:16–24. doi: 10.1016/j.jsat.2011.06.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Sokal RR, Rohlf FJ. Biometry: The Principles And Practice Of Statistics In Biological Research. San Francisco: W. H. Freeman; 1969. [Google Scholar]
  49. Substance Abuse and Mental Health Services Administration. Grants to Expand Substance Abuse Treatment Capacity in Adult and Family Drug Courts. Rockville, MD: U.S. Department of Health and Human Services; 2015a. [Google Scholar]
  50. Substance Abuse and Mental Health Services Administration. [Accessed on January 31, 2016];NREPP: SAMHSA's national registry of evicence-based practices and programs. 2015b http://www.nrepp.samhsa.gov/
  51. Van Horn PS, Green KE, Martinussen M. Survey response rates and survey administration in counseling and clinical psychology: a meta-analysis. Educ. Psychol. Meas. 2009;69:389–403. [Google Scholar]
  52. Walters ST, Matson SA, Baer JS, Ziedonis DM. Effectiveness of workshop training for psychosocial addiction treatments: a systematic review. J. Subst. Abuse Treat. 2005;29:283–293. doi: 10.1016/j.jsat.2005.08.006. [DOI] [PubMed] [Google Scholar]
  53. Weiss NA. Elementary Statistics. Reading, Mass: Addison Wesley; 1999. [Google Scholar]
  54. Whitesell NR, Beals J, Crow CB, Mitchell CM, Novins DK. Epidemiology and etiology of substance use among American Indians and Alaska Natives: risk, protection, and implications for prevention. Am. J. Drug Alcohol Abuse. 2012;38:376–382. doi: 10.3109/00952990.2012.694527. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS759381-supplement-2.docx (28.6KB, docx)

RESOURCES