Associations among state and local organizational contexts: Use of evidence-based practices in the criminal justice system

Abstract

This study used hierarchical linear modeling (HLM) to examine the extent to which the organizational characteristics of state corrections agencies and local criminal justice facilities interacted in their associations with the extent to which local facilities are using evidence-based substance abuse treatment practices (EBPs). The study used data collected from two nationally representative surveys – one of state executives and the other of local prison wardens, justice administrators, and treatment directors – which were conducted as part of the National Criminal Justice Treatment Practices survey [NCJTP; Taxman, F.S., Young, D., Wiersema, B., Mitchell, S., Rhodes, A.G., 2007. The National Criminal Justice Treatment Practices Survey: Multi-level survey methods and procedures. J. Subst. Abuse Treat. 32, 225–238], and includes both adult criminal and juvenile justice samples. Results indicated that several state organizational characteristics were either associated with more EBP use or interacted with local organizational characteristics in associations with EBP use, including: (1) systems integration at the state level was associated with greater EBP use; (2) state staffing adequacy and stability accentuated the association between local training and resources for new programs and EBP use (i.e., in states with better staffing, the relationship between training/resources and EBP use in local facilities was stronger); and (3) state executives’ attitudes regarding the missions and goals of corrections tended to diminish the extent to which corresponding local administrator attitudes were associated with EBP use. The study has implications for future research focused on EBP diffusion and implementation in correctional environments, particularly attempts to influence EBP use by working through state agencies.

1. Introduction

The evidence-based practice (EBP) movement in health services and correctional treatment research has gained significant momentum in recent years. First introduced in the field of medical science in the 1990s (Sackett et al., 1996), advocacy of EBPs has reached the point that some states and cities in the United States now make reimbursement to behavioral health care providers contingent on their use. We should also note that in recent years disseminating empirically supported treatments has become an international priority as well, fueled by interests in improving outcomes and the wide availability of systemic reviews (see the Cochrane Collaboration, www.cochranecollaboration.org). In a paper reviewing the state of evidence-based treatment Miller et al. (2005) observe that “The handwriting is on the wall: Those who are not providing empirically supported interventions are going to have a harder time getting paid for their services, ‘Anything goes’ is gone” (p. 267). The EBP movement is premised on an accumulation of evidence showing that specific treatments and practices lead to better outcomes than others (Carroll, 1998; Miller et al., 2003; Williams and Chang, 2000). Further, a growing body of research suggests these practices can be implemented with fidelity in standard drug abuse treatment settings (Ball et al., 2007; Henggeler et al., 1995; Liddle et al., 2006; Morgenstern et al., 2001; Peirce et al., 2006). This latter research, however, underscores the challenges and complexities of EBP implementation. It is evident from these and a number of related studies that transporting comprehensive, multifaceted treatments to community settings involves strategically affecting the complex interplay between the characteristics of interventions, service providers, and their organizational and service delivery settings (Backer, 2000; Henderson et al., 2006; Liddle et al., 2002; Schoenwald and Hoagwood, 2001).

1.1. Organizational factors related to adoption of evidence-based practices

Recent research has highlighted the importance that organizational characteristics play in the adoption, implementation, and sustainability of effective treatment practices (Hemmelgarn et al., 2006; Roman and Johnson, 2002; Simpson, 2002). There is a body of literature on the diffusion of innovations in organizations (Glisson, 2002; Wejnert, 2002) and implementation of EBPs in mental health settings (Drake et al., 2001; Roman and Johnson, 2002; Stirman et al., 2004), which has identified organizational characteristics conducive to adopting new (and presumed to be improved) treatment technologies. These factors include: (a) organizational structure (Backer et al., 1986; Knudsen et al., 2006; Roman and Johnson, 2002), (b) organizational climate (Aarons and Sawitzky, 2006; Glisson, 2002; Glisson and Hemmelgarn, 1998; Lehman et al., 2002); (c) training opportunities (Brown and Flynn, 2002; Knudsen et al., 2005); (d) resource adequacy (Lehman et al., 2002; Simpson, 2002; Stirman et al., 2004); (e) network connectedness (Knudsen and Roman, 2004), and (f) administrator and staff attitudes (Knudsen et al., 2005; Liddle et al., 2002; Schmidt and Taylor, 2002). Notably, none of these studies have examined the associations between broader contextual factors such as state substance abuse treatment policies and the extent to which substance abuse treatment agencies adopt EBPs (Chriqui et al., 2007, 2008). Further, these studies have almost entirely been limited to community-based treatment settings.

1.2. Organizational context and EBP adoption in corrections facilities

A series of studies analyzing data from the National Criminal Justice Treatment Practices (NCJTP) Survey (Taxman et al., 2007) has begun to demonstrate the associations between organizational context and EBP adoption in corrections settings. Findings from these studies have by and large been consistent with previous research conducted in community-based treatment settings (cf. Bartholomew et al., 2007; Knudsen and Roman, 2004; Roman and Johnson, 2002; Simpson et al., 2007). Friedmann et al. (2007) found that adult offender treatment programs that provided more EBPs had more extensive networking relationships established with various corrections and community agencies, performance-oriented cultures, more resources devoted to training, and were managed by administrators who viewed rehabilitation as a central goal of the criminal justice system. In a companion paper on treatment for juvenile offenders, Henderson et al. (2007) found that programs employing more EBPs also had more extensive networking connections, and received more support for new programs and training opportunities. The organizational culture of these agencies is defined by an emphasis on performance quality and by leaders that understand that public safety and health issues are intertwined. Consistent with findings from these studies, Oser et al. (2007) found that HIV testing (an EBP infrequently used in substance abuse treatment agencies, see Chriqui et al., 2008) more frequently occurred in facilities that had more resources devoted to training, more resources in general, and had more connections with judicial agencies.

1.3. State policy influence on local facility treatment practices

Although this review of recent research indicates that organizational context is indeed related to the extent to which adult criminal and juvenile justice agencies are adopting EBPs, these studies – as well as similar studies focusing on community mental health and substance abuse treatment agencies – do not consider the broader influence that state policies¹ may have on EBP use at local facilities. The few studies that do exist have examined the influence of state mental health and substance abuse treatment policies on the treatment practices taking place in community-based treatment agencies outside of the criminal justice system (Chriqui et al., 2007, 2008). However, a recent trend is for state legislatures to require treatment programs to use evidence-based treatments such as in the states of Oregon and Ohio, and we anticipate that this trend will likely continue in the United States as state governments attempt to improve treatment outcomes.

Investigating policy requirements governing state-authorized outpatient substance abuse treatment programs, Chriqui et al. (2007, 2008) found that state policies influence local substance abuse treatment practices. Chriqui et al. (2007) show that states that administer substance abuse treatment by counselor certification are more likely than states that administer substance abuse treatment by licensure of programs to offer continuing care services and group and family counseling. In terms of EBP use, Chriqui et al. (2008) demonstrate that programs located in states that require comprehensive assessment, family involvement in treatment, and continuing care (among other treatment practices) were significantly more likely to offer these services than states that did not require them. We anticipate that state influence on local treatment practices operate similarly in correctional environments.

The current study extends Chriqui and colleagues’ work by examining the extent to which state corrections organizations are associated with EBP adoption in state prisons and local jail facilities and probation and parole offices operating under state governance. In addition, this study provides a broader sampling of EBPs, as well as incorporating an interval-scaled measure of EBP use as our criterion variable. The study uses hierarchical linear modeling (HLM) to examine the extent to which four domains of organizational context variables (beliefs about the missions and goals of corrections, training and resources, culture and climate, and systems integration), measured at both state and local facility levels, are associated with EBP use in local facilities. HLM is well-suited to examine research questions dealing with how state policies affect local service provision, such as the results we report here. As we discuss more fully below, traditional analytic methods such as regression are inadequate or inappropriate for addressing such questions. Consistent with previous research using the same data source (Friedmann et al., 2007; Henderson et al., 2007, 2008), we hypothesized that prison and jail administrators’ beliefs regarding the missions and goals of corrections, organizational resources, and workplace culture and climate would each be associated with EBP use. Second, we hypothesized that the same domains measured at the state level would be associated with EBP use. Finally, we hypothesized that state-level organizational characteristics would influence the strength of the relationship between local facility characteristics and EBP use (i.e., cross-level interactions, Raudenbush and Bryk, 2002).

2. Methods

The National Criminal Justice Treatment Practices (NCJTP) survey is a multilevel survey designed to assess state and local adult and juvenile justice systems in the United States (US). The primary goals of the survey are to examine organizational factors that affect substance abuse treatment practices in correctional settings in the US as well as to describe available programs and services. The NCJTP survey solicited information from diverse sources ranging from executives of state criminal justice and substance abuse agencies to staff working in correctional facilities and drug treatment programs. Details of the study samples and survey methodology are provided in Taxman et al. (2007). The present study analyzes data concerning the organizational characteristics of state executive agencies (e.g., Departments of Corrections, Probation and/or Parole Departments) and corrections facilities (e.g., prisons, jails, and probation/parole facilities that can be state or locally governed) and the relationships between these organizational characteristics and the extent to which local facilities were using EBPs.

2.1. Sample and procedure

The survey obtained representative samples of adult prisons, juvenile residential facilities, and jails and community corrections agencies using a two-stage stratification scheme (first counties then facilities located within counties) utilizing region of the country and size of the facility or jurisdiction as stratification variables. We report sample sizes and response rates for three targeted populations: (1) a sample (Level 2) of state correctional executives in the adult criminal (n = 100, response rate = 74.6%) and juvenile justice systems (n = 70, response rate = 66.7%), (2) a sample of adult criminal (n = 289, response rate = 70.5%) and juvenile justice administrators (n = 141, response rate = 64.7%) who run local facilities and justice agencies, and (3) a sample of treatment directors² providing services for adult (n = 142, response rate = 61.1%) and juvenile offenders (n = 75, response rate = 56.9%; Level 1 consists of the combined administrator and treatment director samples). Fifty-three percent of the Level 1 administrators worked in probation and/or parole facilities, 29% prisons, and 19% local jails. The sample is nested at the local level in that administrators may be from the same jurisdiction, which served as our sampling unit. The response rates meet or exceed those typically found for mailed, self-administered organizational surveys (Baruch, 1999), and an analysis of response bias in the parent study indicated no systematic differences between responders and non-responders (Taxman et al., 2007). We restricted our Level 1 sample to respondents that had either a direct or indirect link to the state executives (n = 420 of a total sample of 647). Linkages were based on the organizational structure of correctional agencies and on the pool of respondents in the final sample from each state. A direct link refers to a relationship in which a state-level respondent has direct oversight over a local (i.e., Level 1) program. An indirect link refers to one of two relationships: (1) the local-level facility provides services contracted by a state-level agency, but falls under the direct oversight of a different state agency (e.g., Department of Drug and Alcohol Abuse), or (2) the local-level facility falls under the oversight of another division within the same agency. We excluded 132 respondents from local programs that did not have a direct or indirect link to a state-level respondent. The excluded programs consisted of those in which the director of the oversight agency did not provide a survey or were from county-run programs (e.g., jails), which did not fall under the oversight of a state-level agency. Based on these linkages, we used 93 of the 170 possible state executives; these executives represented 42 states. There were few differences between the participants included in the current study and those excluded from it; however, the Level 1 (local facility administrators and treatment directors) participants who were excluded reported being more satisfied with their facilities and having a lower regard for corrections-based substance abuse treatment. Please see Taxman et al. (2007) for more information on the multiple samples of survey participants and the procedure used for obtaining data from them.

2.2. Instrumentation

2.2.1. Use of evidence-based practices

Our measure of EBP use was an item response theory (IRT)-derived measure of the extent to which programs were using 15 specific practices supported either by meta-analyses (cf. Farrington and Welsh, 2005) or recommendations of consensus panels charged with developing recommendations on treatment practices with the best empirical and clinical support (Drug Strategies, 2005; National Institute on Drug Abuse, 2006). As we have reported elsewhere (Friedmann et al., 2007; Henderson et al., 2007, 2008), participants were using approximately one third of the EBPs (M = 5.33, SD = 2.28). The three most commonly occurring EBPs reported by the administrators were using an incentive system to improve program retention (reported by 66% of programs), comprehensive services designed to intervene in offenders’ medical and psychosocial needs (69%), and substance abuse treatment services delivered by staff with specialized training and credentials (68%). The least frequently occurring EBPs were use of a standardized risk assessment tool (23%), using an empirically supported treatment orientation (specifically cognitive behavioral therapy, therapeutic community, or motivational interviewing, 16%), and in the case of the juvenile facilities, using developmentally appropriate treatment (11%). Henderson et al. (2008) used Rasch modeling to derive a continuous, interval-scaled measure of EBP adoption weighting the use of specific practices by the frequency that programs were using them (essentially scaling the EBPs by their difficulty or frequency of occurrence). We used this Rasch-derived measure as our criterion variable in the current study. Other items included in the Rasch measure consist of (1) effective re-entry services designed to build upon initial treatment gains as well as integrated services provided by the justice and treatment systems; (2) use of sanctions to improve program retention; (3) interventions to engage the offender in treatment services and motivate him/her for change; (4) treatment of sufficient duration and intensity to produce change (typically defined as 90 days or longer, Simpson et al., 1999); (5) quality review designed to monitor treatment progress and outcomes; (6) family involvement in treatment; and (7) the use of standardized substance abuse screening tools (Brannigan et al., 2004; Knudsen and Roman, 2004; Landenberger and Lipsey, 2005; Mark et al., 2006; National Institute on Drug Abuse, 2006; Taxman, 1998). See Henderson et al. (2008) for more information on this measure and the advantages of using IRT to develop it.

2.2.2. Organizational characteristics

Organizational characteristics serving as predictor variables in the analysis included the 4 following domains, which previous studies using NCJTP data (e.g., Friedmann et al., 2007; Henderson et al., 2007, 2008) have suggested are important in understanding the relationship between organizational context and EBP adoption: (1) beliefs about the mission and goals of corrections, (2) training and resources, (3) workplace culture and climate, and (4) interagency integration within and between justice and health systems. Nearly all the measures were derived from existing, psychometrically sound measures (Taxman et al., 2007); the systems integration measure was constructed for this research based on multiple theoretical models described in Taxman and Bouffard (2000), Konrad (1996) and Fletcher et al. (in this volume) and was shown to meet conventional psychometric standards.

Respondents’ beliefs and attitudes regarding the mission and goals of corrections and the importance of substance abuse treatment were measured through subscales that assessed beliefs about responses to crime (rehabilitation, deterrence, punishment), as well as support for substance abuse treatment; these scales were adapted from previous similar surveys of public opinion and justice system stakeholders (Cullen et al., 2000). Organizational climate measures included subscales that assessed management emphasis on treatment quality and correctional staff support for treatment (Schneider et al., 1998), and organizational culture measures assessed the extent to which the cultures of the executive and local corrections agencies were characterized by cohesive, hierarchical, performance-oriented, and innovative cultures, as well as the extent to which the organization promoted new learning (Cameron and Quinn, 1999; Denison and Mishra, 1995; Orthner et al., 2004; Scott and Bruce, 1994). Please note that only the facility directors, and not the treatment directors, completed the organizational culture and climate measures. Training and resources measures were adapted from the Survey of Organizational Functioning for correctional institutions (Lehman et al., 2002). Subscales assessed respondents’ views about the adequacy of funding, the physical plant, staffing, resources for training and development, and internal support for new programming. Systems integration was assessed in two ways. First, we examined the extent to which the institution’s service delivery was driven by the activities in which executive agencies participated in establishing collaborative relationships and sharing activities with justice agencies, mental health programs, health clinics, housing services, vocational support agencies, and victim and faith-based organizations. Please see Fletcher et al. (in this volume) for more information on this aspect of systems integration and how it is measured. We developed a second conceptualization of systems integration from a social networking perspective, in which we examined the working relationships state executives had developed with other professionals (e.g., mental health, housing, employment, social services, etc.) that may provide services to offenders. For reasons described below, we included only state executives’ systems integration measures. See Table 1 for ranges on all measures used in the analyses.

Table 1.

Sample characteristics.

Variable	Level 1 (Facility)			Level 2 (State)
	%	M (SD)	Range	%	M (SD)	Range
Use of evidence-based practice		−0.33 (1.05)	−3.9 to 3.3		N/A	N/A
Goals of corrections
Rehabilitation		4.56 (0.49)	2–5		4.68 (0.47)	1–5
Deterrence		2.34 (0.71)	1–5		1.94 (0.83)	1–5
Importance of corrections-based substance abuse treatment		9.18 (1.96)	1–10		8.84 (1.71)	1–10
Importance of community-based substance abuse treatment		9.25 (2.49)	1–10		9.34 (1.27)	1–10
Organizational culture and climate
Management emphasis on quality treatment		3.89 (0.55)	1.3–5		3.46 (0.56)	1.3–4.9
Performance/Achievement		3.51 (0.67)	1.3–5		3.57 (0.66)	1.3–5
Innovation/Adaptability		3.39 (0.71)	1–5		3.41 (0.73)	1.3–5
Hierarchy		N/A	N/A		3.65 (0.60)	2–5
Cohesion		N/A	N/A		3.99 (0.67)	1.3–5
Training and resources
Training		3.63 (0.63)	1–5		3.44 (0.76)	1–4.8
Funding		2.40 (0.79)	1–4.7		2.15 (0.74)	1–4
Physical Plant		3.32 (0.89)	1–5		2.99 (0.84)	1–4.7
Resources		3.38 (0.78)	1–5		3.06 (0.80)	1–4.8
Staffing		2.82 (0.83)	1–5		2.48 (0.72)	1–5
Internal Support		3.58 (0.69)	0–5		N/A	N/A
Systems integration
Shared activities with substance abuse treatment programs		N/A	N/A		4.46 (3.27)	0–10
Shared Activities with probation and parole agencies		N/A	N/A		3.11 (3.36)	0–10
Shared activities with judiciary		N/A	N/A		2.21 (1.78)	0–10
Level of involvement with criminal justice agencies		N/A	N/A		3.06 (0.69)	1.5–5
Level of involvement with non-criminal justice agencies		N/A	N/A		3.04 (0.68)	1–4.7
Direct contact between executive and substance abuse treatment staff		N/A	N/A		12.67 (9.08)	0–36
Direct contact between executive and general staff		N/A	N/A		10.71 (7.95)	0–30
Direct contact between executive and health service staff/medical director		N/A	N/A		4.49 (3.87)	0–11
Direct contact between executive and substance abuse treatment director		N/A	N/A		5.34 (4.04)	0–11
Direct contact between executive and mental health services director		N/A	N/A		5.64 (3.83)	0–11
Level of contact between executive and health programs staff		N/A	N/A		3.52 (0.86)	1–5
Level of contact between executive and other programs staff		N/A	N/A		2.68 (0.76)	1–4.5
Percent adult corrections				59
Education or experience in human services				53
Graduate degree				65
Centralized agency				49

Note. Use of evidence based practice mean negative due to Rasch model-based scaling of the latent trait (Embretson & Reise, 2000). M = Mean, SD = Standard Deviation, N/A = Not Applicable.

2.2.3. Executive agency factors

The analyses incorporated several items that characterized the state executive agencies. We defined centralization of the corrections system on the basis of whether the state executive corrections agency was responsible for administering both prison and probation/parole services in the same organization (or not) (1 = Centralized, 0 = Decentralized). A second item indicated whether the agency served adult criminal or juvenile offenders (1 = Adult, 2 = Juvenile). A third item indicated whether the agency executive had education and/or previous experience in the human services field (0 = No, 1 = Yes), and a fourth item indicated whether the executive had a graduate degree (0 = No, 1 = Yes).

2.3. Data analysis

Study hypotheses were tested using hierarchical linear modeling (HLM; Raudenbush and Bryk, 2002). HLM was developed to address research questions involving multilevel data; in our example, data collected from local facility administrators and treatment directors (level 1) and data collected from state corrections administrators (e.g., prison directors, probation and parole directors, treatment directors, etc.; level 2). When analyzed as independent observations, hierarchically nested data violates critical assumptions on which multiple regression (and other analytic approaches based on the general linear model) rests, typically resulting in downwardly biased standard errors and inflated Type I error rates (Kreft and deLeeuw, 1998). HLM deals with this issue by simultaneously estimating relationships at both the facility (level 1) and state (level 2) level. The criterion variable in HLM models is specified at the facility level; however, regression models can be constructed with both local facility (e.g., resources) and state agency level (e.g., centralization of the corrections system) predictors. In addition, the relationship between a facility-level predictor (resources) and criterion (EBP use) may vary across states. HLM decomposes the relationship between predictor and criterion variables into a fixed portion (i.e., the relationship common to all groups) and a random portion (i.e., the slope estimates that vary across the level 2 state units). The goal of the HLM analysis is then to explain the random effects as a function of state-level predictors.

Our HLM models were specified as follows. Prior to testing any predictor–EBP use relationships at either the facility (level 1) or state (level 2) level, we tested a fully unconditional model including no covariates. This null model is a useful starting point for further analyses, as it provides a point estimate for the grand mean of the outcome variable (EBP use), as well as indicating whether sufficient variability exists among the level 2 units to justify using an HLM approach (Gillespie, 2005; Raudenbush and Bryk, 2002). Because this model resulted in significant variability at the state level (see below), we proceeded with a model consisting of level 1 predictor variables, but no predictors at level 2. This model examines whether facilities aggregated across states differed in their average levels of EBP use. Substantively, this model allows us to address the research question of whether facilities aggregated within state differ in their average levels of EBP use as a function of state-level predictors (e.g., systems integration). Consistent with our previous research on this topic (Friedmann et al., 2007; Henderson et al., 2007), we hypothesized that EBP use will be associated with organizational characteristics such as resources, administrator attitudes, and local organizational culture and climate. Our third model examined variability in regression slopes between level 1 predictor and EBP use as a function of the level 2 unit. This same model-testing procedure was repeated for each primary domain that we were interested in examining: (1) mission and goals of corrections, (2) resources, and (3) culture and climate. We also examined the impact that systems integration had on EBP use, but because level 1 systems integration factored into our construction of the EBP use criterion, we did not include any level 1 predictors in this analysis. In each HLM model we tested, we controlled for facility type (adult or juvenile) given the well-documented differences in the philosophy and goals of the two justice systems (Steinberg and Cauffman, 1999). We examined the HLM models separately for each domain so that we would not overfit the models (i.e., include too many predictors for a given sample size), as most statisticians recommend that the sample-size-to-predictor ratio should not be smaller than 10:1 (Tabachnick and Fidell, 2007). Recommendations for overall sample size indicate that sample sizes over 50 at Level 2 produce stable estimates and adequate power to test hypotheses using HLM (Clarke and Wheaton, 2007; Maas and Hox, 2005). As noted in Tables 2 and 3, the number of facilities included in the HLM analyses differed depending on whether either the facility administrator, treatment director, or both types of participants responded to the questions in a given domain. All of the models presented in this paper were estimated using HLM, Version 6.02 (Raudenbush et al., 2007), using restricted maximum likelihood (REML) estimation.

Table 2.

Results of random intercepts models for use of evidence-based practices.

	Coefficient	S.E.	t
Level 1 fixed effects
Goals of corrections (n = 257)
Rehabilitation	0.30	0.14	2.10*
Punishment	−0.36	0.11	−3.23**
Importance of corrections-based substance abuse treatment	0.06	0.03	1.87
Importance of community-based substance abuse treatment	0.02	0.03	0.62
Organizational culture and climate (n = 225)a
Management emphasis on quality treatment	0.37	0.16	2.35*
Performance/Achievement	0.08	0.10	0.77
Innovation/Adaptability	0.04	0.12	0.33
Training and resources (n = 412)
Training	0.30	0.07	4.06***
Funding	0.01	0.07	0.08
Physical plant	−0.08	0.17	−0.52
Resources	−0.03	0.19	−0.15
Staffing	−0.10	0.06	−1.59
Internal support	0.37	0.09	4.32***
Level 2 fixed effects
Goals of corrections (n = 73)
Rehabilitation	−0.01	0.10	−0.08
Deterrence	0.05	0.08	0.61
Importance of corrections-based SA Tx	<0.01	0.04	0.04
Importance of community-based SA Tx	0.18	0.05	3.70***
Organizational culture and climate (n = 62)a
Management emphasis on quality treatment	0.15	0.14	1.07
Cohesion	0.34	0.17	1.94*
Hierarchy	0.08	0.17	0.44
Performance/Achievement	−0.14	0.16	−0.92
Innovation/Adaptability	−0.22	0.16	−1.56
Resources (n = 91)
Training	0.22	0.06	3.47**
Funding	−0.16	0.09	−1.69
Physical Plant	<0.01	0.17	0.02
Resources	−0.18	0.18	−0.98
Staffing	0.02	0.12	0.17
Systems integration (n = 90)
Shared activities with Substance abuse treatment programs	−0.01	0.02	−0.61
Shared activities with probation and parole agencies	0.05	0.02	3.14**
Shared activities with judiciary	0.02	0.03	0.60
Level of involvement with criminal justice agencies	0.24	0.08	2.82**
Level of involvement with non-criminal justice agencies	0.02	0.12	0.17
Direct contact between executive and SA Tx staff	0.03	0.02	2.11*
Direct contact between executive and general staff	−0.03	0.02	−1.67
Direct contact between executive and health service staff/medical director	0.02	0.03	0.69
Direct Contact between executive and SA Tx director	0.01	0.02	0.27
Direct contact between executive and mental health services director	−0.01	0.03	−0.36
Level of contact between executive and health programs staff	0.07	0.08	0.80
Level of contact between executive and other programs staff	0.06	0.09	0.72

Note. SE: Standard Error, SA Tx: Substance Abuse Treatment. All models control for whether the respondent oversees adult or juvenile corrections organizations. Random effects are presented separately for each domain, Missions and Goals, Resources, Culture and Climate, and Systems Integration. Results of variance components not presented here to conserve space. They are available from the first author by request.

^aThe number of respondents is lower for the Culture and Climate domain because the treatment directors did not complete these measures.

^*p < .05.

^**p < .01.

^***p < .001.

Table 3.

Results of fixed effects for hierarchical linear models (random intercepts and slopes) for use of evidence-based practices.

Variable	Coefficient	SE	t
Goals of corrections (Level 1 n = 73; Level 2 n = 257)
Rehabilitation (Facility)	0.30	0.19	1.60
Rehabilitation (State)	0.13	0.40	0.31
Deterrence (State)	0.13	0.18	0.68
Importance of corrections-based SA Tx (State)	<0.01	0.10	0.07
Importance of community-based SA Tx (State)	0.01	0.17	0.11
Punishment (Facility)	−0.41	0.09	−4.50***
Rehabilitation (State)	0.26	0.20	1.31
Deterrence (State)	0.19	0.11	1.76
Importance of corrections-based SA Tx (State)	−0.13	0.06	−2.20*
Importance of community-based SA Tx (State)	<−0.01	0.07	−0.07
Importance of corrections-based substance abuse treatment (Facility)a	0.05	0.06	0.87
Rehabilitation (State)	−0.01	0.13	−0.04
Deterrence (State)	−0.05	0.04	−1.26
Importance of corrections-based SA Tx (State)	−0.02	0.02	−0.66
Importance of community-based SA Tx (State)	0.02	0.11	0.16
Importance of community-based substance abuse treatment (Facility)	0.01	0.08	0.12
Rehabilitation (State)	0.14	0.08	1.63
Deterrence (State)	0.02	0.04	0.61
Importance of corrections-based SA Tx (State)	0.03	0.03	0.94
Importance of community-based SA Tx (State)	−0.02	0.19	−0.16
Organizational culture and climate (Level 1 n = 62; Level 2 n = 225)b
Management emphasis on quality treatment (Facility)	0.49	0.19	2.64**
Management Emphasis on quality treatment (State)	0.62	0.34	1.81
Cohesion (State)	0.08	0.40	0.21
Hierarchy (State)	−0.50	0.34	−1.46
Performance/Achievement (State)	0.10	0.30	0.35
Innovation (State)	0.03	0.41	0.08
Performance/Achievement (Facility)	0.14	0.13	1.13
Management Emphasis on quality treatment (State)	−0.42	0.35	−1.22
Cohesion (State)	0.29	0.29	1.01
Hierarchy (State)	0.18	0.27	0.66
Performance/Achievement (State)	−0.12	0.25	−0.49
Innovation (State)	0.05	0.31	0.15
Innovation/Adaptability (Facility)	−0.02	0.08	−0.18
Management Emphasis on quality treatment (State)	−0.35	0.47	−0.75
Cohesion (State)	0.28	0.34	0.81
Hierarchy (State)	1.14	0.31	3.70***
Performance/Achievement (State)	−0.26	0.26	−1.00
Innovation (State)	−0.18	0.20	−0.89
Resources (Level 1 n = 91; Level 2 n = 412)
Training (Facility)a	0.22	0.08	2.92**
Training (State)	−0.04	0.12	−0.31
Funding (State)	−0.14	0.15	−0.92
Physical plant (State)	0.13	0.23	0.58
Resources (State)	−0.33	0.25	−1.28
Staffing (State)	0.27	0.14	1.97*
Funding (Facility)	−0.05	0.07	−0.74
Training (State)	−0.01	0.10	−0.14
Funding (State)	−0.12	0.08	−1.55
Physical plant (State)	0.06	0.24	0.27
Resources (State)	−0.02	0.25	−0.10
Staffing (State)	0.27	0.12	2.20*
Physical plant (Facility)	−0.24	0.18	−1.34
Training (State)	0.30	0.24	1.28
Funding (State)	−0.26	0.29	−0.89
Physical plant (State)	−0.39	0.78	−0.51
Resources (State)	0.18	0.75	0.25
Staffing (State)	0.55	0.29	1.90
Resources (Facility)	0.17	0.21	0.80
Training (State)	−0.25	0.28	−0.90
Funding (State)	0.38	0.31	1.22
Physical plant (State)	0.20	0.76	0.26
Resources (State)	0.05	0.74	0.07
Staffing (State)	−0.75	0.34	−2.16*
Staffing (Facility)a	−0.09	0.06	−1.44
Training (State)	0.10	0.09	1.04
Funding (State)	0.04	0.10	0.37
Physical plant (State)	−0.26	0.29	−0.89
Resources (State)	0.25	0.29	0.89
Staffing (State)	0.01	0.12	0.13
Internal support (Facility)	0.35	0.07	4.66***
Training (State)	<−0.01	0.11	−0.04
Funding (State)	−0.19	0.13	−1.42
Physical plant (State)	0.78	0.23	3.40***
Resources (State)	−0.54	0.26	−2.08*
Staffing (State)	<0.01	0.16	<0.01

Note. SE: Standard error, SA Tx: Substance Abuse Treatment. All models control for whether the respondent oversees adult or juvenile corrections organizations. Random effects were estimated in separate regression models for each domain: Missions and Goals, Culture and Climate, Resources.

^aVariance component fixed to zero due to its small magnitude when examined as a random effect.

^bThe number of respondents is lower for the Culture and Climate domain because the treatment directors did not complete these measures.

^*p < .05.

^**p < .01.

^***p < .001.

3. Results

3.1. Preliminary analyses

Descriptive statistics for the EBP measure and all facility- and state-level predictor variables are presented in Table 1. Results of the completely unconditional model indicated that the aggregated facilities within the states differed in the extent to which they were using EBPs (variance component = .091, χ² (88) = 116.37, p = .023), and that approximately 8.3% of the variation in EBP use occurred at the state agency level. These results suggest that although the majority of the variation in EBP use occurs at the level of the local facilities, state context (both policy and practice) is important in understanding the extent to which facilities are using EBPs.

3.2. Means-as-outcomes or random intercept models

Having established that we needed to proceed with model testing that incorporated state administrative units at level 2, we proceeded with examining whether state-level covariates were associated with differences in the average levels of EBP use at the local facilities, as well as whether the local facility organizational characteristics were associated with EBP use. Results of these analyses by domain are presented in Table 2. None of the state-level structural characteristics (centralization, adult correctional or juvenile justice department) or administrator background factors (education or experience in human services, graduate degree) predicted differences in the average levels of EBP use at the local corrections facilities. In terms of the administrator beliefs about the missions and goals of corrections, several level 1 and level 2 covariates predicted EBP use. At the level of the local facilities, wardens and treatment directors who placed higher value on corrections-based substance abuse treatment worked in facilities that were using marginally more EBPs (B = 0.06, t = 1.87, p = .063). Further, more emphasis on rehabilitation (B = 0.30, t = 2.10, p = .036) and less on punishment (B = −0.36, t = −3.23, p = .002) was also associated with more EBP use.

Perceptions of facility and state training and resources were also associated with more EBP use. Administrators who reported that the training is available and adequate was associated with more EBP use, both at level 1 (i.e., local administrators and treatment directors, B = 0.30, t = 4.06, p < .001) and level 2 (i.e., state administrators, B = 0.22, t = 3.47, p = .001). The same is true with perceptions of internal support for new programming at the level of the local facility (B = 0.37, t = 4.32, p ≤.001). More management emphasis on the quality of treatment in the local facilities was also associated with more EBP use (B = 0.37, t = 2.35, p = .020), but this relationship was not replicated at the state level. Finally, several indicators of state-level systems integration were associated with more EBP use, including shared activities between substance abuse treatment and probation and parole agencies (B = 0.05, t = 3.14, p = .003), involvement with other criminal justice agencies (prisons, jails, and community corrections) to provide substance abuse treatment in corrections facilities (B = 0.24, t = 2.82, p = .006), and more direct contact between the corrections agency administration and individuals within that agency who oversee substance abuse treatment in corrections environments (B = 0.03, t = 2.11, p = .038).

3.3. Cross-level interactions

Having established that both facility- and state-level organizational characteristics were associated with higher average levels of EBP use, we went on to examine whether state organizational characteristics moderated the relationship between facility-level predictors and EBP use. In other words, these analyses examined whether state-level organizational characteristics were associated with the strength of the relationship between local organizational characteristics and EBP use. Results of these HLM models are presented by domain in Table 3.

In terms of the mission and goals of corrections, when state executives placed a high level of importance on corrections-based substance abuse treatment, the relationship between local administrators’ punishment attitudes and EBP use was significant and negative (B = −0.13, t = −2.20, p = .029), suggesting that the influence of the executive agency regarding the missions and goals of corrections influenced local sentiments.

With respect to state and local training and resources, staffing and retention at the state level was related to several facility organizational characteristics–EBP use relationships, including training (B = 0.27, t = 1.97, p = .049), funding for new programs (B = 0.27, t = 2.20, p = .031) and marginally physical facilities (B = 0.55, t = 1.90, p = .058). These findings indicate that when the state agency directors perceive that their agencies were adequately staffed, and there was little turnover, the relationships between training opportunities, funding and physical facilities at local facilities were related to more EBP use. Regarding the relationship between local internal support for new programming and EBP use, this relationship was maximized in states that had better physical facilities (B = 0.78, t = 3.40, p = .001), and minimized in states that had more resources (B = −0.54, t = −2.08, p = .038).

4. Discussion

Findings from the current study suggest that both state and local organizational characteristics are associated with the extent to which EBPs are present in correctional agencies, supporting our first two hypotheses. However, our multilevel modeling analyses indicate that relationships between local organizational characteristics and EBP use must be considered in light of the perspectives of state corrections executives and the organizational characteristics of such agencies, supporting our third hypothesis. First, more extensive use of EBPs seems to be associated with several indices of systems integration. We found that facilities that reported more use of EBPs when the state corrections departments in which they are nested: (1) provided more opportunities for substance abuse treatment and probation and parole agencies to share activities, (2) had more networking relationships with other criminal justice agencies, and (3) had more contact between the state executive and substance abuse treatment staff. Given the variations in the delivery of corrections across the US, these findings highlight the importance of state policies and practices that focus on creating “seamless systems of care” (Taxman, 1998), which in this study are linked to more EBP adoption.

The complexity of the treatment delivery system for drug-involved offenders (both adults and youth) often complicates the processes of EBP adoption in these organizations. These findings suggest that this complexity may be decreased to some extent in correctional departments that foster legislation, regulations, and policy initiatives that focus on advancing the use of EBPs. Further, state policies and practices appear to be associated with local operations regardless of whether the local units are run by a state correctional or local government agency, or non-profit. Examples of policies in the US that have led to more EBP use include the initiatives by the New York Division of Probation and Correctional Alternatives which has provided support to advance the use of standardized risk and need tools in local probation agencies throughout New York (which are county-run); the Oklahoma Department of Corrections which has implemented similar initiatives in their state-run probation and community correctional units and provided funding for substance abuse treatment programs in their offices; and the Illinois Department of Corrections which has an in-prison treatment program that works with local parole offices and treatment providers to smooth the re-entry process and continue treatment in the community. Outside of the criminal justice system, Oregon and other states are making block grant funding contingent on treatment programs demonstrating that they are using EBPs. Similar initiatives are happening internationally as well (e.g., Canada, the United Kingdom, and other European countries that provide a centralized mechanism for identifying correctional programs that are suitable for use in the system). Such actions set the tone about the importance of integrating EBPs into treatment services for offenders, which the current study’s findings suggest are associated with more EBP use in operating units.

These findings are consistent with preliminary findings from Robert Wood Johnson’s Advancing Recovery Initiative, the goals of which are focused on promoting the use of EBPs through establishing partnerships between state government substance abuse authorities and substance abuse treatment providers (Schmidt et al., 2007). Using a case study approach, Schmidt et al. have found that one of the most successful innovators in this initiative was a state in which the substance abuse treatment authority was located in the same department as Medicaid, and thus the executives of these agencies were able to work directly together to solve problems such as cost coverage for services. In contrast, less successful innovators tended to have more complex and fragmented department structures. Likewise, states that had difficulty maintaining effective communication with substance abuse treatment providers were less successful in adopting EBPs. To our knowledge, the current study is the first to examine similar processes in the criminal justice system. Along with systems integration, more cohesive organizational cultures, training resources, and higher importance given to community-based substance abuse treatment (each measured at the level of the state executive) were related to more EBP use in local operating programs/facilities.

We also found that the relationships between local organizational characteristics and EBP use were moderated by the influence of state organizational characteristics. One of the more interesting patterns we found was the influence that state staffing and retention had on the relationship between resources and EBP use within local facilities. Among state executives that reported having adequate staff to meet their organizational responsibilities and greater stability, the relationship between local training resources and funding for new programming was more strongly related to EBP use. Along the same line, in states with better physical facilities, local facilities were able to maximize their internal support and adopt more EBPs. These findings suggest that state agencies that are stable and better resourced provide a foundation that may enable local facilities to invest funding, training resources, and internal support in EBP adoption. However, it is also possible that as local facilities establish effective, evidence-based substance abuse programs, the success of the local facilities may contribute to more organizational stability at the state level.

Another interesting set of findings from our cross-level interactions is the influence that executives’ perspectives on the missions and goals of corrections have on the relationship between local administrators’ views and EBP use. We found that executives’ perspectives regarding crime deterrence and rehabilitation can potentiate EBP use through associations with local administrators’ attitudes regarding the same goals. For example, in states that placed a high level of importance on corrections-based substance abuse treatment, these attitudes tended to override local administrators’ emphasis on crime deterrence. This finding suggests that executives’ perceptions of the goals of the criminal justice system may be more influential in shaping EBP use than local administrators’ perceptions, at least in facilities that tend to be more punishment oriented. However, given the cross-sectional nature of the study, it is not possible to determine whether the way in which state executives communicate their values and attitudes causally influences EBP adoption in local facilities.

4.1. Limitations

The current study is limited in certain respects. First, the study does not measure state policies regarding substance abuse treatment in the criminal justice system per se, instead focusing on correctional executive attitudes and organizational characteristics. Chriqui et al. (2008) have found that state substance abuse policies influence treatment practices at local programs; however, state substance abuse authorities constitute a different system, with different goals and operational parameters. While the trend is for more correctional executive agencies to explicitly require the use of EBPs, EBP use tends to be more prevalent in community based treatment programs. Perhaps this is due to competing value premises regarding service provision in the criminal justice system (see Henderson and Taxman, in this volume).

Second, although Taxman et al. (2007) found no evidence of non-response bias in the parent study, we selected state executive and local facility and treatment director respondents who had direct and indirect linkages to one another. Therefore, our selection of specific respondents that have direct or indirect linkages may have injected some bias into the results, and does not allow us to fully evaluate the reach of state policy directives. Third, this is a cross sectional survey and longitudinal data would provide a greater understanding of EBP use, as well as allow us to determine whether state policies precede local EBP implementation or vice versa. Qualitative data that depicts state narratives on the adoption and implementation processes associated with EBPs could also provide fruitful information. Finally, the data are limited to self-reports of state executives and local program administrators; therefore, we have no way of verifying the extent to which EBPs are actually used or examining the quality or fidelity with which they are used.

The NCJTP was designed to examine a range of organizational issues including the extent to which criminal justice facilities were using EBPs. Because it was designed with several goals, it is limited in terms of the detail with which we can study implementation processes. Case study approaches such as Schmidt et al. (2007) and Magnabosco (2006), along with nationally representative surveys such as the NCJTP, are needed to advance the field’s understanding of EBP adoption and implementation.

4.2. Conclusions

Despite these limitations, the current study also possesses noteworthy strengths. Foremost among these is the fact that the parent study obtained nationally representative estimates of substance abuse treatment practices in juvenile and adult correctional and community settings (Taxman et al., 2007). Second, to our knowledge, this is the first study focusing on the hierarchical relationships between corrections state executive and local facility organizational characteristics and their associations with EBP use. These findings have important implications for EBP diffusion and implementation. In terms of EBP diffusion (Rogers, 2003), the study findings suggest that EBP use in the criminal justice system may be more likely to occur in facilities located in states that have more integrated criminal justice-health services agencies and more stable and adequately staffed executive agencies with executives who place a high importance on corrections-based substance abuse treatment. In terms of researchers’ attempts to influence EBP adoption, it appears that an important first step in top-down implementation efforts (e.g., RWJ’s Advancing Recovery Initiative or NIC’s EBP initiative) focused on the criminal justice system is polling executives regarding their attitudes toward offender rehabilitation, crime deterrence, the importance of substance abuse treatment, etc., as well as targeting states that have adequate staffing and stability in key executive positions. At the same time, local enthusiasm for EBPs has influenced state policies, as has occurred in the States of Oregon and Ohio, as well as in Canada, the United Kingdom, and other European countries. The study findings may help guide future interventions aimed at modifying the organizational context of corrections departments and agencies so that EBPs may be implemented more effectively.