Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Apr 15.
Published in final edited form as: AIDS Educ Prev. 2014 Oct;26(5):411–428. doi: 10.1521/aeap.2014.26.5.411

THE EFFECT OF A LOCAL CHANGE TEAM INTERVENTION ON STAFF ATTITUDES TOWARDS HIV SERVICE DELIVERY IN CORRECTIONAL SETTINGS:A RANDOMIZED TRIAL

Christy A Visher 1, Matthew Hiller 1, Steven Belenko 1, Jennifer Pankow 1, Richard Dembo 1, Linda K Frisman 1, Frank S Pearson 1, Holly Swan 1, Tisha R A Wiley 1
PMCID: PMC4397895  NIHMSID: NIHMS677512  PMID: 25299806

Abstract

The National Criminal Justice Drug Abuse Treatment Studies research program conducted cluster randomized trials to test an organizational process improvement strategy for implementing evidence-based improvements in HIV services for preventing, detecting, and/or treating HIV for individuals under correctional supervision. Nine research centers conducted cluster randomized trials in which one correctional facility used a modified Network for Improvement of Addiction Treatment (NIATx) change team approach to implementing improved HIV services and the other facility used their own approach to implement the improved HIV services. This paper examines whether the intervention increased the perceived value of HIV services among staff of correctional and community HIV organizations. Baseline and follow-up measures of the perceived acceptability, feasibility, and organizational support for implementing HIV service improvements were collected from correctional, medical, and community HIV treatment staff. Results indicated that the perceived acceptability and feasibility of implementing HIV services improved among staff in the facilities using the modified NIATx change team approach as compared to staff in the comparison facilities.

United States prisons and jails house over 2.2 million men and women (Glaze & Parks, 2012). This vulnerable population is at high risk for mental health disorders, substance abuse, and physical health problems such as HIV/AIDS, hepatitis, and other infectious diseases. The Bureau of Justice Statistics reports that the prevalence of HIV and AIDS for incarcerated individuals in 2010 was 1.5% of the total prison population (Maruschak, 2012), which is nearly 30 times the population-level rate of infection reported by the Centers for Disease Control and Prevention (2012). For individuals with HIV/AIDS, early detection, linkage to treatment, and support services (particularly to promote adherence to antiretroviral [ARV] medications and uninterrupted care), are essential best practices for managing the disease and reducing the risk of transmission (Conviser & Pounds, 2002; Giordano et al., 2007; Girardi, Sabin, & Monforte, 2007; Janssen et al., 2001; Spaulding et al., 2002). Unfortunately, numerous HIV services gaps have been found in prisons and jails, as well as lack of adherence to best practices in HIV testing, prevention, and treatment access (Belenko, Hiller et al., 2013; Booker et al., 2013; Braithwaite & Arriola, 2003; Grinstead et al., 2003). Thus, it is important for correctional facilities to begin to close gaps in service delivery by initiating and maintaining programs that (1) raise awareness of HIV status for infected individuals through testing access, (2) enhance prevention through education programs, and (3) facilitate linkages to community- based HIV services.

There are significant individual and public health implications associated with gaps in the HIV/AIDS services treatment cascade (i.e., uninterrupted continuous engagement in HIV treatment; Dilernia et al., 2013; Gardner, McLees, Steiner, Del Rio, & Burman, 2011). Thus, there are substantial potential benefits associated with implementing improvements in services delivery within and between stakeholder agencies, especially continuous care during and beyond incarceration (Althoff et al., 2013; Booker at al., 2013; Gordon, Kinlock, McKenzie, Wilson, & Rich, 2013; Rapp et al., 2013). However, achieving more widespread adoption and implementation of HIV services by service systems, organizations, and clinicians requires coordinated action and support from stakeholders at multiple system and organizational levels—a process often strained by differing attitudes about HIV, conflicting missions, and funding challenges. Further, in the correctional system, where the primary mission is security and control, administrators and security staff may have antagonistic attitudes toward improving HIV services. Improving public health services such as HIV care within correctional systems is a complex undertaking that may require changes in organizational climate and staff attitudes, cross-systems training and services integration, and realigning missions (Chandler, Fletcher, & Volkow, 2009; Fletcher et al., 2009; Sabharwal et al., 2010; Taxman & Belenko, 2012; Taxman, Henderson, & Belenko, 2009).

To gain a better understanding of the processes that promote the integration of evidence-based substance abuse treatment and HIV services into practice, the National Institute on Drug Abuse (NIDA) initiated the second phase of the Criminal Justice Drug Abuse Treatment Studies (CJ-DATS 2) with a multisite cooperative of research centers (RCs) across the U.S. CJ-DATS protocols targeted three areas of service delivery to offenders with substance abuse problems: (1) assessment and case planning; (2) medication-assisted treatment options; and (3) HIV services (see Ducharme, Chandler, & Wiley, 2013). This article is concerned with the third protocol, HIV Services and Treatment Implementation in Corrections (HIV-STIC), which involved nine RCs, criminal justice partner agencies, and a coordinating center. The study utilized a multisite cluster randomized trial to test an organizational process improvement strategy designed to improve HIV services in correctional settings, with emphasis on increased detection of unidentified infections, enhanced education and prevention programs, and/or improved linkage to care for seropositive offenders under correctional supervision (Belenko, Visher et al., 2013).

The HIV-STIC protocol used a multi-level conceptual model to understand, measure, and evaluate implementation processes and implementation outcomes. This model represented the various stakeholders at individual, agency, and system levels. The model provided a framework for investigating the relationships between implementation strategies and implementation outcomes in real world settings, an especially relevant but complex undertaking in correctional environments. This conceptualization was informed by the work of several prominent implementation science researchers (Aarons, Hurlburt, & Horwitz, 2011; Damschroder et al., 2009; Fixsen, Blasé, Naoom, & Wallace, 2009; Proctor et al., 2009). According to the conceptual model of Proctor and colleagues (Proctor et al., 2009, 2011), effective implementation of evidence-based practices (EBPs) is characterized by the penetration of an intervention within an organization, acceptability of the improvement, uptake by multiple stakeholders, feasibility of its use, and sustainability over time within a service system setting. These areas, coupled with consideration of costs of implementing changes and fidelity to the implementation process, represent key areas that influence service outcomes, which in turn will impact client outcomes. Thus, successful implementation of innovative best practices is the outcome of implementing feasible, sustainable services that are found to be acceptable and achieve uptake and penetration within relevant stakeholders.

The HIV-STIC design, consistent with the Proctor model, views implementation outcomes as affected by characteristics of organizations and views some organizational characteristics as capable of being modified to successfully implement the use of EBPs. Thus in the current study, we focus on modifiable organizational characteristics, reflecting the implementation outcomes which we collectively labeled as value—acceptability, feasibility, and organizational support. Each of these three constructs links to HIV services, consistent with the definitions in the Proctor model. Acceptability encompasses the extent to which staff perceives value in the areas of improvement, specific to HIV services and related components in the continuum of care (i.e., testing, prevention, ARV medications, and linkage with continuing care after release from prison). Dimensions associated with acceptability include reasonableness, appropriateness, and perceived effectiveness. Feasibility addresses questions that concern the practicality of incorporating HIV improvements into organizational practices with regard to concerns about increasing burden (on staff and clients) and disruption to existing service delivery. Perceived organizational support targets organizational acceptance of planned improvements to HIV services and practices, and the sustainability of these improvements by way of organizational commitment at multiple staff levels to provide resources to support the adoption of changes into routine practice.

The primary objective of this investigation was to evaluate the impact of a local change team process improvement intervention, modified from the Network for Improvement of Addiction Treatment (NIATx) model (McCarty et al., 2007), on the perceived value of HIV services among correctional staff and community HIV service providers. Central to this line of inquiry is a need to recognize that organizational climate and culture play an important role in achieving successful (sustainable) implementation of best practices (Glisson, 2007). Organizational climate and culture can be deconstructed into organizational climate (e.g., stress and engagement) and social contexts (defined by attitudes, norms, values, expectations, and perceptions; Glisson, 2007) which are likely to impact the quality of HIV services delivery within and between agencies.

The theoretical and empirical literature on organizational climate and culture relative to HIV service delivery is limited. However, qualitative studies conducted by Robillard and colleagues (Robillard et al., 2003; Robillard, Gallito-Zaparaniuk, Braithwaite, Arriola, & Kennedy, 2009) support an association between staff attitudes and HIV services delivery. Robillard and colleagues investigated frontline staff perceptions toward providing HIV services to offenders in prison and on community supervision. Their findings indicate that barriers to HIV care in corrections include conflicting missions between healthcare and corrections, lack of knowledge about HIV, and the difficulties some healthcare employees have understanding the correctional culture (Arriola et al., 2002; Braithwaite, Hammett, & Arriola, 2002; Robillard et al., 2003). Healthcare staff with more personal experiences with HIV and prior experience working in corrections might be more successful in adapting to the culture in correctional settings (Robillard et al., 2009).

Before interventions for improving organizational functioning can be developed, more knowledge is needed about the opinions and experiences of corrections and community healthcare providers in the area of HIV service delivery for criminal justice populations and the role these areas play in HIV care. This paper examines whether a local change team intervention in which correctional, medical, and community HIV service provider staff worked together to address gaps in HIV services (Belenko, Visher et al., 2013), increased the perceived value of HIV services among staff of correctional and community HIV organizations. We operationally define value as perceived acceptability, feasibility, and organizational support for implementing HIV service improvements. We hypothesized that a change team approach to implementing changes in the HIV continuum of care would improve the perceived value of HIV services among staff of correctional, medical, and community HIV organizations. Specifically, we expected that organizations utilizing change teams compared to organizations that were not would rate HIV service improvements as significantly more acceptable, feasible, and supported by their organization.

METHODS

OVERVIEW

In this study, our implementation intervention was adapted from the NIATx approach (McCarty et al., 2007), which has been applied successfully in drug abuse treatment facilities (Hoffman, Ford, Choi, Gustafson, & McCarty, 2008). The modified NIATx approach (Experimental condition) was compared to a conventional staff training approach (Control condition) towards improving the delivery of a continuum of HIV services in correctional settings. The continuum of HIV services includes routine HIV testing, prevention/education programming, and procedures to link HIV-infected individuals to community-based treatment after confinement (Centers for Disease Control and Prevention, 2009).

In both the experimental and control conditions, correctional, medical, and community treatment staff members involved in HIV service delivery were recruited to receive a one-day training on the fundamentals of HIV infection, the HIV services continuum, and a brief guide to relevant web-based resources. Then, sites were cluster randomized, with matched pairs of correctional facilities that shared executive-level staff. For each matched pair, an executive-level staff person selected a primary goal that targeted HIV prevention, testing, or linkage to treatment for both the experimental and control facilities. In both facilities, the executive staff person then directed staff members to implement that goal. In the control condition facilities, staff participants used existing techniques for changing agency practices to implement improvement in a selected area of HIV services. In contrast, in the experimental condition a subset of staff participants were organized into a Local Change Team (LCT) under the guidance of a trained NIATx coach. These teams completed a series of rapid cycle process improvement activities designed to improve a selected part of the HIV services continuum in their facility. The process improvement approach tested in HIV-STIC is modeled after the NIATx approach, but differs in important respects. Notably, the goal of the HIV-STIC is to improve the delivery of HIV services, in contrast to the goals NIATx was designed to target on drug abuse treatment access and utilization. For example, some sites worked to improve the number of HIV infected individuals served by community-based HIV care after leaving prison or jail, others sought to improve their HIV testing practices, and others implemented an evidence-based HIV prevention curriculum. HIV-STIC also spans across organizations (correctional agencies, community health and drug abuse treatment agencies), and NIATx was modified for HIV-STIC to place greater emphasis on cross-agency collaboration and coordination. The intervention period was 10 months during which data were obtained using a battery of survey instruments administered to staff members as well as a collection of aggregate services delivery data (see Belenko, Visher et al., 2013, for additional details on the HIV-STIC study design and measures).

STUDY DESIGN AND SAMPLE

As shown in the CONSORT diagram (Figure 1), a multilevel sampling plan was used to guide the selection of research sites and the staff nested within each site. The HIV-STIC study included at least two matched prison or jail facilities (one experimental, one control) from each of the nine CJDATS2 research centers (RCs) located in nine states or U.S. territories. The paired facilities were located within the same state or county and matched on size and security level. Participating agencies included 28 prisons, jails, or work release centers as well as the community agencies that provided HIV services to the inmates while incarcerated and after their release. Eight of the nine research centers that participated in the protocol collected the required data to test the hypothesis that is the focus of this article.1

FIGURE 1.

FIGURE 1

Open in a new tab

Consort Diagram

Administrators and staff responsible for the oversight and delivery of HIV continuum components were purposively selected to complete the study measures. Staff included medical and correctional personnel such as physicians involved in the direct care of HIV infected individuals, medical and correctional staff involved in HIV testing, and medical and correctional staff involved in HIV prevention services. Although one facility from each pair was randomly assigned to experimental or control (i.e., training plus usual practice) groups, some facilities were co-located within the same correctional agency and some staff, correctional as well as treatment, thus may have divided their time between both an experimental and a control facility. On occasion, some staff members were fully transferred from an experimental facility to a control facility at some point during the study. Both circumstances resulted in some staff overlap across experimental and control facilities (n = 42). Too small to be analyzed as a third condition, these individuals were retained in their original experimental assignment. This was done because, through their contact with the staff in the experimental condition, these individuals were exposed to the intervention and the value that the staff in the experimental condition placed on delivery of HIV services.2 This yielded a final total of 225 staff in the experimental condition and 154 in the control condition who completed at least a baseline or a follow-up interview (see Figure 1). However, given staff attrition throughout the study (primarily due to leaving the agency), sample sizes vary. For the demographics, data were available for 300 respondents (but this varied due to missing data on specific variables). However, as detailed in the CONSORT diagram, staff attrition led to reduced analyzable samples for the HIV Staff Survey (source of the Barriers to Utilization Scale, [BRUS]) and the HIV Services Delivery Staff Survey (source of the Usage Rating Profile, [URP]).

This study was approved by each participating research center’s Institutional Review Board. Each research center used IRB approved informed consent procedures that: (1) explained the purpose of the protocol; (2) explained the benefits and any potential risks to participating individuals; and (3) explained that individuals had the right to refuse to participate, the right to withdraw from participation, and the right to refuse to answer any question without negative consequences. Research center representatives administered informed consent to staff and administrators. All survey responses were kept confidential.

As shown in Table 1, demographically, the majority (68%) of staff were women. Fifty-nine percent were Caucasian, 23% were African American, and 25% reported Hispanic ethnicity. The average age of respondents was 48 (SD = 10.7). Seventy percent of staff had at least a bachelor’s degree and about one-third (34%) had a postgraduate degree. Seventy-two percent of the sample was medical staff and 28% were correctional staff. On average, staff had worked in corrections for 10.6 years and for an average of 5.2 years in their current position. A comparison of staff characteristics within the experimental and control conditions showed that random assignment produced similar groups. There were no statistically significant differences in the proportion of the groups who were female and white. A significantly larger proportion of the experimental group was African American, χ2 (1) = 5.57, p = .018, but groups were similar on the proportion who reported Hispanic ethnicity. Age and educational achievement were similar across both experimental and control groups. Furthermore, no group differences were noted for the length of time spent working in the corrections field, or for the average length of time in current position.

TABLE 1.

Staff Characteristics by Study Condition

Study Condition
Staff Characteristic Control Experimental Total
% Female 66.7
(n = 117)		 69.5
(n = 177)		 68.4
(n = 294)
% White/Caucasian 61.5
(n = 117)		 57.1
(n = 170)		 58.9
(n = 287)
% African American* 16.2
(n = 117)		 28.2
(n = 170)		 23.3
(n = 287)
% Hispanic Ethnicity 21.7
(n = 120)		 18.9
(n = 180)		 20
(n = 300)
Mean age (SD) 49.0(10.1)
(n = 85)		 46.8 (11.2)
(n = 102)		 47.8 (10.7)
(n = 207)
% Baclielor’s degree or higher 66.7
(n = 120)		 72.5
(n = 153)		 70
(n = 273)
% Post-graduate degree 38.3
(n = 120)		 30.1
(n = 153)		 33.7
(n = 273)
% Medical 69.4
(n = 98)		 73.3
(n = 120)		 71.6
(n = 218)
% Corrections 30.6
(n = 98)		 26.7
(n = 120)		 28.4
(n = 218)
Mean length of time in years worked field of corrections (SD) 11.2 (9.7)
(n = 12S)		 10.2 (10.3)
(n = 155)		 10.6 (10.0)
(n = 280)
Mean length of time in years in current position (SD) 5.8 (5.8)
(n = 126)		 4.7 (5.6)
(n = 156)		 5.2 (5.7)
(n = 282)
Open in a new tab

Notes. Cell sizes differ in analyses because of the amount of data missing on each variable.

*

p < .05

DATA COLLECTION INSTRUMENTS AND PROCEDURE

Prior to the beginning of the study, executives and staff associated with each program completed the Brief Survey of Organizational Characteristics (BSOC) questionnaire and the HIV staff survey. The BSOC included items and scales designed to measure respondent demographics, staff needs, program needs, training needs, pressure for change, organizational resources and multiple dimensions of organizational readiness for change, leadership attributes, organizational climate, and interagency coordination (see Broome, Knight, Edwards, & Fkynn, 2009; Lehman, Greener, & Simpson, 2002). It also measures the organizational environment of participating agencies as well as the demographic characteristics (e.g., gender, job title) of the staff. The HIV Staff Survey included items from the Barriers to Research Utilization Scale and the Evidence-Based Practices Attitude Scale. Another questionnaire, the HIV Services Delivery Staff Survey, was collected at the end of the one-day training in the HIV services continuum and before randomization to study condition. This survey included the Usage Rating Profile, and the TCU Workshop Evaluation Form (Bartholomew, Joe, Rowan-Szal, & Simpson, 2007). The staff surveys were administered at baseline and at the end of the intervention. The mean time between the baseline and follow-up assessments was 403 days with a standard deviation of 80 days.

Because our literature review identified no established measures for assessing implementation outcomes related to the HIV services continuum, an extensive literature review, informed by Proctor and colleagues (2009), was completed to identify candidates that might be adapted for the study. Selected were the Barriers to Research Utilization Scale (BRUS; Brown, Wickline, Ecoff, & Glasser, 2008; Funk, Champagne, Wiese, & Tornquist, 1991) and the Usage Rating Profile (URP, Chafouleas, Briesch, Riley-Tillman, & McCoach, 2009). Adaptations to these questionnaires involved rewording them to ask about the HIV services continuum, which was defined for each respondent as the (1) use of HIV education or prevention programs to reduce or eliminate behaviors that increase risk for HIV infection; (2) HIV testing to see whether people have become infected with HIV; and (3) linking those that are HIV-infected to medical treatment in the community after their release.

The Barriers to Research Utilization Scale (BRUS)

This scale was originally developed to study implementation of and attitudes toward evidence-based practices with nurses in medical settings. Analyses reported by Funk and colleagues (1991) and confirmed by Brown and colleagues (2008) identified four factors, including support, resources, priority, and attitude. As adapted for the current study, the first factor for the BRUS, Support, measured organizational support for implementing a new practice within the HIV services continuum. The second factor, Resources reflected whether the respondents’ perceived implementation of the HIV services continuum as taking away resources from other services or activities offered in the organization. Priority, the third factor, indicated whether the HIV services continuum was seen as a priority. The fourth factor, Attitude, measured whether the individual and organizational attitude toward the HIV services continuum was positive or negative.

Usage Rating Profile (URP)

This questionnaire was originally developed to measure teachers’ perceptions of the acceptability of evidence-based interventions within their schools (Chafouleas et al., 2009). Adapted for the current study, it measured the level to which the respondent finds the implementation and use of the HIV service continuum as acceptable in their organization. Understanding indicates whether the respondent feels they understand the HIV services continuum and how it should be implemented. Whether the respondent thinks they will actually be able to implement the HIV services continuum is captured by Feasibility. Finally, Systems Support measures the level of support the respondents’ believe they have for implementing the HIV services continuum.

DEPENDENT VARIABLES

Because the experimental condition involves active participation in a change process related to HIV service delivery, it is expected that staff in this condition will show greater improvement in their perceptions and attitudes related to the value of the HIV services continuum as a whole as well as its specific components, compared to staff at facilities receiving only the baseline training. Value is defined as the acceptability and feasibility of implementing HIV service improvements as well as organizational support as perceived by staff members involved in the delivery of HIV services in the facilities. Thus, informed by the Proctor Implementation Model (Proctor et al., 2011) and the three specific hypotheses, subscales were selected to measure staff perceptions on three primary outcomes: (1) acceptability (URP), (2) feasibility (URP), and (3) organizational support (BRUS).

ANALYTIC PLAN

Because the selected scales were specifically adapted for use with medical and correctional staff in prisons and jails for the current study, confirmatory factor analyses (CFAs) were calculated to determine whether the previously established factor structures for these questionnaires were replicated in the current study (Raykov & Marcoulides, 2006), and four goodness-of-fit statistics were examined. Next, after checking distributional properties and adjusting for outliers, the following analyses were undertaken.

First, to explore the data, one-way ANOVAs were conducted comparing the study conditions on baseline scores on the dependent variables, and another series of ANOVAs to compare study condition on follow-up scores on the dependent variables. However, these exploratory ANOVAs could not take into consideration that there might have been a significant intercorrelation between the observations at baseline and at follow-up.

In the statistical tests of the three primary hypotheses, repeated-measures ANO-VAs were employed, with condition (i.e., Experimental or Control) as the between subjects factor, time (e.g., Baseline or Follow-up) as the within subjects factor and the three scales representing organizational support, acceptability, and feasibility as the dependent variables. However, these analyses did not take into consideration that staff were nested within facility and in these statistical tests only the study participants who had both baseline and follow-up measures (n = 106 for BRUS; n = 49 for URP) could be included (thus excluding many staff who had not completed the baseline survey).

Therefore, we explored a more refined analysis that would simultaneously use Multi-level Modeling (MLM) to address the nested nature of the data and Maximum Likelihood (ML) imputation to address the missing data problem.

After those main analyses, we used follow-up exploratory analyses to determine whether staff type (medical versus correctional) responded differently to the intervention, a final set of 3-way ANOVAs, with staff type (medical versus correctional) and study condition (experimental versus control) as the between-subjects factors and time (baseline, follow-up) as the within subjects factors were computed for the three dependent measures.

RESULTS

CONFIRMATORY FACTOR ANALYSES

Consistent with the previously reported established structures for the BRUS, and the URP, the CFAs replicated the same factors on this study’s sample. Appendix 1 presents the three subscales selected for this study and their corresponding factor loadings at baseline and at follow-up. These factor structures were found to be stable across the baseline and follow-up administrations of each of these scales. Most subscales had strong internal consistency; coefficient alphas remained consistently high across both baseline and follow-up administrations. From the BRUS, this included an alpha of .900 for Organizational Support at baseline and .909 at follow-up and from URP, Acceptability (α = .918; .946) and Feasibility (α = .894; .938).

ONE-WAY ANOVAS

As noted above, the first approach to assessing the impact of study condition (i.e., experimental versus control) was to assume the baseline and the follow-up samples were independent. This was done so that all cases with baseline data and all cases with follow-up data were used at each time point. Findings shown in Table 2 indicated that at baseline there were no statistically-significant differences between the experimental and control conditions on perceptions of acceptability or feasibility. The BRUS organizational support subscale at baseline, the control condition M = 3.97), and the experimental condition M = 3.93) also had statistically equivalent scores, F(1, 229) = 0.14; p = ns.

TABLE 2.

Means and Standard Deviations for One-Way ANOVAs Comparing Control and Experimental Groups at Baseline and at Follow-up on Ratings of Organizational Support, Acceptability, and Feasibility

Study Conditiona
Control Experimental
Outcome/Sourceb Baseline Follow-up Baseline Follow-up Effectc
BRUS-Organizational Support 3.97 (.68)
(n = 103)		 4.00 (.80)
(n = 67)		 3.93 (.73)
(n = 128)		 4.06 (.70)
(n = 106)		 B, p = .682
URP-Acceptability 3.98 (.52)
(n = 49)		 3.73 (.68)
(n = 66) 		4.01 (.61)
(n = 67)		 3.99 (.69)
(n = 102)		 B, p = .016
URP-Feasibility 3.34 (.43)
(n = 48)		 3.31 (.62)
(n = 64)		 3.41 (.63)
(n = 67)		 3.52 (.76)
(n = 101)		 B, p = .073
Open in a new tab

Notes. Standard deviations appear in parentheses.

a

Experimental group includes individuals who were in “both” conditions.

b

BRUS is the Barriers to Research Utilization Scale; URP is the Usage Rating Profile.

c

B indicates difference at posttest

At follow-up, as Table 2 shows, relative to staff in the control condition, staff in Experimental facilities had higher acceptability, URP-Acceptability, F(1, 166) = 5.93; p = .016; ratings and marginally higher feasibility ratings, URP-Feasibility, F(1, 163) = 3.27; p = .073. There were no significant differences between the Experimental and Control groups on ratings of organizational support, BRUS-Organizational Support, F(1, 171) = 0.17; p = 0.682.

REPEATED-MEASURES ANOVA FOR STUDY CONDITION

Because a fairly large number of respondents provided data on both the baseline and follow-up administrations of the scales, resulting in nonindependence among observations, a series of repeated measures ANOVAs were conducted using only those individuals who provided both baseline and follow-up measures for each subscale. As shown in Table 3, there is one significant interaction indicating that that the experimental group significantly increased its ratings on the URP Feasibility scale for implementing improved HIV services relative to the control condition, F(1, 45) = 4.55, p = .038.

TABLE 3.

Means and Standard Deviations by Study Condition Testing for Change in Organizational Support, Acceptability, and Feasibility

Study Condition
Control Experimental
Outcome/Sourcea Baseline Follow-up Baseline Follow-up Effectb
BRUS-Organizational Support 3.97 (.68)
(n = 46)		 3.97 (.69)
(n = 46)		 3.89 (.80)
(n = 60)		 4.10 (.66)
(n = 60)		 NS
URP-Acceptability 4.13 (.53)
(n = 22)		 3.78 (.75)
(n = 22)		 4.11 (.63)
(n = 27)		 4.12 (.72)
(n = 27)		 NS
URP-Feasibility 3.46 (.55)
(n = 20)		 3.29 (.74)
(n = 20)		 3.40 (.74)
(n = 27)		 3.79 (.76)
(n = 27)		 A × B, p = .038
Open in a new tab

Notes. Standard deviations appear in parentheses.

a

BRUS is the Barriers to Research Utilization Scale; URP is the Usage Rating Profile.

b

A is a main effect for group (experimental, control); B is a main effect for time (baseline, follow-up), A × B is group by time interaction; NS indicates effects were not statistically significant.

MULTILEVEL MODELING WITH MAXIMUM LIKELIHOOD ESTIMATION OF MISSING DATA

As noted earlier, since staff respondents were employed at sites which were randomly assigned to the experimental or control conditions, multilevel modeling (MLM) analyses were performed. Separate two-level, fixed-effects regression analyses were completed for each of the values variables using Mplus, version 7.11 (Muthen&Muthen, 1998–2012). Due to the small sample size, these analyses were considered exploratory in nature. Analyses indicated the design effects for each of the dependent variables were each below the level of 2.0. Statistical effects such as MLM are mainly applied to account for clustering when the design effects are 2.0 or greater. Thus, we determined that using the MLM was not appropriate for these analyses.

REPEATED MEASURES ANOVA OF STUDY CONDITION BY STAFF TYPE

A final series of analyses examined the relationship between study condition and staff type on changes in perceived value of the HIV services. One statistically significant three-way interaction emerged for staff perceptions of feasibility, F(1, 24) = 5.49, p = .043. It is apparent from Figure 2 that perceived feasibility did not change for the correctional staff in the control condition, but it decreased for correctional staff in the experimental condition. Also, for perceived feasibility, medical staff in the experimental condition showed an increase in perceptions of feasibility from baseline to follow-up, whereas medical staff in the control condition showed a decline.

FIGURE 2.

FIGURE 2

Open in a new tab

Changes in Perception of Feasibility by Study Condition (Control, Experimental) and Staff Type (Correctional, Medical)

SUPPLEMENTAL OUTCOMES

The process improvements experienced by the experimental facilities reflected the primary goal chosen by the executive staff which targeted HIV prevention, testing, or linkage to treatment. The change teams selected a variety of HIV services for improvement during the course of the intervention, including increasing HIV prevention attendance among female inmates; increasing the percentage of inmates receiving HIV education just prior to release; increasing the percentage of inmates receiving an HIV test at admission; increasing overall HIV testing; improving the linkage to community treatment for HIV positive inmates leaving prison; reducing wait times and no-shows to the community HIV treatment provider; improving continuity of anti-retroviral therapy medications for inmates leaving prison; and expanding peer-led HIV prevention programs.

Using a prospective meta-analytic design, analysis of these outcomes revealed an overall positive effect that was significant at the .05 level, with a point estimate corresponding to an odds ratio of 2.14 (see Pearson et al., 2014). The outcome measure was the odds ratio of successful delivery of an HIV service (prevention, testing, and/ or linkage to treatment) for offenders under correctional supervision (jail or prison). However, the effect sizes in the 14 experiments were quite heterogeneous. Most of the efficacious experimental interventions focused on improving HIV prevention.

DISCUSSION

In this cluster randomized trial of 14 matched pairs of correctional facilities, staff in the facilities that implemented a modified-NIATx change team approach for improving the delivery of the HIV service continuum increased their perceptions of the value of HIV services as compared to staff in the control facilities. Analyses using either all data or a subset of the data restricted to the paired baseline and follow-up sample revealed that two of the three primary outcome measures of value showed significant improvement. That is, at follow-up, staff in the facilities that participated in a change team (i.e., the experimental condition) rated implementing HIV services in their facility as more acceptable and feasible as compared with staff in the control facilities. These results provide preliminary support for the use of a local change team approach to implementing evidence-based practices in criminal justice settings. Moreover, measuring staff perceptions is important to studying the effectiveness of the identified implementation strategy, as suggested by Proctor and colleagues (2009, 2011).

Of particular interest in this exploratory study was the differential effect of the intervention on correctional and medical staff. The perceived feasibility of implementing improved HIV services declined for correctional staff in the experimental condition. This result could be due to additional work that was required among correctional staff to implement improved HIV services in their facility. Conversely, medical staff in the experimental condition reported increases in their perceptions of the feasibility of implementing improved HIV services, whereas medical staff in the control condition reported decreases in perceptions of feasibility. Medical staff, in the experimental condition, were likely influenced by the significant improvements being made in the delivery of HIV services whereas medical staff in the control condition were not involved in any changes in service delivery that would have affected their perceptions of feasibility. These findings indicate the importance of studying the impact of an organizational intervention on different types of staff who may be affected by changes designed to improve HIV service delivery. Future qualitative work is planned to explore these differences in more depth.

This study is one of a very few efforts studying the implementation strategies that might be utilized to improve the application of evidence-based health practices in criminal justice settings. There are anecdotal reports of the use of change team strategies in criminal justice settings such as the modified NIATx process improvement approach tested in this study, but these approaches have not been well-documented in published research (but see Johnson, Belenko, & Rieckmann, 2013; Robillard et al., 2009; Wexler, Zehner, & Melnick, 2012). This study found that increased staff support of improvements in the delivery of HIV services is possible after implementing an organizational process improvement strategy. According to the Proctor and colleague’s (2009) model of implementation research, and other conceptual models of implementation (e.g., Aarons et al., 2011), such improved staff support and perceived feasibility would be expected to lead to improved client outcomes. As mentioned earlier, a separate analysis of improvements in service delivery from this study (Pearson et al., in press) found a significant increase in HIV services (prevention, testing, and/or linkage) in the experimental research sites. This improvement in service delivery may be related to the modest changes in staff attitudes.

Several limitations to this research should be noted. As with many implementation studies conducted in real world settings, there was substantial attrition of staff subjects, due to reassignment, retirement, or other reasons. Some staff also entered the study after baseline. In addition, staff respondents were not randomly selected from among all staff working on HIV services or related services at the facilities. Staff on the change teams volunteered or were appointed by their supervisors to participate. Other staff who completed the baseline and follow-up surveys were recruited from all staff, depending on their availability the day of the survey administration. Thus the findings cannot be generalized to all staff working at these facilities or participating agencies. However, as an implementation research study the goal was to change the attitudes and perceptions of staff who either participated in or were affected by the change team project, and the results indicate that the intervention had significant impacts on staff attitudes.

Future research should focus on (a) other evidence-based practices and (b) other implementation strategies to expand our knowledge of implementing health and behavior interventions in criminal justice settings, including in probation offices which have contact with millions of offenders each year. Such research could profoundly affect the health and other individual behavioral outcomes for individuals in contact with the criminal justice system.

APPENDIX 1.

Confirmatory Factor Analyses for Dependent Variables

Factor Loadingsa
Scale/Subscale/Item Baseline Follow-up
Barriers to Research Utilization Scale (BRUS)
Organizational Support (α = .90; .91)2,3
The correctional administration supports implementation of the HIV services continuum. 0.75 0.80
The medical provider administration supports implementation of the HIV services continuum. 0.73 0.69
The treatment contractor administration supports implementation of the HIV services continuum. 0.68 0.53
Local community based organizations' administration supports implementation of the HIV
services continuum.		 0.61 0.60
Correctional staff is supportive of the implementation of the HIV services continuum. 0.55 0.76
Medical staff is supportive of the implementation of the HIV services continuum. 0.71 0.82
Treatment staff is supportive of the implementation of the HIV services continuum. 0.73 0.74
Community organization staff is supportive of the implementation of the HIV services
continuum.		 0.54 0.63
Usage Rating Profile (URP)
Acceptability (α = .92; .95) b
I like the procedures used in the modified HIV services for people under criminal justice
supervision.		 0.52 0.68
The modified HIV services for people under criminal justice supervision are a good way to address people under criminal justice supervision’s
HIV issues.		 0.64 0.66
I am not interested in implementing the modified HIV services for people under criminal justice
supervision. (R)		 0.60 0.49
I am resistant to use the modified HIV services for people under criminal
justice supervision.		 0.48 0.50
This is an acceptable intervention strategy for dealing with HIV issues for people under criminal
justice supervision.		 0.58 0.64
I have positive attitudes about implementing the modified HIV services for people under criminal
justice supervision.		 0.56 0.67
Overall, the modified HIV services for people under criminal justice supervision are beneficial to
the people under criminal justice supervision.		 0.61 0.79
I am motivated to try the modified HIV services for people under criminal justice supervision. 0.52 0.61
The modified HIV services for people under criminal justice supervision are reasonable for their
HIV issues.		 0.56 0.80
The modified HIV services for people under criminal justice supervision are an effective choice for dealing with a lot of HIV issues. 0.59 0.83
I will implement the modified HIV services for people under criminal justice supervision with a good deal of enthusiasm. 0.57 0.63
The modified HIV services for people under criminal justice supervision are a fair way to handle their HIV issues. 0.55 0.74
Use of the modified HIV services for people under criminal justice supervision will save time dealing with their HIV issues. 0.39 0.60
Feasibility (α = .89; .94) b
The amount of time for record keeping associated with the modified HIV services for people
under criminal justice supervision is reasonable.		 0.49 0.65
The modified HIV services for people under criminal justice supervision could be implemented for
the duration of time as prescribed.		 0.65 0.70
The modified HIV services for people under criminal justice supervision could be implemented
with the intensity as prescribed.		 0.61 0.74
The amount of time to use the modified HIV services for people under criminal justice supervision
is reasonable.		 0.60 0.76
The modified HIV services for people under criminal justice supervision could be implemented as
frequently as described.		 0.53 0.81
The modified HIV services for people under criminal justice supervision will not be disruptive to
other offenders.		 0.45 0.63
All pieces of the modified HIV services for people under criminal justice supervision can be
implemented precisely.		 0.39 0.59
The modified HIV services for people under criminal justice supervision can be implemented
exactly as described.		 0.37 0.59
Open in a new tab

(R) following an item indicates it was reverse coded before it was included in its corresponding subscale score.

a

Factor loadings are the STD standardized coefficients from the Mplus output. Results from the baseline data are presented first, followed by results from the follow-up data.

b

Alphas are reported for baseline and follow-up, respectively.

Acknowledgments

This study is funded under a cooperative agreement from the U.S. Department of Health and Human Services, National Institutes of Health, National Institute on Drug Abuse (NIDA), with support from the Substance Abuse and Mental Health Services Administration and the Bureau of Justice Assistance, U.S. Department of Justice. The authors gratefully acknowledge the collaborative contributions by NIDA; the Coordinating Center, AMAR International, Inc.; and the Research Centers participating in the National Criminal Justice Drug Abuse Treatment Studies (CJ-DATS). The Research Centers include: Arizona State University and Maricopa County Adult Probation (U01DA025307); University of Connecticut and the Connecticut Department of Correction (U01DA016194); University of Delaware and the Delaware Department of Correction (U01DA016230); Friends Research Institute and the Maryland Department of Public Safety Correctional Services’ Division of Parole and Probation (U01DA025233); University of Kentucky and the Kentucky Department of Corrections (U01DA016205); National Development and Research Institutes, Inc. and the Colorado Department of Corrections (U01DA016200); University of Rhode Island, Rhode Island Hospital and the Rhode Island Department of Corrections (U01DA016191); Texas Christian University and the Illinois Department of Corrections (U01DA016190); Temple University and the Pennsylvania Department of Corrections (U01DA025284); and the University of California at Los Angeles and the Washington State Department of Corrections (U01DA016211). The views and opinions expressed in this article are those of the authors and should not be construed to represent the views of NIDA or any of the sponsoring organizations, agencies, CJ-DATS partner sites, or the U.S. government.

Footnotes

1

One of the nine research centers was unable to collect follow-up data because of unforeseen administrative constraints at the research sites.

2

Ongoing study monitoring identified those who were in both the experimental and control conditions. These individuals were asked by study staff to take care not to introduce anything from the experimental condition to the control condition, but ultimately, whether they did so, which also would have introduced treatment crossover effects, was not assessed.

REFERENCES

  1. Aarons G, Hurlburt M, Horwitz S. Advancing a conceptual model of evidence based practice implementation in public service sectors. Administration and Policy in Mental Health. 2011;38:4–23. doi: 10.1007/s10488-010-0327-7. doi:10.1007/ s10488-010-0327-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Althoff A, Zelenev A, Meyer J, Fu J, Brown S, Vagenas P, Altice F. Correlates of retention in HIV care after release from jail: Results from a multi-site study. AIDS and Behavior. 2013;17:S156–S170. doi: 10.1007/s10461-012-0372-1. doi:10.1007/s10461-012-0372-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Arriola KRJ, Kennedy SS, Coltharp JC, Braithwaite RL, Hammett TM, Tinsley MJ. Development and implementation of the cross-site evaluation of the CDC/HRSA corrections demonstration project. AIDS Education and Prevention. 2002;14:107–118. doi: 10.1521/aeap.14.4.107.23883. [DOI] [PubMed] [Google Scholar]
  4. Bartholomew NG, Joe GW, Rowan-Szal GA, Simpson DD. Counselor assessments of training and adoption barriers. Journal of Substance Abuse Treatment. 2007;33:193–199. doi: 10.1016/j.jsat.2007.01.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Belenko S, Hiller M, Visher C, Copenhaver M, O’Connell D, Burdon W, et al. Policies and practices in the delivery of HIV services in correctional agencies and facilities: Results from a multisite survey. Journal of Correctional Health Care. 2013;19:293–310. doi: 10.1177/1078345813499313. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Belenko S, Visher C, Copenhaver M, Hiller M, Melnick G, O’Connell D, et al. A cluster randomized trial of utilizing a local change team approach to improve the delivery of HIV services in correctional settings: Study protocol. Health & Justice. 2013;1:8. doi: 10.1186/2194-7899-1-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Booker C, Flygare C, Solomon L, Ball S, Pustell M, Bazerman L, Spaulding A. Linkage to HIV care for jail detainees: Findings from detention to the first 30 days after release. AIDS and Behavior. 2013;17:S128–S136. doi: 10.1007/s10461-012-0354-3. doi:10.1007/s10461-012-0354-3. [DOI] [PubMed] [Google Scholar]
  8. Braithwaite R, Arriola K. Male prisoners and HIV prevention: A call for action ignored. American Journal of Public Health. 2003;93:759–763. doi: 10.2105/ajph.93.5.759. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Braithwaite RL, Hammett T, Arriola KRJ. Introduction to the special issue: HIV/AIDS in correctional settings. AIDS Education and Prevention. 2002;14:1–6. [Google Scholar]
  10. Broome KM, Knight DK, Edwards JR, Flynn PM. Leadership, burnout, and job satisfaction in outpatient drug-free treatment programs. Journal of Substance Abuse Treatment. 2009;37:160–170. doi: 10.1016/j.jsat.2008.12.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Brown CE, Wickline MA, Ecoff L, Glaser D. Nursing practice, knowledge, attitudes, and perceived barriers to evidence based practice at an academic medical center. Journal of Advanced Nursing. 2008;65:371–381. doi: 10.1111/j.1365-2648.2008.04878.x. [DOI] [PubMed] [Google Scholar]
  12. Centers for Disease Control and Prevention . HIV testing implementation guidance for correctional settings. Author; Atlanta, GA: 2009. [Google Scholar]
  13. Centers for Disease Control and Prevention . HIV at a glance. Author; Atlanta, GA: 2012. [Google Scholar]
  14. Chafouleas SM, Briesch AM, Riley-Tillman TC, McCoach DB. Moving beyond assessment of treatment acceptability: An examination of the factor structure of the Usage Rating Profile—Intervention (URP-I) School Psychology Quarterly. 2009;24:36–47. [Google Scholar]
  15. Chandler R, Fletcher B, Volkow N. Treating drug abuse and addiction in the criminal justice system: Improving public health and safety. Journal of the American Medical Association. 2009;301:183–190. doi: 10.1001/jama.2008.976. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Conviser R, Pounds M. The role of ancillary services in client-centered systems of care. AIDS Care. 2002;14:S119–S131. doi: 10.1080/09540120220150018. [DOI] [PubMed] [Google Scholar]
  17. Damschroder L, Aron D, Keith R, Kirsh S, Alexander J, Lowery J. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science: IS. 2009;4:50. doi: 10.1186/1748-5908-4-50. doi:10.1186/1748-5908-4-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Dilernia D, Monaco D, Cesar C, Krolewiecki A, Friedman S, Cahn P, Salomon H. Estimation of HIV-testing rates to maximize early diagnosis-derived benefits at the individual and population level. PLoS One. 2013;8:e53193. doi: 10.1371/journal.pone.0053193. doi:10.1371/journal. pone.0053193. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Ducharme LJ, Chandler RK, Wiley TR. Implementing drug abuse treatment services in criminal justice settings: Introduction to the CJ-DATS study protocol series. Health & Justice. 2013;1:8. doi: 10.1186/2194-7899-1-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Fixsen DL, Blasé KA, Naoom SF, Wallace F. Core implementation components. Research on Social Work Practice. 2009;19:531–540. doi:10.1177/1049731509335549. [Google Scholar]
  21. Fletcher B, Lehman W, Wexler H, Melnick G, Taxman F, Young D. Measuring collaboration and integration activities in criminal justice and drug abuse treatment agencies. Drug and Alcohol Dependence. 2009;103:S54–S64. doi: 10.1016/j.drugalcdep.2009.01.001. [DOI] [PubMed] [Google Scholar]
  22. Funk SG, Champagne MT, Wiese RA, Tornquist EM. BARRIERS: The barriers to research utilization scale. Applied Nursing Research. 1991;4:39–45. doi: 10.1016/s0897-1897(05)80052-7. [DOI] [PubMed] [Google Scholar]
  23. Gardner E, McLees M, Steiner J, Del Rio C, Burman W. The spectrum of engagement in HIV care and its relevance to test-and-treat strategies for prevention of HIV infection. Clinical Infectious Diseases. 2011;52:793–800. doi: 10.1093/cid/ciq243. doi:10.1093/cid/ciq243. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Giordano TP, Gifford AL, White AC, Jr., Suarez-Almazor ME, Rabeneck L, Hartman C, et al. Retention in care: A challenge to survival with HIV infection. Clinical Infectious Diseases. 2007;44:1493–1499. doi: 10.1086/516778. doi:10.1086/516778. [DOI] [PubMed] [Google Scholar]
  25. Girardi E, Sabin C, Monforte A. Late diagnosis of HIV infection: Epidemiological features, consequences and strategies to encourage earlier testing. Journal of Acquired Immune Deficiency Syndromes. 2007;1999;46:S3–S8. doi: 10.1097/01.qai.0000286597.57066.2b. [DOI] [PubMed] [Google Scholar]
  26. Glaze L, Parks E. Correctional populations in the United States, 2011. U.S. Department of Justice, Bureau of Justice Statistics; Washington, DC: 2012. NCJ239972. [Google Scholar]
  27. Glisson C. Assessing and changing organizational culture and climate for effective services. Research on Social Work Practices. 2007;17:736–747. doi:10.1177/1049731507301659. [Google Scholar]
  28. Gordon M, Kinlock T, McKenzie M, Wilson M, Rich J. Rapid HIV testing for individuals on probation/parole: Outcomes of an intervention trial. AIDS and Behavior. 2013;17:2022–2030. doi: 10.1007/s10461-013-0456-6. doi:10.1007/ s10461-013-0456-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Grinstead O, Seal D, Wolitski R, Flanigan T, Fitzgerald C, Nealey-Moore J, Askew J. HIV and STD testing in prisons: Perspectives of in-prison service providers. AIDS Education and Prevention. 2003;15:547–560. doi: 10.1521/aeap.15.7.547.24045. [DOI] [PubMed] [Google Scholar]
  30. Hoffman KA, Ford JH, II, Choi D, Gustafson DH, McCarty D. Replication and sustainability of improved access and retention within the network for the improvement of addiction treatment. Drug and Alcohol Dependence. 2008;98:63–−69. doi: 10.1016/j.drugalcdep.2008.04.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Janssen R, Holtgrave D, Valdiserri R, Shepherd M, Gayle H, De Cock K. The serostatus approach to fighting the HIV epidemic: Prevention strategies for infected individuals. American Journal of Public Health. 2001;91:1019–1024. doi: 10.2105/ajph.91.7.1019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Johnson I, Belenko S, Rieckmann T. Strategies for improving the integration of drug treatment into probation supervision: Results from a pilot study. Paper presented at the American Society of Criminology Annual Conference; Atlanta GA: Nov, 2013. [Google Scholar]
  33. Lehman WEK, Greener JM, Simpson DD. Assessing organizational readiness for change. Journal of Substance Abuse Treatment. 2002;22:197–209. doi: 10.1016/s0740-5472(02)00233-7. [DOI] [PubMed] [Google Scholar]
  34. Maruschak L. HIV in prisons, 2001–2010. U.S. Department of Justice, Bureau of Justice Statistics; Washington, DC: 2012. (NCJ 228307) [Google Scholar]
  35. McCarty D, Gustafson DH, Wisdom JP, Ford J, Choi D, Molfenter T, Cotter F. The network for the improvement of addiction treatment (NIATx): Enhancing access and retention. Drug and Alcohol Dependence. 2007;88:138–145. doi: 10.1016/j.drugalcdep.2006.10.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Muthén LK, Muthén BO. Mplus user’s guide, version 7. Muthén & Muthén; Los Angeles, CA: 1998–2012. [Google Scholar]
  37. Pearson F, Shafer MS, Dembo R, Vega-Debién G, Pankow J, Duvall JL, et al. Efficacy of a process improvement intervention on delivery of HIV services: A multi-site trial. American Journal of Public Health. 2014;204:11. doi: 10.2105/AJPH.2014.302035. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health. 2009;36:24–34. doi: 10.1007/s10488-008-0197-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Proctor EK, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Hensely M. Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health. 2011;38:65–76. doi: 10.1007/s10488-010-0319-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Rapp R, Ciomcia R, Zaller N, Draine J, Ferguson A, Cagey R. The role of jails in engaging PLWHA in care: From jail to community. AIDS and Behavior. 2013;17:89–99. doi: 10.1007/s10461-012-0298-7. doi:10.1007/s10461-012-0298-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Raykov T, Marcoulides GA. A first course in structural equation modeling (2nd ed . Mahwah, NJ: Lawrence Erlbaum. 2006 [Google Scholar]
  42. Robillard A, Gallito-Zaparaniuk P, Arriola K, Kennedy S, Hammett T, Braithwaite R. Partners and processes in HIV services for inmates and ex-offenders. Facilitating collaboration and service delivery. Evaluation Review. 2003;27:535–562. doi: 10.1177/0193841X03255631. [DOI] [PubMed] [Google Scholar]
  43. Robillard A, Gallito-Zaparaniuk P, Braithwaite R, Arriola K, Kennedy S. Providing HIV services for incarcerated and ex-offender populations: Perspectives of frontline staff. Journal of HIV/AIDS & Social Services. 2009;8:95–113. doi:10.1080/15381500902736384. [Google Scholar]
  44. Sabharwal C, Muse K, Alper H, Begier E, McNeill M, Galeta G, Parvez F. Jail-based providers’ perceptions of challenges to routine HIV testing in New York City jails. Journal of Correctional Health Care. 2010;16:310–321. doi: 10.1177/1078345810378842. doi:10.1177/1078345810378842. [DOI] [PubMed] [Google Scholar]
  45. Spaulding A, Stephenson B, Macalino G, Ruby W, Clarke J, Flanigan T. Human immunodeficiency virus in correctional facilities: A review. Clinical Infectious Diseases. 2002;35:305–312. doi: 10.1086/341418. [DOI] [PubMed] [Google Scholar]
  46. Taxman F, Belenko S. Implementing evidence-based practices in community corrections and addiction treatment. Springer; New York: 2012. [Google Scholar]
  47. Taxman F, Henderson C, Belenko S. Organizational context, systems change, and adopting treatment delivery systems in the criminal justice system. Drug and Alcohol Dependence. 2009;103:S1–S6. doi: 10.1016/j.drugalcdep.2009.03.003. [DOI] [PubMed] [Google Scholar]
  48. Wexler H, Zehner M, Melnick G. Improving drug court operations: NIATx organizational improvement model. Drug Court Review. 2012;8:80–95. [Google Scholar]

RESOURCES