Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Jan 3.
Published in final edited form as: J Drug Issues. 2011;41(4):441–446. doi: 10.1177/002204261104100401

Community Providers’ Impression of HIV Prevention Intervention Research in NIDA’s Clinical Trials Network

Aimee N C Campbell, Bryan Hartzler, Mary Hatch-Maillette, Donald A Calsyn, Gloria M Miele, Susan Tross
PMCID: PMC3536537  NIHMSID: NIHMS369026  PMID: 23293393

Abstract

Research-based approaches to HIV risk reduction are available but not readily adopted by community-based treatment programs. This exploratory survey study assessed staff (N=116) attitudes as a function of direct research participation, treatment program type, and study performance within seven methadone maintenance and eight psychosocial outpatient substance abuse treatment programs that participated in the NIDA Clinical Trials Network HIV risk reduction trials. Clinical staff who directly participated in the research reported intervention components as useful and were more likely to report perceived increases in HIV testing/referrals compared to staff who did not directly participate. However, those directly involved reported less positive attitudes about clinical impact and research impression. Results suggest a positive influence of research participation on awareness of program services, but also the need to address practical and professional issues related to research collaboration. Effectiveness trials offer a valuable opportunity to assess provider-level factors associated with adoption and implementation.

Introduction

The gap between science and practice within substance abuse treatment has been readily acknowledged (Carroll & Rounsaville, 2007; Lamb, Greenlick, & McCarty, 1998; Miller, Sorensen, Selzer, & Brigham, 2006). A reliance on informal diffusion strategies, as opposed to more active dissemination activities (Greenhalgh, Glenn, MacFarlane, Bate, & Kyriakidou, 2004; Miller et al., 2006), as well as a previous funding bias towards early phase clinical trials (Glasgow, Lichtenstein, & Marcus, 2003), has exacerbated this gap. Further, even when empirically-supported interventions are adopted by community programs, they may not be implemented as designed or sustained over time (Fichman & Kemerer, 1999). In response to recommendations set forth in a report published by the Institute of Medicine (Lamb, 1998.) regarding the research to practice gap, the National Institute on Drug Abuse (NIDA) established the Clinical Trials Network (CTN) specifically to implement and evaluate efficacious interventions in community-based effectiveness trials (Hanson, Leshner, & Tai, 2002). Community-based research offers opportunities to inform our understanding of relevant barriers and facilitators of adoption and implementation of empirically-supported interventions (Eke, Neumann, Wilkes, & Jones, 2006; Guydish, Tajima, Manser, & Jessup, 2007). Barriers and facilitators to adoption can include intervention feasibility and acceptability, program resources and priorities, and intervention cost-effectiveness. Many of these issues can be assessed during effectiveness trials with the goal of increasing adoption and sustainability of empirically-supported interventions.

In 2009, injection drug use directly accounted for 12% of HIV infections (Centers for Drug Control [CDC], 2011). However, heterosexual transmission from a high risk partner, including contact with a person known to have or be at high risk for HIV infection, such as an injection drug user, comprised almost 31% of HIV infections. Thus, substance use likely contributes to the spread of HIV/AIDS beyond individual substance use, to sexual partners and other family. Further, extensive research has identified specific risk factors associated with substance use that increase the risk of HIV transmission, including more unprotected sexual encounters, higher numbers of sexual partners, higher instances of sexual activity while under the influence of drugs or alcohol, and exchanging sex for money or drugs (Booth, Kwiatkowski, & Chitwood, 2000; Logan, Cole, & Leukefeld, 2002; Sly, Quadagno, Harrison, Eberstein, & Riehman, 1997). Given the multiple risk factors for HIV transmission among substance users, HIV prevention is a particularly important target for effectiveness and implementation research of empirically-supported treatments (NIH Office of AIDS Research) in substance abuse treatment programs.

Overall, studies of HIV risk reduction interventions conducted in drug abuse treatment programs have been shown to be effective, particularly if they contain attitudinal arguments, educational information, behavioral skills arguments, and behavioral skills training provided across multiple sessions (Albarracín, et al., 2005; Calsyn et al., 2009; Copenhaver et al., 2006; Prendergast, Urada & Podus, 2001; Semaan et al., 2002; Tross et al., 2008). Beginning in 1996, the Centers for Disease Control and Prevention initiated the Replicating Effective Programs (REP) and Diffusion of Effective Behavioral Interventions (DEBI), charged with packaging and facilitating dissemination of evidence-based HIV prevention interventions. Fourteen interventions have gone through REP and DEBI. Despite the existence of efficacious multi-session HIV prevention interventions, most substance abuse treatment programs continue to offer single session HIV prevention focused on education (Shoptaw et al., 2002). Further, HIV risk reduction intervention trials that have been conducted in substance abuse treatment programs have been concentrated in methadone programs (Sorensen & Copeland, 2000), making the ability to generalize across program types and substances of abuse challenging.

A key element to successful integration of empirically-supported treatments is examining factors related to implementation of new treatments (Garner, 2008). In a prior NIDA CTN trial investigating infectious disease services within substance abuse treatment programs, Brown et al. (2007) found clinician attitudes and beliefs were associated with providing infection-related services. Clinicians who were comfortable discussing issues related to sexuality and sexual behavior or who believed that infectious disease prevention and treatment were important were more likely to provide infectious disease services, such as screening, encouraging risk reduction behaviors, and medical exams (Tracy et al., 2009). In a recent study using a national sample (N = 571) of substance abuse treatment providers affiliated with the NIDA CTN, the contribution of several factors—demographics and attitudes and involvement in research—toward providers’ willingness to use research findings in practice was examined (Pinto, Yu, Spector, Gorroochurn, & McCarty, 2010). Providers involved in research and who had more favorable attitudes toward evidence-based practices were significantly more willing to use findings in practice. Research participation can also raise concern among clinicians. In a qualitative study of 85 staff participating in the CSAT Methamphetamine Project (including 50 clinicians), Obert et al. (2005) concluded that a randomized control trial of a manualized treatment may not be the best method of introducing evidence-based treatments into practice settings, as clinicians often confounded research-based constraints and burden with their overall acceptance of the treatment itself (e.g., addressing client needs within the confines of a research protocol, feeling restricted by the treatment manual). Other research has found that program/clinician treatment philosophy (Miller et al., 2006), licensure or certification status (Haug, Shopshire, Tajima, Gruber, & Guydish, 2008), and work climate and self-efficacy (Durlak & DuPre, 2008) can also impact adoption and implementation of evidence-based services. Overall, these findings suggest that staff attitudes, as well as other programmatic factors, can have a strong influence on perception and delivery of HIV and other infectious disease prevention services in the context of community-based research.

The current study used data from professional clinical staff at 15 treatment programs that participated in the NIDA CTN Real Men are Safe/Safer Sex Skills Building for Women multi-site trials. Specifically, the study sought to explore the clinical and research impressions and perceived HIV-related referrals of community-based providers who participated in an effectiveness trial based on 1) whether they directly participated in the research study, 2) whether they were from outpatient psychosocial programs or methadone maintenance programs, and 3) participant retention rates in the study intervention. Based on prior research, it was hypothesized that clinicians who participated in the research study directly, were from methadone maintenance programs, and whose programs produced better intervention retention would report more positive research and clinical impressions and higher HIV-related referrals. The study is unique in sampling both outpatient psychosocial and methadone maintenance programs, including measures of both clinical and research attitudes, and evaluating clinician perception based on empirical trial data. The findings of this study will add to the evidence base on ways to improve researcher/practitioner collaborations and inform implementation efforts of empirically-supported HIV risk reduction interventions.

Methods

Participants

Participants were clinical staff (N = 116) employed at seven methadone maintenance and eight outpatient, psychosocial community treatment programs across the United States. The programs had participated in either one or both national multi-site randomized clinical trials comparing the effectiveness of a five-session, manual-driven, safer sexual skills building intervention to a standard one-session HIV/AIDS education group among men or women enrolled in outpatient substance abuse treatment programs (Real Men Are Safe, Calsyn et al., 2009; Safer Sex Skills Building for Women, Tross et al., 2008).

Inclusion criteria included being 18 years or older, employed by a treatment program that had participated in either of the aforementioned studies, and being either a counselor or other staff member responsible for direct patient care at the program or a manager or director of the program. Excluded was staff that had no direct clinical responsibilities (e.g., administrative support staff).

Procedures

The lead investigators for the Real Men Are Safe and Safer Sex for Women trials distributed Exit Survey Packets to all professional clinical staff at the 15 programs involved in the safer sex protocols within the first six months after the clinic’s participation in those protocols had ended. Of the clinics who received Exit Survey Packets, 14 participated in the men’s study and 11 participated in the women’s study. Nine sites participated in both protocols. The Exit Survey Packet included a self-administered informed consent form, a contact sheet and a paper-and-pencil survey. All Exit Survey materials, consent form, and procedures were approved by Institutional Review Boards at the lead investigators’ institutions. All interested professional clinical staff and managers were invited to participate on a voluntary basis. To protect confidentiality, all surveys and envelopes were labeled only with unique identification numbers. Each Exit Survey Packet contained a return envelope; respondents were instructed to seal their surveys and consent forms in the return envelope and place them into a drop box, to be picked up by the treatment program contact person and returned to the lead investigators.

One contact person at each program was identified ahead of time (in most cases the research coordinator for the safer sex study) to receive, distribute, collect, and mail back the Exit Survey Packets. A total of approximately 212 surveys were distributed with an overall return rate of 59.4%. Two programs did not return surveys.

Data Collection

The Clinician Exit Survey was a brief, self-administered, paper-and-pencil survey to assess professional clinical staff’s attitudes and perceptions of participating in research within their clinic setting. The Exit Survey was adapted from an interview guide used in a prior study examining post-research trial intervention adoption (Guydish et al., 2005). It contained an initial set of questions about clinicians’ demographics, clinic role (e.g., “What do you perceive as your primary job at the clinic?”), research study role (i.e., direct [counselor, supervisor] vs. no direct role), and prior experience delivering HIV/STD prevention or counseling. Using a 5-point Likert-type scale (1=Strongly Agree; 5=Strongly Disagree), the Exit Survey contained 16 attitudinal questions measuring perceived impact of participating in the Safer Sex studies in both clinical (e.g. “The clinic’s participation in the Safer Sex studies increased my awareness of sex education and HIV/STD prevention and counseling with clients.”) and research domains (“My workload increased without additional compensation while the Safer Sex studies were being conducted.”). Reverse coding was used so that higher scores on both attitudinal subscales scales (clinical and research) represent more positive opinions. Respondents were then asked to rate their perception of service change in HIV, STD, and HCV testing/counseling or treatment at their clinics on a 3-point scale (1=Increased; 2=Stayed the same; 3=Decreased). Finally, clinicians who had participated in one of the Safer Sex studies as a study counselor were asked to rate on a 5-point Likert-type scale (1=Not at All Useful; 5=Extremely Useful) the utility of the respective intervention components for participants. Core components of both the men’s and women’s intervention included basic HIV information, risky behavior self-assessment, condom use skills and practice, communication skills with partners, and problem solving skills. The men’s intervention specifically emphasized exploration of the interplay between sex and drug use and discussion about sex roles; although these topics were raised in the women’s intervention they were not considered core elements. Clinicians were instructed to base ratings on the extent to which clients enthusiastically engaged in activities, asked questions and made positive or negative comments about the intervention materials.

Data Analyses

Preliminary analytic processes targeted data reduction of contents of the Exit Survey, given its length and lack of prior psychometric examination. The co-authors independently reviewed content of the attitudinal items, formulated a shortlist of groupings for conceptually-linked items, and then compared groupings and resolved discrepancies via conference. Scale reliability analyses then tested intercorrelation among items for two proposed subscales, designated as Clinical Impact and Research Impression. Cronbach alphas for an eight-item Clinical Impact attitudinal subscale score and seven-item Research Impression attitudinal subscale score were reasonably strong (alpha = .78, .72, respectively). See Table 1 for a final list of questions comprising each scale.

Table 1.

Questions comprising the Clinical Impact and Research Impression Attitudinal Subscales

Subscale Questions
Clinical Impact

  1. The Safer Sex Studies met an important unmet HIV/STD education and prevention service need in our clinic. *

  2. The clinic’s participation in the Safer Sex Studies increased my awareness of sex education and HIV/STD prevention and counseling with clients. *

  3. Clients participating in the Safer Sex Studies in our clinic have increased the initiation of discussions about sexual issues in my individual or group counseling sessions. *

  4. Clients not participating in the Safer Sex Studies in our clinic have increased the initiation of discussions about sexual issues in my individual or group counseling sessions. *

  5. Since the Safer Sex Studies were conducted in our clinic I have increased the frequency of my initiating discussions about sexual issues in my individual or group counseling sessions. *

  6. The Safer Sex Studies identified extra clinical problems in my clients.

  7. The clinic’s participation in the Safer Sex Studies helped to make me a better clinician. *

  8. Since the Safer Sex Studies began I have been asked more questions about sexual issues for which I did not know the answer.
Research Impression

  1. The clinic’s participation in the Safer Sex Studies increased my awareness of research procedures. *

  2. The research doesn’t have any relevance for the treatment/services in this clinic.

  3. The researchers are the only people who will benefit from this research.

  4. My workload increased without additional compensation while the Safer Sex Studies were being conducted.

  5. The Safer Sex Studies adversely disrupted day-to-day operations within the program.

  6. The Safer Sex Studies created extra paperwork for me that I didn’t have time to do.

  7. I want to participate in another research study in the future. *
Open in a new tab

Note. Respondents answered each statement on a Likert-type scale where 1=Strongly agree: 2=Agree: 3=Neither agree nor disagree; 4=Disagree; 5-Strongly disagree. Several items (denoted by *) were reverse coded so that higher subscale scores corresponded to more positive clinical impact or research impression.

Data reduction efforts next focused on Exit Survey items tapping perceived impact of clinic participation on HIV, STD, and HCV services. Due to these items’ restricted range, content-related items differentiating impact on ‘testing/counseling’ and ‘treatment’ were combined and the scale was recoded as a binary (1= increase in services, 0 = lack of increase in services) format. No data reduction processes were applied to Exit Survey items that assessed perceived utility of intervention(s) components, as perceptions of the individual intervention components was of primary interest. Notably, the latter analyses were performed on partial samples given that not all of the involved clinics implemented both women’s and men’s interventions.

Upon determining attitudinal (Clinical Impact, Research Impression) and perceived clinic services impact (increase/not in HIV, STD, & HCV services, respectively) indices, data analyses next focused on describing sample characteristics. Comparisons relied on analysis of variance (ANOVA) and chi-square (χ2) tests to determine the extent of group differences.

Subsequent analyses focused on predicting variation in the noted respondent and program level Exit Survey indices. These analyses first compared staff participating vs. those not participating in CTN protocol implementation in Exit Survey indices via use of ANOVA and χ2 tests. The same data-analytic approach was then employed to compare Exit Survey indices among staff of clinics wherein methadone maintenance was the primary treatment vs. staff of psychosocial clinics. Given the differential success among clinics in promoting client attendance to the interventions, further analyses utilized intervention completion rate as a clinic-level variable. This predictor, conceptualized as a three-level independent variable (i.e., high = 67+% completion rate, medium = 51–66%, low = 50% or lower) was utilized in ANOVA and χ2 tests to explore variance in Exit Survey indices.

Results

Sample Characteristics

Table 2 displays characteristics for the collective sample, differentiated by staff that did and did not actively participate in implementing the two CTN protocols. The sample was split almost evenly by gender, with women comprising 55.2% of respondents. A little over half held a graduate degree (57.8%). Most said their primary job at the treatment program was that of a counselor (68.1%). On average, respondents had been working at the treatment program for 6 years (SD = 5.2) and about three quarters had been trained on or had delivered some type of HIV prevention intervention prior to the study. A majority had reported no prior research experience (64.7%). Slightly more than half the sample worked in methadone maintenance programs (54.3%) and 45.7% worked in outpatient psychosocial programs. No differences were detected between staff who actively participated in CTN protocol implementation and nonparticipating staff in terms of gender, education, clinic role and longevity, prior experience delivering HIV prevention education, prior experience in conducting research, and type of program.

Table 2.

Sample Characteristics

Total Direct Study Involvementa No Direct Study Involvementb
N = 116 n = 47 n = 69
n (%) or M (SD) df χ2 or F
Gender 1 2.39
 Male 52 (44.8%) 17 (36.2%) 35 (50.7%)
 Female 64 (55.2%) 30 (63.8%) 34 (49.3%)
Highest Degree 1 3.45
 ≤ Bachelor’s 49 (42.2%) 15 (31.9%) 34 (49.3%)
 Graduate 67 (57.8%) 32 (68.1%) 35 (50.7%)
Primary Job 7 11.46
 Counselor 79 (68.1%) 31 (66.0%) 48 (69.6%)
 Case Manager 6 (5.2%) 2 (4.3%) 4 (5.8%)
 Clinical Supervisor 8 (6.9%) 2 (4.3%) 6 (8.7%)
 Program Director 11 (9.5%) 5 (10.6%) 6 (8.7%)
 Administrator 5 (4.3%) 1 (2.1%) 4 (5.8%)
 Nursing Staff 2(1.7%) 2 (4.3%) 0
 Other 4 (3.4%) 4 (8.5%) 0
Years at Treatment Program 6.2 (5.2) 5.4 (4.8) 6.7 (5.4) 1 1.88
Past experience/training in HIV Prevention 2 0.39
 None 29 (25.0%) 13 (27.7%) 16 (23.2%)
 Delivered 65 (56.0%) 26 (55.3%) 39 (56.5%)
 Training 22 (19.0%) 8 (17.0%) 14 (20.3%)
Prior Research experience 3 1.22
 Direct 18 (15.5%) 9 (19.1%) 9 (13.0%)
 Indirect 23 (19.8%) 8 (17.0%) 15 (21.7%)
 No Role/No Experience 75 (64.7%) 30 (63.8%) 45 (65.2%)
Program Type (Methadone) 63 (54.3%) 30 (63.8%) 33 (47.8%) 1 2.89
Open in a new tab
a

No significant differences were detected based on study involvement for any variable.

Perceived Usefulness of Intervention Components

Staff who directly participated in the women’s and men’s Safer Sex Studies was asked their perception of how useful study participants found specific intervention components. For the men’s study, across 8 components (e.g., risky behavior self-assessment, barriers to condom use, exploration of interplay between sex and drug use) responses ranged from M = 3.77 (SD = 0.88) to M= 4.16 (SD = 0.78) on a 5-point Likert-type scale (1=not at all useful to 5=extremely useful). Using the same scale, staff who participated in delivering the women’s intervention responded with a mean range of 4.23 (SD = 0.82) to 4.40 (SD = 0.86) across 5 components. Components from the women’s intervention included risky behavior self-assessment, communication skills with partners, and problem solving skills.

Continued Intervention Use and Interest in Training

All respondents were asked how much they agreed with the statements of wanting to continue using the study interventions (for staff who delivered the intervention during the study) or wanting training in the intervention (for staff who did not deliver the intervention during the study). The mean response for those who delivered the intervention was 2.1 (SD = 1.0), corresponding to “agree” with wanting to keep using the intervention. The mean response for wanting training in the Safer Sex interventions was 3.1 (SD = 1.2), corresponding to “neither agree nor disagree.”

Differences in Exit Survey by Study Participation and Treatment Program Type

Table 3 presents the two attitudinal subscales (clinical impact and research impression) and the perception of service increases in HIV, STD, and HCV testing and referral by study participation status and treatment program type (methadone versus outpatient psychosocial). Program staff who directly participated in the implementation of the research study had a less positive attitude toward the study’s clinical impact (F(1, 115) = 14.68, p < .001) and a less positive attitude about research (F(1, 115) = 4.83, p < .05) than those who did not directly implement the study. Those who were directly involved in implementing the research study were more likely to report increases in HIV testing and referrals for testing at the treatment program (χ2(1) = 5.55, p < .05). Clinical staff from methadone programs were also more likely to observe increases in HIV testing and referrals (χ2(1) = 4.32, p < .05) compared to staff from psychosocial programs. There were no differences by study participation status in the perception of STD or HCV testing and referrals. Among methadone and outpatient psychosocial staff, no other significant differences were found in the Exit Survey indices (i.e., clinical impact, research impression, STD or HCV testing and referrals).

Table 3.

Tests of Association by Outcome Variable, Study Involvement Status, and Treatment Program Type Using Combined Real Men are Safe and Safer Sex Skills Building for Women

N Direct Study Involvement No Direct Study Involvement Methadone Outpatient Psychosocial
n = 47 n = 69 n = 63 n = 53
n (%) or M (SD) χ2 or F (df = 1) n (%) or M (SD) χ2 or F (df = 1)
Clinical Impacta 116 2.5 (0.73) 3.0 (0.69) 14.68*** 2.9(0.78) 2.7 (0.70) 2.60
Research Impressiona 116 2.1 (0.60) 2.3 (0.62) 4.83* 2.2 (0.63) 2.3 (0.62) 1.34
HIV Testing/Referralsb 108 26 (57.8%) 22 (34.9%) 5.55* 32 (53.3%) 16 (33.3%) 4.32*
STD Testing/Referralsb 108 18 (40.0%) 16 (25.4%) 2.60 16 (26.7%) 18 (37.5%) 1.45
HCV Testing/Referralsb 107 14 (31.8%) 15 (23.8%) 0.84 17 (28.8%) 12 (25.0%) 0.20
Open in a new tab
a

Attitudinal subscale with items scored on 5-point scale (higher scores correspond to more positive impact or impression).

b

% perceived increase in testing and referral.

*

p <.05.

**

p <.01.

***

p <.001.

Differences in Exit Survey by Intervention Completion

Intervention completion rates (defined as having attended at least 3 out of the 5 Real Men Are Safe/Safer Sex for Women intervention sessions, or the single HIV-Ed intervention depending on randomization status) across CTPs ranged from 40–74% in the men’s study and 35–79% in the women’s study. Differences across the Exit Survey indices by intervention completion status in both studies are shown in Table 4. Respondents’ attitude toward the clinical impact of the men’s study was significantly different depending on intervention completion rates (F(2, 89) = 5.20, p < .01). Attitudes became more positive as the intervention completion increased such that the mean scale score for respondents from sites with less than 50% completion was 2.4 (SD = 0.4), 2.6 (SD = 0.6) for sites with 51–66% completion, and 3.0 (SD = 0.8) for sites with 67% or greater completion. There were no other significant differences in other Exit Survey indices (i.e., research impression or HIV/STD/HCV testing and referrals) for respondents from sites participating in the men’s study. For respondents from sites who participated in the women’s study, there was a significant difference in research impression depending on intervention completion at the CTP. Respondents from sites with better completion rates (51% or greater) had a less positive research impression compared to those from sites with 50% or less completion (F(2, 79) = 4.01, p < .05). There were no significant differences in clinical impact or HIV/STD/HCV testing and referrals for respondents whose clinic participated in the women’s study.

Table 4.

Tests of Association by Outcome Variable and Clinic Level Intervention Completion Rates for All Respondents from Clinics who Participated in Real Men are Safe or Safer Sex Skills Building for Women

N Clinic: Level Intervention Completion Rates
Real Men Are Safe Safer Sex Skills Building for Women
0–50% 51–66% 67+% 0–50% 51–66% 67+%
n = 15 n = 33 n = 43 n = 19 n = 39 n = 23
n (%) or M (SD) χ2 or F (df = 2) N n (%) or M (SD) χ2 or F (df = 2)
Ciinical Impacta 90 2.4 (0.4) 2.6 (0.6) 3.0 (0.8) 5.20** 80 2.8 (0.8) 2.8 (0.8) 3.0 (0.7) 0.40
Research Impressiona 90 2.2 (0.3) 2.0 (0.5) 2.2 (0.5) 0.68 80 2.6 (0.6) 2.1 (0.6) 2.2 (0.6) 4.01*
HIV Testing/Referralsb 91 7 (46.7%) 16 (48.5%) 19 (44.2%) 0.14 81 10 (52.6%) 20 (51.3%) 10 (43.5%) 0.46
STD Tesring/Referralsb 91 5 (33.3%) 14 (42.4%) 11 (25.6%) 2.40 81 9 (47.4%) 13 (33.3%) 5 (21.7%) 3.08
HCV Testing/Referralsb 90 3 (21.4%) 11 (33.3%) 10 (23.3%) 1.20 80 7 (38.9%) 9 (23.1%) 9 (39.1%) 2.37
Open in a new tab
a

Attitudinal subscale with items scored on 5-point scale (higher scores correspond to more positive impact or impression).

b

% perceived increase in testing and referral.

*

p < .05.

**

p < .01

Discussion

This study contributes data on the attitudes and perceptions of substance abuse treatment providers following the implementation of evidence-based HIV prevention interventions. Overall, findings are mixed with respect to the effect of research participation on treatment program staff. Clinical staff participating directly in the research reported high utility of individual intervention components and appeared more attuned to HIV/STI referrals in their clinical work. Results also suggest, however, that the experience of participating in a research study may not be sufficient to increase support for the adoption of an evidence-based intervention; clinicians who actively participated in the research reported less positive attitudes towards research and clinical impact of the intervention. It is important to note, however, that the scores on the attitudinal subscales, regardless of the sample being analyzed, were primarily on the lower end of the scale. The highest scores corresponded to the neutral midpoint of the rating scale.

Although the finding that those who directly participated in the research had less positive impressions runs counter to some prior research (Pinto et al., 2010; Roman, Abraham, Rothrauff, & Knudsen, 2010), it is not without precedent (Obert et al., 2005). Clinicians involved with study implementation had greater burden than those not involved due to additional training, paperwork, and group facilitation responsibilities. Further, research typically requires higher levels of fidelity, monitoring and supervision and emphasis on treatment attendance and retention which may have caused additional stress. Thus, there was a positive influence of research participation on perceived service delivery and perception of the evidence-based HIV prevention intervention, but there appears to be a need to improve the practical and professional experiences of staff participating in research collaborations.

The finding that clinicians participating in the implementation of the research study perceived greater referrals suggests that they themselves may have made more referrals or that they were more aware of the need for HIV/AIDS testing in general. This outcome was found over six months after the active treatment phase of the study was complete and indicates an area of potential long-term impact from research study participation. Future studies should assess related outcomes linked to research participation and intervention implementation, including increased awareness of additional client problems, referrals and service utilization, and staff comfort in addressing relevant problem areas. Clinic-wide involvement in research (as opposed to inviting only a few counselors to participate) may also increase long-term outcomes.

Increasing awareness of positive client outcomes may also help to balance the additional burden inherent to research participation. In a prior study of counselors involved in CTN trials, research burden was associated with higher turnover intention; however, if counselors perceived the intervention as being beneficial to clients or to the program, then turnover intention was significantly lower (Knudsen, Ducharme, & Roman, 2007). Thus, the perceived burden of research can be reduced if the clinical staff is made aware and informed of direct intervention benefit. This may be more difficult in the case of multi-session HIV prevention interventions which may not be part of substance abuse treatment program’s core curricula. The time and space for training and implementation, given large program caseloads and tight staff schedules, are obstacles to adopting intervention procedures. Increasing awareness of positive client outcomes may also be especially difficult for interventions, which target behavior change that is invisible to the clinician (i.e., private sexual behaviors).

Mixed results were found for the association between intervention attendance rates during the study and clinical and research impressions by clinical staff. Higher intervention completion rates were associated with more positive clinical impression of the study in the men’s trial. Clinicians may have perceived that participants with higher attendance rates were more likely to have achieved better clinical outcomes or felt more successful in their own execution of the intervention. This finding underscores the need to focus on attendance rates during study treatment—not only to enhance the ability to assess intervention outcome, but to promote clinician morale and enthusiasm. In the women’s trial, higher completion rates were associated with a less positive research impression. Although counterintuitive, this finding may be the result of clinical staff feeling burdened by the additional work accompanying higher attendance rates.

Clinicians from methadone clinics were significantly more likely to perceive increases in HIV testing and referral compared with their outpatient psychosocial program counterparts. HIV testing and referral may be more common and accessible within methadone programs as there are typically more injection drug users (a significant risk factor for HIV infection), as well as on-site medical staff who may be more accustomed to these type of services. Additional effort may be necessary within psychosocial programs to provide HIV testing onsite or improve HIV testing referral accessibility offsite. Although HIV prevention research is more common within methadone clinics, there were no differences in research or clinical impressions between methadone and psychosocial programs.

Findings from this study suggest other implementation and intervention sustainability issues in need of further research. For example, early clinic staff involvement in the planning of community-based research may reduce feelings of burden. Preparing providers for the research implementation process through early dialogue, integration in study planning, and through ongoing communication during the trial may help counteract some of the burdensome elements of research participation. Prior research suggests that a comprehensive understanding of provider views of facilitators and barriers to adoption is necessary prior to adoption decisions, as implementation challenges may ameliorate the salience of addressing these early concerns (Seffrin, Panzano, & Roth, 2008). Active participation in study decision making may enhance overall impression of the research process and awareness of the clinical impact of the intervention. In the two Safer Sex studies, the majority of counselors were compensated by freeing them of some of their usual duties in order to allow time for the research activities. A minority of counselors were compensated with additional pay. There was anecdotal evidence that for many counselors, they did not experience a true reduction in their usual responsibilities and the research activities were thus perceived as additional work for no additional compensation. Again anecdotally, those who received overtime pay for the extra work involved in learning and implementing a study intervention felt less burdened. These informal reports suggest that counselor buy-in and impressions toward research may be improved through compensation or active participation in clinical effectiveness and implementation research. Given the modest scores on the clinical impact and research impression subscales, more research is needed to determine how to better involve and work with community providers in a research context to improve adoption and implementation of evidence-based treatments.

Limitations

Although an acceptable survey completion rate was obtained, approximately 40% of surveys were not returned. The non-completion rate reduces generalizability of the results and raises questions as to possible bias. Comparable numbers of participating (n = 47) and non-participating (n = 69) clinicians reduces some of this concern. It should be noted that because participating and non-participating clinicians worked within the same programs, sharing of experiences and intervention content may have occurred. This “contamination” was minimized in that participating clinicians completed a 3-day centralized training and received ongoing supervision during the trial. Further, contamination would likely dilute differences between participating and non-participating clinicians, making the current findings more conservative.

An additional limitation of the current study is that the clinician assessments did not include measures of actual post-study intervention implementation rates or obtain challenges related to the continued delivery of the intervention. However, in a separate investigation of the continued use of Real Men Are Safe in participating methadone maintenance programs, none of the programs adopted the intervention wholesale; primary reasons cited were insufficient staff time, competing treatment priorities, and reimbursement challenges (Sterling, 2010). Future research in this area should examine the extent of sustainability of interventions, specific components that are retained or discarded, and adherence and fidelity to the delivery of core components of the intervention post-study. Similarly, programs should be queried about whether they are continuing to assess client outcomes pre- and post-intervention delivery.

Sustainability of evidence-based treatment may require modifications to enhance the likelihood of continued use, taking into account service provision, client population, staff training, and local and state funding and policy. Roman and colleagues, in prior research on clinicians involved in CTN research, found multiple barriers to the use of empirically-supported interventions, including interventions perceived as inconsistent with program ideology, cost of implementing the intervention, and logistics (e.g., lack of time and short treatment episodes) (Roman et al., 2010). The Safer Sex studies used on-site supervisors as a mechanism to support continuation of intervention implementation; however, other practical and logistical issues were not a systematic part of ongoing discussions with staff (e.g., how to maintain a 5-session gender specific HIV prevention intervention when the norm is a single mixed gender group). Although not assessed in the current set of analyses, program constraints and needs should be included as part of ongoing dialogue. Future studies should strive to fully integrate experimental interventions into the service delivery and program mandates in place within community treatment programs. In this way, research results are generalizable to real-world settings and programs may be better able to sustain interventions if they are a part of clinic routine.

Conclusion

This study presents the attitudes and impressions of clinical staff whose community treatment programs participated in HIV risk reduction intervention research. Positive aspects of research participation included perceived increases in clinical services and overall endorsement of the utility of intervention components. Mixed and somewhat disappointing findings related to attitudes about clinical impact and research impression highlight the need for additional research on the practical and professional considerations of clinical staff involved in research. Further, aspects of trial performance, such as intervention attendance, also influenced attitudes. Promising primary outcome findings and perceived usefulness of an HIV-prevention intervention do not necessarily equate with positive clinical impact or research impressions which may in turn impact use of the intervention and other related services. Effectiveness trials, such as those conducted within the CTN, offer a valuable opportunity to study issues related to the implementation and continued use of empirically-supported treatments. This study supports continued research on effective methodology for collaboration between investigators and providers to influence post-study implementation.

Acknowledgments

This research was supported by grants from the National Institute on Drug Abuse: U10 DA13035 (Edward V. Nunes and John Rotrosen, Co-PIs) and U10 DA13714 (Dennis Donovan, PI), both from the Clinical Trials Network, and K23 DA025678 (Bryan Hartzler, PI). The Real Men Are Safe and Safer Sex Skills Building for Women intervention manuals are available on the National Drug Abuse Treatment Clinical Trials Network Dissemination Library website (http://ctndisseminationlibrary.org/). The authors wish to thank the clinical and administrative staff at the participating community treatment programs for their time and support of this research.

Biographies

Aimee N. C. Campbell, Ph.D., is a Research Scientist at the New York State Psychiatric Institute and an Assistant Professor of Clinical Psychiatric Social Work in the Department of Psychiatry at Columbia University. She has been involved in intervention and dissemination research for substance use disorders for 15 years.

Bryan Hartzler, Ph.D., is a research scientist at the University of Washington Alcohol and Drug Abuse Institute, with research interests in design, evaluation, and dissemination of behavioral interventions for substance-involved populations. He is a recent recipient of a career development award (K23DA025678 Integrating Behavioral Interventions in Substance Abuse Treatment).

Mary Hatch-Maillette, Ph.D., is a Research Scientist at the Alcohol and Drug Abuse Institute at the University of Washington and has a part-time private practice. Her research and clinical interests include treatment for substance abuse and post-traumatic stress disorders, HIV risk reduction, and violence risk assessment.

Donald A. Calsyn, Ph.D., is a Professor in Psychiatry and Behavioral Sciences and a Research Affiliate with the Alcohol and Drug Abuse Institute at the University of Washington. He previously served as a staff psychologist and Director of Outpatient Services, Addiction Treatment Center, Department of Veterans Affairs Puget Sound Health Care System. His research has focused on substance abuse treatment outcomes and HIV prevention.

Gloria M. Miele, Ph.D., is Instructor of Clinical Psychology (in Psychiatry) at Columbia University College of Physicians and Surgeons and Training Director for the Greater New York Node of NIDA’s Clinical Trials Network. She has been involved in substance abuse treatment and research for over 20 years.

Susan Tross, Ph.D., is an Associate Professor of Clinical Psychology, Departments of Psychiatry and Pediatrics at Columbia University and an Associate Director of the HIV Center. Since 1986, she has conducted numerous HIV or substance abuse intervention research programs and was the Principal Investigator of the NIDA CTN Safer Sex Study for Women.

References

  1. Albarracin D, Gillette J, Earl A, Glasman LR, Durantini MR, Ho MH. A test of major assumptions about behavior change: A comprehensive look at the effects of passive and active HIV-prevention interventions since the beginning of the epidemic. Psychological Bulletin. 2005;131:856–897. doi: 10.1037/0033-2909.131.6.856. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Booth RE, Kwiatkowski CF, Chitwood DD. Sex related HIV risk behaviors: Differential risks among injection drug users, crack smokers, and injection drug users who smoke crack. Drug and Alcohol Dependence. 2000;58(3):219–226. doi: 10.1016/s0376-8716(99)00094-0. [DOI] [PubMed] [Google Scholar]
  3. Brown LS, Kritz S, Goldsmith RJ, Bini EJ, Robinson J, Alderson D, Rotrosen J. Health services for HIV/AIDS, HCV, and sexually transmitted infections in substance abuse treatment programs. Public Health Reports. 2007;122:441–451. doi: 10.1177/003335490712200404. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Calsyn DA, Hatch-Maillette M, Tross S, Doyle SR, Crits-Christoph P, Song YS, Harrer JM, Lalos G, Berns SB. Motivational and Skills Training HIV/STI Sexual Risk Reduction Groups for Men. Journal of Substance Abuse Treatment. 2009;37:138–150. doi: 10.1016/j.jsat.2008.11.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Carroll KM, Rounsaville BJ. A vision of the next generation of behavioral therapies research in the addictions. Addiction. 2007;102(6):850–869. doi: 10.1111/j.1360-0443.2007.01798.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Center for Disease Control and Prevention. [Accessed May 20, 2011];HIV Surveillance Report 2009. 2011 21 http://www.cdc.gov/hiv/surveillance/resources/reports/2009report/ [Google Scholar]
  7. Copenhaver M, Johnson B, Lee I, Harman J, Carey M the SHARP Research Team. Behavioral HIV risk reduction among people who inject drugs: Meta-analytic evidence of efficacy. Journal of Substance Abuse Treatment. 2006;31:163–171. doi: 10.1016/j.jsat.2006.04.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Durlak JA, DuPre EP. Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology. 2008;41:327–350. doi: 10.1007/s10464-008-9165-0. [DOI] [PubMed] [Google Scholar]
  9. Eke AN, Neumann MS, Wilkes AL, Jones PL. Preparing effective behavioral interventions to be used by prevention providers: The role of researchers during HIV prevention research trials. AIDS Education and Prevention. 2006;18(SA):44–58. doi: 10.1521/aeap.2006.18.supp.44. [DOI] [PubMed] [Google Scholar]
  10. Fichman RG, Kemerer CF. The illusory Diffusion of Innovation: An examination of assimilation gaps. Information Systems Research. 1999;10(3):255–275. [Google Scholar]
  11. Garner BR. Research on the diffusion of evidence-based treatments within substance abuse treatment: A systematic review. Journal of Substance Abuse Treatment. 2008;36(4):376–399. doi: 10.1016/j.jsat.2008.08.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Glasgow RE, Lichtenstein E, Marcus AC. Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. American Journal of Public Health. 2003;93(8):1261–1267. doi: 10.2105/ajph.93.8.1261. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Greenhalgh T, Glenn R, MacFarlane F, Bate P, Kyriakidou O. Diffusion of innovation in service organizations: Systematic review and recommendations. The Milbank Quarterly. 2004;82(4):581–629. doi: 10.1111/j.0887-378X.2004.00325.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Guydish J, Manser ST, Jessup M, Tajima B, Sears C, Montini T. Multi-level assessment protocol (MAP) for adoption in multisite clinical trials. Journal of Drug Issues. 2005;35:529–546. doi: 10.1177/002204260503500306. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Guydish J, Tajima B, Manser ST, Jessup M. Strategies to encourage adoption in multisite clinical trials. Journal of Substance Abuse Treatment. 2007;32:177–188. doi: 10.1016/j.jsat.2006.08.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Hanson GR, Leshner AI, Tai B. Putting drug abuse research to use in real-life settings. Journal of Substance Abuse Treatment. 2002;23:69–70. doi: 10.1016/s0740-5472(02)00269-6. [DOI] [PubMed] [Google Scholar]
  17. Haug NA, Shopshire M, Tajima B, Gruber V, Guydish J. Adoption of evidence-based practices among substance abuse treatment providers. Journal of Drug Education. 2008;38(2):181–192. doi: 10.2190/DE.38.2.f. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Knudsen HK, Ducharme LJ, Roman PM. Research participation and turnover intention: An exploratory analysis of substance abuse counselors. Journal of Substance Abuse Treatment. 2007;33(2):211–217. doi: 10.1016/j.jsat.2006.12.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Lamb S, Greenlick MR, McCarty D, editors. Bridging the Gap Between Practice and Research: Forging Partnerships with Community-based Drug and Alcohol Treatment. Washington, DC: Institute of Medicine; 1998. [PubMed] [Google Scholar]
  20. Logan TK, Cole J, Leukefeld C. Women, sex, and HIV: Social and contextual factors, meta-analysis of published interventions, and implications for practice and research. Psychological Bulletin. 2002;128(6):851–885. doi: 10.1037/0033-2909.128.6.851. [DOI] [PubMed] [Google Scholar]
  21. Miller WR, Sorensen JL, Selzer JA, Brigham GS. Disseminating evidence-based practices in substance abuse treatment: A review with suggestions. Journal of Substance Abuse Treatment. 2006;31:25–39. doi: 10.1016/j.jsat.2006.03.005. [DOI] [PubMed] [Google Scholar]
  22. National Institutes of Health Office of AIDS Research. [Accessed November 15, 2010];Trans-NIH Plan and Presidential By-pass Budget. 2011 Available at http://www.oar.nih.gov/strategicplan/fy2011/index.asp.
  23. Obert JL, Brown AH, Zweben J, Christian D, Delmhorst J, Minsky S, Morrisey P, Vandersloot D, Weiner A. When treatment meets research: Clinical perspectives from the CSAT Methamphetamine Treatment Project. Journal of Substance Abuse Treatment. 2005;28:231–237. doi: 10.1016/j.jsat.2004.12.008. [DOI] [PubMed] [Google Scholar]
  24. Pinto RM, Yu G, Spector AY, Gorroochurn P, McCarty D. Substance abuse treatment providers’ involvement in research is associated with willingness to use findings in practice. Journal of Substance Abuse Treatment. 2010;39:188–194. doi: 10.1016/j.jsat.2010.05.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Prendergast M, Urada D, Podus D. Meta-analysis of HIV risk-reduction interventions within drug abuse treatment programs. Journal of Consulting and Clinical Psychology. 2001;69:389–405. doi: 10.1037//0022-006x.69.3.389. [DOI] [PubMed] [Google Scholar]
  26. Roman PM, Abrahm AJ, Rothrauff TC, Knudsen HK. A longitudinal study of organizational formation, innovation, adoption, and dissemination activities within the Naitonal Drug Abuse Treatment Clinical Trials Network. Journal of Substance Abuse Treatment. 2010;38(4):S1, S44–S52. doi: 10.1016/j.jsat.2009.12.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Seffrin B, Panzano PC, Roth D. What gets noticed: How barrier and facilitator perceptions relate to the adoption and implementation of innovative mental health practices. Community Mental Health. 2008;44:475–484. doi: 10.1007/s10597-008-9151-x. [DOI] [PubMed] [Google Scholar]
  28. Semaan S, Des Jarlais D, Sogolow E, Johnson WD, Hedges LV, Ramirez G, Flores SA, Norman L, Sweat MD, Needle R. A meta-analysis of the effect of HIV prevention interventions on the sex behaviors of drug users in the United States. Journal of Acquired Immune Deficiency Syndromes. 2002;30:S73–S93. [PubMed] [Google Scholar]
  29. Shoptaw S, Tross S, Stephens M, Tai B the NIDA CTN HIV/AIDS Workgroup. A snapshot of HIV/AIDS-related services in the clinical treatment providers for NIDA’s Clinical Trials Network. Drug and Alcohol Dependence. 2002;66(S1):S163. [Google Scholar]
  30. Sly D, Quadagno D, Harrison D, Eberstein I, Riehman K. The association between substance use, condom use, and sexual risk among low-income women. Family Planning Perspectives. 1997;29:132–137. [PubMed] [Google Scholar]
  31. Sorensen JL, Copeland AL. Drug abuse treatment as an HIV prevention strategy: A review. Drug and Alcohol Dependence. 2000;59:17–31. doi: 10.1016/s0376-8716(99)00104-0. [DOI] [PubMed] [Google Scholar]
  32. Sterling RC. Barriers to using the Real Men Are Safe Protocol in methadone maintenance programs. Presentation at the Annual Meeting of the American Association for the Treatment of Opioid Dependence; Chicago, IL. 2010. [Google Scholar]
  33. Tracy K, Brown LS, Kritz S, Alderson D, Robinson J, Bini EJ, Levy MS, Calsyn DA, Rieckmann TR, Fuller BE, McAuliffe P, Rotrosen J. Substance abuse treatment clinician opinions and infectious disease service delivery. Journal of Addictive Diseases. 2009;25(8):8–12. doi: 10.1080/10550880802544625. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Tross S, Campbell A, Cohen LR, Calsyn D, Pavlicova M, Miele G, Hu M, Haynes LR, Nugent N, Gan W, Hatch-Maillette MA, Mandler RN, McLaughlin P, El-Bassel N, Crits-Christoph P, Nunes EV. Effectiveness of HIV/STD Sexual risk reduction groups for women in substance abuse treatment programs: Results of a NIDA Clinical Trials Network trial. Journal of Acquired Immune Deficiency Syndromes. 2008;48(5):581–589. doi: 10.1097/QAI.0b013e31817efb6e. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES