Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Feb 19.
Published in final edited form as: Exp Clin Psychopharmacol. 2014 Feb;22(1):75–85. doi: 10.1037/a0034863

Employment-based abstinence reinforcement following inpatient detoxification in HIV-positive opioid and/or cocaine-dependent patients

Kelly Dunn 1, Michael Fingerhood 1, Conrad J Wong 2, Dace S Svikis 3, Paul Nuzzo 1,4, Kenneth Silverman 1
PMCID: PMC4332775  NIHMSID: NIHMS663175  PMID: 24490712

Abstract

Employment-based reinforcement interventions have been used to promote abstinence from drugs among chronically unemployed injection drug users. The current study utilized an employment-based reinforcement intervention to promote opiate and cocaine abstinence among opioid-dependent, HIV-positive participants who had recently completed a brief inpatient detoxification. Participants (n=46) were randomly assigned to an Abstinence & Work group that was required to provide negative urine samples in order to enter the workplace and earn incentives for work (n=16), a Work Only group that was permitted to enter the workplace and earn incentives independent of drug use (n=15), and a No Voucher control group that did not receive any incentives for working (n=15) over a 26-week period. The primary outcome was urinalysis-confirmed opiate, cocaine, and combined opiate/cocaine abstinence. Participants were 78% male and 89% African American. Results showed no significant between-group differences in urinalysis-verified drug abstinence or HIV risk behaviors during the 6-month intervention. The Work Only group had significantly greater workplace attendance and worked more minutes per day when compared to the No Voucher group. Several features of the study design, including the lack of an induction period, setting the threshold for entering the workplace too high by requiring immediate abstinence from several drugs, and increasing the risk of relapse by providing a brief detoxification that was not supported by any continued pharmacological intervention, likely prevented the workplace from becoming established as a reinforcer that could be used to promote drug abstinence. However, increases in workplace attendance have important implications for adult training programs.

Keywords: HIV, contingency management, therapeutic workplace, incentive, injection drug use

Introduction

Dependence on opioids (e.g. heroin, Oxycontin®) has increased substantially in the past decade (Substance Abuse and Mental Health Services Association, 2010), and is associated with greater utilization of health care services, frequent overdose, and Human Immunodeficiency Virus (HIV) acquisition (Birnbaum et al., 2006; Han, Gfroerer, & Colliver, 2010; Hser, Evans, Huang, Brecht, & Li, 2008; Murphy, Brecht, Herbeck, Evans, Huang, & Hser, 2008; Strassels, 2009). Inpatient opioid detoxification is one approach for the treatment of heroin addiction (Kleber, 2007), but many patients relapse to opiate use after leaving the inpatient unit (Hser et al., 2008; Smyth, Barry, Keenan, & Ducray, 2010). The problem of relapse is of particular concern for HIV-positive injection drug users because of their risk of spreading HIV infection as a result of drug-related risk behaviors (e.g., sharing injection equipment).

Reinforcement of drug abstinence through the provision of monetary-based incentives for objective evidence of drug abstinence has been shown to reliably and consistently promote reductions in drug use in a wide variety of drug treatment populations and settings (Higgins, Silverman & Heil, 2008; Lussier, Heil, Mongeon, Badger, & Higgins, 2006). However, relapse to drug use frequently occurs when the abstinence contingency intervention ends (Silverman, Kiminski, Higgins, & Brady, 2011), making it critical to identify cost effective and practical methods for providing incentives for abstinence over an extended duration of time to promote sustained behavior change. Sustained behavior change is especially important in the treatment of opioid dependence, which is characterized as a chronic relapsing problem that warrants extended duration treatment (McLellan, Lewis, O’Brien, & Kleber, 2000).

Employment-based abstinence reinforcement interventions are one method that has shown considerable success at reinforcing abstinence in patients for an extended period of time. Workplaces can be ideal settings for arranging long-term abstinence reinforcement contingencies for many reasons (Silverman et al., 2005). First, employees maintain frequent and extended contact with the workplace, which provides an opportunity to continually monitor and reinforce drug abstinence. Second, workplaces control powerful reinforcers (e.g. paychecks) that can be used to reinforce drug abstinence. Third, an intervention developed within a workplace setting has high potential for dissemination to numerous real world workplace environments.

Silverman and colleagues have been systematically evaluating the use of employment based abstinence reinforcement contingencies to promote sustained drug abstinence in a variety of different treatment populations. Participants in these studies are provided access to stipend-supported training or paid employment. Drug abstinence is generally reinforced by making access to the workplace contingent upon providing a drug-free urine sample. This model has successfully promoted abstinence from opiates and/or cocaine among pregnant/postpartum women at an initial 6-month evaluation (Silverman, Svikis, Robles, Stitzer, & Bigelow, 2001) and during a 3-year extended intervention period (Silverman, Svikis, Wong, Hamptom, Stitzer, & Bigelow, 2002). It has also been shown effective in initiating (Donlin, Knealing, Needham, Wong, & Silverman, 2008; Silverman et al., 2007) and maintaining (DeFulio et al., 2009) cocaine abstinence among cocaine-abusing methadone maintenance patients; and in initiating abstinence from alcohol among alcohol dependent patients (Koffarnus et al., 2011).

More recently, employment-based abstinence reinforcement interventions have been used to promote sustained adherence with naltrexone, an opioid antagonist that can block the effect of opioids and can be used to prevent relapse following inpatient opioid detoxification (Martin, Jasinski, & Mansky, 1973; Mello, Mendelson, Kuehnle, & Sellers, 1981; Verebey, Volavka, Mile, & Resnick, 1976). Three studies inducted recently detoxified opioid-dependent patients onto oral naltrexone and then randomly assigned participants either to a contingency group that was required to accept oral naltrexone (Dunn et al., 2013), or a 6-dose course of sustained release naltrexone administered every 3 (Everly et al., 2011) or 4 (DeFulio et al., 2012) weeks to enter the workplace, or to a usual care condition that had access to the naltrexone product but could enter the workplace independent of naltrexone adherence. Results of all three studies showed improved adherence to naltrexone dosing in contingency versus control participants.

Insofar that prevention of opioid relapse and reductions in opioid use can be associated with reductions in HIV risk behaviors, these results suggest that an employment-based abstinence intervention may be a promising method for reducing HIV risk among opioid dependent patients. This research is critical because data suggests a large sample of HIV-positive patients continue to inject drugs and share injection equipment with partners who are not HIV-positive or whose status is unknown; thus, abstinence interventions could help slow the transmission of HIV (Metsch et al., 2007). HIV positive individuals who abstain from drug use also incur health benefits such as preservation of immune system functioning, slowing of disease progression, and prevention of additional communicable disease acquisition (Baum, Rafie, Lai, Sales, Page, & Campa, 2009; Lucas, Griswold, Gebo, Keruly, Chaisson, & Moore, 2006; Roux et al., 2009; Webber, Schoenbaum, Gourevitch, Buono, & Klein, 1999). Finally, though there is preliminary evidence that employment-based abstinence interventions may exert secondary effects on HIV risk behaviors (DeFulio, Donlin, Wong, & Silverman, 2009), no employment-based abstinence intervention study has directly targeted HIV positive patients.

The current study evaluated whether an employment-based abstinence reinforcement intervention could be used effectively to promote opiate, cocaine, and alcohol abstinence among opioid and/or cocaine-dependent HIV-positive injection drug users following a 3-day inpatient detoxification, and assessed whether effects on abstinence would be associated with reductions in HIV risk behaviors.

Methods

Study Procedures

The study took place in the Center for Learning and Health’s therapeutic workplace, which is located at the Johns Hopkins Bayview Medical Campus, Baltimore MD. The therapeutic workplace consists of an onsite urinalysis testing laboratory and three workrooms. Participants were assigned to a specific workstation for the duration of their participation and study staff members were available at all times to monitor participant progress and provide assistance. The Johns Hopkins Medicine IRB board approved this study and all participants provided voluntary informed consent to participate. See Silverman et al. (2007) for a detailed description of the therapeutic workplace setting.

Study staff provided participants with transportation to and from the workplace for the first 4 weeks of the study; participants were required to make their own travel arrangements thereafter. Participants were also offered a variety of clinical resources, including monthly HIV education groups, referral for motivational enhancement therapy for drug abstinence, support for transitioning into continuing care (e.g., methadone maintenance treatment), referrals to additional community resources (e.g. housing, food, medical/psychiatric problems), and an intervention to facilitate employment following completion of the study. Utilization of these resources was not tracked or factored into study outcomes.

Participants

Participants were recruited directly from a 3-day inpatient medical detoxification program located in Baltimore MD from October 19, 2000 to August 31, 2001. To be eligible for the study, participants were required to report being HIV positive, self-report injection drug use and have observable track marks, have provided an opiate and/or cocaine-positive urine sample on admission to inpatient detoxification, meet DSM-IV criteria for Opiate and/or Cocaine Dependence, make <$200/month from employment in the past 30 days, and be between the ages of 18-65. Applicants were excluded if they showed evidence of a major psychiatric disorder that could disrupt performance in the workplace (e.g., psychosis, schizophrenia) or were at imminent risk of suicide. The study was originally designed to randomize 156 participants, but was terminated after 46 participants were enrolled and randomized into the study because it was apparent at that point that the experimental intervention was ineffective in maintaining drug abstinence. Participants were 78% male and 89% African American. Additional characteristics are summarized in Table 1.

Table 1.

Demographic and Drug Use Characteristics

Abstinence
and Work
(n=16)		 Work Only
(n=15)		 No Vouchers
(n=15)		 P
Demographic Characteristicsa
 African-American (%) 88 87 93 0.85
 Male (%) 81 73 79 0.87
 Single (%) 56 73 67 0.60
 Completed Grade 12 (%) 75 67 71 0.87
 Days employed past 30 (days) 3.1 ± 1.6 5.1 ± 2.8 7.4 ± 2.6 0.46
 Earned from employment past 30 days 41.6 ± 24.5 47.7 ± 35.9 0.0 ± 0.0 0.38
Drug Use Characteristics
 Drug Use Past 30 Days (days)
  Opiate 23.1 ± 2.3 27.0 ± 1.7 26.9 ± 1.9 0.86
  Cocaine 18.2 ± 3.0 17.7 ± 3.1 13.8 ± 3.3 0.57
  Alcohol 10.6 ± 2.8 12.3 ± 3.4 13.2 ± 3.7 0.84
 Current DSM Dependence (%)
  Opiate 100 100 100 1.00
  Cocaine 88 60 71 0.22
  Alcohol 56 47 29 0.31
Mean ASI Composite Scoresb
 Medical 0.37 ± 0.08 0.39 ± 0.09 0.19 ± 0.07 0.18
 Employment 0.93 ± 0.03 0.86 ± 0.04 0.98 ± 0.02 0.03‡
 Alcohol 0.32 ± 0.09 0.31 ± 0.10 0.31 ± 0.09 0.99
 Drug 0.37 ± 0.02 0.37 ± 0.02 0.36 ± 0.02 0.89
 Legal 0.08 ± 0.04 0.26 ± 0.06 0.18 ± 0.06 0.051 §
 Family Social 0.22 ± 0.05 0.25 ± 0.05 0.16 ± 0.04 0.43
 Psychiatric 0.22 ± 0.06 0.06 ± 0.03 0.11 ± 0.06 0.08
Beck Depression Inventory (range 0-63) 17.4 ± 2.89 15.5 ± 2.61 21.6 ± 3.78 0.37
Open in a new tab
a

Values represent Least Squared Means ± SEM unless otherwise indicated

b

Severity score range 0-1

§

= Significant difference between Abstinence + Work and Work Only groups

‡

= Significant difference between Work Only and No Voucher groups

Study Assessments

Screening procedure

Brief screening procedures were used to confirm study eligibility while patients were completing inpatient detoxification. Those who consented to the study went on to complete the baseline assessment. The initial assessment focused on lifetime and recent (defined as 30 days prior to entering detoxification) alcohol and drug use, as well as psychosocial functioning in 5 additional domains (medical, employment, legal, family/social and psychological). This information was collected using the semi-structured Addiction Severity Index-Lite interview (ASI; McLellan et al., 1985). Participants were also checked for visible signs of injection drug use. DSM-IV Substance Abuse/Dependence diagnoses were assigned using the DSM checklist (Hudziak et al., 1993). Current depressive symptoms were assessed using the 21-item Beck Depression self-report inventory (BDI; Beck & Beck, 1972). Finally, the 41-item Risk Assessment of Behavior (RAB) self-report questionnaire was used to determine frequency of engagement in various HIV risk behaviors (including injection drug use and sexual activities). A urine sample was collected under staff observation and analyzed for evidence of opiate and cocaine use, and a breath alcohol concentration sample (BrAC) was collected to check for evidence of recent alcohol use. Participants earned $50 dollars for completing the intake assessment.

Monthly follow-up assessments

Assessments were completed every 30 days over the 5-month study period. At each visit, participants completed follow-up versions of the ASI and RAB. Participants also provided urine samples that were tested for evidence of opiate and cocaine use, and breath samples that were checked for evidence of recent alcohol use. Participants received $30 for each of the five 30-day assessments they completed, for total potential earnings of $150. Finally, participants completed a 12-month follow-up, which occurred 6 months after scheduled study participation ended.

Experimental Groups

Immediately after leaving the inpatient detoxification unit and before leaving the medical center campus, participants (n=46) were randomly assigned to one of three experimental groups: Abstinence & Work (n=16), Work Only (n=15), or No Voucher (n=15). All three groups were invited to attend the therapeutic workplace Monday through Friday for 26 consecutive weeks. Both the Abstinence & Work and Work Only contingency groups were required to attend the workplace daily to earn vouchers (see below); the Abstinence & Work group was also required to provide biochemical evidence of opiate, cocaine, and alcohol abstinence to gain entry into the workplace. The No Voucher control group had access to workplace training programs however did not earn vouchers for workplace participation.

Thrice Weekly Monitoring of Drug Abstinence

Urine samples were collected under same-sex staff observation from all participants thrice weekly (Monday, Wednesday, Friday) prior to entering the workplace and were tested immediately onsite for opiates (morphine, ≤300ng/ml cutoff) and cocaine (benzoylecgonine, ≤300 ng/ml) using Syva® RapidTest Kits (Syva Company, Dade Behring, Inc. Cupertinio, CA). A breath alcohol concentration sample was collected for evidence of recent alcohol use (≥ 0.003 ng/l). Abstinence & Work group participants could not enter the workplace if they tested positive for any drug, though no abstinence contingencies were placed on the other two groups.

Workplace Training Schedule

All participants were invited to attend the therapeutic workplace training program for 4 hours every weekday for 26 weeks. The workplace was open between 10:00 AM and 3:00 PM, and closed between 12:00 and 1:00 PM for lunch. Participants worked on computerized typing and keypad training programs, which were delivered through a web-based program (Silverman et al., 2005) comprised of individualized home screens that tracked and updated daily voucher earnings. Attendance in the workplace was monitored electronically via computerized ID cards that were swiped by workplace supervisors whenever the participant entered or exited the workplace. The specific details of the workplace computerized training program are available elsewhere (Dillon, Wong, Sylvest, Crone-Todd, & Silverman, 2004).

Incentive Payments

Participants in the Abstinence & Work and Work Only contingency groups earned voucher-based incentives for attendance and productivity in typing and keypad programs. All vouchers were redeemable for goods and services in the community. Study staff made all voucher purchases and no voucher earnings were paid to participants in cash. Although participants in the No Voucher control group had access to the same training and data entry programs as the contingency groups, they did not earn any vouchers for their daily participation.

Daily base pay

Participants could earn a maximum daily base pay wage of $30.00/day. Base pay wages began at $2.00/ hour and escalated (Higgins et al., 1991) by $0.50 with each consecutive day of work up to a total of $7.50/hour. Based pay vouchers were paid contingent on signing into the workplace by 10:00 a.m. and working at least 3.5 hours each workday. Violation of either criterion resulted in a base pay reset to the original $2.00/hr value and required participants to meet all workplace requirements for 9 consecutive days in order to escalate back up to $7.50/hour. However, to provide some flexibility, participants were also provided unpaid personal, sick, and emergency days that were not subject to base pay resets. One personal day was accrued for every 5 consecutive days worked and sick and emergency days that were confirmed via medical documentation did not result in a base pay reset.

Productivity pay

Participants earned daily productivity pay as they progressed through the typing and keyboard training programs. No previous computing experience was required to master the training programs. Participants completed a placement test to demonstrate baseline skill level and were placed into the program appropriate step by a staff member. The training program was divided into small trials that were practiced for 1-minute intervals until criteria for mastery was met. Participants earned money for typing correct characters, lost money for typing incorrect characters, and earned bonuses for passing steps. Results of participant performance in typing programs have been reported elsewhere (Dillon et al., 2004).

Maximum potential earnings

Over the 26-week period, participants in the Abstinence & Work and Work Only groups had the potential to earn $6,800 in voucher-based incentives.

Statistical Analyses

The primary outcome measure was mean percent urinalysis-verified opiate, cocaine, and combined opiate/cocaine abstinence. Secondary outcomes included workplace performance measures and HIV risk behaviors. We hypothesized that participants assigned to the Abstinence & Work group would have greater rates of opioid, cocaine, and combined opioid/cocaine-negative urine samples due to the abstinence contingency. The Abstinence & Work group was also hypothesized to show greater reductions in HIV risk behaviors.

Participant characteristics at baseline were compared using a one-way Analysis of Variance (ANOVA) for continuous variables and a Chi-Square test for dichotomous variables. Analyses included all participants who were randomized into the experimental groups, independent of loss to follow-up, consistent with an intent-to-treat approach. Results of thrice weekly and monthly urine drug testing were analyzed with generalized estimating equations (GEE, Zeger, Liang, & Albert, 1988). Although breath samples were collected thrice weekly to test for evidence of alcohol use, no participant provided an alcohol-positive breath sample; therefore, alcohol use was not included in the analyses. GEE models were used to compare the rate of opioid, cocaine, and combined opioid/cocaine positive urine samples collected thrice weekly (as part of study procedures) and monthly (as part of follow-up assessments), and between-group contrasts were conducted. Chi-Squares were used to compare urinalysis results at the 12-month follow-up. Missing urine samples were analyzed in three ways: by treating missing data as missing (missing missing), by imputing missing values as positive (missing positive), or by interpolating missing samples based on the results of the urine sample collected before and after the missing sample or groups of samples (missing interpolated). Missing interpolated analyses were only conducted for thrice-weekly testing due to the increased number of samples obtained.

Workplace performance measures consisted of attendance in the workplace, mean minutes worked per day, total number of minutes worked, and mean total earnings. Attendance in treatment was assessed by creating a dichotomous variable (did participant work >1 min that day yes/no) for each workday for each participant, which was compared across groups using GEE. Number of minutes worked per day was compared two ways. First, mean number of minutes worked per day was assessed using Proc Mixed and posthoc between-group comparisons were conducted using a Tukey’s test. Second, total mean number of minutes worked (collapsed across days) was analyzed with Proc Mixed and posthoc between-group comparisons were conducted with Tukey’s test. An independent groups t-test was conducted on mean group earnings for the Work Only and Abstinence & Work groups only, because the No Voucher group could not earn any incentive payments. Retention, defined as the number of scheduled days that participants worked ≥1 minute, was assessed using a Proc Mixed and a survival analysis that estimated the rate of study attrition for each group, based on the ultimate final day that a participant worked more than 1 minute in the workplace. Self-reported HIV risk behaviors collected on the RAB were evaluated as independent questions (e.g., were not summed into subscales) and compared for each of the 30-day assessment collection periods using GEE, and at the intake and 12-month follow-up time points using Chi-Square analyses.

Statistical analyses were conducted using SAS 9.3 software and P<.05 was considered significant.

Results

Participant Demographics

All participants (100%) met DSM criteria for opioid dependence, and 73% and 44% met criteria for cocaine and alcohol dependence, respectively. The three groups were largely similar in demographic and drug use characteristics (see Table 1), with the exception of the ASI Employment composite score (F (2,42) = 4.04, P=0.03), which was driven by the No Voucher group who had significantly higher scores on the employment subscale compared to the Work Only group (P=0.02). The legal subscale closely approached significance (F (2,42)=3.19, P=.051), which was driven by the Work Only group who had significantly higher scores on the legal subscale compared to the Abstinence & Work group (P=.04).

Drug Abstinence

As seen in Table 2 and Figures 1 and 2, urine samples collected at the thrice-weekly assessments revealed no significant effect of group assignment for the missing-missing, missing-positive, or missing-interpolated analyses. When missing-missing data were evaluated, there were no between-group effects on cocaine or combined opiate/cocaine abstinence. The analysis for opiates was not conducted because the model converged due to lack of variability in the data. When missing-positive and missing-interpolated (Table 2) analyses were conducted, there were no between-group effects on opiate, cocaine, or combined opiates/cocaine.

Table 2.

Urinalysis, Workplace, and HIV-risk Behavior Outcomes

Abstinence
and Work
(n=16)		 Work Only
(n=15)		 No Vouchers
(n=15)		 P OR (95% CI)
Urinalysis Outcomesa
 Urinalysis Samples Provided (%) 58 76 64 0.51 0.79 (0.25 - 2.51)
 Opiate-negative Urinalysis (%)
  Intake 7 14 7 0.78 --
  Interventionb 45 31 42 0.63 0.57 (0.17- 1.90)
  12-month follow-up 62 33 27 0.99 --
 Cocaine-negative Urinalysis (%)
  Intake 7 20 0 0.16 --
  Interventionb 52 27 42 0.22 0.36 (0.11 - 1.15)
  12-month follow-up 50 40 47 0.85 --
 Opiate and Cocaine-negative Urinalysis (%)
  Intake 0 7 0 0.37 --
  Interventionb 42 24 36 0.39 0.44 (0.13- 1.49)
  12-month follow-up 50 20 27 0.17 --
Workplace Outcomes
 Days Attended Workplace (%) 12 27 5 0.02‡ 6.45 (1.41-29.32)
HIV-related Risk Behaviors
 Injected drugs (%)
  Intake 100 100 92 0.29
  Intervention 54 74 49 0.14 0.49 (0.13- 1.30)
  12-month follow-up 40 69 44 0.31
 Shared needles (%)
  Intake 27 13 25 0.63
  Intervention 12 1 3 0.72 1.00 (0.07- 14.92)
  12-month follow-up 0 0 0 0.99
 Used condoms (%)
  Intake 6 13 0 0.35
  Intervention 7 18 1 0.06‡ 0.35 (0.10 - 1.26)
  12-month follow-up 0 23 0 0.09
Open in a new tab
a

= Missing samples have been interpolated

b

= Intervention data based on thrice-weekly testing

‡

= Significant difference between Work Only and No Voucher groups

Figure 1.

Figure 1

Open in a new tab

Opiate urinalysis results across consecutive urine samples for individual participants in each of the three experimental groups. Individual participant numbers are listed on the Y-axis. Horizontal lines represent opiate urinalysis results for individual participants across the consecutive scheduled urine collections of the study, ranked on the y-axis from most abstinence (top) to least abstinence (bottom). The filled squares in each line represent opiate-negative urinalysis results, the open squares represent opiate-positive urinalysis results, and the blank squares represent missing urine samples.

Figure 2.

Figure 2

Open in a new tab

Cocaine urinalysis results across consecutive urine samples for individual participants in each of the three experimental groups. Individual participant numbers are listed on the Y-axis. Horizontal lines represent cocaine urinalysis results for individual participants across the consecutive scheduled urine collections of the study. Participants are ranked on the y-axis in the same order as Figure 2 to enable a direct comparison of percent opiate and cocaine abstinence between Figures 2 and 3 for each participant. The filled squares in each line represent cocaine-negative urinalysis results, the open squares represent cocaine-positive urinalysis results, and the blank squares represent missing urine samples.

Urine samples collected at the 30-day assessment visits also revealed no significant effect of group assignment for either the missing-missing or missing-positive analyses. When missing-missing data were evaluated, there were no between-group effects on opiate, cocaine, or combined opiate/cocaine abstinence. When missing-positive analyses were conducted, there were no between-group effects on opiate, cocaine, or combined opiates/cocaine. There were no significant between-group differences regarding the rate of UA collection at the monthly follow-up assessments.

Workplace Performance Measures

Significant between-group effects were observed for attendance in the workplace (P=.04), which was driven by a significant difference between the No Voucher and Work Only groups (5% vs. 27% of days worked, respectively; P=0.02, OR=6.45, 95% CI 1.41-29.32; Table 2). The Abstinence & Work group did not vary significantly from either group (12% of days worked). Significant between-group effects were also observed for number of minutes worked per day (P=0.05), which was also driven by a significant difference between the No Voucher vs. Work Only groups (9.42 vs. 51.04 minutes per day, respectively; P=0.05); the Abstinence & Work group did not differ significantly from either group (24.35 minutes per day). The difference in total mean number of minutes worked (collapsed across days) closely approached but did not reach significance (F(2,43)=3.03, P=0.06). There were no significant differences between the Work Only and Abstinence & Work group in the total amount of incentive money earned ($1,375.90 vs. $805.19, respectively, P=0.34).

Treatment Retention

Figure 3 shows retention of participants in the three groups across the consecutive workdays of the study. A high rate of drop out was observed in the No Voucher group (93%) compared to the Work Only (67%) and Abstinence & Work (81%) groups. A survival analysis revealed this was significantly different (F(2,43)=3.97, P=0.03) and was driven by a difference between the No Voucher and Work Only groups (χ2(2)=8.65, P =0.003), though the difference between the Work Only and Abstinence & Work group approached significance (χ2(2)=3.42, P =0.06). As can be seen in Figure 3, the difference in retention between the three groups is evident almost immediately following randomization.

Figure 3.

Figure 3

Open in a new tab

Survival Graph of Workplace Attendance. Lines represent participant retention in treatment over the 26-week study. Groups are represented by black circles (Abstinence & Work Group), open squares (Work Only Group) and gray triangles (No Voucher Group).

HIV-related Risk Behaviors

Of particular importance was the impact this intervention might have on HIV-related risk behaviors among participants. As seen in Table 2, at intake participants reported engaging in high rates of HIV-related risk behaviors, including injection drug use (97.3% of participants), sharing needles (22.6%), and infrequent condom use (6%). No significant between-group changes in the frequency of drug injection, sharing needles, exchanging sex for drugs, or receiving money for sex were observed over the intervention period or the 12-month follow-up. Using condoms to have sex is the only item that approached statistical significance, with the Work Only group being more likely to endorse condom use at both the intervention (P=0.06) and 12-month follow-up assessments (P=0.09).

Discussion

The present study evaluated whether an employment-based abstinence reinforcement intervention could be used effectively to prevent relapse to drug abuse among recently-detoxified, HIV-positive injection drug users. Results did not support our primary hypothesis that contingencies placed on drug abstinence and workplace attendance (Abstinence & Work) would produce greater rates of abstinence than no abstinence contingencies. As seen in Table 2 and Figures 1 and 2, participants in this study who were randomly assigned to an Abstinence & Work contingency group were not significantly more likely to abstain from opiates and cocaine when compared to control participants, which is inconsistent with previous studies of employment-based reinforcement that have reported significant differences in drug abstinence between experimental and control groups (Donlin et al., 2008; Silverman et al., 2001; Silverman et al., 2007).

While the exact reason for the discrepancy between this and prior studies is not known, the current study differs from previous studies in several ways that may have contributed to these outcomes. The most prominent difference is that participants in this study had extremely poor attendance in the workplace and that attendance dropped almost immediately after the study began. For example, examination of Figure 3 shows the majority of participants stopped attending the workplace immediately following study enrollment. Attendance, especially in the early phase of this treatment, is crucial because it enables the workplace to begin functioning as a reinforcer, so that it may then be used to promote abstinence from drugs. The current data indicate the majority of participants did not have sufficient exposure to the workplace for it to begin functioning as a reinforcer, which likely undermined the entire intervention because participants did not value the workplace enough to abstain from opioids and cocaine. There are several differences between this study and previous studies that may have prevented the workplace from taking on the role of a reinforcer.

First, the preceding studies enrolled participants into a workplace induction phase that generally lasted 4 weeks and occurred prior to their randomization to an experimental group. During the induction phase participants were granted contingency-free access to the workplace and were able to earn incentives for productivity in the workroom, and only participants who attended the workplace on a regular basis were eligible for study randomization. As a result, the induction period serves as a screening mechanism to identify individuals for whom the workplace serves as a reinforcer before enrolling them into the larger study. The failure of the current study to produce the hypothesized effects on abstinence suggests that the induction phase may play an important role in the success of an employment-based abstinence reinforcement intervention because it ensures that only those participants who find the workplace reinforcing will proceed into the randomized trial. For example, a recent employment-based abstinence reinforcement study that successfully reinforced oral naltrexone maintenance among recently detoxified opioid and cocaine-dependent patients reported that 54% (84/155) of enrolled participants were never randomized because they did not show up to begin naltrexone therapy or did not complete the induction period (Dunn et al., 2013). This is consistent with the high and immediate rate of study attrition observed in the current study and suggests that lack of an induction phase may have prevented the study from accurately assessing whether an employment-based abstinence reinforcement intervention can promote abstinence among HIV-positive patients. The induction phase may also screen out individuals who are not stable in other ways and therefore may be unable to commit to the high frequency of visits required by this intervention approach. This is supported by a recent large-scale comparison of an intensive employment workshop to standard-care treatment in substance abusers, which hypothesized that the continued drug use observed among 52.8% of the sample may prevented significant between-group differences from being observed (Svikis et al., 2012).

Second, the fact that participants in the Abstinence & Work group were required to simultaneously abstain from opiates, cocaine, and alcohol immediately following study enrollment may have set the threshold for entering the workplace too high and further prevented the workplace from being established as a reinforcer for those participants. Previous contingency-based studies that require abstinence from multiple drugs have introduced those contingencies sequentially by requiring participants to achieve abstinence from their primary drug first (e.g., opiates) before introducing abstinence contingencies for secondary drugs (e.g., cocaine). This approach is supported by a meta-analysis that confirmed participants are less successful when they are required to abstain from multiple drugs simultaneously versus sequentially (e.g., Lussier et al. 2006), and a previous Therapeutic Workplace study that was able to successful promote abstinence from multiple drugs when those contingencies were introduced sequentially (Donlin et al., 2008). Examination of the data in the current study suggests that the abstinence contingencies may have prevented attendance in the Abstinence & Work group. For example, despite the fact that 73% of the Work Only participants in the current study attended the clinic to provide a urine sample during the first week of study enrollment, only 36% of those samples tested negative for both opiates and cocaine. In contrast, only 25% of participants in the Abstinence & Work group attended the clinic to provide a urine sample during the first week of study enrollment, however 60% of those samples were negative for opiates and cocaine (see Figures 1 and 2). These data indicate that Abstinence & Work participants were likely not attending the clinic unless they were abstinent from both drugs, which would have prevented them from having sufficient exposure to establish the workplace as a reinforcer.

Third, several characteristics of the detoxification procedure may have made increased the risk for relapse to opiates which would have further prevented participants from entering the workplace and establishing it as a reinforcer. For instance, all the preceding studies that have been conducted in the Therapeutic Workplace with opiate-dependent patients enrolled individuals who were receiving either methadone (Silverman et al., 2001; Silverman et al., 2002) or naltrexone (DeFulio et al., 2012; Dunn et al., 2013; Everly et al., 2011) to manage their opiate dependence. The successful results of these studies are consistent with a recent meta-analysis that reported opiate detoxification is most successful when a combination of pharmacological and psychosocial support is provided (Amato et al., 2011). Although participants in this study were provided access to naltrexone therapy for relapse prevention, naltrexone induction and maintenance was not required for study participation and the majority of participants declined naltrexone treatment. Further, research has also suggested that longer opioid detoxification durations produce better abstinence outcomes in opiate-dependent patients (Dunn et al., 2011). In contrast, participants in the current study completed a brief, 3-day detoxification that offered no additional pharmacological support to help manage the symptoms of withdrawal and/or prevent relapse, and research on the opioid withdrawal curve suggests these participants were likely still experiencing mild to moderate levels of opiate withdrawal at the time of study enrollment (Gossop, Green, Phillips, & Bradley, 1987; Gossop, Green, Phillips, & Bradley, 1989). These symptoms could have interfered with the study participation and contributed to a rapid relapse to drug use after being discharged from the inpatient detoxification program, which ultimately would have prevented Abstinence & Work participants from entering the workplace.

This study did detect a significant difference in workplace attendance between the No Voucher and Work Only groups, whereby participants who received monetary-based incentives to attend work (Work Only) worked for more days and for more minutes per day compared to the group who received no incentives for work (No Voucher). Work Only participants also displayed a different survival curve compared to Abstinence & Work and No Voucher participants. These results replicate similar effects of incentives in promoting attending in training observed in prior studies (Koffarnus et al., 2011; Silverman, Chutuape, Bigelow, & Stitzer, 1996; Wong, Dillon, Sylvest, & Silverman, 2004), and add to a growing body of literature that suggest incentives may be useful for promoting reliable workplace attendance even among patients who are continuing to engage in high rates of drug and/or alcohol abuse. These results have important implications for adult training programs because they provide empirical evidence that provision of incentives can improve attendance in an adult education program among chronically unemployed injection drug users

There was no effect of the employment-based abstinence reinforcement on self-reported HIV risk behaviors. Given that employment-based abstinence reinforcement did not affect drug use, there is really no reason to expect that it would reduce drug-related HIV risk behaviors. It is also possible that the low rate of risk behaviors reported at baseline (0% -27% of participants reporting engaging in >1 behavior), excluding injection drug use, resulted in a floor effect that made it difficult to establish any change in behavior from baseline. Nevertheless, it is important that additional research evaluate the role that employment-based abstinence reinforcement might have on reducing HIV risk behaviors.

Conclusion

In contrast to previous research, this employment-based abstinence reinforcement training program did not promote significant differences in drug abstinence among chronically unemployed injection drug users following opioid detoxification. Several features of the study design, including the lack of an induction period, setting the threshold for entering the workplace too high by requiring immediate abstinence from several drugs, and increasing the risk of relapse by providing a brief detoxification that was not supported by any continued pharmacological intervention, likely prevented the workplace from becoming established as a reinforcer that could be used to promote drug abstinence. These results provide evidence of how strategically reinforcement contingencies must be arranged in order to produce the intended therapeutic outcomes and suggest that opiate dependent patients may require additional pharmacological support to succeed in an employment-based abstinence reinforcement setting. Failure to establish the workplace as a reinforcer means that these data did not sufficiently address the research question, and that more research is needed to evaluate whether an employment-based abstinence intervention can successfully reduce drug use among HIV-positive substance abusers. Finally, a significant difference was observed in workplace attendance, which suggests that daily provision of employment incentives may be a promising method to encourage workplace attendance among chronically unemployed patients.

Acknowledgements

The project described was supported by Award Numbers R01DA012564 and T32DA07209 from the National Institute On Drug Abuse.

All authors provided substantive contributions to the manuscript and have read and approved the final manuscript.

This research was conducted while Dr’s Svikis and Wong were affiliated with Johns Hopkins University. Dr. Svikis is now at Virginia Commonwealth University and Dr. Wong is now at Eli Lilly Corporation. This study was also completed prior to the development of Clinicaltrials.gov and therefore was not registered on that site.

The authors would like to acknowledge the hard work of the numerous Center for Learning and Health and the Behavioral Pharmacology Research Unit employees who assisted in the administration of this study, without whom this research would not have been possible.

Footnotes

Disclosures

Paul A. Nuzzo has been paid as a statistical consultant / project coordinator for the NIDA Clinical Trials Network (CTN), NIDA Clinical Coordinating Center, and Yaupon Therapeutics, Inc. Dr. Wong is employed at Eli Lilly Corporation but has no additional conflicts of interest to report. The remaining authors report no conflicts of interest or financial disclosures.

References

  1. Amato L, Davoli M, Ferri M, Gowing L, Perucci CA. Effectiveness of interventions on opiate withdrawal treatment: An overview of systematic reviews. Drug and Alcohol Dependence. 2004;73:219–226. doi: 10.1016/j.drugalcdep.2003.11.002. [DOI] [PubMed] [Google Scholar]
  2. Baum MK, Rafie C, Lai S, Sales S, Page B, Campa A. Crack-cocaine use accelerates HIV disease progression in a cohort of HIV-positive drug users. Journal of Acquired Immune Deficiency Syndromes. 2009;50(1):93–99. doi: 10.1097/QAI.0b013e3181900129. [DOI] [PubMed] [Google Scholar]
  3. Beck AT, Beck RW. Screening depressed patients in family practice: A rapid technique. Postgraduate Medicine. 1972;52:81–85. doi: 10.1080/00325481.1972.11713319. [DOI] [PubMed] [Google Scholar]
  4. Birnbaum HG, White AG, Reynolds JL, Greenberg PE, Zhang M, Vallow S, Katz NP. Estimated costs of prescription opioid analgesic abuse in the United States in 2001: A societal perspective. Clinical Journal of Pain. 2006;22:667–676. doi: 10.1097/01.ajp.0000210915.80417.cf. [DOI] [PubMed] [Google Scholar]
  5. DeFulio A, Donlin WD, Wong CJ, Silverman K. Employment-based abstinence reinforcement as a maintenance intervention for the treatment of cocaine dependence: A randomized controlled trial. Addiction. 2009;104(9):1530–1538. doi: 10.1111/j.1360-0443.2009.02657.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. DeFulio A, Everly JJ, Leoutsakos JM, Umbricht A, Fingerhood M, Bigelow GE, Silverman K. Employment-based reinforcement of adherence to an FDA approved extended release formulation of naltrexone in opioid-dependent adults: A randomized controlled trial. Drug and Alcohol Dependence. 2012;120:48–54. doi: 10.1016/j.drugalcdep.2011.06.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Dillon EM, Wong CJ, Sylvest CE, Crone-Todd DR, Silverman K. Computer-based typing and keypad skills training outcomes of unemployed injection drug users in a therapeutic workplace. Substance Use and Misuse. 2004;39(13-14):2325–2353. doi: 10.1081/ja-200034620. [DOI] [PubMed] [Google Scholar]
  8. Donlin WD, Knealing TW, Needham M, Wong CJ, Silverman K. Attendance rates in a workplace predict subsequent outcome of employment-based reinforcement of cocaine abstinence in methadone patients. Journal of Applied Behavior Analysis. 2008;41(4):499–516. doi: 10.1901/jaba.2008.41-499. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Dunn KE, DeFulio A, Everly JJ, Donlin WD, Aklin WM, Nuzzo PA, Silverman K. Randomized trial of employment-based reinforcement of adherence to oral naltrexone treatment in unemployed injection drug users. Experimental and Clinical Psychopharmacology. 2013;21(1):74–83. doi: 10.1037/a0030743. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Dunn KE, Sigmon SC, Strain E, Heil SH, Higgins ST. The association between outpatient buprenorphine detoxification duration and clinical treatment outcomes: A review. Drug and Alcohol Dependence. 2011;119(1-2):1–9. doi: 10.1016/j.drugalcdep.2011.05.033. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Everly JJ, DeFulio A, Koffarnus MN, Leoutsakos JM, Donlin WD, Aklin WM, Silverman K. Employment-based reinforcement of adherence to depot naltrexone in unemployed opioid-dependent adults: A randomized controlled trial. Addiction. 2011;106:1309–1318. doi: 10.1111/j.1360-0443.2011.03400.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Han B, Gfroerer JC, Colliver JD. Associations between duration of illicit drug use and health conditions: Results from the 2005-2007 National surveys on Drug Use and Health. Annals of Epidemiology. 2010;20:289–297. doi: 10.1016/j.annepidem.2010.01.003. [DOI] [PubMed] [Google Scholar]
  13. Higgins ST, Delaney DD, Budney AJ, Bickel WK, Hughes JR, Foerg F, Fenwick JW. A behavioral approach to achieving initial cocaine abstinence. American Journal of Psychiatry. 1991;148(9):1218–1224. doi: 10.1176/ajp.148.9.1218. [DOI] [PubMed] [Google Scholar]
  14. Higgins ST, Silverman K, Heil SH, editors. Contingency Management in Substance Abuse Treatment. Guilford Press; New York, NY: 2008. [Google Scholar]
  15. Hser YI, Evans E, Huang D, Brecht ML, Li L. Comparing the dynamic course of heroin, cocaine and methamphetamine use over 10 years. Addictive Behavior. 2008;33(1-2):1581–1589. doi: 10.1016/j.addbeh.2008.07.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Hudziak JJ, Helzer JE, Wetzel MW, Kessel KB, McGee B, Janca A, Przybeck T. The use of the DSM-III-R checklist for initial diagnostic assessments. Comprehensive Psychiatry. 1993;34(6):375–383. doi: 10.1016/0010-440x(93)90061-8. [DOI] [PubMed] [Google Scholar]
  17. Gossop M, Green L, Phillips G, Bradley B. What happens to opiate addicts immediately after treatment: A prospective follow-up study. British Medical Journal. 1987;294:1377–1380. doi: 10.1136/bmj.294.6584.1377. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Gossop M, Green L, Phillips G, Bradley B. Lapse, relapse, and survival among opiate addicts after treatment. British Journal of Psychiatry. 1989;154:348–353. doi: 10.1192/bjp.154.3.348. [DOI] [PubMed] [Google Scholar]
  19. Kleber HD. Pharmacologic treatments for opioid dependence: detoxification and maintenance options. Dialogues in Clinical Neuroscience. 2007;9(4):455–470. doi: 10.31887/DCNS.2007.9.2/hkleber. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Koffarnus MN, Wong CJ, Diemer K, Needham M, Hampton J, Fingerhood M, Silverman K. A randomized clinical trial of a Therapeutic Workplace for chronically unemployed, homeless, alcohol-dependent adults. Alcohol and Alcoholism. 2011;46(5):561–569. doi: 10.1093/alcalc/agr057. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Lucas GM, Griswold M, Gebo KA, Keruly J, Chaisson RE, Moore RD. Illicit drug use and HIV-1 disease progression: a longitudinal study in the era of the highly active antiretroviral therapy. American Journal of Epidemiology. 2006;163(5):412–420. doi: 10.1093/aje/kwj059. [DOI] [PubMed] [Google Scholar]
  22. Lussier JP, Heil SH, Mongeon JA, Badger GJ, Higgins ST. A meta-analysis of voucher-based reinforcement therapy for substance use disorders. Addiction. 2006;101(2):192–203. doi: 10.1111/j.1360-0443.2006.01311.x. [DOI] [PubMed] [Google Scholar]
  23. Martin WR, Jasinski DR, Mansky PA. Naltrexone, an antagonist for the treatment of heroin dependence. Effects in man. Archives of General Psychiatry. 1973;28(6):784–791. doi: 10.1001/archpsyc.1973.01750360022003. [DOI] [PubMed] [Google Scholar]
  24. McLellan AT, Lewis DC, O’Brien CP, Kleber HD. Drug dependence, a chronic medical illness. Implications for treatment, insurance and outcomes evaluation. Journal of the American Medical Association. 2000;284(13):1689–1695. doi: 10.1001/jama.284.13.1689. [DOI] [PubMed] [Google Scholar]
  25. McLellan AT, Luborsky L, Cacciola J, Griffith J, Evans F, O’Brien CP. New data from the Addiction Severity Index, Reliability and validity in three centers. Journal of Nervous and Mental Disease. 1985;173:412–423. doi: 10.1097/00005053-198507000-00005. [DOI] [PubMed] [Google Scholar]
  26. Mello NK, Mendelson JH, Kuehnle JC, Sellers MS. Operant analysis of human heroin self-administration and the effects of naltrexone. Journal of Pharmacology and Experimental Therapeutics. 1981;216(1):45–54. [PubMed] [Google Scholar]
  27. Metsch LR, Pereyra M, Purcell DW, Latkin CA, Malow R, Gomez CA, INSPIRE Study Team Correlates of lending needles/syringes among HIV-seropositive injection drug users. Journal of Acquired Immune Deficiency Syndrome. 2007;46(Suppl. 1):S72–S79. doi: 10.1097/QAI.0b013e3181576818. [DOI] [PubMed] [Google Scholar]
  28. Murphy D, Brecht M-L, Herbeck D, Evans E, Huang D, Hser Y-I. Longitudinal HIV risk behavior among the Drug Abuse Treatment Outcome Studies (DATOS) adult sample. Evaluation Review. 2008;32(1):83–112. doi: 10.1177/0193841X07307411. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Roux P, Carrieri P, Cohen J, Ravaux I, Poizot-Martin I, Dellamonica P, Spire B. Retention in opioid substitution treatment: A major predictor of long-term virological success for HIV -infected injection drug users receiving antiretroviral treatment. Clinical Infectious Diseases. 2009;49:1433–1440. doi: 10.1086/630209. [DOI] [PubMed] [Google Scholar]
  30. Silverman K, Chutuape MA, Bigelow GE, Stitzer ML. Voucher-based reinforcement of attendance by unemployed methadone patients in a job skills training program. Drug and Alcohol Dependence. 1996;41(3):197–207. doi: 10.1016/0376-8716(96)01252-5. [DOI] [PubMed] [Google Scholar]
  31. Silverman K, Kaminski BJ, Higgins ST, Brady JV. Behavior analysis and treatment of drug addiction. In: Fisher WW, Piazza CC, Roane HS, editors. Handbook of Applied Behavior Analysis. The Guilford Press; New York, NY: 2011. pp. 451–471. [Google Scholar]
  32. Silverman K, Svikis D, Robles E, Stitzer ML, Bigelow GE. A reinforcement-based therapeutic workplace for the treatment of drug abuse: Six-month abstinence outcomes. Experimental and Clinical Psychopharmacology. 2001;9(1):14–23. doi: 10.1037/1064-1297.9.1.14. [DOI] [PubMed] [Google Scholar]
  33. Silverman K, Svikis D, Wong CJ, Hampton J, Stitzer ML, Bigelow GE. A reinforcement-based therapeutic workplace for the treatment of drug abuse: Three-year abstinence outcomes. Experimental and Clinical Psychopharmacology. 2002;10(3):228–240. doi: 10.1037//1064-1297.10.3.228. [DOI] [PubMed] [Google Scholar]
  34. Silverman K, Wong CJ, Grabinski MJ, Hampton J, Sylvest CE, Dillon EM, Wentland RD. A web-based Therapeutic Workplace for the treatment of drug addiction and chronic unemployment. Behavior Modification. 2005;29(2):417–463. doi: 10.1177/0145445504272600. [DOI] [PubMed] [Google Scholar]
  35. Silverman K, Wong CJ, Needham M, Diemer K, Knealing T, Crone-Todd D, Kolodner K. A randomized trial of employment-based reinforcement of cocaine abstinence in injection drug users. Journal of Applied Behavior Analysis. 2007;40(3):387–410. doi: 10.1901/jaba.2007.40-387. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Smyth BP, Barry J, Keenan E, Ducray K. Lapse and relapse following inpatient treatment of opiate dependence. Irish Medical Journal. 2010;103(6):176–179. [PubMed] [Google Scholar]
  37. Strassels SA. Economic burden of prescription opioid misuse and abuse. Journal of managed care pharmacy JMCP. 2009;15(7):556–562. doi: 10.18553/jmcp.2009.15.7.556. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Substance Abuse and Mental Health Services Administration (SAMHSA), Office of Applied Studies Treatment episode data set admissions (TEDS-A) concatenated, 1992 to present. 2010 [Google Scholar]
  39. Svikis DS, Keyser-Marcus L, Stitzer M, Rieckmann T, Safford L, Loeb P, Zweben J. Randomized multi-site trial of the Job Seeker’s Workshop in patients with substance use disorders. Drug and Alcohol Dependence. 2012;120(1-3):55–64. doi: 10.1016/j.drugalcdep.2011.06.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Verebey K, Volavka J, Mule SJ, Resnick RB. Naltrexone: disposition, metabolism, and effects after acute and chronic dosing. Clinical pharmacology and therapeutics. 1976;20(3):315–328. doi: 10.1002/cpt1976203315. [DOI] [PubMed] [Google Scholar]
  41. Webber MP, Schoenbaum EE, Gourevitch MN, Buono D, Klein RS. A prospective study of HIV disease progression in female and male drug users. AIDS. 1999;13(2):257–262. doi: 10.1097/00002030-199902040-00014. [DOI] [PubMed] [Google Scholar]
  42. Wong CJ, Dillon EM, Sylvest CE, Silverman KE. Contingency management of reliable attendance of chronically unemployed substance abusers in a therapeutic workplace. Experimental and Clinical Psychopharmacology. 2004;12(1):39–46. doi: 10.1037/1064-1297.12.1.39. [DOI] [PubMed] [Google Scholar]
  43. Zeger SL, Liang KY, Albert PS. Models for longitudinal data: A generalized estimating equation approach. Biometrics. 1988;44:1049–1060. [PubMed] [Google Scholar]

RESOURCES