Prize-based Contingency Management for the Treatment of Substance Abusers: A Meta-analysis

Abstract

Aim

To review randomized controlled trials to assess efficacy of a prize-based contingency management procedure in reducing substance use (where a drug-free breath or urine sample provides a chance of winning a prize).

Methods

A meta-analysis was conducted on articles published from January 2000 to February 2013 to determine the effect size of studies comparing prize-based contingency management to a treatment-as-usual control condition (k=19 studies). Parallel analyses evaluated the efficacy of both short- (k=9 studies) and long-term outcomes (k=6 studies) of prize-based contingency management .

Results

The average end-of-treatment effect size (Cohen's d) was .46 [95% CI=0.37,0.54). This effect size decreased at the short-term (≤ 3-month) post-intervention follow-up to .33 (95% CI=0.12,0.54) and at the 6-month follow-up time point there was no detectable effect (d=-.09 (95% CI=−0.28,0.10)).

Conclusion

Adding prize-based contingency management to behavioral support for substance use disorders can increase short-term abstinence but the effect does not appear to persist to 6 months.

1. INTRODUCTION

Contingency management (CM) is among the most empirically-supported strategies for increasing drug abstinence [1]. Using principles of operant conditioning, CM protocols arrange for the regular monitoring of a person's drug use (e.g., regular urinalysis) and delivery of an incentive only after verification of drug abstinence (e.g., providing a drug-free urine sample) has occurred [2]. If the incentive functions as a positive reinforcer, it competes with the reinforcing effects of the drug and increases the likelihood that drug abstinence is initiated and maintained [3]. CM's efficacy has been demonstrated in a wide range of substance-using populations, including alcohol-, cannabis-, cocaine-, and opiate-dependent outpatient clients, as well as cigarette smokers. To date, systematic reviews [4-6] and meta-analyses [7-11] have consistently supported the efficacy of various CM approaches.

Perhaps the best known CM procedures for drug abstinence are voucher-based reinforcement therapy (VBRT) and the prize-based (PB) contingency management procedure. In the VBRT paradigm, the desired behavior is rewarded by presenting a voucher exchangeable for goods and services each time it is performed, whereas there is only a chance that the desired behavior will be rewarded in the probabilistic prize-based procedure. In the PB procedure, individuals earn draws from a prize bowl every time the target behavior is exhibited (e.g., providing a drug-free urine sample). The prize bowl typically contains slips of paper or chits labeled “Good Job,” “small,” “large,” or “jumbo”. Approximately half of the slips are labeled “Good Job” and have no monetary value. The majority of the remaining slips are labeled “small” and indicate a low-value prize (e.g., $1). The bowl usually contains one “jumbo” slip indicating a very high value prize (e.g., $100), and the remainder of the slips are labeled “large” indicating a prize of moderate value (e.g., $20). VBRT and PB procedures are similar in several ways: (1) both typically follow escalating schedules of reinforcement (e.g., increased voucher value in VBRT and increased number of draws in PB), and (2) both have employed resets to the starting level when a person fails to perform the behavior and opportunities to return to the prior level of earnings if the behavior is again exhibited a predetermined number of times.

Both VBRT and PB procedures have shown good efficacy in clinical trials, with extensive empirical evidence that VBRT is efficacious for establishing abstinence from a variety of substances (e.g., cocaine, alcohol, opiates, marijuana, nicotine) and with a wide range of substance abusing populations [9,12-18] and that the PB procedure is efficacious in improving drug use outcomes [19-23] when tested in community-based treatment settings [24-27] including residential programs [28], intensive outpatient drug-free programs [29,30], methadone clinics [31], and group treatment settings [32]. Studies also indicate that the PB procedure's efficacy is not related to income [33], ethnicity [34], or gambling history [35].

Importantly, meta-analytic results provide a less biased, more representative estimate of the experimental condition than is obtained when simply reviewing individual study results using a vote-counting procedure [36]. Meta-analyses have greater statistical power because they are based on aggregated data from multiple studies that examine the same dependent variable, and the results are more generalizable since they are derived from numerous studies (rather than a single study; [37]). Confidence intervals associated with effect sizes suggest the magnitude of the treatment effect [38].

Evaluating the efficacy of the short- and long-term effects of the PB procedure on substance use through meta-analysis is critical given its growing dissemination and adoption in a variety of therapeutic contexts. Several meta-analyses of CM procedures have been conducted to date. However, none have exclusively examined the PB procedure or have established its efficacy independent of VBRT. Focused analyses could provide more specific information about the efficacy and clinical utility of the PB procedure, and there now appears to be a sufficient number of published studies to conduct a separate meta-analysis of this procedure. Furthermore, with one exception [10], previous meta-analyses have focused on end-of-treatment results, but not post treatment follow-up outcomes. As such, the current study involves a meta-analysis of the PB procedure and examines its overall efficacy compared to treatment-as-usual not only with respect to reducing substance use at end-of-treatment, but also at short (i.e., ≤3 months) and 6-month follow-up assessments.

2. METHODS

2.1 Article Selection

The literature search procedures were guided by the Cochrane Collaborative suggestions for conducting systematic reviews [39].

2.1.1 Stage 1: Initial systematic review of the literature and article exclusion process

We conducted a systematic review of PB articles published between January 2000 (the year the first PB study with a substance-abusing population was published [19]) and February 8, 2013. The search involved six electronic bibliographic data bases (i.e., CENTRAL, EMBASE, PsycINFO, PubMedPlus, Social Services Abstracts, Web of Science) and used the following combination of search terms: (contingency management, prize*, or voucher*) and (alcohol*, cocaine, opiate*,marijuana, nicotine, drug*, substance* or treatment*) which were applied to title, abstract, descriptor, topic, and/or keyword search fields. We restricted our search to studies based on human participants, written in English, and published in peer-reviewed journals when possible.

Electronic data base search results were merged into a single Endnote™ file (n=2,390 articles); duplicates were deleted, yielding a total of 1,156 unique articles (see Figure 1). Documents were then deleted if they met any of the following exclusion criteria: (1) involved non-human participants, (2) not published in English, (3) not an addictions article, and (4) not utilizing a PB procedure, resulting in a total of 76 PB articles.

Figure 1.

Flow Diagram of Study Retrieval Process

2.1.2 Stage 2: Identification of additional articles

Four methods were used to identify additional articles for inclusion in the meta-analysis. First, a second set of literature searches using the six data bases was conducted to identify relevant meta-analyses and systematic reviews [i.e., search terms: (meta* or review) paired with the substance-related terms utilized in Stage 1 were applied to the title and abstract fields]; the Cochrane Data Base of Systematic Reviews was also searched. Reference sections of these publications were reviewed. Second, reference sections of the 76 PB articles were reviewed. Third, first authors of these 76 PB articles were contacted via email and asked to identify additional PB articles that we had not identified during our search process. Fourth, we searched gray literature sources (i.e., REPORTer, clinicaltrials.gov) using the terms “contingency management” or “prize* or voucher*” to identify clinical trials using a PB procedure with substance abusers. This 4-step process identified five additional PB articles, resulting in 81 PB articles.

2.1.3 Stage 3: Prize bowl study exclusion screening process

Two authors applied more specific exclusion criteria to these 81 PB studies. These criteria included: (1) PB not implemented with a substance abusing population (e.g., implemented with a parent of an adolescent substance abuser), (2) PB not used as an independent variable, (3) biological indicators not used to assess illicit substance use, (4) PB confounded with a co-intervention (e.g., PB combined with a network support intervention), (5) VBRT data combined with PB data, (6) prior article published on same data set, (7) article evaluated PB cost-effectiveness only, (8) PB intervention not applied to behavior of individuals (i.e., group contingencies), or (9) no random assignment to treatment conditions that included a control comparison. Discrepancy meetings were held with a third author to discuss ‘difficult to categorize’ articles and make final inclusion/exclusion decisions. This selection process resulted in a total of 18 study-eligible PB articles [19-26,40-49].

2.1.4 Stage 4: Availability of required data

These 18 PB articles were reviewed to determine if they contained the data necessary to calculate effect sizes for the meta-analysis. Lead authors for two of the articles were contacted via email and asked to provide missing data; one responded to this request. We were able to include partial data from the non-responding author's study (i.e., useable data was available on one of the study's three PB conditions). The overall data collection process resulted in 19 independent end-of-treatment effect size estimates (i.e., one article contained two independent PB studies, [25]), nine short-term follow-up effect size estimates, and six 6-month follow-up effect size estimates.

2.2 Data Coding Procedures and Data Extraction

2.2.1 Variables coded

We coded each article based on three types of variables (Table 1). Study context and methodology variables included sample size, setting, population, control comparison condition, and assessment time window. Prize-based contingency management variables included duration of the PB intervention and expected mean maximum earnings. Outcome variables were selected if they were congruent with the PB intervention (i.e., if the PB intervention focused on cocaine use then cocaine use was the outcome variable) and measured as continuous variables.

Table 1.

Studies Targeting Abstinence Where the Prize Bowl Effect Could Be Isolated

Prize Bowl Target Behavior & Study	Quality Score	n	Setting	Population	Control Comparison	Length of Intervention (wks)	Expected Mean Maximum Earningsx	Assessment Time Window	Dependent Variable for Each Time Window	Effect Size (d)	SE	CI (95%)
												Lower	Upper
Alcohol, Cocaine/Stimulants, Opiates, & Marijuana
Petry et al. (2012a)	14	43	DF IOP	Alcohol, cocaine, opiate, or marijuana abusing or dependent adults	TAU - group counseling w tapered care, biological samples collected 2/wk by therapist	12	$380x	1	LDA (weeks)	.64	.31	.03	1.26
Killeen et al. (2012)	13	31	DF IOP and DF OP	Marijuana abusing or dependent adolescents	TAU - primarily group counseling w option for individual and family counseling; 2 PB draws for providing biological sample	10	$626x	1	LDA (samples)	.14	.36	−.56	.85
Alcohol, Cocaine, & Opiates
Petry et al. (2004)	14	120	DF IOP with step-down	Cocaine using adults (85% diagnosed with cocaine dependence); no diagnosis of opiate dependence	TAU - group counseling w tapered care; biological samples collected 1-4x/mo by clinic	12	$80 condition $240 condition	1	LDA (samples)	.35	.24	−.12	.81
Petry et al. (2005a)	17	89	DF OP	Cocaine or opioid dependent adults	TAU - group & individual counseling w tapered care; psycho-ed to control for individualized attention	12	$543x	1 3 4	LDA (weeks) % negative samples % negative samples	.81 −.04 −.01	.22 .25 .27	.37 −.53 −.55	1.24 .45 .52
Petry et al. (2006)	17	84	DF OP	Cocaine or heroin using, abusing, or dependent adults	TAU - group counseling w tapered care; psycho-ed to control for individualized attention	12	$466x	1 3 4	LDA (weeks) % negative samples % negative samples	.53 .18 −.54	.22 .32 .37	.09 −.44 −1.25	.97 .80 .18
Petry et al. (2010)	17	170	HIV	HIV+ cocaine or opiate abusing or dependent adults	TAU - group counseling w 12-Step Facilitation	24	$330	1 4	LDA (samples) % negative samples	.26 .08	.15 .2	−.0442 −.3087	.560 2 .459 9
Petry et al. (2012b)	17	Cocaine + Arm=109	DF IOP	Cocaine dependent adults	TAU- group counseling w tapered care	12	Cocaine + Arm=$250 Cocaine + Arm=$560	1 4	LDA (weeks) % negative samples	.53 .11	.24 .28	.05 −.44	1.01 .66
		Cocaine – Arm=226				12	Cocaine – Arm=$250	1 4	LDA (weeks) % negative samples	.53 −.15	.14 .16	.27 −.47	.80 .16
Alcohol & Stimulants
Petry et al. (2005c)	15	415	DF OP	Stimulant using adults (84.4% diagnosed with abuse or dependence)	TAU – group counseling w option for family & individual counseling	12	$420	1	LDA (samples)	.42	.10	.22	.61
Roll et al. (2006)	15	113	DF OP	Meth-amphetamine abusing or dependent adults	TAU – Individual, group, family counseling;	12	$420	1 3	LDA (samples) % negative samples	.22 .30	.19 .19	−.15 −.09	.59 .67
Peirce et al. (2006)	17	388	M	Stimulant using adults	TAU – individual & group counseling	12	$420	1 3	LDA (samples) % negative samples	.51 .04	.10 .13	.31 −.22	.72 .30
Tracy et al. (2007)	15	30	HS	Alcohol or cocaine abusing or dependent adults	Assessment only; no counseling	4	$41	1	% negative samples	1.53	.64	.28	2.78
Cocaine & Opiates
Petry & Martin (2002)	17	42	M	Cocaine abusing or dependent adults	TAU – individual counseling	12	$478x	1 3	LDA (weeks) % negative samples	.53 .68	.32 .33	−.10 .04	1.16 1.32
Ghitza et al. (2007)^	14	60	M	Cocaine using and heroin dependent adults; no diagnosis of alcohol or sedative dependence	TAU – individual counseling; non-contingent (yoked) prize control	12	$1,391	1 2	Mean % negative samples Mean % negative samples	.67 1.02	.28 .36	.12 .33	1.22 1.72
Alcohol
Petry et al. (2000)	14	42	DF IOP	Alcohol dependent male veterans	TAU – group counseling w tapered care; psycho-ed to control for individualized attention	8	$156x	1	% negative samples	.67	.36	−.04	1.38
Opiates
Hser et al. (2011)	14.5	319	M	Opiate dependent adults	TAU – no counseling	12	$9x,~	1	LDA (weeks)	.23	.11	.01	.45
Cocaine/Stimulants
Petry et al. (2005b)	15	77	M	Cocaine dependent adults	TAU – group counseling; 1 PB draw for providing biological sample	12	$497x	1 3	LDA (weeks) % negative samples	.42 .68	.23 .32	−.04 .05	.88 1.31
Petry et al. (2007)	17	47	M	Cocaine dependent adults	TAU – group and/or individual counseling	12	$300	1 3 4	LDA (weeks) % negative samples % negative samples	.73 .58 −.33	.31 .34 .34	.13 −.09 −1.01	1.34 1.26 .35
McDonell et al. (2013)	14	176	OP/SUD/CMH	Stimulant dependent adults with SMI	TAU – group counseling and case management	12	$588 x	1 3	Days abstinent Days abstinent	.67 .29	.16 .19	.36 −.08	.98 .67

Note. n = sample size at baseline for control and PB groups combined, may not represent the n used to calculate the effect sizes. Setting = setting in which study occurred: M = Methadone clinic; DF = Drug-Free clinic; IOP = Intensive Outpatient Program; HIV = HIV drop-in center that provided substance abuse treatment; HS = Homeless Shelter; SUD/CMH=combined substance abuse and community mental health program; SMI=seriously mentally ill.

^xif value was not reported in article, it was calculated based on PB earning probabilities stated in article with equations provided by Olmstead & Petry (2009).

^~Amount based on conversion from Yen to American dollars. Assessment Time Window: 1 = End-of treatment assessment/ EOT; 2 = 2-mos post EOT; 3 = 3-mos post-EOT; 4 = 6-mos post-EOT. LDA = Longest duration of abstinence. Bold entry = study that contained two PB conditions; the weighted average ES is shown in table.

^{^}Data available from author for 1 of 3 PB conditions.

2.2.2 Data extraction

All data necessary to calculate effect sizes was extracted from the PB articles and then independently validated by a co-author. Discrepancies were discussed and, if necessary, resolved via discussions with a third co-author.

2.3 Methodological Rigor

A slightly modified Method Guidelines for Systematic Reviews of the Cochrane Back Review Group [50,51] was used to assess the methodological rigor of each study. Eighteen dichotomous items (0=no/not able to determine; 1=yes) were used to evaluate the study's participant selection procedures, interventions, outcome measurement, and statistical methodology. Total scores could range from 0 to 18 with scores greater than 13 (i.e., ≥75% of the criteria were met), indicating a methodologically sound study.

Two bachelor's-level research assistants completed a 4-hour training consisting of (1) a basic overview of meta-analysis, criteria associated with methodological rigor (e.g., random assignment to treatment, intent-to-treat design), and inter-coder agreement, (2) a review of the operational definitions associated with each item contained in our measure, and (3) practice scoring the measure with one PB publication not included in the meta-analysis. The coders then independently rated four additional PB studies that were not included in this study. They obtained an 80% absolute agreement rate, the criterion used to qualify them to code study articles. Discrepancies between coders were resolved via discussions with the first author or, if necessary, a co-author. The percent absolute inter-coder agreement for the PB articles was 0.90.

2.4 Effect Size Calculation

Cohen's d was used as our measure of effect size [52] and, with the exception of effect sizes associated with Mann-Whitney U analyses, calculations and conversions were completed using Wilson's effect size calculator (http://gemini.gmu.edu/cebcp/EffectSizeCalculator/index.html). Following the recommendations of DeCoster [53], effect sizes based on Mann-Whitney U values were calculated by converting the U test statistic to a point-biserial r and then to a biserial r. The d was then calculated from the biserial r. Small, medium, and large effect sizes were defined as .20, .50, and .80, respectively [52]. Effect sizes received a positive valence when the PB condition showed more favorable outcomes than the control condition.

2.5 Analyses

Random effects models were used for all analyses (i.e., end-of-treatment/EOT, short-term follow-up, and 6-month follow-up outcomes) given that the effect size estimates were assumed to be heterogeneous [54,55] and the meta-analyses were based on a small number of effect sizes [54]. Each effect size was weighted by the inverse of its variance [56]. The degree of heterogeneity between the effect sizes was examined using the I² statistic and its 95% confidence interval, a less biased indicator of heterogeneity than the Q statistic [56,57]. I² values of 50% or higher indicate considerable heterogeneity among the effect sizes [57]. Statistical analyses were conducted in SPSS Version 19 using Wilson's macro (http://mason.gmu.edu/~dwilsonb/ma.html). We did not conduct sub-group (i.e., moderator) analyses given the small number of effect sizes in our sample [58].

We chose outcome variables that were continuous in nature (e.g., longest duration of abstinence, percent of patients with negative samples) for the purpose of this analysis. When possible, effect sizes from analyses based on the assumption that missing biological samples were positive for the targeted substance were used. When a study contained two PB conditions [25,26], a single mean weighted effect size was calculated to satisfy the assumption that all effect sizes used in the meta-analysis were based on independent data points. The primary meta-analysis is based on data collected at the end-of-treatment; secondary analyses were conducted on effect sizes associated with short-term (i.e., ≤3 months) and 6-month follow-up data.

2.6 Publication Bias

Following the methodology used in prior CM meta-analyses [8-10], we also examined the extent to which publication bias may have inflated our overall EOT effect size (i.e., “file drawer problem” [59]). This was important given that only published studies were included in the meta-analysis and journals’ tendencies to more frequently publish articles that report statistically significant findings. As a result, meta-analyses may overestimate the overall effect of the PB intervention. A minimum of 10 studies is necessary to test for publication bias due to the decreased power when testing for funnel plot asymmetry with a small number of studies [39]. As such, publication bias was assessed using Begg and Mazumdar's [60] rank order correlation test for the EOT effect sizes. The small number of effect sizes for the short- (k=9 studies) and 6-month follow-up outcomes (k=6 studies) precluded such analyses.

3. RESULTS

3.1 Study Characteristics

Eighteen studies met the inclusion criteria for this meta-analysis and yielded a total of 19 EOT effect sizes. Only one study was conducted outside of the United States [45]. These studies included 1,235 control participants and 1,346 PB participants.

The average sample size of the studies was 136 (range=30-415); 53% and 32% of these studies were conducted in drug-free treatment programs or methadone programs, respectively, whereas 15% were located at an HIV drop-in center, a homeless shelter, or an integrated SUD/mental health program. Fifteen of the studies (79%) employed a PB procedure which lasted 12 weeks (range=4-24 weeks), and the mean average maximum PB earnings was $402 (range=$9-$1,391). The average methodological rigor score for these studies was 15.4 (range=13-17; see Table 1, Quality Score), indicating that, on average, 86% of the rigor criteria were met. This value was above the 75% criteria used to define a methodologically sound study.

3.2 End-of-Treatment Analyses

Table 1 contains the EOT, short-, and 6-month follow-up effect sizes, standard errors, and 95% confidence intervals for the 19 control PB comparisons; Figures 2, 3, and 4 provide forest plots of the EOT, short-, and long-term overall effects sizes, study-specific effect sizes, and confidence intervals, respectively. We identified one non-significant, six small, ten medium, and two large EOT effect sizes. The random-effects weighted mean effect size of these 19 comparisons was d=0.46 (p<0.001, 95%; CI=0.37,0.54) which represents a moderate effect. Based on the I² index, the effect sizes had a low degree of heterogeneity (I²=3.69%; 95% CI=0%,50.82%).

Figure 2.

End-of-Treatment Effect Sizes (d) with 95% Confidence Intervals

Figure 3.

Short-term Follow-up Effect Sizes (d) with 95% Confidence Intervals

Figure 4.

6-month Follow-up Effect Sizes (d) with 95% Confidence Intervals

3.3 Follow-up Analyses

3.3.1 Short-term follow-up

The nine studies that included short-term follow-up assessments (Table 1; Figure 3) identified two small, three medium, and one large effect size; non-significant effects were identified for three of the studies. The weighted mean effect size of associated with these studies was d=0.33 (p=0.002; 95% CI=0.12,0.54) indicating a small to medium effect. The I² index indicated a low degree of heterogeneity (I²= 39.35%; CI=0%,72.07%).

3.3.2 6-month follow-up

Four of the six 6-month outcome effect sizes showed non-significant effects. There was one small and one moderate effect, both in favor of the control condition; however, the test statistics associated with these effects were not significant (Table 1; Figure 4). The weighted mean effect size for these studies was d=−0.09 (p=0.351; 95% CI=−0.28,0.10), indicating a non-significant effect. Again, the I² index suggested the presence of no heterogeneity (I²≤ 0%; 95% CI=0%,62.9%).

The combined results for the two follow-up meta-analyses suggest a decrease in the effect size over time. In comparison to the EOT effect size (d=0.46), the short- and 6-month follow-up effect sizes were 0.33 and -0.09, respectively. The 95% confidence interval indicates that the long-term abstinence rate effect size was not significantly different from zero.

3.4 Publication Bias

The Begg rank order correlation coefficient did not identify significant publication bias in the EOT meta-analysis (Z=1.48,p=0.139).

4. DISCUSSION

4.1 End-of-Treatment Results

As expected, the findings from this meta-analysis indicate that, when compared to treatment-as-usual, the prize-based contingency management approach increased abstinence during the time that the intervention was being implemented. The overall moderate effect size of .46 at the end of treatment suggests that the PB procedure has a meaningful impact on substance use. Twelve of the 19 effect sizes were medium to large in nature, and the overall effect size was in the moderate range.

We examined the results of this meta-analysis in the context of results from four other meta-analyses of CM with substance-abusing populations [7-10]. With the exception of Lussier et al. ([9]; d=.68), the overall EOT PB effect size of .46 (CI: 0.27,0.54) reported in this study falls within the range of those found in prior CM meta-analyses conducted on data derived from substance-abusing populations (i.e., Dutra [7] d=.58; Griffith [8] d=.52; Prendergast [10] d=.40). Caution should be exercised in making such comparisons, as the previous meta-analyses included several different types of CM procedures, and three of them included a small number of PB studies (Dutra-1 of 14 studies; Lussier-3 of 30 studies; Prendergast-2 of 47 studies) that were also included in the current meta-analysis. While this overlap may have a slight effect on the results of our comparisons, this bias is likely minimal given the small number of redundant studies and, as such, the differences in effect sizes are worth further comment.

The effect size differences are particularly interesting because the .46 effect size for the PB procedure found in this study is substantially lower than the .68 effect reported by Lussier at al. in the meta-analysis that had a greater focus on VBRT [9]. A moderator analysis by Lussier et al. showed that reinforcer magnitude was positively correlated with effect size, thus their larger effect would be predicted by the preponderance of VBRT studies that employed higher maximum earnings [9]. This direct relationship between cost and effect size has been demonstrated even within the PB procedure [26].

4.2 Follow-up Results

As is the case for most treatments for substance abuse and dependence [61], the effects were not maintained after the intervention ended. Specifically, the effect size decreased from .46 at the end of treatment to .33 at the time of the short-term follow-up and was no longer present at the 6-month follow-up (i.e., −.09). Importantly, the fact that there was a small to medium effect of the PB procedure for the shorter follow-up assessment suggests that the effects of PB do not stop as soon as the contingencies are removed. However, declines in effectiveness shortly after an intervention ends has been widely reported across most behavioral treatments including the CM literature [10]. Although strategies for maintaining the effects of CM interventions have been considered for many years in areas outside of addictions [62,63], the importance of continuing care in addiction treatment has been emphasized only recently, and methods of effective continuing care have only recently been explored [64-66]. Further research is needed to examine continuing care strategies related to CM and other behavioral interventions.

4.3 Limitations

There are three notable limitations associated with this study. First, the small number of studies that met our study inclusion criteria did not afford us adequate power to look for potential end-of-treatment and follow-up moderator variables which may have provided additional information on ways to best structure and apply a PB procedure. Furthermore, the results associated with our follow-up analyses should be interpreted with caution given the limited number of longitudinal studies. As such, we are restricted in our ability to make data-driven practical implementation recommendations. Second, the developer of the PB procedure (N. M. Petry) authored 78% of the studies included in this meta-analysis, suggesting the possible presence of an operator bias. However, similar effects have been obtained by other investigators, lending credibility to the results. Implementation of the PB procedure by additional research teams would further strengthen confidence in these results. Finally, our meta-analytic data set did not include studies that were based on non-randomized designs or published in non-peer-reviewed journals. As such, we may have not included data associated with small sample studies or studies that reported non-significant findings and, therefore, were less likely to be accepted for publication [59]. The publication bias analysis conducted in our study indicated that this had no impact on our overall EOT effect size.

4.4 Strengths

There are also a number of notable strengths associated with this study. First, given that illicit substance abusers may under-report their drug use [67-69], only data from biological indicators for those drugs were used in the meta-analysis, whereas self-report data on alcohol use was allowed [19,24] given that it has been shown to be as valid as biologically-confirmed data [70]. As such, these results are likely to be more valid than findings associated solely with self-report indices. Second, only studies that employed random assignment to treatment were included. As such, these results can be linked to true causal relationships between the PB procedure and substance use. Finally, rather than only focusing on the efficacy of the PB procedure during the time it was actually being implemented, this study provides preliminary information about its longer-term, post-intervention effects.

4.5 Future Directions

Additional studies based on randomized designs with a control comparison group are needed to more fully evaluate the evidence base for the PB procedure. This will allow us to identify moderator variables in future meta-analyses that will suggest for whom and under what circumstances the PB procedure would be most beneficial. Also, further studies are needed so that stronger analyses of longer term effects can be conducted.

Importantly, the extent to which the effect size associated with the PB procedure compares to VBRT is unknown. The belief that PB is a lower cost alternative to VBRT is now fairly widely held, with little consideration that this may be have been achieved at the cost of producing smaller effect sizes. Meta-analyses conducted by an independent research team exclusively on PB and VBRT procedures would shed much needed comparative light on their relative efficacy and allow a more systematic examination of the procedural variables contributing to variance in cost and efficacy.

Finally, apparent difficulty maintaining the positive effects of CM procedures such as the PB procedure may, in part, be related to the need to conceptualize and treat addiction like other chronic diseases that require long-term management and care (e.g., hypertension, diabetes; [71,72]). Several continuing care strategies have reported promising results at maintaining abstinence (e.g., Web-based multi-media programs; periodic contact with recovery coaches; [64-66]), but additional research is warranted.