Performance-based contingency management in cognitive remediation training: A pilot study

Abstract

Impairments in attention, working memory, and executive function are common among substance users and may adversely affect SUD treatment outcomes. The ability of cognitive remediation (CR) interventions to improve these deficits is hindered in part because levels of engagement in CR training may be inadequate to achieve benefit. This pilot study aimed to increase CR engagement and improve outcome by implementing contingency management (CM) procedures that reinforce performance improvements on CR tasks. Participants were forty individuals (50% male; 65% African American) in an outpatient substance use treatment facility with mild cognitive impairment who had ≥ 30-days of abstinence from alcohol and drugs. They were randomized to standard (CR-S; n = 21) or CM-enhanced (CR-CM; n = 19) cognitive remediation training. CR consisted of 1-hour sessions, three times per week for four weeks (12 sessions). A neuropsychological assessment battery was administered prior to and after the four-week intervention. Both groups had high rates of CR session attendance (mean CR-S = 11.7, CR-CM = 10.9 sessions). Performance on 8 of the 9 CR tasks significantly improved over time for both conditions, with the CR-CM condition demonstrating greater improvement on a CR Sequenced Recall task [F(1,37) = 5.81, p < .05]. Significant improvement was also evident on 4 of 9 neuropsychological assessment measures, with the CR-CM condition showing differential improvement on the Trail Making Test – Part B [F (1,37) = 5.34, p < .05]. These findings support the feasibility of using CM procedures to enhance substance users’ engagement with CR training and suggest the potential value of more research in this area.

1.1 INTRODUCTION

The effectiveness of treatment for individuals with substance use disorders (SUDs), particularly for cognitively demanding approaches like cognitive behavioral therapy (CBT), may be undermined by diminished cognitive resources associated with chronic substance use (Bolla et al., 2004; Bolla, Rothman, & Cadet, 1999; Verdejo-Garcia, Rivas-Perez, Lopez-Torrecillas, & Perez-Garcia, 2006; Vik, Cellucci, Jarchow, & Hedt, 2004). Impairments with respect to attention, working memory, and executive function among SUD patients have been associated with poorer treatment outcomes such as less abstinence (Passetti, Clark, Mehta, Joyce, & King, 2008), shorter treatment retention (Aharonovich et al., 2006; Aharonovich, Nunes, & Hasin, 2003; Streeter et al., 2008; Turner, LaRowe, Horner, Herron, & Malcolm, 2009; Verdejo-Garcia et al., 2012), lower self-efficacy (Bates, Pawlak, Tonigan, & Buckman, 2006) and poorer coping skills acquisition (Kiluk, Nich, & Carroll, 2011). Cognitive remediation (CR) interventions, typically computer-administered training exercises, are designed to harness the brain’s neuroplastic capabilities to enhance or restore these types of impaired cognitive processes (Keshavan, Vinogradov, Rumsey, Sherrill, & Wagner, 2014). Despite the strong evidence base on the effectiveness of CR for improving cognitive impairments among schizophrenic populations (McGurk, Twamley, Sitzer, McHugo, & Mueser, 2007; Medalia & Choi, 2009; Til Wykes, Huddy, Cellard, McGurk, & Czobor, 2011), tests of CR in individuals with SUDs have yielded mixed results. Some studies have shown training-related improvement in attention and working memory (e.g., Goldstein, Haas, Shemansky, Barnett, & Salmon-Cox, 2005; Rass et al., 2015; Rupp, Kemmler, Kurz, Hinterhuber, & Fleischhacker, 2012), and others limited or no cognitive benefit (e.g., Bickel, Yi, Landes, Hill, & Baxter, 2011; Peterson, Patterson, Pillman, & Battista, 2002).

One potential reason for mixed effects among SUD patients may be their limited motivation to engage in CR. Many CR interventions require participants to complete multiple trials of monotonous tasks. This may lead to boredom and reduced engagement, which in turn impedes task performance (Hawkins, Rae, Nesbitt, & Brown, 2013). Low motivation has been noted as a moderator of CR effects among patients with schizophrenia (Medalia & Choi, 2009; Medalia & Richardson, 2005; Velligan, Kern, & Gold, 2006), yet has received relatively little attention in CR studies with substance users. Although some computer-based CR programs include game-like motivational elements to increase engagement and enjoyment (e.g., real-time scoring system, virtual prizes and certificates), the actual effect on participant motivation, engagement, and performance may be relatively limited (Hawkins et al., 2013; Katz, Jaeggi, Buschkuehl, Stegman, & Shah, 2014). Thus, the potential for these cognitive enhancing interventions to serve as a viable adjunct treatment for addictions (Bickel, Moody, & Quisenberry, 2014; Sofuoglu, DeVito, Waters, & Carroll, 2013) may be undermined by the inadequate level of motivation and training engagement within this population.

Contingency management (CM; e.g., voucher or prize-based reinforcement) has strong empirical support for improving treatment retention and increasing abstinence among SUD patients (e.g., Higgins, Alessi, & Dantona, 2002; Higgins et al., 1994; Petry et al., 2006; Prendergast, Podus, Finney, Greenwell, & Roll, 2006), and so might be a useful approach for improving engagement and performance on cognitive tasks in this population. This supposition is supported by studies in which an attention shaping procedure that included monetary rewards for achieving attentiveness duration goals enhanced a conversational skills training program with schizophrenic patients (Silverstein et al., 2005; Silverstein et al., 2009). A study by Bickel et al. (2011) used performance-based monetary rewards in CR training with individuals diagnosed with stimulant abuse/dependence and demonstrated a positive effect of CR training on a delay discounting measure (although no effect on working memory). This study set a precedent for use of performance-based CM in CR training, but did not provide an evaluation of the approach.

This pilot study was conducted to gather preliminary data on whether a performance-based CM intervention would result in improved performance during computerized CR training, and if so, whether this would translate into improved cognitive function as measured by standard neuropsychological tests.

2.1 MATERIAL AND METHODS

2.1.1 Overview

This pilot study enrolled individuals with substance use disorders but who had been abstinent for at least 30 days to a 4-week CR intervention. All participants received reinforcement for attending CR sessions; those assigned to the CR-CM condition received reinforcement for improvements on CR tasks. Potential participants were screened for eligibility, then completed pre-training assessments that included neuropsychological assessment prior to being randomized to one of the CR intervention conditions. Following the 4-week CR intervention, participants repeated the neuropsychological assessment battery.

2.1.2 Participants

Participants were recruited from a community outpatient substance use treatment facility as well as through online advertisements. To be eligible, participants had to be between 18-60 years of age, meet DSM-IV criteria for substance dependence within the past year, report no use of alcohol or drugs for the past 30 days (and provide a negative urine drug screen at time of screening), and demonstrate evidence of mild cognitive impairment (score <26 on Montreal Cognitive Assessment (Nasreddine et al., 2005)). Individuals were excluded if they met DSM-IV criteria for a current bipolar or psychotic disorder, if they would be unable to complete the 4-week intervention period due to an anticipated event (e.g., planned move out of the area, facing incarceration, etc.), or if they were colorblind (due to inclusion of the Stroop Color Word Test in the assessment battery). The maximum age was set at 60 years, as age has been found to be a predictor of improvement from CR (Kontis, Huddy, Reeder, Landau, & Wykes, 2013; Vita et al., 2013; Wykes et al., 2009). Alcohol and drug abstinence for at least 30 days was required to limit the negative cognitive impact of acute drug withdrawal and/or reduce the potential confound of cognitive recovery associated with short-term abstinence (Pace-Schott et al., 2008; Stavro, Pelletier, & Potvin, 2013). Moreover, there is some evidence that the effectiveness of CR is moderated by initial cognitive function, such that those with greater impairment demonstrate larger effects (Fiszdon, Cardenas, Bryson, & Bell, 2005; Fiszdon, Choi, Bryson, & Bell, 2006).

2.1.3 Assessments

Pre-training screening included the Montreal Cognitive Assessment (MoCA)(Nasreddine et al., 2005), a brief, 10-minute cognitive screening instrument that was used to determine the presence of mild cognitive impairment. It has demonstrated greater sensitivity to subtle cognitive deficits than the Mini-Mental State Examination (MMSE) in a variety of populations (Dong et al., 2010; Hoops et al., 2009; Popovic, Seric, & Demarin, 2007), and has good agreement with the lengthier Neuropsychological Assessment Battery-Screening Module at identifying cognitive impairment in patients with substance use disorders (Copersino et al., 2009). The Structured Clinical Interview for the DSM-IV (SCID; First, Spitzer, Gibbon, & Williams, 1995) was used determine diagnostic eligibility, and the Substance Use Calendar (SUC), similar to the Timeline Follow-back (Sobell & Sobell, 1992), was used to assesses self-reported days of substance use in the past calendar month. Urine drug screening (cocaine, marijuana, opioids, benzodiazepines, methamphetamine) was used to confirm recent drug abstinence at the time of screening, as well as weekly during the course of training. Alcohol breathalyzer tests were conducted at each study visit.

Assessments also included: (1) the Addiction Severity Index (ASI; McLellan et al., 1992); (2) the Shipley Institute of Living Scale (Zachary, 1991) was used to assess general intellectual functioning with scores from the vocabulary and abstract subtests converted to estimated IQ scores; (3) the Patient’s Assessment of Own Functioning Inventory (PAOFI; Richardson-Vejlgaard, Dawes, Heaton, & Bell, 2009) was administered at pre- and post-training time points to evaluate patients’ self-reported cognitive impairment using a Likert-type response scale from 1 (“almost always”) to 6 (“almost never”) for a series of items regarding everyday cognitive complaints (e.g., “how often do you lose things or have trouble remembering where they are?”). A higher rating on any item indicates a lesser degree of impairment. The PAOFI includes subscales assessing memory, language and communication, and higher cognitive functions. (4) The Intrinsic Motivation Inventory for Schizophrenia Research (IMI-SR; Choi, Mogami, & Medalia, 2010) is a 21 item self-report, Likert-format measure tapping three domains relating to motivation for treatment: interest/enjoyment, perceived choice, and value/usefulness. The IMI-SR was adapted from the original IMI in order to assess the motivational structures for a learning activity specifically in an experimental setting and has been shown to have good internal consistency and test-retest reliability. Participants indicate how true each of the statements were in regard to completing the computer learning activity (e.g., “I enjoyed doing this activity very much”) using a response scale from 1 (“not at all true”) to 7 (“very true”).

The following seven neuropsychological tests yielding ten measures were administered at pre- and post-training:

Digit Symbol subtest from WAIS-IV – this test consists of nine digit-symbol pairs followed by a list of digits with empty boxes. Participants are asked to write down the corresponding symbol below the correct number as fast as possible, completing as many pairs as possible within 120 seconds. The correct number of pairs is considered a measure of processing speed (Joy, Kaplan, & Fein, 2004).

Trail Making Test (Part A & B) – this timed test requires participants to draw a line connecting a series of targets, either numbers only (Trails A) or numbers and letters (Trails B), on a sheet of paper. Time to complete the task is a measure of visual attention and cognitive flexibility (Kortte, Horner, & Windham, 2002); separate scores are obtained for part A and B.

Hopkins Verbal Learning Test - Revised (HVLT-R; Brandt & Benedict, 2001) – this is a widely used task of verbal and learning and memory that consists of a 12-item word list composed of four words from each of three semantic categories. The examiner reads the word list and participants are asked to recall as many as they can remember (this process is repeated two more times). Following a 20-25-minute delay, participants are asked to again recall as many words as they can remember. Scores include the total recalled and the delayed recall; the latter is reported. Alternate, equivalent forms were used to reduce practice effects from pre- to post-training.

Stroop Color Word Test (Golden & Freshwater, 2002) – this is considered a general measure of cognitive flexibility and control (Uttl & Graf, 1997) or executive functioning (Moering, Schinka, Mortimer, & Graves, 2004). Participants are asked to name the ink color of an incongruously named color word as quickly as possible within a time limit. Their performance is compared to a basic task of reading names of colors (all printed in black ink), which produces a color-word interference T-score (reflecting the difference between the age/education adjusted predicted color-word score and the actual uncorrected raw color-word score; lower T-scores indicate greater color-word interference problems).

Continuous Performance Test-II (CPT-II; Connors, 2004) – this is a computer-administered general measure of sustained attention. Participants are required to press the space bar or click the mouse whenever any letter except the letter “X” appears on the screen. Measures include separate T-scores for errors of omission and commission, reported here.

Balloon Analogue Risk Task (BART; Lejuez et al., 2002) – this is a computerized decision-making task that provides a test of behavioral risk taking. Participants are instructed to use the computer mouse to inflate a balloon on the computer screen to a desired level. Each click on the mouse inflates the balloon one degree and earns the participant money in a “temporary bank” that would be lost if the balloon pops with more successive clicks on the mouse. The adjusted average number of pumps is the BART’s primary dependent measure; number of explosions is also reported.

Quick Discounting Operant Task (QDOT; Johnson, 2012) – This is an assessment of delay discounting that assesses risk/reward decision-making in vivo and directly assesses behavioral discounting with actual time delays and real-time rewards (coins mechanically dispensed). Reported is the area under the curve, which allows the analysis of discounting data based on a standardization of indifference points; steeper discounting is associated with a smaller area under the curve (Dallery & Raiff, 2007; Myerson, Green, & Warusawitharana, 2001). It is positively correlated with the Experiential Discounting Task (EDT; Reynolds & Schiffbauer, 2004), which is another measure of discounting found to decrease following CR training in substance users (Bickel et al., 2011).

2.1.4 Intervention

CR was delivered via the PSSCogRehab2012 (Psychological Software Services) program, which has been used in prior trials with substance users (e.g., Bickel et al., 2011; Fals-Stewart & Lam, 2010). Of the more than 65 tasks contained in the program, 9 were selected for this study based on the cognitive domain of interest (attention, working memory, and problem solving). Table 1 presents a description of each task.

Table 1.

PSSCogRehab2012 tasks

Domain (task name)	Description	Performance Indicator
Attention
Simple Choice Auditory Reaction	Participants must press space bar when they hear target musical note; must inhibit press when hearing non-target musical note.	Average reaction time; number omissions; number commissions
Simultaneous Multiple Attention	Four parallel lines of multi-colored window frames moving across the screen; participants must click when ‘target’ color appeared in center frame; speed of lines could be modified (slower or faster).	Number of correct ‘target’ clicks out of the total possible
Detecting Differences	Four large identical frames (with the exception of one minor detail in one frame) presented on screen filled with either alphabet characters or random graphics patterns (depending on setting chosen); participants must click the frame with the difference as quickly as possible.	Number of correct frames detected on first click
Working Memory
Spatial Memory	Participants presented with a series of randomly selected shapes in specific locations and must correctly place shapes after a brief wait period during which the shapes disappear; the study time (1-5 sec/item) and wait time (1-60 sec) could be modified.	Most correctly recalled in one trial
Sequenced Recall	Participants must correctly recall a series of numbers in the order presented. The number of digits presented (1-7), time lapse in between presentations (0.5 – 2 sec), and wait time (1 – 60 sec) could be modified.	Most correct numbers recalled in one trial
Verbal Memory	Participants presented with 20 words and asked to place them into one of four categories. Categories then cleared and participants had to correctly place the words again into categories after a brief wait period (1 – 60 sec).	Percent correctly recalled
Paired Associates	Participants presented with either 3, 5, or 10 pairs of graphic designs and numbers for a varied period of time to study (3 – 20 seconds). Following the study period, the graphic design was presented and participants were instructed to supply the number that was its pair.	Number of correct pairs
Problem Solving
Simply Logical	Participants presented three to five squares (depending on the difficulty level) and were asked to click on each square multiple times to change its color (green, red, or blue) until the colors of the squares were in a “correct” order. Participants were unaware of the correct order, requiring them to use a trial and error process to determine the correct order within a given number of attempts.	Number of guesses until correct order achieved
Pyramids II	Participants presented with three posts, with one post containing four, five, or six red disks of various sizes, and one post colored gold. Participants must move the disks back and forth, from post to post, until placed in proper order (from largest to smallest) upon the gold post.	Number of moves until correct placement

2.1.5 Procedures

Upon completion of the screening and pre-training assessments, eligible participants were randomly assigned (using an urn randomization program that balanced for gender, race, MoCA score, and primary drug (Stout, Wirtz, Carbonari, & DelBoca, 1994)) to one of the following four-week training conditions:

CR-Standard (CR-S) – participants assigned to this condition were asked to engage in CR sessions three times per week for four weeks. Participants completed three tasks per cognitive training session (20 minutes per task for total of 1-hour training session), thereby engaging in each task once per week during the study. The tasks during each visit varied randomly and were counterbalanced across participants and conditions, such that participants were to spend equal amounts of time on each task during the course of the study. At the start of each session, research staff instructed participants to give their best effort on the tasks. Participants’ performance on the initial trial for each task was used to establish their baseline level of performance. A score summary screen appeared upon the completion of each trial for a given task, which was viewed by participants. The difficulty level of the task was increased each time a participant’s performance on that task improved from their previous level (e.g., decreased time to completion, fewer errors, etc.); research staff informed participants each time the difficulty level was increased. After 3 consecutive attempts at the task without maintaining or improving performance, the task was discontinued and the participant moved on to the next task. Participants in the CR-S condition were compensated $15 per training session attended, with an additional $15 bonus if they completed all three sessions in a given week (maximum for attendance = $240).

CR-Contingency Management (CR-CM) - Participants randomized to this condition

received the same training procedures and instructions as those in the CR-S condition, as well as payment for attendance at each training session. In addition, they could also receive monetary payments based on their performance on the cognitive training tasks. The first trial for each task was used to determine the ‘baseline’ level of performance for that session. Following baseline task trial completion, participants received a monetary reward each time their performance on the subsequent trial improved from their prior performance, with rewards increasing in value as the difficulty of the task increased during the session. Participants did not receive a monetary reward for failing to successfully complete a trial or if their performance on that trial decreased (e.g., greater number of errors). Research staff informed participants at the end of each task trial whether their performance improved/decreased from the prior trial’s performance, and the amount of money earned for that trial. The more trials completed during the training session while improving performance, the greater the total monetary amount received at the end of that session. Maintaining performance also resulted in reinforcement, although not as much as improving performance. Further, monetary rewards varied according to the difficulty level of the task (ranging from $1- $5 depending on the task). Participants could earn up to an additional $25 per session if improved performance was achieved on each trial (maximum for attendance + improved performance = $540).

2.1.6 Data Analysis

Demographic and baseline characteristic differences were evaluated across training assignment (CR-S vs. CR-CM) using analysis of variance (ANOVA) or Chi-square tests. The mean number of CR sessions attended during the 4-week training period, as well as the percentage of participants completing all 12 CR sessions were compared across groups using ANOVAs and Chi-square to determine whether performance-based CM had an effect over and above attendance-based CM on CR attendance. The primary outcome was improvement on the CR tasks, which was defined in two ways: (1) change in mean CR task scores (weighted according to difficulty level) from week 1 to week 4; and (2) the percentage of the task trials wherein the participant’s performance improved from the prior trial. The change in weighted CR task scores was evaluated with repeated measures ANOVA, using scores from the first session’s trial (baseline) as week 1, and the best score achieved during the final session to represent week 4. One-way ANOVAs were used to evaluate a difference in the mean percentage of task trials showing improvement across the two training conditions. The secondary outcome was a change in performance on the neuropsychological tests from pre- to post-training, which was evaluated across treatment condition using repeated measures ANOVA. The change in neuropsychological test performance was also evaluated with a general linear model according to the percentage of CR task trials showing improvement, in order to determine whether improved performance on CR tasks was associated with change in neuropsychological test performance. This was evaluated based on the percentage of trials showing improvement for all CR tasks combined, as well as within each CR task domain (attention, working memory, problem solving). Self-reported days of alcohol or drug use during the 4-week treatment period were included as a covariate in these analyses. Exploratory analyses were conducted to evaluate changes in self-reported cognitive function from pre- to post-training (PAOFI scores), intrinsic motivation from week 1 to week 4 (IMI scores), and rates of drug abstinence (percentage of self-reported days of abstinence, and percentage of individuals submitting at least 1 urine sample positive for drugs during training period).

3.1 RESULTS

3.1.1 Participants

Of the 55 participants screened for eligibility, 40 were determined to be eligible and randomized to a training condition (CR-S = 21; CR-CM = 19; see Figure 1). There were no demographic or baseline characteristic differences across training conditions (Table 2); 50% of the participants were female; the majority were African American (65%), completed high school (73%), and were unemployed (80%). In terms of substance use, the majority reported alcohol as their primary drug (55%), followed by cocaine (40%), and marijuana (5%). Nearly 18% (n = 7) of the sample reported at least one serious head injury (e.g., concussion, cerebral contusion, traumatic brain injury) in their lifetime; of these seven individuals, two (29%) reported a loss of consciousness at the time of the head injury. The mean MOCA score for the full sample was 21.9 (SD = 3.6), indicating mild cognitive impairment.

Figure 1.

CONSORT Diagram of Participant Flow

Table 2.

Demographic and Baseline Characteristics

	CR-S (n = 21)	CR-CM (n = 19)	Total (n = 40)	Χ 2	p
	n (%)	n (%)	n (%)
Female	8 (38.1)	12 (63.2)	20 (50)	2.51	.11
Race
Caucasian	5 (23.8)	3 (15.8)	8 (20)	1.07	.78
African-American	13 (61.9)	13 (68.4)	26 (65)
Other	1 (4.8)	2 (10.5)	3 (7.5)
Multiracial	2 (9.5)	1 (5.3)	3 (7.5)
Completed High School	16 (76.2)	13 (68.4)	29 (72.5)	0.30	.58
Never married/living alone	18 (85.7)	15 (78.9)	33 (82.5)	0.32	.57
Unemployed	17 (81)	15 (78.9)	32 (80)	0.03	.87
Referred by criminal justice
system	2 (9.5)	1 (5.3)	3 (7.5)	0.26	.61
On Probation	6 (28.6)	2 (10.5)	8 (20)	2.03	.15
On Public Assistance	18 (85.7)	12 (63.2)	30 (75)	2.71	.10
Primary Drug
Alcohol	10 (47.6)	12 (63.2)	22 (55)	2.34	.31
Cocaine	9 (42.9)	7 (36.8)	16 (40)
Marijuana	2 (9.5)	0	2 (5)
Prior serious head injury (YES)	4 (19.0)	3 (15.8)	7 (17.5)	0.07	.79
Lost consciousness (YES)	1 (25)	1 (33.3)	2 (28.5)	0.20	.66
	Mean (SD)	Mean (SD)	Mean (SD)	F	p
Age	44.8 (8.7)	43.2 (7.8)	44 (8.2)	0.35	.56
Years of alcohol use	14.2 (13.2)	15.8 (10.7)	15 (11.9)	0.18	.68
Years of cocaine use	9.6 (9.1)	14.3 (14.4)	11.8 (12.0)	1.56	.22
ASI Medical Composite	.10 (.24)	.10 (.23)	.10 (.23)	0.01	.94
ASI Employment Composite	.16 (.20)	.30 (.29)	.22 (.25)	3.37	.07
ASI Alcohol Composite	.10 (.15)	.05 (.10)	.07 (.13)	1.37	.25
ASI Cocaine Composite	.07 (.19)	.02 (.10)	.05 (.15)	1.16	.29
ASI other Drug composite	.002 (.01)	.001 (.004)	.002 (.01)	0.29	.59
ASI Legal Composite	.04 (.09)	.02 (.07)	.03 (.08)	0.21	.65
ASI Family Composite	.10 (.14)	.12 (.15)	.11 (.15)	0.20	.66
ASI Psychological Composite	.18 (.22)	.12 (.16)	.15 (.20)	1.22	.28
Number of psych inpatient	.67 (2.2)	1.2 (1.9)	0.9 (2.0)	0.57	.46
Number of psych outpatient	1.2 (2.2)	2.4 (4.1)	1.8 (3.3)	1.31	.26
Number of months since last head injury	130.3 (148.3)	184.0 (60.4)	153.3 (114.2)	0.34	.59
MOCA	21.6 (3.8)	22.2 (3.4)	21.9 (3.6)	0.26	.61

3.1.2 CR Engagement

There was a significant difference in the mean number of CR sessions attended across groups during the study, favoring the CR-S condition (CR-S: M = 11.7, SD = 0.6; CR-CM: M = 10.9, SD = 1.6, F (1,38) = 4.48, p < .05). However, this difference was no longer significant after excluding the participant assigned to the CM condition who became incarcerated during the 4-week training period (CR-S: M = 11.7, SD = 0.6; CR-CM: M = 11.1, SD = 1.3, F (1,37) = 3.21, p = .08). There were no differences across groups in terms of the number of participants who attended all 12 CR sessions during the 4-week period (CR-S = 71% vs. CR-CM = 58%, Χ² = 0.80, p = .37). On average, the CR-CM participants earned an additional $14.90 (SD = $3.60) per session based on their performance on the tasks. Mean earnings for the 4-week study across conditions were: CR-S = $231 (SD = $17); CR-CM = $374 (SD = $62).

3.1.3 Improvement on CR Tasks

Table 3a displays the change in mean weighted CR tasks scores (task’s primary performance measure weighted according to level of difficulty) from week 1 to week 4, controlling for self-reported days of alcohol and/or drug use. Results indicated a significant effect of time, suggesting overall improvement irrespective of training condition, for 8 out of the 9 tasks. A significant time × group interaction effect was present for the Sequenced Recall task indicating a differential improvement over time favoring the CR-CM condition compared with CR-S. Table 3b displays results of between-group ANOVAs comparing the percentage of CR task trials within a given task where the participant demonstrated an improvement in performance from the prior trial. Participants assigned to the CR-CM condition improved on a greater percentage of the Simply Logical trials compared to participants assigned to CR-S (15.2% vs. 9.3%, respectively; F (1,37) = 10.60 p<.01). However, the percentage of trials with improved performance did not differ across training conditions for the other eight CR tasks, nor was this difference reflected in differential improvement during training (Table 3a).

Table 3a.

Weighted CR task scores over time^a

Task Name & Measure	CR-S	CR-CM	Time	Group	Time × Group
	Mean (sd)	Mean (sd)	F	F	F
Simple Choice Auditory Reaction†			86.50***	0.68	0.01
Average Reaction Time (ms) Week 1  Average Reaction Time (ms) Week 4	751.5 (219.1) 398.7 (77.3)	696.6 (255.5) 367.8 (61.0)
Simultaneous Multiple Attention			360.57***	1.43	2.71
Correct Hits Week 1  Correct Hits Week 4	19.4 (1.1) 61.1 (17.3)	18.8 (5.7) 66.5 (13.4)
Detecting Differences			27.33***	0.71	0.80
Number Correct First Click Week 1  Number Correct First Click Week 4	14.7 (2.7) 22.3 (11.9)	14.2 (4.4) 24.3 (9.9)
Spatial Memory			47.11***	2.75	1.88
Most Recalled in One Trial Week 1  Most Recalled in One Trial Week 4	4.8 (4.7) 26.2 (17.2)	4.8 (6.2) 19.2 (13.1)
Sequenced Recall			29.47***	2.19	5.81*
Most Recalled in One Trial Week 1  Most Recalled in One Trial Week 4	11.9 (3.1) 14.4 (3.7)	11.4 (2.3) 17.4 (6.3)
Verbal Memory			4.67*	0.04	0.01
Percent Recalled Week 1  Percent Recalled Week 4	14.3 (5.1) 18.9 (18.1)	14.9 (4.6) 19.5 (8.9)
Paired Associates			43.32***	0.30	0.33
Number Correct Week 1  Number Correct Week 4	2.7 (2.2) 33.6 (28.2)	3.1 (3.0) 28.2 (23.7)
Pyramids†			11.12**	0.09	0.31
Number of Moves Week 1  Number of Moves Week 4	443.3 (259.2) 204.9 (10.1)	498.8 (451.7) 212.1 (202.5)
Simply Logical			2.99	1.54	0.01
Number Correctly Solved Week 1  Number Correctly Solved Week 4	5.7 (8.1) 9.5 (14.3)	3.2 (4.8) 7.4 (12.4)

^*p<05;

^**p<01;

^***p<001

^†lower score indicates better performance

^acontrolled for days of alcohol and/or drug use during trial period

Table 3b.

Percentage of CR trials with improved performance^a

Task	CR-S Mean (sd)	CR-CM Mean (sd)	F	p
Simple Choice Auditory Reaction	62.9 (6.7)	63.9 (6.6)	0.08	.78
Simultaneous Multiple Attention	36.0 (8.2)	37.2 (7.4)	0.55	.46
Detecting Differences	24.5 (26.8)	27.9 (15.7)	0.14	.71
Spatial Memory	28.8 (13.1)	30.1 (9.4)	0.05	.82
Sequenced Recall	7.6 (2.3)	10.7 (17.8)	0.40	.53
Verbal Memory	3.6 (8.4)	2.4 (6.6)	0.10	.75
Paired Associates	36.6 (6.5)	32.3 (10.2)	1.07	.31
Pyramids	37.1 (11.9)	34.1 (16.2)	0.88	.35
Simply Logical	9.1 (6.4)	15.1 (4.3)	10.60	<.01

^acontrolled for self-reported days of alcohol and/or drug use during trial period

3.1.4 Improvement on Neuropsychological Measures

Results of repeated measures ANOVA evaluating change over time (pre- to post-training) across training condition on the neuropsychological assessment measures are displayed in Table 4. A significant effect of time was present for four of nine neuropsychological measures (Digit Symbol, Trail Making Test – Part A, Trail Making Test – Part B, CPT Commissions). A significant ‘time × group’ effect was present only for the Trail Making Test – Part B [F(1, 37) = 5.34, p < .05], with participants assigned to the CR-CM condition improving their performance (decreased time to completion) more so than those assigned to the CR-S condition.

Table 4.

Change in Neuropsychological Assessment Performance^a

Measure	CR-S	CR-CM	Time	Time × Group
	Mean (sd)	Mean (sd)	F	F
Digit Symbol score			10.55***	2.47
Week 1  Week 4	46.6 (12.3) 48.5 (13.7)	50.0 (12.0) 55.7 (12.3)
Trail Making Test – A (time in sec)			5.65*	0.67
Week 1  Week 4	40.6 (13.4) 37.1 (12.0)	36.9 (10.4) 29.1 (8.2)
Trail Making Test – B (time in sec)			14.31***	5.34*
Week 1  Week 4	90.6 (31.4) 84.3 (40.2)	105.9 (41.4) 79.9 (42.6)
HVLT – Delayed Recall T score			0.45	0.08
Week 1  Week 4	34.2 (11.5) 35.0 (11.4)	38.1 (12.4) 14.2 (10.1)
Stroop – Interference T score			2.24	0.79
Week 1  Week 4	47.3 (8.8) 48.4 (10.8)	47.9 (10.4) 53.3 (7.6)
CPT – Omissions T score			0.37	0.17
Week 1  Week 4	58.1 (36.7) 66.2 (57.4)	57.7 (22.3) 59.7 (31.3)
CPT – Commissions T score			7.10**	0.40
Week 1  Week 4	48.8 (15.1) 52.3 (13.2)	47.9 (12.3) 53.1 (14.0)
BART – Pumps Adjusted Average			0.16	1.34
Week 1  Week 4	27.4 (13.5) 24.9 (12.6)	32.1 (13.9) 28.7 (10.8)
BART – Explosions			2.08	1.39
Week 1  Week 4	4.7 (2.5) 3.5 (2.3)	5.1 (2.2) 4.5 (2.1)
QDOT – Area Under the Curve			>0.01	0.74
Week 1  Week 4	39.9 (22.7) 42.3 (22.7)	41.5 (22.6) 38.8 (25.9)

^*p<05;

^**p<.01;

^***p<.001

^acontrolled for self-reported days of alcohol and/or drug use during trial period

General linear modeling did not indicate a significant association between improved performance on CR task trials (all combined) and change in performance on any neuropsychological measure. When the CR tasks were separated according to the targeted cognitive domain, there was a significant interaction between improvement on CR attention tasks and change in BART performance (‘time × % improved’ for BART Pumps - adjusted average: F(1, 74) = 4.0, p < .05; ‘time × % improved’ for BART – explosions: F(1, 74) = 4.83, p < .05). Specifically, improved performance on CR attention tasks was associated with less risk-taking on the BART (fewer pumps and explosions). No significant interactions were found in domains of working memory or problem solving.

3.1.5 Changes in Substance Use and Other Exploratory Measures

The percentage of participants who provided a urine sample positive for illicit drugs during the training period did not differ across conditions (CR-S = 6% vs. CR-CM =27%, χ² = 2.39, p = .12), nor did the self-reported days abstinent (CR-S = 99.4% vs. CR-CM = 94.7%, F(1,38) = 2.55, p = .12). Results of repeated measures ANOVA for self-reported cognitive impairment (PAOFI subscale scores) revealed a significant effect of ‘time’ on the memory [F(1,32) = 6.80, p = .01] and the higher cognitive functions subscale [F(1,32) = 5.05, p = .02] indicating all participants reported fewer cognitive complaints in these areas from pre- to post-training. There were no significant ‘time × group’ effects on any PAOFI subscale. In terms of motivation, there were no significant ‘time’ or ‘time × group’ effects on the IMI-SR total or subscale scores. Across training conditions, mean IMI scores during the first week of training were 5.95 (SD = 0.89) and during the final week of training were 5.85 (0.77), reflecting relatively high ratings of interest/enjoyment, personal choice, and value/usefulness.

4.1 DISCUSSION

This pilot study implemented a performance-based monetary reinforcement system based on contingency management procedures to determine its impact on cognitive remediation training among abstinent substance users. Overall, results did not indicate a strong effect of the performance-based CM in a context where high levels of session attendance (94% of sessions) were observed. All participants showed similar significant levels of task improvement over the course of training as well as improvements in some neuropsychological function domains from pre-to-post training. Nevertheless, the few differential effects observed were in the predicted direction. In particular, the CM as compared to the standard group showed greater improvement on a sequenced recall task during training and greater pre-to-post training improvement on a task (Trail Making Test – Part B) related to visual attention and cognitive flexibility. These results demonstrate feasibility and support further evaluation of the performance-based CM intervention.

Adequate engagement with cognitive training tasks has been the major challenge with implementing CR procedures with substance users, particularly in outpatient settings, including 2 previous failed pilots by our group. The few existing studies with relatively high rates of treatment engagement included samples from inpatient or residential facilities (e.g., Fals-Stewart & Lam, 2010; Goldstein et al., 2005; Rupp et al., 2012). It is possible that compensation for attendance may be all that is needed to promote engagement with cognitive training. We observed very high attendance rates when participants could earn a total of $240 for attendance over a 4-week period. Attendance compensation has also been included in prior studies with a similar sample/setting, yet reported lower attendance rates than found here (e.g., Bell, Vissicchio, & Weinstein, in press; Eack et al., 2015; Rass et al., 2015). The magnitude of compensation for attendance (which appears slightly higher than prior studies), in addition to weekly bonuses contingent on perfect attendance, likely contributed to the strong attendance rates in this study. Although the small group difference in attendance favored the CR-S condition when the full sample was examined, the difference was no longer significant after excluding the participant assigned to CR-CM who became incarcerated during the training period.

Performance-based CM procedures appeared to have minimal impact on CR task performance over and above the impact of monetary incentives for attendance. There were few significant differences across groups whether CR task performance was measured using a weighted scoring system based on difficulty level from week 1 to week 4, or when measured as a mean percentage of trials with demonstrated improvement. Participants assigned to the CR-CM condition significantly improved their performance over time on the Sequenced Recall task and demonstrated improvement on a greater percentage of the Simply Logical task trials, compared to those in the CR-S condition. While it is possible these significant differences occurred due to chance given the number of comparisons, the overall pattern of findings suggests performance-based CM is worthy of further investigation in future CR trials.

An important question regarding CR interventions is whether the improvements on CR tasks will generalize to improvements in neurocognitive functioning (Owen et al., 2010). The data presented here suggested mixed effects. While prior studies have reported improvements in neurocognitive function associated with improvements in CR performance (e.g., Fisher, Holland, Merzenich, & Vinogradov, 2009; Jaeggi, Buschkuehl, Jonides, & Perrig, 2008; Rass et al., 2015), our exploratory analysis did not reveal much specificity in the relationship between improvement on CR tasks and measures of neurocognitive function. There was no differential change pre- to post-training on any neuropsychological assessments related to the percentage of overall CR task trials showing improvement. When tasks were grouped according to the targeted cognitive domain, improvement on the CR attention tasks were related to change in a measure of risk-taking behavior (BART). However, improvement on the CR attention tasks was not associated with differential changes over time on primary neuropsychological measures of attention (e.g., Digit Symbol, CPT). Also, there were no other significant interactions that would have suggested improved performance on specific CR task domains were associated with changes in neuropsychological assessment performance, limiting the conclusions drawn from the association with the BART. Lastly, the findings of Bickel et al (2011) regarding an effect of CR on decreasing delayed discounting rates were not replicated here (i.e., no significant pre-post change on the QDOT overall, or according to working memory task improvement). These relationships are important for validating a mediating role of CR training and need further examination in future research.

Significant limitations of the study are the small sample size and lack of a comparison condition that did not receive CR training. Other features of this pilot study may also have impacted the findings. First, although there is no defined optimal ‘dose’ of cognitive training necessary to produce significant improvement in neurocognitive function (Wykes et al., 2011), the duration of the CR intervention here was comparatively brief (4 weeks) given the average across CR studies in schizophrenia populations is 13 weeks (McGurk et al., 2007). Also, we incorporated a broad-based training approach targeting multiple aspects of cognitive function associated with substance use disorders, rather than a more focused training paradigm. The benefits of performance-based CM may have been more apparent within a longer, more intense or more tightly focused CR intervention (Wykes & Spaulding, 2011). We chose not to yoke payment amounts across the two conditions, which resulted in the two treatment groups receiving different amounts of total compensation. However, this would be more of an issue if strong differential effects had been present across the groups. Finally, some significant differences may have been expected to occur by chance given the large number of comparisons.

5.1 CONCLUSIONS

Cognitive remediation training is currently being explored as a means to improve outcomes of substance users exposed to SUD treatments that utilize cognitive processes such as information retention and skills acquisition. The potential effectiveness of CR approaches in this regard may hinge on patients’ level of engagement and effort with the CR training program. The high rate of attendance, and improvements in neurocognitive function by both self-report and objective assessment found in this study supports the feasibility of using CM techniques as a means to enhance outcomes of CR training. While data did not strongly support efficacy of the intervention, effects that were evident were in the predicted direction. Thus, the study findings support the feasibility of using CM procedures to enhance substance users’ engagement with CR training and suggest the potential value of more research in this area.

HIGHLIGHTS.

• CM could be used to improve engagement and performance during cognitive remediation (CR) training.

• Substance users received CR with or without CM for performance improvement

• Groups generally did not differ on training performance although the few differences observed favored the CM group.

• CM for performance improvement is feasible and should continue to be evaluated.