Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 May 1.
Published in final edited form as: Behav Anal (Wash D C). 2017 Feb 2;17(2):114–128. doi: 10.1037/bar0000077

Reinforcing Productivity in a Job-Skills Training Program for Unemployed Substance-Abusing Adults

Shrinidhi Subramaniam 1, Jeffrey J Everly 2, Kenneth Silverman 3
PMCID: PMC5560428  NIHMSID: NIHMS840822  PMID: 28824954

Abstract

Chronically unemployed adults may benefit from intensive job-skills training; however, training programs do not always reliably engage participants in mastering skills. This study examined effects of voucher reinforcement for performance on a job-skills training program in the therapeutic workplace. Participants were four unemployed, substance abusing adults who earned monetary vouchers for working on programs targeting typing skills. Participants were exposed to two payment conditions that differed in whether or not pay was dependent on performance in a within-subject reversal design. In the productivity-pay condition, participants earned $8.00 per hour for attending the workplace plus a bonus for performance. In the base-pay condition, participants were paid an hourly wage that was equivalent to the total hourly earnings from the previous productivity-pay condition. Participants completed less work on the typing programs in the base- than the productivity-pay condition, but the amount of time spent in the workroom and the accuracy and rate of typing were not affected by the pay manipulation. All participants reported preferring base pay over productivity pay. Explicit reinforcement of productivity maintains consistent work in training programs, but more aspects of productivity pay need to be refined for effective, efficient, and socially valid implementation with unemployed, substance-abusing adults.

Keywords: contingency management, reinforcement, drug addiction, employment, vocational training

Unemployment is a common problem among substance-abusing adults. Although employment is often considered an important treatment goal for adults with histories of drug addiction, many of those individuals lack the necessary skills to acquire and maintain jobs (Magura, Staines, Blankertz, & Madison, 2004; Silverman, Chutaupe, Bigelow, & Stitzer, 1996; Silverman, Chutaupe, Svikis, Bigelow, & Stitzer, 1995; Wong & Silverman, 2007). Some drug-abuse treatment programs offer vocational counseling and job-search assistance to promote employment, but the success of these programs is limited (for a review, see Magura et al., 2004).

Providing chronically unemployed adults with assistance in searching for a job does not address the issue that those individuals might lack the skills needed to obtain and retain gainful employment. Holtyn, DeFulio, and Silverman (2015) analyzed academic performance of unemployed, drug-addicted adults across six studies. Most participants were at or below a seventh-grade level in reading, spelling, and math, despite the fact that 60% of participants reported completing high school. Similarly, Silverman et al. (1995) showed that pregnant women who were in treatment for substance abuse were interested in obtaining jobs as clerical or office workers; however, their basic math, reading, and spelling skills were insufficient to perform those types of jobs successfully. For some substance abusers, promoting long-term and successful employment may require intensive adult education and job-skills training.

Identifying causal links between adult education and future employment in substance abusers is a challenging pursuit, but relations between education and employment have been studied in welfare-to-work populations. Descriptive analyses in one large-scale study of a welfare-to-work program suggested that intensive adult education was associated with increased employment and wages (Bos et al., 2002; Gueron & Hamilton, 2002). Unfortunately, the effectiveness of the education intervention was limited because many welfare recipients did not reliably attend training.

If a potential solution for chronic unemployment is training and education, then that training should be designed for optimal adherence (e.g., attendance and interaction with training materials). Unfortunately, similar to welfare recipients, there is evidence that unemployed adults who have histories of chronic drug addiction will not attend training consistently under typical conditions. For example, Silverman et al. (1996) showed that unemployed adults enrolled in methadone treatment did not reliably attend a training program unless they received relatively high stipends that exceeded minimum wage. In that study, unemployed adults who were in methadone treatment for opioid dependence were offered data-entry training for two hours each weekday. By attending training sessions, participants earned monetary vouchers exchangeable for goods and services. Across experimental conditions, the magnitude of the vouchers was manipulated. Attendance was high when the magnitude of the vouchers was high. However, most participants stopped attending when the magnitude of the vouchers was reduced to levels below minimum wage. Other studies (Wong, Dillon, Sylvest, & Silverman, 2004a, 2004b) have shown that unemployed methadone patients may not work complete shifts or arrive on time consistently in a job-skills training program, even when paid a substantial wage for attendance. Importantly, those studies showed that special reinforcement contingencies can be arranged to increase punctuality and the frequency of complete work shifts.

Although reinforcement contingencies can be arranged to ensure that participants attend and spend adequate amounts of time in training each day (Silverman et al., 1996; Wong et al., 2004a, 2004b), those contingencies may not be sufficient to maximize success in training. For example, Silverman et al. (1996) noted considerable variability in the amount of time participants were engaged in the training programs while present at the training sessions. In an effort to promote active participation, Silverman and colleagues adopted a practice of providing a portion of pay in a job-skills training program contingent on attendance and a portion contingent on performance on the training programs (see Donlin, Knealing, Needham, Wong, & Silverman, 2008; Silverman et al., 2007). Participants earned a maximum of $8 per hour in base pay for attending training sessions and approximately $2 per hour in productivity pay for performance on the training programs. The productivity-pay system involved bonuses for correct responses within programs, bonuses for mastering target behavior, and small costs for incorrect responses within programs.

Numerous controlled laboratory studies (e.g., Berger, Cummings, & Heneman, 1975; Bucklin, McGee & Dickinson, 2003; Frisch & Dickinson, 1990; Matthews & Dickinson, 2000; Pritchard, Hollenback, & DeLeo, 1980; Riedel, Nebeker, & Cooper, 1988; Smoot & Duncan, 1997; Yukl, Wexley, & Seymore, 1972) examined the relation between productivity pay and performance. While these studies demonstrated that productivity pay generally increased work output, they were conducted with college students working in simulated laboratory settings, often for relatively brief periods of time and for relatively little overall wages. There are field studies and interventions (e.g., Austin, Kessler, Riccobono, & Bailey, 1996; LaFleur & Hyten, 1995; LaMere, Dickinson, Henry, Henry, & Poling, 1996; Luthans, Paul, & Baker, 1981; Riedel, Nebeker, & Cooper, 1988; Yukl, Latham, & Pursell, 1976) showing that monetary incentives increased performance, but these studies were conducted with employees who had no reported history of chronic unemployment or drug abuse. An exception was a study by Nicol and Hantula (2001), which examined effects of performance goals and feedback on the productivity of adult employees working in a transitional environment for the previously unemployed. In that study, performance goals and feedback increased participants’ productivity.

Although laboratory and field studies indicated that productivity pay increased work output, the question remained as to whether productivity pay could compete with other reinforcers in substance abusing populations who required high-magnitude wages to simply attend job-skills training programs. One study (Koffarnus, Wong, et al., 2013) enrolled homeless, alcohol-abusing adults who were unemployed in a job-skills training program with or without incentives. Monetary incentives were more effective than no incentives in promoting attendance and engagement with training materials. Koffarnus, DeFulio, Sigurdsson, and Silverman (2013) evaluated the same productivity-pay system across 42 injection heroin users in two job skills training programs. Participants were assigned to receive base pay in one training program and base pay plus productivity pay in another program in a multi-element design. Importantly, participants earned approximately the same amount of money in base- and productivity-pay conditions. Participants engaged with and advanced through the program resulting in productivity pay more than they did in the base-pay program. Participants also reported a strong preference for the productivity-pay system over the base-pay system and generally opted to work under the productivity-pay system.

The fact that participants in Koffarnus, DeFulio, et al.’s (2013) study reported a preference for productivity pay contrasts with assumptions about performance effort discounting the value of incentives (e.g., Mazur & Kralik, 1990). Productivity-pay systems typically require sustained attention, consistent or high rates of responding, and receptiveness to feedback to maximize reinforcement. These demands placed on behavior might be taxing and therefore less preferred than a system in which participants are paid simply for attendance. In fact, Long, Wilder, Betz, and Dutta (2012) found that two out of three participants in a check-processing program opted to work for hourly base pay more than productivity pay despite the fact that those participants earned less money in the base-pay system. There are several potential reasons for this inter-study variability in preference including differences in task difficulty (typing versus check processing), contrast between contingencies (frequency and reliability of cues about changes in condition), participant-generated rules, and motivational factors (participant employment). Given the relative complexity of productivity pay over base pay, identifying the conditions under which productivity pay is effective and preferred facilitates data-based decisions concerning implementation of productivity-pay systems.

The present study serves as a systematic replication of Koffarnus, Defulio, Sigurdsson, and Silverman (2013) to determine the extent to which productivity pay increases performance in a job-skills training program of unemployed adults with long histories of heroin addiction. The present study is an extension of that research in that context effects generated by productivity pay occurring in conjunction with base pay were minimized by presenting the two conditions in a within-subjects reversal design. The goals of the study were to evaluate the effectiveness of the two payment systems, examine whether performance reversed when productivity pay was removed, and measure participants’ preferences when they experienced the same system for all training programs. Participants attended a stipend-supported job-skills training program. In the productivity-pay condition, participants were paid base pay for attending training and productivity pay for their performance on the training programs. In the base-pay condition, participants were only paid a base pay wage that was equivalent to their average hourly earnings (base pay plus productivity pay) from the previous productivity-pay condition. At the end of the study, participants completed a survey to identify preferences in pay.

Method

Setting and Participants

This study was conducted in the therapeutic workplace, which is a model employment setting that offers substance-abuse treatment and paid job-skills training. Participants in the present study were four adults who were part of a larger clinical trial (Everly et al., 2011) designed to determine if contingent access to the therapeutic workplace increased acceptance of prescribed depot naltrexone injections. Participants in the clinical trial were recruited from inpatient opiate detoxification programs in Baltimore, MD and through street outreach. Individuals were eligible for the clinical trial if they were between the ages of 18 and 65 years old, were opioid dependent, and unemployed within the last 30 days.

Participants eligible for the clinical trial were randomly assigned to one of two groups and invited to attend the therapeutic workplace for 26 weeks. Participants assigned to the Naltrexone Offered group could attend the workplace independent of whether or not they took the depot naltrexone injections. Participants in the Naltrexone Contingency group were required to take scheduled naltrexone injections to continue attending the workplace and maintain maximum pay. Participants were eligible for the present study if they had been in the main clinical trial for at least 10 weeks. To ensure that sufficient data were obtained to evaluate the relation between payment type and measures of productivity, participants were included in the present study if they consistently attended the workplace and accessed the training programs. The clinical trial and the present study were approved by the Johns Hopkins Medicine Institutional Review Board.

Table 1 shows clinical trial group assignment, number of weeks enrolled in the clinical trial prior to the present study, and demographic characteristics for each of the four participants. Demographic data were obtained at intake from two assessments: The Addiction Severity Index – Lite (McLellan et al., 1985), which assesses substance abuse, employment, legal, medical, psychosocial, and family histories; and the Composite International Diagnostic Interview, 2nd ed. (Cottler, 1991), an instrument that screens for psychiatric disorders such as substance abuse and dependence.

Table 1.

Participant Characteristics at Intake to the Clinical Trial, the Number of Weeks in the Clinical Trial at the Start of the Present Study, and the Group Assignment for the Clinical Trial

Participant
Characteristic S1 S2 S3 S4
Age 48 51 44 51
Sex Male Male Female Female
Race Black Black Black Black
Years of Education 9 14 12 14
Usual Employment Pattern
(Last 3 Years)		 Full Time Unemployed Full Time Unemployed
Monthly Incomea
  Welfare, Pension,
  Benefits,
  Social Security		 $702 $671 $0 $750
  Illegal $0 $6000 $3000 $0
  Other $0 $0 $2000 $0
Money Spent on Drugsa $6000 $640 $5000 $110
Cocaine Dependentb No Yes Yes Yes
Parole/Probation Yes Yes No No
Weeks in Clinical Trial 15 13 16 10
Groupc Contingency Offered Contingency Offered
Open in a new tab

Note. All participant characteristics were self-reported in the Addiction Severity Index – Lite unless otherwise noted.

a

Dollar amounts are estimates from the 30-day period prior to intake.

b

Characteristics were self-reported in the Composite International Diagnostic Interview, 2nd ed.

c

In the clinical trial, participants were assigned to the Naltrexone Contingency group (Contingency) or the Naltrexone Offered group (Offered).

Pre-Experimental Procedure

Since participants were enrolled in the clinical trial for at least 10 weeks prior to the present study, they were experienced with the therapeutic workplace procedures. This section describes the pre-experimental procedure including general procedures for the therapeutic workplace, an overview of protocols for the typing and keypad training programs targeted in the present study, and voucher reviews, which were used to ensure that participants understood the contingencies for earning vouchers.

General therapeutic workplace procedures

The general procedures of the therapeutic workplace intervention are described in this section and in more detail elsewhere (e.g., Everly et al., 2011; Silverman et al., 2007). Participants attended the therapeutic workplace for a maximum of four hours per day, five days per week. Upon arrival at the workplace on Mondays, Wednesdays, and Fridays, urine samples were collected under observation and tested for opiates and cocaine. At the start of each workday, participants were given a personal bar-coded photo ID and reported to an assigned workroom and workstation. A therapeutic workplace staff member swiped the participant’s ID card through a card reader, which signed the participant in or out each time they entered or left the workroom. Signing participants in and out allowed for precise recording of the amount of time spent in the workroom. Participants were paid $8.00 per hour in base pay, approximately $2.00 per hour in productivity pay, and were allotted a 5-minute paid break for each 55 minutes worked. Base pay and productivity pay (described below) were delivered in the form of vouchers which were exchangeable for goods and services. Vouchers were used instead of cash to reduce the likelihood that earnings could be used to purchase illicit substances.

Typing and keypad training programs

Participants worked on custom-developed training programs which were web-based and almost fully automated. All four participants were assigned two types of computer keyboard training programs. One program focused on typing using the alphanumeric keyboard and the other focused on the numeric keypad (see Silverman et al., 2005, for a detailed description of typing and keypad training programs). The typing program was available for two hours in the morning and the keypad program was available for two hours in the afternoon.

Participants engaged with both programs through 1-min practice trials called “timings,” which participants could initiate whenever the programs were available. During a timing, participants copied sequences of characters presented in a random order. Specifically, a sample (i.e., string of characters) was displayed on the screen and participants could respond by typing characters on an entry line just below the sample. After the participant finished typing the sample, they pressed the enter key to move on to the next sample. The computer collected and saved response data and scored participants’ responding. Correct responses were characters that matched the sample with 1:1 correspondence and errors were omitted characters or characters that did not match the sample.

Previous research (Koffarnus, DeFulio, et al., 2013; Silverman et al., 2005) detailed the typing and keypad training programs used in the present study. In short, each program was organized into steps. A step focused on one or more three-character sets and included a mastery criterion. Participants mastered a step when they typed a minimum number of correct characters without exceeding a specified number of incorrect characters within a single timing. To illustrate, in the first step of the keypad program mastered by Participant S1, the sets of target characters (separated by a comma and a space) were 8-0, 7-5, 6-3, 5.0, .45, 67., 4+0, 7+9, 1+5. One set was presented per sample and the mastery criterion was 60 correct characters with no more than three incorrect characters within a 1-min timing. Table 2 shows the duration of each timing, target characters each participant practiced during the experimental procedures, how many three-character sets were presented in each sample, the range of minimum correct characters per timing required for mastery, and the maximum number of errors participants could make in a single timing and still meet the mastery criterion.

Table 2.

Parameters of the Typing and Keypad Programs for each Participant

Timing
Dur (s)		 Target Characters
Keypada 		Target Characters
Typingb 		Min, Max Min Corr
(range)		 Max Inc
S1 60 0 1 2 4 5 6 7 8 9.
− +		 d f j k 1, 1 or
2, 4		 90–240 3
S2 60 0 1 2 3 4 5 6 7 8 9.
− + / *		 r u
a s d f g h j k l;		 1, 1 or
2, 4		 90– 250 3
S3 60 0 1 2 3 4 5 6 7 8 9.
− + / *		 q w p o
a s d f j k l;
z x c v b n m.		 1, 1; 1, 4;
or 2, 4		 90– 250 3
S4 60 0 1 2 3 4 5 6 7 8 9
− + / *		 d f j k 1, 1 or
2, 4		 90– 230 3
Open in a new tab

Note. Timing Dur is the duration in s of a single timing. Target characters were arranged in sets, or groups of 3 characters. Min, Max represents the minimum and maximum number of sets per sample. Min corr and max inc is the minimum number of correct characters and maximum number errors to meet mastery criteria. Because the minimum correct characters varied across steps, a range is presented.

a

Target Characters Keypad could be placed in any position within a set (e.g., .45, 6+7, 42/, 357).

b

Target characters for the typing program are listed in the order found on the keyboard, but were presented in different combinations within a set (e.g., fkd, kdf, dfk, jkd).

Participants could repeat timings until the mastery criterion was met, after which the program advanced to the next step. Some steps increased the response requirement to master a set of characters (which also increased the difficulty of the program) and other steps introduced new sets of target characters. For example, Participant S1 received nine sets of characters (listed above) in the first through third steps of the keypad program. When S1 reached the fourth step of the keypad program, the multiplication and division sign were introduced in six additional sets of target characters (1/8, 9*4, −7+, .09, 63/, 4*4). The fourth through eleventh steps of Participant S1’s keypad training program used the same character sets, but required an increase in the number of correct characters per timing by ten from one step to the next. In the fourth step, S1 was required to have 90 correct characters per min with no more than three incorrect characters. In the fifth step, that requirement was increased to 100 correct characters per min with no more than three incorrect characters, and so on until the eleventh step.

Prior to experimental conditions, participants earned productivity pay for performance in typing and keypad training programs. Participants typically earned $0.03 per 20 correct characters and lost $0.01 per two incorrect characters. Participants also earned bonuses, which were typically $0.10, for meeting the mastery criterion of a step. Although productivity pay varied depending upon individual performance, the system was arranged such that reasonable performances produced approximately $2.00 per hour as in previous research (e.g., Koffarnus, DeFulio, et al., 2013).

The computer program provided participants with information about the response requirements for the current step, contingencies for earning money, and their performance in the previous timing. Before each timing, the mastery criterion for the current step, the amount that could be earned for correct characters and lost for incorrect characters, and the bonus for completing the step was displayed on the screen by the program. After a timing, the program displayed the number of correct and incorrect characters the participant typed, and the amount the participant earned (earnings were not displayed during timings) on the computer screen. If a participant met the mastery criterion for a step, the screen displayed a “congratulations” message and stated that the step was completed.

Voucher reviews and baseline

Voucher reviews were used to ensure that behavior contacted the relevant stimuli correlated with the contingencies. A voucher review consisted of a brief questionnaire that asked participants to report, in writing, information about their earnings. Before and after a timing, participants were asked to report their earnings for correct characters and step completion, and their losses for incorrect characters. Participants were also asked to report their total base pay, productivity pay, and their base-pay rate.

Prior to the experimental conditions, voucher reviews were given once per day until 100% accuracy was achieved on three consecutive days. Once the accuracy requirement was met, S1, S3, and S4 continued to work during a baseline period during which no instructions were given. The baseline period lasted five complete workdays or until timings per hour were stable, and was in place to ensure that potential effects of the voucher reviews did not interact with the experimental manipulation. No lingering effects of the voucher reviews were observed in these three participants. Participant S2 received instructions for the first experimental condition following voucher reviews.

Experimental Procedures

Design and Response Measurement

The present study used a within-subject reversal design (Kazdin, 1982) to evaluate effects of productivity pay on performance on the training programs. The study also used a steady-state design (Kazdin, 1982; Sidman, 1988), in which conditions remained in effect until performance was judged stable (i.e., recent performance was relatively unchanged). Each payment condition was in effect for at least five complete workdays. A workday was considered complete if participants were clocked into the workroom for at least 1 hour. When the rate of timings per hour (number of timings / number of hours worked) across the five most recent workdays were free of monotonic trends, behavior was considered stable and the condition ended. The number of timings per hour was used to determine stability because it was a measure of effort on the training programs independent of individual typing skills or changes in program difficulty.

Response measurement was automated within the custom computer-based training programs. The computer recorded each attempted timing, the number of keystrokes participants made, the number of correct and incorrect characters typed, and the productivity pay earned for each timing. As reported, a staff member swiped each participant’s ID card through a card reader upon entry into and exit from the workplace. This process automatically logged the number of hours participants spent in the workroom each day.

Experimental Conditions

There were two experimental conditions that differed in terms of how participants earned vouchers. In the productivity pay condition, participants earned $8.00 per hour in base pay and approximately $2.00 per hour in productivity pay as described in the typing and keypad training programs procedure. In the base-pay condition, participants were paid an hourly wage that equaled the average hourly earnings (base pay plus productivity pay) from the five stable workdays of the previous productivity-pay condition, but did not earn productivity pay. By equating the hourly pay in the two conditions, we could identify effects of payment type independent of changes in total earnings (however, S1 and S2 received hourly wages in the first base-pay condition that were slightly higher than they should have been due to a calculation error). For each participant, the order of conditions, the number of workdays in each condition, and the total earnings in each condition are shown in Table 3. With the exception of S4, participants were exposed to both conditions at least twice. Participant S4 dropped out of the clinical trial after completing three days in the second productivity-pay condition; hence the behavior in that condition was not stable. Participant S3 completed the clinical trial after four days in the second base pay condition prior to meeting stability criteria.

Table 3.

The Order of Conditions (Cond), Number of Workdays, Hourly Earnings, and Primary (Characters per Hour) and Secondary (Timings per Hour, Percent Correct per Timing, Total Correct Per Timing, and Hours in Workroom) Outcome Measures for Each Participant in Each Sequential Productivity Pay (Prod) and Base Pay (Base) Condition.

Outcome Measures
Pt Cond. #
Days		 Hourly
Earningsa 		Characters per
Hour		 Timings per Hour Percent Correct per
Timing		 Total Correct Per
Timing		 Hours in
Workroom
S1 Prod 5 11.04 6377.5 (511.16) 37.2 (2.2) 95.9 (1.2) 164.3 (6.5) 3.8 (0.2)
Base 10 11.06 1704.3 (1048.13) 9.5 (5.6) 96.4 (1.4) 170.4 (17.7) 3.8 (0.1)
Prod 5 11.26 6083.8 (557.42) 36.6 (2.1) 97.4 (0.4) 162.1 (13.2) 3.6 (0.4)
Base 15 11.26 1684.2 (1073.43) 9.5 (5.7) 96.8 (1.1) 169.7 (15.4) 3.6 (0.3)
S2 Prod 5 12.2 55598.8 (506.62) 40.6 (2.8) 95.6 (1.1) 131.9 (7.5) 3.5 (0.2)
Base 7 12.85 5493.2 (925.56) 36.8 (5.1) 97.5 (0.7) 145.7 (16.0) 3.6 (0.2)
Prod 21 10.91 5845.1 (727.40) 40.7 (3.7) 97.2 (0.5) 140.1 (16.6) 3.3 (0.7)
Base 7 10.91 2839.0 (589.44) 22.3 (4.9) 96.8 (0.7) 123.5 (5.1) 3.4 (0.3)
Prod 11 11.14 4999.6 (1006.09) 34.4 (6.6) 97.5 (0.5) 141.3 (9.2) 2.6 (0.6)
S3 Prod 5 15.58 5043.3 (618.69) 39.1 (3.8) 99.3 (0.2) 128.1 (7.1) 3.4 (0.1)
Base 20 15.58 4442.1 (1081.47) 30.8 (5.2) 98.7 (0.5) 141.7 (22.9) 3.4 (0.4)
Prod 8 12.56 7459.4 (1010.79) 44.5 (4.7) 99.4 (0.1) 166.7 (13.7) 2.9 (0.7)
Base 4 12.56 3793.6 (2038.73) 24.4 (9.9) 98.5 (1.3) 150.5 (43.2) 3.3 (0.3)
S4 Prod 5 12.21 6160.6 (315.49) 43.3 (1.9) 98.4 (0.4) 139.9 (2.9) 4.0 (0.0)
Base 16 12.21 5092.6 (953.93) 36.5 (4.9) 98.3 (0.3) 136.5 (14.4) 4.0 (0.0)
Prod 3 12.71 7273.1 (109.45) 44.5 (1.3) 98.3 (0.4) 160.7 (3.3) 3.7 (0.5)
Grp Prod 6224.8 (876.98) 39.8 (5.0) 97.6 (1.2) 146.2 (17.2) 3.3 (0.7)
Base 3821.2 (1204.00) 24.7 (12.4) 97.1 (1.2) 148.4 (24.0) 3.6 (0.3)
Open in a new tab

Note. Values for the Outcome Measures are means for the entire condition. Standard deviations are in parentheses.

a

Hourly Earnings are the mean earnings (base pay and, if applicable, productivity pay) from the last five sessions of the condition

Written information and Voucher Review

At the start of each condition, participants were given written information describing the payment method for that condition (an exception was S2, who did not receive written instructions prior to the first productivity pay condition). The written information summarized how much the participant averaged per hour in the previous condition (in base pay or base plus productivity pay). If the upcoming condition was a productivity-pay condition, the participants were informed that if they continued to perform on the training program as they had in the past, they would earn approximately the same amount of base pay and productivity pay. If the upcoming condition was a base-pay condition, the participants were informed that they would not be paid productivity pay for their work on the training programs. Participants were also informed that, to ensure that they did not make less money in the base-pay condition, their base pay would be increased to equal what they previously earned when they received both base and productivity pay. To ensure that participants understood the voucher contingencies, participants completed a voucher review (with no accuracy criterion) with a staff member on the first day of each condition.

Preference assessment

At the end of the study, participants’ preference for the two payment methods was assessed. One participant (S1) completed all conditions prior to completing the clinical trial, so he was permitted to choose his payment method for each of his last eight days in the clinical trial. At the end of each workday during the preference assessment, S1 was presented with a sheet that detailed how much he would make according to each of the two payment methods (the information was similar to the written information presented at the beginning of each condition). On days that S1 chose the base-pay condition, the hourly wage was the same as in the last base-pay condition.

All participants except S4 were given a brief preference questionnaire on their last day in the clinical trial. The questionnaire asked participants to indicate which of the two payment methods they preferred, and to choose from a list of options that specified why they preferred the chosen method. Options for the productivity-pay condition were, “I have a chance to make more money if I do really well on the training programs”, and “The training programs give me something to do, but I don’t like to work on them unless I get paid productivity pay.” Options for the base-pay condition were, “I always know how much money I will make,” “Base pay only is easier to understand,” and “I can make the same amount of money even if I do less work on the training programs.” Participants could also write comments about their chosen payment method.

Dependent Variables and Data Analysis

The typing and keypad training programs focused on different parts of the keyboard. Despite the differences between the two programs, the relation between payment type and timings per hour held across typing and keypad programs in previous research (e.g., Koffarnus, DeFulio, et al., 2013). Therefore, we considered the programs to be functionally similar enough to justify combining data. Hereafter, the combined performances on the typing and keypad programs will simply be referred to as “typing”.

Across conditions, the payment manipulation resulted in changes in the proportion of base pay and productivity pay. The measures we analyzed were those that could be affected by changes in base- and productivity-pay earnings. These measures included the number of characters typed per hour and the number of timings completed per hour because productivity pay could only be earned while timings were in progress; the percentage of correct characters per timing (correct characters / total characters) and the number of correct characters per timing (total correct characters / number of timings) because they are aspects of typing performance that determined how much productivity pay was earned during timings; and the number of hours signed into the workroom because of the direct relation between time in the workroom and the amount of base pay earned. The rate of completing training steps could be influenced by two factors: Changes in productivity pay and changes in the mastery criterion (which affected the difficulty of the program). Because the mastery criterion necessarily varied independent of the payment manipulation, an analysis of steps completed was not included. For each measure, means and measures of variability were calculated and compared across productivity pay and base pay conditions for each individual participant.

Results

Figure 1 shows the number of characters participants typed per hour across workdays for each individual participant in each payment condition. The dashed horizontal lines are mean characters per hour for the corresponding condition. Table 3 shows means and standard deviations of the number of characters typed per hour and the number of timings completed per hour for each payment condition for individual participants and the group. In general, participants typed a relatively high number of characters per hour during the productivity-pay conditions, but substantially fewer characters per hour in the base-pay conditions. This pattern of relatively high work output with productivity pay and low output with base pay was also apparent in timings per hour. On average, participants completed approximately 40 timings per hour with productivity pay and 25 timings per hour with base pay. Additionally, changes in conditions resulted in rather abrupt changes in behavior. A robust effect of the payment manipulation was evident for S1 (top panel of Figure 1), who had 0% of overlap in characters typed per hour on days with productivity and base pay (productivity pay minimum was 5324 characters per min, base pay maximum was 4555 characters per min). There was no effect of changing payment type on Participant S2’s characters per hour (second panel) in the first base-pay condition. However, S2 typed fewer characters per hour in the second base-pay condition relative to the second and third productivity-pay conditions. Participants S3 and S4 (third and bottom panels, respectively) both decreased in the number of characters typed per hour during the base-pay conditions. Results for S3 and S4 should be interpreted cautiously, however, because the final conditions were incomplete, and had not yet met the stability criteria. Regardless, rate of typing was sensitive to the payment manipulation in all four participants. Although the magnitude of the changes varied among the individual participants, the number of characters typed per hour decreased when earnings were independent of work output.

Figure 1.

Figure 1

Open in a new tab

The number of characters typed per hour each workday. Each panel shows data from an individual participant. The vertical lines separate the experimental conditions. Labels for the productivity pay condition (Prod) and base pay condition (Base) are located across the top of each panel. The horizontal dashed lines are the mean characters per hour for the entire condition.

Table 3 shows individual and group means (and standard deviations) for other variables that may have been influenced by the payment manipulation: The percentage of correct characters per timing (a measure of typing accuracy), the number of correct characters per timing (a measure of the rate of accurate typing), and the number of hours signed into the workroom each day. Typing accuracy was high and consistent across all conditions with minimal variation. There were minimal differences across conditions in the number of hours that participants were signed into the workroom. For the group, there were no systematic differences between payment types on mean typing accuracy and hours spent in the workroom.

Participant S1, who could choose his payment method on each of his last eight days in the clinical trial, chose the base-pay condition on six of the eight days. Participants S1, S2, and S3 completed the preference-assessment questionnaire, and all three of those participants reported preferring the base-pay condition over the productivity-pay condition. S1 reported that he preferred the base-pay condition because he always knew how much money he would make and because the amount he earned was independent of the number of errors he made. S2 stated that he preferred base pay because, in the productivity-pay conditions, he was not able to make as much money when the training programs increased in difficulty. S3 reported preferring the base-pay condition because she always knew how much money she would make, and her total earnings were consistent even when she did less work. Across participants, the preference for the base-pay condition was influenced by the predictability of the earnings and the fact that the amount earned was independent of variations in the amount or quality of the work they did.

Discussion

The present study was as a systematic replication of Koffarnus, DeFulio, et al. (2013) to examine the relation between productivity pay and performance in training programs in unemployed adults with histories of heroin addiction. Context effects generated by productivity pay occurring in conjunction with base pay were minimized by presenting the two conditions in a within-subjects reversal design. In the productivity-pay condition, daily earnings depended on performance on the training programs and time spent in the workroom. In the base-pay condition, earnings were only contingent upon spending time in the workroom. Participants typed more characters per hour and completed more training trials (i.e., timings) when a portion of their pay was contingent on performance than when all of their pay was contingent upon being in the workroom. In spite of increased output in the productivity-pay condition relative to the base-pay condition, participants reported a preference for the latter payment type.

The present study replicated the reinforcing effect of productivity pay on work output (e.g., Koffarnus, DeFulio, et al., 2013; Long et al., 2012). The contingencies arranged in the payment manipulation, however, may not have been the only influence on the number of characters typed and timings completed per hour. It is possible that the written information participants received at the start of each condition exerted instructional control over behavior (e.g., Catania, Matthews, & Shimoff, 1982). The description of the payment methods may have been responsible for the abrupt changes in the number of characters typed and timings completed per hour, particularly with S1, at the beginning of the conditions. Unfortunately, the design of the present study does not permit conclusions about the relative influence of instructional control independent of the monetary reinforcement for performance. Koffarnus, DeFulio, et al. (2013) found that participants who had a better declarative understanding of contingencies also had a greater increase in performance in productivity over base pay. Future studies can manipulate understanding of instructions directly by providing descriptions of contingencies, omitting those descriptions, or by misinforming participants about changes in payment type.

Although we found an effect of payment type on characters typed per hour and the number of timings completed, some measures of performance were not sensitive to the payment manipulation. Namely, typing accuracy and rate of correct responses within a timing did not differ across productivity- and base-pay conditions. This finding was interesting because payment was delivered for correct responses, there was a cost to incorrect responding, and high rates of typing increased opportunities for payment in the productivity-pay condition. If accuracy and speed of correct responses within a timing were sensitive to the withdrawal of productivity pay, then accuracy and rates of correct responses within timings should have dropped in base-pay conditions. The payment manipulation had little effect on those measures. It is possible that the typing task was sufficiently engaging and low in effort that, once participants initiated a timing, the payment no longer served as a reinforcer. If that was the case, then the measure of response sensitive to the productivity-pay manipulation in the present study was timing initiation.

Initiating contact with training materials might be reinforced by productivity pay, but participants must attend training. When the primary measure of interest is attendance, base pay might be a sufficient incentive. Accordingly, the payment method in the present study did not affect the number of hours participants spent in the workroom. The amount of base pay that participants could earn was contingent upon the amount of time they were signed into the workroom. The base pay rate was $8.00 per hour in the productivity-pay conditions, but was increased in base-pay conditions to encompass the total earnings in the previous productivity-pay condition. Despite changes in base-pay rate, the amount of time participants spent in the workroom did not vary across conditions. An adequate wage was sufficient to keep participants in the workroom regardless of how the wage was earned. In this respect, our results are consistent with the finding that reasonable earnings can maintain reliable attendance in a job-skills training program (Silverman et al., 1996). More noteworthy is the finding that making a portion of earnings dependent on productivity increased the number of characters typed per hour and the number of timings participants initiated.

The present study and previous research (see Bucklin & Dickinson, 2001, for a review) indicate clearly that productivity pay is more effective in maintaining high performance than a flat hourly wage. What is less clear across studies is whether individuals prefer productivity pay or would opt to work for productivity over base pay. In the present study, the three participants who received preference assessments reported preferring base pay over productivity pay. Two (S1 and S3) favored the consistent earnings under base pay and one (S2) may have found the response requirements in productivity pay aversive when faced with increasingly difficult typing programs. Given time constraints in the clinical trial, only one participant in the present study (S1) chose their payment type. That participant selected base pay in a majority of workdays. This preference for a flat hourly wage over productivity pay is consistent with one previous study (i.e., Long et al., 2012), which found that most participants in a check-processing program preferred to be paid hourly.

Under some circumstances, however, participants self-report a preference for productivity pay and elect to work under a productivity-pay system rather than a base-pay system. For example, Koffarnus et al. (2013) found that a majority of participants preferred productivity pay and opted to work for that payment method when those conditions occurred in rapid alternation, but were signaled by different programs. The present study used a survey to measure the preference of three participants. This self-report measure may not correlate highly with other measures of preference. Long et al. (2012) found differences in the payment type one participant reported to prefer and the payment type that participant opted to work under when given the choice. Future studies can address whether preference is stable as a function of measurement strategy, and whether productivity and base pay can be arranged to promote a preference for the more effective payment type. Taken together, the results of the present study and previous research suggest that clear contrasts between payment contingencies, arranging consistent earnings, and providing additional assistance when tasks become difficult may lead to a preference for productivity pay over base pay.

Implications for Job-Skills Training

Bos et al. (2002) and Gueron and Hamilton (2002) found that adult education can benefit those who have limited basic academic and job skills. Unfortunately, keeping people engaged in training programs has been an obstacle to success in adult education (Bos et al. 2002). The present results suggest that, for unemployed substance abusers, it may be necessary to reinforce even the most basic academic and work-related behaviors explicitly. Furthermore, monetary vouchers can function as a reinforcer for initiating contact with training materials. In this regard, the results are consistent with other studies of performance in the therapeutic workplace. For example, Koffarnus, DeFulio, et al. (2013) found that participants completed more work in programs with productivity pay than those with base pay. Wong et al. (2003) showed that ten-fold increases in productivity pay increased work on data entry tasks. Wong at el. (2004a, 2004b) also showed that arranging special reinforcement contingencies increased the frequency with which participants arrived at work on time and stayed for the entire shifts. Collectively, these studies and the present study show that reinforcement contingencies can be arranged to promote contact with training programs, which has the potential benefit of reducing the time and cost needed to train necessary academic and job skills sufficiently.

The present results replicate the well-established finding that pay for performance increases productivity (e.g., LaFleur & Hyten, 1995; LaMere et al., 1996; Matthews & Dickinson, 2000; Smoot & Duncan, 1997; Yukl, Latham, & Pursell, 1976), and extends the finding to a population with the serious social problems of substance abuse, chronic unemployment, and poverty. Participants could work for extended periods of time (four hours per day, five days per week for 26 weeks), and earned substantial wages that possibly equaled or exceeded all of their other sources of legal income.

For chronically unemployed substance abusers living in poverty, monetary incentives need to be high in magnitude to effect behavior change (e.g., Silverman et al., 1996; Wong et al., 2004a, 2004b). It is thus promising that the present study found that an additional payment averaging two dollars an hour can have large effects on productivity. These results are consistent with the findings of Nicol and Hantula (2001); Koffarnus, Wong, et al. (2013); and Koffarnus, DeFulio, et al. (2013), who showed that behavioral interventions can increase the productivity of adults with problematic work histories. Nicol and Hantula used feedback combined with performance goals to improve performances of employees who worked in a transitional environment for the previously unemployed. Koffarnus and colleagues used abstinence and productivity pay contingencies to improve typing skills in substance abusing populations. Thus, various low-magnitude reinforcement contingencies can be used to positively impact the work performance of this clinically important population.

Limitations and Conclusions

Although effects of payment method on the number of characters typed per hour and timings completed per hour was replicated within and across subjects, there are limitations to the generality of the results. The experimental design required that behavior was stable enough to make clear comparisons across conditions. Because of a concern that it might be difficult to obtain sufficient amounts of relatively stable data, participants were included if, for several weeks prior to the start of the present study, they showed consistent attendance and performance on the training programs. Many more participants in the clinical trial were excluded because of substantial variability in their attendance. Considering the impact of the payment manipulation on participants who were fairly consistent workers to begin with, it would be worthwhile to see how it would have affected those who showed variable attendance and performance.

Another limitation of the present study is that the reversal design did not allow for conclusions to be drawn about skill acquisition because mastery criteria varied independently of the payment manipulation. Over the course of the study, Participant S1 mastered 13 steps, S2 mastered 87 steps, S3 mastered 140 steps, and S4 mastered 16 steps. Although this skill acquisition cannot be attributed to the payment manipulation, it is likely that productivity pay promoted mastery. Koffarnus, DeFulio, and colleagues (2013) found that participants mastered significantly more steps with productivity pay over base pay.

In summary, persistent unemployment is a problem in adults with histories of drug addiction. Intensive education and job-skills training programs may be needed to address chronic employment problems (Wong & Silverman, 2007). If so, this study suggests that those programs need to not only provide reinforcement for attendance, but also for performance on training programs if reliable attendance and effort are to be maintained. The development of efficient and effective education and job-skills training programs could be critical to preparing the many adults who have histories of drug addiction and chronic unemployment for gainful and long-term employment.

Acknowledgments

This research was supported by grants R01DA019386, R01DA019497, DA037314 and T32DA07209 from the National Institute on Drug Abuse. The authors wish to thank Anthony DeFulio, Will Aklin, Siggi Sigurdsson, and John Borrero for their helpful comments on this research. The authors also wish to thank Mick Needham, Carina Iati and Donna Glaze for their assistance with collecting the data, and Paul Nuzzo for conducting the statistical analyses.

Footnotes

A portion of these results were presented at the annual meeting of the Association for Behavior Analysis in Chicago, IL (2008). There was no other prior dissemination of the results.

Contributor Information

Shrinidhi Subramaniam, Department of Psychiatry and Behavioral Sciences, Johns Hopkins University School of Medicine.

Jeffrey J. Everly, Department of Psychiatry and Behavioral Sciences, Johns Hopkins University School of Medicine

Kenneth Silverman, Department of Psychiatry and Behavioral Sciences, Johns Hopkins University School of Medicine.

References

  1. Austin J, Kessler ML, Riccobono JE, Bailey JS. Using feedback and reinforcement to improve the performance and safety of a roofing crew. Journal of Organizational Behavior Management. 1996;16(2):49–75. [Google Scholar]
  2. Berger CJ, Cummings LL, Heneman HG. Expectancy theory and operant conditioning predictions of performance under variable ratio and continuous schedules of reinforcement. Organizational Behavior and Human Performance. 1975;14(2):227–243. [Google Scholar]
  3. Bos JM, Scrivener S, Snipes J, Hamilton G, Schwartz C, Walter J. National evaluation of welfare-to-work strategies, improving basic skills: The effects of adult education in welfare-to-work programs. Washington, DC: U.S. Department of Education, Office of the Under Secretary, Office of Vocational and Adult Education; U.S. Department of Health and Human Services, Administration for Children and Families, Office of the Assistant Secretary for Planning and Evaluation; 2002. [Google Scholar]
  4. Bucklin BR, Dickinson AM. Individual monetary incentives: A review of different types of arrangements between performance and pay. Journal of Organizational Behavior Management. 2001;21(3):45–137. [Google Scholar]
  5. Bucklin BR, McGee HM, Dickinson AM. The effects of individual monetary incentives with and without feedback. Journal of Organizational Behavior Management. 2003;23(2–3):65–93. [Google Scholar]
  6. Catania AC, Matthews BA, Shimoff E. Instructed versus shaped human verbal behavior: Interactions with nonverbal responding. Journal of the Experimental Analysis of Behavior. 1982;38(3):233–248. doi: 10.1901/jeab.1982.38-233. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Cottler LB. The CIDI and CIDI-substance abuse module (SAM): Cross-cultural instruments for assessing DSM-III, DSM-III-R, and ICD-10 criteria. In: Harris LS, editor. Proceedings of the Committee on Problems of Drug Dependence 1990 (NIDA Research Monograph 105, DHHS Pub. No. ADM 91-1754. Rockville, MD: Department of Health and Human Services; 1991. pp. 167–177. [PubMed] [Google Scholar]
  8. Donlin WD, Knealing TK, Needham M, Wong CJ, Silverman K. Attendance rates in a workplace predict subsequent outcome of employment-based reinforcement of cocaine abstinence in methadone patients. Journal of Applied Behavior Analysis. 2008;41(4):499–516. doi: 10.1901/jaba.2008.41-499. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Everly JJ, DeFulio A, Koffarnus MN, Leoutsakos JMS, Donlin WD, Aklin WM, Silverman K. Employment-based reinforcement of adherence to depot naltrexone in unemployed opioid-dependent adults: A randomized controlled trial. Addiction. 2011;106(7):1309–1318. doi: 10.1111/j.1360-0443.2011.03400.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Frisch CJ, Dickinson AM. Work productivity as a function of the percentage of monetary incentives to base pay. Journal of Organizational Behavior Management. 1990;11(1):13–33. [Google Scholar]
  11. Gueron JM, Hamilton G. The role of education and training in welfare reform (Policy Brief No. 20) Washington, DC: Welfare Reforms & Beyond, The Brookings Institute; 2002. [Google Scholar]
  12. Holtyn AF, DeFulio A, Silverman K. Academic skills of chronically unemployed drug-addicted adults. Journal of Vocational Rehabilitation. 2015;42(1):67–74. doi: 10.3233/JVR-140724. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Kazdin AE. Single-case research designs. New York: Oxford: 1982. [Google Scholar]
  14. Koffarnus MN, DeFulio A, Sigurdsson SO, Silverman K. Performance pay improves engagement, progress, and satisfaction in computer-based job skills training of low-income adults. Journal of Applied Behavior Analysis. 2013;46(2):395–406. doi: 10.1002/jaba.51. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Koffarnus MN, Wong CJ, Fingerhood M, Svikis DS, Bigelow GE, Silverman K. Monetary incentives to reinforce engagement and achievement in a job-skills training program for homeless, unemployed adults. Journal of Applied Behavior Analysis. 2013;46(3):582–591. doi: 10.1002/jaba.60. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. LaFleur T, Hyten C. Improving the quality of hotel banquet staff performance. Journal of Organizational Behavior Management. 1995;15(1–2):69–93. [Google Scholar]
  17. LaMere JM, Dickinson AM, Henry M, Henry G, Poling A. Effects of a multi-component monetary incentive program on the performance of truck drivers: A longitudinal study. Behavior Modification. 1996;20(4):385–406. [Google Scholar]
  18. Long RD, Wilder DA, Betz A, Dutta A. Effects of and preference for pay for performance: An analogue analysis. Journal of Applied Behavior Analysis. 2012;45(4):821–826. doi: 10.1901/jaba.2012.45-821. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Luthans F, Paul R, Baker D. An experimental analysis of the impact of contingent reinforcement on salespersons’ performance behavior. Journal of Applied Psychology. 1981;66(3):314–323. [Google Scholar]
  20. Magura S, Staines GL, Blankertz L, Madison EM. The effectiveness of vocational services for substance users in treatment. Substance Use and Misuse. 2004;39(13–14):2165–2213. doi: 10.1081/ja-200034589. [DOI] [PubMed] [Google Scholar]
  21. Matthews GA, Dickinson AM. Effects of alternative activities on time allocated to task performance under different percentages on incentive pay. Journal of Organizational Behavior Management. 2000;20(1):3–27. [Google Scholar]
  22. Mazur JE, Kralik JD. Choice between delayed reinforcers and fixed-ratio schedules requiring forceful responding. Journal of the Experimental Analysis of Behavior. 1990;53(1):175–187. doi: 10.1901/jeab.1990.53-175. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. McLellan AT, Luborsky L, Cacciola J, Griffith J, Evans F, Barr HL. New data from the Addiction Severity Index: Reliability and validity in three centers. The Journal of Nervous and Mental Disease. 1985;173(7):412–423. doi: 10.1097/00005053-198507000-00005. [DOI] [PubMed] [Google Scholar]
  24. Nicol N, Hantula DA. Decreasing delivery drivers’ departure times. Journal of Organizational Behavior Management. 2001;21(4):105–116. [Google Scholar]
  25. Pritchard RD, Hollenback J, DeLeo PJ. The effects of continuous and partial schedules of reinforcement on effort, performance, and satisfaction. Organizational Behavior and Human Performance. 1980;25(3):336–353. [Google Scholar]
  26. Riedel JA, Nebeker DM, Cooper BL. The influence of monetary incentives on goal choice, goal commitment, and task performance. Organizational Behavior and Human Decision Processes. 1988;42(2):155–180. [Google Scholar]
  27. Sidman M. Tactics of scientific research: Evaluating experimental data in psychology. Boston: Author’s Cooperative; 1988. (Original work published 1960) [Google Scholar]
  28. Silverman K, Chutuape MAD, Bigelow GE, Stitzer ML. Voucher-based reinforcement of attendance by unemployed methadone patients in a job skills training program. Drug and Alcohol Dependence. 1996;41(3):197–207. doi: 10.1016/0376-8716(96)01252-5. [DOI] [PubMed] [Google Scholar]
  29. Silverman K, Chutuape MAD, Svikis DS, Bigelow GE, Stitzer ML. Incongruity between occupational interest and academic skills in drug abusing women. Drug and Alcohol Dependence. 1995;40(2):115–123. doi: 10.1016/0376-8716(95)01196-x. [DOI] [PubMed] [Google Scholar]
  30. Silverman K, Wong CJ, Grabinski MJ, Hampton J, Sylvest CE, Dillon EM, Wentland RD. A web-based therapeutic workplace for the treatment of drug addiction and chronic unemployment. Behavior Modification. 2005;29(2):417–463. doi: 10.1177/0145445504272600. [DOI] [PubMed] [Google Scholar]
  31. Silverman K, Wong CJ, Needham M, Deimer KN, Knealing T, Crone-Todd D, Fingerhood M, Nuzzo P, Kolonder K. A randomized trial of employment-based reinforcement of cocaine abstinence in injection drug users. Journal of Applied Behavior Analysis. 2007;40(3):387–410. doi: 10.1901/jaba.2007.40-387. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Smoot DA, Duncan PK. The search for the optimum individual monetary incentive pay system: A comparison of the effects of flat pay and linear and non-linear incentive pay systems on worker productivity. Journal of Organizational Behavior Management. 1997;17(2):5–75. [Google Scholar]
  33. Wong CJ, Dillon EM, Sylvest C, Silverman K. Contingency management of reliable attendance of chronically unemployed substance abusers in a therapeutic workplace. Experimental and Clinical Psychopharmacology. 2004a;12(1):39–46. doi: 10.1037/1064-1297.12.1.39. [DOI] [PubMed] [Google Scholar]
  34. Wong CJ, Dillon EM, Sylvest C, Silverman K. Evaluation of a modified contingency management intervention for consistent attendance in therapeutic workplace participants. Drug and Alcohol Dependence. 2004b;74(3):319–323. doi: 10.1016/j.drugalcdep.2003.12.013. [DOI] [PubMed] [Google Scholar]
  35. Wong CJ, Sheppard JM, Dallery J, Bedient G, Robles E, Svikis D, Silverman K. Effects of reinforcer magnitude on data-entry productivity in chronically unemployed drug abusers participating in a therapeutic workplace. Experimental and Clinical Psychopharmacology. 2003;11(1):46–55. doi: 10.1037//1064-1297.11.1.46. [DOI] [PubMed] [Google Scholar]
  36. Wong CJ, Silverman K. Establishing and maintaining job skills and professional behaviors in chronically unemployed drug abusers. Substance Use & Misuse. 2007;42(7):1127–1140. doi: 10.1080/10826080701407952. [DOI] [PubMed] [Google Scholar]
  37. Yukl GA, Latham GP, Pursell ED. The effectiveness of performance incentives under continuous and variable ratio schedules of reinforcement. Personnel Psychology. 1976;29(2):221–231. [Google Scholar]
  38. Yukl G, Wexley KN, Seymore JD. Effectiveness of pay incentives under variable ratio and continuous reinforcement schedules. Journal of Applied Psychology. 1972;56(1):19–23. [Google Scholar]

RESOURCES