Performance pay improves engagement, progress, and satisfaction in computer-based job skills training of low-income adults

Abstract

Advancing the education of low-income adults could increase employment and income, but adult education programs have not successfully engaged low-income adults. Monetary reinforcement may be effective in promoting progress in adult education. This experiment evaluated the benefits of providing incentives for performance in a job-skills training program for low-income, unemployed adults. Participants worked on Typing and Keypad programs for 7 months. Participants randomly assigned to Group A (n=23) earned hourly and productivity pay on the Typing program (Productivity Pay), but earned only equalized hourly pay on the Keypad program (Hourly Pay). Group B (n=19) participants had the opposite contingencies. Participants worked more on, advanced further on, and preferred their productivity pay program. These results show that monetary incentives can increase performance in a job-skills training program, and indicate that payment in adult education programs should be delivered contingent on performance in the training program instead of simply on attendance.

Poverty and lack of a high school education are two of the greatest risk factors for poor health in the United States (Institute of Medicine, 2010; Muennig, Fiscella, Tancredi, & Franks, 2010). Improving education could be an ideal means to improve the lives of poorly educated and low-income populations because increasing education appears to increase income (Bauman and Ryan, 2001; Day and Newburger, 2002). Health policy experts writing for the New York Academy of Sciences concluded that increasing education is one of the best methods to decrease socioeconomic disparities in health (Dow, Schoeni, Adler, & Stewart, 2010).

Despite the need and broad potential benefits of education, very few adults who do not have a high school diploma or equivalent participate in adult education. The U.S. Department of Education, National Center for Education Statistics (2007) estimated that among adults over the age of 16 who did not have a high school diploma or equivalent in 2005, only 1.0% participated in degree or diploma programs, and only 4.2% participated in work-related courses. In their efforts to combat poverty, governments and private foundations have attempted to promote the education of low-income adults, but these efforts have generally failed because of low rates of participation in the available educational opportunities (Hamilton et al., 2001; Porter et al., 2005). For example, a large-scale randomized controlled study evaluated “education-focused” programs for adult welfare-recipients, but failed to show definitive benefit of the education-focused approach because most participants did not attend the education and training programs long enough to reap their potential benefits (Hamilton et al., 2001). Importantly, secondary analyses showed that increased retention and participation in the educational programs were associated with improvements in basic literacy and math skills, increased attainment of GEDs, and greater employment earnings (Bos et al., 2002).

The health and well-being of many low-income individuals could be improved through adult education, but new methods are needed to promote engagement and retention in adult education programs. Extensive evidence from diverse sources suggests that monetary incentives could be used to reinforce engagement and retention of adults in education programs. Monetary incentives have been used to reinforce a range of health behaviors, from simply attending health care visits or getting immunized (Kane, Johnson, Town, & Butler, 2004) to behaviors that are otherwise refractory to change like cigarette smoking (Sigmon & Patrick, 2012), medication adherence (DeFulio & Silverman, 2012), and illicit drug use (see Higgins, Silverman, & Heil, 2008 for review). Incentives also have been used to reinforce employment or work-related behaviors (Berlin, 2007; Bucklin & Dickinson, 2001), including attendance in the therapeutic workplace model used in the current study (Koffarnus et al., under review). Thus, incentives for attending academic and job skills training could be delivered as training stipends, similar to those that are currently and widely offered to college students based on need or merit.

Conditional cash transfer programs offer a model of how governments could use training stipends to reinforce engagement and progress in adult education programs. These programs have been used by many governments of low- and middle-income countries to reduce poverty and improve the health of low-income families by offering cash incentives to low-income families for engaging in health and educational behaviors (Lagarde, Haines, & Palmer, 2007). New York City implemented the first conditional cash transfer program in the U.S. called Opportunity NYC. Under the program, families could earn incentives over a 3-year period for utilizing health services (e.g., medical checkups), for meeting educational goals (e.g., child attendance and achievement), and for meeting employment-related goals (e.g., attending job skills training and sustaining full-time employment). Opportunity NYC had some positive effects, but the monetary incentives, called Family Rewards, had no effect on parent engagement in adult education programs (Riccio et al., 2010). Although 50% of parents did not have a high school diploma or GED at the start of the study and could earn up to $3,000 in family rewards for completing adult education classes, fewer than 2% of the parents earned rewards for attending education or training classes, and only 7.4% of parents ever participated in adult basic education, GED classes, or high school classes. As should be expected based on this low rate of participation in adult education program, Opportunity NYC did not increase attainment of GED certificates or high school diplomas. In considering how the Opportunity NYC family rewards program might be improved, the evaluators considered “reducing the complexity of program, improving families’ understanding of the incentives offer early on, reducing the lag time between meeting conditions and receiving reward payments (p. 259),” among other features. Indeed, family rewards for adult education could be earned only if the parent maintained at least 10 hours per week of employment, and the rewards were given only after the parent completed a course, and then were provided only after some delay.

The existence and success of conditional cash transfer programs throughout the world shows that governments can and do offer incentives for health and education behaviors to improve health and reduce poverty. Opportunity NYC shows that a government within the U.S. will provide incentives for participation in job and academic skills training to promote employment and reduce poverty. However, conditional cash transfer programs, including Opportunity NYC, have typically been implemented with hundreds or thousands of individuals without prior small-scale development. Carefully controlled small-scaled research is needed on design features (e.g., the size and frequency of incentives) that may be critical to effective incentive programs.

Over the past 16 years, we have been developing an intervention for chronic unemployment and drug addiction called the therapeutic workplace in which monetary incentives are used to reinforce a range of academic and professional behaviors needed for employment, and to promote and maintain drug abstinence (Silverman, 2004; Silverman, Defulio, & Sigurdsson, 2012). Stipend-supported adult education is a central feature of the therapeutic workplace intervention and could provide a model of how monetary incentives could be employed to enhance the effectiveness of adult education programs. In the therapeutic workplace’s stipend-supported educational program, low-income and unemployed adults are invited to attend an education and jobs skills training program and paid a stipend for attending the training program and for performance on the training program. Our studies have shown that stipends can promote attendance in training, but they did not evaluate the effects of providing incentives contingent on performance in the training programs (Koffarnus et al., 2011; Silverman, Chutuape, Bigelow, & Stitzer, 1996). Analyses of companies that provide pay for performance and laboratory experiments conducted in analogue work settings have shown that productivity increases when a portion of employees’ total earnings is based on productivity (Bucklin & Dickinson, 2001). However, this principle has not been applied to promote skill acquisition in education programs for low-income, undereducated and unemployed adults.

The purpose of the current experiment was to evaluate the benefits of providing incentives for performance in a job-skills training program for low-income, unemployed adults. All participants worked on two computer-based keyboarding training programs and were randomly assigned to two groups with two different earned pay schedules (productivity pay and hourly pay). Overall pay was yoked (i.e., equalized) on an individual basis across conditions for all participants. This study was designed to assess the effect of performance pay under a situation where overall earning was equal to better isolate and measure this key variable. It was expected that participants would work more and advance more in the training programs in the productivity pay condition than in the hourly pay condition. Various measures of satisfaction and preference were also collected to assess which payment system was more acceptable to the participants.

Method

Setting and Participant Selection

All participants were enrolled in a clinical trial to evaluate the therapeutic workplace intervention in promoting adherence to methadone treatment and abstinence from heroin and cocaine in unemployed, out-of-treatment injection heroin users. The details of the parent trial, when complete, will be reported elsewhere. General therapeutic workplace procedures have been reported previously (Silverman et al., 2005; Silverman et al., 2007).

This study was conducted at the Center for Learning and Health on the Johns Hopkins Bayview campus in Baltimore, MD between January 2010 and July 2011. All 58 of the participants recruited for the parent trial during this time period were included in the present study. Of those 58, 42 completed a 4-week induction period that was a pre-requisite for enrollment in the parent trial, and the results for those 42 participants are reported here. To be included in the parent trial, all participants had to be at least 18 years old, meet DSM-IV criteria for heroin dependence, provide a positive opiate urine sample at intake, self-report of recent injection drug use, have visible track marks, be unemployed, be out-of-treatment (no treatment episodes within 30 days), and reside in Baltimore city. The protocol was approved by the Johns Hopkins Medicine Institutional Review Board and all participants completed an informed consent process and signed consent forms prior to participating in the study. Participant characteristics are detailed in Table 1.

Table 1.

Participant characteristics at study intake

	Group A (n = 23)	Group B (n = 19)	t test p value	Fisher’s exact p value
Percent female (no.)	43% (10)	26% (5)		.5
Age, mean (SD)	42.3 (7.3)	44.8 (6.5)	.3
% African American (no.)	74% (17)	58% (11)		.3
% High school education (no.)	48% (11)	53% (10)		1.0
% Unemployed (no.) a	100% (23)	100% (19)		–
% Opiate dependent (no.) a	100% (23)	100% (19)		–
% Cocaine dependent (no.)	65% (15)	84% (16)		.3
% Married (no.)	22% (5)	32% (6)		.5
% Homeless (no.)	0% (0)	21% (4)		.03
Wide Range Achievement Test standard scores
Reading, mean (SD)	84.0 (11.6)	81.9 (12.4)	.6
Spelling, mean (SD)	84.0 (14.9)	80.6 (14.7)	.5
Math, mean (SD)	84.0 (11.4)	79.2 (12.5)	.2

Note: Opiate and cocaine dependence measured by the Composite International Diagnostic Interview-2. Other demographic information from the Addiction Severity Index.

^aInclusion criteria for the main trial

Experimental Design and Study Groups

While at the therapeutic workplace, participants worked on a variety of training programs. The present study involved two programs designed to teach basic keyboarding skills: “Typing” and “Keypad.” The basic design of the Typing and Keypad programs have been described previously (Dillon et al., 2004; Silverman et al., 2007; DeFulio et al., 2011). The Typing program taught participants to become proficient at using a standard QWERTY keyboard. The Keypad program was similar to the Typing program but was designed to train rapid entry of characters on a numeric keypad. The Typing and Keypad programs are described in more detail below.

During the 4-week induction period at the beginning of the parent trial, trainees worked on the training programs for 2 hours in the morning (10:00 a.m. to 12:00 p.m.) and for 2 hours in the afternoon (1:00 p.m. to 3:00 p.m.). For the final 26 weeks of the parent trial, trainees worked on other training programs in the morning and worked on the Typing and Keypad programs in the afternoon. Access to the Typing and Keypad programs was determined randomly, such that participants did not know which program would be available when they arrived at the workplace. This feature of the study design prevented participants from self-selecting greater exposure to either of the study conditions.

Participants were randomly assigned to one of two groups for the present study. Participants in “Group A” (n = 23) received all of their earnings as an hourly wage while working on the Keypad program. While working on the Typing program a portion of their earnings was based on hourly wage and a portion was based on productivity pay. Specifically, for productivity pay, participants earned and lost money for correct and incorrect characters keyed, respectively, and earned bonuses for each step they passed. Payment conditions for participants randomized to “Group B” (n = 19) were reversed, with an hourly wage earned for Typing and hourly and productivity pay available for Keypad. At the start of the study the hourly wage in the hourly pay condition was set to $10 per hour and the wage in the productivity pay condition was set to $8 per hour in base pay plus whatever was earned in productivity pay. Based on extensive prior experience with this payment system, we expected that participants would earn approximately $2 per hour in productivity pay. Great effort was made to ensure that total money earned across groups and payment conditions was equalized within each subject so that any differences in performance between payment conditions could be attributed to the effects of productivity pay on performance and not the amount earned. To equalize total payment earned, the payment rate in the hourly pay condition for each participant was periodically adjusted (yoked) to be equal to the total amount earned in the productivity pay condition for that person (i.e., average amount earned in productivity pay per hour in the prior week or month plus current hourly base pay). These adjustments were made weekly for the first 4 weeks and monthly thereafter, for nine total adjustments. At each pay adjustment, the amount the participant earned during the last period in the productivity pay condition and the new payment amounts for the hourly pay condition were read to the participant and the participant could ask any questions about these pay amounts. The participants also were explicitly told that the hourly pay rate was being equalized to the total earned in the productivity pay condition and that hourly pay was based on attendance only, not on timings completed.

A short 8-item multiple-choice quiz was then administered on these new amounts to ensure the new payment conditions were understood. In six of the questions, participants were asked to report the payment amounts just read to them. Participants then completed a timing in front of the staff member, and were then asked two questions about the program and productivity pay earned for that timing, ensuring that the participants were aware of how to find this information on their computer screens. If any questions were answered incorrectly on these quizzes, the misunderstood payment condition was reviewed with the participant.

All earnings were paid in monetary vouchers that were automatically added to each participant’s voucher account and displayed on the participant’s computer screen. Participants also earned vouchers for completing monthly assessments for the main trial. Voucher earnings were exchangeable for gift cards to area retailers or payments made to third parties on behalf of participants (e.g., bills or rent payments). Voucher redemption options were flexible as long as they weren’t likely to perpetuate drug use, and a supply of gift cards were kept on-site and were generally available without delay to the participants.

To assess any preferences for one program that may have developed after being exposed to the two payment conditions, some participants (n = 24) were allowed to choose which keying program to work on each afternoon during the final six days of the parent trial. Of those, 18 participants made program choices. At the end of the parent trial, an exit questionnaire was administered that assessed understanding of payment contingencies, keying program preferences, and keying program liking with visual analog scale ratings for each keying program ranging from “Hated It” to “Loved It.”

The Typing and Keypad Training Programs

The Typing program taught participants to type nearly all of the keys on the QWERTY keyboard (see Silverman et al., 2005 for more detailed information). Trainees were presented with a series of characters on a line, and were required to key an identical string of characters in an entry line that appeared directly below the line of presented characters. Characters that matched the criterion characters were considered correct and mismatched characters or omissions were considered incorrect. After 1 min of keying (a “timing”), the number of incorrect and correct characters were displayed on the participant’s screen along with any earnings for those characters. Participants could then initiate a new 1-min timing. The program was arranged in a series of steps. Some steps introduced a small number of new characters, and participants had to type 50–60 characters in a minute with no more than three errors to master these steps. These steps were intermixed with steps designed to increase proficiency that required trainees to key previously-learned characters at increased rates. Initially, participants in Group A earned $0.02 for every 13 correct characters, lost $0.01 for every four incorrect characters, and earned $0.11 for mastering the step. Payment amounts decreased over time to maintain an approximate $2 per hour rate of productivity pay.

The Keypad program included two types of steps: New-Key steps and Fluency-Building steps. Between 1 and 4 Keypad characters (i.e., the ten digits and five symbols on a standard numeric keypad) were presented per line in random order. New-Key steps introduced one new character per step and randomly inter-mixed that new character with all previously trained characters, with 75% new characters and 25% previously trained characters. After introducing three or four new characters, Fluency-Building steps were presented which randomly intermixed all previously trained characters with gradually increasing fluency criteria. On New-Key steps, Participants in Group B earned $0.03 for every 15 correct responses, lost $0.01 for every 2 incorrect responses (except on the first three steps in which no error penalty was imposed), earned $0.15 for mastering each step, and earned a range of bonuses for Fluency steps (up to $3.00). Payment amounts decreased over time to maintain an approximate $2 per hour rate of productivity pay.

Outcome Measures

The primary outcome measure to quantify program achievement was steps completed. Since the two keying training programs have different numbers of steps, this outcome measure was converted into standard scores ([participant steps completed – mean steps completed]/standard deviation of steps completed) for each program individually so that progression on Typing and Keypad could be directly compared. The primary outcome measure to quantify program engagement was the number of 1-min timings initiated per hour in the workplace. Keying accuracy (percentage of characters correct) and keying speed (characters per minute) were measured to characterize other aspects of keying performance. To determine whether the groups were not different on important aspects of the procedure, total hours worked on each training program, obtained pay per hour, and average quiz score were obtained and compared. All performance measures were based on the entire duration of participation in the parent trial except the final 6 days when some participants were allowed to choose their keying program. Participants’ choices of the keying programs during the last 6 days of the trial were compared across groups. Finally, participants were asked to rate “how much they like” the two keying programs on a 100 mm visual analog scales from “Hated It” (0) to “Loved It” (100). Participant were also asked to compare the two programs by answering three additional questions: (1) “Do you like the way you were paid better on Typing or Keypad?”, (2) “Which training program did you work harder on?”, and (3) “If you had to choose to work on only one training program all day, which would it be?”.

Data Analysis

Dichotomous participant characteristics were compared between groups with Fisher’s exact tests, while continuous variables were compared with independent t tests. Steps completed (standard score), timings initiated per hour, keying accuracy, keying speed, total hours worked, and keying program visual analog scale ratings were compared with generalized estimating equations (GEE) using an exchangeable correlation matrix. For each measure, the model included payment condition (hourly pay vs. productivity pay) entered as a within-subject variable, group entered as a between-subject variable, and the interaction between productivity pay condition and group. Planned post hoc comparisons of the payment condition within each group were also conducted. Obtained productivity pay per hour and average quiz score were each compared with a one-way analysis of variance (ANOVA). Keying program choice (percentage choice of Typing) was compared with a Mann-Whitney test. Dichotomous keying program ratings were compared with Fisher’s exact tests. Pearson product-moment correlations were conducted between selected pairs of outcome measures. All statistical tests were conducted in SPSS 17.03 (IBM Corporation, Somers, NY, USA).

Results

Participant characteristics did not differ between groups for any characteristic except the percentage reporting being homeless within the month prior to study intake (see Table 1). Each group completed significantly more steps on the keying program on which they earned productivity pay (Table 2, Figure 1 top). Similarly, each group completed significantly more 1-min timings per hour on the productivity pay condition than in the hourly pay condition (Table 2, Figure 1 bottom). The productivity pay condition engendered faster and more accurate responding than did the hourly pay condition (Table 2). A payment condition by group interaction on keying accuracy also indicated that the effect of payment condition was larger in Group B than Group A.

Table 2.

Mean values of each outcome variable for each keying program for both groups, and the results of GEE tests comparing payment condition, group, and the interaction of the two (Stat = χ², df = 1), unless otherwise noted.

	Group A (n = 23)		Group B (n = 19)		Payment Condition main effect		Group main effect		Payment Condition by Group interaction
	Typing	Keypad	Keypad	Typing	Stat	p value	Stat	p value	Stat	p value
Steps completed (standard score)	0.30 ***	−0.22	0.27 ***	−0.36	31.4	< .001	0.1	n.s.	0.3	n.s.
One-minute timings per hour	28.0 ***	15.0	27.7 ***	14.4	100.5	< .001	0.0	n.s.	0.0	n.s.
Keying accuracy (% characters correct)	90.7	91.7	93.5 ***	80.7	9.2	.002	1.8	n.s.	12.7	< .001
Keying speed (characters per minute)	71.2 *	57.5	57.1	54.0	4.4	.04	1.2	n.s.	1.7	n.s.
Total hours worked	87.4	87.4	93.2	91.9	0.3	n.s.	0.3	n.s.	0.4	n.s.
Obtained pay per hour	$10.26	$10.43	$10.58	$10.64	3.1	n.s.	0.3	n.s.	0.8	n.s.
Average quiz score a	84.6%		82.2%		–		0.4	n.s.	–
Percent choice of keying programs b	73%	27%	90%	10%	–		10.0	.006	–
Percent that selected each keying program in response to questions.
Do you like the way you were paid better on Typing or Keypad? c	91%	9%	89%	11%	–			< .001	–
Which training program did you work harder on? c	95%	5%	89%	11%	–			< .001	–
If you had to choose to work on only one training program all day, which would it be? c	73%	27%	84%	16%	–			< .001	–
Keying program VAS ratings of liking	80.0 *	55.4	82.8 **	63.5	11.3	.001	1.0	n.s.	0.2	n.s.

Note:

Asterisks denote significant effect of planned comparisons between payment conditions within Group A or Group B (* p < .05, ** p < .01, *** p < .001).

VAS = visual analog scale. n.s. = not statistically significant.

^aOne-way ANOVA (Stat = F, df = 1,40).

^bMann Whitney test comparing percentage choice of Typing for each participant as a function of group (Stat = U).

^cFisher’s exact test.

Figure 1.

Steps completed (top), represented as a standard score, and timings initiated per hour (bottom) as a function of group and payment condition. Each point represents an individual participant, and the horizontal lines represent the group means. Asterisks indicate a significant effect of planned comparisons between payment conditions for each group (*** p < .001).

Across all methods of measuring preference, participants preferred the productivity pay condition over the hourly pay condition (see Table 2). Participants overwhelmingly chose the productivity pay program when asked about preference. Similarly, participants overwhelmingly said they worked harder on their productivity pay program and would rather work on their productivity pay program if they had to work on only one program all day. Participants rated their productivity pay program higher on visual analog scales from “Hated It” (0) to “Loved It” (100) for each keying program (see Figure 2) and were more likely to choose to work on their productivity pay program during the final 6 days of the trial (Table 2). Each of these effects was evident in both groups, demonstrating that the payment condition was more influential in determining program preferences than any inherent or pre-existing preferences that may have existed for the Typing and Keypad programs.

Figure 2.

VAS ratings for each program as a function of group and payment condition. Each point represents an individual participant, and the horizontal lines represent the group means. Asterisks indicate a significant effect of planned comparisons between payment conditions for each group (* p < .05, ** p < .01).

Importantly, the pay earned in the hourly and productivity pay conditions were nearly identical. There was a strong correlation between pay earned in the productivity pay and hourly pay conditions for each participant (r = 0.97, p < 0.001) and overall obtained pay did not differ between groups or payment conditions (see Table 2), suggesting that the pay adjustments were successful in equating pay between the two payment conditions within each participant. Additionally, participants were approximately equally likely to receive an hourly pay update amount that was greater than (M = 4.5 pay updates, SD = 1.9) or less than (M = 3.9 pay updates, SD = 2.0) their previous amount (p = .3).

Participants generally answered most of the quiz questions correctly, and there was no significant difference in quiz scores between groups (Table 2). There was a significant correlation between the average quiz score for each participant and the magnitude of the effect of the payment contingencies on timings per hour for that participant, as measured by the difference in average timings per hour in the two payment conditions (r = 0.41, p = 0.007). This demonstrates that those participants who initiated more timings per hour in the productivity pay condition relative to the hourly pay condition were also those participants who had a better understanding of the payment contingencies as measured by the periodic quizzes.

Finally, Figure 3 shows that most participants completed more timings per hour in the productivity condition and sustained that difference fairly consistently over all weeks of the study. Some participants (e.g., S14, S9, S16 and S35) showed a progressive increase in the difference between the productivity and hourly conditions. One of these participants (S16) informally explained her engagement in her hourly pay condition early in the study was due to a desire to learn the keying skills offered, but her level of performance in this condition did not persist throughout the study. Some participants almost stopped working in the hourly pay condition, while continuing to work consistently in the productivity pay condition (e.g., S34, S14, S9, S28 and S35). Some participants showed no difference between the productivity and hourly conditions at any time in the study (e.g., S19, S13), but no participants consistently completed more timings per hour under the hourly pay condition. Most participants maintained fairly high and stable rates of work in the productivity condition. Only one participant (S7) completed more overall timings in the hourly pay condition, but this participant was relatively unengaged throughout the study and informally expressed satisfaction with the $8 per hourly pay, not finding the additional productivity pay worth the increased effort.

Figure 3.

Number of 1-min timings initiated per hour by study week for each participant for the payment conditions and the type of training programs. Participants are sorted by the overall difference in timings initiated between the two payment conditions. The x-axes depict the 4- to 5-week induction period and the first 25 weeks of the main study.

Discussion

Results show that monetary incentives can increase performance in a job-skills training program and indicate that the payment should be delivered contingent on performance in the training program instead of simply on attendance. As we originally expected, participants worked harder and achieved more when a portion of their pay was contingent upon their performance, despite equal earning rates across conditions. The one exception to this trend was with keying accuracy, for which payment condition significantly interacted with group (Table 2). This interaction may have been a result of differences in difficulty between the programs or differences in participant characteristics (i.e., homelessness) affecting the value of monetary incentives. Review of the data from individual participants show that the effects of performance pay can be enormous in some learners: Some individuals almost stopped working completely when pay was solely contingent on attendance in training (i.e., hourly pay), but worked consistently when pay was contingent on performance in the training program. For some participants, it is possible that experiencing both payment conditions may have increased the contrast between them, resulting in bigger disparities in performance. Future research should examine what factors promote sensitivity to this type of payment contingency.

A clear preference among the participants for the use of productivity pay also was evident, both in survey responses and participant choice of payment condition, providing substantial evidence for the acceptability of productivity-based payment contingencies in job-skills training programs. The present results also suggest the importance of effective and clear instructions about payment contingencies, as the beneficial effect of the productivity pay condition was greater in those individuals who had a greater understanding of the payment conditions and scored better on the payment quizzes.

The use of incentives to promote positive behavior change is receiving increased support in the scientific literature, and is receiving concomitant support by governments to counter unhealthy behaviors, particularly in low-income populations. Opportunity NYC stands as a key example of a government health incentive program, and has shown a beneficial impact on a number of health-related outcomes (Riccio et al., 2010). However, Opportunity NYC did not demonstrate an increase in the utilization of job-skills training or educational programs, despite this being one of the targets of the program. The contingencies used in Opportunity NYC differed considerably from the productivity pay contingencies used in this study. Under Opportunity NYC, parents earned incentives for adult education only if they maintained at least 10 hours per week of employment and the rewards were given only after the parent completed a course and then after some delay. The failure of Opportunity NYC’s incentives to increase engagement in adult education program suggests that the contingencies used in that program may not be ideal. Our prior research showed that daily monetary incentives can promote attendance in an adult education program (Silverman et al., 1996; Koffarnus et al., 2011). Importantly, this study also shows that most individuals show a strong preference for earning pay contingent on performance. Taken together, our current and previous research suggest that adult education programs for low-income adults like Opportunity NYC could be highly effective and attractive to participants if the training programs provide frequent small payments contingent on performance and progress in training.

The model used in this study of providing frequent productivity pay for training performance could be effective in developing a broad range of academic and jobs skills that low-income, unemployed adults need to obtain diplomas, degrees, and certifications. A potential limitation of frequent small payments, however, is the increased effort required to arrange and distribute such payments. Additionally, not all behaviors or skills are easily divided into discrete bouts for reinforcement purposes. Frequent delivery of monetary incentives for performance was feasible in our Typing and Keypad programs only because they were computerized. Integrating monetary reinforcement contingencies with computer-based training courses may be ideal for targeting other skills (e.g., mathematics, literacy, or computer skills). The incentives could be considered training stipends, similar to those that are currently and widely offered to college students based on need or merit. As suggested by the current research and a review of employment incentive programs (Bucklin & Dickinson, 2001), the size of the incentives need not be large to be effective. Additional productivity pay in the current study averaged less than $3.00 per hour. The development of stipend-supported computer-based training courses to teach undereducated adults who live in poverty a wide range of academic and job skills could substantially improve the effectiveness of adult education programs for this population, and could improve their education, employment, income, and health.