PARAMETRIC ANALYSIS OF DELAYED PRIMARY AND CONDITIONED REINFORCERS

Abstract

We examined the effects of delayed reinforcement on the responding of individuals with intellectual disabilities. Three conditions were evaluated: (a) food reinforcement, (b) token reinforcement with a postsession exchange opportunity, and (c) token reinforcement with a posttrial exchange opportunity. Within each condition, we assessed responding given (a) a no-reinforcement baseline, (b) immediate reinforcement, and (c) delayed reinforcement, in which responses produced a reinforcer after 1 of 6 delays. Results suggest that delayed food produced greater response persistence than did delayed tokens.

Several reinforcement parameters may influence responding, including rate, magnitude, quality, and delay (Neef, Mace, Shea, & Shade, 1992; Neef, Shade, & Miller, 1994; Pliskoff, Shull, & Gollub, 1968; Reed & Wright, 1988). Delayed reinforcement in particular has been a focus of basic research for decades and has proven to be an exceptionally influential parameter (Chung, 1965; Sizemore & Lattal, 1978). Basic research with nonhumans has examined the effect of delayed reinforcement in the context of response acquisition (e.g., Lattal & Gleeson, 1990; Wilkenfield, Nickel, Blakley, & Poling, 1992), response maintenance (e.g., Schaal & Branch, 1988; Williams, 1976), and response allocation (Chung & Herrnstein, 1967). For example, Chung (1965) examined the effects of delayed reinforcement on the key pecking of pigeons by using a concurrent-schedules arrangement during which pecks on one key resulted in immediate reinforcement on a variable-interval (VI) schedule, and pecks on a second key resulted in delayed reinforcement on the same VI schedule. Chung demonstrated that as the duration of the delay interval increased, a systematic decrease in responding on the delay key was obtained for all pigeons. In addition, results illustrated a range of values at which responding decreased for all pigeons, suggesting a critical range of delays at which the contingent delivery of a stimulus may no longer function as a reinforcer in a concurrent-schedules arrangement when the alternative is immediate reinforcement. Specifically, at delays between 4 s and 8 s, responding decreased on the delay key relative to the immediate key for all subjects. Further, little to no responding was observed on the delay key at delays greater than 12 s for all subjects. Research that has examined delayed reinforcement using a single-schedule arrangement has reported disrupted responding with delays as short as 1 s (e.g., Schaal & Branch, 1988) and 3 s (e.g., Sizemore & Lattal, 1978; Williams, 1976) when delays are unsignaled.

The ubiquitous nature of delay to reinforcement makes it an especially important parameter to study (Stromer, McComas, & Rehfeldt, 2000). Many studies that have examined delayed reinforcement with human subjects examined the extent to which procedures used in basic research can facilitate delay tolerance in clinical settings. For example, basic research with nonhumans has demonstrated that signaled delays produce greater response persistence than unsignaled delays (Lattal, 1984; Schaal & Branch, 1988). In an analogous application, Vollmer, Borrero, Lalli, and Daniel (1999) examined signaled versus unsignaled delays in the context of differential reinforcement of appropriate communication for two individuals with developmental disabilities. Vollmer et al. demonstrated that a communication response could be maintained and aggression suppressed under conditions of delayed reinforcement if the delay was signaled. However, under conditions of unsignaled delays, there was a reemergence of problem behavior as well as a decrease in appropriate communication. In addition, when given a choice within a concurrent-schedules arrangement, subjects in the study by Vollmer et al. were more likely to select the larger delayed reinforcer when that alternative was signaled and were more likely to select the smaller immediate reinforcer when the delay was unsignaled. Thus, when reinforcers cannot be provided immediately, both basic and applied research has demonstrated that signaled delays promote response persistence.

Kelley, Lerman, Fisher, Roane, and Zangrillo (2011) examined the effects of signals during delay fading of a communication response for a variety of reinforcers (i.e., access to materials and escape from task demands). In both the signaled and unsignaled delay conditions, Kelley et al. systematically increased the delay between the communication response and delivery of the reinforcer. In the signaled delay condition, a continuous signal was present throughout the entire delay interval. Kelley et al. found greater response maintenance under equal delays in the signaled condition than in the unsignaled condition for two of three subjects. The results of this study provide further support for the role of signals in promoting response persistence when reinforcers are delayed.

One common arrangement used to facilitate delayed reinforcement in educational settings is the use of token systems (Kazdin & Bootzin, 1972). A token system involves a second order schedule of reinforcement in which a pattern of behavior is reinforced according to one schedule, treated as a unitary response, and is subsequently reinforced according to a second schedule (Kelleher, 1966). As Hackenberg (2009) noted, token systems involve three interconnected schedule components: (a) token production, (b) exchange production, and (c) token exchange. The token-production schedule describes the schedule by which responses produce tokens. The exchange-production schedule describes the schedule by which opportunities to trade tokens, for back-up reinforcers, is produced, and the token-exchange schedule describes the ratio of individual tokens to back-up or primary reinforcers. Although token systems are often used when immediate delivery of primary reinforcers is not feasible, little is known about the extent to which delays affect the efficacy of tokens themselves (i.e., as conditioned reinforcers). In basic research that has examined conditioned reinforcement, tokens are typically delivered immediately, and primary reinforcer provision is delayed. Although a delay is unlikely and unnecessary in laboratory procedures, it is possible that token delivery itself may not be immediate in clinical settings. Delivery of conditioned reinforcers in laboratory settings is often programmed and delivered via an experimental apparatus (e.g., a food hopper in the pigeon laboratory). By contrast, delivery of conditioned reinforcers in clinical settings frequently involves the behavior of another person (e.g., clinician or educator). In addition to the delivery of programmed reinforcers (primary or conditioned), clinicians often have other competing responsibilities (e.g., teaching, monitoring of other students) that may contribute to the delayed provision of either primary or conditioned reinforcers.

Although basic research has demonstrated that contingent token delivery will maintain responding at levels similar to food-reinforced behavior (Kelleher & Gollub, 1962), little is known about the direct effects of delayed token delivery on token-reinforced behavior. However, Jackson and Hackenberg (1996) reported a series of four experiments in which they examined pigeons’ choices for immediate versus delayed token delivery. Jackson and Hackenberg found that when delays to and amount of food were equal across conditions, pigeons’ choices between immediate and delayed token delivery were nearly indistinguishable. In the context of choice, Hyten, Madden, and Field (1994) similarly showed that delays to token delivery were less influential than delays to exchange periods.

In addition to basic research on choice, research on temporal discounting with human subjects provides a strong conceptual basis for interpreting the effects of delay on contingent primary or conditioned reinforcers. In brief, several studies in the discounting literature have suggested that when given a choice between a smaller reinforcer now and a larger reinforcer later, subjects will switch to the smaller-sooner reinforcer more quickly if primary reinforcers are used (e.g., alcohol, food) instead of conditioned reinforcers (e.g., tokens or money; Bickel, Odum, & Madden, 1999; Estle, Green, Myerson, & Holt, 2007; Madden, Petry, Badger, & Bickel, 1997; Odum & Rainaud, 2003; Petry, 2001). These general findings suggest that token systems may promote greater delay tolerance compared to behavioral programs that involve primary reinforcers alone.

In sum, previous research on delayed conditioned reinforcers suggests that (a) nonhumans are relatively insensitive to delayed token delivery in concurrent choice arrangements and (b) humans discount conditioned reinforcers less steeply than primary reinforcers. Given these findings, we sought to evaluate whether the delayed delivery of conditioned reinforcers produces greater response persistence under increasing delays compared to delayed delivery of primary reinforcers under single-schedule arrangements. Despite the fact that delays to (a) primary reinforcers, (b) conditioned reinforcers, and (c) exchange opportunities are highly probable in the application of token systems, the effects of these delays have not been comprehensively examined for the behavior of individuals with intellectual or developmental disabilities. Although much translational research in behavior analysis has involved the extension of laboratory principles to clinically important problems (Mace & Critchfield, 2010), this particular line of inquiry represents a different approach. Mace (1994) suggested that practical problems should set the occasion for translational work that will ultimately lead to a greater understanding of behavioral principles and an extension of fundamental basic knowledge. That is, although basic research has not directly examined the effects of delayed conditioned reinforcers in single-schedule arrangements, examination of the effects of delay under these circumstances is warranted because this is a likely scenario in application. Specifically, the objectives were (a) to assess how increasing the delay to the contingent delivery of a stimulus would affect responding and (b) to examine what, if any, differences would be observed under conditions of delayed provision of a primary reinforcer, delayed provision of a conditioned reinforcer with a postsession exchange opportunity, and immediate delivery of a conditioned reinforcer with a posttrial delayed exchange opportunity.

METHOD

Subjects and Setting

Three children with intellectual disabilities who had been admitted to an inpatient hospital for the assessment and treatment of severe behavior disorders served as subjects. David was a 13-year-old boy who had been diagnosed with autism and intellectual disability (severity unspecified). David communicated vocally in one- and two-word phrases. Chris was a 7-year-old boy who had been diagnosed with autism; mood disorder, not otherwise specified; attention deficit hyperactivity disorder (ADHD); and moderate intellectual disability. Chris had limited verbal skills and communicated primarily with gestures. Alex was an 8-year-old boy who had been diagnosed with pervasive developmental disorder, not otherwise specified; ADHD; and mild intellectual disability. Alex had an extensive verbal repertoire and communicated vocally in full sentences. All subjects could follow simple one-step instructions.

Sessions with David were conducted in a small bedroom located in the hospital (approximately 6 m by 3.6 m). The room contained a patient bed, a table, and several chairs. Sessions with Chris and Alex were conducted in small rooms (approximately 2.4 m by 3 m) located in the hospital that were equipped with one-way observation panels. Rooms contained one table and two chairs.

Materials

Task materials remained constant for each subject for the duration of the experiment. The target response was selected from each subject’s individualized education plan and was one that the individual could complete independently in the absence of prompting. The tasks selected were putting towels in a bin, matching colors, and folding towels for David, Chris, and Alex, respectively. Both primary and conditioned reinforcers remained constant for the entire experiment for each subject. In addition, back-up reinforcers during the token analysis and exchange analysis were the same as those used during the food analysis. That is, if a subject earned chips in the food analysis, tokens were exchangeable for chips in the token and exchange analyses.

Response Measurement and Interobserver Agreement

During the preference assessment (described below), data were collected with paper and pencil on selection and consumption. Selection was defined as reaching toward or asking for a stimulus by name. Consumption was defined as the edible item passing the plane of the lips. Occurrence data were collected for selection and consumption for each preference-assessment trial. The number of selections was divided by the number of opportunities and the result was converted to a selection percentage for each item. During the delay analysis, trained observers collected data on laptop computers. Observers collected data on (a) the number of correct responses completed, (b) consumption of the reinforcer, (c) token exchange, and (d) frequency of problem behavior. Four experimenter responses were also measured: (a) task presentation, (b) delivery of the reinforcer, (c) exchange signal (exchange analysis only), and (d) presentation of the “stop” stimulus (described in detail under general procedure). The primary dependent variable was total number of responses per session.

Interobserver agreement was assessed for all portions of the study. Agreement for the preference assessment was defined as both observers recording the same selection during each trial. Agreement percentages were calculated by dividing the number of agreements, on a trial-by-trial basis, by the number of agreements plus disagreements and converting the result to a percentage. Interobserver agreement data for the preference assessment were assessed for 100%, 100%, and 33% of sessions and had means of 100%, 97% (range, 95% to 98%), and 100% for David, Chris, and Alex, respectively. Interobserver agreement for the delay analysis was assessed using the block-by-block method (Mudford, Martin, Hui, & Taylor, 2009). Each session was divided into consecutive 10-s intervals. Intervals in which both observers scored the exact same number of responses were assigned a value of 1 (including intervals in which no responses were scored). Intervals in which only one observer scored an instance of the target behavior were assigned a value of 0. For intervals in which observers scored different instances of the target response, a proportion was calculated by dividing the smaller number by the larger number. These measures were summed, divided by the total number of intervals, converted to a percentage, and averaged across sessions. Interobserver agreement data for the delay analysis were assessed during 49%, 28%, and 31%, of sessions and produced means of 98% (range, 84% to 100%), 96% (range, 89% to 100%), and 94% (range, 75% to 100%), for task completion for David, Chris, and Alex, respectively.

Stimulus Preference Assessment

Before the experiment, all subjects were exposed to a stimulus preference assessment. Alex and David were exposed to a multiple-stimulus-without-replacement preference assessment (MSWO; DeLeon & Iwata, 1996). Eight items were assessed. Before the assessment, subjects were prompted to sample each edible item. Each item was placed on an individual plate, and plates were arranged in a quasirandom sequence in front of the subject. Subjects were prompted to select one item. Selection of an item resulted in access to the item. The item was not replaced in the sequence. The remaining items were rearranged, and the subject was prompted to select one item from those remaining in the array. This procedure was repeated until each item had been selected. The entire procedure was repeated six times for Alex and three times for David, and results were averaged.

Chris was exposed to a paired-stimulus preference assessment using procedures similar to those described by Fisher et al. (1992). Twelve edible items were assessed. Before the assessment, Chris was prompted to sample each item. Edible items were presented in pairs, and he was prompted to choose one. Selection of one item resulted in access to that item and removal of the nonchosen item. If he attempted to select both items at once, the items were removed and re-presented. This procedure was repeated until each item had been paired with every other item once. The entire procedure was repeated three times, and results were averaged. For both preference assessment procedures, selection percentages were calculated by dividing the number of times each item was selected by the number of times it was available and converting the result to a percentage. We selected the following items that resulted in the highest selection percentage for use in the delay analysis: chips (for David), M&Ms (for Chris), and Raisinets (for Alex).

Token Training

Token training was conducted with David and Chris before the token analysis began. During token training, subjects were first taught to exchange tokens for primary reinforcers and then to earn tokens that were exchanged after the session. The mastery criterion for token training was three consecutive sessions during which the subjects (a) earned 10 tokens and (b) independently exchanged each token for a single back-up reinforcer on 90% of trials. The sequence of the token-training procedure is described in the Supporting Information. A token system was part of Alex’s behavioral treatment before his participation in this study. Therefore, token probes were conducted at the terminal criterion (i.e., 10 earned tokens). Alex met mastery criterion during the token probes; therefore, token training was not conducted. (Note that the tokens used for Alex in this study were qualitatively different from those used as part of his behavioral treatment.) For all subjects, tokens were poker chips. For Chris, tokens were placed on a token board (a laminated card-stock construction board affixed with Velcro). For Alex and David, tokens were placed in a translucent plastic container.

GENERAL PROCEDURE

Before the start of each session, one prompted exposure trial was conducted with each subject. However, three prompted exposure trials were conducted for the first session of each new delay. After the prompted trial, the experimenter said, “You can work if you want.” There were no additional instructions or prompts to complete the task. A brief auditory stimulus (buzzer) sounded when the response requirement had been met. There were no programmed stimuli to signal the duration of the delay. Sessions were arranged in a trial-based format and were terminated according to one of three criteria: (a) The subject earned 30 reinforcers; (b) the subject did not respond (complete a task) for 2 consecutive minutes, measured from the time the task materials were presented; or (c) the subject communicated that he was finished working (Alex only). To restrict the opportunity to work during the delay, instructional materials were removed when the subject met the response requirement and were reintroduced 5 s after the delivery of the reinforcer. Removal of the task materials was considered important because any intervening responding may have been adventiously reinforced and thus complicated interpretations of the effect of the delay (Sidman, 1960).

Three separate analyses were conducted to evaluate the effects of delayed food, delayed token delivery, and delayed exchange opportunities, respectively. For each analysis, subjects were exposed to the experimental phases in the following order: (a) no-reinforcement baseline, (b) immediate reinforcement (i.e., 0-s delay), and (c) delayed reinforcement (delay values assessed for each subject are summarized in Table 1). During all phases, a minimum of three sessions was conducted, and sessions continued until stable responding was observed.

Table 1.

Delay Values (in Seconds) Assessed in Each Condition

Subject	Food analysis	Token analysis	Exchange analysis
David	0, 3, 6, 10, 20, 60	0, 3, 6	0, 3, 6, 10, 20, 60
Chris	0, 3, 6, 10	0, 3, 6	0, 3, 6, 10
Alex	0, 3, 6, 10, 20, 60, 120	0, 3, 6, 10, 20, 60

The first phase in each analysis was a no-reinforcement baseline, during which there were no programmed consequences for task completion. After the no-reinforcement baseline, subjects were exposed to an immediate reinforcement phase during which responses produced reinforcers (i.e., food or token) on a fixed-ratio (FR) 1 schedule, and reinforcers were delivered immediately. The immediate reinforcement phase was carried out to determine levels of responding when there was no delay and served as point of comparison to phases in which a delay was imposed. The immediate reinforcement phase was followed by a series of delayed reinforcement phases during which responses produced reinforcers on an FR 1 schedule following one of six delays. During the immediate reinforcement phase, after completion of the schedule requirement, the buzzer sounded and the reinforcer was delivered immediately. During the delayed food and token reinforcement phases, after completion of the schedule requirement, the buzzer sounded immediately and the reinforcer (food or token) was delivered after a programmed delay. During the delayed exchange phases, after completion of the schedule requirement, the buzzer sounded and a token was delivered immediately. However, the opportunity to exchange was delayed. Delays were systematically increased across sessions. During the delayed reinforcement phases, delay values were increased until, at minimum, responding decreased by 50% or greater relative to responding during the last three sessions in the immediately preceding phase. The only exception was the food analysis for Chris, during which responding at the largest delay value (6 s) decreased by 45% relative to the previous phase¹. After completion of the delayed reinforcement phase within each analysis, a reversal to the immediate reinforcement phase was implemented to ensure that any decreases in responding observed were a function of increases in delay and not a global decline in reinforcing efficacy of the stimulus. All subjects were exposed to the conditions in the following order: food analysis, token analysis, and exchange analysis.

Delay Analysis

Food Analysis

No-reinforcement baseline

During baseline, subjects had the opportunity to complete up to 30 responses; however, there were no programmed consequences for task completion. Sessions in the no-reinforcement baseline and in all subsequent phases were terminated as previously described. No-reinforcement baseline sessions were identical for the food, token, and exchange analyses.

Immediate reinforcement (0-s delay)

During the immediate reinforcement phase of the food analysis, task completion produced an edible item immediately. Subjects could consume the reinforcer immediately or save it and consume it at the end of the session in this and subsequent phases. Alex occasionally accumulated two or three edible items.

Delayed reinforcement

In delayed reinforcement phases of the food analysis, task completion produced an edible item following one of six programmed delays. For example, in the 6-s delay phase, an item was delivered 6 s after the subject completed the target task.

Token Analysis

No-reinforcement baseline

The no-reinforcement baseline of the token analysis was identical to the no-reinforcement baseline described above.

Immediate reinforcement (0-s delay)

The immediate reinforcement phase of the token analysis was similar to the immediate reinforcement phase of the food analysis except that a token was delivered contingent on each response (instead of an edible item) and tokens were exchangeable for the back-up reinforcer after the session. That is, in the immediate reinforcement phase of the token analysis, task completion resulted in immediate token delivery. Subjects accumulated tokens during the session and exchanged them for back-up reinforcers immediately after the session. Recall that a session ended after 30 responses had been completed, 2 min had passed without responding, or the subject had communicated that he was finished working (Alex only). Back-up reinforcers used in the token analysis were the same as those used in the food analysis. There was a 1:1 correspondence between token trade-in and edible-item delivery, and tokens were traded in one at a time during the postsession exchange period.

Delayed reinforcement

The delayed reinforcement phase of the token analysis was identical to the immediate reinforcement phase except that task completion resulted in token delivery after a programmed delay. That is, the delay was imposed between task completion and delivery of the token. For example, in the 6-s delay phase, a token was delivered 6 s after the subject completed the schedule requirement. There was no delay between token exchange and edible-item delivery during the exchange period.

Exchange Analysis

No-reinforcement baseline

The no-reinforcement baseline of the exchange analysis was identical to the no-reinforcement baseline described above.

Immediate reinforcement (0-s delay)

In the immediate reinforcement phase of the exchange analysis, task completion resulted in immediate token delivery and an immediate opportunity to exchange the token for the back-up reinforcer after every earned token (rather than after the session). The opportunity to exchange was signaled by moving the token board or container within arm’s reach of the subject, and the experimenter extended a hand in front of the subject. That is, after completion of each target response, the experimenter placed a token on the token board or in the token container and simultaneously extended a hand, which signaled the exchange period. Similar to the token analysis, the token board or container was in sight but not within reach of the subject. There was no delay between token exchange and edible-item delivery.

Delayed reinforcement

In the delayed reinforcement phases of the exchange analysis, task completion resulted in immediate token delivery and an exchange opportunity after a programmed delay. That is, the delay was imposed between the delivery of the token and the exchange opportunity. For example, in the 6-s delay phase, a token was delivered immediately after completion of the target response. After 6 s, the experimenter extended a hand and moved the token board or container toward the subject, signaling the exchange period. Note that this analysis differed from the token analysis in that subjects accumulated tokens during the token analysis (that were exchanged after the session) but had the opportunity to exchange after every earned token in the exchange analysis.

Procedural Integrity

We calculated individual obtained delay values for each reinforcer delivery and compared the mean obtained delay to the programmed delay for each session. Individual obtained delay values were determined by calculating the time (in seconds) between completion of the task and delivery of the reinforcer (food, token, or exchange signal in the food, token, and exchange analyses, respectively). Procedural integrity data are depicted in Table 2.

Table 2.

Mean Obtained Delay (in Seconds) and Mean Differential Values

Programmed delay (in seconds)	Food analysis		Token analysis		Exchange analysis
	Mean obtained delay	Mean differential	Mean obtained delay	Mean differential	Mean obtained delay	Mean differential
David
	0.70	0.7	0.81	0.81	1.22	1.22
0
3	4.57	1.57	4.14	1.14	4.78	1.78
6	7.80	1.8	7.34	1.34	6.87	0.87
10	11.12	1.12			11.67	1.67
20	20.34	0.34			21.37	1.37
60	60.64	0.64			62.0	2.0
120	118.33	−1.67
0	1.25	1.25	0.81	0.81	1.33	1.33
Chris
0	1.16	1.16	1.12	1.12	0.82	0.82
3	4.64	1.64	4.03	1.03	4.09	1.09
6	8.14	2.14	8.10	2.1	7.90	1.9
10					12.18	2.18
0	0.72	0.72	0.57	0.57	0.56	0.56
Alex
0	0.59	0.59	0.61	0.61
3	4.59	1.59	3.98	0.98
6	8.98	2.98	8.21	2.21
10	17.21	7.21	13.2	3.2
20	25.01	5.01	23.66	3.66
60	61.26	1.26	63.0	3.0
120	121.85	1.85
0	1.01	1.01	0.69	0.69

RESULTS

Figures 1 through 3 depict the total number of responses completed per session for David, Chris, and Alex, respectively, during the food analysis, token analysis, and exchange analysis (David and Chris only). During the no-reinforcement baseline phase, David’s responding was low and decreased to near-zero levels during all analyses. Further, he completed all 30 responses during each immediate reinforcement condition. During the delayed reinforcement phases of the food analysis, he continued to complete all 30 responses during the majority of sessions until a 60-s delay was introduced. We calculated the percentage decrease in responding across phases by calculating the mean of the last three sessions in each phase and comparing responding to the last three sessions in the immediately preceding phase. During the 60-s delay phase, responding decreased by 71% relative to the previous phase and further decreased by an additional 73% (relative to 60 s) during the 120-s delay phase. That is, a delay of 120 s resulted in a 92% decrease in responding relative to immediate reinforcement. Thus, for David, a delay of 60 s did not maintain responding at levels that might be considered to be acceptable work levels, but a delay of 20 s did. Figure 1 (middle) depicts responding during the token analysis. Recall that during the token analysis, the delay was imposed between completion of the target response and delivery of the token. During the token analysis, responding quickly decreased during the delayed reinforcement phase. A 3-s delay produced a 43% decrease relative to the immediate reinforcement phase, and a 6-s delay produced an additional 88% decrease relative to responding in the 3-s delay phase. That is, a 6-s delay produced a 93% decrease relative to immediate reinforcement. Figure 1 (bottom) depicts David’s responding during the exchange analysis. Recall that during the exchange analysis, tokens were delivered immediately, and the opportunity to exchange was delayed. Responding persisted at high levels during the 3-s, 6-s, 10-s, and 20-s delay phases. A 60-s delay resulted in a 99% decrease in responding relative to the previous phase. David completed all 30 responses during the majority of sessions during the reversal to immediate reinforcement during all three analyses.

Figure 1.

Frequency of responses completed per session for David during the food analysis (top), token analysis (middle), and exchange analysis (bottom). The 0-s delay is synonymous with the immediate reinforcement phase, described in the text. Sr = reinforcement.

Figure 3.

Frequency of responses completed per session for Alex during the food analysis (top) and token analysis (bottom). The 0-s delay is synonymous with the immediate reinforcement phase, described in the text. Sr = reinforcement.

Chris’s responding in the food analysis (Figure 2, top) was more variable than the other two subjects. However, he completed more than 22 responses during four of five sessions in the 3-s delay phase. Responding decreased by 45% relative to the previous phase during the 6-s delay phase. Figure 2 (middle) shows responding during the token analysis. Responding did not initially increase during the immediate reinforcement phase of the token analysis. Token training was conducted before the start of the no-reinforcement baseline for this analysis. Therefore, additional token training was conducted after Session 10. After the token-training booster sessions, Chris completed all 30 responses for five consecutive immediate reinforcement sessions. A 3-s delay resulted in a 66% decrease in responding relative to the immediate reinforcement phase, and a 6-s delay further decreased responding by an additional 52% (relative to 3 s). That is, a 6-s delay resulted in an 83% decrease in responding relative to immediate reinforcement. Figure 2 (bottom) shows responding during the exchange analysis. Responding generally persisted at near-maximum levels during the 3-s and 6-s phases. A 10-s delay resulted in a 97% decrease in responding relative to the previous phase. Chris completed all 30 responses during most sessions (i.e., seven of eight sessions) during the reversal to immediate reinforcement during the food and exchange analyses. Responding was generally lower and more variable during the reversal to immediate reinforcement in the token analysis. However, a clear increasing trend was evident during the return to immediate reinforcement.

Figure 2.

Frequency of responses completed per session for Chris during the food analysis (top), token analysis (middle), and exchange analysis (bottom). The 0-s delay is synonymous with the immediate reinforcement phase, described in the text. Sr = reinforcement.

Figure 3 depicts Alex’s responding during the food and token analysis. (Recall that Alex did not complete the exchange analysis.) Similar to the results obtained for David, Alex completed all 30 responses during the majority of the delayed reinforcement phases during the food analysis (top). Responding decreased during the 120-s delay phase by 71% relative to responding in the previous phase. During the token analysis (bottom) responding persisted at near-maximum levels during the 3-s, 6-s, 10-s, and 20-s delay phases. A 60-s delay resulted in a 99% decrease in responding relative to the previous phase. Responding increased to near-maximum levels during the reversal to immediate reinforcement in both analyses. Notably, responding during the reversal to immediate reinforcement during the token analysis was more variable than during all other immediate reinforcement phases (i.e., food and exchange analysis) for all subjects.

Figure 4 depicts the mean number of responses completed in the last three sessions at each delay value for David, Chris, and Alex. As illustrated by the session-by-session data depicted in Figures 1 through 3, responding decreased as a function of delay within each condition for all subjects. In addition, responding decreased at smaller delay values in the delayed token condition relative to both the delayed food and delayed exchange conditions for all subjects.

Figure 4.

Mean number of responses completed in the final three sessions at each delay value during the delayed food (filled circles), delayed token (open triangles), and delayed exchange (open diamonds) analyses for David (top), Chris (middle), and Alex (bottom). Sr = reinforcement.

Figure 5 depicts the mean latency to respond at each delay value for David (top), Chris (middle), and Alex (bottom). We calculated the mean latency to respond by taking the mean of all individual response latencies within each delay phase. Individual latencies were determined by calculating the time between presentation of the task materials and initiation of the task. Recall that task materials were removed after task completion and were re-presented after reinforcer delivery. Overall, there was a positive relation between latency to respond and the delay value for each individual analysis for all subjects. That is, as the delay value increased, the latency to initiate the task also increased. For David, the latency to respond was relatively stable during both the food and exchange analyses between 0-s and 10-s delays. The latency to respond increased during the 20-s delay phase for both delayed food and delayed exchange roughly equally. Further increases in the delay (i.e., 60 s) produced a longer mean latency in the exchange analysis than in the food analysis. However, it should be noted that David completed only one response in the 60-s delay phase of the exchange analysis; therefore, the mean latency depicted for that phase is based on a single measurement opportunity. The mean latency to respond during the token analysis was higher than the mean latencies to respond in both the food and exchange analyses at each delay value. Specifically, the mean latency to respond was roughly three times as high in the token analysis than in the food and exchange analyses at 0-s and 3-s delay and roughly 10 times as high at a delay of 6 s. The mean latency to respond for Chris generally increased with increases in the delay interval during all analyses. Mean latencies were nearly equal during the 0-s delay phase across analyses. Mean latencies were similar during the food and token analyses at 3-s and 6-s delays. However, the mean latencies during the exchange analysis were significantly lower than those obtained during the food and token analyses at 3-s and 6-s delays and remained low at a delay of 10 s. The mean latency to respond for Alex was relatively low and roughly equal across analyses during the 0-s, 3-s, 6-s, and 10-s delay phases. The mean latency to respond increased at 20 s during the token analysis and increased further at 60 s. These data corroborate findings reported in the basic literature with nonhuman subjects that showed that response rate decreases as a function of delay (Pierce, Hanford, & Zimmerman, 1972).

Figure 5.

Mean latency to respond at each delay value during the delayed food (filled circles), delayed token (open triangles), and delayed exchange (open diamonds) analyses for David (top), Chris (middle), and Alex (bottom).

DISCUSSION

We examined the effects of delayed food delivery, delayed token delivery, and immediate token delivery with delayed exchange opportunities on the responding of three individuals with developmental disabilities. Overall, as the delay to reinforcement increased, the total number of responses completed per session decreased across all programmed contingencies. Although there were considerable differences in the value at which responding deteriorated across subjects, a similar pattern emerged, in that delayed food generally produced the greatest response persistence and delayed tokens produced the most rapid decreases in responding across increasing delays for all subjects. Immediate token delivery with a delayed exchange opportunity resulted in decreased responding at the same delay value as delayed food for one subject (David) and at a greater value than delayed food for another subject (Chris).

Another consistent pattern observed across subjects was that responding during the reversal to immediate reinforcement at the end of the token analysis was more variable than responding during the corresponding phase of the food analysis (all subjects) and exchange analysis (David and Chris). In the food and exchange analyses, the return to immediate reinforcement resulted in maximum responding (i.e., 30 reinforcers earned) in all sessions for David, seven of eight sessions for Chris, and all sessions for Alex (food analysis only); whereas the return to immediate reinforcement in the token analysis resulted in maximum responding in two of four sessions for David, two of eight sessions (with five of the remaining six sessions below 20 responses) for Chris, and six of nine sessions for Alex (with the remaining three sessions below 10 responses). It is possible that two factors weakened the conditioned reinforcing effects of the tokens in this analysis: (a) the delay between completion of the response and delivery of the token or (b) the delay between the delivery of tokens and delivery of food during the exchange at the end of the session. This pattern of responding was not observed in the exchange analysis, which involved immediate token delivery and a delayed exchange opportunity after every earned token (instead of one exchange period at the end of the session). Because both of those contingencies were present in the token analysis and not in the exchange analysis, it remains unclear which factor resulted in the apparent variability.

Although similar patterns emerged across subjects, the delay values that produced decreases in responding differed. It is possible that individual-subject characteristics may have contributed to differences in sensitivity to delay. For example, Alex, the subject who continued to respond at increasingly larger delays, made several statements during delay sessions such as “Alex, you have to wait.” In addition, he frequently counted audibly during the delay. Thus, self-generated rules may have promoted relatively greater delay tolerance (Catania, 1998). Further, by audibly counting during the delay, Alex converted what was programmed to be an unsignaled delay into a signaled delay.

In a recent study that examined delayed reinforcement in the context of discrimination using food and toy items as reinforcers, Sy and Vollmer (2012) demonstrated that delays of 20 s and 40 s produced acquisition similar to that produced by immediate reinforcement. In the current study, relatively large delays to food produced responding similar to immediate food reinforcement for David (i.e., 20 s) and Alex (i.e., 60 s). Similar to the findings of Sy and Vollmer, food reinforcement was exceptionally robust in the face of long delays. To date, no research has examined the extent to which delayed token delivery affects response acquisition. Given the current findings on the effects of delayed token delivery on response maintenance and the relatively short delay values (i.e., 3 s and 6 s for Chris and David, respectively) that resulted in decreased responding in the token analysis, future research could examine delayed token delivery in the context of response acquisition.

All responses were immediately followed by an auditory stimulus (buzzer). The purpose of the buzzer was to signal that the response requirement had been met. We did not evaluate the effect of the buzzer alone on responding; therefore, it is unclear whether the buzzer affected responding in this study. It is possible that the buzzer functioned as a conditioned reinforcer and produced more persistent responding due to a conditioned reinforcement effect by way of immediate pairing with an unconditioned reinforcer during the food analysis (the first analysis conducted for all three subjects). Alternatively, it is possible that the buzzer became discriminative for the onset of the delay and functioned as a conditioned punisher, thereby producing less overall responding than would have occurred in its absence. Future research could examine the effect of including a secondary, previously neutral stimulus (like the buzzer) at the onset of a delay on the behavior of individuals with developmental disabilities in academic settings.

In this study, tokens were exchangeable only for the edible item used during the food analysis. That is, the tokens were not generalized conditioned reinforcers. Moher, Gould, Hegg, and Mahoney (2008) examined the effects of satiation and deprivation in the context of token reinforcement. Moher et al. evaluated a satiation condition, during which subjects were given noncontingent presession access to a high-preference item. Next, two conditions were evaluated in which (a) tokens were exchangeable only for the high-preference item that was made available before the session and (b) tokens were exchangeable for a variety of high-preference items. Moher et al. found that responding decreased during sessions in which tokens were exchangeable only for the high-preference item and persisted at levels similar to the deprivation condition when the tokens were exchangeable for a variety of high-preference items. These results suggest that responding may be more durable in the face of treatment challenges (e.g., delay) if tokens are exchangeable for a variety of reinforcers. The present study focused on one back-up reinforcer so that findings would not be conflated by exchanges that involved multiple back-up reinforcers. However, in practice, exchange opportunities commonly involve multiple back-up reinforcers. Thus, future research could examine the effect of delayed token delivery when the token is a generalized conditioned reinforcer (i.e., exchangeable for multiple items) rather than exchangeable for just one item.

Discounting research on primary and secondary reinforcers posits that humans discount delayed conditioned reinforcers less steeply than delayed primary reinforcers. However, findings from the current study run counter to this notion. Why the present findings contradict findings from the discounting literature is not entirely clear, but we may speculate on three possible explanations. First, relative to food as a consequence for behavior, tokens were relatively novel for Chris and David. That is, these subjects had a relatively abbreviated history with tokens as consequences. In fact, experiences with tokens as consequences were established before this experiment. This is to be contrasted with the experiences of subjects in discounting research who bring an extensive history to bear on choices emitted during experimental sessions. For example, adult subjects in discounting research will have had years of experience in the production and exchange of money as a form of conditioned reinforcement. Second, the nature of the conditioned reinforcer used in this study differs from that typically used in discounting research. Specifically, and as stated previously, we used one back-up reinforcer (the same food item used in the delayed food analysis), whereas discounting research typically involves a conditioned reinforcer that is exchangeable for a variety of back-up reinforcers (i.e., a generalized conditioned reinforcer). It is conceivable that an individual will discount a generalized conditioned reinforcer (e.g., money) less steeply than a secondary reinforcer that is not generalized (i.e., exchangeable for only one item). Third, the current experiment examined delayed reinforcement under a single-schedule arrangement, whereas temporal discounting research involves choice under a concurrent-schedules arrangement. Future research could examine the effects of delayed reinforcement on the responding of individuals with developmental disabilities in a concurrent-schedules context.

There are several notable limitations to the current experiment. First, we capped sessions at 30 earned reinforcers. It is possible that the subjects’ ceiling of responding, especially for the immediate reinforcement condition, was much higher than 30. If sessions had not been capped, it is possible that responding could have decreased more steadily across delays. However, we elected not to do so to limit the amount of extra food the subjects consumed each day. Second, exchange opportunities were not fixed during the token analysis. That is, subjects could exchange the tokens they earned during the session at the end of every session. Recall that sessions ended when subjects (a) earned 30 reinforcers, (b) did not respond for 2 min, or (c) said “I’m done.” That is, subjects could end a session by not responding and cash in their tokens, functionally shortening the exchange delay. Jackson and Hackenberg (1996) demonstrated that pigeons became insensitive to delays in token delivery only when their behavior had no effect on exchange opportunities. Otherwise, their behavior was largely allocated to the response option that produced more immediate exchange opportunities (i.e., more immediate access to food). Future research could examine responding under delayed token conditions with a fixed exchange opportunity with human subjects. We gradually increased the duration of delay values from 0 s to 120 s. This procedure may in fact have functioned as delay fading (Rooker, Jessel, Kurtz, & Hagopian, 2013), and subjects may have become tolerant to longer and longer delays. In research and practice (Rooker et al., 2013) an ascending sequence of delays is common, and although this approach may have confounded an understanding of delayed food, tokens, and exchange, presenting a descending sequence of delays or random delays might be contraindicated clinically, at least initially. Third, session-to-session intervals were not held constant. Therefore, it is possible that decreases in responding that were observed as a function of increasing delays were in fact a function of decreases in reinforcer density. Future researchers could arrange yoked intersession intervals across delays to control for total number of reinforcers that could be earned per unit time. Finally, subjects were exposed to the individual delay values only in single phases. That is, we did not arrange for within-subject replications of responding at the different delay values. The only phase that was repeated was 0-s delay. Future research could arrange for within-subject replications of the delay value that produces decreases in responding.

The present study involved a systematic evaluation of three delay scenarios that might arise in applied contexts (e.g., classrooms). Responding decreased consistently, for all subjects, as a function of increasing delays to token production. That is, delays to token production maintained responding most weakly. The present results suggest that delaying provision of tokens by only a few seconds, which might be conceptualized as a treatment-integrity challenge, can have a substantial impact on responding. If the delay to token provision is problematic in practice, future researchers might consider ways to enhance the value of tokens so that their effects remain durable in the face of delay.