Response Bias is Unaffected by Delay Length in a Delay Discounting Paradigm

Abstract

This study examined the contribution of response bias to measures of delay discounting in Long-Evans rats (n = 8) using the adjusting amount procedure. Under this procedure, we assessed preference for 150 μl of 10% sucrose solution delivered following a delay over a variable-amount alternative delivered immediately. Bias was calculated based on relative preference when reinforcers were delivered immediately from both alternatives. We extended this assessment procedure to examine preference when rewards from both alternatives were equally delayed (2, 4, 8, or 16 s) in addition to assessing a traditional delay discounting function. Relative preference was similar across delays and slightly larger than 150 μl. These results indicate that response bias was stable and suggests a relative aversion for the adjusting alternative, which may be due to the variability in reward size associated with that alternative.

1. Introduction

Choice between two alternatives is a function of the reinforcement parameters associated with each alternative, but also a function of factors unrelated to the prevailing reinforcement contingencies (response bias: Baum, 1974). Response bias occurs when animals choose between two similar response alternatives that differ in subtle but important ways and is usually manifested as a side or location preference. Very small differences in reinforcement rate and required effort may contribute to side preferences, for example, if reinforcers are slightly more difficult to retrieve due to apparatus imperfections or the physical characteristics of the subject. Bias may also arise from preexisting preferences for one side or the other. It is important to note that response bias is not the result of errors by the organism, but due to unmeasured independent variables.

Psychophysical procedures in which subjects adjust the value of one alternative to “match” the value of the other (adjustment procedures: Stevens 1951) are well-suited to assess both the magnitude and direction of response bias as well as any systematic effects of experimental variables that might indirectly contribute to response bias. However, the most prevalent use of such adjustment procedures recently has been to examine the extent to which the value of a delayed reinforcer is discounted (delay discounting). Two assumptions are made when interpreting delay discounting data. First, it is assumed that the value of the delayed alternative is a function of the reinforcer devaluation induced by the delay as well as any response bias, and second, that the magnitude of any response bias associated with the alternatives is independent of the length of the delay. To the best of our knowledge, the latter assumption is made in all mathematical models of delay discounting, including simple hyperbolic discounting (Mazur 1987), as well as in other models of operant choice involving delayed rewards. However, this critical assumption has not been systematically investigated.

One of the two adjusting procedures that could be used to assess this assumption is the adjusting amount procedure, in which rats choose between a large, fixed amount of a reinforcer delivered after a specific delay (the “standard”) and a variable amount of the reinforcer available immediately (Richards et al., 1997). Choices of the large, delayed standard reward cause the size of the variable reward to increase, whereas choices of the variable reward cause its size to decrease. After a number of trials, the size of the variable reward is titrated to a value at which rats become indifferent between the two reward alternatives. The indifference values obtained using this procedure are assumed to reflect the subjective value of the delayed reward, e.g., 100 μl of water in 8 s may only have a value of 50 μl of water immediately while 100 μl of water delivered sooner (in 4 s) would have a higher value, such as 75 μl of water immediately. Previous research has reported substantial individual differences in response bias when both the adjusting and standard alternatives were delayed by 0 s (Richards et al, 1997; Wilhelm & Mitchell, 2008; 2009). However, these studies have not assessed the assumption that response bias is independent of the length of the response-reinforcer delay.

A more widely used adjusting procedure is the adjusting delay procedure, in which subjects choose between a large, fixed amount of food delivered after a variable delay and a smaller, fixed amount available after a short fixed delay (e.g., Mazur, 1987). Choices of the larger more delayed reward cause the length of the delay for that alternative to increase, whereas choices of the smaller, sooner reward cause the variable delay to the larger reward to decrease. Studies using the adjusting delay procedure have yielded results consistent with increasing response bias as a function of delay length. Mazur (1984; Figure 3) found pigeons had a slight overall preference for the adjusting delay alternative when the standard alternative delay was 1, 5, 10 and 18 s and both alternatives delivered 3 s access to grain. But more importantly, a graphical analysis indicated that the mean degree of bias increased by approximately 15% as a function of delay length. Grace (1996), also using an adjusting delay procedure, found that pigeons exhibited a preference for the standard alternative at longer delays (20 and 30 s) but not at shorter delays (10 s) when both alternatives delivered 2 s access to grain, suggesting that response bias might increase with overall delay. However, caution should be exercised before concluding that these data indicate response bias is a function of delay and that the assumption made in mathematical models of discounting is incorrect. These two studies were not designed to assess response bias directly. Indeed, the physical location of the standard and adjusting alternatives was assigned randomly on each trial, which would reduce the development of side preferences.

The purpose of this study was to examine whether response bias was indeed independent of the reinforcement delays associated with each alternative using the adjusting amount procedure. To achieve this, we included a number of additional conditions when the delay to both the standard and adjusting alternatives was the same but nonzero. This allowed us to determine if response bias remained constant as a function of delay, implying the factors contributing to response bias either are not affected by delay or are discounted to an equal degree for both alternatives.

2. Method

2.1 Subjects

Male Long-Evans rats (n = 8) were obtained from Charles River Laboratories. On arrival, the rats weighed between 268 and 325g. During the experiment, animals were maintained at approximately 90% of their age-adjusted free-feeding body weights by supplemental feedings of lab chow after experimental sessions. Animals had constant free access to water in the home cage and were housed in pairs on a 12:12 light/dark cycle (lights on = 0600 h, lights off 1800 h). Experimental sessions occurred during the light cycle, 5 days a week.

2.2 Apparatus

Mitchell & Rosenthal (2003) provide a detailed description of the apparatus. The operant conditioning chambers were designed with three force-sensitive levers on the front panel (in accordance with Notterman & Mintz, 1965). A guard around the side and bottom of each lever ensured that animals could only contact the lever from above. A stimulus light was located 5 cm above each lever, and a circular (diameter = 5 cm) recess containing a food cup was located under each lever.

Lever pressing constituted isometric exercise, as force applied to the lever caused virtually no movement (< 1 mm) of the shaft. Strain gauges bonded to the shaft of each lever allowed force applied to the lever to be measured on a ms-by-ms basis (sensitivity of 0.001N [0.1 gram-force], range 0 to 0.15 N). Effort exerted during responding was measured as the amount of force applied over time (units: Ns). Responses were reinforced if and when the sum of the effort exerted during a response exceeded a specific threshold. For example, at a threshold of 0.1 Ns, if the rat exerted 10 grams-force the response would be reinforced after 1 s, however if the rat exerted 50 grams-force the response would be reinforced after 0.2 s.

Reinforcers consisted of a 10% w/v solution of sucrose (table sugar) dissolved in deionized water. When reinforcement criteria were met, the microcomputer activated a syringe pump and the programmed amount of sucrose solution was delivered to a food cup located under the appropriate lever. Food delivery was accompanied by a programmed feedback click and the noise associated with pump activation.

2.3 Procedure

Task training occurred in 3 phases. In Phase 1, animals responded on either of the two outer levers and were reinforced according to a FR1 schedule. Phase 2 required animals to press the middle lever prior to one of the two outer levers, introduced forced choice trials (see Adjusting Amount Procedure), and assigned one lever as the “adjusting” and one as the “standard” lever. Phase 3 exposed animals to each of the possible delays on the standard lever in ascending order over successive sessions.

Following training, rats were exposed to nine different conditions in a random order (sampling without replacement, see Table 1). In five conditions, the rats chose between the standard lever, which delivered 150-μl of sucrose after a delay (0, 2, 4, 8 or 16 s), and the adjusting lever, which delivered a variable amount of sucrose immediately (standard delay discounting conditions: 0-0, 2-0, 4-0, 8-0 and 16-0). In the remaining conditions, the rats chose between the standard and adjusting levers when they had equal delays to reinforcer receipt (2-2, 4-4, 8-8, or 16-16, in seconds). Under all nine conditions, responses on the standard lever delivered 150 μl of sucrose, while the adjusting lever delivered a variable amount of sucrose (as described below). Each condition occurred on six sessions according to a randomized design (Table 1).

Table 1.

The schedule used to vary the order in which individuals experienced the nine conditions.

Block	Delay (in seconds on standard and adjusting lever respectively)
1	8-0	4-0	16-0	2-2	4-4	8-8	2-0	16-16	0-0
2	16-16	0-0	16-0	4-0	4-4	2-2	2-0	8-0	8-8
3*	16-0	4-0	8-0	2-0	2-2	4-4	16-16	4-0	0-0
4	2-2	8-0	0-0	2-0	8-8	4-4	4-0	16-0	16-16
5	16-0	16-16	2-2	8-8	4-0	2-0	4-4	8-0	0-0
6	4-4	8-8	4-0	16-16	2-2	16-0	0-0	2-0	8-0

Note.

^*Due to experimenter error, Block 3 is missing the 8-8 condition. Subjects B1 and B5-B8 performed 8-8 after completing all scheduled sessions. B2-B4 performed 8-8 between 0-0 and 2-0 in the 6^th Block.

2.3.1 Adjusting Amount Procedure (adapted from Richards et al., 1997)

Each session was composed of 60 choice trials, plus a variable number of forced choice trials. To signal the beginning of a trial, the houselights and the light above the middle lever were lit. The rat was then required to perform a response of at least 0.01 Ns on that lever. This ensured that subjects were positioned midway between the two outer levers, which were associated with the delivery of the sucrose reinforcers, thereby minimizing one source of response bias. When the rat responded on the middle lever, a brief click sounded, the light above the middle lever was extinguished and the lights were lit above the outer levers. If the rat chose the adjusting lever, a brief click sounded and a variable amount of sucrose solution was delivered to the food cup under that lever. If the rat chose the standard lever a brief click sounded and 150 μl of sucrose solution was delivered to the food cup under that lever, concurrent with another click after the delay associated with that session (0, 2, 4, 8, or 16s) elapsed. At the start of each session, the amount of sucrose associated with the adjusting alternative was 75 μl. However, within each session this amount varied. If the rat chose the standard lever, the adjusting amount increased by 10% of its current volume for the next trial. If the rat chose the adjusting lever, the adjusting amount decreased by 10% of its current volume for the next trial. Thus, if an animal were to choose the adjusting alternative on the first three consecutive trials of the session, it would receive 75 μl of sucrose on the first trial, 75 × 0.90 = 67.5 μl of sucrose on the second trial, and 67.5 × 0.90 = 60.75 μl of sucrose on the third trial.

A timeout period began following reinforcer delivery. All lights in the operant chamber were extinguished throughout this period. The length of the timeout varied but was designed so that, regardless of which alternative the animal chose, a new trial would begin every 30s. The beginning of a new trial was signaled by the houselights and the light above the middle lever being lit.

To ensure that subjects were exposed to the contingencies of each alternative, a forced choice trial occurred whenever the rat made two consecutive choices of the same lever. Forced choice trials differed from choice trials in two ways. First, on a forced choice trial, after the rat had pressed the middle lever, the light above either the standard or the adjusting lever was lit, depending on which alternative had not been chosen on the previous two choice trials. Responses on the other lever were recorded but did not result in reinforcer delivery. Second, the magnitude of the reinforcer associated with the adjusting lever was not adjusted for the next trial. Data from the forced choice trials were not included in any of the analyses reported.

2.4 Data Analysis

The main dependent variable was the amount of sucrose reinforcer associated with the adjusting alternative over the final 30 trials of a session (trials 31-60, with forced choice trials excluded from analysis). Medians were used because the adjusting nature of the task resulted in amount data that were positively skewed. The final 30 trials were used because inspection of the data and previous studies suggest rats become relatively indifferent between the standard and adjusting alternatives by this point (Richards et al., 1997). If a rat did not complete a session, the analysis incorporated completed trials beyond 30 for that session. We excluded from the analyses sessions where animals completed fewer than 30 trials (10 out of 432 sessions; 2%), or where animals exhibited a greater than 80% preference for either the standard or adjusting alternatives over the final 30 trials (20 out of 432 sessions; 5%). Overall, animals completed 56.4 ± 0.5 trials per session, with the vast majority of incomplete trials occurring in the 16-16 condition (average trials completed = 42.8 ± 2.2). The average of the median indifference points for the final five sessions was calculated for each condition and analyzed using within-subject analyses of variance (ANOVA). Delay discounting (0-0, 2-0, 4-0, 8-0, 16-0) and Bias (0-0, 2-2, 4-4, 8-8, 16-16) conditions were analyzed separately with delay as a within subject factor. Similar analyses were performed for reaction time (i.e. time between the beginning of a trial and the initial response on the middle lever), and choice reaction time (i.e. time between middle lever press and choice of the adjusting or standard alternative). Huynh-Feldt corrections were applied where the sphericity assumption was violated, and adjusted degrees of freedom are presented. For all analyses the threshold for significance was maintained at p < 0.05.

The hyperbolic discount function (Mazur 1987) was fitted to the indifference points derived for the delay discounting conditions (0-0, 2-0, 4-0, 8-0, 16-0).

$$\mathrm{V}=\mathrm{bA}∕(1+\mathrm{kD})$$ Equ 1

V represents the subjective value of the reinforcer from the standard lever (amount, A = 150 μl), approximated by the indifference point, available after a delay (D). k is a fitted parameter that measures the rate of discounting as a function of delay, and b represents response bias. Values for b were calculated based on each animal's indifference point at the 0-0 condition:

$$\mathrm{b}={\mathrm{V}}_{0-0}∕\mathrm{A}.$$ Equ 2

As a secondary analysis, b values were fitted using Equ 1 and compared to b values calculated for the Bias conditions.

3. Results

3.1 Sensitivity to delay

Sensitivity to delay was examined by offering animals a choice between a standard 150 μl of 10% sucrose solution after a 0, 2, 4, 8 or 16s delay, or an adjusting amount of 10% sucrose solution available immediately. Over the final 30 trials, animals chose the standard alternative (mean ± SEM) 52.6 ± 0.7% of the time. A one-sample t-test indicated that this value was significantly larger than 50% (p < 0.01), indicating that, on average, animals selected the standard alternative more than the adjusting alternative over the final 30 trials. There was no significant correlation, however, between percent choice over the final 30 trials and b values (Pearson r = −0.31, p = 0.45, n = 8), suggesting that the slight preference was not systematically contributing to the measured response bias.

Indifference points decreased as the delay to reward increased (F(3.4, 24.1) = 29.94, p < 0.001), indicating that animals devalued the 150 μl standard alternative as a function of the delay to its delivery (Figure 1). In general, the data for each subject were well described by the hyperbolic discount function (Equation 1; R² = 0.90 ± 0.03). The coefficients of determination ranged from 0.77 to 1.00. The mean k value was 0.15 ± 0.03, which is consistent with previously reported discount rates observed in Long-Evans rats responding for water (e.g. Richards et al. 1997) or sucrose solution (e.g. Mitchell & Rosenthal 2003). The mean b value was 1.16 ± 0.10 (Equ 2).

Figure 1.

Indifference points (mean ± SEM) for each condition tested. Abscissa depicts length of delay in seconds to the delivery of the 150 μl reward. Standard delay discounting conditions (0-0, 2-0, 4-0, 8-0 and 16-0) are denoted with solid boxes. The hyperbolic discount function was fitted to indifference points associated with these conditions (Equation 1). Conditions to specifically assess bias (0-0, 2-2, 4-4, 8-8 and 16-16) are denoted as open triangles. Note the 0-0 point is the same for the standard delay discounting conditions and the bias assessment conditions.

3.2 Response Bias

Under conditions where the standard and adjusting alternatives were available with the same delay there were no significant differences in indifference point as a function of delay (F(2.2, 15.7) = 1.15, p = 0.35).

In the absence of any inherent or learned biases, animals would be expected to titrate the amount of the adjusting alternative to 150 μl when the standard and adjusting alternatives were available with the same delays. However, animals consistently exhibited a response bias toward the standard alternative (average indifference point for all bias conditions including 0-0 = 197.9 ± 11.0 μl) under these conditions. One-sample t-tests comparing indifference points at each of the bias conditions (0-0, 2-2, 4-4, 8-8 and 16-16) indicated that indifference points were significantly greater than 150 μl for the 2-2 and 16-16 conditions (ps < 0.05), while the other indifference points did not differ significantly from the expected 150 μl value (0-0: p = 0.09; 4-4: p = 0.15; 8-8: p = 0.23). The indifference points across all bias conditions did not differ from the 0-0 condition (t-tests; all ps > 0.10). As a follow-up analysis, we fitted b values to the observed indifference points generated from the unequal delay conditions (2-0, 4-0, 8-0 and 16-0). The b values from this analysis (1.68 ± 0.29) did not differ from b values calculated from any of the bias conditions (t-tests; all ps > 0.10).

3.3 Response Times

To examine potential differences in local reinforcement rate (overall rate was fixed by use of the ITI), we examined reaction times and choice reaction times as a function of delay and subsequent lever choice (Table 2). Bias conditions and standard delay discounting conditions were examined separately.

Table 2.

Mean ± S.E.M.s for the reaction times and choice reaction times (in seconds) as a function of condition (delay in seconds on standard and adjusting lever respectively) and the lever chosen (Delay or Immediate)

	0-0	2-0	4-0	8-0	16-0
Reaction time
Delay	1.63 ± 0.22	2.23 ± 0.79	2.01 ± 0.19	2.22 ± 0.46	4.40 ± 1.24
Immediate	1.86 ± 0.32	2.34 ± 0.45	2.14 ± 0.49	2.21 ± 0.46	3.65 ± 0.71
Choice reaction time
Delay	0.53 ± 0.06	0.49 ± 0.05	0.54 ± 0.08	0.55 ± 0.10	0.79 ± 0.21
Immediate	0.66 ± 0.09	0.69 ± 0.09	0.69 ± 0.08	0.71 ± 0.10	0.78 ± 0.15
		2-2	4-4	8-8	16-16
Reaction time
Delay		1.87 ± 0.31	1.67 ± 0.13	4.13 ± 0.89	10.97 ± 1.34
Immediate		1.94 ± 0.32	2.18 ± 0.38	3.10 ± 0.64	12.58 ± 2.66
Choice reaction time
Delay		0.57 ± 0.05	0.54 ± 0.07	0.68 ± 0.15	0.85 ± 0.20
Immediate		0.73 ± 0.11	0.77 ± 0.11	1.08 ± 0.42	1.47 ± 0.33

Note.

The reaction time is the time from trial initiation until the middle lever has been pressed. The choice reaction time is the time from middle lever press until the animal chooses either the adjusting or standard alternatives.

Under standard delay discounting conditions, analysis of reaction times indicated no effect of subsequent choice on that trial (F(1, 7) = 0.03, p > 0.05), delay (F(1.5, 10.5) = 3.84, p > 0.05), or choice × delay interaction (F(2.2, 15.7) = 0.73, p > 0.05). Similarly, analysis of choice reaction times also indicated no effect of choice (F(1, 7) = 1.97, p > 0.05), delay (F(2.3, 16.0) = 1.33, p > 0.05), or choice × delay interaction (F(3.7, 26.2) = 1.01, p > 0.05) under these conditions.

In contrast, under bias conditions, reaction times were slower when the delay on both levers became longer (delay; F(1.3, 9.2) = 24.10, p < 0.001), but were unaffected by lever choice (F(1, 7) = 0.75, p > 0.05) and did not exhibit a choice × delay interaction (F(1.3, 9.0) = 1.13, p > 0.05). Meanwhile, choice reaction times were unaffected by choice (F(1, 7) = 2.06, p > 0.05), delay (F(1.1, 7.3) = 1.78, p > 0.05), or choice × delay interaction (F(1.0, 7.3) = 1.25, p > 0.05). The lack of choice or choice × delay effects for the reaction time and choice reaction time measures indicates that local rates of reinforcement were quantitatively similar for all of these conditions tested and therefore would likely not be a source of response bias.

4. Discussion

To our knowledge, this is the first study using the adjusting amount procedure to examine response bias at conditions other than 0-0 to determine if bias is uniform across other delays. We found no systematic differences in response bias across any of the conditions tested, i.e. bias at 0-0 was representative of bias under the other conditions tested in the experiment (2-2, 4-4, 8-8 and 16-16). As such, it appears that for the purposes of assessing sensitivity to delay, determining bias at the 0-0 condition provides sufficient information to assess response bias across each of the delays. Further, while it is difficult to be fully confident of a null result, our data support the appropriateness of the assumption of stable levels of bias inherent in theories of delay discounting and other temporal choice models. That is not to say that there was no response bias, only that it was stable across delays. These results contrast with those of Grace (1996), who found that pigeons exhibited a preference toward a standard alternative in the adjusting delay task when the delay was long but not when it was short, and Mazur (1984) who found that pigeons exhibited an increasing preference for the adjusting alternative in the adjusting delay task. Species or procedural differences may be the cause of the apparent disparities between these studies and the current research, which could be examined in the future.

One assumption in psychophysical procedures, such as the adjusting amount procedure, is that animals choose whichever alternative is most valuable on any given trial as opposed to adopting a global optimization strategy which would maximize the amount of reinforcer received. In the adjusting amount procedure, if a subject were to repeatedly choose the standard alternative, the amount of sucrose associated with the adjusting alternative could become very large (up to 300 μl, the limit programmed under our procedure). Once the maximum amount became available, the subject could then alternate between the standard and adjusting alternatives, thereby maximizing the amount of sucrose delivered over the entire session. Although we cannot definitively rule out the possibility that animals adopted this strategy at any time during the session, the data do not support the contention that their behavior was driven by a global maximization rule. Though indifference points were somewhat larger than the expected 150 μl under some conditions, they never approached the program-limited value available from the adjusting alternative, as would be predicted by global maximization.

Although response bias was unaffected by delay length during the bias conditions, there was a consistent response bias towards the standard alternative. The source of this observed bias is not entirely clear. While we cannot rule out some of the potential causes of bias noted by Baum (1974), the experimental design was such that many of the potential biases would be balanced. For instance, the adjusting and standard alternatives were counterbalanced across animals. Thus, any differences in lever sensitivity, positional bias, or even apparatus asymmetry should be balanced. This would not, however, account for asymmetries within the organism such as handedness, cerebral dominance, or muscular asymmetries. Furthermore, reaction times and choice reactions times were unaffected by choice of the adjusting or standard lever, suggesting that response rates and therefore reinforcement rates were similar on both alternatives under bias conditions.

Isles et al. (2004) hypothesized that response bias within a delay discounting task may be the result of learned associations that lead to a conditioned place aversion or preference for one of the levers. The observation in the present study that animals preferred the standard to the adjusting lever suggests that animals may have developed a slight aversion to the inconsistent nature of the adjusting alternative or a preference for the more stable standard lever. In contrast to the amount of reinforcer, which varied from trial-to-trial within every session on the adjusting alternative, the delay to reinforcer delivery varied for both levers across sessions, though somewhat more frequently on the standard lever. Research examining preference for variable versus fixed alternatives has indicated that when the source of the variability is in food amount, there is a preference for the fixed alternative (Bateson, 2002), and when the source of variability is in delay to reinforcement, there is a preference for the variable alternative (e.g. Schuck-Paim & Kacelnik, 2002). Early research had suggested that food restriction was associated with a preference for the variable alternative (e.g., Caraco, Martindale & Whitham, 1980) but a more recent review suggests that the effects of food deprivation are negligible in the face of the parameter being varied (Kacelnik & Bateson, 1996). Our data would suggest that the response bias towards the standard alternative was driven by the more rapid cycling of the amount than delay. Variability in delay may have also impacted preference; however this effect appears to be less robust than the effect of amount variability. Individual differences in response bias would reflect the interaction of these amount and delay factors, and account for the lack of consistent biases observed in some studies, e.g., Richards et al (1997). Given the hypothesis that response bias in the procedure is driven by relative sensitivity to variability cycles, we would predict that any future research examining bias in adjusting delay procedures (e.g., Mazur, 1987; Rodriguez & Logue 1988) would find a bias towards the variable alternative, which is ordinarily associated with the larger reward. Thus, in both the adjusting delay and adjusting amount procedures we would predict preferences for the larger, more delayed alternative, increasing the likelihood of the procedures yielding concordant data (Green et al., 2007). It may be that the preference of animals in this study for the fixed alternative was driven by a relative aversion to a trial-to-trial variability in amount of the sucrose reinforcer, and that this aversion outweighed the day-to-day variability of the delay changes associated with the fixed alternative. Further, it should be noted that Wilhelm and Mitchell (2008) observed that rats selectively bred to prefer alcohol exhibited b values consistently larger than 1 (indicating preference for the standard alternative), while rats bred to avoid alcohol exhibited b values equal to or less than 1. Likewise, among a panel of inbred rat strains, most rat lines tended to have b values higher than 1, with Copenhagen rats being the lone exception (Wilhelm & Mitchell, 2009). Genetics may play a role in the individual differences in sensitivity to amount and delay variability, potentially leading to differences in response bias which should be taken into account.