A bout analysis of operant response disruption

Abstract

Operant behavior appears to be organized in bouts of responses, whose parameters are differentially sensitive to various manipulations. This study investigated potential differential effects of three forms of operant response disruption—extinction (EXT), non-contingent reinforcement (NCR), and prefeeding (PRE)—on response bouts. In Experiment 1, Wistar Kyoto rats (WKY) were trained on a tandem variable-time (VT) 120 s fixed-ratio (FR) 5 schedule of reinforcement; after stability was established, their responding was disrupted for three sessions with one of the three disrupters (EXT, NCR, or PRE). In Experiment 2, Long Evans (LE) rats were trained on a tandem VT 240 s FR 5 to stability, and their responding disrupted with EXT or NCR. In EXT and NCR, response rates declined significantly and progressively over the course of the session, primarily due to a declining bout-initiation rate in EXT, and to fewer responses per bout in NCR. In contrast, a session-wide drop in response rate was observed in PRE, primarily due to a reduction in bout-initiation rate at the start of the session. These findings suggest that each form of disruption differentially impacts dissociable aspects of behavior. Theories of behavioral persistence should account for these functional relations, which appear to be obscured in response rate measures.

There are many ways to disrupt operant responding. Sating a rat prior to the experimental session may reduce its lever pressing for food. Alternatively, withholding the food reinforcer (extinction) or providing food independently of the operant (non-contingent reinforcement) may undermine the response-reinforcer contingency. Each of these methods decrease response rate, but presumably in ways that are functionally distinct (Bindra, 1978; Bouton, 2004; Lachter, Cole, & Schoenfeld, 1971; Skinner, 1948). This study asks whether changes in the temporal organization of responses reveals the distinct ways in which each of these response disrupters—extinction (EXT), non-contingent reinforcement (NCR), and prefeeding (PRE)—changes operant performance.

Operant responses are typically organized in clusters (bouts). Within each bout, short inter-response times (IRTs) separate individual responses; longer IRTs separate bouts (Brackney, Cheung, Herbst, Hill, & Sanabria, 2012; Brackney, Cheung, Neisewander, & Sanabria, 2011; Brackney & Sanabria, 2015; Hill, Herbst, & Sanabria, 2012; Johnson, Pesek, & Newland, 2009; Shull, Gaynor, & Grimes, 2001, 2002; Shull, 2004, 2011; Smith, McLean, Shull, Hughes, & Pitts, 2014; Tolkamp, Schweitzer, & Kyriazakis, 2000). The response bout has three characteristics: (a) the mean bout length, or how many responses the animal makes while in a bout, (b) the within-bout response rate, or how fast the animal responds while in a bout, and (c) the bout-initiation rate, or how frequently the animal begins a new bout.

Brackney and colleagues (2011) used the bi-exponential refractory model (BERM) to estimate bout-organization parameters, showing that those estimates change systematically in response to different reinforcement contingencies and other experimental manipulations. For example, Brackney et al (2011) found that depriving a rat of food increases the overall bout-initiation rate, whereas extinction decreases the bout-initiation rate gradually over the course of the session, while neither manipulation had an appreciable effect on other parameters. In contrast, increasing the number of responses required to collect a reinforcer increased both the within-bout response rate and the mean bout length.

This study is a partial replication and extension of Brackney et al (2011). There are several critical differences between Brackney et al (2011) and the current study. First, in Brackney et al (2011), the effect of acute food deprivation on normally free-fed rats was examined during variable-interval (VI) training, whereas this study examines the effect of acute prefeeding on rats that are normally food-deprived. Second, the current study adds an additional comparison condition, NCR. Third, whereas conditions in Brackney et al (2011) were confounded by training order, exposure to each disruption condition in the current study was counterbalanced across rats. Fourth and finally, the current study examines response bouts in two strains of rats, Wistar Kyoto (WKY; Experiment 1) and Long Evans (LE, Experiment 2), whereas Brackney et al (2011) examined the performance of Sprague Dawley rats. WKYs are an inbred strain commonly used as control strain for the spontaneously hypertensive rat (SHR), an animal model of both hypertension and attention deficit hyperactivity disorder (Sagvolden et al., 2009). WKYs generally display low rates of operant responding, and are sometimes used as an animal model of depression and anxiety (Will, Aird, & Redei, 2003). LE rats are a commonly-used outbred strain that originally resulted from a cross between Wistar and wild gray rats (Oiso, Riddle, Serikawa, Kuramoto, & Spritz, 2004).

The primary goal of this study was to assess the relative contributions of each response bout parameter to the decline in responding observed in each response-disruption condition. To achieve this goal, a dynamic extension of BERM, DBERM (Brackney et al., 2011; Cheung, Neisewander, & Sanabria, 2012) was fit to responding during maintenance training and to responding during the first disruption session of each condition. DBERM extends BERM by assuming that bout parameters change exponentially over time within a session. Differences between estimates of DBERM parameters during maintenance and disruption were then assessed.

METHODS

Subjects

Experiment 1

Twelve experimentally experienced, pair-housed Wistar Kyoto rats (WKY/NHsd, Harlan Laboratories, US) served as subjects. They were approximately one year old [post-natal day, (PND) 336] at the start of the study. The rats were food restricted: 30 min after each experimental session, free access to homecage chow (Harlan 2920X rodent diet) was allowed for 1 h. This feeding regimen remained in effect unless noted otherwise. It maintained subjects at approximately 85% of their ad libitum weight based on a logistic function fitted to growth curves provided by breeder.

All twelve subjects participated in several previous experiments and were well trained in lever pressing for sucrose pellets at the start of the experiment. Their experimental histories included training on simple variable interval (VI) schedules, extinction following VI training, latent inhibition (using pre-exposed or novel tones to signal sucrose pellets), and fixed minimum interval training (reinforcing timed lever-press-head-entry sequences with sucrose pellets)..

Experiment 2

Ten experimentally naïve Long Evans rats (LE; Charles River Laboratory, US) served as subjects. The experiment started on PND 60. All other details were identical to Experiment 1 unless otherwise stated.

Apparatus

Experimental sessions were conducted in six identical Med Associates ® chambers, 305 mm long, 241 mm wide, and 210 mm high. The chambers were housed in sound and light attenuating cabinets, in which a ventilation fan provided white noise at approximately 60 dB. The chambers were arranged according to the standard dual lever configuration—two retractable levers 21 mm above the floor flanked a food receptacle aperture in (51 mm sides, 15 mm from the chamber floor). The walls orthogonal to levers and food receptacle aperture were made of transparent Plexiglas, whereas the remaining two walls were made of aluminum. A houselight mounted outside the experimental chamber provided dim illumination inside the chamber when on. Forty-five mg sucrose pellets (TestDiet^™ 5TUT) served as reinforcers and were delivered into the food receptacle aperture via a pellet dispenser mounted outside the chamber. The experimental equipment was identical in Experiments 1 and 2.

Procedure

Experiment 1

Each daily session began with a 300-s acclimation period, during which no experimental events occurred. Following acclimation, the left lever (farthest from the door) was extended. All sessions were conducted with the houselight off except when noted otherwise. Sessions were 60-min long and were conducted 7 days/week. Due to the rats’ considerable experimental history, pretraining such as autoshaping or chamber habituation was not conducted.

The experiment consisted of alternating phases of maintenance training (MAINT) and response disruption (Table 1; see Table 2 for abbreviations). During MAINT, lever presses were reinforced according to a tandem variable-time (VT) 120 s fixed ratio (FR) 5 schedule of reinforcement. Reinforcement was thus contingent upon completing 5 lever presses after an unsignaled interval had elapsed; the interval was randomly sampled without replacement from a 12-item list drawn from a Flesher-Hoffman distribution (Fleshler & Hoffman, 1962) with a mean of 120 s. Reinforcement consisted of the delivery of a sucrose pellet and was signaled by a noticeable but brief (0.1 s) flash of the house light.

Table 1.

Order of experimental phases for each subject in Experiment 1.

Subject	Phase
3-1	MAINT1	RemT	MAINT2	NCR	MAINT3	PRE	MAINT4	EXT
3-2	MAINT1	EXT	MAINT2	NCR	MAINT3	PRE	MAINT4	NCR
3-3	MAINT1	PRE	MAINT2	PRE	MAINT3	NCR	MAINT4	EXT
3-4	MAINT1	PRE	MAINT2	PRE	MAINT3	NCR	MAINT4	EXT
3-5	MAINT1	NCR	MAINT2	EXT	MAINT3	EXT	MAINT4	PRE
3-6	MAINT1	NCR	MAINT2	EXT	MAINT3	EXT	MAINT4	PRE
4-1	MAINT1	RemT	MAINT2	PRE	MAINT3	EXT	MAINT4	NCR
4-2	MAINT1	NCR	MAINT2	PRE	MAINT3	EXT	MAINT4	NCR
4-3	MAINT1	EXT	MAINT2	NCR	MAINT3	PRE	MAINT4	NCR
4-4	MAINT1	RemT	MAINT2	NCR	MAINT3	PRE	MAINT4	EXT
4-5	MAINT1	PRE	MAINT2	EXT	MAINT3	NCR	MAINT4	EXT
4-6	MAINT1	PRE	MAINT2	EXT	MAINT3	NCR	MAINT4	EXT
Sessions in Phase	13	3	16	3	14	3	13	3

Note. The order in which rats experienced each phase progresses from left to right. The number of sessions in each phase is listed in the bottom row. If a subject experienced a disruption condition twice, only the first disruption condition was analyzed. Refer to Table 2 for abbreviations.

Table 2.

Key abbreviations and DBERM parameters

Schedules
MAINT	Maintenance. Reinforcement was scheduled according to a tandem variable-time (VT) fixed ratio (FR) schedule.
EXT	Extinction disrupter. Lever presses had no programmed consequences.
NCR	Non-contingent reinforcement disrupter. Reinforcement was scheduled according to a variable-time (VT) schedule.
PRE	Pre-feeding disrupter. Same as MAINT, but access to chow was provided prior to session.
DBERM Parameters
Lt	Mean length of a response bout at time t.
wt	Within-bout response rate at time t.
bt	Bout-initiation rate at time t.
HL(X)	Half-life of parameter X. The time it would take for X to decline to half of its estimated value at t = 0.

All subjects began the experiment in MAINT. After each MAINT session, the stability of the response bouts was assessed over a 5-session window. Once stability was detected (see Appendix A for details), all rats were switched to a response disruption phase.

During a response disruption phase, each subject was exposed to a response disrupter for 3 consecutive sessions. There were 3 types of response disrupters:

1. Extinction (EXT). The experimental contingencies were identical to MAINT, except that lever pressing never resulted in a houselight flash or sucrose pellet delivery. Lever presses were recorded but had no programmed consequences.

2. Non-contingent reinforcement (NCR). Pellets were delivered according to a VT 120 s schedule. The experimental contingencies in NCR were similar to MAINT, except that the houselight flash and sucrose pellet delivery occurred at the end of the programed interval independent of lever pressing.¹ Lever presses were recorded but had no programmed consequences.

3. Prefeeding (PRE). Rats were provided ad libitum access to their homecage chow for 1 h immediately prior to the experimental session. The experimental contingencies were identical to those in MAINT.

During all 3 sessions of a response disruption condition, each rat was exposed to only one response disrupter. The order in which rats were exposed to the response disrupters was counterbalanced (Table 1). MAINT resumed on the session immediately following the end of each disruption condition and continued until response-rate stability was reestablished.

The majority of subjects began responding at high rates within the first 7 sessions. However, 3 rats (3-1, 4-1, and 4-3) exhibited low response rates, fewer than 12 responses/min, during the first MAINT phase, whereas the other 9 rats were responding at more than 20 responses/min. In order to establish higher levels of responding, rats responding at low rates were excluded from stability analyses for the first MAINT phases, and introduced to three sessions of remedial training (RemT) instead of the first response disruption phase (Table 1). During RemT, subjects were exposed to an ascending schedule sequence of VI 24 s, VI 46 s and VI 96 s, with a new schedule each day. On the fourth day, RemT was deemed effective in raising response rate, and rats were returned to MAINT.

Due to experimenter error, subjects 3-3 and 3-4 were exposed to PRE twice, and subjects 3-5 and 3-6 were exposed to EXT twice. A fourth treatment phase was added to the end of the experiment to expose all subjects to all treatment conditions. Where subjects experienced a disrupter condition twice, only the first exposure was analyzed.

Experiment 2

LE rats were trained on a tandem VT FR schedule of reinforcement, as in MAINT in Experiment 1, and were exposed to two disrupter conditions, EXT and NCR. Because the LE rats were experimentally naïve, they were first pretrained, which involved chamber habituation (day 1), magazine training (day 2), autoshaping (days 3 and 4), and the gradual decrease in reinforcement rate (days 5–10) until the target tandem VT 120 s FR 5 schedule was reached. During chamber habituation, each subject was placed in its chamber for 1 h; no experimental events occurred, except for the delivery of 5 sucrose pellets at the start of the session. Magazine training consisted of 45 individual sucrose pellet deliveries on a VT schedule ranging from 45 to 90 s. Autoshaping consisted of 45 daily trials in which the left lever was presented for 8 s followed by a single sucrose pellet delivery; a lever press ended the trial and immediately delivered a sucrose pellet. The inter-trial interval during autoshaping was variable and ranged from 45 to 90 s.

Days 5 to 13 consisted of one day each of the following schedules in consecutive order: continuous reinforcement, FR 5, tandem VT 3-s FR 5, tandem VT 6-s FR 5, tandem VT 12-s FR 5, tandem VT 24 s FR 5, tandem VT 49-s FR 5, and tandem VT 98-s FR 5. On day 14, rats began training on a tandem VT 120-s FR 5, where contingencies of reinforcement were identical to those in Experiment 1.

After 23 days of training on the VT 120 s FR 5 schedule, bout-like responding was not apparent. The mean of the median daily response rate for sessions 13–23 was 51.4 responses/min. In comparison, the overall mean response rate for the WKY during the last 5 sessions of MAINT in Experiment 1 was 28.3 responses/min. Examination of log survivor plots of the IRTs (not shown) suggested that rats were responding at a nearly constant average rate without noticeable bouts (i.e., IRTs were exponentially distributed). To allow for detectable bouts, the reinforcement rate was halved, changing the schedule to a tandem VT 240-s FR 5. This schedule produced more appreciable response bouts.

Rats then proceeded to train on the tandem VT 240-s FR 5 for both maintenance components (MAINT1 and MAINT2). Similar to Experiment 1, rats were first trained on the VT 240-s FR 5 (MAINT1) to stability before being exposed to 3 sessions of a disruption condition (either EXT or NCR; NCR was a VT 240 s instead of VT 120 s). Rats were then retrained on the tandem VT 240-s FR 5 (MAINT2) to stability before being exposed to 3 sessions of the alternate disruption condition. Table 3 describes the order and duration of each condition for each rat.

Table 3.

Order of experimental phases for each subject in Experiment 2.

Subject	Phase Order
5-1	MAINT1	EXT	MAINT2	NCR
5-2	MAINT1	NCR	MAINT2	EXT
5-3	MAINT1	EXT	MAINT2	NCR
5-4	MAINT1	NCR	MAINT2	EXT
5-5	MAINT1	EXT	MAINT2	NCR
5-6	MAINT1	NCR	MAINT2	EXT
5-7	MAINT1	EXT	MAINT2	NCR
5-8	MAINT1	NCR	MAINT2	EXT
5-9	MAINT1	EXT	MAINT2	NCR
5-10	MAINT1	NCR	MAINT2	EXT
Session in Phase	24	3	19	3

Note. The order in which subjects experienced each phase progresses from left to right. The number of sessions in each phase is listed in the bottom row. Refer to Table 2 for abbreviations.

Data Analysis

Response rates generally declined across consecutive disruption sessions, which indicated that (a) there were fewer responses to model in the second and third disruption sessions relative to the first one, and (b) it would be inappropriate to pool the responses across all three sessions, as their estimated parameters would almost certainly be different. For these reasons, only performance in the first disruption session of each condition was analyzed.

DBERM Parameter Estimation

The dynamic bi-exponential refractory model (DBERM; Brackney et al., 2011; Cheung, Neisewander, & Sanabria, 2012) assumes that IRTs are generated by a mixture of two independent Poisson processes, which underlie the within-bout response rate (w_t) and the bout-initiation rate (b_t). The mixture weighting parameter, p_t, is the probability of remaining in a bout after each response, and δ is the estimated shortest IRT possible (response refractory period).

$$\begin{array}{l}Pr({\mathit{IRT}}_t=\tau \mid \tau <\delta )=0\\ Pr({\mathit{IRT}}_t=\tau \mid \tau \ge \delta )=p_t{w}_t{e}^{-w_t(\tau -\delta )}+(1-p_t)b_t{e}^{-b_t(\tau -\delta )},min(\text{IRT})\ge \delta >0;w_t\ge b_t>0;1\ge p_t\ge 0\end{array}$$ (1)

where t is the time in the session. The parameter p_t is a function of the mean bout length (excluding the bout-initiating response), L_t,

$$p_t=\frac{L_t}{1+L_t}.$$ (2)

At any time t, run rate (response rate computed without post-reinforcement pauses) may be recovered from Equations 2 and 3,

$$\text{Run}\text{Rate}\text{at}\text{time}t=\frac{1}{p_t(1/w_t)+(1-p_t)(1/b_t)}=\frac{(1+L_t)w_t{b}_t}{w_t+L_t{b}_t}.$$ (3)

Changes in the parameters of Equations 1 and 2 are difficult to characterize. A common method to estimate these parameters is to fit them to the semi-log survivor plot of IRTs using the method of least squares (e.g., Shull et al., 2001). Shull and colleagues (2002) attempted to estimate changes in IRT distribution parameters during extinction by segmenting the extinction session and implementing the survivor-plot method in each method. This approach failed primarily because reliable parameter estimates require a large number of IRTs per segment, which reduces the number of segments and undermines the characterization of change. An alternative method involves defining a cutoff IRT length, below which IRTs are classified as within-bout, and above which IRTs are classified as between-bouts (Mellgren & Elsmore, 1991; Shull et al., 2002). This method has multiple limitations, including a high likelihood of misclassifying short between-bout IRTs as within-bout IRTs², and setting a ceiling to within-bout IRTs at an arbitrary cutoff length (Brackney & Sanabria, 2015). An alternative method assumes that parameters L_t, w_t, and b_t decay exponentially over the course of each disrupter session. This method, which minimizes classification errors and eliminates ceiling estimates, has been successfully implemented in past research (Brackney & Sanabria, 2015; Brackney et al., 2011, 2012; Cheung et al., 2012) and is implemented again here.

L_t, w_t, and b_t, at the start of the session (when t = 0) are referred to as L₀, w₀, and b₀, which together constitute the baseline parameters. Over the course of the session, L_t, w_t, and b_t, are assumed to decay exponentially from their starting values at rates α, β, and γ, such that

$$\begin{array}{l}{L}_t=L_0{e}^{-\gamma t}\\ w_t=w_0{e}^{-\alpha t}\\ b_t=b_0{e}^{-\beta t}.L_0,w_0,b_0,\gamma \ge 0;\beta \ge \alpha \ge 0\end{array}$$ (4)

To ease interpretation, the decay parameters are reported as half-lives [e.g., HL(L) = ln(2)/γ], the time taken for the parameter to reach half of its baseline value (L₀, w₀, or b₀). Table 2 lists and describes the key DBERM parameters that served as dependent measures.

DBERM was fit to each rat’s IRTs during each disruption condition (NCR, EXT, and PRE), using the method of maximum likelihood (Myung, 2003), implemented on custom-written software in Matlab® 2013a (Mathworks, Inc; Natick, MA). DBERM was fit to the aggregate IRTs of the last 5 days of each MAINT condition (i.e., one set of parameters were estimated for the entire 5 days, per subject, per MAINT condition). DBERM was also fit to the first session of each disruption phase. When a rat was exposed to the same disruption condition twice (see Table 1), the second exposure to that disruption condition (all 3 sessions) and the MAINT condition prior to it were excluded from analysis for that rat.

Response Rate Recovery by Simulation

Monte Carlo simulations were conducted to ensure that the underlying DBERM parameter estimates accurately reflected the observed response rates. The simulation was conducted as follows: for each rat and phase a series of Bernoulli trials were conducted with success probability p_t (Equation 1). A success yielded an IRT sampled from an exponential distribution with a mean of 1/w_t; a failure yielded an IRT sampled from an exponential distribution with a mean of 1/b_t. The sampled IRT then advanced the simulation clock by its respective value, and another trial began. The simulation ended when the session clock exceeded 55 min, and the final IRT was removed from the list of simulated IRTs. Each simulation also included enforced pauses that corresponded to post-reinforcement pauses observed for each subject. During those times, the simulation clock still advanced, but no responses could be produced. One hundred simulations were run for each rat and phase, and then averaged to produce mean predicted response rates.

Null Hypothesis Significance Tests (NHST)

NHST on response rates were conducted in IBM SPSS v22 (Armonk, NY: IBM Corp). To assess whether response rate changed significantly during the first session of disruption in each condition relative to MAINT, responding in each session was partitioned into eight equal-length bins of 6.9 min. The 5 sessions of MAINT per animal prior to the disruption session was collapsed into a single measure per bin by taking the median response rate in each bin. A 2 (phases) × 8 (bins) repeated measures ANOVA was then conducted on log-transformed response rates. As the goal of these tests was simply to verify that response rate was generally lower in disruption relative to MAINT, post-hoc pairwise tests using Fisher’s LSD were conducted when a significant interaction was observed. Bins between phases were first compared, and if the source of the interaction was not revealed, bins within phases were also compared. Pair-wise test outcomes are only reported when the corrected p < 0.05.

To test whether there was a significant change in DBERM parameters between MAINT and each disruption condition, log-transformed parameter estimates in disruption and in the preceding MAINT were examined with paired t-tests. Because there are six free parameters in DBERM [L₀, w₀, b₀, HL(L), HL(w), and HL(b)]³, a Dunn–Šidák correction for six comparisons was applied using an expected Type I error rate of 5%. As a result of this correction, p-values lower only than 0.0085 were reported as significant effects. Cohen’s d is reported for every significant difference between parameter estimates.

RESULTS

Response Rates

Figure 1 displays the mean response rates for each experiment during MAINT and the first disruption session of each phase. Response rate during bin 1 varied markedly across phases in WKY, from 32 resp/min (MAINT) to 18 resp/min (PRE). In contrast, response rate during bin 1 was consistently about 52 resp/min in LE. Response rate declined within session in all phases, most steeply during extinction (75–90% decline between bins 1 and 8) and least so in PRE (16% decline). MAINT response rates declined between bins 1 and 8 by 36% in WKY and by 41% in LE.

Figure 1.

Binned response rates during maintenance (MAINT) and response disruption (EXT = extinction, NCR = non-contingent reinforcement, PRE = prefeeding). The markers are the observed mean (±SEM) response rates for Wistar Kyoto (WKY, Experiment 1) and Long Evans (LE, Experiment 2) rats. The lines are mean predicted response rates according to the DBERM simulation.

Experiment 1 (WKY)

For EXT, there was a significant phase × bin interaction effect on response rate [F(7,77) = 2.230, p = 0.041]. Follow up tests revealed that response rate was significantly lower in EXT than in MAINT in all bins. The source of the interaction was revealed clearly by an examination of each bin in MAINT relative to a central bin (bin 5), and by the same examination in EXT. In MAINT, response rates in bin 5 were only significantly lower than in bins 1 and 3, and only significantly higher than in bin 8. In contrast, in EXT, response rates in bin 5 were significantly lower than in bins 1, 2, and 3, and significantly higher than in bins 7 and 8. Combined, these tests indicate that response rate declined more rapidly in EXT than in MAINT.

For NCR, there was a significant phase × bin interaction effect on response rate [F(7,77) = 2.448, p = 0.025]. Follow up test indicated that response rate was significantly lower in NCR relative to MAINT in all bins except 1 and 7, indicating that response rate was similar at the beginning of NCR and MAINT, but quickly dropped during NCR.

For PRE, there was a significant main effect of phase [F(1,11) = 30.927, p < 0.001] and of bin [F(7,77)= 5.078, p < 0.001] on response rate. These results indicate that response rate was lower in PRE than MAINT, but both decreased at similar rates.

Experiment 2 (LE)

For EXT, there was a significant phase × bin interaction effect on response rate [F(7,63) = 17.523, p < 0.001]. Follow up tests indicated that response rate was significantly lower in EXT than in MAINT for all bins except the first. These results indicate that response rate at the start of EXT was similar to MAINT, but declined more rapidly.

For NCR, there was a significant phase × bin interaction effect on response rate [F(7,63) = 2.64, p = 0.019)]. Follow up tests indicated that response rate was significantly lower in NCR than in MAINT in bins 2, 4, 5, and 6. These results indicate that response rate at the start of NCR was similar to MAINT, but rapidly declined during the middle of the session.

DBERM parameters

Figure 2 displays the mean (±SEM) estimates of DBERM baseline parameters and their half-lives, averaged across instantiations of MAINT. At the onset of MAINT sessions, rats produced, on average, bouts of 8–10 (L₀ + 1) rapid responses (2.0–3.4 resp/s), emitted 7–9 times per minute. Bouts declined in length at a barely detectable pace (half-lives > 3.5 months) during MAINT sessions. Within-bout rate was virtually constant for WKY, but declined over the course of a 60-min MAINT session by about 3% in LE (half-life ≈ 22 h). Bout-initiation rates declined more steeply, by about 13% in WKY (half-life = 5.0 h) and 41% in LE (half-life = 1.3 h). These changes in IRT distribution parameters accounted for a substantial proportion of the decline in response rate between bins 1 and 8 of MAINT (Fig. 1): 32% for WKY and 74% for LE. Changes in post-reinforcement pauses, which DBERM does not model, potential binning artifacts in Fig. 1, and sampling error accounted for the remainder.

Figure 2.

Mean (±SEM) estimates of DBERM baseline parameters (left panel) and their half-lives (right panel) during MAINT, for WKY (black columns) and LE (gray columns) rats. Baseline bout length (L₀) is reported in number of responses (left y-axis), and excludes the bout-initiating response. Baseline rates are reported in responses (w₀) and bouts (b₀) per minute (min⁻¹, right y-axis). Half-lives [HL(L), HL(w), and HL(b)] are reported in minutes.

DBERM also accounted for a substantial proportion of the decline in response rate between bins 1 and 8 of disrupter sessions: on average, 76% for WKY and 69% for LE. DBERM almost completely accounted for this decline in EXT (on average, 91%), where there are no post-reinforcer pauses. Figure 3 shows the mean (±99% CI) log-transformed ratios of estimated disrupted DBERM parameters (left panels) and their half-lives (right panels) relative to maintenance estimates.

Figure 3.

Mean (±99% CI) log-transformed ratios of estimated disrupted DBERM parameters (left panels) and their half-lives (right panels) relative to preceding maintenance estimates, for WKY (top panels) and LE (bottom panels) rats. Each disrupter is identified with a symbol of a different color. Asterisks indicate significant changes in estimates between disruption and preceding maintenance, p < 0.0085 (Type I error rate = 5% with Dunn–Šidák correction for 6 comparisons). There was no PRE disruption in Experiment 2.

Experiment 1 (WKY)

EXT significantly decreased HL(b) [t(11) = 3.33, p = 0.003, d = 1.09], and HL(w) [(t(11) = 3.78, p = 0.007, d = 0.96], indicating that the within-bout response rate and bout initiation rates decreased more rapidly in EXT, compared to MAINT. NCR significantly decreased HL(L) [t(11) = 3.28, p = 0.007, d = 0.95], indicating that bout lengths decreased more rapidly in NCR than in MAINT. PRE decreased b₀ [t(11) = 6.24, p < 0.001, d = 1.81], indicating that initial bout initiation rates were slower in PRE compared to MAINT.

Experiment 2 (LE)

EXT decreased HL(b) ([t(9) = 7.22, p < 0.001, d = 2.29] and L₀ [t(9) = 3.402, p = 0.008, d = 1.08] relative to MAINT. Although no significant effects of NCR were detected on DBERM parameters, NCR reduced mean HL(L) noticeably (d = 0.76, the highest for any non-significant effect in Experiment 2). We highlight this change because (a) it is consistent with a similar, but significant effect in Experiment 1, and (b) it appears to be the main contributor to the significant phase (MAINT vs. NCR) × bin interaction effect earlier reported on response rate (Figure 1, right panel, green curve).

DISCUSSION

The notion that operant behavior is organized in bouts implies that multiple, potentially independent processes may underlie changes in response rate. Response rate may decline because the frequency of bouts declines, responding within bouts declines, or bouts become shorter. Results from the present study suggest that these components of response rate are differentially sensitive to three changes in training conditions: rate of reinforcement (EXT), response-reinforcement contingency (NCR), and reinforcer efficacy (PRE). The effects of these changes were generally consistent in two strains of rat, WKY (Experiment 1) and LE (Experiment 2): whereas NCR primarily reduced bout length, EXT and PRE primarily reduced bout-initiation rate, with EXT accelerating its decline over the course of the session and PRE reducing it at the onset of the session (the latter yielding a reduction in response rate by a constant proportion across the whole session). These results suggest that rate of reinforcement and reinforcer efficacy primarily control the rate at which response bouts are initiated (Brackney et al., 2011; Podlesnik et al., 2006; Reed, 2011, 2015; Shull, 2004; Shull et al., 2001, 2004; Shull & Grimes, 2003), whereas response-reinforcement contingency primarily controls the length of those bouts (Brackney & Sanabria, 2015; Reed, 2011; Shull et al., 2001, 2004; Shull & Grimes, 2003; Tanno, 2016). The dissociability of motivational processes and schedule control is a key feature of various theories of operant performance (e.g., Baum, 2012; Killeen, 1994), which the analysis of operant bouts may aid in quantifying.

Although not explicitly tested, response rate appears to decline faster in EXT than in NCR in both strains (Figure 1). This effect is consistent with the notion that non-contingent reinforcers slow down the decline of non-reinforced responding (Rescorla & Skucy, 1969). NCR effects on DBERM parameters appear to provide some insights on the mechanisms underlying this effect. Unlike EXT, NCR did not induce a significant decline in bout-initiation rate in either experiment. Instead, NCR appears to induce a faster decline in bout length (a significant effect in WKY, a trend in LE). NCR-induced bout-length effects may be partially explained by adventitious reinforcement of alternative behaviors that compete with the operant (Skinner, 1948). In such case, however, a decline in bout-initiation rate would also be expected (Smith et al., 2014). Alternatively, when reinforcement occurs while a rat is responding within a bout, the sudden reinforcer delivery may interrupt the bout and thus reinforce shorter bouts, with repeated interruptions gradually reducing the average bout length over time. Future research may explicitly examine whether interrupting bouts at different points in their occurrence (e.g., at the initiation versus several responses into the bout) have a differential effect on response disruption.

There were two noticeable divergences from the expected effects of EXT on the organization of operant behavior. First, within-bout response rates appear to decline faster during EXT than during MAINT in WKY rats. Brackney et al. (2012) also reported an unusual pattern of extinction performance in WKY rats: extinction appears to induce a remarkably fast decline in bout length in this strain (a trend of which was observed also in Experiment 1). It is thus likely that inbred strains of rat such as WKY may display extinction-induced changes in aspects of behavior that are robust to extinction in outbred strains such as LE and Sprague Dawley. In particular, it is possible that multiple components of the organization of operant behavior decline simultaneously with extinction, each at a rate that varies with the state of the other components. Indeed, Podlesnik & Sanabria (2011) suggests an analogous model, in which changes in arousal modulate the rate at which learning occurs. The processes governing bout-initiation rate may exert a similar modulatory effect on other parameters of behavioral organization, but more weakly in WKY than in other strains of rat.

The second unexpected effect was that EXT reduced estimates of baseline bout length (a significant effect in LE, a trend in WKY). Bout length, however, could not have declined at the onset of the EXT session, when extinction contingencies had not yet been experienced. Instead, this effect suggests a limitation in the bout-length decay function assumed by DBERM (first line of Equation 4); more complex decay functions may overcome this limitation (e.g., Cheung et al., 2012). In any case, in neither WKY nor in LE, the size of the EXT effect on baseline bout length (Cohen’s d = 0.60 and 1.08, respectively) was as large as the effect on the half-life of bout-initiation rate (Cohen’s d = 1.09 and 2.29, respectively).

Although all response disrupters decreased response rate, the sources of those differences appear to vary across disrupters, as evidenced by selective changes in response bout parameters. Theories of behavioral persistence, such as behavioral momentum theory (Nevin & Grace, 2000) generally treat these disrupters as functionally interchangeable, assuming response rate as the critical dependent measure to be explained. The present study suggests that focusing instead on the parameters of the organization of behavior reveals distinct behavioral effects associated with each disrupter. Such approach has already been proven useful in elucidating the dynamics of schedule and temporal control of behavior (Daniels & Sanabria, 2017; Higa, 1996; Higa, Wynne & Staddon, 1991; Palya, 1992).

HIGHLIGHTS.

• The organization of operant behavior was disrupted in two strains of rat

• Extinction progressively reduced the rate at which response bouts were initiated

• Non-contingent reinforcement progressively reduced the length of response bouts

• Pre-feeding abruptly reduced the rate at which response bouts were initiated

• Analyses focused on response rate obscure key features of behavioral persistence

Appendix A: Stability Assessment

To assess the stability of response bouts, each individual subjects’ IRTs were fit to DBERM (Equations 1 and 3) by maximum likelihood estimation using custom written Matlab® software, on a session-by-session basis. For simplicity, it was assumed that α = β = γ = 0. A linear regression was fit to each parameter—L, w, and b—over a moving window of 5 sessions, using the method of least squares. AICc (Anderson and Burnham, 2002; Hurvich & Tsai 1989) was then used to assess whether the arithmetic mean of each BERM parameter over the 5-session window was a better fit than the linear regression. If the AICc of the mean was lower than the AICc of the regression by more than 4 units, then regressed parameter estimates were judged to be stable. If the parameters L, b, and w were all stable, performance was judge stable and a treatment condition was initiated on the following day.