Partial Reinforcement Effects on Acquisition and Extinction of a Conditioned Taste Aversion

Abstract

Four experiments with rat subjects asked whether a partial reinforcement extinction effect (PREE) occurs in taste aversion learning. The question has received little attention in the literature, and to our knowledge no taste aversion experiment has previously demonstrated a PREE. In each of the present experiments, experimental groups received a taste mixed in drinking water for 20 min; such taste exposures were sometimes paired with a lithium chloride (LiCl) injection and sometimes not. Control groups received only taste-LiCl pairings. There was evidence that each reinforced and nonreinforced trial produced increments and decrements in aversion strength (respectively), and trials mattered more than accumulated time during the conditioned stimulus and during the background (as emphasized in time-accumulation models like those of Gallistel and Gibbon, 2001, and Gibbon and Balsam, 1981). In addition, a partial reinforcement extinction effect was observed when there was a relatively large number of conditioning trials. The results extend our understanding of extinction in taste aversion learning and provide more evidence that aversion learning might follow rules that are qualitatively similar to those of other forms of learning.

Taste aversion learning has sometimes been considered a unique form of learning. When it was first discovered and studied in the 1950s and 1960s, at least two of its features suggested that it was special. First, aversion learning could occur when the taste conditioned stimulus (CS) and the illness unconditioned stimulus (US) were separated by hours rather than a matter of seconds (e.g., Garcia, Kimeldorf, & Koelling, 1966; Smith & Roll, 1967) (“long delay learning”). Second, illness was readily associated with taste but not audiovisual cues, whereas footshock was readily associated with audiovisual cues and not taste (e.g., Domjan & Wilson, 1972; Garcia & Koelling, 1966; see also Gemberling & Domjan, 1982) (the “cue to consequence” or “preparedness” effect). Many discussions of the uniqueness of aversion learning followed (e.g., Rozin & Kalat, 1971; Seligman, 1970; Shettleworth, 1972). The view of Rozin and Kalat (1971) has been especially enduring: The unique way in which the taste of a poisonous food and its gastrointestinal consequences might typically occur in nature might encourage evolution of a unique learning system as a biological adaptation. However, by the early 1980s, the taste aversion phenomenon had been more widely researched, and it began to be accommodated by an expanding set of general learning principles, as Michael Domjan concluded in at least two scholarly reviews (Domjan, 1983; Domjan & Galef, 1983).

Extinction is one of the many well-known conditioning phenomena that occur in taste aversion learning. That is, a taste aversion created by a taste-illness pairing will become weaker if the animal then receives repeated exposures to the taste alone. Moreover, extinction of taste aversions seems to follow rules that may be similar to those derived from other conditioning preparations. For example, extinction of a taste aversion appears to be sensitive to the context, as suggested by reports of the renewal effect (in which an extinguished aversion conditioned in one context recovers when it is returned there after extinction in another, e.g., Rosas & Bouton, 1997, 1998; Rosas, García-Guttiérrez, & Callejas-Aguilera, 2007; see also Sjödén & Archer, 1989). Taste aversion extinction is also sensitive to retention interval; spontaneous recovery of the aversion can occur if time is allowed to elapse after extinction (e.g., Brooks, Palmatier, Garcia, & Johnson, 1999; Rosas & Bouton, 1996, 1998). Thus, as with other forms of classical and instrumental conditioning, extinction in taste aversion learning does not appear to “erase” or destroy the original learning, but might create context-specific new learning, where “context” may be provided by conditioning apparatus, place, or time (e.g., Bouton, 2017; Bouton, Maren, & McNally, 2021). Two other response-recovery phenomena that occur after extinction in other preparations may be more difficult to produce in taste aversion learning (reinstatement: Bouton, 1982; but see Schachtman, Brown, & Miller, 1985; rapid reacquisition, Danguir & Nicolaidis, 1977; Hart, Bourne, & Schachtman, 1995); an explanation has not been fully explored (see Ricker & Bouton, 1996).

The present experiments were designed to study another well-known extinction phenomenon, the partial reinforcement extinction effect, in a taste aversion preparation. In this phenomenon, CSs or responses that are intermittently paired with a reinforcer are known to extinguish more slowly than CSs or responses that are reinforced whenever they occur, i.e., continuously (see Mackintosh, 1974, for one review). The phenomenon is widely known in appetitive instrumental learning, but has also been shown in classical conditioning preparations (e.g., Harris, 2019). In an especially striking version of the effect, a partially-reinforced appetitive CS can evoke more responding than a continuously-reinforced CS in extinction even though it elicited less responding at the end of conditioning—that is, the extinction curves can cross (e.g., Bouton, Woods, & Todd, 2014). Importantly, however, the PREE has received very little attention in taste aversion learning, even though it might have clinical significance. For example, some drugs that cause taste aversion conditioning also cause the taste to elicit conditioned immune responses whose medical effectiveness might be prolonged if extinction could be slowed when the taste is presented without the drug (see Hadamitzky, Lückemann, Pacheco-López, & Schedlowski, 2020). We are aware, however, of only one published experiment testing the PREE in taste aversion learning (Dwyer, Gasalla, & López, 2017). In that experiment, rats drank a saline CS and then received LiCl either immediately after every saline presentation or after every other one. The partially-reinforced group received three paired trials and the continuous group received six, and the partially-reinforced group received an LiCl dose of twice the volume received by the continuously-reinforced group. Interestingly, during extinction, the aversion extinguished at a comparable rate when measured either by saline consumption (the standard measure of taste aversion) or by lick-cluster size (thought to index the rat’s hedonic reaction, e.g., Davis & Smith, 1992; Dwyer, Boakes, & Hayward, 2012). The results suggest that a PREE does not occur with all taste aversion procedures, although the results of a single experiment with one partially- and one continuously-reinforced group clearly deserved follow-up. The present experiments did that, and used the opportunity to study the trial-by-trial effects of partial reinforcement on acquisition as well.

Experiment 1

The first experiment took an initial look at whether the PREE occurs in taste aversion learning. Its design is illustrated in Figure 1, where each group’s treatment is summarized in a row in which each symbol represents the event that occurred on each day (R = reinforced trial; N = nonreinforced trial; and - = neither R nor N trials). After becoming accustomed to a two-drinks-a-day schedule, four groups of water-deprived rats received the treatments shown. The experimental events always occurred during the first scheduled drink (which allowed the rat to drink what it needed for the day in the second drink if consumption of the first drink was suppressed). Two CRF (continuous reinforcement) groups received a saline solution (the CS) on trials administered at four-day intervals. Each saline presentation was followed immediately by an ip injection of isotonic (.15 M) lithium chloride (LiCl). To prevent rapid approach to a consumption floor, we gave LiCl at half the volume we often use (10 ml/kg instead of 20 ml/kg, e.g., Bouton, Allan, Tavakkoli, Steinfeld, & Thrailkill, 2021; Thrailkill, Michaud, & Bouton, 2021). Two other groups (PRF, for partial reinforcement) received the same reinforced trials, but additional saline exposures without LiCl on each of the three intervening days. In the extinction phase, all groups then received a series of nonreinforced exposures to saline. One of the groups in each condition received these trials on consecutive days, whereas the other received them four days apart (“Spaced”). Notice that the scheduling of extinction trials maintained the intervals between CS exposures received by the CRF and PRF groups in acquisition and thus helped control for generalization decrement created by differences in trial spacing between the acquisition and extinction phases. If the PREE occurs in taste aversion learning, we expected slower extinction of the aversion in the PRF than the CRF groups.

Figure 1.

Designs of the experiments. Each symbol in a row represents a day. R = reinforced (CS-US) trial; N = nonreinforced (CS – no US) trial; - = water only day.

Method

Subjects and apparatus

Twenty-three female Wistar rats purchased from Charles River (Raleigh, NC) were used. They were approximately 120 days old at the start of the experiment and had previously served in an instrumental learning experiment in which they had been food-deprived and learned to lever press and chain pull for grain-based food pellets. The rats had been individually housed on sawdust in plastic home cages (31.75 x 19.05 x 19.05 cm), and all of the present experimental procedures were conducted in those cages. Experimental solutions were delivered through a standard metal spout that extended from a plastic water bottle affixed to the top of the cage. The saline drink CS was a 0.15 M NaCl (isotonic) solution that involved mixing table salt (non-iodized, PICS by Price Chopper) with ultra-pure reverse osmosis water. Injections were given ip with a 10 ml/kg dose of a 0.15 M lithium chloride (LiCl) solution.

Procedure

Drink Training

For the first five days of the experiment, the rats were water deprived except for a 20-min drink of tap water in the home cage at 9:30 am and a second 20-min drink approximately two hours later. Food was available on an ad lib basis, but was removed immediately prior to the first drink and returned immediately following the second. The animals were weighed each day prior to the first drink. Water bottles were weighed before and after each drink throughout the experiment. The twice-daily drink schedule was maintained throughout the experiment.

Conditioning

Following the five days of drink training, the animals were randomly assigned to two groups: PRF (partial reinforcement, n = 12) or CRF (continuous reinforcement, n = 11). Conditioning was then conducted over the next 13 days. As summarized in Figure 1, all rats received four reinforced trials (R) over these days. The saline CS was only given during the first drink. Reinforced trials consisted of a 20-min saline drink followed immediately by LiCl injection. Nonreinforced (N) trials consisted of a 20-min drink of saline with no subsequent injection. (We did not give sham injections on N trials to avoid unnecessary perforation of the rats’ abdomens; in our hands, there are usually no effects of sham injections on consumption, and they are rarely if ever used on taste aversion extinction trials.) The second daily drink (always water) allowed the rats to compensate if consumption had been suppressed during the first drink.

Extinction

The PRF and CRF groups were then each divided into two extinction groups, Spaced and Not Spaced, which received extinction of saline over trials separated by three days or one day, respectively. The two intervals maintained the spacing of CS exposure that the CRF and PRF groups had received during conditioning. Extinction itself followed the procedure used in conditioning, except that LiCl was never administered. All rats received four extinction trials.

Results

Acquisition

The results of the conditioning phase are summarized in Figure 2, which shows saline consumption each time it was presented over the 13 conditioning days. Rats in the CRF condition displayed substantially suppressed saline consumption relative to the PRF group. A 2 (Group) x 4 (Trial) ANOVA on saline consumption on the four reinforced trials revealed significant effects of group, F(1, 21) = 43.61, MSE = 1564.27, p < .01, and trial, F(3, 63) = 15.07, MSE = 297.97, p < .01, and a group x trial interaction, F(3, 63) = 15.17, MSE = 300.00, p < .01.

Figure 2.

Taste aversion acquisition over trials in Experiment 1, with arrows representing reinforced (R) trials. Error bars represent standard errors of the means.

Extinction

The extinction results are summarized in Figure 3. Here, a 2 (Reinforcement condition: CRF vs. PRF) x 2 (Extinction spacing: Spaced vs. Not Spaced) x 4 (Extinction trial) ANOVA revealed significant main effects of Reinforcement condition, F(1) = 33.84, MSE = 4282.23, p < .01, and Extinction Trial, F(3) = 11.88, MSE = 72.06, p < .01. The analysis also revealed a significant Reinforcement x Extinction Trial interaction, F(3) = 3.21, MSE = 19.49, p = .03, that took the form of faster extinction in the PRF groups. There was also a Trial x Extinction Spacing interaction, F(3) = 7.14, MSE = 43.30, p < .01, suggesting that the saline aversion extinguished more rapidly when the trials were three days rather than one day apart. There were no other significant effects, largest F = 2.10.

Figure 3.

Results of the four extinction trials in Experiment 1. Error bars represent standard errors of the means.

Discussion

Partial reinforcement involving three N trials between each R trial weakened aversion conditioning substantially. However, there was no evidence of a PREE: If anything, the weaker aversion in the PRF groups appeared to extinguish somewhat faster than the aversion in the CRF groups, perhaps because consumption in the CRF groups was not far from a consumption floor. Interestingly, there was also an effect of the spacing of extinction trials suggesting that spaced extinction was quicker, particularly in the PRF condition. However, the weaker aversion in Group PRF Spaced was evident even on the first extinction trial, before spacing was actually manipulated. This suggests that the effect was a result of sampling error rather than a real effect of trial spacing on extinction.

Experiment 2

In an effort to generate stronger aversion conditioning with a PRF procedure, the second experiment tested for a PREE in PRF groups that had only two (rather than three) N trials between each R trial. The experiment also took a closer look at the importance of the actual R and N trials in the acquisition of the aversion. Some models of conditioning, though rarely applied to taste aversion learning, suppose that trials themselves are less relevant than the amount of time spent in the CS and in the intervals between each reinforcer presentation (e.g., Gallistel & Gibbon, 2001; Gibbon & Balsam, 1981). Gibbon and Balsam (1981), for example, proposed that animals remember time in the CS (trial time, or T) as well as the time between reinforcers (C) and respond proportionate to the ratio between them (C/T). Importantly, time in the trial is assumed to accumulate between reinforcers in the calculation of T.

As explored in previous work (e.g., Bouton & Sunsay, 2003; Harris & Bouton, 2020), the emphasis on C/T rather than actual R and N trials makes novel predictions about the effects of partial reinforcement on acquisition. For example, if a series of reinforced trials has an average interval of C, adding nonreinforced N trials between the R trials (an “Add-N” procedure) will increase the accumulating T without affecting C and thus decrease the C/T ratio—resulting in less conditioning in the PRF than the CRF condition. However, if one instead removes the US from scheduled R trials, the method would create a PRF schedule that proportionately increases both C and T. Thus, if an animal was reinforced on every third scheduled CS presentation, both C and T would triple compared to a condition where every CS was reinforced, preserving the C/T ratio and thus predicting no effect of partial reinforcement on acquisition in this arrangement.

Experiment 2 tested these predictions for the first time in taste aversion learning. As shown in Figure 1, a core CRF group (Group CRF) had CS-LiCl pairings spaced every three days. One PRF group (Add N) had two nonreinforced trials added between each CS-LiCl pairing, whereas a second PRF group (Subtract R) omitted LiCl on two out of the three scheduled R trials—tripling both C and T. According to the Gibbon-Balsam model (and the related Gallistel-Gibbon model), we should observe a decrement in conditioning relative to Group CRF in the Add N group, but not in the Subtract R group. Of course, if the aversion increases after each R trial and decreases after each N trial, as other models of conditioning assume, then we might expect to observe a partial reinforcement decrement in both groups. A fourth group (CRF Long) received a CRF procedure with the same intervals between R trials as the Subtract-R group. (Group Subtract R therefore had an “Add N” relation to Group CRF Long.) After the end of conditioning, which was arranged so that all groups finished conditioning on the same calendar day, extinction began with ITIs approximating the geometric mean of the intervals received by the two CRF groups and the adjacent PRF groups in the figure (with incidentally had add N relationships). A PREE would be represented by slower extinction in the two PRF groups. It is worth noting that the Gibbon-Balsam model makes no predictions about extinction, and that the Gallistel-Gibbon model does not use C/T (in their case, I/T, where I is the accumulating intervals between CSs) in explaining the PREE and would therefore predict a similar PREE in the two PRF groups tested here.

Method

Subjects

Thirty-two naïve female Wistar rats (75-90 days old, Charles River) were housed and maintained identically to those in Experiment 1. As in Experiment 1, conditioning and extinction were conducted in the animals’ home cages. The saline CS drink and 0.15 M LiCl solution (injected ip at 10 ml/kg) were also the same as in Experiment 1.

Procedure

Drink Training

The rats were again trained to drink in twice-daily 20-min sessions for the first five days of the experiment. The rats were water deprived except for a 20-min drink at 2:30 pm and a second 20-min drink approximately two hours later. Food was available on an ad lib basis, but was removed prior to the first drink and returned following the second. The animals were weighed each day before the first drink.

Acquisition

Conditioning then occurred following the daily procedure illustrated in Figure 1. As in Experiment 1, all exposures to saline (and LiCl injections) occurred with the first drink. The animals were assigned to four groups: Add N (partial reinforcement, adding nonreinforced trials), CRF (continuous reinforcement trials occurring on the same days as Group Add N), Subtract R (partial reinforcement, removing reinforced trials), and CRF Long (continuous reinforcement trials occurring on the same days as those of Group Subtract R). The scheduling of trials over days was arranged so that all groups received the final conditioning trial on the same calendar day, and hence after the same amount of time on the water deprivation schedule. All told, the conditioning phase lasted 28 days.

Extinction

All groups then received four extinction trials in which saline was presented in the first drink period without LiCl. The trials were spaced by 48 hrs in Groups PRF Add-N and CRF short, and 120 hrs in the PRF Subtract-R and CRF Long group. These intervals approximated the geometric mean of the intervals between saline exposures in the two sets of PRF-CRF groups during conditioning in order to control generalization between conditioning and extinction that might be influenced by the intertrial intervals (e.g., Church & Deluty, 1977).

Results

Data from one rat in the CRF Long group was excluded due to lack of aversion conditioning.

Acquisition

As in Experiment 1, the rats in the partial reinforcement groups demonstrated weaker aversion conditioning than the continuously reinforced groups (see Figure 4). A 2 (Reinforcement condition: PRF vs CRF) X 2 (Interval between reinforced trials: Short vs Long) x 4 (Trial) ANOVA on the reinforced trials revealed significant effects of Reinforcement condition, F(1, 27) = 65.40, MSE = 1287.07, p < .01, and trial, F(3, 81) = 51.98, MSE = 366.86, p < .01. There were also significant interactions between Reinforcement condition and trial, F(3, 81) = 39.80, MSE = 280.91, p < .01, and Short vs Long interval and trial F(3, 81) = 4.87, MSE = 34.34, p = .004, as well as a three-way interaction F(3, 81) = 8.34, MSE = 58.86, p < .01. The three-way interaction suggests that the interaction between the reinforcement condition and interval suggested by consumption on the second R trial was unique to that trial. There were no other main effects or interactions, and notably no significant effect of short versus long, suggesting that the rate of learning was not dependent on intertrial interval (highest F = 1.34).

Figure 4.

Taste aversion acquisition over trials in Experiment 2, with arrows indicating reinforced (R) trials. Error bars represent standard errors of the means.

Figure 5 isolates the reinforced trials for the CRF and Subtract R groups. Recall that the subtraction of the reinforcer on scheduled trials in Group Subtract R group gave these groups equal C/T or I/T ratios. As the figure makes clear, a robust effect of partial reinforcement was still evident, suggesting that the partial reinforcement effect on acquisition did not depend on C/T or I/T. A group x trial ANOVA that isolated these groups revealed a highly reliable group by trial interaction, F(3, 42) = 45.74, MSE = 229.39, p < .01, as well as main effects of trial, F(3, 42) = 51.58, MSE = 258.68, p < .01, and group, F(1, 14) = 25.63, MSE = 489.52, p < .01.

Figure 5.

Taste aversion acquisition in Experiment 2 isolating R trials for the Subtract R and CRF groups. Error bars represent standard errors of the means

Extinction

The results of the extinction phase are summarized in Figure 6. Interestingly, extinction occurred rapidly in the CRF Long group, but appeared equally slow in the other three groups. A 2 (PRF vs CRF) X 2 (Short vs Long) X 4 (Trial) ANOVA revealed significant effects of PRF vs CRF, F(1, 27) = 110.40, MSE = 3531.45, p < .01, Short vs Long, F(1, 27) = 16.16, MSE = 516.89, p < .01, and trial, F(3, 81) = 41.35, MSE = 211.64, p < .01. There was also a significant interaction for PRF vs CRF x Short vs Long, F(1, 27) = 6.17, MSE = 197.40, p = .019, as well as a three-way interaction, F(3, 81) = 16.51, MSE = 84.50, p < .01, that was consistent with the relatively rapid extinction observed in the CRF Long group. There were no other significant interactions (largest F = 1.83).

Figure 6.

Results of the four extinction trials in Experiment 2. Error bars represent standard errors of the means.

Discussion

During acquisition, there was a clear effect of partial reinforcement that was present in the groups that had equal C/T and I/T ratios (Groups CRF and Subtract R) (see also Dwyer et al., 2017). This result appears consistent with the conclusion, also supported by the trial-by-trial changes after R and N trials that was otherwise evident (see General Discussion), that each R and N trial has a discrete consequence in taste aversion conditioning. The effect of partial reinforcement on extinction rather than acquisition was somewhat more ambiguous. Although the relatively rapid extinction observed in one of the CRF groups (Group CRF Long) suggests that extinction was faster after CRF than PRF training, rapid extinction in Group CRF Long was peculiar to that group: Extinction in Group CRF Short was no faster than that in the PRF groups. The pattern suggests that some factor other than a PREE was responsible for the quicker extinction in Group CRF Long. Since its ITI was longer than that in Group CRF Short, a possible cause was the longer ITI.

Experiment 3a and 3b

Experiment 3 was designed to test for the PREE after further modification of the method. To address the possible presence of a ceiling effect on consumption in the PRF groups during extinction, we examined the effects of a conditioning procedure that involved only one N trial for each R trial. (The experimental design actually compared one-N and two-N procedures.) Although fewer N trials could result in a weaker PREE, the tradeoff was that we should also observe stronger conditioning, thus moving consumption away from a ceiling effect and perhaps making it easier to observe an unambiguous PREE. A second modification is that Experiment 3 used a saccharin solution instead of saline as the CS. Use of saccharin might reduce the chance that animals would generalize between the CS and water-exposure trials. If there were any such generalization, exposure to water during the longer extinction intertrial intervals in Group CRF Long could have been a factor creating the rapid extinction observed in that group.

The design of Experiments 3a and 3b is again summarized in Figure 1. In both experiments, a CRF group received LiCl after every exposure to the saccharin CS, whereas two PRF groups received nonreinforced exposures to saccharin between each R trial. Group PRF-2 received 2 N trials (as in Experiment 2), and Group PRF-1 received only one. In Experiment 3a, the rats received four reinforced trials, as shown in Figure 1, just as the rats in Experiments 1 and 2 had. However, because most studies of the PREE in other conditioning preparations have used more R and N trials than we had been administering here, Experiment 3b doubled the number of those trials by putting the groups through the trial sequence shown in Figure 1 twice. (Theories of the PREE also imply a bigger PREE with more conditioning trials; see General Discussion.) To compensate for the likelihood that the added reinforced trials would produce floor effects in saccharin consumption, we also halved the 10 ml/kg dose of LiCl used in the preceding experiments to 5 ml/kg in Experiment 3b.

Method

Subjects and apparatus

Experiments 3a and 3b respectively involved 23 and 24 naïve female Wistar rats (75-90 days old, Charles River). The rats were individually housed in plastic cages and maintained identically to those in Experiments 1 and 2. As before, conditioning and extinction were conducted in the home cage. However, here the experimental drink (CS) was a 0.1% (w/v) saccharin solution (Acros Organics, Geel, Belgium). LiCl injections were again given ip. Experiment 3a used the 10 ml/kg dose of 0.15 M LiCl used in the previous experiments, whereas Experiment 3b used a weaker 5 ml/kg dose.

Procedure

Drink Training and Acquisition

Drink training followed the procedure used in Experiments 1 and 2. Here the rats received the first drink at 2:30 pm. After five days, all rats received the daily events summarized in Figure 1. Reinforced trials consisted of a 20-min saccharin drink followed immediately by injection of LiCl. Nonreinforced trials consisted of a 20-min saccharin drink with no subsequent injection. As usual, experimental events always occurred during the first drink.

There were three groups in both Experiments 3a and 3b: CRF (continuous reinforcement), PRF-1 (partial reinforcement, with one N trial added between R trials), and PRF-2 (partial reinforcement, with two N trials added between R trials) (see Figure 1). In Experiment 3a, each group received the reinforced, nonreinforced, and water trials illustrated in the figure. In Experiment 3b, the number of trials was doubled by putting each group through the Figure 1 sequence twice.

Extinction

All groups then received 10 extinction trials. The intertrial interval (48 hrs) approximated the geometric mean of the intervals between saccharin presentations for the different conditioning schedules.

Results

Experiment 3a

Acquisition

The results of the acquisition phase are summarized in Figure 7. Rats in group CRF and PRF-1 demonstrated stronger conditioning than group PRF-2. A 3 (Group) x 4 (Trial) ANOVA isolating the reinforced trials revealed significant effects of group, F(2, 20) = 37.01, MSE = 275.29, p < 0.01, and trial, F(3, 60) = 18.93, MSE = 57.99, p < 0.01, as well as an interaction, F(6, 60) = 13.58 MSE = 41.61, p < 0.01.

Figure 7.

Taste aversion acquisition over trials in Experiment 3a. Arrows indicate reinforced (R) trials. Error bars represent standard errors of the means.

Extinction

While saccharin consumption in Group PRF-2 was higher than that in the other groups throughout extinction, the rate at which extinction proceeded did not appear to differ between the groups (see Figure 8). We ran separate group x trial ANOVAs comparing each of the PRF groups with the CRF control. In the PRF-2 vs CRF ANOVA, there were significant effects of group, F(1,13) = 32.88, MSE = 1181.25, p < .01, and trial, F(9, 117) = 7.62, MSE = 51.77, p < .01, but the interaction did not approach significance, F = 1.57. In the PRF-1 vs. CRF ANOVA, there was a significant effect of trial, F(9,117) = 5.69, MSE = 16.72, p < 0.01, and no other significant effects, Fs < 1.

Figure 8.

Results of the 10 extinction trials in Experiment 3a. Error bars represent standard errors of the means.

Experiment 3b

Acquisition

The results of the acquisition phase are summarized in Figure 9. Rats in group CRF and PRF-1 again demonstrated stronger conditioning than group PRF-2. A 3 (Group) x 8 (trial) ANOVA isolating the reinforced trials revealed significant effects of group, F(2, 21) = 29.88, MSE = 871.97, p < .01, and trial, F(7, 147) = 5.92, MSE = 21.88, p < .01, as well as a significant group x trial interaction, F(14, 147) = 6.40, MSE = 23.64, p < .01.

Figure 9.

Taste aversion acquisition over trials in Experiment 3b, with arrows indicating reinforced (R) trials. Error bars represent standard errors of the means.

Extinction

Experiment 3b’s extinction data are summarized in Figure 10. As in Experiment 3a, saccharin consumption was highest in Group PRF-2 throughout extinction, but this time the rate of extinction was higher in Group CRF than in the PRF groups-- suggesting a PREE. We again ran separate group x trial ANOVAs comparing each of the PRF groups with the CRF control. In the PRF-2 vs. CRF ANOVA, there were significant effects of group, F(1, 14) = 58.68, MSE = 1991.63, p < 0.01, and trial, F(9, 126) = 6.60, MSE = 37.85, p < .01, as well as a significant group x trial interaction, F(9, 126) = 3.37, MSE = 19.30, p = 0.001. In the PRF-1 vs CRF ANOVA there were also significant effects of group, F(1, 14) = 5.78, MSE = 558.76, p = 0.03, and trial, F(9, 126) = 4.33, MSE = 29.89, p < .01, and a group x trial interaction, F(9, 126) = 3.62, MSE = 24.98, p < .01. Notice that the CRF group’s consumption extinguished rapidly and eventually overlapped with consumption in the PRF-1 group. It is also worth noting that an ANOVA comparing Groups PRF-1 and PRF-2 revealed a significant effect of group, F(1, 14) = 5.65, MSE = 440.56, p = .032, and trial, F(9, 126) = 2.43, MSE = 11.10, p = .014, but no interaction, F(9, 126) = .354, MSE = 1.62, p = .954. The fact that Group PRF-1 consumed less saccharin than Group PRF-2 during extinction indicates that Group PRF-1’s consumption was not on the response ceiling. Overall, the results of this experiment thus provide evidence of a PREE in taste aversion learning.

Figure 10.

Results of the 10 extinction trials in Experiment 3b. Error bars represent standard errors of the means.

Discussion

The results of Experiment 3b suggest that extinction occurred more rapidly in a continuously reinforced group than in the two PRF groups. Particularly when comparing Groups CRF and PRF-1, the result seems difficult to interpret as a scaling effect created by less sensitivity to consumption differences at higher regions of the consumption scale. That is because Group PRF-1 had significantly lower consumption than Group PRF-2, indicating that PRF-1’s consumption was not on the ceiling, and Group CRF extinguished to the point where its consumption overlapped with that of Group PRF-1. The fact that a similar PREE was not produced in Experiment 3a, where half the number of conditioning trials were used, is consistent with the idea that the PREE might depend on procedures with more conditioning trials (e.g., Amsel, 1958; Capaldi, 1967). It is interesting to observe that, because taste aversion learning occurs very rapidly with typical parameters, taste aversion investigators might use a small number of trials and potentially miss a PREE.

Other differences between Experiments 3a and 3b should be noted. In Experiment 3a, adding a single N trial (Group PRF-1) did not weaken the aversion compared to the CRF group, whereas it did in Experiment 3b. The reason for the difference is not clear, although it could be due in principle to either the difference in the number of R and N trials or to the weaker US used in Experiment 3b. Strictly speaking, the occurrence of the PREE in Experiment 3b could relate to either of these variables. Of course, it is also true that other uncontrolled and unknown variables might have differed between the two experiments. It is notable, though, that among the different possibilities, a role for the number of trials is consistent with available theory (e.g., Amsel, 1958; Capaldi, 1967).

General Discussion

The results of Experiment 3b clearly suggest that a PREE can occur in taste aversion learning. The results of that experiment, which involved twice the number of R and N trials than the very similar Experiment 3a, suggest that the PREE may be most likely to occur after a relatively large number of conditioning trials. Of course, the number of trials there (8 R and 7 N) was still far fewer than typically employed in other experiments studying the PREE (but see, e.g., McCain & Brown, 1967). Since asymptotic taste aversion learning with standard methods can often occur with very few trials, it would be natural to use too few conditioning trials in taste aversion to observe a PREE—as was true for us in Experiments 1 and 2.

Another factor that might complicate observing a PREE in the taste aversion preparation is the possible presence of floor and ceiling effects. Near-zero consumption in a continuously-reinforced group could easily obscure unmeasured (and potentially “rapid”) extinction, whereas weak conditioning in a partially-reinforced group might make extinction look slow if consumption is at the ceiling of what the animal can drink. Given the rapidity with taste aversion learning can occur, floor effects are always a potential issue. And the results of the present experiments suggest that the inclusion of nonreinforced trials during conditioning can substantially weaken aversion acquisition. With the present methods, it was necessary to use only one N trial per R trial to produce a level of conditioning in a PRF group that was unambiguously off the consumption ceiling.

The results of the acquisition phase of Experiment 2 further confirmed that the local effects of each R trial and N trial matter in aversion conditioning. Models that argue for the importance of rate of reinforcement in the CS and in the background (calculated with the C/T or I/T ratio; Gibbon & Balsam, 1981; Gallistel & Gibbon, 2000), rather than the effects of individual trials, were challenged by the fact that there was a clear effect of partial reinforcement in acquisition with a “subtract R” procedure that preserved these ratios (Figure 5; see also Dwyer et al., 2017). The idea that the aversion might strengthen after every R trial and weaken after every N trial, which is assumed by most other theories of conditioning, was also consistent with the acquisition data in each of the present experiments. To examine this further, we collapsed the data from all PRF groups over all four experiments and analyzed the change in consumption from each R trial to the next trial in separate Trial x Experiment ANOVAs. There was a significant decrease in consumption (Trial main effect) after every R trial, smallest F(1, 56) = 23.78. In contrast, when we analyzed the change in consumption after every N trial that immediately followed those R trials (where consumption was never near the ceiling), there was a significant increase in consumption after every trial, smallest F(1,48) = 25.95. Overall, the results suggest that individual trials do matter in taste aversion learning.

The PREE has often been explained by two types of theory. According to frustration theory (e.g., Amsel, 1958), during partial reinforcement, the response is reinforced in the presence of frustration generated on previous N trials. During extinction, the presence of frustration elicited by extinction will therefore evoke or set the occasion for the response and cause responding to persist. Alternatively, other approaches have emphasized that extinction is harder to discriminate from acquisition under partial reinforcement than continuous reinforcement conditions, or that there is less generalization decrement when extinction trials begin (e.g., Mackintosh, 1974). The generalization decrement concept is elegantly built into Capaldi’s sequential theory (e.g., Capaldi, 1967; Capaldi & Martins, 2010), which suggests that during partial reinforcement the response is reinforced in the presence of memories of the preceding string of N trials, which can vary in length. During extinction, the presence of N memories generated by previous extinction trials will similarly cause behavioral persistence. The application of frustration theory to taste aversion learning may seem challenged by the fact that N trials in aversion learning are not likely to cause frustration. However, a different emotional response (e.g., relief) could be elicited by N trials and play an analogous role. It is worth noting that frustration theory predicts that the emotional response would grow stronger with more conditioning—potentially consistent with the observation of a stronger PREE with more trials (Experiment 3b). However, another issue is that the long intervals between an N trial and the next R trial (24-48 hrs in Experiment 3b) might allow any emotion to decay or dissipate before the next R trial. Turning to sequential theory, continuous and partial reinforcement procedures were originally supposed to affect S-R “habit” learning (because the theory was primarily designed to explain appetitive instrumental conditioning experiments). Although habit per se seems unlikely to result from a pairing of a taste and illness, “associative strength” can substitute for “habit strength” without damaging the theory. And sequential theory notably predicts that a PREE may increase with more trials in which R trials follow N trials.

Harris (2019) has recently reviewed partial reinforcement effects in appetitive Pavlovian conditioning experiments in his laboratory and in others. According to his analysis, an explanation of the PREE, particularly when observed in within-subject procedures where it is now known to occur (e.g., Chan & Harris, 2017, 2019; Harris & Kwok, 2018; Rescorla, 1999), implies that the animal must encode a rich and episodic-like memory of reinforced and nonreinforced trials with a given CS. For example, Chan and Harris (2019) gave rats intermixed trials in which one CS was reinforced on every trial and another was reinforced on every third trial. When both were extinguished, responding to the partially-reinforced stimulus took longer to extinguish, demonstrating a within-subject PREE, but more interestingly, the CS reinforced on a third of the trials took three times as many extinction trials before responding began to decrease (see also Bouton et al., 2014, for a between-subjects demonstration); a CS reinforced on 1/5 the trials took five times as many trials to begin extinction as a CS that had been continuously reinforced on intermixed trials. Such results suggest a fairly precise encoding of nonreinforced trials with a given CS. Harris (2019) noted that a within-subject PREE is difficult for either sequential theory or frustration theory to handle unless they are amended to allow the animal to remember occasions of reinforcement or nonreinforcement (in sequential theory) or frustration (in frustration theory) with a specific CS. The animal further remembers the entire trial and encodes it as nonreinforced. Given the common view that taste aversion learning is a rather visceral and non-cognitive form of learning (e.g., part of the gut feedback or gut defense system, e.g., Garcia, 1989; Garcia, Lasiter, Bermudez-Rattoni, & Deems, 1985), the application of this idea to taste aversion learning may be especially interesting and provocative. Of course, the current demonstration of the PREE was from a between-subjects rather than within-subject experiment, and the present results do not necessarily force the same conclusion.

The possibility that a PREE occurs in taste aversion learning again suggests that aversion learning follows rules that may be qualitatively similar to those of other forms of associative learning. This may be true because the rules of operation required to learn how to identify poisonous foods in nature are not truly “functionally incompatible” with those required for other forms of associative learning (e.g., Sherry & Schacter, 1987). Thus, it might make sense for the animal to stay away from a food that has occasionally been poisoned even when extinction trials begin—just as it might make sense to persist when a partially-reinforced food-seeking response begins extinction. These observations may explain why, as Michael Domjan (1983) observed, “detailed experimental analyses of various constraints on learning have led to explanation of these phenomena in terms of generally applicable principles of behavior” (pp. 265-266).