Associative mechanisms involved in specific Pavlovian-to-instrumental transfer in human learning tasks

Abstract

Four experiments compared the effect of forward and backward conditioning procedures on the ability of conditioned stimuli (CS) to elevate instrumental responding in a Pavlovian-to-instrumental transfer (PIT) task. Two responses were each trained with one distinct outcome (R_1-> O₁, R₂->O₂), either concurrently (Experiment 1) or separately (Experiments 2, 3 and 4). Then, in Experiments 1 and 2, four CSs were either followed or preceded by one outcome (A->O₁, B->O₂, O₁->C, O₂->D). In Experiment 3, each CS was preceded and followed by an outcome: for one group of participants, both outcomes were identical (e.g., O₂, O₁->A->O₁, O₂->B->O₂), but for the other, they were different (e.g., O₁->A->O₂, O₂->B->O₁). In Experiment 4, two CSs were preceded and followed by identical outcomes, and two CSs by different outcomes. In the PIT tests, participants performed R₁ and R₂ in the presence and absence of the CSs. In Experiments 1 and 2, only the CSs followed by outcomes in Pavlovian training elevated responding. In Experiments 3 and 4, all the CSs elevated responding but based on the outcome that followed them in training. These results support the stimulus-outcome-response (S-O-R) mechanism of specific PIT, according to which CSs elevate responding via activation of its associated outcome representation.

It has been well established that Pavlovian conditioned stimuli (CS) can affect behavior in multiple ways. For example, in appetitive conditioning procedures, the CS has been shown to elicit “goal-tracking” conditioned responses (CRs) directed to the location of the unconditioned stimulus (US), and also “sign-tracking” CRs directed to the stimulus itself (e.g., Brown & Jenkins, 1968; Farwell & Ayres, 1979; Holland, 1979). In both cases, the CS is said to acquire some of the incentive motivational properties of the US with which it was previously paired (e.g., Bindra, 1968; Boakes, 1979; Konorski, 1967; Robinson & Berridge, 1993). These “incentive motivational” effects of CSs can be seen in other ways as well, perhaps the most straightforward of which is the acquired ability of the stimulus to support higher order conditioning (e.g., Holland & Rescorla, 1975). Yet another way in which incentive motivational effects of CSs have been studied is with a procedure known as Pavlovian-to-instrumental transfer (PIT, for a review see Cartoni, Balleine, & Baldassarre, 2016; Holmes, Marchand, & Coutureau, 2010). The task consists of three phases: an instrumental training phase, in which one or more responses are each reinforced with a distinctive outcome (e.g. R₁->O₁, R₂->O₂), a Pavlovian conditioning phase, in which one or more CSs is each paired with one of the outcomes (e.g., CS₁->O₁, CS₂-O₂), and a test phase in which subjects are allowed to perform the instrumental responses, usually under extinction conditions, in the presence and absence of the CSs. This task is especially useful in revealing acquired incentive motivational effects of stimuli because it illustrates how a CS comes to invigorate responses other than those directly conditioned to the stimulus. It is this feature of the test that suggests the stimulus has acquired incentive motivational properties.

This task has been used extensively in both human and non-human animal experiments, and it has been consistently found that CS presentations can exert both selective (e.g., Bray, Rangel, Shimojo, Balleine, & O ‘Doherty, 2008; Colwill & Rescorla, 1988; Delamater, 1995; Kruse, Overmier, Konz, & Rokke, 1983; Paredes-Olay, Abad, Gámez, & Rosas, 2002), and general (e.g., Corbit & Balleine, 2005, 2011; Lewis, Niznikiewicz, Delamater, & Delgado, 2013; Nadler, Delgado, & Delamater, 2011) incentive motivational effects on instrumental performance. The specific PIT effect is revealed when presentations of a CS selectively increase performance of an instrumental response previously reinforced with the same outcome as that signaled by the stimulus, for example, in the presence of CS₁, R₁ > R₂ (in the scenario described above). The general PIT effect is revealed when a CS non-selectively energizes instrumental responding above baseline levels, even when the CS and response were reinforced previously with different outcomes (e.g., Corbit & Balleine, 2005, 2011).

These PIT phenomena have been extensively studied in recent years, and researchers have discovered that distinct neural mechanisms underlie the specific and general forms of PIT (Corbit & Balleine, 2005, 2011; Corbit, Fischbach, & Janak, 2016). This suggests that there are multiple kinds of incentive motivational properties acquired by a CS as a result of Pavlovian conditioning (see also Robinson & Berridge, 1993; Robinson & Flagel, 2009). Nonetheless, currently there is some disagreement as to the nature of the psychological mechanisms involved, especially, in selective PIT. The first theoretical explanation of this phenomenon is based on the ideas of Trapold and Overmier (1972) in their “expectancy” version of two-process learning theory (c.f., Rescorla & Solomon, 1967). Following Trapold and Overmier (1972), the specific PIT effect can be explained through a stimulus-outcome-response (S-O-R) associative chain mechanism. More specifically, during instrumental conditioning, they assumed (along the lines of Thorndike, 1898) that at the time of reinforcement an association is strengthened between the stimuli present in the situation and the response. During initial lever training, for instance, the outcome just earned and consumed could itself serve as a discriminative stimulus when the next lever press is reinforced. In addition, the animal is also thought to develop a Pavlovian association between the experimental chamber and the specific food reward. Upon being placed into the apparatus, this association could generate an expectation of the outcome, and so long as that expectation is present at the time that the response is reinforced the O-R association could be further strengthened. Furthermore, when explicit Pavlovian training occurs with a discrete CS, this could also result in the establishment of a highly specific S-O association, such that the S evokes a specific expectation of the very same US that also separately entered into an association with the response. These two associations, then, could be used in an associative S-O-R chain during a test for specific PIT (e.g., Kruse et al., 1983).

A critical aspect of this S-O-R account is that instrumental conditioning results in the formation of O-R associations, an assumption that is not consistent with evidence indicating that instrumental performance is mainly governed by R-O associations (Adams & Dickinson, 1981; Colwill & Rescorla, 1985; Rescorla, 1992). A second explanation of PIT acknowledges that S-O and R-O associations both contribute to specific PIT. In order for this explanation to work, however, it has been necessary to assume that R-O associations can work in a bidirectional manner (Asratyan, 1974; Mackintosh & Dickinson, 1979; Pavlov, 1932); thus, activation of an outcome representation at test by the CS can produce the response associated with that outcome by utilizing the R-O association in a backward direction (e.g., Gilroy, Everett, & Delamater, 2014; Rescorla, 1992).

Although these two versions of an S-O-R associative chain account can easily explain the specific PIT effect, recent data challenge these views. Cohen-Hatton, Haddon, George, and Honey (2013) trained rats to perform two responses, each of them reinforced by either pellets or a sucrose solution, i.e., R₁->O₁, R₂->O₂. In a subsequent Pavlovian conditioning phase, presentations of one CS were followed by one of the outcomes, i.e., forward CS₁->O₁ pairings, and presentations of a second CS were preceded by the other outcome, i.e., backward O₂->CS₂ pairings. A larger PIT effect was produced by the backward-trained CS₂ than by the forward-trained CS₁ in this study (Cohen-Hatton et al., 2013). This result is problematic for the S-O-R accounts described above because according to those accounts the specific PIT effect depends on the excitatory strength of the S-O associations at the time of test, and backward training has often been observed to produce either weaker excitatory learning than forward training, or inhibitory, rather than excitatory, CS-US associations (e.g., Hall, 1984; Maier, Rapaport, & Wheatley, 1976; Moscovitch & LoLordo, 1968; Siegel & Domjan, 1971, 1974).

Cohen-Hatton et al. (2013) proposed an alternative mechanism to explain specific PIT. They suggested that bidirectional R-O associations are formed during instrumental training. Then, in the Pavlovian conditioning phase that follows, each time an outcome is delivered, it is assumed to activate a representation of the response with which it was trained. By virtue of the contiguity between the CS and this representation of the response, an association is further assumed to be formed between the CS and the response. In this way, a specific S-R association can be established during the Pavlovian training phase. Then at test, the CS presentation directly would elicit responding without any mediation by the outcome representation, i.e., through an S->R association. This framework explains superior PIT by backward than forward-trained CSs on the grounds that the response representation is more temporally contiguous with the CS in the backward procedure than in the forward procedure. In the forward procedure, the outcome occurs at CS offset and, consequently, the presumed response representation could only be activated by the outcome at a time when the CS is no longer present. In the backward procedure; however, the outcome occurs just before the CS is presented. In this case, the outcome-evoked response representation would coincide with the presentation of the CS, so long as the US-CS interval is short, and for this reason, superior S-R learning should arise in the backward procedure (Cohen-Hatton et al., 2013).

One potential problem with this explanation is that sometimes the opposite effect is observed. In one experiment, Delamater, LoLordo, and Sosa (2003) studied backward conditioning using a PIT task and observed that a backward-trained CS reduced the level of responding on a lever trained with the outcome paired with the backward CS (see also Laurent, Wong, & Balleine, 2015). As argued by Delamater et al. (2003), this result is consistent with more standard views of PIT that emphasize the effect the CS has on the outcome representation at the time of the PIT test, provided one assumes that their backward CSs had acquired outcome-specific inhibitory, rather than excitatory, associations.

These contradictory results (Cohen-Hatton et al., 2013; Delamater et al., 2003; Laurent et al., 2015) might be explained by procedural differences in these studies. In the experiments reported by Delamater et al. (2003) and Laurent et al. (2015), the CS was presented 10 s after the outcome was delivered, in order to ensure that the animals had time to approach the food trough, retrieve the pellet, and consume it prior to presentation of the CS. In contrast, in the backward trials of the Cohen-Hatton et al. (2013) experiments, the CS was presented only |1 s after the subjects entered the food magazine. If subjects were still consuming the outcome at the time of CS presentation, this would have generated a simultaneous rather than backward pairing of the CS and US. Other research has suggested (Wagner, 1981) and shown (e.g., Rescorla, 1980, 1985) that simultaneous training can produce stronger associative learning than forward pairings (for a review, see Nasser & Delamater, 2016).

If this is the basis of the conflicting results described above, then a useful procedure would be one that allows for greater control over the timing of events. Although this might be challenging with animal subjects (due to difficulties in controlling when an appetitive outcome is consumed), it can easily be achieved by using human participants. Indeed, there has recently been a large number of studies and an explosion of interest in examining PIT effects in various human learning tasks, both with an eye toward examining psychological and neural mechanisms (Alarcón & Bonardi, 2016; Allman, DeLeon, Cataldo, Holland, & Johnson, 2010; Bray et al., 2008; Colagiuri & Lovibond, 2015; Hogarth & Chase, 2011; Hogarth, Maynard, & Munafò, 2015; Lewis et al., 2013; Lovibond & Colagiuri, 2013; Lovibond, Satkunarajah, & Colagiuri, 2015; Nadler et al., 2011; Paredes-Olay et al., 2002; Prévost, Liljeholm, Tyszka & O ‘Doherty, 2012; Rosas, Paredes-Olay, García-Gutiérrez, Espinosa & Abad, 2010; Talmi, Seymour, Dayan, & Dolan, 2008; Trick, Hogarth, & Duka, 2011; Watson, Wiers, Hommel, & De Wit, 2014). In many of these studies, neutral images have been used as CSs and appetitive images, such as food or drink pictures, as outcomes. This type of task allows us to control the exact duration of each of these stimulus presentations in order to compare the effect of CSs trained in a backward or forward conditioning procedure in a PIT task. In the series of experiments reported here, two instrumental responses were initially trained with distinct outcomes (i.e., R₁->O₁, R₂->O₂). These responses were either trained concurrently (Experiment 1) or in sequential phases (Experiment 2, 3 and 4). In the Pavlovian phase of Experiments 1 and 2, each of two CSs were immediately followed by one of the outcomes, i.e., were given forward pairings (A->O₁, B->0₂), and each of two additional CSs were immediately preceded by one of the outcomes, i.e., were given backward pairings (O₁ >C, 0₂->D). In Experiments 3 (between-subjects) and 4 (within-subjects), each of the CSs was preceded and followed by either the same or a different outcome on every conditioning trial to directly compare, within trials, the relative magnitudes of backward and forward conditioning. In all experiments, we assessed the relative degree of specific PIT resulting from these forward and backward conditioning procedures, in which participants had the chance to perform R₁ and R₂ in the presence and absence of the CSs. We consistently observed stronger selective PIT effects with forward than backward-trained CSs in all of these studies.

Experiment 1

The goal of this experiment was to compare the specific PIT effect produced by CSs given forward or backward conditioning procedures, in order to resolve conflicting evidence supporting the S-O-R and the S-R accounts of PIT. A computer task was created using a PIT procedure similar to that used in the animal literature (see Table 1). This task comprised an instrumental training phase, a Pavlovian conditioning phase and a PIT test. In addition, at the end of the experiment, subjects answered a series of questions designed to assess the degree to which the participants had explicitly learned the Pavlovian (S-O) and instrumental (R-O) contingencies. The stimuli used in this task were neutral fractal images as CSs (A, B, C, D), and images of different drinks and foods as outcomes (O₁, O₂).

Table 1.

Design of Experiment 1.

Instrumental	Pavlovian	PIT test	Questionnaires
R1->O1	A->O1	A: R1 vs R2	A, B: O1, O2?
R2->O2	B->O2	B: R1 vs R2	C, D: O1, O2?
	O1->C	C: R1 vs R2	R1, R2: O1, O2?
	O2->D	D: R1 vs R2

A, B, C and D: neutral fractal images; R₁ and R₂ : keyboard responses; O₁ and O₂ : food and drink images.

In the instrumental phase participants were asked to press two keys in order to obtain as many outcomes as they could (R₁->O₁, R₂->O₂). In the subsequent Pavlovian phase, one of four CSs was presented on the screen at a time. Two of these CSs were followed (forward CSs: A->O₁, B->O₂) and the other two were preceded (backward CSs: O₁->C, O₂->D) by one of the outcomes. Importantly, participants were asked to press a key each time they saw the outcomes (one key for O₁ and another for O₂), and these response keys were different and spatially orthogonal to those trained in the instrumental phase. This was done in order to differentiate the stimuli designated as outcomes from those designated as CSs, and can be thought of as “instructed unconditioned responses.” In the PIT test, participants were free to perform both instrumental responses (R₁ vs R₂) while each of the CSs was presented. These responses and CSs were tested under extinction conditions. The rate of responding was measured in the presence and absence of the CSs during this test. It was expected that the forward CSs (A and B) would exert selective control over instrumental responding indicative of specific PIT, i.e., A: R₁ > R₂ and B: R₂ > R₁. The critical question concerned the backward CSs (C and D). According to the S-R mechanism, these cues should produce a larger selective increment of instrumental responding relative to the forward CSs because of their more favorable temporal contiguity relations. In contrast, the S-O-R account predicts that backward CSs should produce weaker selective PIT to the extent that forward conditioning results in stronger associative learning than backward conditioning (see also Delamater & Joseph, 2000). Finally, participants were asked about each of the CS-outcome and response-outcome relationships in order to obtain a measure of explicit contingency learning.

Method

Participants

Thirty-five students aged between 18 and 41 years (18 males and 17 females) from Brooklyn College-CUNY participated in this study. Three of them were excluded from this experiment: one failed to complete the instrumental phase, one did not perform any response during the Pavlovian phase and another abandoned the task during the PIT test to ask questions to the researcher. All the participants received course credit for their participation.

Apparatus and materials

The task was programmed and run in PsychoPy (Peirce, 2007), using a standard computer with a 22-inch screen. Four neutral fractal images served as CSs, and in each of the CS trials, two identical fractal images were presented, symmetrically 5 cm to the left or right of the center of the screen. A set of six drink images and six food images were used as outcomes (O₁, O₂). Each time an outcome was delivered one image from the corresponding set was randomly selected and presented at the center of the screen. All the CSs and outcomes were 8 × 8 cm in size (see Figure 1 for CSs and outcomes images). A fixation cross (10 × 10 mm) was positioned at the center of the screen during the instrumental phase and PIT test, and this cross was replaced by a fixation dot (3 × 3 mm) during the Pavlovian phase. The responses involved pressing the keys “z” and “m” during the instrumental training phase, and the keys “t” and “g” during the Pavlovian phase. Questionnaire items were answered by clicking on a visual analog scale (Questionnaire 1) or by pressing the numbers “1,” “2” or “3” on the keyboard (Questionnaire 2). Participants were asked to press the “space bar” to advance through each of the phases. The physical stimuli designated as A, B, C and D were counterbalanced across subjects such that two of the fractal images were assigned to the roles of A and B and the other two assigned to C and D, or the reverse. The specific stimuli assigned to play the roles of A and B as well as C and D were counterbalanced with each other, resulting in 8 counterbalancing conditions. In addition, for one set of 8 conditions food images were designated as O₁ and drink images as O₂, and the reverse was true for another set of 8 conditions, resulting in a total of 16 counterbalancing conditions.

Figure 1.

Stimuli Used in Experiment 1. Top: Images Used as CSs. Middle: Images Used as Food Outcomes. Bottom: Images Used as Drink Outcomes. All the stimuli were color images.

Procedure

Participants were seated in front of the computer in a quiet room and were provided with an informed consent form for the experiment. Task instructions were presented on the computer screen at the beginning and prior to each phase of the experiment, which lasted approximately 20 min. At the beginning of the task, the following instructions were presented on the screen:

This is a simple game in which your goal is to discover the relationships between different images, actions and rewards (images of FOOD and DRINKS).

You will have to pay attention to the screen and press some keys to obtain or to predict rewards.

When you are ready please press SPACE BAR to continue.

Good luck!

Instrumental phase.

Participants were instructed to discover the relations between the keys (R₁, R₂) and the rewards (O₁, O₂), and to obtain 50 rewards of each type. The following instructions were presented on the screen:

In this part your goal is to press different keys (“Z” and “M” keys) to obtain different types of rewards (images of DRINKS or FOODS). It will be helpful to learn the relationships between your key presses and the different rewards.

A “+” will appear on the screen throughout this part of the game. Please keep your eyes focused on this. You can press either the “Z” or the “M” key in order to produce the rewards but nothing will happen if you press “Z” or “M” during the reward presentations.

Your goal is to get 50 of each reward but you will not get rewards all the time, so please keep trying- you can press as many times or as quickly as you see fit! Two counters will let you know how many of each reward you have earned.

When you are ready please press space to continue.

The responses were reinforced with the corresponding outcome according to a concurrent random ratio 5 (conc RR 5 RR 5) schedule. On average, O₁ was presented after 5 R₁ responses and O₂ after 5 R₂ responses. For each reinforced response, the outcome was presented for 0.8 s, replacing the fixation cross. If any key was pressed during this period, the outcome was immediately removed. The texts “Drink=0” in orange letters and “Food= 0” in blue letters were used as counters. They were positioned at the left and right top corners of the screen, respectively. Each time an outcome was delivered the value of the corresponding counter was incremented by 1. This phase ended when both counters were greater than or equal to 50.

Pavlovian phase.

Participants received the following instructions:

Now you will see a black dot centered on the screen. Once again please keep your eyes focused on this. At some point either a neutral image or a reward (DRINK or FOOD) will appear on the screen and if it is a DRINK you have to press “T,” but if it is a FOOD you have to press “G.” Your goal is to press as quickly and accurately as you can. Please do not press any key during the neutral images.

Sometimes the neutral images will appear before the rewards but other times after them. You may find these neutral images useful in predicting which type of reward may occur at any given time, so please pay attention to the neutral images as well as the rewards because later on you will have to answer some questions about these.

When you are ready, please press space button to continue.

At the beginning of each trial the text, “T ‘= Drink” in orange letters was positioned above the fixation dot, and text “G‘ = Food” in blue letter below the dot. After 3 s the texts were removed, leaving the dot alone for one additional second. On forward conditioning trials, this was followed by the CS presentation, in which each of the two identical images was presented at one side of the fixation dot. After 3 s the CS and the dot were removed, and the corresponding outcome was presented. Feedback was delivered immediately after participants performed a response. If the response was correct, the text “Correct! RT =X” in green letters appeared at the top of the screen, in which X was the reaction time (in ms) for that response. If the response was incorrect, or no key was pressed after 2 s, the text “Oops! That was wrong” in red letters was positioned below the outcome. The outcome and the feedback text remained on the screen for 1 s and then a new trial began. The backward trials were identical, except that the outcome was presented before the CSs. The phase comprised two blocks, each of them with 6 trials of A->O₁ B->O₂, O₁->C and O₂->D, and their order of presentation was pseudorandom.

PIT test.

Participants received the following instructions:

In this part of the experiment, you will see either a “+” sign or one of the neutral images and you have to press either the “Z” or the “M” key in order to obtain the rewards, just as in the first phase of the game. You can press the keys as many times or as quickly as you see fit.

When you are ready, please press space bar to continue.

On each trial, the fixation cross remained centered on the screen for 3 s (pre CS period), after which two identical CS images were positioned at each side of the cross (CS period). These images disappeared after 3 s, leaving the cross alone for one additional second (ITI period), and then a new trial began. The test comprised two blocks, each of them consisting of two trials of A, B, C and D, and the order of the CS presentations was pseudorandom.

Questionnaire 1.

In this section, participants were asked to indicate with which outcome each of the CSs and responses was previously related, by using a visual analog scale presented on the screen. Participants received the following instructions:

During this game you saw pairs of neutral images and rewards, and also pressing some keys was followed by those rewards. Now you need to determine with which reward each of the neutral images and responses was related.

You will see a scale on the screen and you need to use the mouse to select the correct answer.

When you are ready press SPACE to continue.

In the trials related to the Pavlovian associations, the text “This image was paired with” was presented at the top of the screen, and the corresponding CS image at the center of the screen. In the trials related to the instrumental associations, the text was “The ‘Z’/’M’ key was followed by” and no image was presented. In both types of trial, a visual analog scale was located at the bottom of the screen. At the left of the scale, a drink image with the word “Drink” below it was presented, and at the right, a food image with the word “Food” below it. At the middle of the scale, the word “Nothing” was positioned (below the scale). Participants had to answer by clicking at any point of the scale. This sections comprised two blocks, each of them with one trial of A, B, C, D, Z and M. The order of the trial presentations was pseudorandom.

Questionnaire 2.

Then participants received the following instructions:

Finally, you will see a few more questions about the relationships between the neutral images, the keys and the rewards. This time, please answer by using the number on the keyboard.

When you are ready press SPACE to continue.

In the questions referring to the Pavlovian associations, the text “This image was followed/preceded by” appeared at the top of the screen, and the corresponding CS below it. In the instrumental questions, the text was replaced by “The Z/M key was followed by” and no image was presented. In both types of questions, the text “1) Drink 2) Food 3) Dot/Cross” was positioned at the bottom of the screen. This test comprised two blocks, each of them with two trials of A, B, C, D, R₁ and R₂, and the order of the questions was pseudorandom.

Data treatment.

The data were analyzed using one-way repeated measures analysis of variance (ANOVA) tests. Significant differences were further explored with post-hoc tests following the method of Rodger (1974, 1975). This method entails construction of a set of ν₁ linearly independent and mutually orthogonal post-hoc contrasts for statistical evaluation, and assigning non-zero values to each rejected contrast and a value of zero for each non-rejected contrast (for more details see Rodger, 1974). Based on these statistical decisions, two measures of effect size were computed. The first is an estimate of the noncentrality parameter (Δ) that defines the F distribution when the null is false, and reflects the overall amount of variation among the population means justified by the empirical data. This parameter is estimated using Perlman and Rasmussen’s (1975) uniformly minimum variance unbiased estimator. The second measure of effect size is a computation of the implied population means, themselves, arising from the collective set of statistical decisions for the post-hoc contrasts within the set. These implied means are expressed as a difference between an individual mean, in question, and the overall average of all the population means comprising the analysis (μ_j-μ), and the computed values are reported in σ units that can be interpreted in a manner that is analogous to Cohen’s d (Cohen, 1988). Thus, a difference of 1 σ unit represents a large difference between two conditions.

Rodger’s method defines type I error in terms of the rate of erroneously rejecting null contrasts (within sets of ν₁ linearly independent contrasts) when they are true, and in the present studies, this expected rate was fixed at Eα = 0.05. Furthermore, with the sample sizes used in the present studies, the rate of accurately rejecting false null contrasts, i.e., statistical power, for moderately sized effects was at least Eβ = 0.95.

These methods of analysis were chosen over other ANOVA techniques because they are the most powerful and quantitatively precise methods currently available (Rodger & Roberts, 2013). All statistical procedures were conducted using software that is freely available for download from the following site: https://sites.google.com/site/spsprogram/home.

Before the analysis, participants’ reaction times in the Pavlovian phase were grouped according to trial type, i.e., forward and backward trials. The trials in which participants performed an incorrect response or did not perform any response were excluded from the analysis. The data of the PIT test were also grouped according to trial type (i.e., forward and backward CS). In addition, the responses made in the pre CS and CS periods were grouped as “same” or “different,” depending on the CS being previously paired with the same or different outcome than those paired with the respective responses. For instance, in the pre CS and CS period of A and C, which were paired with O₁, R₁ responses were grouped as “same” and R₂ responses as “different,” and the reverse for B and D. PIT scores were calculated by subtracting the responses made in the pre CS period from those made in the CS period. Thus, an elevation on performance produced by the CS presentation was reflected by positive PIT scores, and the specific PIT effect was found when this elevation was larger in the “same” than in the “different” responses. The data of “Questionnaire 1” were converted to values ranging between −100 and 100, where −100 represents selecting the opposite outcome, and 100 the correct outcome. The data of “Questionnaire 2” were grouped into “correct” and “incorrect” outcome responses. For example, if the question referred to which outcome followed A the “correct” outcome response was O₁ and the “incorrect” outcome O₂. The answers in which participants chose the option cross/dot were not considered in the analysis.

Results

All participants learned the two instrumental responses without incident and earned the required 50 outcomes of each type. The mean reaction times to the outcomes during the Pavlovian phase were collapsed across outcome identity and are shown separately for forward and backward training conditions over three 4-trial blocks of training (see left panel of Figure 2). The figure suggests that participants’ reaction times decreased after the first trial block and that they were slightly faster in the forward than in the backward trials. An ANOVA performed on these data revealed significant differences across training, F(5, 155)=3.62, mean squared error (MSE) = 0.007, p< 0.05, Δ = 12.87. Post-hoc tests showed no significant differences between forward and backward trials, F < 1, but they revealed that mean RTs in block 1 were greater than in Blocks 2 and 3, which did not differ, in both the forward, F(5, 155) = 1.55, p< 0.05, and backward trials, F(5, 155) = 1.72, p< 0.05. These data show that subjects learned to respond to the outcomes more efficiently across training.

Figure 2.

Left Panel: Mean Reaction Times Per Trial Block for Each Trial Type in the Pavlovian Phase of Experiment 1. Right panel: Mean Rates of “same” and “different” Responses for Forward and backward CSs Averaged Over Both Blocks of the PIT Test of Experiment 1. The Population Means Implied by our Statistical Analysis is also Reported in σ Units.

The data of most interest came from the PIT tests and are shown averaged over the two blocks of testing in the right panel of Figure 2. The data are shown separately for the response that had been reinforced with the same or different outcome as that paired with the stimuli, and these are further broken down for the stimuli trained in a forward or backward relation to the outcomes. The forward CSs selectively elevated the instrumental response rewarded with the same outcome as that paired with the stimuli, and had little effect on the response rewarded with the different outcome. In contrast, this effect was greatly attenuated in the presence of the backward CSs. An ANOVA performed on these data showed a significant main effect, F(3, 93)=2.82, MSE = 1100.01, p < 0.05, Δ= 5.29. Post hoc comparisons confirmed that participants performed more “same” than “different” responses in the presence of the forward CSs, F(3, 93)=2.52, p < 0.05, but not the backward CSs, F < 1. The implied population means derived from our statistical analysis are also presented in the right panel of Figure 2, illustrating the size of these differences in σ units. The mean rates of responding in the pre CS period were 71.3 (standard error of the mean [SEM] = 9.99) and 77.3 (SEM = 10.52) for “Same” and “Different” in the forward trials, and 70.3 (SEM = 10.59) and 81.0 (SEM = 11.23) for “Same” and “Different” in the backward trials. An ANOVA performed on these data showed significant differences, F(3, 93)=2.02, MSE=409.47, p < 0.05, Δ=2.92, and post-hoc comparisons showed a slightly higher rate of “different” responses relative to “same” responses in both forward and backward trials. However, this inexplicable difference was small (σ = 0.3) and affected both forward and backward trials similarly.

The scores obtained from the rating scale of “Questionnaire 1” were 7.2, −1.4, and 57.5, for the questions referring to the forward, backward, and instrumental contingencies, respectively (100=correct outcome, −10 0 = incorrect outcome). An ANOVA on these data showed a significant main effect, F(2, 62) = 18.86, MSE = 1718.19, p < 0.05, Δ = 34.51, and post-hoc comparisons confirmed that the scores to the instrumental questions were significantly higher than those to the questions about the Pavlovian associations, F(2, 62) = 18.52, p < 0.05, indicating superior knowledge of the instrumental contingencies relative to the Pavlovian relations. No difference was found between the scores to the forward and backward contingencies. The data of “Questionnaire 2” are presented in Figure 3. This figure shows the mean percentages of questions in which the correct and opposite outcomes were identified for the forward and backward stimuli and for the instrumental responses. The graph suggests that participants accurately identified the specific response-outcome but not the stimulus-outcome relationships. An ANOVA on these data revealed significant differences across ratings, F(5, 155) = 11.88, MSE =1031.1, p < 0.05, Δ= 53.64, and post-hoc comparisons confirmed a significant difference between correct and incorrect outcome responses in the questions about the instrumental relations, F(5, 155) = 11.09, p < 0.05, but not in the questions about forward or backward Pavlovian relations, Fs < 1. The implied population means are also indicated in the figure and reflect the differences just described.

Figure 3.

Mean Percentages of “correct” and “incorrect” Outcome Responses in the Questionnaire of Experiment 1. The Population Means Implied by Our Statistical Analysis is Also Reported in σ Units.

Discussion

The results of the PIT test in this experiment indicate that only the cues that were followed by the outcomes during training, i.e. forward CSs, produced the specific PIT effect. The backward CSs numerically displayed a specific PIT effect, but this difference fell short of statistical significance. As noted in the general introduction, both S-O-R and S-R accounts predict that forward CSs will produce the specific PIT effect, but only S-O-R accounts predict that this effect will be stronger than that produced by the backward CSs. The remaining experiments in this report explored the relative strengths of backward and forward training procedures in producing selective PIT by using different variants of the procedures employed here.

Experiment 2

The aim of Experiment 2 was to replicate the previous findings using a procedure that is more similar to those used in PIT studies conducted with rodents. First, in Experiment 1, the instrumental responses were trained concurrently, i.e., participants could perform both R1 and R2 within the same experimental phase. However, in most animal studies, the two instrumental responses are trained in separate sessions (Cohen-Hatton et al., 2013; Delamater et al., 2003; Laurent et al., 2015). This difference might be critical considering that backward (O->R) associations may be formed during instrumental training, as noted in the general introduction (see also Ostlund & Balleine, 2007). In a concurrent training procedure, it would, therefore, be possible for each outcome to be associated with each instrumental response, and this would be expected to reduce the specific PIT effect. Thus, in Experiment 2, we avoided this problem by training the two instrumental responses separately.

In addition, in Experiment 1, each of the “outcomes” used actually consisted of a set of 6 different stimuli taken from a single category (food or drink). This differs from rodent studies in that only 2 specific outcomes differing in their sensory features are generally used. The use of multiple food and drink instances may further dilute the magnitude of specific PIT, as well, to the extent that the effect is mediated by a specific common associate connected with both the stimulus and response. In Experiment 2, therefore, we only used one specific drink and one food image as our reinforcing outcomes.

Method

Participants

Seventeen students aged between 18 and 28 years (5 males and 12 females) from Brooklyn College-CUNY participated in this study. One participant was excluded because he did not perform any responses during the Pavlovian phase.

Procedure

Everything was the same as in Experiment 1, unless otherwise stated.

Instrumental phase.

Participants’ first goal was to press “z” to obtain as many drink images as they could. After 25 outcomes were earned, participants were instructed to press “m” to obtain as many food images as they could, which was also terminated after 25 outcomes were delivered. This sequence was repeated, so by the end of this phase participants earned 50 presentations of each outcome. These responses were reinforced by their appropriate outcomes with a RR 5 schedule, and, as in Experiment 1, subjects were asked to learn about the relationships between the responses and the specific outcomes.

Results

Instrumental training was successfully completed with all subjects earning 50 outcomes of each type. In addition, mean reaction times to the outcomes during the Pavlovian phase decreased over training, and there were no differences between backward and forward training trials, as in Experiment 1 (data not shown).

The PIT data are displayed in Figure 4 (as in Experiment 1). The graph shows a large specific PIT effect in the presence of the forward CSs, but this effect was numerically reversed in the presence of the backward CSs. An ANOVA performed on these data revealed a significant difference in responding across the two types of stimulus, F(3, 45)=2.77, MSE = 1,467.41, p < 0.05, Δ = 4.95. Post-hoc comparisons showed a significant difference between “same” and “different” responses on forward trials, F(3, 45)=2.47, p < 0.05, but this difference was not significant on backward trials, F < 1, which is also reflected by the implied population means shown in the figure. No response differences were found, overall, on forward and backward trials, Fs < 1. The mean rates of pre CS responding were 112.97 (SEM=22.05) and 132.81 (SEM=26.11), for “Same” and “Different” responses in the forward trials, and 126.09 (SEM=23.24) and 115.47 (SEM=26.1), for “Same” and “Different” in the backward trials, and an ANOVA on these data confirmed no differences, F < 1. Responding in the pre CS period appeared larger than those in Experiment 1, which was likely due to the fact that responses were trained using a concurrent schedule in Experiment 1 but with independently trained simple schedules in Experiment 2.

Figure 4.

Mean rates of “same” and “different” responses for forward and backward CSs averaged over both blocks of the PIT Test of Experiment 2. The population means implied by our statistical analysis is also reported in σ units.

No differences were found in the data from “Questionnaire 1.” The scores were 29.1, 11.7, and 37.7, for the questions about the forward, backward, and instrumental contingencies, respectively (100 = correct outcome, −100 = incorrect outcome). The data from “Questionnaire 2” are presented in Figure 5. The figure shows that participants more often identified the correct than incorrect outcomes for both the Pavlovian S-O and instrumental R-O contingencies, although to different degrees. An ANOVA performed on the data revealed significant differences across the conditions, F(5, 75) = 3.87, MSE = 1219.71, p < 0.05, Δ = 13.85. Post-hoc comparisons showed a significant difference between “correct” and “incorrect” outcome estimates in the three types of questions, but this difference was somewhat smaller with the backward stimuli (see implied population means in the figure). These data suggest that subjects acquired an explicit understanding of the forward, backward, and instrumental contingencies, though the absolute magnitude of this explicit understanding was not overwhelming.

Figure 5.

Mean percentages of “correct” and “incorrect” outcome responses in the questionnaire of Experiment 2. The population means implied by our statistical analysis is also reported in σ units.

Discussion

As in Experiment 1, only the forward CSs selectively elevated instrumental responding at test, which is consistent with the predictions of the S-O-R account of the specific PIT effect. The most important modification made in this experiment was that during instrumental training the responses were trained with independent RR schedules rather than concurrently. This may be expected to have especially strengthened the formation of specific O-R associations during instrumental training and, consequently, should have facilitated selective PIT according to both S-O-R and S-R accounts. The absolute size of selective PIT with the forward cues was numerically slightly larger than was observed in Experiment 1 (a roughly 0.6 σ vs 0.8 σ effect), but it is unclear whether this factor was responsible for that seemingly small difference.

One potential concern with both Experiments 1 and 2 is that the instructions of the Pavlovian phase might have especially encouraged learning of the forward associations. Participants were informed that the neutral cues could be useful to “predict” the outcomes, and this might have biased their attention and learning of the forward relations at the cost of the backward ones. The questionnaire data did reveal a slightly weaker explicit knowledge of the backward relative to the forward relations. Thus, it is possible that these instructions may have fostered greater specific PIT with forward than backward stimuli. This issue was addressed in the next two experiments.

Experiment 3

The goal of Experiment 3 was to assess the relative contribution of backward and forward associations to the specific PIT effect by contrasting these within individual training and test trials. In the Pavlovian phase of Experiment 3, each CS presentation was preceded and followed by an outcome delivery (see Table 2). For one group of participants (Group Consistent), the outcomes that preceded and followed the CSs were the same, i.e., O₁>A->O₁; O₂->B->O₂, but for another group (Group Inconsistent), these outcomes were different, i.e., O₁->A->O₂; O₂->B->O₁. The importance of this manipulation is that within individual trials each CS has the opportunity to enter into associations with both the backward- and forward-trained outcomes. This should be an especially sensitive way of detecting the relative strengths of learning processes based on backward versus forward relations.

Table 2.

Design of Experiment 3.

	Instrumental		Pavlovian	PIT test	Questionnaire
Inconsistent	R1->O1	R2->02	O1->A->O2	A	A, B: O1, O2?
			O2->B->O1	B	C, D: O1, O2?
			O1->C->O2	C	R1, R2: O1, O2?
			O2->D->O1	D
Consistent	R1->O1	R2->02	O1->A->O1	A	A, B: O1, O2?
			O2->B->O2	B	C, D: O1, O2?
			O1->C->O1	C	R1, R2: O1, O2?
			O2->D->O2	D

A, B, C and D: neutral fractal images; R₁ and R₂ : keyboard responses; O₁ and O₂ : food and drink images.

Both S-O-R and S-R accounts make the same prediction for the consistent group (because the preceding and following outcomes were identical), but the critical question referred to the specific PIT effect in the inconsistent group (see also Cohen-Hatton et al., 2013). According to the S-O-R account, the CSs should selectively elevate performance based on the outcome that followed the CSs during training (because forward pairings should result in stronger associations than backward pairings). In contrast, the S-R account predicts the specific PIT effect to be based on the outcome that preceded the CSs in training. This is because, according to the S-R mechanism, the outcomes presented before and after the CS activate a response representation, but the representation activated by the preceding outcome should be more temporally contiguous with the CS, resulting in a stronger S-R association. For instance, in each of the O₁->A->O₂ trials, O₁ activates a representation of R₁ close in time to the presentation of A, forming a strong A-R₁ association. However, O₂ is presented when A is no longer present, thus the activation of R₂ is relatively distant to A, resulting in a relatively weak A-R₂ association.

A second important difference in our procedures from the previous experiments was that in the present study participants were not given any instructional manipulation that might bias them to process the forward relations more effectively than the backward relations. In particular, participants were not instructed to expect that the stimuli could be used to predict the outcomes on any given trial. Instead, they were merely informed that each trial would consist of an outcome followed by a fractal image followed by an outcome, and they were encouraged to learn about the relations involving these events because they would be questioned about them later.

In order to raise the difficulty of the task and to maintain participants’ engagement with it, the number of CSs used was increased to four (see Table 2). In Group Consistent 2, CSs were preceded and followed by O₁, i.e., O₁->A->O₁; O₁->C->O₁, and two other CSs by O2, i.e., O₂->B->O₂; O₂->D->O₂. In Group Inconsistent 2, CSs were preceded by O₁ and followed by O₂ and two others were preceded by O₂ and followed by O₁. Due to the added complexity of the task, the number of Pavlovian conditioning trials was also increased in this study. Finally, it was considered that “Questionnaire 2” was more informative about the degree to which participants learned the Pavlovian and instrumental associations, thus “Questionnaire 1” was removed.

Method

Participants

Thirty-two students from the Department of Psychology of Brooklyn College participated for course credit in this experiment (8 males and 24 females), aged between 19 and 58 years old.

Procedure

The same general procedures were used as in Experiment 2, except where noted below.

Instrumental phase.

The same procedures were used as in Experiment 2.

Pavlovian phase.

On each conditioning trial, outcomes were presented both before and after the CS. Participants were instructed to press the keys “t” and “g” when they saw the drink and food images, respectively. The phase was divided into three blocks comprising 6 trials each of O₁->A->O₁, O₁->B->O₁, O₂->C->O₂ and O₂->D->O₂ in Group Consistent, and 6 trials of O₁->A->O₂, O₁->B->O₂, O₂->C->O₁ and O₂->D->O₁ in the Group Inconsistent.

Importantly, the instructions for the Pavlovian phase also differed from those used in Experiment 2. In particular, the phrase that could potentially bias processing of the forward relations on a trial was eliminated. The specific instructions appear below:

Now you will see a black dot centered on the screen. Once again please keep your eyes focused on this.

Neutral fractal images and rewards (DRINK or FOOD) will appear on the screen. When a DRINK appears you have to press “T,” and when FOOD appears you have to press “G.” Your goal is to press as quickly and accurately as you can.

In addition, please pay close attention to the relationship between the neutral images and rewards because later on you will have to answer some questions about these. Please do not press any key during the neutral images.

When you are ready, please press space button to continue.

Data analysis.

Data were analyzed by conducting separate between-group ANOVAs for each response (same, diff) based on a common error mean square (estimate of σ²) as well as an overall repeated measures main effect test across the two responses. This approach (Rodger, personal communication; also, Delamater, Derman, & Harris, 2017) accounts for all of the sum of squares found in the ordinary split plot form of analysis, but with the sum of squares repartitioned to conform to a different underlying linear model. Significant differences were then assessed using post-hoc contrasts following Rodger (1974, 1975).

PIT scores were coded with reference to the forward CS-Outcome relations during training. For instance, in the inconsistent group A was preceded by O₁ but followed by O₂. Thus, R2 responses were grouped as “same” and R₁ as “different.” With this index, there should be more “same” than “different” responses if the forward relations govern specific PIT, but the reverse should be true if the backward relations govern specific PIT.

Results

Participants in both groups completed instrumental training by successfully earning 50 outcomes of each type as in Experiment 2. The mean reaction times in responding to the backward and forward outcomes during the Pavlovian phase were also analyzed, but there were no differences between or within groups except that overall RTs decreased over training (data not shown).

The data of primary interest came from the PIT tests, and these are shown in Figure 6 for both groups averaged over the two blocks of testing. The graph indicates that the CSs produced a comparable specific PIT effect in the consistent and inconsistent groups. Critically, this effect was based on the outcome that followed the CSs during Pavlovian training.

Figure 6.

Mean rates of “same” and “different” responses for forward and backward CSs averaged over both blocks of the PIT Test of Experiment 3. The population means implied by our statistical analysis is also reported in σ units.

An ANOVA on the data confirmed that the groups did not differ in the number of “same” responses, F < 1, nor “different” responses, F < 1, but it revealed an overall significant difference between “same” and “different” responses, F(1, 30) = 4.17, M SE=4117.82, p < 0.05, Δ =4.74. The mean rates of pre CS responding were 82.5 (SEM = 16.44), and 90.8 (SEM = 20.72) for “same/different” in the consistent, and 113.9 (SEM = 22.81), and 102.8 (SEM = 22.32) for “same/different” in the inconsistent group. An ANOVA on these data showed no significant differences, largest F(1, 30) = 1.15,p > 0.05.

The mean “correct” and “incorrect” outcome responses on the questionnaire are presented in Figure 7. In general, participants more accurately identified the correct than incorrect outcomes for the CSs and instrumental responses, although they were not equally good in all the conditions. An ANOVA on the data showed differences in group consistent, F(5, 30) = 6.71, M SE=6585.94, p < 0.05, and in group inconsistent, F(5, 30)=4.71, MSE=4618.82, p < 0.05, but no overall difference between groups, F < 1. Post-hoc comparisons revealed that Group Consistent was slightly more accurate when judging the instrumental than Pavlovian contingencies, but that Group Inconsistent was slightly more accurate at identifying the backward Pavlovian contingencies than the forward Pavlovian or instrumental contingencies (see implied population means in Figure 7).

Figure 7.

Mean percentages of “correct” and “incorrect” outcome responses in the consistent and inconsistent group in the questionnaire of Experiment 3. The population means implied by our statistical analysis is also reported in σ units.

Discussion

The CSs in Group Inconsistent were preceded and followed by different outcomes, but at test these CSs elevated responding based on the outcome that followed them during training. Indeed, it did so to the same extent as in Group Consistent. These results, once again, are at odds with predictions derived from an S-R mechanism.

It may have been anticipated, according to either the S-R or the S-O-R accounts, that the CSs during Pavlovian training could have formed two different associations in Group Inconsistent—one based on the backward O-S relation and one based on the forward S-O relation. In either case, such learning would have diluted the specific PIT effect, at least to some degree, in this group. We have no evidence, however, to suggest that participants learned in this way. In particular, the magnitude of specific PIT in Group Inconsistent was no different from that seen in Group Consistent. Had the former group acquired multiple associations, with either different Os or Rs, a smaller specific PIT effect should have been found in Group Inconsistent. Apparently, learning was based solely on the forward S-O relations.

A noteworthy feature of the present study was that our instructional manipulation did not include any statements that might bias participants toward processing forward over backward relations on any given Pavlovian training trial. In spite of this, we found similar results to those seen in Experiments 1 and 2. Thus, we think it is unlikely that our results can be attributed to this aspect of our procedures.

One potential problem with the present study is that during Pavlovian training participants could have used the first outcome to predict the second outcome, and this may have somehow interacted with the nature of learning involving the stimuli. For instance, Group Consistent participants could have rapidly learned that an O₁ delivery signaled a second O₁ delivery on that trial, regardless of which CS occurred in between these presentations, e.g., O₁->A->O₁; O₁->C->O₁. Similarly, Group Inconsistent participants could have learned that O₁ signaled O₂ and vice versa. Both our specific PIT and questionnaire data show that participants did, indeed, process and learn about both the forward and backward S-O relations, at least to some degree. However, this aspect of the experimental design is of some concern, and the issue was addressed in the next experiment.

Experiment 4

This study aimed to replicate the results of Experiment 3 by using a within-subjects design. During the Pavlovian training phase, all participants received presentations of four different CSs, and each of these was preceded and followed by an outcome. On consistent trials, these outcomes were the same, i.e., O₁->A->O₁; O₂->B->O₂, and on inconsistent trials they were different, i.e., O₁->C->O₂; O₂->D->O₁. With this arrangement, participants cannot use the first outcome of a trial to anticipate the second outcome. In this way, we eliminated the potential problem that could arise in the between group version of this study employed in Experiment 3. Once again, however, we anticipate that specific PIT will be based on the forward S-O relations, as suggested by the S-O-R view that forward S-O associations are stronger than backward ones.

Method

Participants

Seventeen students from Brooklyn College participated in this experiment (3 males and 14 females), aged between 18 and 50 years old. One participant was removed because he performed only one type of response in the PIT test.

Procedure

Everything was the same as in Experiment 3, unless otherwise stated.

Pavlovian phase.

The phase was divided in three blocks, each of them comprising 6 consistent trials of O₁->A->O₁ and O₂->B->O₂, and 6 inconsistent trials of O₁->C->O₂ and O₂->D->O₁.

Data analysis.

The data were analyzed as in Experiments 1 and 2. The calculation of the PIT scores was exactly as in Experiment 3.

Results

Instrumental training proceeded uneventfully as in our earlier studies. The mean RTs to the outcomes during Pavlovian training showed a similar improvement over training with no differences other than those seen in Experiment 3; however, an ANOVA performed on these data (not shown) failed to reveal any significant differences, F < 1.

The PIT data are plotted in Figure 8. The graph shows that both consistent and inconsistent CSs produced the specific PIT effect, and it was based on the outcome that followed the CSs during training. Although this effect seems to be slightly lower in the presence of the consistent CSs than the inconsistent CSs, this difference was not statistically significant. An ANOVA performed on the data revealed a significant main effect across stimuli and responses, F(3, 45)=2.61, M SE=4179.77, p < 0.05, Δ =4.488. Post-hoc comparisons showed that both types of CS produced more “same” than “different” responses based on the forward S-O relations, F(3, 45) = 2.43, p < 0.05, but the number of “same” or “different” responses did not differ across consistent and inconsistent trials. The mean rates of pre CS responding were 63.4 (SEM = 15.72) and 77.8 (SEM = 15.78) for “same” and “different” in the consistent CS, and 86.6 (SEM = 19.43) and 86.9 (SEM=21.67) for “same” and “different” in the inconsistent CS. An ANOVA on these data showed no significant differences, F < 1.

Figure 8.

Mean rates of “same” and “different” responses for forward and backward CSs averaged over both blocks of the PIT Test of Experiment 4. The population means implied by our statistical analysis is also reported in σ units.

The questionnaire data showed that participants tended to identify the correct more than the incorrect outcome for the various Pavlovian and instrumental contingencies. However, an ANOVA performed on these data revealed that these apparent differences were not statistically reliable, F < 1 (data not shown).

Discussion

In this experiment, we once again observed that the CSs produced a specific PIT effect based only on the outcome that followed these cues in training. Furthermore, this effect was not reduced on inconsistent compared to consistent trials (if anything, the reverse occurred), suggesting that the learning was entirely based on forward S-O relations. This pattern of results was very similar to that seen in Experiment 3, and this suggests that the ability to use the first outcome to predict the second on a given trial within the Pavlovian training phase did not likely contribute to our results. These data, once again, more strongly support the S-O-R mechanism and are inconsistent with the predictions of the S-R account. We may note that Cohen-Hatton et al. (Experiment 4, 2013) performed a conceptually similar study to ours with rats and found stronger learning on the basis of backward than forward relations. We discuss this discrepancy in more detail in the general discussion section, but it seems possible that their training procedure resulted in stronger CS-US associations in their backward than their forward training trials.

Unlike Experiments 2 and 3, the results of the questionnaire did not confirm that participants were able to distinguish the Pavlovian or instrumental associations, although the results were in the expected direction. Nevertheless, if participants did not learn these associations, then no specific PIT effect should have been found. It is possible that this questionnaire was not sensitive enough to adequately assess participants’ explicit knowledge of the contingencies because of the added complexity of the task and the manner in which we assessed contingency knowledge. Nonetheless, the PIT data show that these contingencies were, in fact, learned.

General Discussion

The present studies aimed to identify the psychological mechanisms underlying specific PIT effects in a human learning task. We contrasted predictions based on different hypotheses developed to account for conflicting evidence obtained in rodent studies, namely, that specific PIT is produced by an S-O-R (e.g., Delamater et al., 2003; Laurent et al., 2015) or an S-R associative mechanism (Cohen-Hatton et al., 2013). In Experiments 1 and 2, the incentive motivational effect of CSs that were trained in a forward (A->O₁; B->O₂) or a backward (O₁>C; O₂->D) relation with distinct outcomes was assessed on the performance of two responses, each of them previously trained with one of these outcomes (R₁>O₁; R₂->O₂). In both of these cases, we observed stronger selective PIT with forward- than backward-trained CSs. Moreover, in Experiments 3 and 4, we contrasted Pavlovian learning in situations where CS presentations were preceded and followed by the same or different outcomes on each conditioning trial. In the PIT test of both experiments, the CSs only produced a detectable specific PIT effect based on the outcome that followed them during Pavlovian training. In addition, this effect was found regardless of whether the CSs were preceded and followed by the same or different outcomes. These results indicate that with our procedures forward S-O relationships more strongly contribute to specific PIT than do backward O-S relationships.

The results obtained in our studies support the S-O-R mechanism of PIT, which states that at the time of PIT testing, the CS activates a representation of its associated outcome that, in turn, triggers the performance of the response linked with the activated outcome, via an O-R (e.g., Kruse et al., 1983; Trapold & Overmier, 1972) and/or an R-O association acting in the backward direction (e.g., Rescorla, 1992). Because there is good reason to expect forward conditioning to result in stronger excitatory associative learning than backward conditioning (which may more likely engender inhibitory learning), S-O-R mechanisms successfully anticipate stronger selective PIT with forward than backward-trained cues. In contrast, as noted in the general introduction, these results cannot easily be explained by the S-R account of PIT postulated by Cohen-Hatton et al. (2013). According to this view, there should be greater contiguity between the response representation evoked by the outcome and the CS when the outcome is presented just before rather than just after the CS. This should lead to stronger PIT based on the backward O-S than the forward S-O relation.

Another interesting aspect of our findings is that specific PIT was found whether the instrumental responses were trained concurrently (Experiment 1) or separately (Experiments 2–4). If the instrumental association that supports PIT is an O-R rather than an R-O link (e.g.,Trapold & Overmier, 1972), then the specific PIT effect should be more difficult to find when both instrumental responses are trained concurrently. Because the subject can switch from one response to the other at any time in a concurrent training procedure, each outcome has the opportunity to associate with each of the two subsequent responses. This would tend to dilute the specificity of the PIT effect that operates through an O-R link. Although the specific PIT effect observed in Experiment 1 (that used a concurrent training procedure) appeared to be smaller (same—different = 0.58σ) than that found in Experiment 2 (same—different = 0.78σ), further studies would need to directly compare these effect sizes to reach a definitive conclusion.

Given our general conclusion that PIT is due to an S-OR, rather than an S-R, mechanism, what are we to make of the discrepancies found in the rodent literature (Cohen-Hatton et al., 2013; Delamater et al., 2003; Laurent et al., 2015) and alsofound between the Cohen-Hatten et al. (2013) study and the results we present here? One obvious consideration is that our human learning PIT task may simply not be relevant to rodent studies because it differs in many ways. Other research will be required to determine the generality of our findings to other species. We currently have preliminary data with rats, however, showing a similar forward learning bias to those reported in Experiments 3 and 4 here (Gilroy, Schneider, & Delamater, 2015).

Another difference between our tasks and animal studies is that our procedure may be described as one that uses conditioned, rather than primary reinforcers. Other researchers have studied PIT with humans using primary rewards (e.g., Bezzina, Lee, Lovibond, & Colagiuri, 2016; Colagiuri & Lovibond, 2015; Lovibond & Colagiuri, 2013), but it remains to be seen whether the forward learning bias (relative to backward learning) that we observed here would also be observed when using primary rewards. Another important consideration is that in our studies we had direct control over the specific timing relations between stimulus and outcome presentation, and this may be critical. In the rodent studies, this timing is less certain because rodents, and not the experimenters, have control over when they approach and how long it takes them to consume the presented outcomes following their delivery. It seems quite likely that the discrepant findings between Cohen-Hatton et al. (2013), on the one hand, and Delamater et al. (2003) and Laurent et al. (2015), on the other, arose from different temporal parameters used in these studies. For instance, Cohen-Hatton et al. (2013) might have inadvertently used a simultaneous rather than backward conditioning procedure since in their “backward” task the CS was presented only 1 s after subjects had access to the US, leaving open the possibility that subjects might have been consuming the US through much of the CS presentation. In contrast, Delamater et al. (2003) and Laurent et al. (2015) presented the CS 10 s after the US delivery, which makes it less likely that animals were still consuming the outcome simultaneously with the CS presentation. Indeed, Cohen-Hatton et al. (2013) recognized this difference and suggested that the 10-s US-CS interval used in the Delamater et al. (2003) and Laurent et al. (2015) procedures would have sufficiently disrupted the temporal contiguity between S and R so as to disrupt PIT based on that relation.

We think that the S-R explanation of PIT would not easily apply to the results we present here, however, because, like Cohen-Hatton et al. (2013), we used a very short US-CS interval during Pavlovian training. Had the sort of mechanisms hypothesized by Cohen-Hatton et al. (2013) applied to our procedure, the backward relations would have resulted in stronger S-R learning than the forward relations we employed due to a more favorable temporal contiguity between the R representation evoked by the backward outcome and the stimulus. From this perspective, though, one additional consideration may be relevant to our demonstration of a forward bias in learning. In the Pavlovian phase, subjects were instructed to press the appropriate response button whenever the outcome was presented. This overt “Pavlovian unconditioned response” may have disrupted processing of the representation of the instrumental response presumed to be associatively activated by the outcome. This would make it difficult for the CS to associate with the representation of the instrumental response. However, for this account to apply to our results, greater interference would have to occur to the response memories evoked by the backward than the forward outcome. In the absence of a set of very specific ad hoc processing assumptions, it is not obvious that this should be true. Furthermore, in our task, the outcome remained on the screen for an additional 1 s after the Pavlovian response was made. During this time, the outcome had an additional opportunity to evoke its associated instrumental response memory. For both of these reasons, we find this interference account an unlikely explanation of our observed forward bias.

Although, so far, we have only discussed the predictions of the S-O-R and the S-R models of PIT, a propositional version of the hierarchical (S:R-O) model of instrumental conditioning (cf., Rescorla, 1991, 1994); has recently been used to explain the specific PIT effect in humans (e.g., Hardy, Mitchell, Seabrooke, & Hogarth, 2017; Hogarth et al., 2014; Seabrooke, Hogarth, & Mitchell, 2016). According to this approach, participants learn that in the presence of each CS a common tacit response is paired with a particular outcome (e.g., CS₁: CR->O₁; CS₂: CR->O₂). Also, each instrumental response is reinforced with a particular outcome, but both of them in the presence of common contextual cues (e.g., Ctx: R₁ >O₁; Ctx: R₂->O₂). At test, these contextual cues are assumed to retrieve both instrumental relations equally. However, the CS acts by facilitating the retrieval of one of these instrumental relationships, rather than simply activating an outcome representation (i.e., CS₁: R₁->O₁; CS₂: R₂->O₂). In other words, according to this view, CS presentations allow participants to infer which instrumental relation is more likely to be in force.

While this hierarchical view may seem plausible, we are not sure how it applies to our studies. In our task, subjects were explicitly instructed to respond to the outcomes, and not to the CSs. Indeed, during the Pavlovian phase of Experiment 4 participants performed one response or the other (“t” or “g”) during the actual presentation of the CSs on only 4% of the trials. This demonstrates that the CSs provoked a tacit “withholding response” that suppressed (not elicited) these instructed Pavlovian responses. Thus, it is not so clear to us that subjects should have inferred to engage in instrumental responses during the CSs in the PIT test. Of course, other inferential reasoning accounts can be constructed, but the basic problem these accounts face is explaining why subjects’ choice of one response is more “reasonable” than their choice of the other. For instance, would it be equally reasonable to suppose that if in the presence of a CS one outcome were to be freely delivered, the subject should choose the action that leads to the alternative outcome in order to maximize the total amount of rewards gained? An answer to this question may require a deeper understanding of how various biases affect human decision making.

One final point is worth some consideration. Although the present studies were not focused on evaluating the role of awareness in PIT, we did collect questionnaire data probing subjects’ explicit knowledge of the various Pavlovian and instrumental contingencies. While we observed specific PIT in Experiments 1–4, subjects either could not report the Pavlovian S-O contingencies or they could do so only relatively weakly. One possible conclusion for this pattern of results is that participants’ awareness of the S-O relations is not required for specific PIT to be observed. However, recent human studies suggest the opposite conclusion (e.g., Bezzina et al., 2016; Hogarth, Dickinson, Wright, Kouvaraki, & Duka, 2007; Lovibond et al., 2015; Seabrooke et al., 2016; Trick et al., 2011). In order to reconcile these differences, we note that there was considerable individual variation in subjects’ contingency and PIT scores, and, because of this, we explored the relation between Pavlovian contingency knowledge and the size of the PIT effect in our studies.¹ Across the four experiments (n = 80), a significant positive correlation (Pearson’s r) was obtained between Pavlovian forward S-O contingency knowledge and the magnitude of forward PIT, r(78) = 0.38, p < 0.001. In addition, we observed a nonsignificant correlation between Pavlovian backward O-S contingency knowledge and the magnitude of backward PIT, r(78)=−0.02. These results are consistent with the evidence mentioned above that explicit Pavlovian knowledge contributes to the expression of PIT. However, our study was not designed to directly assess the role of contingency knowledge in PIT. Doing so would require that we manipulate contingency awareness and use more sensitive contingency assessment procedures than were used in the present studies.

In summary, our findings contribute to our understanding of the mechanisms involved in PIT. We consistently found that forward S-O relations learned during Pavlovian conditioning played a more substantial role than backward O-S relations at supporting PIT. These findings are more naturally understood from the perspective that what PIT measures is the strength of the Pavlovian S-O association at the time of test, rather than a mediated S-R association learned during Pavlovian training. Furthermore, we believe that the task used in these experiments provides us with a useful tool, more generally, to address additional questions regarding the nature of the PIT phenomenon in humans.