Pavlovian to Instrumental Transfer of Control in a Human Learning Task

Abstract

Pavlovian learning tasks have been widely used as tools to understand basic cognitive and emotional processes in humans. The present studies investigated one particular task, Pavlovian-to-instrumental transfer (PIT), with human participants in an effort to examine potential cognitive and emotional effects of Pavlovian cues upon instrumentally-trained performance. In two experiments subjects first learned two separate instrumental response-outcome relationships (R1-O1, R2-O2) and then were exposed to various stimulus-outcome relationships (S1-O1, S2-O2, S3-O3, S4-) before the effects of the Pavlovian stimuli on instrumental responding were assessed during a nonreinforced test. In Experiment 1 instrumental responding was established using a positive reinforcement procedure whereas in Experiment 2 a quasi-avoidance learning task was used. In both cases the Pavlovian stimuli exerted selective control over instrumental responding, whereby S1 & S2 selectively elevated the instrumental response with which it shared an outcome. In addition, in Experiment 2, S3 exerted a nonselective transfer of control effect, whereby both responses were elevated over baseline levels. These data identify two ways, one specific and one general, in which Pavlovian processes can exert control over instrumental responding in human learning paradigms, and suggest that this method may serve as a useful tool in the study of basic cognitive and emotional processes in human learning.

It has increasingly been recognized that Pavlovian conditioning paradigms offer useful tools to study basic cognitive and emotional processes in humans (e.g., Bechara, et al., 1995; Delgado, et al, 2006; Knight, et al., 2005; Phelps & LeDoux, 2005). In applying such tools, however, it becomes important to understand how these may effectively reveal the contribution to performance of different cognitive and emotional processes. To illustrate the problem concretely, suppose, for instance, an investigator was interested in studying fear conditioning in humans by pairing a red colored square with cutaneous shock. After relatively few pairings, the red square may come to alter skin conductance more than would a control stimulus (a green square) that was not paired with shock (e.g., LaBar et al.,1995, LaBar et al., 1998; Delgado et al., 2006). One important issue in interpreting these results is determining whether the red square elicits a skin conductance change because, on the one hand, it signals that a painful shock is about to occur, or, on the other hand, because it creates an unpleasant emotional state independent of any specific expectation of shock. In some sense, this is akin to the distinction between a specific fear, for which there is an identifiable referent, and a free-floating anxiety, for which there is no obvious referent (e.g., Davis, et al., 2010; Waddell, Morris, & Bouton, 2006). In both cases, a change in performance can occur but for different reasons.

This issue has been of interest to learning theorists. In the study of Pavlovian learning, for instance, it has long been recognized that both appetitive and aversive unconditioned stimuli, US, are complex events consisting of many different attributes, any number of which may enter into an association with the predictive conditioned stimulus, CS, (e.g., Delamater & Oakeshott, 2007). Some theories of conditioning have emphasized the distinction between learning about highly specific sensory and more general emotional (or motivational) attributes of reward (e.g., Konorski, 1967, pps 270–280; Wagner & Brandon, 1989), while other theories have emphasized the importance of temporal aspects of reward (Arcediano & Miller, 2002; Gallistel & Gibbon, 2000; Gibbon & Balsam, 1981) or specific response processes (Donahoe & Vegas, 2004). A number of different tasks have been developed in non-human animal studies to demonstrate that associations are sometimes formed between the CS and the specific sensory properties of the unconditioned stimulus (e.g., Betts, Brandon, & Wagner 1996; Colwill & Motzkin, 1994; Delamater, 1995; 1997; Galarce, Crombag, & Holland, 2007; Ostlund & Balleine, 2007; 2008; Rescorla, 1999). In addition, studies involving Pavlovian to instrumental transfer of control (PIT), have provided evidence for the claim that the CS can enter into associations not only with the specific sensory features of reward (e.g. Delamater & Holland, 2008; Galarce, et al. 2007; Kruse, Overmier, Konz, & Rokke, 1983) but with their more general emotional properties as well (e.g. Dickinson & Dawson, 1987). To the extent that some arbitrary stimulus enters into associations with both specific sensory and more general emotional properties of a biologically important event, then that stimulus is said to have acquired both “cognitive” and “emotional” significance.

Recent work with rat subjects nicely illustrates this distinction in the PIT task. Corbit and Balleine (2005) gave rats Pavlovian training in which three stimuli (tone, white noise, and clicker) were each paired with distinct outcomes (pellet, sucrose, or Polycose). In an instrumental training phase, two different responses (left or right lever press) were reinforced differentially with two of the three outcomes (e.g. pellet & sucrose) also used during Pavlovian training. During the test, each of the three CSs was presented while the animals engaged in the different instrumental responses (in different sessions under extinction conditions). Outcome-specific PIT was observed when two of the CSs selectively increased responding on the lever with which they shared an outcome (e.g., if the CS signaled pellets, then the lever press response previously rewarded with pellets, but not sucrose, was increased over baseline levels). This result indicates that the CS had evoked a specific neural representation of the reward with which it was paired, and that this neural representation, in turn, influenced the instrumental response similarly associated with that reward (see Balleine and Ostlund, 2007). Furthermore, outcome-general transfer was observed when the third stimulus (associated with an outcome other than those used during instrumental training) non-selectively increased both instrumental responses over baseline levels. This result implies that the CS had evoked an arousing emotional state (e.g., excitement) that non-specifically increased responding (e.g., see Rescorla and Solomon, 1967). Additional support for these claims was provided by Corbit and Balleine's (2005) finding of a double dissociation between the effects of different pretraining brain lesions on each form of transfer. Specifically, basolateral amygdala lesions eliminated the outcome-specific PIT effect (without affecting general PIT), while central nucleus of the amygdala lesions eliminated the outcome-general PIT effect (without affecting specific PIT). Thus, using the same task one can effectively distinguish learning that is based on associations with more cognitive or emotional elements of reward.

Recently, there has been interest in using PIT tasks with humans. This endeavor is potentially of great significance because with neuroimaging techniques it should be possible to distinguish between different cognitive and emotion circuits engaged by the task. To date, several studies with humans have successfully provided evidence for specific PIT (Bray, Rangel, Shimojo, Balleine, & O'Doherty, 2008; Paredes-Olay, Abad, & Gamez, 2002; see also Gamez & Rosas, 2007; Hogarth, Dickinson, Wright, Kouvaraki, & Duka, 2007), and another study used a procedure in which the specific and general forms of PIT could not be differentiated (Talmi, Seymour, Gayan, & Dolan, 2008). There has not yet been a human PIT study that has successfully identified both the specific (cognitive) and general (emotional) forms of PIT in the same experiment. Developing such a task could be especially helpful in further human neuroimaging studies directed at separating these different types of neural circuits.

In the present study, we adapted the experimental design introduced by Corbit and Balleine (2005) to human learning tasks in an attempt to identify both specific and general PIT effects in a single experiment. Since the human learning studies cited above employed both positive reinforcement and avoidance learning paradigms, we investigated these effects using a positive reinforcement task in Experiment 1 and an avoidance-related task in Experiment 2 similar to that devised by Paredes-Olay, et al. (2002).

EXPERIMENT 1a

The first experiment was intended to determine if the PIT design used by Corbit & Balleine (2005) could be used with humans to identify both specific and general PIT. The design of this task is outlined in Table 1. A computer “game” was developed consisting of three phases. Initially, participants were asked to learn about the relationships between different button press responses and different outcomes (USs) that may have appeared on the screen as a consequence of responding (R1-O1, R2-O2). Although these outcomes were pictures presented on the computer screen (and not biologically significant), we regarded them as “positively reinforcing” because of task instructions. Subsequently, the participants were asked to learn the relationships between different stimuli and the outcomes that may have followed these stimuli. Two of these stimuli were paired with the instrumental outcomes (S1-O1, S2-O2), a third stimulus was paired with a third outcome (S3-O3), while a fourth stimulus was presented alone (S4-). In a test phase, participants were encouraged to engage in R1 and R2 responding, both in the presence and absence of each of the 4 stimuli. Specific PIT would be observed in this test by a selective increase in the response that shares an instrumental outcome with the Pavlovian stimulus (i.e., more R1 than R2 responding in the presence of S1, but the opposite in the presence of S2). A general PIT effect would be revealed by a non-selective increase in R1 and R2 responding in the presence of S3. Tests involving S4 were included to assess the effects of a nonreinforced stimulus upon instrumental responding.

Table 1.

Experimental Designs used in Experiments 1 & 2. R1 and R2 refer to different instrumental responses, O1, O2, and O3 refer to different reinforcing outcomes used during instrumental and Pavlovian training phases, and CS1, CS2, CS3, CS- refer to different conditioned stimuli used during Pavlovian training.

Instrumental Training	Pavlovian Training	Transfer Test
R1–O1	CS1 – O1	CS1: R1 vs. R2
R2–O2	CS2 – O2	CS2: R1 vs. R2
	CS3 – O3	CS3: R1 vs. R2
	CS–	CS- : R1 vs. R2

Methods

Participants

Thirty-seven Brooklyn College students (22 female; 15 male) were recruited from introductory psychology and advanced psychology classes. The students' ages ranged from approximately 18 – 25 yrs old. All students received course credit for their participation, and had normal or corrected vision.

Stimuli and Materials

The PIT task was a computer based task designed using Superlab software (Cedris Corporation) and was conducted on a Macintosh computer (PowerMac G4). Responses were made by pressing two buttons on a Cedrus response pad (model RB-730) using the index and middle finger of the dominant hand. The stimuli used were 4 rectangular images of different colors (red, blue, purple, and black). All were created in Microsoft Paint and were 38mm × 25mm in size, and each was presented on the center of the screen. The “reinforcing” outcomes used were images of a coin (60mm diameter), star (80mm × 60mm), and key (45mm × 50 mm), and each was presented on screen for 0.8 s in both the instrumental and Pavlovian conditioning phases. During the instrumental conditioning phase, a fixation cross (10mm × 10mm) was presented on screen until an outcome appeared based on the participant's response. Two types of intertrial stimuli were used. Both were black and were presented in the center of the white screen. One, a fixation dot (3mm × 3mm), was on screen for 2 s in the Pavlovian conditioning phase, and the other a fixation cross, was presented on screen for 2 s in the test phase. All stimuli, objects, and fixation cues were singly presented in the center of the screen (Figure 1). All participants performed the experiment individually in a dark room seated at a viewing distance of approximately 18 inches from the screen.

Figure 1.

This figure depicts the two fixation cues, stimuli, and outcomes used (images not to scale) for each experiment. The stimuli used were 4 rectangular images of different colors (an orange colored image replaced the blue rectangular image in Experiment 2). The three outcomes used in Experiment 1 were images of a coin, star, and key (1a) and goblet, star, and moneybag (1b). The outcomes used in Experiment 2 were images of a warplane, warship, and tank.

Procedure

Upon entering the lab, participants were instructed that they will be playing a computer game which has three parts, and that they would be asked to learn about the relationships between different button press responses and different objects that may appear on the screen as well as between different colored boxes and objects that may appear on the screen. As an incentive, participants were informed that they could win a $25 gift certificate to either the campus bookstore or Starbucks coffee provided that they 1) correctly learned the response-outcome (R-O) and the stimulus-outcome (S-O) relationships, and 2) earned the most outcomes.

Instrumental phase

In this phase, participants were asked to focus on a black fixation cross and press either of 2 yellow buttons in order to produce the outcomes. Participants' responses were reinforced according to a concurrent variable ratio 5 variable ratio 5 schedule of reinforcement (conc VR 5 VR 5). Each button press was reinforced with a probability of 0.2. One response was reinforced with the image of a coin outcome and the other response with the image of a star outcome. When the reinforcer was presented, it occurred immediately after the button press and was removed after 0.8s or when the next button press occurred, whichever came first. This phase lasted 5 minutes and began with the following instructions:

In the first part of this experiment, you are asked to learn the relationship between button presses and the appearance of objects on the screen. A “+” sign will appear in the middle of the screen; when you see the + sign, you may press either of the two yellow buttons on the response pad as often as you would like. Please pay attention to the relationships between the left and the right button presses and the objects on the screen that follow them. Try to earn as many objects as possible.

Pavlovian phase

In this phase, participants were asked not to press buttons, but to simply observe the relationships between different colored boxes (conditioned stimuli, CS) and the different outcomes that followed. Two of the CSs preceded the outcomes used previously to reinforce the responses in the instrumental phase (images of a coin or star), another CS was paired with a 3rd outcome (image of a key), and the final stimulus was presented alone (CS-). The particular S-O assignments were counterbalanced using a Latin Square procedure. These four trial types were presented in random order, each occurring 10 times for a total of 40 trials. On each trial, the stimulus was presented for 3s and followed by the outcome or in the case of the CS-, the intertrial interval. A fixation point (presented for 2s) was used instead of a fixation cross during the intertrial interval to help remind subjects that they were to simply observe the relationships presented on screen and not to press buttons. This phase began with the following instructions:

At this point of the experiment you will see a black dot in the middle of the screen. Please focus on the black dot. You will then be presented with one of four (4) different colored boxes - black, blue, purple, or red. These boxes may or may not lead to a particular object (coin, key, star). Please pay close attention to the relationships between the colored boxes and the objects, but do not press any buttons during this part of the experiment (except to begin this phase). You can press the spacebar when you are ready to resume the experiment.

Test phase

In the test phase, the participants were once again encouraged to press the response buttons in an effort to earn as many outcomes as possible. The Pavlovian stimuli (colored boxes) were randomly presented in six blocks of trials. Each block contained a single trial of each of the four boxes. Thus, across blocks each stimulus was tested 6 times. The stimuli were each presented for 2s and the inter-trial interval was 2s long. No outcomes were presented in this phase. Participants' responses (button presses) were recorded both during the presentation of the stimuli as well as during the pre-stimulus intervals. This phase began with the following instructions:

At this point of the experiment, either a “+” sign or one of the colored boxes you've seen previously will be presented in the middle of the screen. During this part of the experiment, you are encouraged to press the two yellow buttons at any time as you see fit in order to earn as many objects as possible. Please press the spacebar when you are ready begin.

Assessment phase

After completing the test phase, participants were asked to complete a contingency assessment questionnaire to determine whether the participant was knowledgeable about the relationships presented in the experiment. Participants answered six multiple-choice questions about the R-O and S-O relationships presented in the experiment. For example:

When you pressed the LEFT button, which object was obtainead?

• 1)Coin
• 2)Star

When the RED box was presented on the screen, which object followed?

• 1)Coin
• 2)Star
• 3)Key
• 4)Nothing

Only those participants who could report the various S-O and R-O relationships with 100% accuracy were included in the data analysis. Based on these criteria, 11 subjects were dropped from the study leaving 26 qualifying participants. Separate analyses performed on the data from excluded participants revealed similar general trends, but more muted effects.

Statistical analysis

Here and throughout, all statistical analyses were performed using analysis of variance (ANOVA) techniques. Significant interactions were evaluated with simple main effect tests using appropriated pooled error terms and follow-up post hoc tests were conducted using the methods of Rodger (1974). Use of these procedures ensured that our type I error rate was no greater than .05, and that our statistical power did not decline with increasing numerator degrees of freedom making it easier to detect significant interactions.

Results

Instrumental training proceeded uneventfully. The mean total number of Rcoin responses during the 5-min instrumental training period was 243.6 and this response was reinforced an average of 47.4 times, while the mean total number of Rstar responses was 217.4 and this response was reinforced an average of 44.0 times. Thus, the experienced VR schedule for each response was close to the programmed VR 5 schedule (5.1 for Rcoin and 4.9 for Rstar). The difference between the mean total Rcoin and Rstar responses was not significant.

Although not instructed, participants frequently responded during the time when the reinforcing outcome was on the screen. Which of the two buttons they pressed in the presence of each outcome was also recorded. When the coin outcome was on screen, participants made an average of 30.6 button presses on the “same” response, i.e., the response that was associated with the coin outcome, and an average of 5.8 “different” responses, i.e., responses on the button associated with the star outcome. Similarly, when the star outcome was on screen, participants made an average of 26.4 “same” responses, and an average of 6.0 “different” responses. A twoway ANOVA with Response (same, different) and Outcome (coin, star) as separate factors found only a main effect of Response [F(1,25) = 34.75, MSerr = 383.351].

The test phase data for Experiment 1a is presented in figure 2. The figure shows the mean rate of Rcoin and Rstar responses in the pre stimulus period (Pre, Rcoin & Pre, Rstar) and during the stimulus presentation (CS, Rcoin & CS, Rstar) for each conditioned stimulus (CScoin, CSstar, CSkey, and CS-). The data reveals that there was a specific transfer effect as indicated by the selective increase in Rcoin responding during the CScoin stimulus presentation and the selective increase in Rstar responding during the CSstar stimulus presentation. However, CSkey and CS- both failed to affect instrumental responding.

Figure 2.

Mean rate of instrumental responding during the PIT test of Experiment 1a. Data are shown separately in the pre stimulus (Pre Rcoin, Pre Rstar) and stimulus periods (Rcoin & Rstar) for each test conditioned stimulus (CScoin, CSstar, CSkey, and CS-).

A three way ANOVA examining the effects of Interval (Pre CS, CS), Response (Rcoin, Rstar), and Stimulus (CScoin, CSstar, CSkey, CS-) revealed significant main effects of Interval [F(1,25) = 6.38, MSerr = 66.45] and Stimulus [F(1,25) = 7.21, MSerr = 23.16]. In addition, significant Interval × Stimulus [F(3,75) = 10.44, MSerr = 26.17], Response × Stimulus [F(3,75) = 6.01, MSerr = 54.60], and Interval × Response × Stimulus [F(3,75) = 7.65, MSerr = 44.98] interactions were also found. To examine the source of the three-way interaction, one-way ANOVAs were performed across the four levels of responding for each CS using a pooled error term (MSerr = 38.92). Significant main effects were obtained for CScoin [F(3,300) = 15.95] and for CSstar [F(3,300) = 11.52], but not for CSkey or the CS-. Post hoc analyses revealed that CScoin selectively elevated Rcoin responding but did not affect Rstar responding relative to the pre-stimulus period [F(3,300) = 15.81]. The post hoc analysis for the CSstar similarly revealed that this stimulus selectively elevated Rstar responding but did not affect Rcoin responding relative to the pre-stimulus period [F(3,300) = 11.17].

Discussion

The purpose of this experiment was to determine if the Corbit and Balleine design (2005) could be used to identify both forms of PIT in a human learning task. This aim was met with mixed success. In the test phase, we observed specific PIT by stimuli associated with the outcomes used during instrumental training, i.e., those playing the roles of S1 and S2. However, we failed to observe general PIT by the stimulus paired with the outcome that was not used during instrumental training, i.e., by S3. One explanation for this lack of general PIT is that the outcomes used in this study were relatively neutral. In contrast, in the animal literature (i.e. Corbit & Balleine, 2005; Hall, Parkinson, Conner, Dickinson, & Everitt, 2001) the outcomes used were food substances and, as such, were biologically significant in hungry rats. General transfer may depend upon the use of outcomes that are more emotionally significant than was used here. The next experiment attempted to address this issue by increasing the significance of our third outcome.

EXPERIMENT 1b

In Experiment 1b, the same general procedures were followed as in Experiment 1a. However, in an attempt to increase the significance of the third outcome we directly associated it with money in the present experiment. The third outcome used here was an image of a moneybag that the participants were instructed was worth 25¢ each time it occurred. Thus, the stimulus paired with the moneybag outcome, i.e., S3, might acquire more emotional significance due to the associated monetary reward, and, therefore, non-selectively increase both instrumental responses during the test session. Further, we anticipate S1 and S2 to exert specific transfer as seen in Experiment 1a.

Methods

Participants

Twenty Brooklyn College students (11 female; 9 male) were recruited from introductory psychology and advanced psychology classes, and their ages ranged between 18 – 25 years. All students received course credit for their participation. All participants had normal or corrected vision.

Stimuli and Materials

The stimuli and materials were the same as in Experiment 1a except for the following change. The “reinforcing” outcomes used were images of a goblet (50mm × 40mm), star (80mm × 60mm), and moneybag (90mm × 100mm). The CSs, outcomes and fixation cues were singly presented in the center of the screen (Figure 1), and the fixation cues and CSs were presented for 3s in the Pavlovian and test phases.

Procedure

The general procedures used in Experiment 1b were the same as those followed in Experiment 1a. However, in addition to the use of a moneybag image for the third stimulus, an image of a goblet replaced the coin image outcome used in Experiment 1a so as not to interfere with the monetary incentive of the moneybag outcome. Except for changing the outcomes used (to goblet, star, & moneybag images), the instructions, the criteria used to determine the winner of the gift certificate, and the incentive (a $25 gift certificate to either the campus bookstore or Starbucks coffee) were the same.

Instrumental phase

The instructions and procedures for the instrumental phase in Experiment 1b were identical to the instructions and procedures in Experiment 1a with the exception that an image of a goblet was used instead of a coin for one of the reinforcing outcomes (counterbalanced across responses).

Pavlovian phase

The Pavlovian instructions for Experiment 1b were identical to the instructions of Experiment 1a except for the difference in outcomes used. As in Experiment 1a, participants were asked not to press buttons, but to simply observe the relationships between different colored boxes (CSs) and the different outcomes that followed. As before, two of the colored boxes predicted the outcomes used earlier to reinforce the responses in the instrumental phase (images of a goblet or star), one box was paired with a 3rd outcome (image of a moneybag), and the final stimulus was presented alone (CS-). Again we used a Latin Square procedure to counterbalance the specific S-O relationships. The order of events was the same as in Experiment 1a, and the intertrial and CS durations were both 3s.

Test phase

The instructions and procedures for the test phase in Experiment 1b were identical to those used in Experiment 1a, except that the intertrial and CS durations were 3s (instead of 2s in Experiment 1a).

Assessment phase

After completing the test phase, participants were asked to fill out a contingency assessment questionnaire as in Experiment 1a. Again only those participants who could report the various S-O and R-O relationships with 100% accuracy were included in the data analysis. Based on these criteria, 5 subjects were dropped from the analysis leaving 15 qualifying participants.

Results

Instrumental responding emerged as it had in Experiment 1a. The mean total number of Rgoblet responses was 228.4 and this response was reinforced an average of 45.6 times. The mean number of Rstar responses was 191.0 and this response was reinforced an average of 38.1 times. Thus both of these responses were reinforced on VR5 schedules as programmed. The difference between the mean number of Rgoblet and Rstar responses was significant (t(14) = 2.49), but it is not clear why this difference occurred.

As in Experiment 1a, subjects occasionally made button press responses when the outcomes were presented on the screen. When the goblet outcome was on screen, participants made an average of 27.9 Rgoblet responses and an average of 5.7 Rstar responses. Similarly, when the star outcome was on screen, participants made an average of 18.0 Rstar responses, and an average of 4.6 Rgoblet responses. A Response (same, different) × Outcome (goblet, star) ANOVA found significant main effects of Response [F(1,14) = 9.43, MSerr = 502.14], and Outcome [F(1,14) = 8.08, MSerr = 56.14], and also a significant Response × Outcome interaction [F(1,14) = 6.33, MSerr = 45.20]. Follow up one-way ANOVAs using a pooled error term (MSerr = 273.67) found that same responding was greater than different responding in the presence of both the goblet [F(1,28) = 13.43] and the star [F(1,28) = 4.92] outcomes.

The test phase data for Experiment 1b is presented in figure 3. The figure presents the mean rate of Rgoblet and Rstar responding in the Pre CS and CS periods for each stimulus (CSgoblet, CSstar, CSmoneybag, and CS-). The data again reveals that there was a specific transfer effect. This effect is indicated by the selective increase in Rgoblet responses when the CSgoblet stimulus was presented and a selective increase in Rstar responses when the CSstar stimulus was presented. However, again there were no significant differences in responding in the Pre CS and CS presentation periods on CSmoneybag or the CS- trials.

Figure 3.

Mean rate of instrumental responding during the PIT test of Experiment 1b. Data are shown separately in the pre stimulus (Pre Rgoblet, Pre Rstar) and stimulus periods (Rgoblet & Rstar) for each test conditioned stimulus (CSgoblet, CSstar, CSmoney, and CS-).

A three way ANOVA examining the effects of Interval (Pre CS, CS), Response (Rgoblet, Rstar), and Stimulus (CSgoblet, CSstar, CSmoneybag, CS-) revealed a significant main effect of Stimulus [F(3,42) = 2.05, MSerr = 36.65], and significant Stimulus × Interval [F(3,42) = 2.65, MSerr = 34.63] and Stimulus × Interval × Response [F(3,42) = 3.48, MSerr = 113.33] interactions. Separate one way ANOVAs examining the four levels of response to each CS were performed with a pooled error term (MSerr = 103.96) to assess the source of this three way interaction. These analyses revealed significant main effects for the CSgoblet [F(3,168) = 2.76] and for the CSstar [F(3,168) = 2.92]. There were no significant differences found for the CSmoneybag or the CS-. Further post hoc analyses supported the finding that the CSgoblet selectively elevated Rgoblet responding but not Rstar responding relative to the pre-stimulus period [F(3,168) = 2.43]. The post hoc analysis for the CSstar similarly revealed that this stimulus selectively elevated Rstar responding but did not affect Rgoblet responding relative to the pre-stimulus period [F(3,168) = 2.63].

Discussion

In Experiment 1b, changes were made to the third outcome, O3, with the rationale of increasing the emotional significance of the stimulus associated with it, S3, in the hope of generating a general transfer effect. Though the specific PIT effect with S1 and S2 was replicated, we failed to observe general PIT in the presence of S3. Perhaps the monetary incentive used in this study was not successful because the monetary amount (.25¢) was not large enough. Alternatively, perhaps the task, itself, is biased towards observing specific, rather than general, transfer effects because of its somewhat arbitrary nature. Other procedures that are more successful at enhancing the salience of emotional processes in the task may produce a different pattern of results. The next experiment attempted to accomplish this by using a more naturalistic paradigm, i.e., one that is more relevant to potential real-world life experiences.

EXPERIMENT 2

In Experiment 2, we modified the procedure developed by Paredes-Olay et al. (2002) in order to study specific and general transfer with the experimental design used in Experiment 1. Our task can be considered a quasi-avoidance problem, in which participants are asked to defend their fictitious country against enemy attack by firing missiles at potentially attacking vessels. We then asked subjects to observe the relationships between different cues and different types of attack, before finally using these cues to assist them in making decisions about which type of missile to fire in an effort to efficiently defend their country. Because this task is more naturalistic than the one used in Experiment 1, perhaps subjects will not only be sensitive to the specific components of the outcomes (i.e., different attacking vessels) but also to the more general emotional components of these outcomes as well (i.e., the fact that all of these attacking vessels are dangerous).

It is noteworthy that specific and general PIT effects have not been extensively studied using avoidance paradigms either with human or non-human animal subjects. However, general transfer effects have been found using avoidance procedures with rats and dogs (Henderson, Patterson, & Jackson, 1980; LoLordo, 1967) and specific transfer effects have been reported with rats (Henderson, et al., 1980).

Methods

Participants

Fifty-one Brooklyn College students (29 female; 22 male) were recruited from introductory psychology and advanced psychology classes. Their ages ranged between 18 and 25 years. All students received course credit for their participation. All participants had normal or corrected vision.

Stimuli and Materials

The materials used were the same as in Experiments 1a and 1b except for the following changes. The conditioned stimuli used were the same except the blue box color was replaced with an orange colored box, in an effort to equate the overall luminance levels in the different boxes while maintaining their discriminability. The reinforcing outcomes used were black and white photographic images of a warplane (110mm × 152mm), warship (112mm × 150mm), and tank (110mm × 145mm) (see Figure 1). In the instrumental phase, these objects were presented with text below indicating that the enemy vessel was destroyed (e.g., Warplane Destroyed). In the Pavlovian phase, the same objects were presented with text below indicating that an attack was taking place (e.g., Tank Attack). The conditioned stimuli (except for the color change of one stimulus) and intertrial stimuli were the same images and were presented in the same fashion as in Experiments 1a and 1b. All stimuli, objects, and fixation cues were singly presented in the center of the screen with a 3s duration. An additional difference was that the instrumental instructions were presented on screen for 1 min.

Procedure

Participants were verbally instructed that they will be playing a computer game consisting of three parts. Participants were further instructed that in this game they would play the role of a commander of an army with the job of protecting their fictitious country, “Viltoma”, from enemy attack. During the first part of the experiment, the response options were identified (two yellow buttons) and participants were asked to use the middle and index fingers of their dominant hand to press them in order to destroy an attacking enemy vessel (i.e. warplane & warship). In the second phase it was explained that no buttons should be pressed, but that they should observe the S-O relationships presented on the screen. In the third phase, button pressing was again encouraged. Participants were told that questions would appear throughout the experiment and that the keyboard keys would be used for that purpose only. In this experiment there was no winner, no monetary payout, and no additional incentive beyond class credit as these did not seem to be required in the present study to encourage participants to respond.

Instrumental phase

After the presentation of the instrumental instructions for 1 min, the procedure here was the same as in Experiments 1a and 1b, except that participants' instrumental responses were now reinforced with an image of a warplane and the text `Warplane Destroyed', and an image of a warship with the text `Warship Destroyed'. These specific R-O relations were counterbalanced across subjects. This phase began with the following instructions:

Welcome. You are asked to play a game in which you are a military unit commander in charge of protecting Viltoma, your country, from enemy attacks.

Viltoma is being attacked by air and by sea. Your work will be to defend Viltoma by pressing the 2 yellow buttons on the response pad. One of these buttons fires missiles at oncoming warplanes, whereas the other one fires missiles at oncoming warships, but because of a malfunction in the missile launch mechanism, you do not know which button fires missiles at each type of target. Sometimes your missiles will hit their targets, but often they will miss their targets.

Your mission consists of destroying the warplanes and warships before they reach the Viltoma coast by firing your missiles. The sooner you discover the functions of the buttons, the more efficient your defense will be.

Good Luck! The people of Viltoma are depending on you.

Pavlovian phase

In this phase, the procedure was the same as in the previous two experiments except the story as explained in the instructions below was different. Similar to the other experiments, two of the colored boxes predicted the outcomes used previously to reinforce the responses in the instrumental phase (images of a plane or ship), one box was paired with a 3rd outcome (image of a tank), and the final stimulus was presented alone (CS-). In this phase, the text underneath each outcome indicated a vessel attack, for example `Warship Attack'. The specific S-O relations were counterbalanced following a Latin Square procedure. The rest of this phase followed the same procedures as Experiments 1a and 1b. This phase began with the following instructions:

You have done good work and successfully defended against enemy attacks to your coastline. However, the enemy has regrouped and begun to attack another region of Viltoma defended by another unit commander.

At this point, you can only observe what happens and be ready to offer assistance if called upon. This other unit commander will keep you informed of his current status and will send you a code using colored boxes to indicate which type of attack he is attempting to defend against. In this case, the enemy directs its attacks either from warplanes, warships, or tanks.

Your mission in this part of the game is to closely follow this other commander's progress and discover which colored box indicates whether there is an attack and which type attack has just occurred.

Test phase

In this phase, the procedure was the same as in Experiments 1 except the story as explained in the instructions below was modified to reflect the avoidance learning nature of the task. All CS and intertrial durations were 3s. This phase began with the following instructions:

Final Attack!!! You are called in to assist the other commanders in their effort to protect Viltoma. The other commanders will continue to send you the coded information to alert you to the type of attack they are facing and they need your help. You are now asked to press the buttons to fire missiles from your position to help fend off the enemy. Good luck… Viltoma is in your hands!!

Assessment phase

A contingency assessment questionnaire was again given, however in this experiment the assessment questions were incorporated into the task. Participants answered the same questions about the R-O relationships directly after the instrumental phase and about the S-O relationships directly after the Pavlovian stage. Again, the only difference in these assessment questions for Experiment 2 from the previous experiments was the specific outcome choices. Here the outcomes were warplane, warship, and tank. Again only those participants who could report the various S-O and R-O relationships with 100% accuracy were included in the data analysis (criteria 1). Additionally, if participants earned fewer than 5 of each outcome in the instrumental phase, then they were excluded as well (criteria 2). This criterion was not required in Experiments 1a and 1b because of the additional incentive given to participants to respond frequently. Based on these criteria, 22 subjects were excluded, leaving 29 qualifying participants. Seventeen participants were excluded based on criteria 1, two were excluded based on criteria 2, and three were excluded due to not completing the task and computer error. Color vision variability for the orange box stimulus accounted for seven of the seventeen excluded participants (based on criteria 1) as these participants saw the orange box as yellow. So when asked in the assessment about which object followed the orange box they chose nothing exclaiming post test that they never saw an orange box.

Results

The instrumental training data for Experiment 2 offered no surprises on the basis of the previous studies. The mean total number of Rplane and Rship responses during the 5-min instrumental phase were, respectively, 197.1 and 201.6, and these were reinforced, respectively, an average of 39.9 and 41.2 times. The obtained values for the variable ratio schedules were 4.9 for Rplane and 4.9 for Rship, both close to the programmed value of 5.0.

The number of each button press during each outcome presentation was also recorded as in Experiment 1. Once again, when subjects made button press responses during the times when the outcomes were presented, they tended to choose the response that had just produced the outcome. Specifically, participants made an average of 22.6 Rplane responses and 10.0 Rship responses when the Warplane Destroyed outcome was on screen, but 21.7 Rship and 12.1 Rplane responses when the Warship Destroyed outcome was on screen. A Response (same, different) × Outcome (ship, plane) ANOVA found only a main effect of Response [F(1,28) = 7.80, p = .01, MSerr = 460.07].

The test phase data for Experiment 2 is presented in figure 4. The figure displays the average number of Rplane and Rship responses in the Pre CS and CS periods for each stimulus (CSplane, CSship, CStank, and CS-). The data once again reveals that there was a significant specific transfer effect. This is indicated by a selective increase in Rplane responses when the CSplane stimulus was presented and a selective increase in Rship responses when the CSship stimulus was presented. Contrary to the findings of the previous Experiments (1a & 1b), in this experiment, a significant general transfer effect was also found. The mean number of Rplane and Rship responses increased during the CStank stimulus compared to the Pre CS period. Additionally, both responses decreased during the CS- compared to responding during the pre CS period.

Figure 4.

Mean rate of instrumental responding during the PIT test of Experiment 2. Data are shown separately in the pre stimulus (Pre Rplane, Pre Rship) and stimulus periods (Rplane & Rship) for each test conditioned stimulus (CSplane, CSship, CStank, and CS-).

A three way ANOVA examining the effects of Interval (Pre CS, CS), Response (Rplane, Rship), and Stimulus (CSplane, CSship, CStank, CS-), revealed significant Interval [F(1,28) = 11.02, MSerr = 190.13] and Stimulus [F(3,84) = 6.14, MSerr = 97.24] main effects and also significant Interval × Stimulus [F(3,84) = 11.49, MSerr = 83.02], Response × Stimulus [F(3,84) = 6.38, MSerr = 90.73], and Interval × Stimulus × Response [F(3,84) = 6.29, MSerr = 160.56] interactions. This three way interaction was further analyzed via one-way ANOVAs with a pooled error term (MSerr = 111.86) across the four levels of responding for each CS. These analyses revealed significant main effects for the CSplane [F(3,336) = 12.63], the CSship [F(3,336) = 10.71], the CStank [F(3,336) = 4.05], but not for the CS-. Post hoc analyses supported the finding that the CSplane selectively elevated Rplane responding, but not Rship responding relative to the pre-stimulus period [F(3,336) = 12.46]. The post hoc analysis for the CS ship similarly revealed that this stimulus selectively elevated Rship responding, but not Rplane responding relative to the pre-stimulus period [F(3,336) = 10.40]. Post hoc analysis for the CStank revealed that Rplane and Rship responding during the CStank did not differ but was greater than Rplane and Rship responding during the pre-stimulus period [F(3,336) = 4.00].

Although this analysis failed to demonstrate that CS- significantly decreased instrumental responding over Pre CS levels, a supplementary analysis did reveal such an effect. The MSerror on CS- trials (36.84) was considerably less than on the other three trial types (CSplane = 146.58; CSship = 180.12; CStank = 83.90). This suggests that the pooling procedure may not have been appropriate. A second analysis, using one-way ANOVAs based on each stimulus' own MSerror term, revealed a significant and reliable main effect for each stimulus; for the CSplane [F(3,84) = 9.64], the CSship [F(3,84) = 6.65], the CStank [F(3,84) = 5.40], and for the CS- [F(3,84) = 4.97]. Subsequent post-hoc tests revealed the same pattern of data as that described above for CSplane, CSship, and CStank. However, post-hoc tests performed on the CS- data also revealed that both responses were lower in the presence of the stimulus compared to the pre CS period [F(3,84) = 4.85].

Discussion

The aim of Experiment 2 was to investigate the possibility of both specific and general PIT effects using a more naturalistic, quasi-avoidance learning task similar to that introduced by Paredes-Olay et al. (2002). In this task, we successfully observed both specific and general effects. There were a number of procedural differences introduced in the present study, any one of which could account for the different results seen here. One of these is that our task rendered the emotional features of the outcomes more salient. Indeed, instrumental responding in the presence of S1, S2, and S3 in this study reached higher absolute levels than in Experiment 1a and 1b. Another difference is that this experiment used an avoidance-type training procedure as opposed to the positive reinforcement procedure used in Experiment 1. At the present time we cannot determine which of these features of the present task was responsible for the different results. Nevertheless, these findings are the first, to our knowledge, in which both specific and general PIT has been observed in the same task with humans. Additionally, we observed that our control stimulus, S4, which had never been paired with an outcome nonselectively decreased both instrumental responses. This finding is suggestive of the possibility that S4 functioned as a conditioned inhibitor, denoting the absence of an attacking vessel. However, our experimental design did not include an appropriate control condition to adequately assess this claim, so this conclusion must remain tentative.

General Discussion

The experiments presented here consistently revealed a specific Pavlovian to instrumental transfer effect as indicated by a selective increase of instrumental button press responses in the presence of a stimulus that, itself, was paired with the same outcome as that used to reinforce the response. Furthermore, in Experiment 2, in addition to the observation of a specific transfer effect, a general Pavlovian to instrumental transfer effect was observed as well. The general transfer effect was seen as a non-selective increase of instrumental button press responses in the presence of a stimulus that was reinforced with a third outcome not previously used to reinforce either instrumental response. In addition to these findings, in Experiment 2, a significant decrease in responding to the CS- stimulus (the stimulus associated with no outcome) was found. The results from these studies are consistent with other human learning studies in showing specific PIT (Bray et al., 2008; Paredes-Olay et al., 2002; see also Hogarth et al., 2007), but are the first to separately identify both specific and general PIT within a single experiment. The present results have a number of implications for understanding PIT effects as well as their applications and these will be discussed in turn.

The psychological processes that underlie Pavlovian to instrumental transfer are often discussed in associative terms. According to this framework specific and general PIT effects are based on the formation of associations between the CS and different components of the reinforcing outcome (e.g., see Konorski, 1967, pps 270–280). Specific PIT is thought to reflect a more “cognitive” association between the stimulus and highly specific sensory properties of the outcome (e.g., Corbit & Balleine, 2005; Delamater, 2007; Holland, 2004; Rescorla, 2001), while general PIT is thought to reflect a more “emotional” association between the stimulus and some general emotional or motivational property of the outcome (e.g., Corbit & Balleine, 2005; Dickinson & Dawson,1987; Rescorla & Solomon, 1967). But how does the presumed presence of these different associations result in the observed specific and general effects of Pavlovian stimuli upon instrumental behaviors? According to the bidirectional hypothesis (e.g., see Mackintosh & Dickinson, 1979; Pavlov, 1932) when a response is paired with a reinforcing outcome, this is assumed to result in a response-outcome association that can act in either the forward or backward direction. When a conditioned stimulus separately activates a representation of that same outcome, this should control a specific response by virtue of the backwardly acting response-outcome association. Specific transfer only requires that the outcome representations are distinctive. General PIT, on the other hand, requires a different mechanism. The most common explanation is that the CS activates a central motivational state that has general activating effects on performance (e.g., Rescorla & Solomon, 1967).

In the present studies, we assume that our instrumental responses and Pavlovian stimuli formed specific associations with their respective outcomes to explain specific PIT. In addition, in Experiment 2, the CS paired with a third outcome not used previously to reinforce the instrumental responses, S3, might have generally increased both instrumental responses because it associatively activated a generally arousing emotional state (e.g., perceived danger in the context of our task). Accordingly, it is possible that general transfer effects were not observed in Experiment 1 because neither the outcomes nor the monetary incentives used in that experiment were emotionally engaging enough. Thus, it could be argued that the stimuli in that study had little opportunity to enter into associations with some central motivational or emotional state. Whereas in neither of our studies did we truly employ outcomes that were traditional USs, it remains to be seen whether both specific and general PIT effects would be obtained under such circumstances.

As noted above, the present results are largely consistent with others in the literature; however, there are some discrepancies as well. For example Bray et al. (2008) used an experimental design similar to ours but with traditional appetitive USs (juice) instead of instructed USs (as in our tasks). Both specific and general PIT, in principle, could have been obtained in this experiment, but only specific PIT was reported. Our experiments differed from Bray et al. (2008) in a number of ways, but, perhaps, the key procedural difference was that we assessed transfer by measuring rate of responding. In the Bray et al. (2008) task, subjects also chose between two response options during the PIT test but the trial was terminated after the first response. While this task allows for the identification of a specific transfer effect (through selective choice responses), the only way in which it could be used to assess general transfer is by measuring response latency on S3 trials. Unfortunately, no such results were reported. Perhaps response rate is a more sensitive measure than latency for the simple reason that PIT effects can be assessed over time.

The Paredes-Olay et al. (2002) study, like the experiments reported here, found outcome-specific transfer. However, we consistently observed a selective increase in responding while Paredes-Olay et al. (2002) consistently found a selective decrease. One explanation for this difference could be that responding during the pre-CS period in the test phase was considerably higher in the Paredes-Olay et al. study compared to our Experiment 2 (which was a slightly modified version of the Paredes-Olay et al. task). Studies examining specific PIT in rats have also often found either selective increases (e.g., Corbit & Balleine, 2005; Delamater & Holland, 2008; Holland, 2004) or decreases (e.g., Colwill & Rescorla, 1988; Delamater & Holland, 2008), and some have speculated that instrumental baseline levels may be important in determining which of these might occur (e.g., Colwill & Rescorla,1988).

Another explanation for why selective increases or decreases may be found in PIT studies is the presence of potentially competing Pavlovian conditioned responses (Colwill & Rescorla, 1988; Delamater & Holland, 2008). For example, in situations where the CRs are incompatible with the instrumental responses, any CRs that occur during the transfer test will potentially decrease instrumental responding. It is noteworthy that in the Paredes-Olay et al. (2002) study subjects were instructed during the Pavlovian phase to press different response buttons than those used during instrumental training to indicate their differential outcome predictions during different Pavlovian cues. In our task, subjects were merely instructed to observe the different S-O relationships in effect, without being asked to make any competing responses. Thus, the presence of competing CRs may have led to a selective decrease in the Paredes-Olay et al. study, whereas the absence of any competing Pavlovian CRs in our studies may have led to selective increases during our selective PIT tests.

Another issue worth some comment concerns the nature of our task used in Experiment 2. We have conceptualized this as an avoidance-like task, but, strictly speaking, the task may not be best described as a true avoidance procedure. For instance, our instrumental training phase in some ways resembles an unsignaled avoidance procedure as the participants could learn that pressing response buttons lead to the absence of an enemy attack. Our subjects were given instructions that indicated they were currently under enemy attack. Thus, when a given button press response resulted in the destruction of an attacking vessel, by implication, this also led to the absence of an attack by that vessel. Subjects could, therefore, have learned that a particular button press response led to avoidance of a particular type of attack. However, it is also likely that subjects could have learned that different responses led to the destruction of different types of vessels. Characterizing the task in this way emphasizes more the potential excitatory, rather than inhibitory, relations between responding and the different outcomes.

It remains an empirical question whether or not the PIT results of our study would be different had we used a “true” avoidance procedure. In order to accomplish this, our task could be modified by presenting signs of successful attacks by enemy vessels unless subjects were to respond appropriately. In our task, we only presented information regarding a successful destruction of an enemy vessel without providing signs of a successful enemy attack. The “true” avoidance procedure may be more effective, than ours, at emphasizing the inhibitory associations between responding and different outcomes, but we have no reason to think that this change in procedure would result in different patterns of transfer results in the PIT test. Nevertheless, this remains an issue for further research.

One final issue concerns the nature of the underlying mechanisms at work in our tasks. While we have been emphasizing an associative account of PIT it is worth pointing out that an inferential reasoning approach has also been used to discuss various human learning phenomena (e.g., Declercq and De Houwer, 2009; De Houwer, 2009; Waldmann & Holyoak, 1992). According to this view, subjects performing in an associative learning task represent the programmed relations between events (e.g., CS-US or action-outcome) in propositional form, and then use these propositions to guide performance. Although we have not seen this idea applied to PIT, our results may be understood in these terms. For instance, if subjects were to learn during the instrumental phase that “a left button press causes the destruction of a ship”, and during the Pavlovian phase that “a red box indicates an ensuing ship attack,” then during the test phase subjects might rationally conclude that they should press the left button in the presence of a red box. This would explain our specific PIT effect in propositional terms.

It is less obvious to us how subjects might have integrated the various propositions they learned during the instrumental and Pavlovian phases in such a way to result in the general PIT effect we observed in Experiment 2. Instances of general PIT imply a certain degree of nonrationality (see also Dickinson & Dawson, 1987) because there is no direct link established between the propositions presumed to have been learned during the instrumental and Pavlovian phases. In our case, the third CS signaled an attacking vessel that participants had no experience defending against. Indeed, the only experience our participants had with firing different types of missiles was that these missiles were very specific in their effectiveness, destroying only one of two vessel types. In other words, participants had no basis for inferring that these missiles could be effective for any other type of attacking vessel. Given these circumstances, we suggest that the third CS may have elicited a heightened emotional state, “danger” or “panic,” that could have generally increased button press responding without really knowing what else could be done. A more “cognitive” inferential reasoning account of these data, on the other hand, may suppose that in the absence of any additional information one might as well fire both types of missiles in the presence of the third CS on the grounds that each of these has at least been effective in the past against one attacking vessel. Either of these explanations could potentially apply to our general transfer effect, and future work will be needed to resolve this issue. Regardless of how we interpret these findings, however, it should be apparent that the specific and general effects reported here point to the operation of different underlying psychological mechanisms – either cognitive and emotional in nature, on the one hand, or inferential reasoning in light of certain versus uncertain outcomes in the other – and for this reason the PIT task we developed here may be especially useful as a tool for further study in human learning.

In summary, the aim of the present studies was to provide evidence for both specific and general forms of PIT in a human learning task because these two forms of PIT have been understood to reflect different underlying “cognitive” and “emotional” associative processes. While Experiment 1 successfully identified specific PIT using a task involving positive response-reinforcer relations, Experiment 2 provided evidence for both specific and general PIT in a single avoidance-related task with human subjects. There has been a considerable amount of attention recently directed towards an analysis of the neural mechanisms of PIT effects in both rats (Blundell, Hall, & Killcross, 2001; Corbit, Muir, & Balleine, 2001; Corbit & Balleine, 2005; Hall et al., 2001; Murschall & Hauber, 2006; Ostlund & Balleine, 2007; 2008) and humans (Bray et al., 2008; Talmi et al., 2008). Because the task we developed in Experiment 2 separately identifies both specific and general PIT, the use of this task in further human neuroimaging studies could potentially provide useful information regarding the neural bases of the cognitive and emotion circuits underlying these phenomena.