Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Mar 4.
Published in final edited form as: Neuroimage. 2008 Aug 28;43(4):748–755. doi: 10.1016/j.neuroimage.2008.08.021

Dissociable Roles of Medial Orbitofrontal Cortex in Human Operant Extinction Learning

Elizabeth C Finger 1,2, Derek GV Mitchell 1,3, Matthew Jones 1, R J R Blair 1
PMCID: PMC3941699  NIHMSID: NIHMS79240  PMID: 18793731

Abstract

Operant extinction, which features modification of instrumental responses to stimuli following a change in associated reinforcement, is an important form of learning for organisms in dynamic environments. Animal studies have highlighted orbital and medial prefrontal cortex and amygdala as mediators of operant extinction. Yet little is known about the neural mediators of operant extinction learning in humans. Using a novel fMRI paradigm, we report dissociable functional responses in distinct regions of medial orbitofrontal cortex (mOFC) during successful appetitive and aversive based operant extinction. During successful operant extinction, increased activity was observed in frontopolar OFC, while decreased activity was observed in caudal mOFC and rostral anterior cingulate cortex (rACC) relative to both (i) successful control trials where the reinforcement associated with the stimulus does not change; and (ii) successful acquisition trials during initial learning of the stimulus- reinforcement associations. Functional connectivity analysis demonstrated inverse connectivity between frontopolar OFC and both rACC and the amygdala. These data support animal models suggesting the importance of mOFC - amygdala interaction during operant extinction and expand our knowledge of the neural systems in humans. These findings suggest that in humans, frontopolar OFC modulates activity in caudal mOFC, rACC and amygdala during successful operant extinction learning.

Keywords: frontopolar, amygdala, instrumental, anterior cingulate cortex

Extinction learning in humans facilitates behavioural flexibility in response to changing contingencies and, when impaired, has been associated with neuropsychiatric disorders such as psychopathy and addiction. Two forms of extinction can be considered: Pavlovian extinction, the extinction of a classically conditioned response when the conditioned stimulus is no longer paired with the unconditioned stimulus (Pavlov, 1927), and operant extinction, the extinction of a conditioned operant response when the response is no longer followed by the reinforcing stimulus (Skinner, 1938). Recent human neuroimaging work has examined the neural systems involved in Pavlovian extinction (Gottfried and Dolan, 2004; LaBar et al., 1998; Milad et al., 2005; Phelps et al., 2004) but, to our knowledge, none has considered operant extinction.

On the basis of animal studies, the successful extinction of an operant response results from the replacement of the original stimulus-outcome association with an updated stimulus-outcome association which then guides behaviour (Ostlund and Balleine, 2005; Schoenbaum et al., 2003b). As in Pavlovian extinction, in operant extinction learning the original stimulus-outcome association is not erased but the new association is preferentially selected to guide behaviour (Ellson, 1938; Rescorla R.A. and Wagner and Skucy, 1969). Orbital frontal cortex (OFC) is critically implicated in this process. When OFC is lesioned, an increase in perseverative errors (failures to modify original response) is observed in both operant extinction and reversal learning paradigms in animal and human studies (Butter, 1969; Chudasama and Robbins, 2003; Gallagher et al., 1999; Izquierdo and Murray, 2005; Schoenbaum et al., 2002; Schoenbaum et al., 2003a). It has been proposed that OFC facilitates changes in stimulus-reinforcement encoding in other brain regions such as the amygdala, particularly when contingencies change (Saddoris et al., 2005; Schoenbaum et al., 2007). Currently, it remains unclear which regions of human OFC may be critically involved in this process. However, studies of emotion regulation suggest that frontopolar regions of rostral medial orbitofrontal cortex may be the region of human PFC that facilitates selection of the new stimulus-outcome association to guide behaviour. Frontopolar OFC is activated during successful operant responses in the setting of positive and negative emotional distracters (Blair et al., 2007) and has been implicated in the adaptive modulation of emotional information during instrumental tasks (Mitchell et al., 2008). This region of frontopolar OFC is also activated during voluntary suppression of emotional stimuli (Levesque et al., 2003; Ohira et al., 2006).

In contrast, a second region of rodent medial prefrontal cortex (mPFC), the infralimbic region, has been implicated in the retention or recall of Pavlovian extinction learning (Milad and Quirk, 2002; Phelps et al., 2004; Quirk et al., 2000). Lesions of this region result in an increase in regressive errors (the return to the previously learned but now invalid response) (Chudasama and Robbins, 2003; Ragozzino, 2007). In human Pavlovian extinction, caudal regions of mOFC have been proposed to serve an analogous role. In a classical fear extinction paradigm, this region displayed decreased BOLD signal in early extinction relative to late and was correlated with expression of the conditioned response, leading to the suggestion that it may play a role in extinction retention (Phelps et al., 2004). Furthermore, during recall of classically conditioned fear extinction, proximal regions of caudal mOFC demonstrate augmented activity (Milad et al., 2007), and thickness of this region is positively associated with fear extinction recall (Milad et al., 2005; Rauch et al., 2005).

In the present study, we use a passive avoidance extinction paradigm to examine operant extinction learning (Kosson et al., 2006; Newman and Kosson, 1986). This paradigm is based on animal go-no-go paradigms in which a single stimulus is presented in association with either reward or punishment (Schoenbaum et al., 1998, 1999). Following a change in contingency, the valence of the reinforcement associate with the stimulus is changed. In the extinction phase, the participant extinguishes instrumental responses to the previously rewarding stimulus, and learns to respond to the previously punishing but now rewarding stimulus. Performance in this task is thought to rely on the formation of appropriate stimulus-outcome associations during acquisition and then, in extinction, the replacement of the original stimulus-outcome association with an updated stimulus-outcome association (Saddoris et al., 2005; Schoenbaum et al., 2000).

While this form of operant extinction likely shares functional components and neural substrates with response reversal tasks, the two forms of learning differ in several important ways. First, human and non-human primate reversal learning paradigms (Budhani, 2006; Cools et al., 2002; O'Doherty et al., 2003) typically involve object discrimination reversal learning: the same pair of stimuli is repeatedly presented and the subject must learn to respond to one component of the pair, and then to the other member of the pair. It has been argued that such visual discrimination reversal learning paradigms are solved by updating stimulus-response associations or response-outcome associations rather than stimulus-reinforcement associations (Baxter and Murray, 2002). In such paradigms, it has been suggested that the simultaneous presentation of two stimuli can be considered as one compound stimulus (Baxter and Murray, 2002). Thus the reinforcement associated with the (compound) stimulus remains constant, it is only the response to the stimulus (respond to component A rather than B) that need be updated. The stimulus is not good or bad; rather to gain reward, the correct response must be made towards the stimulus (respond towards component A rather than B). In contrast, the paradigm used in the current study is thought to rely on the formation of appropriate stimulus-outcome associations (during acquisition) and then, in extinction, the replacement of the original stimulus-outcome association with an updated stimulusoutcome association (Saddoris et al., 2005). Effectively, the individual learns that some of the stimuli are good and responses should be made to them while some stimuli are bad and responses should be withheld. In extinction, some of the “good” stimuli become bad and some of the bad stimuli good.

Furthermore, lesion studies indicate this form of operant extinction and reversal learning are dissociable. The amygdala is not necessary for successful object discrimination and reversal learning as lesions do not impact performance (Izquierdo and Murray, 2007). In contrast, amygdala lesions do impair performance following the contingency change in stimulus-outcome based instrumental tasks (Schoenbaum et al., 2003a). Recent functional imaging studies in healthy adults further suggest that reversal learning tasks and passive avoidance learning paradigms recruit both shared and distinct regions of orbitofrontal and medial prefrontal cortex. The suggestion is that both rely on the appropriate representation of reinforcement information by medial regions of OFC. However, reversal learning paradigms also consistently reveal activation in dorsomedial and ventrolateral PFC with increased activity found during reversal errors relative to correct responses (Cools et al., 2002; O'Doherty et al., 2003). These regions also show significant activation to incorrect responses relative to correct responses during the acquisition phase of reversal learning paradigms (Budhani et al., 2007). In the context of these paradigms, dorsomedial prefrontal cortex is thought to respond to response conflict generated by either an internal error signal or external punishment and that ventrolateral prefrontal cortex is recruited to aid selection of an alternate response option (Budhani et al., 2007). We suggest that successful behavioral extinction may not be associated with the level of response conflict seen in reversal learning paradigms. However, we argue that successful behavioral extinction critically involves appropriate recoding of the stimulus-reinforcement associations that are thought to drive correct behavioral selection (respond vs. not respond to the stimulus) during acquisition (cf. Kosson et al., 2006). Based on studies of emotion regulation we believe that a frontopolar region of vmPFC may be critically involved in appropriate recoding of the stimulus-reinforcement associations.

Based on the animal and human literature described above, we hypothesized that successful operant extinction learning would entail suppression of the original stimulus-action-outcome representations following the change in contingency. We predicted that frontopolar OFC would show augmented activity during successful operant extinction learning in humans. Furthermore, if the role of frontopolar OFC is to facilitate updating of stimulus-outcome expectations, it should be augmented during successful extinction of operant responses to both previously appetitive and aversive cues. In contrast, based on findings of decreased caudomedial OFC activity early Pavlovian extinction learning (extinction learning conducted immediately after acquisition) (Phelps et al., 2004), we predicted that caudal mOFC would show a decrease in activity during successful extinction of responses to both appetitive and aversive cues, particularly early in extinction learning when the old outcome associations should be suppressed in favour of the new associations. Finally, if frontopolar OFC suppresses the initial stimulus-outcome representations in caudal mOFC, we predicted that activity between these regions should be negatively correlated.

Materials and Methods

To test the hypothesis described above we developed an operant extinction learning fMRI task based on the “go-no-go” task used extensively in animal work (Roesch et al., 2006; Schoenbaum et al., 1998, 1999; Schoenbaum and Eichenbaum, 1995; Setlow et al., 2003) and its human homologue, the passive avoidance learning paradigm (Kosson et al., 2006; Lykken, 1957; Newman and Kosson, 1986). In these tasks, the organism learns to respond to stimuli associated with reward, and refrain from responding to stimuli associated with punishment. The contingency then changes for a subset of the stimuli, and the organism must learn to extinguish prior responses to rewarding stimuli and learn to respond to previously punishing stimuli.

Participants

Twenty right handed participants took part in the study (10 women and 10 men; mean age= 26 years old). One participant demonstrated a low response rate (> 2 S.D. below the mean response rate) and was not included in the subsequent analysis. All participants were in good health with no past history of psychiatric or neurologic disease and all gave informed written consent. The study was approved by the NIMH Institutional Review Board.

Passive Avoidance Extinction Task and Experimental Procedure

In the current study, we used the human passive avoidance learning paradigm studied previously (Kosson et al., 2006) and modified it by including an extinction phase following acquisition (see figure 1). In this extinction phase, the reinforcement value of half of the stimuli changed. The participant’s goal was to learn to extinguish responses to previously rewarding stimuli and similarly, to extinguish avoidances and learn to respond to previously punishing stimuli.

Figure 1.

Figure 1

Open in a new tab

Passive Avoidance Extinction Task. In an event-related design stimuli were shown serially for 1100 ms followed by feedback for 1000ms and a fixation cross for 200ms. Participants responded to stimuli by pressing a button to a stimulus and were awarded or penalized with 100 points. Passively avoiding a stimulus by not pressing the response button resulted in a blank screen during the feedback window. In the extinction phase, half of the original stimulus-reinforcement associations were changed (3 rewarding and 3 punishing), so that participants had to learn to respond to some previously punishing stimuli, and avoid responding to some previously rewarding stimuli. Control stimuli did not change their reinforcement contingency during the extinction phase. Participants were presented with each of the 12 stimuli once per block in a randomized order, for 8 blocks in the acquisition phase and 8 blocks in the extinction phase.

The test stimuli consisted of 12 white Snodgrass images per run presented on a black background (see Figure 1). As in Kosson (Kosson et al., 2006) six stimuli were initially associated with positive reinforcement: if a button press was made while the stimulus was displayed, the participant received positive feedback (“you WIN 100 points”) and a running point total was displayed. Six were associated with negative reinforcement and a response resulted in negative feedback (“you LOSE 100 points”). If the participant did not respond to a stimulus, they neither won nor lost points and a blank screen replaced the feedback screen. Participants were told, “You will be presented with a series of images. Some of these images are good and will gain you points if you press the button when they are showing. Some images are bad and will lose you points if you press the button when they are showing. If you do nothing you will neither gain nor lose points. Your goal is to win as many points as you can.”

The acquisition phase consisted of 8 blocks: in each block each stimulus was presented once in a randomized order. The acquisition phase was followed immediately by the extinction phase, in which the reinforcement value of 3 good stimuli and 3 bad stimuli were changed. The remaining 3 “good” and 3 “bad” stimuli served as control stimuli whose reinforcement value was not changed in the extinction phase. During the extinction phase, the twelve stimuli (3 always “good,” 3 newly “good”, 3 always “bad”, 3 newly “bad”) were presented in a randomized order once per block for 8 blocks. Each trial lasted 2300 ms, beginning with a Snodgrass image for 1100 ms followed by the feedback screen for 1000ms and finally a fixation cross for 200ms. Fixation trials (2300ms) served as the baseline and were presented for 25% of trials (4 randomly interspersed per block). Participants were trained outside of the scanner on a practice run containing 8 sample stimuli displayed for 2 blocks. Participants subsequently completed 4 runs presented in a random order in the MRI scanner. Each run lasted 10 minutes and 18 seconds. Four different task versions counterbalancing stimuli with reinforcement assignments were developed and randomly administered to control for any inadvertent differences in processing of the cue Snodgrass images.

MRI Parameters

Participants were scanned during task performance using a 3.0 Tesla GE Signa scanner. A total of 269 functional images per run were taken with a gradient echo planar imaging (EPI) sequence (repetition time = 2300ms, echo time = 23 ms, 64 × 64 matrix, flip angle 90°, FOV 24cm). Whole brain coverage was obtained with 34 axial slices (thickness 3.3mm; in-plane resolution 3.75 × 3.75mm). A high resolution anatomical scan (three-dimensional Spoiled GRASS; repetition time = 8.1ms, echo time = 3.2ms; field of view = 24cm; flip angle = 20°; 124 axial slices; thickness = 1.0 mm; 256 × 256 matrix) in register with the EPI dataset was obtained covering the whole brain.

Imaging Data Analysis

The AFNI software package (http://afni.nimh.nih.gov/afni) was used for image data processing (Cox and Hyde, 1997). Functional images from the first 6 trials of each run collected before equilibrium magnetization was reached were discarded. Functional images from the 4 time series were motion corrected and spatially smoothed with a 6 mm full-width half-maximum Gaussian filter. The time series were normalized by dividing the signal intensity of a voxel at each time point by the mean signal intensity of that voxel for each run and multiplying the result by 100. Resultant regression coefficients represented a percent signal change from the mean. Following this, 16 regressors characterizing the trial types (accuracy: correct vs. error), response type (response vs. avoidance), and trial type (extinction vs. control) were created by convolving the train of stimulus events with a gamma-variate hemodynamic response function to account for the slow hemodynamic response. Linear regression modelling was performed using the 16 regressors described above plus regressors to model a first order baseline drift function. This produced a beta coefficient and associated t-statistic for each voxel and regressor.

Individual anatomic and functional images were normalized to the standardized space of Talairach and Tournoux (Talairach, 1988). The group analysis was then performed through ANOVAs on whole brain data. The threshold was set at p< 0.005 (corrected at p< 0.05 for multiple comparisons). Threshold correction was done using the AlphaSim program in AFNI, which applies Monte Carlo simulation to calculate the probability of false positive detection, taking into consideration both the individual voxel probability thresholding and cluster size. Average percent signal change was measured within each significant cluster of 50 mm3 or greater. Significant main effects were followed up with ANOVAs to further characterize the percent signal change according to trial type and response type during the extinction phase (blocks 9–16). To aid interpretation of main effects of interest, post-hoc examination of average percent signal change in BOLD responses during acquisition (blocks 1–8) and extinction errors was conducted using functional ROIs with p<0.005.

Functional Connectivity Analysis

Functional connectivity, or covariation of the BOLD responses throughout the brain, was assessed with respect to a seed voxel of maximum intensity in frontopolar OFC. Following conversion of individual participants’ time series to Talairach space, the time series of the seed voxel in right frontopolar OFC (BA 10) was extracted. After correction for baseline, linear and quadratic trends, the average of the brain time series was treated as a global signal and used as a covariate in the correlation analysis. A voxel-wise correlation analysis was conducted between individual voxel time series and that of the identified seed voxel in frontopolar OFC. The resulting correlation coefficient was squared to calculate the proportion of signal variance attributable to correlation with the seed. To reduce the skew and normalize the sampling distribution, correlation coefficients were converted to a Gaussian variable with a Fisher transformation formula. A one-sample t-test was conducted on the transformed correlation coefficients to identify regions positively or negatively correlated with the target voxel at group level at a threshold of p<0.001.

Results

Behavioral Results

A 2 (trial type: extinction vs. control) × 8 (block) repeated measures ANOVA was conducted on correct hits (trials when the participant responded to a stimulus associated with reward) in the extinction phase (blocks 8–16). This revealed main effects for extinction (F(1,18) = 13.51, p<0.005) and block (F(7,126) = 6.76, p<0.001), and an extinction × block interaction (F(7,126) = 14.39, p<0.001) (see figure 2a). Unsurprisingly, participants made more correct responses on control trials (stimuli where the reinforcement value had not changed since the acquisition phase) relative to extinction trials, particularly for the early blocks following reinforcement change.

Figure 2.

Figure 2

Open in a new tab

Behavioral Results. (a) Mean Correct Hit Responses for control and extinction trial types over the extinction phase (blocks 9–16). (b) Mean Correct Avoidance Responses for control and extinction trial types over the extinction phase (blocks 9–16). (c) Mean Reaction times for Correct Hit Responses for control and extinction trial types over the extinction phase (blocks 9–16).

A second 2(trial type: extinction vs. control) × 8(block) ANOVA conducted on correct avoidances in the extinction phase of the task (blocks 9–16) (figure 2b). This revealed main effects of block (F(7,126) = 44.69, p<0.001) and a significant trial type × block interaction (F(7,126) = 10.15, p<0.001), but no significant main effect of trial type (F 1,18) = 2.3, p <0.15). Correct responses increased with greater exposure (increasing block number) for both extinction and control trial types across the blocks. In block 9 alone (the first block of the contingency change) participants made more correct avoidance responses on control trials (to stimuli which did not change reinforcement association) relative to extinction trials (to stimuli which were no longer rewarding).

Reaction time Data

Reaction time data in the extinction phase was assessed with a 2 (trial type: extinction vs. control) × 8(block) ANOVA on correct responses (figure 2c). This demonstrated a main effect of block (F(7,126)= 5.07, p<0.001). There was no main effect of response type (F(1,18) =2.23, p=0.15) nor response type × block interaction (F<1, n.s.). Examination of the main effect of block demonstrated that reaction times increased following the first block in extinction phase (block 9) when participants were exposed to new stimulus-outcome associations.

FMRI Results

Successful Extinction

A 2 (trial type: extinction vs. control) × 2 (response type: response vs. avoidance) ANOVA was conducted on BOLD response data from correct responses during the extinction phase (blocks 9–16). This yielded main effects of both extinction and response type, with no significant interactions.

The main effect of extinction identified regions which showed a differential response during successful extinction trials compared to successful control trials. In line with predictions, BOLD responses were increased during extinction trials relative to non-extinction control trials in regions of frontopolar OFC at the frontal pole: right and left medial frontal gyrus (BA 10/11) (F(1,18)= 26.50; p<0.001 and F(1,18) =20.93; p<0.001 respectively) and the left middle frontal gyrus (BA 11) (F(1,18)= 16.51; p=0.001) (see Figure 3a+bi). This increased activation was present during successful extinction for both response and avoidance trials (see figure 3a+bii). A distinct pattern of BOLD response was observed in more caudal regions of mOFC and rACC. In these regions, BOLD responses were reduced during successful extinction trials relative to control trials: medial frontal gyrus (BA 25) (F(1,18) = 12.3; p<0.005) and rACC (F(1,18)= 14.18; p=0.001) (see figure 3c+di). This was found for both response and avoidance successful extinction trials (see figure 3c+dii).

Figure 3.

Figure 3

Open in a new tab

Differential BOLD Responses during Successful Extinction. BOLD response and % signal change analysis demonstrate increased responses during successful extinction compared to control trials in (a) frontopolar OFC (left medial frontal gyrus/ BA 11 and (b) right medial frontal gyrus/ BA 10). Decreased responses were observed in caudal mPFC including (c) right anterior cingulate cortex and (d) right medial frontal gyrus/ BA 25. Asterisk (*) denotes variables used in the initial ANOVA on BOLD response data. BOLD responses during acquisition and extinction error trials are also shown for comparison in (i). (ii) Comparison of successful appetitive (response) and aversive (avoidance) control and extinction trials.

The main effect of response type identified regions which all demonstrated increased activity during responses relative to avoidances (see supplemental table 2). These regions included the temporal-occipital cortex, thalami, caudate, anterior and dorsal cingulate cortex, superior and inferior parietal lobules, inferior, middle and superior frontal gyri, and precentral gyrus.

We next investigated whether regions involved in extinction learning would differ in early (first 4 blocks) relative to late (last four blocks) in the extinction process. A 2 (phase: early extinction vs. late extinction) × 2 (trial type: extinction vs. control) × 2 (response type: response vs. avoidance) ANOVA was conducted on the BOLD data from correct responses. This demonstrated significant main effects of trial type (extinction vs. control) and main effects of phase (early vs. late), and response type, with no significant trial type × phase interactions, and one phase × response type × trial type interaction (see supplemental table A). Examination of the main effect of phase revealed regions implicated in attentional processing, including dorsolateral prefrontal cortex, were more active during early relative to late blocks of extinction learning. Examination of the main effect of trial type (extinction vs. control) demonstrated that the frontopolar region of mOFC was more active for extinction trials than control trials in both the early and late phases of the extinction phase, while anterior cingulate cortex was less active in both early and late phases of extinction trials compared to control trials.

Functional Connectivity

Following the findings above demonstrating differential activity in frontopolar OFC compared to anterior cingulate cortex and caudal mOFC during extinction learning, we next examined whether activity in the region of frontopolar OFC identified during successful extinction correlated with activity in the anterior cingulate cortex and caudal mOFC. Functional connectivity throughout the brain was assessed with respect to a seed voxel of maximum intensity in right frontopolar OFC (right medial frontal gyrus/ BA 10 Talairach (−15,−64,−5) selected because it was the voxel of greatest mean % signal change identified by the main effect of trial type during successful extinction. This analysis demonstrated that activity in frontopolar OFC was negatively correlated with activity in rACC (p<0.005). Furthermore, frontopolar OFC activity also demonstrated significant negative correlations with activity in the right and left amygdalae (p<0.0001). (see figure 4 and table 2).

Figure 4.

Figure 4

Open in a new tab

Functional Connectivity Analysis. Seed voxel in the region of frontopolar OFC (right medial frontal gyrus) (a) activated by extinction errors and successful extinction (BA 10) demonstrated negative correlations (in blue) with the left and right amygdalae (b), and rostral anterior cingulate cortex (c).

Table 2.

Regions demonstrating significantly correlated activity with seed voxel in right medial frontal gyrus (BA 10)

Region L/R BA x y z cluster size mm3
Negatively correlated regions
amygdala and parahippocampal gyrus L −27 −7 −12 1768
amygdala R 27 −9 −10 616
rostral anterior cingulate cortex R 24 −13 −29 −10 440*
red nucleus and substantia nigra L −7 −17 −6 6328
culmen/cerebellum L −31 −51 −24 648
culmen/cerebellum R 27 −45 −24 208
Positively correlated regions
cuneus R 18 15 −101 10 1864
cuneus L −27 −91 30 200
middle occipital gyrus L 18 −21 −101 4 592
middle occipital gyrus R 21 −91 14 152
middle frontal gyrus R 11 31 41 −8 328
medial frontal gyrus L 10 −11 57 −6 112
Open in a new tab

MNI coordinates of peak activation. Regions and Brodmann area (BA) according to Talairach Daemon Atlas. All regions significant a threshold p<0.0001 except * threshold at p<0.01.

Discussion

The present study investigated the neural bases of successful operant extinction learning in humans. Using a whole-brain voxel wise analysis, we identified extinction related activity with two dissociable functional patterns in regions of mOFC. First, we report the frontopolar region of frontopolar OFC in particular was the region of mPFC associated with increased activity during successful operant extinction learning in humans. Second, successful operant extinction was associated with decreased BOLD responses in caudal mOFC and rostral anterior cingulate cortex. Third, functional connectivity analysis demonstrated an inverse relationship between activity in frontopolar OFC and both the amygdala and rACC. Together these results suggest that in operant extinction learning in humans, frontopolar OFC may modulate activity within the amygdala and rostral anterior cingulate to facilitate rapid updating of stimulus-reinforcement associations.

Successful operant extinction requires updating of responses based on the new contingency. It has been proposed that in the setting of a contingency change, successful operant extinction learning entails formation and use of alternate stimulus-reinforcement associations to guide motor responding (Schoenbaum et al., 2002). In non-human animal models of successful operant learning, OFC is thought to process outcome expectancies based on stimulus-reinforcement associations (Izquierdo et al., 2004; McDannald et al., 2005; Pickens et al., 2005; Schoenbaum and Roesch, 2005; Tremblay and Schultz, 2000). In the present study, regions of frontopolar OFC at the frontal pole were the only areas to demonstrate significantly increased activation during successful extinction relative to control trials. In short, BOLD responses within frontopolar OFC were augmented for successful extinction in comparison to previously learned and unchanged correct responses, to initial learning in acquisition, or to errors. Extrapolating from animal models (Schoenbaum and Roesch, 2005; Schoenbaum et al., 2007), we suggest the increased BOLD responses in frontopolar OFC during successful extinction may reflect the manipulation of stimulus-reinforcement associations mediated by the amygdala. We suggest that during human operant extinction learning, frontopolar OFC may gate original representations of stimulus-reinforcement associations in the amygdala when they are no longer consistent with the current contingency. In line with this, the regions identified here were proximal to those implicated in previous emotional regulation studies thought to involve manipulation of stimulus-reinforcement associations mediated by the amygdala (Blair et al., 2007; Mitchell et al., 2007; Ohira et al., 2006). Moreover, it is notable that the functional connectivity analysis identified a highly significant inverse relationship between this region of frontopolar OFC and the amygdala.

It is worth noting that frontopolar OFC demonstrated increased activation during successful operant extinction trials whether the updated response was a successful new approach for reward or a successful new avoidance of punishment. This finding further supports previous suggestions that OFC is not mediating inhibition but rather updating stimulus-reinforcement associations generally (Berlin et al., 2004; Schoenbaum et al., 2007). Our voxel-wise whole brain analysis also identified a more caudal region of mOFC (medial frontal gyrus (BA 25)) and anterior cingulate cortex which demonstrated a different pattern during successful extinction. In these regions, successful extinction trials were associated with decreased activity relative to control trials (i.e., trials to stimuli which did not change reinforcement contingency)(see figure 3c+d). This region of caudal mOFC (MNI coordinates: 12, 36, −12) is proximal to the subgenual region previously correlated with extinction of Pavlovian fear conditioned responses by Phelps (Phelps et al., 2004) (Talairach coordinates: −2, 38 −3 and −4, 31, −6) and the regions of medial gyrus rectus and caudal mOFC identified during classical extinction by Gottfried (Gottfried and Dolan, 2004) (MNI: 15, 27, −21; 12, 21, −12). It is also proximal to the regions of mPFC identified by Milad (Milad et al., 2005) (Talairach: 4, 30, −12 and 6, 28, −15) which demonstrated significant correlations between gray matter volume and retention of fear extinction.

This region of caudal mOFC has been implicated in the retention or recall of extinction learning (Milad and Quirk, 2002; Phelps et al., 2004; Quirk et al., 2000). However, while our results match predictions of the involvement of this region based on Pavlovian extinction work, as we observed a suppression of this region when subjects behaviourally exhibited successful operant extinction our data do not clearly support the role of this region in extinction retention. Phelps et al. did report decreased activity within this region to what had been the CS+, relative to the CS−, during the early phase of extinction (one day after acquisition). However, it was the absence of this suppression which predicted extinction on day 2. In contrast, Milad (Milad et al., 2007) reported increased BOLD responses in this region during late day 1 classical fear extinction learning relative to control trials, and an increase on day 2 of extinction. The reason for these mixed results is not yet clear. Of course, it should be remembered that there are considerable differences between operant and Pavlovian extinction. Moreover, it is notable that the contrasts implicating this region in the Pavlovian extinction studies differ considerably across studies. Phelps et al. (2004) examined differences between a stimulus previously associated with a shock on 100% of trials relative to a stimulus never presented with shocks. Milad et al. (2007) compared responses to a stimulus previously associated with shock on 60% trials but now extinguished relative to a stimulus now associated with shocks during early extinction, and relative a stimulus associated with shocks that was not extinguished during late extinction. Furthermore, Milad et al. compared the first four trials of early extinction on day 1 with the last 12 trials of “late” extinction, while Phelps et al. examined early and late (first ½ of trials vs. last ½ of trials during both day 1 and 2. Finally, the current study did not examine activity within this region a day or longer after extinction had occurred. Based on Pavlovian conditioning studies demonstrating differences in regions of mOFC activity between early and late extinction learning (Milad et al., 2007; Phelps et al., 2004), we predicted we would observe phase related differences in mOFC. Our results did not confirm this prediction. Instead, we observed main effects of phase (early vs. late) in a network of regions implicated in attention. While this contrast replicated main effects of extinction in the frontal pole and ACC, no significant phase interactions were present in these regions. The lack of findings may relate to differences in time course contrasts between this study and prior work. In the present study, we examined phase effects by comparing the first half of trials after the contingency change to the second half of trials. In prior Pavlovian studies, early extinction learning took place immediately after extinction, but late learning occurred several hours (Milad et al., 2007) to a day later (Phelps et al., 2004). We conclude that the lack of a significant temporal separation between the early and late phases as assessed in this study may underlie the lack of phase related findings in frontopolar cortex, ACC, and subgenual cortex.

We suggest it is worth considering an alternative hypothesis regarding the role of this region in operant extinction. Recent work in humans using an instrumental devaluation paradigm has demonstrated that an adjacent region of caudal mOFC displays reduced activity during operant responses linked to a devalued reinforcer compared to a valued reinforcer (Valentin et al., 2007) . It has also been argued that this region is involved in the representation of outcome information that is used to guide behavior (Blair et al., 2006; O'Doherty, 2007). It is possible that the reduced activity of this region during successful extinction trials reflects the suppression of the representation of outcome information relating to the old stimulus-reinforcement association that is no longer valid. Interesting in this regard is the inverse relationship, revealed by the functional connectivity analysis, between frontopolar OFC and this caudal region. By suppressing the original representations of stimulus-reinforcement associations in the amygdala when they are no longer consistent with the current contingency, frontopolar OFC may give rise to a suppression of outcome information in caudal mOFC. This model of interaction is supported by anatomical studies in non-human primates which demonstrate reciprocal connections between rostral mOFC and frontopolar regions with caudomedial OFC and rostral ACC (Barbas and Pandya, 1989; Schmahmann and Pandya, 2006) , and the amygdala (Amaral and Price, 1984; Moran et al., 1987). It should be noted that the anatomic connections between these regions are reciprocal and functional connectivity analysis cannot determine the direction of modulation between regions. We suggest modulation is more likely to involve active regions suppressing deactivated regions. However, it is also possible, though we suggest less likely, that rostral ACC, caudomedial OFC, or the amygdala could inhibit frontopolar cortex, and that decreased activity in ACC and caudomedial OFC would result in less inhibition of frontopolar cortex.

The neural correlates of operant extinction learning in humans identified in the present study complement those of seminal work in animal models and Pavlovian extinction, and newly highlight dissociable functional roles of specific regions of frontopolar and caudomedial OFC as mediators of this type of adaptive behavior in humans. Further characterization of the neural mechanisms of operant extinction learning in humans may help to elucidate the pathophysiology of disorders such as addiction, psychopathy, and frontotemporal dementia which feature devastating impairments in the extinction of instrumental behavior.

Supplementary Material

01
02

Table 1.

Regions Demonstrating Differential BOLD Responses during Successful Extinction

Region L/R BA x y z cluster size mm3
p<0.005
Successful extinction > control
Medial frontal gyrus R 10 15 69 −5 1278
Medial frontal gyrus L 11 −16 62 −9 267
Middle frontal gyrus L 11 −30 42 −11 263
Successful extinction < control
Anterior cingulate cortex R 14 35 −5 860
Medial frontal gyrus/rectal gyrus R 25/47 12 36 −12 78*
Open in a new tab

MNI coordinates of peak activation. Regions and Brodmann area (BA) according to Talairach Daemon Atlas. All regions significant at p<0.005 corrected for multiple comparisons except *.

Acknowledgments

Funding

This research was funded by the National Institute of Mental Health Intramural Research Program.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Amaral DG, Price JL. Amygdalo-cortical projections in the monkey (Macaca fascicularis) J Comp Neurol. 1984;230:465–496. doi: 10.1002/cne.902300402. [DOI] [PubMed] [Google Scholar]
  2. Barbas H, Pandya DN. Architecture and intrinsic connections of the prefrontal cortex in the rhesus monkey. J Comp Neurol. 1989;286:353–375. doi: 10.1002/cne.902860306. [DOI] [PubMed] [Google Scholar]
  3. Baxter MG, Murray EA. The amygdala and reward. Nat Rev Neurosci. 2002;3:563–573. doi: 10.1038/nrn875. [DOI] [PubMed] [Google Scholar]
  4. Berlin HA, Rolls ET, Kischka U. Impulsivity, time perception, emotion and reinforcement sensitivity in patients with orbitofrontal cortex lesions. Brain. 2004;127:1108–1126. doi: 10.1093/brain/awh135. [DOI] [PubMed] [Google Scholar]
  5. Blair K, Marsh AA, Morton J, Vythilingam M, Jones M, Mondillo K, Pine DC, Drevets WC, Blair JR. Choosing the lesser of two evils, the better of two goods: specifying the roles of ventromedial prefrontal cortex and dorsal anterior cingulated in object choice. J Neurosci. 2006;26:11379–11386. doi: 10.1523/JNEUROSCI.1640-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Blair KS, Smith BW, Mitchell DG, Morton J, Vythilingam M, Pessoa L, Fridberg D, Zametkin A, Sturman D, Nelson EE, Drevets WC, Pine DS, Martin A, Blair RJ. Modulation of emotion by cognition and cognition by emotion. Neuroimage. 2007;35:430–440. doi: 10.1016/j.neuroimage.2006.11.048. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Budhani S, Marsh AA, Pine DS, Blair RJ. Neural correlates of response reversal: considering acquisition. Neuroimage. 2007;34:1754–1765. doi: 10.1016/j.neuroimage.2006.08.060. [DOI] [PubMed] [Google Scholar]
  8. Budhani S, Marsh AA, Pine DS, Blair RJR. Neural correlates of response reversal: Considering acquisition. Neuroimage. 2006 doi: 10.1016/j.neuroimage.2006.08.060. In press. [DOI] [PubMed] [Google Scholar]
  9. Butter CM. Perseveration in Extinction and in Discrimination Reversal Tasks Following Selective Frontal Ablations in Macaca Mulatta. Physiology and Behavior. 1969;4:163–171. [Google Scholar]
  10. Chudasama Y, Robbins TW. Dissociable contributions of the orbitofrontal and infralimbic cortex to pavlovian autoshaping and discrimination reversal learning: further evidence for the functional heterogeneity of the rodent frontal cortex. J Neurosci. 2003;23:8771–8780. doi: 10.1523/JNEUROSCI.23-25-08771.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Cools R, Clark L, Owen AM, Robbins TW. Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J Neurosci. 2002;22:4563–4567. doi: 10.1523/JNEUROSCI.22-11-04563.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Cox RW, Hyde JS. Software tools for analysis and visualization of fMRI data. NMR Biomed. 1997;10:171–178. doi: 10.1002/(sici)1099-1492(199706/08)10:4/5<171::aid-nbm453>3.0.co;2-l. [DOI] [PubMed] [Google Scholar]
  13. Ellson DG. Quantitative studies of the interaction of simple habits: I. Recovery from specific and generalized effcts of extinction. Journal of Experimental Psychology. 1938;23:349–355. [Google Scholar]
  14. Gallagher M, McMahan RW, Schoenbaum G. Orbitofrontal cortex and representation of incentive value in associative learning. J Neurosci. 1999;19:6610–6614. doi: 10.1523/JNEUROSCI.19-15-06610.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Gottfried JA, Dolan RJ. Human orbitofrontal cortex mediates extinction learning while accessing conditioned representations of value. Nat Neurosci. 2004;7:1144–1152. doi: 10.1038/nn1314. [DOI] [PubMed] [Google Scholar]
  16. Izquierdo A, Murray EA. Opposing effects of amygdala and orbital prefrontal cortex lesions on the extinction of instrumental responding in macaque monkeys. Eur J Neurosci. 2005;22:2341–2346. doi: 10.1111/j.1460-9568.2005.04434.x. [DOI] [PubMed] [Google Scholar]
  17. Izquierdo A, Murray EA. Selective bilateral amygdala lesions in rhesus monkeys fail to disrupt object reversal learning. J Neurosci. 2007;27:1054–1062. doi: 10.1523/JNEUROSCI.3616-06.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Izquierdo A, Suda RK, Murray EA. Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J Neurosci. 2004;24:7540–7548. doi: 10.1523/JNEUROSCI.1921-04.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Kosson DS, Budhani S, Nakic M, Chen G, Saad ZS, Vythilingam M, Pine DS, Blair RJ. The role of the amygdala and rostral anterior cingulate in encoding expected outcomes during learning. Neuroimage. 2006;29:1161–1172. doi: 10.1016/j.neuroimage.2005.07.060. [DOI] [PubMed] [Google Scholar]
  20. LaBar KS, Gatenby JC, Gore JC, LeDoux JE, Phelps EA. Human amygdala activation during conditioned fear acquisition and extinction: a mixed-trial fMRI study. Neuron. 1998;20:937–945. doi: 10.1016/s0896-6273(00)80475-4. [DOI] [PubMed] [Google Scholar]
  21. Levesque J, Eugene F, Joanette Y, Paquette V, Mensour B, Beaudoin G, Leroux JM, Bourgouin P, Beauregard M. Neural circuitry underlying voluntary suppression of sadness. Biol Psychiatry. 2003;53:502–510. doi: 10.1016/s0006-3223(02)01817-6. [DOI] [PubMed] [Google Scholar]
  22. Lykken DT. A study of anxiety in the sociopathic personality. J Abnorm Psychol. 1957;55:6–10. doi: 10.1037/h0047232. [DOI] [PubMed] [Google Scholar]
  23. McDannald MA, Saddoris MP, Gallagher M, Holland PC. Lesions of orbitofrontal cortex impair rats' differential outcome expectancy learning but not conditioned stimulus-potentiated feeding. J Neurosci. 2005;25:4626–4632. doi: 10.1523/JNEUROSCI.5301-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Milad MR, Quinn BT, Pitman RK, Orr SP, Fischl B, Rauch SL. Thickness of ventromedial prefrontal cortex in humans is correlated with extinction memory. Proc Natl Acad Sci U S A. 2005;102:10706–10711. doi: 10.1073/pnas.0502441102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Milad MR, Quirk GJ. Neurons in medial prefrontal cortex signal memory for fear extinction. Nature. 2002;420:70–74. doi: 10.1038/nature01138. [DOI] [PubMed] [Google Scholar]
  26. Milad MR, Wright CI, Orr SP, Pitman RK, Quirk GJ, Rauch SL. Recall of fear extinction in humans activates the ventromedial prefrontal cortex and hippocampus in concert. Biol Psychiatry. 2007;62:446–454. doi: 10.1016/j.biopsych.2006.10.011. [DOI] [PubMed] [Google Scholar]
  27. Mitchell DG, Luo Q, Mondillo K, Vythilingam M, Finger EC, Blair RJ. The interference of operant task performance by emotional distracters: An antagonistic relationship between the amygdala and frontoparietal cortices. Neuroimage. 2007 doi: 10.1016/j.neuroimage.2007.08.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Mitchell DG, Luo Q, Mondillo K, Vythilingam M, Finger EC, Blair RJ. The interference of operant task performance by emotional distracters: An antagonistic relationship between the amygdala and frontoparietal cortices. Neuroimage. 2008;40:859–868. doi: 10.1016/j.neuroimage.2007.08.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Moran MA, Mufson EJ, Mesulam MM. Neural inputs into the temporopolar cortex of the rhesus monkey. J Comp Neurol. 1987;256:88–103. doi: 10.1002/cne.902560108. [DOI] [PubMed] [Google Scholar]
  30. Newman JP, Kosson DS. Passive avoidance learning in psychopathic and nonpsychopathic offenders. J Abnorm Psychol. 1986;95:252–256. [PubMed] [Google Scholar]
  31. O'Doherty J, Critchley H, Deichmann R, Dolan RJ. Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J Neurosci. 2003;23:7931–7939. doi: 10.1523/JNEUROSCI.23-21-07931.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. O'Doherty JP. Lights, camembert, action! The role of human orbitofrontal cortex in encoding stimuli, rewards, and choices. Ann N Y Acad Sci. 2007;1121:254–272. doi: 10.1196/annals.1401.036. [DOI] [PubMed] [Google Scholar]
  33. Ohira H, Nomura M, Ichikawa N, Isowa T, Iidaka T, Sato A, Fukuyama S, Nakajima T, Yamada J. Association of neural and physiological responses during voluntary emotion suppression. Neuroimage. 2006;29:721–733. doi: 10.1016/j.neuroimage.2005.08.047. [DOI] [PubMed] [Google Scholar]
  34. Ostlund SB, Balleine BW. Lesions of medial prefrontal cortex disrupt the acquisition but not the expression of goal-directed learning. J Neurosci. 2005;25:7763–7770. doi: 10.1523/JNEUROSCI.1921-05.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Pavlov IP. Conditioned Reflexes. London: Oxford University Press; 1927. [Google Scholar]
  36. Phelps EA, Delgado MR, Nearing KI, LeDoux JE. Extinction learning in humans: role of the amygdala and vmPFC. Neuron. 2004;43:897–905. doi: 10.1016/j.neuron.2004.08.042. [DOI] [PubMed] [Google Scholar]
  37. Pickens CL, Saddoris MP, Gallagher M, Holland PC. Orbitofrontal lesions impair use of cue-outcome associations in a devaluation task. Behav Neurosci. 2005;119:317–322. doi: 10.1037/0735-7044.119.1.317. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Quirk GJ, Russo GK, Barron JL, Lebron K. The role of ventromedial prefrontal cortex in the recovery of extinguished fear. J Neurosci. 2000;20:6225–6231. doi: 10.1523/JNEUROSCI.20-16-06225.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Ragozzino ME. The contribution of the medial prefrontal cortex, orbitofrontal cortex, and dorsomedial striatum to behavioral flexibility. Ann N Y Acad Sci. 2007;1121:355–375. doi: 10.1196/annals.1401.013. [DOI] [PubMed] [Google Scholar]
  40. Rauch SL, Milad MR, Orr SP, Quinn BT, Fischl B, Pitman RK. Orbitofrontal thickness, retention of fear extinction, and extraversion. Neuroreport. 2005;16:1909–1912. doi: 10.1097/01.wnr.0000186599.66243.50. [DOI] [PubMed] [Google Scholar]
  41. Rescorla RA, Wagner ARA, Skucy JC. The effect of response-independent reinforcers during extinction. Journal of Comparative and Physiological Psychology. 1969;67:381–389. [Google Scholar]
  42. Roesch MR, Stalnaker TA, Schoenbaum G. Associative Encoding in Anterior Piriform Cortex versus Orbitofrontal Cortex during Odor Discrimination and Reversal Learning. Cereb Cortex. 2006 doi: 10.1093/cercor/bhk009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Saddoris MP, Gallagher M, Schoenbaum G. Rapid associative encoding in basolateral amygdala depends on connections with orbitofrontal cortex. Neuron. 2005;46:321–331. doi: 10.1016/j.neuron.2005.02.018. [DOI] [PubMed] [Google Scholar]
  44. Schmahmann JD, Pandya DN. Fiber pathways of the brain. Oxford ; New York: Oxford University Press; 2006. [Google Scholar]
  45. Schoenbaum G, Chiba AA, Gallagher M. Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat Neurosci. 1998;1:155–159. doi: 10.1038/407. [DOI] [PubMed] [Google Scholar]
  46. Schoenbaum G, Chiba AA, Gallagher M. Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. J Neurosci. 1999;19:1876–1884. doi: 10.1523/JNEUROSCI.19-05-01876.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Schoenbaum G, Chiba AA, Gallagher M. Changes in functional connectivity in orbitofrontal cortex and basolateral amygdala during learning and reversal training. J Neurosci. 2000;20:5179–5189. doi: 10.1523/JNEUROSCI.20-13-05179.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Schoenbaum G, Eichenbaum H. Information coding in the rodent prefrontal cortex. II. Ensemble activity in orbitofrontal cortex. J Neurophysiol. 1995;74:751–762. doi: 10.1152/jn.1995.74.2.751. [DOI] [PubMed] [Google Scholar]
  49. Schoenbaum G, Nugent SL, Saddoris MP, Setlow B. Orbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations. Neuroreport. 2002;13:885–890. doi: 10.1097/00001756-200205070-00030. [DOI] [PubMed] [Google Scholar]
  50. Schoenbaum G, Roesch M. Orbitofrontal cortex, associative learning, and expectancies. Neuron. 2005;47:633–636. doi: 10.1016/j.neuron.2005.07.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Schoenbaum G, Saddoris MP, Stalnaker TA. Reconciling the roles of orbitofrontal cortex in reversal learning and the encoding of outcome expectancies. Ann N Y Acad Sci. 2007;1121:320–335. doi: 10.1196/annals.1401.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Schoenbaum G, Setlow B, Nugent SL, Saddoris MP, Gallagher M. Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals. Learn Mem. 2003a;10:129–140. doi: 10.1101/lm.55203. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Schoenbaum G, Setlow B, Saddoris MP, Gallagher M. Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron. 2003b;39:855–867. doi: 10.1016/s0896-6273(03)00474-4. [DOI] [PubMed] [Google Scholar]
  54. Setlow B, Schoenbaum G, Gallagher M. Neural encoding in ventral striatum during olfactory discrimination learning. Neuron. 2003;38:625–636. doi: 10.1016/s0896-6273(03)00264-2. [DOI] [PubMed] [Google Scholar]
  55. Skinner BF. The Behavior of Organisms. Acton: Copley Publishing Group; 1938. [Google Scholar]
  56. Talairach J, Tournoux P. Co-planar stereotaxic atlas of the human brain. Stuttgart: Thieme; 1988. [Google Scholar]
  57. Tremblay L, Schultz W. Reward-related neuronal activity during go-nogo task performance in primate orbitofrontal cortex. J Neurophysiol. 2000;83:1864–1876. doi: 10.1152/jn.2000.83.4.1864. [DOI] [PubMed] [Google Scholar]
  58. Valentin VV, Dickinson A, O'Doherty JP. Determining the neural substrates of goal-directed learning in the human brain. J Neurosci. 2007;27:4019–4026. doi: 10.1523/JNEUROSCI.0564-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

01
NIHMS79240-supplement-01.pdf (59.6KB, pdf)
02
NIHMS79240-supplement-02.pdf (55.5KB, pdf)

RESOURCES