Skip to main content
PMC home page
Journal of the Experimental Analysis of Behavior logoLink to Journal of the Experimental Analysis of Behavior
. 2005 Nov;84(3):505–519. doi: 10.1901/jeab.2005.89-04

Integrating Functional Neuroimaging and Human Operant Research: Brain Activation Correlated with Presentation of Discriminative Stimuli.

Michael W Schlund 2,✉, Michael F Cataldo 1
PMCID: PMC1389778  PMID: 16596977

Abstract

Results of numerous human imaging studies and nonhuman neurophysiological studies on “reward” highlight a role for frontal, striatal, and thalamic regions in operant learning. By integrating operant and functional neuroimaging methodologies, the present investigation examined brain activation to two types of discriminative stimuli correlated with different contingencies. Prior to neuroimaging, 10 adult human subjects completed operant discrimination training in which money was delivered following button pressing (press-money contingency) in the presence of one set of discriminative stimuli, and termination of trials followed not responding (no response-next trial contingency) in the presence of a second set of discriminative stimuli. After operant training, subjects were instructed to memorize a third set of control stimuli unassociated with contingencies. Several hours after training, functional magnetic resonance imaging was performed while subjects viewed discriminative and control stimuli that were presented individually for 1,500 ms per trial, with stimulus presentations occurring, on average, every 6 s. Activation was found in frontal and striatal brain regions to both sets of discriminative stimuli relative to control stimuli. In addition, exploratory analyses highlighted activation differences between discriminative stimuli. The results demonstrate the utility of coupling operant and imaging technologies for investigating the neural substrates of operant learning in humans.

Keywords: neuroimaging, discriminative stimulus, striatum, caudate, observing, human

Magnetic resonance imaging (MRI) provides a noninvasive, minimal-risk technique for studying a variety of brain-behavior relations and brain-related diseases. The range of uses of MRI technology includes quantifying: (a) the size and position of discrete brain structures (i.e., structural MRI), (b) changes in activation of specific brain regions under differing stimulus and/or performance conditions (i.e., functional MRI or fMRI), (c) certain biochemical changes related to neurotransmitters (MR spectroscopy), and (d) the location and direction of neural activity along the fiber tracts that connect brain structures and regions (filter tract mapping). Further, this technology is expanding exponentially, with new techniques and uses being reported with almost every journal cycle.

Studies involving neuroimaging and the Experimental Analysis of Behavior (EAB) can make at least two important contributions to the advancement of behavioral science. The first is a better understanding of the neurobiology of operant learning. Studies on the relation between operant learning and brain function have never been considered an unimportant or unnecessary pursuit, just difficult. Indeed, these relations are particularly important to understanding learning deficits as well as understanding the action of different therapeutic approaches, such as drug versus behavior therapy.

A second contribution is the degree of precision that the EAB can offer fMRI research. Despite its rapid development, fMRI research is not without its own unique methodological concerns, some of which stem from a lack of precision in arranging stimulus conditions, response repertoires, and the like. Although not always the case, most experiments of brain function involve the comparison of neural activation during differing conditions of stimulus presentations, responding, or both. Thus a science such as EAB that is based on and has developed because of considerable rigor in arranging environmental conditions to precisely control behavior can make a significant contribution to imaging research on operant learning, particularly on discriminative stimulus control of human behavior. To this end, the present investigation coupled methods typical of the EAB with blood-oxygen-level-dependent functional magnetic resonance imaging (BOLD fMRI) to examine frontal, striatal, and thalamic activation correlated with presentation of discriminative stimuli.

The reciprocal connections between the frontal lobe, striatum, and thalamus appear to play a role in human and nonhuman learning (Groenewegen, Wright, Beijer, & Voorn, 1999; Lynd-Balta & Haber, 1994; Ongur & Price, 2000; Wise, Murray, & Gerfew, 1996). In frontal regions, humans with lesions to the ventromedial cortex and diffuse frontal injury show impairments in discriminating contingencies (Bechara, Damasio, Damasio, & Anderson, 1994; Schlund, 2002; Schlund & Pace, 2000). In the striatum, dysfunction associated with human neurodegenerative conditions such as Huntington's or Parkinson's disease also negatively affects learning (Jackson, Jackson, Harrison, Henderson, & Kennard, 1995; Knopman & Nissen, 1991; Swenson & Butters, 1996). Results of numerous lesion and neurophysiological studies with nonhuman subjects implicate frontal-striatal connections in learning and coding reward and reward-associated stimuli (Apicella, Ljungberg, Scarnati, & Schultz, 1991; Hikosaka, Sakamoto, & Usui, 1989; Kawagoe, Takikawa, & Hikosaka, 1998; Robbins & Everitt, 1996; Schultz, Tremblay, & Hollerman, 1998; Tremblay & Schultz, 2000b). For example, Tremblay and Schultz (2000a) used a modified go/no-go task with monkeys to investigate responses of neurons in the orbitofrontal cortex and caudate to different types of discriminative stimuli. Reinforcement contingencies were used to bring responding under the control of three different discriminative stimuli, each correlated with a different contingency: respond-reinforcer, no respond-reinforcer, and respond-no reinforcer (in this case, responding produced a conditioned stimulus followed by termination of the trial). Response accuracy was high and orbitofrontal and caudate neural activity was consistently greater during the presentation of discriminative stimuli correlated with reinforcement.

The emergence of methodological differences between cognitive neuroscience imaging studies on reward and human and animal operant studies on reinforcement are relevant here. For example, results of many human imaging studies on reward report activation in frontal and striatal regions, but relatively few investigations arrange reinforcement contingencies to control behavior (e.g., O'Doherty, Kringelbach, Rolls, Hornak, & Andrews, 2001; Rogers et al., 1999; Rogers et al., 2004). By comparison to nonhuman studies, many human imaging studies rely on experimenter instructions and do not employ reinforcement contingencies or discriminative stimuli. Thus behavior appears primarily under the control of instructions that are often structured, implicitly or explicitly, to generate the “perception” of contingencies (e.g., Breiter, Aharon, Kahneman, Dale, & Shizgal, 2001; Delgado, Nystrom, Fissell, Noll, & Fiez, 2000; Knutson, Westdorp, Kaiser, & Hommer, 2000; Tricomi, Delgado, & Fiez, 2004). These differences involving contingencies and verbal instructions suggest that the variables controlling formally similar human and animal behavior may differ functionally, which may in turn recruit different neural systems.

At a broader level, many human imaging studies often interpret striatal activation to reward (i.e., money delivery) as reflecting “reward processing.” Yet similar results recently have been reported to other stimuli (often referred to as “nonrewards”). For instance, caudate activation has been observed when experimenter instructions emphasize responding to particular stimulus targets (ones unassociated with money gain/loss; Zink, Pagnoni, Martin, Dhamala, & Berns, 2003); during the presentation of cues associated with avoidable shock in an avoidance paradigm (Jensen et al., 2003); presentation of noxious thermal stimuli (Becerra, Brieter, Wise, Gonzalez, & Borsook, 2001); and presentation of aversive auditory stimuli (Zald & Pardo, 2002). Such results have fueled support for the notion that striatal activity (including its major dopaminergic inputs) reflects the “salience” or “behavioral significance” of stimuli (Horvitz, 2000; Kapur, 2003), an idea based on results of nonhuman research showing striatal activity to a wide range of events, including appetitive, aversive, high-intensity, and novel stimuli. This view suggests that simply viewing discriminative stimuli should also produce increases in striatal activation, especially in the caudate.

The primary aims of the present investigation were to demonstrate an efficient, yet rigorous, approach for integrating operant and imaging methodologies to map brain activation correlated with presentation of discriminative stimuli. In contrast with typical human imaging studies, we chose to establish the learning histories associated with discriminative stimuli, rather than employ verbal instructions to control behavior or use stimuli with a presumed learning history (e.g., words, pictures, etc.). Our design allowed us to compare brain activation to discriminative stimuli: (a) established through operant contingencies versus verbal instructions, (b) correlated with contingencies involving the delivery of a positive reinforcer (money) versus no tangible reinforcer (trial termination), and (c) correlated with reinforcer magnitudes differing logarithmically ($.05, $.50, and $5.00).

In choosing a research strategy, we also were concerned with designing an approach that would minimize imaging resources (time and cost) and eventually be suitable for studying populations with stimulus preferences and learning difficulties of interest (e.g., individuals with autism, mental retardation, learning disabilities, etc.). That is, we chose an approach that allowed subjects first to be trained until they met a discrimination criterion and then be imaged, rather than have the training occur during imaging. This involved a passive viewing paradigm in which subjects viewed discriminative stimuli under imaging conditions without responding or reinforcement. Benefits of the passive approach include minimizing activation correlated with learning (Haruno et al., 2004), response-reward interactions (Elliott, Newman, Longe, & Deakin, 2004), upcoming money gain (Breiter et al., 2001), and contextual factors, such as recent runs of money gains (Elliott, Friston, & Dolan, 2000). Because “activation” in fMRI research reflects BOLD signal differences between experimental and control conditions, our control condition for discriminative stimuli previously correlated with contingencies was a set of instructed control stimuli not associated with any programmed contingencies and that subjects had exposure to prior to imaging to eliminate activation associated with stimulus novelty.

Method

Subjects

Ten healthy, right-handed males (n  =  3) and females (n  =  7) participated in discrimination training that lasted approximately 2 hr and a 1-hr fMRI session that occurred 2 to 3 hr after training was completed. Subjects reported being between 18 and 50 years of age, right handed, free of medications affecting the central nervous system or the autonomic system for at least 2 weeks, and without a personal history of psychiatric disorder or a psychiatric history in first-degree relatives.

Discrimination Training

Apparatus

During discrimination training outside of the fMRI scanner, subjects responded by pressing a spacebar on a standard computer keyboard connected to a desktop computer. QBASIC© software was used to program stimulus presentation and record data. Training took place in a quiet room with the subject seated in front of the computer and keyboard.

Stimuli

Stimuli consisted of nine Greek letters (α, Π, Σ, ∩, μ, λ, δ, β, Ω), approximately 7.6 cm by 7.6 cm in size. All stimuli used during discrimination training and as instructed control stimuli were randomly assigned and counterbalanced across subjects.

Instructions

Instructions emphasized paying attention to stimuli, but did not provide subjects with specific information about programmed contingencies. During training, subjects were seated in front of a computer keyboard and monitor and were provided the following instructions. The experimenter initially read aloud the instructions printed on the computer screen while the subject followed along:

In front of you is a spacebar. Pressing the spacebar will <sometimes> produce money. It is up to you to learn when it is best to press and not to press. Earn as much money as you can. The computer will prompt you each time you earn money. Pay careful attention to the stimuli you see because you will see these same stimuli later during imaging. Any questions?

Operant Training

Training consisted of two phases. Both phases employed contingencies to establish stimulus control over responding by two 3-member stimulus sets. During the first phase of training, two stimuli (Greek letters) were used and their presentation alternated strictly across trials. Each stimulus was correlated with a different contingency. For the stimulus correlated with a no press-next trial contingency, a period of 10 s without a response in the presence of the stimulus was required to terminate the trial and initiate the next trial. For stimuli correlated with the press-money contingency, reinforcers (the prompt “Earn 5 Cents”) were delivered according to a variable-ratio 3 reinforcement schedule for responding in the presence of the stimulus. After earning five reinforcers, the trial was terminated. The intertrial interval was approximately 100 ms. Each session lasted 20 trials (10 trials with each stimulus). Training continued until the total number of responses emitted in the presence of stimuli correlated with the press-money contingency was greater than 90% of the total number of responses emitted during a session. The first phase of training ended once the 90% criterion was met.

The second phase of training modeled the first phase of training and was completed immediately after phase one was completed. The two stimuli from phase one and four additional stimuli were presented, each on separate trials, and in a randomized order during a session. Three discriminative stimuli were correlated with the no press-next trial contingency. The remaining three discriminative stimuli were correlated with the press-money contingency and different amounts of money ($0.05, $0.50, and $5.00). Each session lasted 42 trials (seven presentations of each discriminative stimulus). Again, training continued until the total number of responses emitted in the presence of stimuli correlated with the press-money contingency was greater than 90% of the total number of responses emitted during a session. The second phase of training ended once the 90% criterion was met.

Instructed Training

This procedure employed instructions to establish stimulus control by a third set of three stimuli. Immediately after completing operant training, three stimuli were printed on the computer screen and subjects were told: “Please memorize these stimuli over the next 6 min. Please pay careful attention to these stimuli because you will see these same stimuli later during imaging.” Instructed control stimuli served as control comparisons for imaging analyses. Control stimuli differed from discriminative stimuli by not being correlated with any programmed contingency. This procedure also was necessary to provide subjects with preexposure to stimuli. Without it, the novelty of control stimuli would increase activation during imaging and confound results.

Neuroimaging

Neuroimaging occurred approximately 2 to 3 hr after instructed training was completed.

Stimulus Recognition Task

To ensure that discriminative and instructed control stimuli retained their stimulus-control properties before imaging occurred, subjects were required to complete a pencil and paper stimulus recognition task. Instructions stated that subjects were to circle all stimuli that were seen during training. The task was organized such that training stimuli (six discriminative stimuli and three control stimuli) were positioned randomly in a field containing an additional nine distractor stimuli (also Greek symbols).

Imaging Apparatus and Parameters

Functional MRI images were obtained on a 1.5 T Philips® MRI scanner. E-prime® software controlled stimulus presentation and recorded timing data. Task instructions and stimuli were presented on a rear screen monitor viewed through a mirror anchored to a standard head coil. After an initial series of sagittal T1-weighted localizers, a set of oblique T1-weighted images, angled parallel to the intercommissural line, were gathered. The fMRI data were acquired at the same slice locations. The T1 parameters were repetition time (TR) of 500 ms and an estimation time (TE) of 11 ms. Functional MRI data were gathered using a single-shot echo planar imaging (EPI) sequence for data acquisition, with a TR of 2 s, a TE of 50 ms, and a 90° flip angle. The matrix size was 64 × 64 and the field of view 24 cm, yielding voxels measuring 3.75 × 3.75 mm in plane. Using these parameters, 20 contiguous 7-mm thick sections were obtained angled parallel to the intercommissural line.

Instructions and Task

Subjects were placed in the scanner and handed a button box. The following instructions were printed on a viewing screen located in the scanner room and read aloud by the experimenter: “During your training you learned something about several different stimuli. During this task, you will look at stimuli presented on your monitor. We also would like you to press the hand-held button each time you see a white * (star/asterisk) on the screen. OK? This task will last about 9 minutes.”

During imaging, subjects viewed six discriminative stimuli, three control stimuli, and a white asterisk target, presented in a randomized order (i.e., an event-related design). Press-money stimuli, no press-next trial stimuli, and control stimuli were each presented on 21 trials (for a total of 63 trials). The target asterisk was presented on seven trials. Each trial consisted of a “+” sign (500 ms), a stimulus presentation (1,500 ms), and a blank computer screen. Trial duration varied from 5.0 s to 7.0 s (mean, 6.0 s; “jittered” in 200 ms steps). Jittering trial durations is a conventional neuroimaging approach that ensures that image acquisition, here every 2 s (TR  =  2 s), occurs at different points during trials to improve sampling of the hemodynamic response.

Analysis

For a subject's imaging data to be included in the analysis, head movement was limited to less than 2 mm. Functional EPI images were first reconstructed from k-space to image space for further processing. All preprocessing and data analysis were performed using statistical parametric mapping software, version 2 (SPM2; Wellcome Department of Cognitive Neurology, London, UK). EPI images were slice-timing corrected to adjust for the lag between slices during each TR, corrected for head motion during scanning, and normalized to a standard template brain from the Montreal Neurological Institute (MNI) to get all participants into the same space (Friston, Ashburner, et al., 1995). After normalization, voxels were resampled with a 2 × 2 × 2 mm voxel size. EPI images then were spatially smoothed using a 6 × 6 × 8 mm full-width-half-maximum (FWHM) Gaussian kernel. High-pass filtering was applied to the time series of EPI images to remove the low-frequency drift in EPI signal and then subjected to analysis.

Imaging analyses involve using t tests to identify voxels that show significantly greater activation (>) in one condition relative to a second condition, referred to as a “contrast.” In the present investigation, four contrasts were performed: (a) press-money > control, (b) no response-next trial > control, (c) press-money > no response-next trial, and (d) no response-next trial > press-money. To identify activation, a conventional two-level analysis procedure was employed.

At the first level, individual-subject models are constructed in which a linear regression analysis is performed between the observed event-related EPI signals and onset times of stimuli (regressors) to identify patterns of activation (Friston, Holmes, et al., 1995). This produces a beta weight (regression coefficient) for each voxel (with voxels measuring 2 × 2 × 2 mm and a whole brain consisting of ∼194, 858 voxels). Voxel beta weights were based on aggregated fMRI signal changes correlated with presentations, or onset times, of press-money stimuli, no response-next trial stimuli, instructed control stimuli, and the target asterisk stimulus. Figure 1A provides an illustration of how each voxel beta weight approximates percentage change in fMRI signal relative to a global fixed constant, which is an average value of all time points in a session. Using the beta weights from the model, contrasts involve calculating a beta difference at each voxel for each subject and for each contrast [e.g., for contrast (a): beta difference  =  press-money beta–control beta].

Figure 1. Illustration of the steps used to analyze BOLD fMRI data in SPM2.

Figure 1

Open in a new tab

(A) shows the relation between a hypothetical BOLD fMRI signal change at a brain voxel for presentations of a discriminative stimulus (left) and a control stimulus (right). For each stimulus type, regression analyses produce different beta weights that approximate signal change. (B) shows beta-weight differences at the voxel for a hypothetical group of subjects when the control stimulus beta is subtracted from the discriminative stimulus beta. If the group difference is significant, the voxel is then highlighted on an anatomical image for display.

At the second level, separate whole-brain “random effect” analyses were performed for each contrast (a through d) (Holmes & Friston, 1998). An illustration of this step appears in Figure 1B. For each voxel, a group mean beta difference was calculated and one-sample t tests were employed to determine whether the mean beta difference was significantly greater than 0. Statistical thresholds of p < .005 and 10 contiguous voxels were used to find “activation.” This approach excludes from results nonsignificant voxels and all voxel clusters containing less than 10 contiguous voxels. As shown in Figure 1B, voxels that survive the thresholds are plotted on a statistical parametric map, which simply highlights brain regions with a mean beta difference significantly greater than 0. Due to a design miscalculation, each stimulus associated with money was not presented with sufficient frequency to perform group analyses of magnitude effects (i.e., increases in activation correlated with increases in money). Consequently, activation reported to presentations of press-money discriminative stimuli reflect collapsing across magnitude.

Our a priori primary region of interest was the caudate in addition to frontal regions, the anterior cingulate, lentiform nucleus, putamen, thalamus, and fusiform gyrus. A small volume correction was employed to interrogate the caudate head using an anatomically defined mask created with the Wake Forest University PickAtlas SPM2 plug-in (Maldjian, Laurienti, Burdette, & Kraft, 2003). The location of voxels with significant activation was summarized by their local maxima separated by at least 8 mm, and by converting the maxima coordinates from MNI to Talairach coordinate space using the formulas provided by Matthew Brett (2002). These coordinates were finally assigned neuroanatomic labels using the Talairach Daemon (2003).

Results

Behavioral

Operant and instructed training procedures each successfully established stimulus control over behavior. Figure 2 plots total responses emitted under each contingency for each subject during both training phases. Results highlight behavioral changes across sessions and the emergence of differential discriminative control. By the end of each training phase, each subject emitted more than 90% of the total responses in the presence of discriminative stimuli correlated with the press-money contingency (thus less than 10% of the total responses were emitted in the presence of discriminative stimuli correlated with the no press-next trial contingency). In addition, each subject correctly identified all training (discriminative and instructed) stimuli on the recognition task completed prior to neuroimaging. Lastly, during neuroimaging each subject responded to each target presentation (the asterisk).

Figure 2. Total responses emitted during each training session prior to neuroimaging.

Figure 2

Open in a new tab

Responding was reinforced with money in the presence of press-money discriminative stimuli (filled circles). Trials ended after a 10-s period of no responding in the presence of no response-next trial discriminative stimuli (open circles). The dashed line separates the two training phases.

Imaging

Figure 3 presents statistical parametric maps showing activation for the press-money > control contrast (red) and no press-next trial > control contrast (green). Overlapping activation appears in yellow. The insert in Figure 3 shows bilateral activation in the caudate head to discriminative stimuli correlated with the no press-next trial contingency (Left: 86 voxels centered at −10, 10, 1; p(corrected cluster)  =  .006, z  =  3.76; Right: 13 voxels centered at 10, 6, 3; p(corrected cluster)  =  .094, z  =  2.99), and in the left caudate head to discriminative stimuli correlated with the press-money contingency (Left: 43 voxels centered at −12, 10, 3; p(corrected cluster)  =  .02, z  =  3.35). For both contrasts, individual subject beta differences are plotted in Figure 4. Statistical parametric maps shown in Figure 3 also highlight activation to press-money discriminative stimuli in the left medial frontal gyrus, putamen, and thalamus and in the right precentral and inferior frontal gyri, putamen, anterior cingulated, and fusiform gyrus. Similarly, maps show activation to no press-next trial discriminative stimuli bilaterally in superior, medial, and middle frontal gyri, and the right putamen.

Figure 3. Results of whole-brain group random effect analyses.

Figure 3

Open in a new tab

Statistical parametric maps highlight voxels with significantly greater activation for the contrasts: press-money > instructed control (red) and no press-next trial > instructed control (green). Yellow voxels represent overlapping regions with activation. The insert highlights activation in the caudate head following small volume corrections. Images are arranged in 4-mm slices (moving left to right) extending upwards (bottom to top of brain) and with the left cerebrum on the left (i.e., left  =  left).

Figure 4. Beta weight differences for individual subjects in the left and right caudate head (see insert in Figure 3).

Figure 4

Open in a new tab

Figure 5 presents results of an exploratory analysis for the press-money > no press-next trial contrast, and the no press-next trial > press-money contrast. None of our regions of interest showed activation for the press-money > no press-next trial contrast. However, the no press-next trial > press-money contrast showed bilateral activation in numerous frontal regions, the left anterior cingulate, and the right putamen and fusiform gyrus (shown in green). Table 1 summarizes results shown in Figures 3 and 5 for our a priori regions of interest and other regions with activation that may benefit subsequent investigations.

Figure 5. Results of whole brain group random effect analyses.

Figure 5

Open in a new tab

Statistical parametric maps highlight voxels with significantly greater activation for the contrasts: press-money > no press-next trial (red), and no press-next trial > press-money (green). Images are arranged in 4-mm slices (moving left to right) extending upwards (bottom to top of brain) and with the left cerebrum on the left (i.e., left  =  left).

Table 1. Regions activated for each contrast performed (a. through d.) beyond the thresholds p < .005 for 10 contiguous voxels.
a. Press-money > control
 b. No-press next trial > control
Region Talairach
 Cluster sizea Z Region Talairach
 Cluster sizea Z
x y z x y z
Left Frontal Medial −4 48 18 31 3 17 Left Frontal Medial −4 −24 60 12 2.97
Inferior −51 18 10 33 2.98
Superior −12 48 29 193 3.38
Superior −6 52 25 (193) 3.29
Superior −12 35 46 17 2.96
Middle −42 40 18 56 3.17
Panetal Postcentral −30 −36 61 10 3.12 Parietal Precuneus −14 −56 36 23 3.48
Sub-lobar Putamen −20 9 −6 189 3.59 Sub-lobar Insula −28 17 −6 355 3.8
Caudate head −12 10 3 189 3.35
Thalamus −6 −12 0 11 2.9
Temporal Middle −61 −37 −5 15 3.73
Right Frontal Precentral 57 1 13 (115) 3.26 Right Frontal Superior 8 16 56 56 3.36
Inferior 51 29 −12 85 3.01 Superior 10 −16 63 (43) 3.01
Superior 14 62 −11 10 3.19
Superior 30 33 32 14 3.15
Middle 42 36 20 30 3.29
Medial 6 −15 54 43 3.13
Sub-lobar Putamen 20 0 9 113 2.79 Sub-lobar Putamen 32 −18 −1 74 3.67
Claustrum 34 −21 7 29 3.15 Putamen 14 8 0 72 3.41
Insula 38 6 −5 19 3.2
Temporal Middle 65 −37 −2 66 3.57 Temporal Middle 57 1 −17 36 3.49
Superior 59 −6 −1 115 3.93
Precentral 57 −5 8 (115) 2.79
Occipital Fusiform 22 −84 −11 18 3.2
Parietal Postcentral 61 −17 16 13 3.09
Limbic Anterior cingulate 4 9 25 23 4.07
Parahippocampus 20 5 −15 31 3.23
c. Press-money > no-press next trial
 d. No-press next trial > press-money
Region Talairach
 Cluster sizea Z Region Talairach
 Cluster sizea Z
x y z x y z
Left Posterior Uvula −10 −83 −31 22 3.03 Left Anterior Culmen −24 −28 −22 40 3.2
Sub-lobar Insula −42 −12 2 29 3.39 Anterior Culmen −20 −36 −20 10 2.69
Frontal Superior −18 14 53 90 4.2
Superior −20 15 60 (90) 3.84
Superior −26 48 18 339 3.98
Middle −30 43 13 (339) 3.82
Middle −32 21 30 201 3.54
Precentral −57 −8 32 11 2.94
Limbic Cingulate −10 −18 29 27 3.74
Cingulate −12 −19 38 (27) 2.8
Anterior cingulate −8 33 8 72 2.9
Parietal Angular −50 −59 34 36 4.06
Precuneus −14 −55 36 12 3.26
Posterior Declive −28 −57 −11 66 3.69
Temporal Inferior −61 −18 −19 15 3.22
Supenor −51 −57 19 15 3.17
Right Anterior Culmen 8 −53 −9 36 3.48 Right Frontal Middle 48 38 20 107 4.26
Inferior 44 38 13 (107) 3.86
Posterior Pyramis 24 −71 −27 30 3.64 Precentral 42 1 29 92 3.99
Temporal Middle 55 2 −30 58 3.19 Anterior cingulate 4 41 2 27 3.3
Inferior 44 −7 −28 (58) 3.05 Paracentral 6 −42 56 17 2.86
Transverse 42 −29 11 (66) 2.91 Superior 32 45 16 12 2.8
Superior 46 −27 0 (66) 2.75 Superior 18 51 16 12 2.78
Limbic Posterior cingulate 22 −62 12 11 3 Limbic Cingulate 14 23 30 29 3.22
Occipital Fusiform 38 −69 −13 51 3.08
Parietal Postcentral 53 −23 14 66 3.17 Parietal Precuneus 28 −48 43 37 3.29
Sub-lobar Insula 46 10 12 (92) 3.23
Putamen 28 4 9 15 3.05
Open in a new tab
a

Number of voxels in a cluster.

Note. Voxel clusters in parentheses highlight secondary local maxima > 8 mm apart from similar-sized clusters. Anterior and posterior refer to the cerebellum.

Formal group analyses of magnitude effects were not feasible, but it still was possible to perform an exploratory analysis for individual subjects to examine the relation between activation observed in the left caudate head and different reinforcer magnitudes correlated with press-money discriminative stimuli. Figure 6 plots beta differences for individual subjects in the left caudate head (x, y, z: −12, 10, 3) as a function of the three monetary amounts ($5.00, $0.50, and $0.05). This display reveals no consistent decreases in activation with decreases in reinforcer magnitude and considerable between-subject variability, both of which may have resulted from modeling too few presentations of each monetary amount.

Figure 6. Beta weight differences for individual subjects (S1 through S10) in the left caudate head for press-money discriminative stimuli plotted as a function of decreasing money amounts ($5.00, $0.50, $0.05, respectively).

Figure 6

Open in a new tab

Discussion

The present investigation demonstrated one efficient approach for integrating operant and imaging methodologies. Results of group and individual-subject analyses revealed significant increases in activation in the caudate, several frontal regions, and the putamen to discriminative stimuli previously correlated with operant contingencies compared to instructed control stimuli not correlated with operant contingencies. Such findings are consistent with results of nonhuman neurophysiological and human imaging “reward” research employing reinforcement contingencies (O'Doherty et al., 2001; Tremblay & Schultz, 2000a), human imaging studies with instructed contingencies (Tricomi et al., 2004), instructed responding to targets (Zink et al., 2003), avoidance (Jensen et al., 2003), and presentation of aversive stimuli (Becerra et al., 2001; Zald & Pardo, 2002). The present findings implicating frontal and striatal brain regions in discriminative stimulus control also lend support to the salience-system hypothesis that suggests striatal activity may reflect the salience or behavioral significance of stimuli (Horvitz, 2000; Kapur, 2003). The present findings are significant in the sense that frontal and striatal activation to discriminative stimuli could not be attributed to learning, response preparation, responding, runs of money gain or loss, or upcoming reinforcer delivery. Overall, we interpret activation correlated with presentation of discriminative stimuli as reflecting differences in control by learning histories involving programmed contingencies.

There is much discussion within cognitive neuroscience on the topics of imaging designs and analyses, but two topics that may surface at some point within the EAB are “what is the place of human operant research in fMRI research on operant learning processes?” and “what do contrast results reflect?” The present investigation suggests that conventional human operant research methods (ones involving operant contingencies) can be coupled successfully with fMRI designs and imaging analyses in ways that will contribute to cognitive neuroscience imaging research on operant learning processes. Regarding contrast results, a common approach used in imaging research is the “cognitive subtraction method.” This approach involves comparing activation from an experimental condition containing both a process of interest (X) and related sensory-motor processes (Y) with a control condition containing only the sensory-motor processes (Y). By contrasting activation between experimental and control conditions via inferential statistics, the assumption is that voxels showing statistically greater activation correspond to the process of interest X less Y via “subtraction” [(X,Y) − Y  =  X].

An alternative approach is to view results of contrasts as a difference between controlling variables rather than as a difference between additive, hypothetical processes. This view suggests that fMRI designs could be guided by questions about differences (or similarities) in the neural correlates that underlie differential stimulus control or “where in the brain is activation in the presence of X > activation in the presence of Y (or even X  =  Y)?” In the present investigation, we contrasted discriminative stimuli that previously exerted stimulus control over behavior via contingencies with stimuli that previously exerted stimulus control over behavior via verbal instructions (and no programmed contingency). The imaging data revealed several brain regions, especially the caudate head, where control by discriminative stimuli was greater than control by instructed stimuli. Imaging data also revealed that control by no press-next trial discriminative stimuli was relatively greater than control by press-money discriminative stimuli in numerous frontal regions, the anterior cingulate, fusiform gyrus, and putamen. Why differences appeared between these discriminative stimuli requires further investigation. Although further discussions are needed on these issues, we suggest that experimental questions, selection of fMRI designs, and interpretations of contrasts can be structured to focus on controlling variables, which avoids some of the assumptions of the cognitive subtraction approach.

Some cautions are important to note here. First, activation should never be taken to mean that the locus of a relation or stimulus property has been found. That is, the areas reported here do not necessarily represent the area of the brain where operant learning occurs, or stimulus discrimination, or the like. The brain is a complex organ of interrelated activity. Thus, at best, studies such as these should be considered as viewing only part of the picture.

That the effects of operant discrimination training employed in this investigation can be detected with BOLD fMRI allows for systematic replications with other controls, schedules, populations, and the like. Additional manipulations for future studies could include: (a) prebaseline imaging of the effects of stimuli to discern any differences across presumably neutral stimuli before training occurs, (b) the use of more-sensitive imaging equipment such as a 3.0 Tesla scanner, (c) an increased number of trials of varied money values to look at reinforcer magnitude effects, and (d) employing reversal designs to demonstrate that regional brain activation is a direct function of discrimination training procedures rather than structural properties of stimuli. With regard to schedules, the operant discrimination methodology employed in this investigation permits any variety of positive and negative reinforcement contingencies to be studied. Lastly, with regard to populations, we are especially interested in developing a methodology useful with populations having a variety of developmental disabilities including mental retardation. We believe that the current methods are particularly suitable, because training (which could require a large number of trials) can occur outside of the scanner.

Operant-based imaging studies, such as this, represent an attempt at methodological integration of BOLD fMRI methods and analyses with some aspects of the EAB. Over the past half-century, the study of human operant learning has matured to the degree where such interactions with other scientific disciplines are appropriate and should be fruitful. Several previous successes in this regard reinforce the current effort: applications of operant methods in mental retardation research, education, behavioral pharmacology, and behavioral medicine, to name a few. Operant-based imaging studies can improve the rigor of some current research approaches in cognitive neuroscience, advance our understanding of the neurobiology of learning and brain-behavior relations, as well as lead to clinical advances in neurology. After all, the clinical endpoint in neurology is behavior.

Acknowledgments

Research and manuscript preparation supported by NIH Grant R03 HD43178-01A1 awarded to the first author. We thank Amy Shelton for her insight and technical assistance with the experimental design and imaging analysis. Thanks also are due to Corrine Durisko and Jeff Phillips for invaluable comments and suggestions regarding the manuscript. Portions of these data were presented at the Association for Behavior Analysis meeting, May 2002.

Footnotes

Portions Of These Data Were Presented At The Association For Behavior Analysis Meeting, May 2002.

References

  1. Apicella P, Ljungberg T, Scarnati E, Schultz W. Responses to reward in monkey dorsal and ventral striatum. Experimental Brain Research. 1991;85:491–500. doi: 10.1007/BF00231732. [DOI] [PubMed] [Google Scholar]
  2. Becerra L, Brieter H.C, Wise R, Gonzalez R.G, Borsook D. Reward circuitry activation by noxious thermal stimuli. Neuron. 2001;32:927–946. doi: 10.1016/s0896-6273(01)00533-5. [DOI] [PubMed] [Google Scholar]
  3. Bechara A, Damasio A.R, Damasio H, Anderson S.W. Insensitivity to future consequences following damage to human prefrontal cortex. Cognition. 1994;50:7–15. doi: 10.1016/0010-0277(94)90018-3. [DOI] [PubMed] [Google Scholar]
  4. Breiter H.C, Aharon I, Kahneman D, Dale A, Shizgal P. Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron. 2001;30:619–639. doi: 10.1016/s0896-6273(01)00303-8. [DOI] [PubMed] [Google Scholar]
  5. Brett M. 2002. Retrieved from http://www.mrc-cbu.cam.ac.uk/Imaging/common/mnispace.shtml. [Google Scholar]
  6. Delgado M.R, Nystrom L.E, Fissell C, Noll D.C, Fiez J.A. Tracking the hemodynamic responses to reward and punishment in the striatum. Journal of Neurophysiology. 2000;6:3072–3077. doi: 10.1152/jn.2000.84.6.3072. [DOI] [PubMed] [Google Scholar]
  7. Elliott R, Friston K.J, Dolan R.J. Dissociable neural responses in human reward systems. Journal of Neuroscience. 2000;20:6159–6165. doi: 10.1523/JNEUROSCI.20-16-06159.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Elliott R, Newman J, Longe O, Deakin J. Instrumental responding for rewards is associated with enhanced neuronal response in subcortical reward systems. NeuroImage. 2004;21:984–990. doi: 10.1016/j.neuroimage.2003.10.010. [DOI] [PubMed] [Google Scholar]
  9. Friston K.J, Ashburner J, Frith C.D, Poline J.B, Heather J.D, Frackowiak R.S.J. Spatial registration and normalization of images. Human Brain Mapping. 1995;3:165–189. [Google Scholar]
  10. Friston K.J, Holmes A.P, Worsley K.J, Poline J.B, Frith C.D, Frackowiak R.S.J. Statistical parametric maps in functional imaging: A general linear approach. Human Brain Mapping. 1995;2:189–210. [Google Scholar]
  11. Groenewegen H.J, Wright C.I, Beijer A.V, Voorn P. Convergence and segregation of ventral striatal inputs and outputs. Annals of the New York Academy of Sciences. 1999;877:49–63. doi: 10.1111/j.1749-6632.1999.tb09260.x. [DOI] [PubMed] [Google Scholar]
  12. Haruno M, Kuroda T, Doya K, Toyama K, Kimura M, Samejima K, Imamizu H, Kawato M. A neural correlate of reward-based behavioral learning in caudate nucleus: A functional magnetic resonance imaging study of a stochastic decision task. Journal of Neuroscience. 2004;24:1660–1665. doi: 10.1523/JNEUROSCI.3417-03.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Hikosaka O, Sakamoto M, Usui S. Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. Journal of Neurophysiology. 1989;61:814–832. doi: 10.1152/jn.1989.61.4.814. [DOI] [PubMed] [Google Scholar]
  14. Holmes A.P, Friston K.J. Generalisability, random effects and population inference. NeuroImage. 1998;7:S754. [Google Scholar]
  15. Horvitz J.C. Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience. 2000;96:651–656. doi: 10.1016/s0306-4522(00)00019-1. [DOI] [PubMed] [Google Scholar]
  16. Jackson G.M, Jackson S.R, Harrison J, Henderson L, Kennard C. Serial reaction time learning and Parkinson's disease: Evidence for a procedural learning deficit. Neuropsychologia. 1995;33:577–593. doi: 10.1016/0028-3932(95)00010-z. [DOI] [PubMed] [Google Scholar]
  17. Jensen J, McIntosh A.R, Crawley A.P, Mikulis D.J, Remington G, Kapur S. Direct activation of the ventral striatum in anticipation of aversive stimuli. Neuron. 2003;40:1251–1257. doi: 10.1016/s0896-6273(03)00724-4. [DOI] [PubMed] [Google Scholar]
  18. Kapur S. Psychosis as a state of aberrant salience: A framework linking biology, phenomenology, and pharmacology in schizophrenia. American Journal of Psychiatry. 2003;160:13–23. doi: 10.1176/appi.ajp.160.1.13. [DOI] [PubMed] [Google Scholar]
  19. Kawagoe R, Takikawa Y, Hikosaka O. Expectation of reward modulates cognitive signals in the basal ganglia. Nature Neuroscience. 1998;1:411–416. doi: 10.1038/1625. [DOI] [PubMed] [Google Scholar]
  20. Knopman D, Nissen M.J. Procedural learning is impaired in Huntington's disease: Evidence from the serial reaction time task. Neuropsychologia. 1991;29:245–254. doi: 10.1016/0028-3932(91)90085-m. [DOI] [PubMed] [Google Scholar]
  21. Knutson B, Westdorp A, Kaiser E, Hommer D. FMRI visualization of brain activity during a monetary incentive delay task. NeuroImage. 2000;12:20–27. doi: 10.1006/nimg.2000.0593. [DOI] [PubMed] [Google Scholar]
  22. Lynd-Balta E, Haber S.N. The organization of midbrain projections to the ventral striatum in the primate. Neuroscience. 1994;59:609–623. doi: 10.1016/0306-4522(94)90181-3. [DOI] [PubMed] [Google Scholar]
  23. Maldjian J.A, Laurienti P.J, Burdette J.H, Kraft R.A. An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets. NeuroImage. 2003;19:1233–1239. doi: 10.1016/s1053-8119(03)00169-1. [DOI] [PubMed] [Google Scholar]
  24. O'Doherty J, Kringelbach M.L, Rolls E.T, Hornak J, Andrews C. Abstract reward and punishment representations in the human orbitofrontal cortex. Nature Neuroscience. 2001;4:95–102. doi: 10.1038/82959. [DOI] [PubMed] [Google Scholar]
  25. Ongur D, Price J.L. Intrinsic and extrinsic connections of networks within the orbital and medial prefrontal cortex. Cerebral Cortex. 2000;10:206–219. doi: 10.1093/cercor/10.3.206. [DOI] [PubMed] [Google Scholar]
  26. Robbins T.W, Everitt B.J. Neurobehavioural mechanisms of reward and motivation. Current Opinion in Neurobiology. 1996;6:228–236. doi: 10.1016/s0959-4388(96)80077-8. [DOI] [PubMed] [Google Scholar]
  27. Rogers R.D, Owen A.M, Middleton H.C, Williams E.J, Pickard J.D, Sahakian B.J, Robbins T.W. Choosing between small, likely rewards and large, unlikely rewards activates inferior and orbital prefrontal cortex. The Journal of Neuroscience. 1999;20:9029–9038. doi: 10.1523/JNEUROSCI.19-20-09029.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Rogers R.D, Ramnani N, Mackay C, Wilson J.L, Jezzard P, Carter C.S, Smith S.M. Distinct portions of the anterior cingulate cortex and medial prefrontal cortex are activated by reward processing in separable phases of decision-making cognition. Biological Psychiatry. 2004;55:594–602. doi: 10.1016/j.biopsych.2003.11.012. [DOI] [PubMed] [Google Scholar]
  29. Schlund M. Effects of acquired brain injury on adaptive choice and the role of reduced sensitivity to contingencies. Brain Injury. 2002;16:527–535. doi: 10.1080/02699050110113679. [DOI] [PubMed] [Google Scholar]
  30. Schlund M.W, Pace G. The effects of traumatic brain injury on reporting and responding to causal relations: An investigation of sensitivity to reinforcement contingencies. Brain Injury. 2000;14:573–583. doi: 10.1080/026990500120475. [DOI] [PubMed] [Google Scholar]
  31. Schultz W, Tremblay L, Hollerman J.R. Reward prediction in primate basal ganglia and frontal cortex. Neuropharmacology. 1998;37:421–429. doi: 10.1016/s0028-3908(98)00071-9. [DOI] [PubMed] [Google Scholar]
  32. Swenson M, Butters N. Dissociations within nondeclarative memory in Huntington's disease. Neuropsychology. 1996;10:538–548. [Google Scholar]
  33. Talairach Daemon . Talairach Daemon Client Download. 2003. Retrieved from http://ric.uthscsa.edu/projects/talairachdaemon.html. [Google Scholar]
  34. Tremblay L, Schultz W. Reward-related neuronal activity during go-nogo task performance in primate orbitofrontal cortex. Journal of Neurophysiology. 2000a;83:1864–1876. doi: 10.1152/jn.2000.83.4.1864. [DOI] [PubMed] [Google Scholar]
  35. Tremblay L, Schultz W. Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. Journal of Neurophysiology. 2000b;83:1877–1885. doi: 10.1152/jn.2000.83.4.1877. [DOI] [PubMed] [Google Scholar]
  36. Tricomi E.M, Delgado M.R, Fiez J.A. Modulation of caudate activity by action contingency. Neuron. 2004;41:281–292. doi: 10.1016/s0896-6273(03)00848-1. [DOI] [PubMed] [Google Scholar]
  37. Wise S.P, Murray E.A, Gerfew C.R. The frontal cortex–basal ganglia system in primates. Critical Reviews in Neurobiology. 1996;10:317–356. doi: 10.1615/critrevneurobiol.v10.i3-4.30. [DOI] [PubMed] [Google Scholar]
  38. Zald D.H, Pardo J.V. The neural correlates of aversive auditory stimulation. NeuroImage. 2002;16:746–753. doi: 10.1006/nimg.2002.1115. [DOI] [PubMed] [Google Scholar]
  39. Zink C.F, Pagnoni G, Martin M.E, Dhamala M, Berns G.S. Human striatal response to salient nonrewarding stimuli. Journal of Neuroscience. 2003;23:8092–8097. doi: 10.1523/JNEUROSCI.23-22-08092.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of the Experimental Analysis of Behavior are provided here courtesy of Society for the Experimental Analysis of Behavior

RESOURCES