Affective Learning Enhances Visual Detection and Responses in Primary Visual Cortex

Abstract

The affective significance of a visual item is thought to lead to enhanced visual processing. However, the precise link between enhanced visual perception of emotion-laden items and increased visual responses remains poorly understood. To investigate this link, we acquired functional magnetic resonance imaging (fMRI) data while participants performed a challenging visual detection task. Grating stimuli were physically identical and differed only as a function of their previous exposure history; CS+ stimuli were initially paired with shock, whereas CS− stimuli were not. Behaviorally, subjects were both faster and more accurate during CS+ relative to CS− target detection. These behavioral results were paralleled by increases in fMRI responses across early, retinotopically organized visual cortex, which was mapped in a separate fMRI session. Logistic regression analyses revealed that trial-by-trial fluctuations in fMRI responses were closely linked to trial type, such that fMRI signal strength reliably predicted the probability of a hit trial across retinotopically organized visual cortex, including area V1. For instance, during the CS+ condition, a 0.5% signal change increased the probability of a hit from chance to 67.3–73.5% in V1–V4 (the highest increase was observed in area V1). Furthermore, across participants, differential fMRI responses to hits versus correct rejects were correlated with behavioral performance. Our findings provide a close link between increased activation in early visual cortex and improved behavioral performance as a function of the affective significance of an item.

Introduction

The affective significance of a visual item is thought to lead to enhanced visual processing (Phelps, 2006; Vuilleumier and Driver, 2007; Pessoa, 2008). For example, subjects exhibit fast, involuntary responses to emotional stimuli, such as faces with fearful expressions or aversive pictures (Globisch et al., 1999). In addition, negative stimuli appear to be a more effective source of involuntary interference to on-going tasks than neutral or positive ones (Hartikainen et al., 2000; Tipples and Sharma, 2000; Pereira et al., 2006), and to more readily recruit attention (Bradley et al., 1997; Eastwood et al., 2001). Enhanced visual processing of affective information is also suggested by neuroimaging data. For instance, functional magnetic resonance imaging (fMRI) responses evoked while viewing emotional faces is stronger relative to viewing neutral ones and responses evoked to viewing emotional scenes (e.g., a war scene) are larger than to viewing neutral scenes (e.g., a lake scene) (Pessoa et al., 2002b; Vuilleumier, 2005). Increased activation has been reported across many visual areas, including “late” areas such as the fusiform gyrus and superior temporal sulcus, and early visual cortex involving Brodmann's areas 17 and 18. However, because previous studies did not perform retinotopic mapping for individual participants, it is not possible to precisely map the reported results to early, retinotopically organized visual areas, such as V1 and V2. Thus, the precise impact of emotion-laden visual stimuli on early visual cortical activation remains unclear.

Furthermore, despite the findings summarized above, the link between enhanced visual perception of emotion-laden items and increased fMRI responses remains poorly understood. The aim of the present study was to fill-in this gap. We reasoned that if the affective significance of an item enhances visual processing, it should be possible to (1) demonstrate enhanced visual performance during a basic visual task, and to (2) link potential improvements in behavioral performance to fMRI responses evoked in early visual cortex in a trial-by-trial manner. Critically, it should be possible to probe these two questions while controlling for stimulus confounds, which significantly complicate the interpretation of the findings reviewed above (e.g., both emotional and neutral faces and unpleasant and neutral scenes differ in their visual features). To address these questions, in the present study, we acquired fMRI data while participants performed a challenging visual detection task involving simple grating stimuli (see Fig. 1), so-called Gabor gratings. To modulate affective significance, during an initial learning phase, certain gratings were paired with shock (CS+ gratings; e.g., those presented in the left hemifield) 50% of the time, whereas others were never paired with shock (CS− gratings; e.g., those in the right hemifield). We were chiefly interested in assessing two separate questions. First, would subjects demonstrate improved detection ability to CS+ gratings relative to CS− gratings? Second, would potential improvements in performance be paralleled by evoked responses in retinotopically organized visual cortex, including primary visual cortex (V1)? By using a slow event-related design, we thus aimed at investigating how trial-by-trial fluctuations in fMRI responses were linked to detection performance in regions of interest (ROIs) in early visual cortex.

Figure 1.

Experimental paradigm. A, Affective significance was manipulated via an item's history. In this example, during an initial conditioning procedure, visual gratings paired with the auditory stimulus “one” were paired with mild shock 50% of the time, whereas those paired with the stimulus “two” were not. The figure is schematic only. B, During the detection phase, trials occurred every 16 s. Subjects were required to indicate whether or not a low-contrast grating (indicated by the dashed circle; not displayed during actual task performance) was shown on that trial. Because the task was demanding, an auditory cue was sounded 100 ms after a grating may have been shown to indicate that subjects should respond. The figure is not drawn to scale; for simplicity, noncritical parts of the trial were omitted.

Materials and Methods

Subjects.

Nine volunteers (26 ± 4 years old; three females) participated in the study, which was approved by the Institutional Review Board of Indiana University, Bloomington, IN. All subjects were in good health with no past history of psychiatric or neurological disease and had normal or corrected-to-normal vision. All participants gave informed written consent.

Stimuli and behavioral task.

The target stimulus used was a low-contrast Gabor grating (Gaussian smoothed sinusoidal grating) oriented at minus 45°. The grating size was 2° (radius) and it was shown at 4° of eccentricity (grating center). For each participant, grating contrast was calibrated during behavioral sessions in a mock scanner (with identical stimulus settings and stimulation conditions as used in the fMRI session), such that detection accuracy was ∼65% correct.

During an initial learning phase, aversive conditioning was used to manipulate the affective significance of visual stimuli. Conditioning, which took place during the first experimental run, used the same detection task and trial structure as the subsequent runs (see below) (Fig. 1), with the exception that trials lasted 12 s and that a high-contrast grating was used (thus, behavioral performance was near ceiling). For the initial four participants (conditioning 1), Gabor gratings shown on the left visual field (or right) were paired with mildly aversive electrical stimulation (CS+ condition), and Gabor gratings shown on the right visual field (or left) were never paired with shock (CS− condition); left/right presentation was counterbalanced across participants. For the CS+ condition, shocks occurred in 50% of the target-present trials and were administered 1 s after the onset of the grating; thus, shocks never occurred during target-absent trials. CS+ and CS− conditions were indicated at the beginning of the trial by the auditory stimulus “left” or “right” (i.e., the spoken word “left” or “right”). For the subsequent five participants (conditioning 2), Gabor gratings were always shown in a fixed visual field, either left or right (counterbalanced across subjects). Trials starting with the auditory stimulus “one” (i.e., the spoken word “one”) were paired with shock (CS+ condition), and trials starting with the auditory stimulus “two” were never paired with shock (CS− condition); “one”/“two” stimuli were counterbalanced across participants. Again, shocks only occurred during target-present trials.

After the conditioning run, subjects performed nine detection runs in which they were asked to indicate on each trial whether or not a briefly presented low-contrast Gabor grating was presented. In a slow event-related design, each trial began with a fixation cross shown for 1700 ms on a black background. After 200 ms from the onset of the fixation, a 500 ms auditory stimulus indicated the experimental condition (CS+ or CS−) by using the auditory stimuli “left”/“right” (conditioning 1) or “one”/“two” (conditioning 2). The Gabor grating was presented for 50 ms, followed by a 100 ms blank screen and a 100 ms tone prompting the subject to respond (Fig. 1). The same trial structure was used for the conditioning 1 and 2 groups. Trials occurred every 16 s. To minimize the extinction of conditioned responses, zero to two shocks were administered per run. Because they involved electrical stimulation, these trials were excluded from additional analyses. A total of 432 trials were performed for each subject.

Subjects participated in a total of five to six experimental sessions: two to three initial practice behavioral sessions in the mock scanner that did not involve conditioning; two fMRI sessions that followed the conditioning-plus-detection structure outlined above; a final fMRI session for retinotopic mapping that was used to determine the cortical representation of visual areas V1–V4. The two main sessions (conditioning plus detection) occurred in consecutive days for most subjects, except for three participants (3 d maximum separation).

Conditioning procedure.

As an unconditioned stimulus (US), a 500 ms electric shock (50 Hz) was delivered to the distal phalanges of the third and fourth fingers of the right hand by a shock stimulator (E13–22; Coulbourn Instruments). Before the experiment, subjects were instructed of the contingency rule, but were not informed about the probability of US delivery. The intensity of electric shock, which ranged between 0.8 and 4.0 mA, was determined separately for each participant so as to be “highly unpleasant but not painful”.

Skin conductance responses (SCRs) were recorded with the MP-150 system (BIOPAC Systems) and Ag/AgCl electrodes placed on the distal phalanges of the index and middle finger of the nondominant (left) hand. SCR was amplified and sampled at 250 Hz and the analysis of waveforms was conducted using Matlab. SCR data were first detrended and then smoothed with a median filter of 50 samples to filter out MRI-induced noise. To reduce novelty effects, the first four “habituation” trials, defined as the first trial occurrence of each trial type (CS+/CS− × target present/target absent), were excluded from the analyses. On each trial, the level of SCR was calculated by subtracting a baseline (average signal between 0 and 1 s) from the peak amplitude during the 1–6 s time window after stimulus onset (Prokasy and Raskin, 1974).

MR data acquisition.

MR data were collected using a 3 Tesla Siemens TRIO scanner. During conditioning-plus-detection sessions, echoplanar imaging volumes were acquired with a repetition time (TR) of 2000 and echo time (TE) of 25 ms. A single volume consisted of 34 slices positioned obliquely, roughly perpendicular to the calcarine fissure with a thickness of 3.8 mm and an in-plane resolution of 3.8 × 3.8 mm (240 mm field of view). This slice positioning was used to improve signal quality and the attribution of voxels to regions surrounding the calcarine fissure (V1 and V2) (Ress et al., 2000; Cheng et al., 2001). Given this positioning, we were unable to cover the entire brain, and lost portions of the frontal lobe, among others. During the retinotopy session, similar imaging parameters were used, except that a volume consisted of 32 slices positioned obliquely, again roughly perpendicular to the calcarine fissure, but with a thickness of 3 mm and an in-plane resolution of 3 × 3 mm (192 mm field of view). At the beginning of each scanning session, a high-resolution magnetization-prepared rapid-acquisition gradient echo anatomical sequence was performed (TR, 1900 ms; TE, 4.15 ms; inversion time, 1100 ms; 1 mm isotropic voxels; 256 mm field of view).

Behavioral data analysis.

We analyzed performance in terms of the signal detection theory measure d′ (Green and Swets, 1966), which is defined in terms of hit (correct response; target-present trial) and false alarm (incorrect response; target-absent trial) rates: d′ = z(H) − z(F), where H and F are the hit and false alarm rate, respectively, and z(x) is the cumulative normal probability distribution function (i.e., left tail z scores). Because participants made a very low number of false alarms, in addition to analyzing performance in terms of the sensitivity measure d′, we also analyzed it in terms of the rate of hits and the rate of false alarms. The latter analysis was desirable especially because three participants made no false alarms, which is even more problematic for the estimation of d′. When no false alarms were made, the false alarm rate was arbitrarily set to 1/2N, where N is the number of trials (Macmillan and Creelman, 1991). For the d′ analysis, we considered the nine fMRI participants in addition to three additional ones that participated in a behavioral session that took place in the mock-scanner using identical parameters to those used during scanning. When only the nine fMRI participants were compared, the difference between d′ for the CS+ and CS− conditions approached significance (p = 0.1).

General fMRI data analysis.

Preprocessing steps, which were the same for the fMRI sessions (see also below), used AFNI tools (Cox, 1996) (http://afni.nimh.nih.gov/afni). The first five volumes of each functional run were discarded to account for equilibration effects. The remaining volumes were slice-time corrected and spatially registered for motion correction to the volume acquired closest in time to the particular subject's high-resolution anatomy. No normalization or spatial smoothing of functional volumes was applied for analyses involving early visual ROIs (see below). For these ROIs, signal intensity at each voxel was scaled to a mean of 100 and detrended to remove slow-varying drifts in the fMRI signal (both linear and nonlinear terms were used). For other ROIs, functional data normalized to Talairach space (Talairach and Tournoux, 1988) and spatially smoothed were considered (Gaussian filter, 7.6 mm Gaussian full-width at half maximum; i.e., two times the voxel dimension). Again, for each voxel, signal intensity was scaled to a mean of 100. In addition, for nonretinotopically organized ROIs, the functional data from the second fMRI session were aligned to first-session data. To do so, a transformation matrix was first determined that aligned the second-session anatomical data set to that of the first-session data set. This transformation was then applied to the functional data set so as to bring the two sessions in register. Both steps used the 3dvolreg tool from the ANFI package. First- and second-session data were then concatenated.

Two types of fMRI data analysis were performed by focusing on retinotopically organized visual areas V1–V4, in addition to a few targeted ROIs (see below): selective averaging and logistic regression analysis (see below). Selective averaging was performed in terms of the factors condition (CS+ and CS−), target (present and absent), and accuracy (correct and incorrect). The following trial types were considered: hit (correct, target-present), correct reject (correct, target-absent), and miss (incorrect, target-present); false alarms (incorrect, trial-absent) were not considered because of insufficient data for this trial type.

Controlling for the effect of reaction time on fMRI responses.

Reaction time (RT) differed as a function of trial type (see Fig. 2A) and also influenced evoked responses in visual cortex. For instance, miss trials evoked greater fMRI responses than hit trials across retinotopically organized visual areas and also exhibited longer RTs. We formally assessed this relationship by considering the “control” condition (CS−) during which target stimuli were not paired with shock. For this condition, we correlated the difference in RTs between miss and hit trials with the difference in evoked single-trial responses in V1 for the same trial types, which revealed a significant correlation (r = 0.76; p < 0.05). In other words, across subjects, increases in RT during miss trials (relative to hit trials) were correlated to increases in evoked fMRI responses in V1 for miss trials (relative to hit trials); note that both hits and misses were physically identical, and only differed in terms of the behavioral decision. For this analysis, single-trial response amplitude was indexed by averaging activation at three points around the peak (i.e., at times 4, 6, and 8 s relative to trial onset).

Figure 2.

Behavioral and skin conductance responses. A, Subjects exhibited the fastest reaction times to CS+ hit (correct, target present) trials. B, Differential SCRs (target–nontarget) were significantly higher during the CS+ relative to the CS− condition. Error bars indicate SEM.

Because RT and evoked responses were correlated, we determined RT-corrected fMRI responses by simply dividing the uncorrected single-trial fMRI responses by the RT on that trial. Because miss trials exhibited the longest RTs, RT-corrected fMRI responses were used in analyses involving hits versus misses. For parsimony, other analyses did not use RT-corrected activations, although the results obtained with the correction were in general stronger than the ones shown here. Note that we did not use a more standard strategy of having a nuisance regressor for RT to deal with RT effects because ours was not a traditional general linear model-type analysis with a set of regressors, but instead a trial-by-trial analysis (see below).

ROI analysis.

Each subject participated in a separate session for retinotopic mapping, which used standard procedures of meridian, polar angle, and eccentricity mapping (Engel et al., 1994, 1997). Retinotopic data were then used to delineate the contralateral cortical representations of early visual areas V1, V2, V3 (also called VP), and V4. A representative time series for each ROI was obtained by averaging all of the voxels identified via retinotopic mapping that were also reliably driven by the task during the conditioning-plus-detection fMRI sessions, as determined by a standard contrast of task versus baseline. Because no smoothing was used, a threshold of p = 0.05 (uncorrected) was used for voxel selection. The following number of voxels was used on average: V1, 26; V2, 31; V3, 22; V4, 27.

Two additional control analyses were performed, one for voxels at the fovea and another for voxels in contralateral regions V1–V4 that corresponded to a high-eccentricity patch at ∼12–16°. The following number of voxels was used on average, again based on the task versus baseline contrast: fovea, 24; high-eccentricity patch, V1, 16; V2, 10; V3, 14; V4, 16.

Because of the visual nature of the task, other regions in ventral temporal cortex were also activated. In particular, we observed activation in the inferior temporal gyrus (ITG) (right ITG, x = 51, y = −56, z = −10; left ITG, x = −32, y = −56, z = −9) within a site slightly lateral to the fusiform gyrus. We also probed activation in an ROI in the intraparietal sulcus (IPS) (right IPS, x = 28, y = −56, z = 45; left IPS, x = −26, y = −56, z = 41). We chose this region because, as expected given the roles of this region in several attentional functions, it was strongly activated in the task versus baseline contrast. In particular, we observed performance-related activity in this area in a previous working memory study (Pessoa et al., 2002a). Furthermore, because we used auditory cues to signal trial type, we also defined ROIs in auditory cortex (right Heschel's gyrus, x = 50, y = −20, z = 11; left Heschel's gyrus, x = −42, y = −20, z = 11). Left and right hemisphere ROIs for the regions above were created based on the contrast of task versus baseline (p < 0.05). Finally, although our coverage originally extended beyond visual cortex to include the amygdala, the signal quality in this region was very poor because of severe “drop out” within this area given our scanning parameters (a 3T system and oblique slices). Therefore, we were unable to analyze responses in this region.

Trial-based analysis.

To quantify the link between fluctuations in fMRI single-trial amplitude and behavioral performance at the level of the individual, we performed a trial-by-trial logistic regression analysis, which was possible given our slow event-related design. For each trial, response amplitude was indexed by the average of activations at time points at 4, 6, and 8 s post trial onset (three points around the expected peak response); a very similar pattern of results was obtained when other strength measures were used, such as magnitude at the response peak. Average responses were obtained by selective averaging as a function of the factors condition (CS+ and CS−), target (present and absent), and accuracy (correct and incorrect). A standard logistic regression analysis was performed (Hosmer and Lemeshow, 2000) by modeling the probability of a trial being a hit trial or a correct reject trial as a function of fMRI response strength. A second logistic regression analysis modeled the probability of a trial being a hit trial or a miss trial as a function of fMRI response strength. The slope of the regression fit indicates the magnitude of the predictive effect. Group inferences were made by contrasting logistic slope values across participants via paired t tests (random effects).

Note that a contrast of CS+ and CS− logistic regression slopes logically corresponds to a test of a statistical interaction. Consider a logistic regression involving hits and correct rejects. The ensuing regression slope itself quantifies differences between the two trial types. Thus, the contrast between CS+ and CS− slopes across subjects implements a test for a condition × trial type interaction.

Whereas logistic regression intuitively described the relationship between fMRI signal strength and trial type, it relies on assuming a logistic functional form. To confirm that our results were not dependent of this parametric assumption, we reanalyzed our results in terms of a nonparametric receiver operating characteristic analysis (Pessoa and Padmala, 2005; Thielscher and Pessoa, 2007), for which permutation-based statistical tests were used. Our results were confirmed by the nonparametric method and are not discussed further.

Results

Behavior and skin conductance responses

Behavioral performance was analyzed by comparing the sensitivity measure d′ (see Materials and Methods) as a function of condition (CS+ and CS−), which revealed improved detection performance during CS+ trials relative to CS− trials (CS+, 2.64 ± 0.16; CS−, 2.29 ± 0.15; paired t test, p < 0.05). These results are consistent with a considerably higher hit rate during CS+ trials (75.1 ± 1.8%) relative to during CS− trials (61.3 ± 1.6%; paired t test, p < 0.001), whereas no significant difference was observed for false alarm rate (CS+, 4.5 ± 1.2%; CS−, 3.87 ± 1.1%; paired t test, p = 0.56).

Mean RTs (Fig. 2A) were analyzed in terms of a 2 condition (CS+ and CS−) × 3 trial type (hit, correct reject, miss) repeated-measures ANOVA, which revealed a main effect of trial type (p < 0.001) and a significant statistical interaction (p < 0.005); false alarms were not included because subjects made very few errors of this type (e.g., three subjects made no false alarms). Subjects were slowest during miss trials (711 ms), exhibited intermediate RTs during correct rejects (657 ms) and were fastest during hit trials (593 ms); differences between hits and correct rejects, hits and misses, and correct rejects and miss were all significant (p values <0.01). Critically, only RTs during hit trials differed as a function of condition (t test, p < 0.01); no significant differences were observed for misses or correct rejects. Overall, subjects were both faster and more accurate during CS+ trials.

During the initial conditioning run, because Gabor patches of relatively high contrast were used, subjects made very few errors (98.0 ± 1.7% correct). Therefore, we analyzed SCRs in terms of a 2 condition (CS+ and CS−) × 2 trial type (hits and correct rejects) repeated-measures ANOVA, which revealed main effects of condition (p < 0.05) and trial type (p < 0.01), as well as a significant statistical interaction (p < 0.05). The latter reflected the larger differential SCRs evoked to target-present (hit) versus target-absent (correct reject) trials during the CS+ relative to the CS− condition (Fig. 2B).

Retinotopically organized visual cortex: hits versus correct rejects

In investigating the link between behavioral performance and fMRI responses, we were particularly interested in probing visual responses as a function of both target (present vs absent) and accuracy (correct vs incorrect). Because subjects made very few false alarms, these analyses focused on contrasting hits versus correct rejects and hits versus misses, respectively. These two analyses are complementary and characterize the link between performance and evoked responses. Specifically, hits and correct rejects are matched in terms of performance (both are correct trials), but differ in terms of the physical stimulus (target present vs absent); hits and misses are matched in terms of physical stimulus (both trial types contain a target), but differ in terms of accuracy (correct vs incorrect). We first present the results for hits versus correct rejects and discuss results for hits versus misses in the next section.

We investigated fMRI responses evoked in retinotopically organized areas V1, V2, V3, and V4 as a function of condition (CS+ and CS−) and target (present and absent). Figure 3A illustrates responses during hit (target present) and correct reject (target absent) trials in contralateral V1 for both CS+ and CS− stimuli. Stronger responses during hit relative to correct reject trials were observed reliably only for the CS+ condition. No differences as a function of condition were evident when control analyses were performed in the fovea (where only a fixation cross was shown) (Fig. 3B) or at a high eccentricity (data not shown) (see Materials and Methods).

Figure 3.

Average single-trial responses for hits and correct rejects (n = 9). A, In contralateral V1, responses evoked during CS+ hits were strongest. B, No differences as a function of trial type were observed in the fovea (which contained a fixation cross only). Error bars indicate SEM and are shown for one condition to maximize clarity. H, Hit; CR, correct reject. Error bars indicate SEM.

Because we were interested in the potential role of affective significance in visual detection, we tested for condition × target statistical interactions. In other words, we sought to test whether hit versus correct reject responses differed during the CS+ and CS− conditions. In this manner, differential attention and/or arousal during the CS+ and CS− conditions were effectively subtracted out (e.g., subjects may have been more aroused after hearing an auditory cue signaling a CS+ trial relative to a CS− trial) (see below). A significant condition × target interaction was observed in all visual areas (V1–V4; all p values < 0.05). A significant main effect of target was also observed in all areas (all p values < 0.05); condition was significant in area V3 (p < 0.05) and a near-significant result was found in V1 (p = 0.07). A summary of these results is provided in Table 1. No significant interactions (or main effects) were observed for the control analyses at the fovea or at a high eccentricity.

Table 1.

Condition × target ANOVA (p values)

Visual area	Condition main effect	Target main effect	Condition × target interaction
V1	0.07*	0.003	0.001
V2	n.s.	0.005	0.004
V3	0.04	0.04	0.001
V4	n.s.	0.006	0.02

Entries in bold font are statistically significant; asterisks denote near-significant results. n.s., Not significant.

To further investigate the link between single-trial amplitude and behavioral performance at the level of the individual, we performed a logistic regression analysis in which single-trial amplitude was used to predict whether a correct trial was a hit or a correct reject. The results of the logistic fit for the CS+ condition are shown in Figure 4A for a representative individual in area V1. The probability of a given trial being a hit trial increased as a function of fMRI response strength. To test for potential differences at the group level, paired t tests were performed to compare CS+ and CS− regression slopes for each visual region (Fig. 4B). In all cases, significant differences were found (all p values < 0.05).

Figure 4.

Logistic regression analysis with hit and correct reject trials. A, The probability of a hit trial as a function of fMRI single-trial response in V1 was modeled via a logistic regression analysis for data from a representative individual. The regression slope indicates the strength of the predictive effect. B, Regression slopes averaged across the group indicated a stronger relationship during CS+ trials. Error bars indicate SEM.

Both hit and correct reject trials are, by definition, correct trials. We reasoned that if fMRI responses reflected behavioral responses during the CS+ condition, across participants, the increase in fMRI response to hit versus correct reject trials should be correlated with behavioral performance. In other words, the increase in response during a hit versus a correct reject trial (which can be viewed as a baseline condition) should be linked to the behavioral hit rate (i.e., the proportion of correct target-present trials) given that very few false alarms were made. This was indeed the case in regions V1 (Fig. 5) and V3, and a trend was observed in area V4 (V1, r = 0.74, p < 0.05; V2, r = 0.41, p = 0.26; V3, r = 0.81, p < 0.01; V4, r = 0.56, p = 0.11). During the CS− condition, no such link was observed (all r values < 0.2).

Figure 5.

Relationship between fMRI responses and behavior across participants. An individual's hit rate was significantly correlated (r = 0.74) with the magnitude of fMRI responses during hit trials (relative to correct reject trials) in primary visual cortex.

Retinotopically organized visual cortex: hits versus misses

To further investigate the link between affective significance and performance, we investigated fMRI responses evoked in retinotopically organized visual cortex as a function of condition (CS+ and CS−) and accuracy (correct and incorrect). Figure 6A illustrates responses during hit (correct) and miss (incorrect) trials in contralateral V1 for both CS+ and CS− stimuli. Stronger responses during hit relative to miss trials were observed reliably only for the CS+ condition. As before, we investigated the impact of affective significance in visual detection by probing statistical interactions, in this case condition × accuracy, thereby accounting for differential attention/arousal effects. A significant statistical interaction was observed in areas V1 and V4 (p values < 0.05), and near-significant interactions in areas V2 and V3 (p = 0.08 and p = 0.07, respectively). A significant main effect of accuracy was observed in areas V2 and V4 (p values < 0.01). A main effect of condition was significant in area V1 (p < 0.01) (see Table 2 for a summary). No significant interactions (or main effects) were observed for the control analyses at the fovea (Fig. 6B) or at a high eccentricity.

Figure 6.

Average single-trial responses for hits and misses (n = 9). A, In contralateral V1, responses evoked during CS+ hits were strongest. B, No differences as a function of trial type were observed in the fovea (which contained a fixation cross only). Error bars indicate SEM and are shown for one condition to maximize clarity. H, Hit; M, miss. Error bars indicate SEM.

Table 2.

Condition × accuracy ANOVA (p values)

Visual area	Condition main effect	Accuracy main effect	Condition × accuracy interaction
V1	0.01	n.s.	0.009
V2	n.s.	0.009	0.08*
V3	n.s.	n.s.	0.07*
V4	n.s.	0.009	0.02

Entries in bold font are statistically significant; asterisks denotes near-significant results. n.s., Not significant.

Next, we performed a logistic regression analysis at the level of the individual in which single-trial amplitude was used to predict whether a target-present trial was detected (hit) or not (miss). The results of the logistic fit for the CS+ condition are shown in Figure 7A for a representative individual for area V1. The probability of a given trial being a hit trial increased as a function of fMRI response strength. To test for potential differences at the group level, paired t tests were performed to compare CS+ and CS− regression slopes for each visual region (Fig. 7B). In all cases, significant differences were found (all p values < 0.05).

Figure 7.

Logistic regression analysis with hit and miss trials. A, The probability of a hit trial as a function of fMRI single-trial response in V1 was modeled via a logistic regression analysis for data from a representative individual. The regression slope indicates the strength of the predictive effect. B, Regression slopes averaged across the group indicated a stronger relationship during CS+ trials. Error bars indicate SEM.

For the CS+ condition, we also correlated the differential fMRI responses to hits versus misses with the behavioral hit rate across participants. Although the correlations were lower than those observed in the hit versus correct reject analysis, they approached statistical significance in areas V1 and V4 (V1, r = 0.58, p = 0.1; V2, r = 0.36, p = 0.3; V3, r = 0.38, p = 0.3; V4, r = 0.61, p = 0.07). Again, no link was observed during CS− trials (all r values < 0.2).

Retinotopically organized visual cortex: comparing effects in V1–V4

As Figures 4 and 7 illustrate, single-trial responses across regions V1–V4 were predictive of behavioral performance during the CS+ condition, but not during the CS− condition. We attempted to more explicitly compare the predictive power of visual regions by comparing their regression slopes. Figure 8 plots the difference in CS+ and CS− slopes obtained from the hits vs correct rejects and from the hits vs misses logistic regression analyses. Although the effects appeared to be largest in V1 (Fig. 8B) (a t tests comparing V1 and V2, and V1 and V4 were statistically significant; p = 0.01 and p = 0.03, respectively), no statistically significant differences were observed when the areas were compared via a one-way ANOVA. Like other studies in the past (Ress et al., 2000), ours was likely statistically underpowered to detect differences across early visual areas at the group level.

Figure 8.

Comparison across V1–V4. A, B, The difference in logistic regression slopes is shown for the analysis of hits vs correct rejects (A) and hits versus misses (B). The relationship between fMRI responses and behavior appeared to be strongest in area V1 (see Results). Error bars indicate SEM.

Additional ROIs

Although the focus of our investigation involved early, retinotopically organized visual cortex, we probed fMRI responses in three additional ROIs: inferior temporal gyrus (whose responses likely reflected “late” visual processing), IPS [which is involved in attention and is correlated with behavioral performance (Pessoa et al., 2002a)], and the auditory cortex surrounding Heschel's gyrus (given the use of auditory cues at trial onset). Selective averaging was performed in terms of the factors condition (CS+ and CS−), target (present and absent), and accuracy (correct and incorrect) (supplemental Figs. 1, 2, available at www.jneurosci.org as supplemental material). Repeated-measures ANOVA revealed a main effect of target in the left ITG and bilateral IPS, and a significant main effect of accuracy in bilateral IPS (a near-significant effect was observed in the left ITG) (supplemental material, available at www.jneurosci.org). No significant differences were observed in auditory cortex. Thus, the pattern of results in these additional ROIs was quite distinct from that observed in early visual cortex.

Discussion

In the present study, participants were asked to detect visual gratings presented under challenging conditions. Gratings were physically identical and only differed as a function of their previous exposure history, as some were paired with shock during a learning phase, whereas others were never paired with shock. Behaviorally, subjects were both faster and more accurate during CS+ relative to CS− conditions. These behavioral results were paralleled by increases in fMRI responses across early, retinotopically organized visual cortex, including primary visual cortex. At the individual level, logistic regression analyses revealed that trial-by-trial fluctuations in fMRI responses were closely linked to trial type such that fMRI signal strength reliably predicted the probability of a hit trial. Furthermore, across participants, differential fMRI responses to hits versus correct rejects were correlated with behavioral performance. Together, our findings provide a close link between increased activation in early visual cortex and improved behavioral performance as a function of the affective significance of an item.

Ress, Heeger, and colleagues have shown that trial-by-trial fluctuations in fMRI responses closely follow a subject's behavioral performance in demanding visual tasks (Ress et al., 2000; Ress and Heeger, 2003; Silver et al., 2007). Activity in early visual cortex was dominated by a large response that was relatively independent of the presence of a grating pattern, suggesting that, when sensory inputs are weak and attentional demands are high, activity in early visual cortex may be dominated by a stimulus-independent response that is related to visual attention. Our findings followed their results quite closely. Target-absent trials exhibited sizeable responses that were comparable in magnitude to target-present trials, especially for the CS− condition. Overall, responses observed during target-absent trials may have reflected a “baseline response” that has been characterized by single-unit and fMRI studies (Luck et al., 1997; Chawla et al., 1999; Kastner et al., 1999).

Behaviorally, affective significance considerably increased detection sensitivity as subjects improved from a 61 to 75% hit rate. Reaction time was also considerably faster during CS+ hit trials. In terms of fMRI responses, within participants, improvements in behavioral performance were reflected in larger differential signals during the CS+ relative to the CS− condition. Specifically, trial-by-trial fluctuations in single-trial response amplitude were quantitatively linked to the probability of a trial being a hit trial. This link was formally assessed by performing a logistic regression analysis for hits versus correct rejects and, separately, hits versus misses. In this manner, we could reliably distinguish a hit from a correct reject trial. During the CS+ condition, a 0.5% signal change increased the probability of a hit from chance to 67.3–73.5% in V1–V4 (the highest value was observed in V1). Likewise, we could reliably distinguish a hit from a miss trial. In this case, a 0.5% signal change increased the probability of a hit from chance to 66.6–72.4% in V1–V4 (again, the highest value was observed in V1). Furthermore, across participants, increases in differential fMRI responses were linked to behavioral performance. In particular, increases in responses evoked to hit relative to correct reject trials were correlated with the behavioral hit rate in areas V1 (r = 0.74) and V3 (r = 0.81). This link was observed during the CS+ condition but not during the CS− condition.

Several investigators have proposed that the processing advantage of emotional items is dependent on the amygdala (Anderson and Phelps, 2001). Here, we reasoned that if the amygdala biases visual competition in favor of emotion-laden items, then stimuli that undergo aversive conditioning should exhibit enhanced processing. Our prediction was based on strong extant evidence that fear conditioning is dependent on the amygdala in both nonhuman species and humans (Davis and Whalen, 2001; Phelps and LeDoux, 2005). The increased differential SCRs for the CS+ relative to the CS− condition is consistent with this proposal (Bechara et al., 1995; LaBar et al., 1995; Knight et al., 2005). However, because of our slice positioning, the signals from the amygdala exhibited substantial signal “drop out” in our 3T system. We were thus unable to investigate how trial-by-trial fluctuations in the amygdala correlated with either behavior or visual cortical responses.

As mentioned in the Introduction, increases in evoked responses in visual cortex during the viewing of emotion-laden visual stimuli have been well documented. However, the relationship of these activations to retinotopically organized areas was unknown. In the present study, the mapping of visual areas V1–V4 was determined in a separate experimental session and allowed us to determine the cortical representation of the Gabor gratings shown during the main experimental sessions. Our results demonstrate that the primary visual cortex, in addition to other retinotopically organized visual areas, is modulated by emotional content. Thus, just as visual attention is known to modulate responses in primary visual cortex (Ghandi et al., 1999; Martinez et al., 1999; Somers et al., 1999), so does the affective significance of an item. It should also be noted that our findings of robust emotional modulation in V1–V4, and especially in V1 (Fig. 8), may have been tied to using a task that used stimuli known to effectively engage early visual cortex (Marcelja, 1980; Pollen et al., 1988). Future studies are needed to investigate how facial expressions (Vuilleumier et al., 2001) and emotional scenes (Mourão-Miranda et al., 2003) may modulate early visual cortex by mapping retinotopically organized cortex and by carefully linking brain and behavior.

A potential concern with the present findings is that increased responses in visual cortex may have reflected an unspecific arousal response to the CS+ stimulus. Arousal per se would be expected to generate a main effect of condition (CS+ vs CS−) but to not generate a statistical interaction. Contrary to these predictions, our results exhibited a different pattern. For instance, CS+ hit trials evoked larger responses than correct reject trials, but such difference was strongly attenuated for CS− trials, leading to significant statistical interactions in all early visual areas investigated. Thus, although a near-significant effect of condition was observed in, for example, area V1 (p = 0.07), the main effect was entirely driven by the CS+ hit condition (Fig. 3). In other words, CS+ stimuli did not generate unspecific increases in visual activation, as would be expected for an unspecific arousal-related response. Furthermore, condition effects were not observed in the fovea or high-eccentricity control analyses. Interestingly, analyses in additional ROIs also did not show evidence of condition effects. For instance, in “late” visual cortex (ITG), we observed a main effect of target (present and absent) and some indication of an effect of accuracy. Unspecific arousal would have been more consistent with a robust condition effect, namely, stronger responses across different CS+ trial types. Finally, the RT pattern was also not suggestive of a nonspecific arousal effect given that the speeding up of RT was observed only during CS+ hit trials (and not during CS+ correct rejects or misses). Overall, we suggest that increased activation in visual cortex reflected the affective significance of the stimulus, which was manifested as improved behavioral performance for CS+ stimuli.

A related concern is whether our results reflected an increase in visual attention. Given that the auditory cue at the beginning of the trial indicated whether it was a CS+ or a CS− trial, subjects may have simply increased attention after the auditory cue. Indeed, the type of improvement in behavioral performance observed during the CS+ condition has been suggested to be one of the functions of attention (Carrasco, 2006). However, fMRI responses in the IPS, a structure important in attentional control (Kastner and Ungerleider, 2000; Corbetta and Shulman, 2002), were not increased for all types of CS+ trials (i.e., a main effect of condition was not observed), which would be expected if participants simply paid increased attention after an auditory cue signaling a CS+ trial. We thus suggest that increased selective attention, at least of the kind that reliably engages the IPS, does not explain our results.

There are at least two additional mechanisms that may have subserved our findings. First, it is possible that changes in retinotopically organized visual cortex occurred during conditioning, possibly altering receptive field properties, resulting in improved behavioral performance and increased visual responses. For example, training produces conditioning-specific receptive field plasticity in the primary auditory cortex of guinea pigs (Bakin and Weinberger, 1990), and receptive field tuning is also found in rats (Weinberger, 1995). Thus, classical conditioning to a tone retunes receptive fields, increasing responses to the CS+ while reducing responses to non-CS+ frequencies. Second, it is possible that no changes in visual cortex per se occurred, but that enhanced “functional coupling” between the amygdala and visual cortex took place during the detection phase of the experiment. Increased coupling between the amygdala and the fusiform gyrus has been reported during the perception of fearful faces (Pessoa et al., 2002c), and patients including amygdala lesions show less evidence of increased responses in visual cortex during the viewing of fearful faces (Vuilleumier et al., 2004). Additional studies are needed to further elucidate the mechanisms that endow participants with superior behavioral performance when confronted with affectively significant stimuli.