Neural Correlates of Perceptual Choice and Decision Making during Fear–Disgust Discrimination

Abstract

In the present study, we tested the hypothesis that brain activation would reflect perceptual choices. To probe this question, we used functional magnetic resonance imaging (fMRI) during a challenging fear–disgust, two-choice discrimination task. We investigated how moment-to-moment fluctuations in fMRI signals were correlated with perceptual choice by computing a choice probability index that quantified how well behavioral choice could be predicted by single-trial fMRI amplitude. Our analyses revealed that reporting a neutral face as “fearful” was associated with activation in a broad network of brain regions that process emotionally arousing stimuli, whereas reporting a neutral face as “disgusted” was associated with activation in a focused set of sites that included the putamen and anterior insula. Responses predictive of perceptual reports were not only observed at the group level but also at the single-subject level. Thus, voxel-by-voxel fluctuations in fMRI amplitude for an individual participant could be used to reliably predict the perceptual choice of individual trials for that subject. In addition to the investigation of choice, we also isolated the neural correlates of decision making per se by using reaction time as an index of decision processes. Overall, our findings revealed that brain responses dynamically shifted according to perceptual choices. In addition, the neural correlates of decision making involved at least the anterior cingulate cortex, middle frontal gyrus, and inferior frontal gyrus/insula, consistent with recent proposals that decisions may emerge from distributed processes.

Introduction

One strategy to probe neural mechanisms involved in perceptual decision making has been to record neuronal firing from cells in sensory cortex while a monkey makes difficult perceptual decisions. In one study, Newsome et al. (1989) determined the activity of directionally selective neurons in the visual middle temporal area MT while the monkey decided about the direction of motion of an array of moving dots (Newsome et al., 1989). To probe how cell responses were related to perceptual decisions, they determined whether activity would predict perceptual choice (e.g., left vs right) even in trials in which there was no net motion in any direction, thus eliminating potential stimulus-related confounds. Their findings indicate that the activity of single cells does indeed predict choice in a statistically significant manner (Britten et al., 1996).

Humans frequently make perceptual decisions about facial expressions, such as when determining whether a face is “positive” or “negative.” Although this type of perceptual decision is pervasive in everyday life, little is known about the associated neural correlates of both perceptual choice and decision making. Previous work on the perception of emotional faces has focused on how the physical properties of the stimuli affect brain responses. Such work has often emphasized the idea that facial expressions (especially fear) are processed in a primarily stimulus-driven manner. The present investigation focused instead on how fluctuations in perceptual choices were correlated with brain responses when stimulus valence was fixed (i.e., neutral faces were viewed). In the present study, we used functional magnetic resonance imaging (fMRI) during a challenging fear–disgust, two-choice discrimination task. We tested the hypothesis that brain activation would reflect the perceptual report, such that reporting “fearful” or “disgusted” when viewing a briefly presented neutral face would be associated with distinct patterns of brain activation. Specifically, we investigated how moment-to-moment fluctuations in fMRI signals were correlated with perceptual choice by computing a choice probability index that quantified how well choice could be predicted by single-trial fMRI amplitude (Pessoa and Padmala, 2005). In this manner, we probed how the trial-by-trial fluctuation in fMRI signals reflected the participant's perceptual decision. In particular, much like in the monkey physiology work, we determined whether single-trial fMRI data would support robust prediction of a challenging perceptual decision even when no physical stimulus difference occurred, namely, when the face was neutral (see Fig. 1A), thus eliminating potential stimulus-based confounds.

Figure 1.

A, Example of the graded emotional facial expressions used in the task ranging from 100% disgusted to 100% fearful. B, Representative behavioral response pattern of a subject showing the rate of fearful responses (y-axis) as a function of graded stimulus level (x-axis). The overall pattern exhibits a sigmoidal shape that was significantly fit by a Weibull function (p < 0.05); the filled black dot depicts the 50% point of the fitted curve. C, Group activation map for the contrast of 100% fearful/disgusted stimuli versus baseline. The task robustly activated regions along occipitotemporal cortex, including bilateral fusiform gyrus, as well as the amygdala, bilaterally. The bar plots show activation strength in the amygdala for different levels of graded stimuli (the white arrows indicate the corresponding left/right amygdala of origin). Note that no significant differences were observed for fearful and disgusted stimuli at corresponding stimulus levels (e.g., 100% disgusted and fearful). The level of the slice is indicated according to the MNI template. L, Left; R, right.

Perceptual choices reflect the outcome of decision-making processes that are believed to weigh the evidence in favor of possibly competing “perceptual hypotheses” (Gold and Shadlen, 2001). To investigate the neural correlates of decision making per se during the fear–disgust discrimination task, we reasoned that reaction time (RT) would provide an index of the engagement of decision processes (Luce, 1991). In general, however, RT can be decomposed into independent contributions of decision and nondecision components. To minimize the contribution of the latter, we required that decision-related activation meet several criteria, including that trial-by-trial RTs be correlated with trial-by-trial fMRI amplitude when unspecific signal contributions attributable to attention and/or task difficulty were factored out.

Materials and Methods

Subjects.

Twenty-five volunteers (mean ± SD age, 23 ± 3 years; 15 females) participated in the study, which was approved by the Institutional Review Board of both Brown University and Memorial Hospital of Rhode Island. All subjects were in good health with no past history of psychiatric or neurological disease and had normal or corrected-to-normal vision. All participants gave informed written consent.

Stimuli and behavioral task.

Thirty-three instances of identity-matched fearful, neutral, and disgusted faces (11 for each expression) were used from the Ekman set. For every individual identity, a complete graded-stimulus series was generated by morphing the neutral face with the 100% (original) prototypical fearful face and, separately, with the 100% (original) prototypical disgusted face. Two intermediate levels between emotional and neutral faces were generated (37 and 75%). Morphing was obtained by using the Abrosoft (Beijing, China) FantaMorph software package. Thus, for every identity, a graded-stimulus series involved seven stimuli, starting with the 100% disgusted face, passing through the neutral face, and ending in the 100% fearful face, as indicated in Figure 1A. A total of 77 stimuli were used (11 identities × 7 stimuli per series).

Subjects performed a two-choice task in which they were asked to indicate on each trial whether a briefly presented face was either fearful or disgusted. They were instructed that some of the stimuli had rather weak emotional expressions. Subjects were encouraged to be accurate in their judgments but to respond “reasonably fast.” In our slow event-related design, each trial began with a fixation cross shown for 1000 ms on a black background, followed by the presentation of a face for 70 ms, followed by a blank screen for 14,100 ms (overall trial length was ∼15 s). Subjects indicated “fearful” or “disgusted” via button presses, which were counterbalanced across subjects (13 of 25 used the right and left thumbs for fearful and disgusted responses, respectively; the reverse mapping was used in 12 of 25 subjects). Each of the 77 faces was presented three times, except for neutral faces, which were shown six times each. Altogether, each subject performed 264 trials. Overall, neutral-face trials were preceded by fear or disgust-containing trials nearly equally often. Thus, fearful or disgusted reports (and the associated neural correlates) were not affected systematically by sequence effects.

MR data acquisition.

MR data were collected using a 1.5 tesla Symphony Magnetom scanner (Siemens Medical Systems, Erlangen, Germany). Each subject performed eight experimental runs, each lasting 8 min and 36 s. During each run, 236 gradient-echo echo planar imaging volumes were acquired with a repetition time (TR) of 2170 ms and an echo time (TE) of 38 ms. A single volume consisted of 27 axial slices with a thickness of 4 mm and an in-plane resolution of 3.8 × 3.8 mm (240 mm field of view). At the end of each scanning session, a high-resolution magnetization-prepared rapid-acquisition gradient echo anatomical sequence was performed (TR, 1900 ms; TE, 4.15 ms; inversion time, 1100 ms; 1 mm isotropic voxels; 256 mm field of view).

General fMRI data analysis.

Preprocessing steps used AFNI tools (http://afni.nimh.nih.gov/afni), unless indicated otherwise. The first three volumes of each functional run were discarded to account for equilibration effects. The remaining volumes were slice-time corrected and spatially registered (for motion correction) to the volume acquired closest in time to the particular subject's high-resolution anatomy. Both the anatomical and functional data were then normalized to Montreal Neurological Institute (MNI) space using the BET and FLIRT tools from the FSL package (http://www.fmrib.ox.ac.uk/fsl/). Subsequently, the functional volumes were spatially smoothed with an 8 mm (full-width half-maximum) Gaussian filter. Finally, for each voxel, signal intensity was scaled to a mean of 100.

Standard fMRI analysis.

Two types of fMRI data analysis were performed: standard analysis using the general linear model (GLM) and receiver operating characteristic (ROC) analysis (see below). The goal of the GLM analysis was to identify voxels whose time series were robustly modulated by general task demands (i.e., regardless of trial type or other factors). Each subject's data were analyzed using a standard GLM model that included constant and linear terms for each run (as covariates of no interest) to control for drifts of the MR signal. In addition, when a head-motion parameter was significantly correlated with the experimental conditions (tested via regressing the motion parameter on the regressors linked to the experimental conditions; p < 0.05), it was included as an additional covariate of no interest. At the same time, seven regressors of interest were defined according to the emotional content of the face stimuli (100, 75, and 37% fearful and disgusted, as well as neutral). A two-stage mixed-effects analysis was performed in which regression coefficients for each condition (fixed factor) were tested across subjects (random factor) via t tests. To identify voxels robustly modulated by the task, they were required to survive a p < 0.001 threshold and a 26-voxel extent (corresponding to p < 0.05 at the cluster level as determined by permutation testing). The objective of this initial analysis was to constrain the search space in the subsequent ROC analysis (see below) and thus address the multiple-comparisons problem. Only voxels that exhibited robust task-related activation were additionally probed. A second t statistic group map depicting the contrast of the 100% fearful and 100% disgusted faces versus baseline (i.e., rest; threshold of p = 10⁻⁶) was used to explore the brain regions that were involved in the processing of fearful and disgusted expressions during the two-choice discrimination task (Fig. 1C).

Receiver operating characteristic analysis.

To quantify the link between fluctuations in fMRI single-trial amplitude and trial-by-trial perceptual choice (disgusted vs fearful responses) or stimulus condition (100% disgusted vs 100% fearful stimuli), we performed ROC analyses (Pessoa and Padmala, 2005). For each trial, response strength was indexed by averaging the responses around the expected peak (time points at 4.3 and 6.5 s after stimulus onset) and subtracting an estimate of “baseline” activation (average of time points −2.2, 0, 15.2, and 17.4 s relative to stimulus onset). Such a method of determining response strength was not critical for the ROC analysis, and other methods of relative strength would have been suitable as well (e.g., fitting individual trials with the canonical hemodynamic response). Trial amplitudes for two conditions of interest (e.g., trials in which the subject reported fearful and trials in which the subject indicated disgusted) were then used to construct ROC curves. The corresponding trials of the two conditions may be thought of as the signal and noise distributions in signal-detection theory (SDT) (for additional methods, see supplemental data and Fig. S1, available at www.jneurosci.org as supplemental material).

To determine brain regions in which trial-to-trial fluctuations in fMRI responses were correlated with perceptual choice, we analyzed trials in which subjects viewed neutral faces only. ROC analysis was used to compare fMRI amplitude during trials in which subjects reported fearful relative to those trials in which they reported disgusted. We computed a choice probability index at every voxel. For voxels whose fearful responses were greater than to disgusted responses (i.e., the fearful distribution was to the right of the disgusted distribution), the area under the ROC curve would be >0.5. We called such voxels predictive of fearful responses. When the opposite was true, namely fearful responses were smaller than disgusted responses (i.e., the fearful distribution was to the left of the disgusted distribution), we called such voxels predictive of disgusted responses. In such cases, the area under the ROC curve would be <0.5 (see also Fig. S1, available at www.jneurosci.org as supplemental material).

For consistency, to investigate brain regions in which fMRI responses were stimulus responsive, we performed an analogous analysis that compared activations between 100% fearful and 100% disgusted stimuli; in this context, we call the index “stimulus probability.” Thus, choice and stimulus maps were generated by computing ROC-derived indexes based on two specific trials types. In all cases, group activation maps were subsequently determined by testing if such indexes deviated significantly from 0.5 using a t test across subjects in a second-level analysis. Group maps were then thresholded at p < 0.05, and a 10-voxel extent cluster size was required. Note that, to address the multiple-comparisons problem, only task-related voxels were considered (see above). In addition, the robustness of activation clusters observed in group maps was further investigated at the region of interest (ROI) level (see below).

Because participants indicated perceptual decisions via a button press, we controlled for any potential hand-specific effects via a masking procedure. To do so, in all cases, we compared the maps of the two groups of subjects (13 of 25 used the right and left thumbs for fearful and disgusted responses, respectively; the reverse mapping was used in 12 of 25 subjects) by performing a t test. Voxels that exhibited hand-specific differences were included in the mask and were thus excluded from additional analysis. In this process, we used a threshold of p = 0.1 (uncorrected) such that even very weak evidence of hand-specific effects eliminated the voxel from additional consideration. By doing so, we aimed to eliminate potential contamination of hand-specific responses when reporting perceptual choice (which was indicated via a button press).

Choice-related analyses focused on neutral faces only and included 19 subjects for whom fearful or disgusted responses did not exceed 70% of the total trials, thus ensuring a sufficient number of trials for both response outcomes. For stimulus-related analyses, all 25 subjects were used.

We performed an additional analysis of choice-related activation in which RTs were equated for both reported fearful and reported disgusted trials. To equate RTs, for every subject, we iteratively removed reported fearful and reported disgusted trials until no significant difference (p > 0.1) in RT for that individual was observed (trials were removed by starting with the fastest and slowest RTs and continuing, if needed, by considering the next fastest and slowest trials). In general, RTs were equated by removing very few trials (zero to eight trials). The results of such analysis (Table 1) thus cannot be attributed to differences in RT between the two trial types.

Table 1.

Neural correlates of perceptual choice

	MNI coordinates				Size	Peak t value (df = 17)	Median CP (p value)		Min/max CP (p value)
Regions predictive of fearful choices
Superior parietal gyrus	Left	−7	−74	45	283	5.88	0.55	(0.002)	0.72	(0.004)
Superior temporal sulcus	Right	58	−59	14	90	4.11	0.55	(0.003)	0.69	(<0.001)
Inferior frontal gyrus	Right	44	19	14	113	4.50	0.55	(0.002)	0.70	(0.003)
Anterior insula	Right	49	22	−10	99	4.47	0.55	(<0.001)	0.73	(<0.001)
Anterior cingulate cortex	Bilat.	−2	32	26	122	4.43	0.56	(0.002)	0.69	(<0.001)
Orbitofrontal cortex	Left	−26	41	−18	15	4.32	0.56	(0.002)	0.66	(0.002)
Orbitofrontal cortex	Right	29	39	−19	79	6.41	0.55	(<0.001)	0.71	(<0.001)
Inferior frontal gyrus (anterior)	Right	41	48	0	124	4.33	0.55	(0.002)	0.72	(<0.001)
Middle frontal gyrus	Left	−35	56	18	43	4.97	0.55	(<0.001)	0.73	(<0.001)
Thalamus	Bilat.	−1	−15	15	102	4.28	0.55	(0.002)	0.69	(<0.001)
Regions predictive of disgusted choices
Supramarginal gyrus	Right	43	−28	33	37	−4.08	0.44	(0.005)	0.33	(0.007)
Precentral gyrus	Right	24	−17	64	41	−3.55	0.45	(0.003)	0.34	(0.006)
Putamen / Insula	Right	28	−10	10	62	−5.25	0.45	(<0.001)	0.32	(0.003)

Cortical regions are listed according to anteroposterior dimension. Repeated names refer to distinct sites within the same structure. Bilat., Site along the midline with bilateral activation; CP, choice probability.

ROI analyses.

In addition to group analyses at the voxel level, we investigated choice probability values at the ROI level. Because our goal was to further probe the strength and reliability of choice-related activations, we focused our ROI investigation on a set of regions that exhibited clustered choice-related activation at the group level. ROIs were generated by centering a sphere of 8 mm radius around cluster centers of choice-related activation observed on the group map. Voxels that exhibited robust task-related activation (p < 0.001, cluster size of 26 voxels) and choice-related activation (p < 0.05, cluster size of 10 voxels) were considered below for additional ROI analyses.

To further characterize the link between fMRI responses and behavioral choice, we summarized both median and minimum/maximum choice probability values across ROIs (Table 1). For each ROI, we determined the median choice probability value. Values >0.5 indicated that an ROI was predictive of fearful choices and values smaller than 0.5 indicated that an ROI was predictive of disgusted choices. We then tested whether the medians differed significantly from the chance level of 0.5 via a Wilcoxon's signed-rank test (p values are given in Table 1). In addition, for each ROI, we determined the voxel with the largest choice probability value that was predictive of fearful responses [CP_max(fearful)] and the voxel with the smallest choice probability value that was predictive of disgusted responses [CP_min(disgusted)], i.e., the voxel that was most predictive of disgusted responses. For each ROI, we then compared the deviations of such maximum and minimum values from 0.5 via a Wilcoxon's signed-rank test (p values are given in Table 1); specifically, we compared [CP_max(fearful) − 0.5] with [0.5 − CP_min(disgusted)], across subjects. Here, our goal was to test whether the most predictive voxels exhibited consistency in their prediction of behavioral choice. For instance, a voxel that was predictive for fearful reports (high values of [CP_max(fearful) − 0.5]) in some subjects but predictive for disgusted reports (high values of [0.5 − CP_min(disgusted)]) in other subjects would exhibit comparable deviations from 0.5 across subjects and thus would not pass this test. Finally, for each ROI, we also counted the number of voxels that were significantly predictive of fearful responses (as determined by permutation testing; see below) and the number of voxels that were significantly predictive of disgusted responses. For each ROI, we then compared these two numbers across subjects via a Wilcoxon's signed-rank test. All of the regions listed in Table 1 exhibited significantly different numbers of predictive voxels, indicating that activation in the region was linked to a particular perceptual choice. Overall, the goal of the above tests was to further characterize and test whether the selectivity (fearful or disgusted choice) of the predictive signals in these ROIs was sufficiently robust. For consistency, we performed analogous tests for stimulus-driven activation by testing stimulus probability values (Table 2).

Table 2.

Stimulus-related activations

	MNI coordinates				Size	Peak t value (df = 23)	Median SP (p value)		Min/max SP (p value)
Regions driven by 100% fearful faces
Lingual gyrus	Left	−4	−90	−5	16	2.95	0.54	(0.016)	0.67	(0.004)
Precuneus	Right	15	−55	39	24	3.73	0.54	(<0.001)	0.63	(<0.001)
Fusiform gyrus	Left	−33	−51	−14	22	4.65	0.55	(0.001)	0.64	(0.002)
Entorhinal cortex	Right	34	−18	−19	29	3.60	0.53	(0.002)	0.63	(0.002)
Anterior insula/inferior frontal gyrus	Left	−42	11	−4	65	4.38	0.54	(<0.001)	0.66	(<0.001)
Anterior insula/inferior frontal gyrus	Right	38	11	2	17	2.89	0.53	(0.005)	0.63	(<0.001)
Inferior frontal sulcus	Right	36	20	33	27	4.00	0.54	(0.002)	0.63	(<0.001)
Orbital gyrus	Right	27	54	8	34	3.56	0.54	(0.003)	0.69	(<0.001)
Thalamus	Right	13	−9	7	24	3.93	0.53	(0.001)	0.62	(<0.001)
Caudate	Left	−14	10	15	33	4.30	0.54	(<0.001)	0.64	(0.005)
Regions driven by 100% disgusted faces
Middle occipital gyrus	Right	53	−70	6	71	−3.58	0.46	(0.012)	0.28	(0.001)
Intraparietal sulcus	Left	−24	−67	44	48	−3.96	0.47	(0.006)	0.37	(0.003)
Middle temporal gyrus	Right	54	−38	−9	21	−2.99	0.47	(0.011)	0.39	(0.009)

SP, stimulus probability.

As stated previously, ROIs were selected such that they exhibited choice probability values that were consistently different from 0.5 across participants. Although this result is informative, it would be possible to meet this criterion with modest choice probability values in the, say, 0.51–0.53 range. Another important goal of our ROI-based analysis was to ask whether the time series from an individual voxel from an individual participant could be used to reliably predict behavioral choice on a trial-by-trial basis, much like an individual neuron whose responses are significantly predictive of the monkey's trial-by-trial choices (Britten et al., 1996). Thus, individual voxels from the candidate ROIs were assessed for the significance of their choice predictability. To do so, we applied a permutation test (Efron and Tibshirani, 1993) to compare the choice probability value of an individual's voxel with the chance expectation of 0.5. For each voxel, the critical value was calculated by randomly reassigning the fMRI amplitude of each trial to one of the two perceptual choices (fearful or disgusted), thus disrupting any correlation between fMRI responses and choices but leaving the two distributions unchanged. For each voxel, we generated the distribution of choice probabilities expected in the absence of any association from 2000 permutations (i.e., the null distribution). Observed choice probability values were deemed statistically significant if they lay outside the central 95% of the null distribution. In addition, for the ROI analysis, suprathreshold voxels were counted for reported fearful and reported disgusted trials and compared across subjects, as indicated in the previous paragraph.

Decision-related activation.

To probe the neural correlates of decision processes, we correlated RT with fMRI amplitude. For the main analysis, trial RT and single-trial fMRI amplitude were correlated after mean values were removed (mean removal was performed in a condition-specific manner). Such correlations were performed by either considering all seven trial types or, separately, considering neutral faces only. Group maps were obtained by applying a t test on the correlation values after they were Fisher's z transformed. A separate correlation map was also computed based on the overall mean RT pattern to probe for voxels whose responses exhibited an inverted-U pattern (see Fig. 5, insets). As described in text, voxels were considered decision related if they exhibited significant mean-corrected correlations for all seven conditions and for the neutral condition and exhibited the general inverted-U pattern. Each criterion was required to be significant at the 0.05 cluster level (p < 0.001, cluster size of 26 voxels), and only task-related voxels were considered (which were significant at the 0.05 cluster level, as previously described).

Figure 5.

Neural correlates of decision making. The correlates of decision making were isolated by using RT as an index of decision processes. Activated voxels met our criteria for decision-related activation (see Results). In the ACC and the anterior insula, the average fMRI response as a function of condition exhibited an inverted-U pattern that closely matched the pattern of average RTs (see insets). The levels of the slices are indicated according to the MNI template. L, Left; R, right.

Results

To investigate the link between perceptual decisions and fMRI responses, we performed an ROC analysis (Green and Swets, 1966). In the present context, an ROC analysis can quantify the extent to which an ideal observer, based on single-trial fMRI amplitude, would be able to determine whether that trial was associated with one of two events, namely, reported fearful or reported disgusted trials (see Materials and Methods). In particular, the area under the ROC curve provides an index, called choice probability (Britten et al., 1996), with which to quantify the separation between the fMRI response distributions for reported fearful and reported disgusted trials. As illustrated in Figure S1 in the supplemental data (available at www.jneurosci.org as supplemental material), when the two response distributions are partly, but not completely, overlapping, the choice probability will differ from 0.5. When this difference is significant, an ideal observer would be able to reliably (i.e., better than chance) classify the trial type based on fMRI response amplitude. Note that, if the two distributions are completely overlapping, the choice probability will be 0.5 (no relationship between fMRI signal strength and the two event types), and, if the two distributions do not overlap, the choice probability will be 1.0 or 0.0 (in the latter cases, perfect predictability would be possible).

Behavioral results

Performance for a representative individual is illustrated in Figure 1B: the percentage of fearful responses was near zero for prototypical disgusted faces, near 100% for prototypical fearful faces, and exhibited an overall “sigmoidal” shape with ∼40% fearful responses for the neutral face. Such behavior was typical of our subjects, who indicated fearful 40 ± 4% (SD) of the time when neutral faces were presented. Here, we summarize the behavioral results more directly related to the imaging results reported below. In particular, to probe the neural correlates of perceptual choice, we focused the imaging analysis on the neutral faces only. Additional analyses focused on stimulus-driven activation, namely, responses evoked by 100% fearful/disgusted faces. When neutral faces were viewed, subjects were slower to indicate fearful (1347 ± 384 ms) than to indicate disgusted (1231 ± 387 ms); the difference in RT was statistically significant (p < 0.01, paired t test). Reaction time in response to prototypical 100% fearful faces averaged 937 ± 231 and 949 ± 223 ms in response to prototypical 100% disgusted stimuli; RTs did not differ significantly between the two conditions (p = 0.67, paired t test). Accuracy during prototypical fearful faces averaged 95 ± 5% correct and 97 ± 3% correct during prototypical disgusted faces; the number of correct responses did not differ significantly between the two conditions (p > 0.05, Wilcoxon's signed-rank test).

Imaging results

General task activation with 100% stimuli

Initially, we assessed activations evoked while viewing emotional faces by comparing the combined 100% fearful/disgusted conditions relative to baseline (i.e., blank screen). The goal of this analysis was simply to confirm that, during our forced-choice discrimination task, we obtained activations similar to those reported previously during passive viewing or during implicit tasks (e.g., male/female). In our task, viewing emotional faces evoked robust activation throughout occipitotemporal regions, including the fusiform gyrus, superior temporal sulcus (STS), as well as amygdala, insula, and putamen (all bilaterally) (Fig. 1C). Thus, in line with previous studies, both visual regions and regions important for evaluating stimulus significance (such as the amygdala and insula) were engaged during the fear–disgust discrimination task. In addition, we also observed activation in frontoparietal regions, including bilateral intraparietal sulcus, anterior cingulate cortex (ACC), bilateral superior frontal gyrus, bilateral inferior frontal gyrus (IFG), and bilateral thalamus. Finally, we observed bilateral activations around the central sulcus likely related to motor function.

Neural correlates of perceptual choice

A central goal of the present study was to characterize the link between moment-to-moment fluctuations in fMRI signals and perceptual decisions in a quantitative manner. To do so, our analyses focused exclusively on neutral faces. Because only neutral faces were considered, stimulus-related contributions were eliminated, much as done in monkey physiology when neutral stimuli are used to investigate the link between cell responses and choice during visual tasks [e.g., no net-motion stimuli (Britten et al., 1996); zero-disparity stimuli (Dodd et al., 2001)]. For every subject and for every voxel, we computed an index that quantified how well fMRI amplitude predicted behavioral responses on a trial-by-trial basis, namely, the choice probability. To determine the reliability of the results across subjects, we performed a standard two-stage, random-effects group analysis. The first stage corresponded to the individual choice probability values, which were then submitted to a one-sample t test (second stage) that evaluated whether the observed values differed significantly from that expected by chance (0.5) on a per-voxel basis. Resulting group choice probability maps are shown in Figure 2A. Voxels whose signals predicted fearful responses were observed in several brain regions, including the calcarine fissure, superior temporal sulcus, superior parietal gyrus, anterior insula, ACC, middle frontal gyrus (MFG), IFG, orbitofrontal cortex (OFC), and thalamus. The bilateral putamen and the right insula (this site abutted the right putamen) were the most prominent regions that predicted the behavioral choice disgusted. In Table 1, we list the activations that were also observed in additional analyses in which the RTs were equated for both reported fearful and reported disgusted trials. Thus, such activations cannot be attributed to differences in RT between these two trial types. In additional analyses, we confirmed that the group choice probability maps based on neutral faces was nearly identical to group maps obtained when neutral faces were combined with the 37% faces (supplemental Fig. S2, available at www.jneurosci.org as supplemental material).

Figure 2.

Group choice probability map. A, Group choice map based on trials with neutral faces. Voxels whose fMRI signals predicted fearful responses are depicted in red/yellow, and voxels whose fMRI signals predicted disgusted responses are depicted in blue. B, Combined group map depicting choice-related activation (red), stimulus-related activation (blue), and their intersection (green). Interestingly, there was little overlap in the activations, but they neighbored each other in the left anterior insula and the right IFG/IFS. The levels of the slices are indicated according to the MNI template. L, Left; R, right; SPG, superior parietal gyrus.

The above results demonstrate that, at the group level, fMRI signals in several brain regions can be used to predict behavioral choice in a trial-by-trial basis even when faces were neutral. Next, we sought to investigate whether voxel-based fMRI responses were reliably predictive of behavioral choice at the single-participant level. Such analysis seeks to establish whether the time series of an individual voxel can be used to predict behavioral choice and is analogous to investigations in monkey physiology in which choice probability values are assessed for an individual neuron. To test whether choice probabilities were significant at the single-participant level, for each voxel, we used a permutation test that assessed whether choice probabilities differed significantly from 0.5 (see Materials and Methods). Peak choice probability values (averaged across subjects) were ∼0.7/0.3 for fearful/disgusted predictive regions; for several participants, individual values were more extreme than 0.8/0.2, indicating that single-trial fMRI amplitudes correctly predicted the behavioral response >80% of the time.

To further characterize choice-predictive activations, we performed additional ROI analyses (see Materials and Methods). Overall, our goal was to characterize the selectivity of the predictive signals in these ROIs, namely, whether the ROIs were consistently predictive of fearful or disgusted choices across participants. Thus, we summarized both median and minimum/maximum choice probability values across ROIs. The regions listed in Table 1 were the ones whose predictive voxels were consistently linked to a particular perceptual choice (for additional elaboration of the tests used, see Materials and Methods). Figure 3 illustrates the predictability of the right anterior IFG (average maximum choice probability value of 0.72), a region predictive of fearful choices, and the right putamen/insula (average minimum choice probability value of 0.32), a region predictive of disgusted choices. The ROI analyses revealed that the regions listed in Table 1 reliably exhibited suprathreshold choice probability voxels predictive of fearful or disgusted responses at the single-participant level. The consistency of the results across subjects is further illustrated in Figure 4, which shows the distribution of median choice probability values for the regions listed in Table 1 (one value for each subject). All distributions are clearly skewed to the right or left, illustrating the consistency of the selectivity for both fearful and disgusted reports, respectively.

Figure 3.

Average ROC curves for representative ROIs predictive of fearful and disgusted responses. Average ROC curves for the right anterior IFG and right putamen/insula ROIs. The curves were constructed by determining the most predictive voxel within the ROI for each subject; the ROC curves of these voxels were then averaged across subjects to yield the mean curve. The error bars and gray-level bands depict the region of the mean ± SD.

Figure 4.

Distribution of choice probability values. Median choice probability values (1 value for each subject) for the regions listed in Table 1. Choice probability values (x-axis) >0.5 indicate regions predictive of fearful reports (red), and choice probability values smaller than 0.5 indicate regions predictive of disgusted reports (blue). The y-axis shows the subject count. All distributions are clearly skewed to the right (columns 1, 2) or left (column 3), illustrating the consistency of the selectivity for both fearful and disgusted reports, respectively.

The above analyses excluded voxels that exhibited hand-specific differences (see Materials and Methods). In light of findings suggesting the involvement of the motor system in decision-making processes (Shadlen and Newsome, 2001; Wyss et al., 2004), our approach may have been overly conservative. In additional analyses, when we relaxed our exclusion criterion and did not censor voxels that may have been involved in motor-related processes, we observed three sites in parietal cortex that were modulated by perceptual choice, all of which were relatively superior in the brain (fearful choice: x = −22, y = −25, z = 65; disgusted choice: x = 27, y = −16, z = 66; x = 21, y = −49, z = 58). In addition, we observed sites in motor cortex and cerebellum that were likely involved in hand presses (motor cortex, disgusted choice: x = 36, y = −33, z = 49; cerebellum, fearful choice: x = 7, y = −57, z = −10; x = −9, y = −43, z = −21).

Stimulus-related activations

Given the pattern of choice-related activations, it was informative to probe brain regions that exhibited fMRI signals linked to the physical stimulus. For consistency, we applied the same ROC methodology used above to investigate, in this case, the link between physical stimulus and fMRI responses. In this case, the analysis focused on prototypical 100% fearful and 100% disgusted stimuli. Stimulus-driven regions were observed in visual cortex, in addition to right thalamus, left caudate, bilateral anterior insula, and in the inferior frontal sulcus (IFS) and adjacent sites. Table 2 lists whether these sites were more strongly driven by 100% fearful or 100% disgusted faces. Interestingly, stimulus-related activations in the left anterior insula and right IFS (Fig. 2B, blue) neighbored choice-related activations (red) but did not appear to significantly overlap with them (green). Surprisingly, we did not find stimulus-driven differences in activation in the amygdala (for a similar result, see Fitzgerald et al., 2006). To probe this result further, we determined responses for all stimulus conditions. As illustrated in Figure 1C, amygdala responses were equivalent for fearful and disgusted faces at all levels of our graded stimuli (37, 75, and 100%). It may be the case that differential responses in the amygdala during the viewing of fearful and disgusted expressions are observed more reliably during passive viewing (Phillips et al., 2004) or implicit tasks, such as determining the sex of a face (Phillips et al., 1997). Finally, we found a few voxels in the putamen and adjacent insula that were more strongly driven by disgusted faces relative to fearful ones, but they did not survive the stricter ROI criteria and are not listed in Table 2.

Neural correlates of decision making

The analysis of choice-related activations focused on neutral faces to investigate whether evoked responses were reliably modulated by the perceptual decision. Although such analysis reveals the neural correlates of perceptual choice, it does not necessarily reflect decision processes per se. Here, we reasoned that RT would provide an index of decision-making processes that could be meaningfully used to probe the associated neural correlates. As expected, the average RT exhibited an inverted-U shape, with the maximum RT occurring when subjects viewed neutral faces and minimum RTs observed for the 100% prototypical expressions (Fig. 5, insets).

In general, it is possible to conceptualize RT as composed of decision and nondecision (also referred to as residual) components (Luce, 1991). A common assumption is to consider these two components as additive independent contributions to RT (Palmer et al., 2005). Residual RT includes several subprocesses, including those involving perceptual processing and motor initiation. In the present context, because we used a single task, we suggest that such residual contributions were relatively constant across conditions; in fact, it could be argued that the residual RT was <960 ms, which was the RT for the least ambiguous conditions (100% fear/disgust stimuli).

One approach to probe the neural correlates of decision making would be to correlate average RT per condition with the average fMRI response for those conditions. In other words, voxels whose average responses followed the inverted-U pattern shown in Figure 5 (insets) would be potentially linked to decision making. One concern with such an approach, however, is that RT would be expected to be modulated by additional “high-level” factors such as attention and/or task difficulty. Thus, observed correlations of mean RT and fMRI responses could be primarily driven by these factors. To control for these extraneous contributions, we (1) correlated RT with fMRI amplitude on a trial-by-trial basis and (2) removed the mean RT and the mean fMRI amplitude for each stimulus condition separately (by subtracting them out). In other words, only trial-by-trial fluctuations around the (condition-specific) mean were correlated. In this manner, overall attentional and/or task difficulty contributions were essentially eliminated. In addition, we reasoned that decision processes engaged by the task should be observed for the neutral-face condition, too. In fact, fluctuations in RT while viewing neutral faces should primarily reflect decision process because both the task and the stimulus were kept constant. Thus, we required that, in addition to conditions 1 and 2 above, (3) decision-related voxels exhibit significant trial-by-trial RT-fMRI correlations when only the neutral-face condition was considered (again, only mean-removed values were used). Finally, we required that (4) decision-related voxels exhibit an inverted-U pattern when the average RT and the average fMRI responses were correlated. In this way, collectively, fMRI responses followed the pattern of RT values, but significant correlations were not spuriously driven by high-level factors.

Voxels that simultaneously met criteria 1–4 above were considered to reflect decision processes and were observed in the ACC, MFG, and IFG/anterior insula (Fig. 5, Table 3). We also probed the intersection of decision-related activation with activation that was predictive of perceptual choice. Interestingly, sites in the ACC, right MFG, and right IFG/anterior insula not only reflected decision processes but also expressed the ultimate effects of the decision (i.e., choice). Note that, because our criteria for decision-related activation was somewhat restrictive, we may have failed to observe other important sites, such as the posterior dorsolateral prefrontal cortex (DLPFC) (Heekeren et al., 2004, 2006).

Table 3.

Neural correlates of decision making

Brain region	MNI coordinates
Anterior cingulate cortex	Bilat.	0	20	40
Middle frontal gyrus	Left	−43	21	20
Inferior frontal gyrus/anterior insula	Left	−37	24	−2
Inferior frontal gyrus/anterior insula	Right	35	23	−5

Bilat., Site along the midline with bilateral activation.

Discussion

In the present study, we investigated the neural basis of perceptual decision making during a challenging fear–disgust discrimination task. First, we probed the link between trial-by-trial fluctuations in perceptual choice and fluctuations in fMRI single-trial response amplitude. To do so, we used an ROC analysis that quantified such association. Our analyses, which focused on neutral stimuli only (but see supplemental data, available at www.jneurosci.org as supplemental material), revealed a remarkable shift in the pattern of brain activation depending on the subject's perceptual choice. At the group level, fearful reports were associated with responses in a broad network of brain regions, whereas disgusted reports were linked with responses in a focused set of sites. Functional MRI responses predictive of perceptual reports were not only observed at the group level but also at the single-subject level. Voxel-by-voxel fluctuations in fMRI amplitude for an individual participant could be used to reliably predict the behavioral choice of individual trials for that subject. In addition to the investigation of choice, we also isolated the neural correlates of decision making during the fear–disgust discrimination task by using RT as an index of decision processes.

Dynamic shift in activation depending on perceptual choice

The present investigation focused on how moment-to-moment fluctuations in perceptual choices were correlated with brain responses when stimulus valence was fixed (i.e., neutral-face condition). Our findings revealed that brain responses dynamically shifted according to subjective reports.

During reported fearful trials, perceptual choice modulated fMRI responses in the superior parietal gyrus, ACC, MFG, IFG, OFC, anterior insula, and thalamus. Why does reporting fearful engage such broad network of regions? We tentatively suggest that the conscious decision that a fearful face is viewed is represented across a network of brain regions that prepare the organism to appropriately handle emotionally challenging stimuli and that regulate the associated emotional response [for comparison with our previous study (Pessoa and Padmala, 2005), see supplemental data, available at www.jneurosci.org as supplemental material]. Thus, both regions important for attention and regions important for affective evaluation and regulation would be expected to be engaged. For instance, the insula is a complex structure with important autonomic and other affective functions (Mesulam, 2000). The OFC has important functions in stimulus evaluation and is believed to be involved in determining the biological significance of a stimulus (Rolls, 2000). The ACC is believed to participate in both cognitive and affective functions (Bush et al., 2000). The IFG has been proposed to be a node within a network of regions that process emotionally arousing visual stimuli (Yamasaki et al., 2002). The MFG is known to be important for working memory processes and attention. The superior parietal gyrus and thalamus are important for attentional processes, too (Kastner and Ungerleider, 2000). Our results do not imply, however, that the entire network is directly responsible for deciding that a fearful face is present. Although some of the observed regions may have been more directly involved in perceptual processes per se (especially the voxels that overlapped with decision making), fearful-predictive activations should be viewed as involving a more general set of processes that are associated with the conscious decision that fear is present. Such activations likely included several components that could be described as behavioral or physiological (e.g., autonomic, facial expression generation, and the manner in which gaze is directed). Therefore, the observed dynamic shift in activation contingent on perceptual choice should be viewed as a general change in “brain state” that encompasses several perceptual, cognitive, and affective dimensions.

Many of the regions activated during reported fearful trials, notably the anterior insula, ACC, thalamus, and OFC, are centrally involved in the generation of “visceral” reactions and modulation of bodily arousal (e.g., changes in heart rate and skin conductance) that may contribute to the embodied component of an emotional reaction (Critchley et al., 2000, 2003, 2005). In addition, activity in the right anterior insula has been shown to reflect explicit awareness of interoceptive information, suggesting that this region supports a representation of visceral responses accessible to awareness (Critchley et al., 2004). Accordingly, in the present experiment, the anterior insula may have been important in the actual generation of the perceptual choice.

Another group of brain regions exhibiting choice-related activation predictive of fearful reports involved the visual cortex, including the STS, fusiform gyrus, and the calcarine fissure (the latter two regions did not survive the strict criteria used in our ROI analysis). Our findings indicate that fMRI responses in these regions can be modulated by perceptual interpretation even when no valence differences occur. In this context, it is interesting to note that ∼90% of the cells in STS and inferotemporal cortex in the monkey reflect the perceptual state of the animal during binocular rivalry (Sheinberg and Logothetis, 1997). In addition, in humans, activity in the fusiform gyrus is modulated by stimulus visibility, awareness, and explicit object recognition (Tong et al., 1998; Bar et al., 2001; Pessoa et al., 2006).

As stated above, during disgusted reports, we observed relatively focused activation in bilateral putamen and right insula. Both human lesion and neuroimaging studies have implicated the basal ganglia and the anterior insula in the perception of disgust (Calder et al., 2001). For instance, lesions of these sites preferentially impair the recognition of disgust relative to other facial expressions, including fear. Again, our findings suggest that responses in these areas are not only modulated by the physical stimulus but also by perceptual interpretation.

Our findings revealed that brain responses dynamically shifted according to perceptual choice. Does this mean that subjects really “saw” the neutral stimulus as fearful or disgusted? Within an SDT type of framework, a subject's answer results from both perceptual and decision processes (Green and Swets, 1966). Thus, the types of activation that we observed could more closely reflect either decision processes or perceptual processes. Although in the present experiment we cannot, strictly speaking, separate the two types of processes, it is noteworthy that regions commonly viewed as “perceptual,” such as the STS, fusiform gyrus, and even to some extent the calcarine fissure, were modulated by perceptual choice. More generally, although SDT models often assume that “percepts” and “decisions” are independent, in general, they may interact such that they are more inextricably bound (Grossberg, 1980; Churchland et al., 1994; Goldstone et al., 2000). In this latter framework, differences in activation should not be viewed as strictly perceptual or decision related, but instead, as resulting from the interplay between perceptual and decision processes.

Decision making

Gold and Shadlen (2001) have proposed that a key mechanism of perceptual decision making involves evidence accumulation, such that decision-making circuits compute something like the time integral of the activity “read” from visual cortex. The work of Romo and Salinas (2003) has investigated related questions in the tactile domain. In a recent study, Heekeren and colleagues showed that activity in human DLPFC covaried with the difference signal between face- and house-responsive regions in ventral temporal cortex in a face/house categorization task (Heekeren et al., 2004) and suggested that such difference signals may underlie a general mechanism for perceptual decision making (Heekeren et al., 2006). In another study, Binder et al. (2004) investigated the neural correlates of decision processes in auditory perception. Participants discriminated two synthesized speech syllables, /ba/ and /da/, which differed only in minor ways. To probe the neural correlates of decision processes, they used RT as an index of decision making and searched for voxels whose time series exhibited significant correlations with RT. Their results suggested that the anterior insular and adjacent frontal opercular cortex were involved in decision processes.

Here, we also reasoned that RT would provide an index of decision processes. However, in principle, RT could also be modulated by other factors, including attention and/or task difficulty. Thus, we searched for decision-making voxels by correlating per-condition mean-corrected RTs and fMRI amplitude. When such an approach is adopted in the context of a single task and a single stimulus type (neutral faces), we propose that RT provides a more direct index of decision processes. Voxels in the ACC, MFG, and IFG/anterior insula met the criteria for decision-related activation. As reviewed above, these regions are involved in important cognitive and affective functions. Our findings suggest that these regions are also engaged in perceptual decision-making processes. In this context, the MFG has been shown to contribute to perceptual decisions during motion-direction discrimination (Kim and Shadlen, 1999). Additionally, as stated, the anterior insula has been linked to perceptual decision making during phonemic decisions (Binder et al., 2004).

In previous studies by Heekeren et al. (2004, 2006), it was suggested that activation in the left DLPFC was linked to perceptual decision making. The present study suggests, however, that, in our task, the neural correlates of decision making appear to be more distributed and involve at least the ACC, MFG, and IFG/insula. Such a view is consistent with the proposal by Romo and Salinas (2003) that decisions may emerge from highly distributed processes in which the details of a motor plan are specified by sensory information.