Attentional Modulation of fMRI Responses in Human V1 Is Consistent with Distinct Spatial Maps for Chromatically Defined Orientation and Contrast

Abstract

Attending to different stimulus features such as contrast or orientation can change the pattern of neural responses in human V1 measured with fMRI. We show that these pattern changes are much more distinct for colored stimuli than for achromatic stimuli. This is evidence for a classic model of V1 functional architecture in which chromatic contrast and orientation are coded in spatially distinct neural domains, while achromatic contrast and orientation are not.

Introduction

When examining locations or objects in the natural world, we often want to make fine discriminations along different feature dimensions—for example, grating orientation (perhaps to detect a camouflaged animal) or color (“Is it a tiger or a zebra?”). To achieve this, visual attention modulates responses in the neural population most informative for the attended features (Martinez-Trujillo and Treue, 2004). This change can be detected using fMRI (Serences et al., 2009).

Here, we ask whether switching attention between contrast and orientation in the same location produces the same type of neural modulation for achromatic and isoluminant chromatic stimuli. We instructed participants to attend to orientation (AO) or contrast (AC) in either chromatic or achromatic gratings (Gc or Ga), and to detect small changes in the attended dimension (Fig. 1). Regardless of the task, orientation changes and contrast changes occurred with the same average frequency in all experiments. Using a linear classifier, we measured fMRI pattern classification accuracies in V1 for combinations of two stimulus conditions (Gc, Ga) and the two attentional tasks (AO, AC).

Figure 1.

a, Achromatic gratings were presented in a 1 s on, 1 s off design for 16 s followed by 10 s with only a fixation point visible. The red fixation point cued the subject to attend to orientation changes (indicated by “O”) and ignore contrast changes (“C”). b, Isoluminant chromatic gratings were used. The blue fixation point cued subjects to attend to contrast decrements. Orientation and contrast changes are exaggerated for illustrative purposes.

We hypothesized that our ability to classify attentional state might depend on the color of the target because neurons sensitive to chromatic signals are often reported as having a special spatial arrangement in early visual areas. In primate V1, cytochrome oxidase staining reveals patches or “blobs” of neurons (Wong-Riley, 1979; Horton and Hubel, 1981; Humphrey and Hendrickson, 1983) that have strong chromatic tuning and weak spatial frequency and orientation tuning and respond relatively strongly to isoluminant color (Livingstone and Hubel, 1984; Ts'o et al., 1986; Lu and Roe, 2008). In comparison, the intervening “interblob” regions appear to contain cells with stronger orientation tuning and increased sensitivity to achromatic contrast. This segregated pattern of spatiochromatic tuning may also extend to the thick/thin/pale stripes of V2 (Livingstone and Hubel, 1984, 1987; DeYoe and Van Essen, 1985; Shipp and Zeki, 1985; Moutoussis and Zeki, 2002; Federer et al., 2009). However, this model has been questioned by other groups, who find little evidence of chromatic specialization in either the V1 blob regions (Lennie et al., 1990; Leventhal et al., 1995; Friedman et al., 2003) or the V2 thin stripes (Gegenfurtner et al., 1996).

If “blobs” contain cells that code primarily chromatic contrast and “interblobs” contain a mixture of cells coding chromatic and achromatic contrast, discrimination of either the contrast or orientation of achromatic targets would both depend on cells in the “interblob” region in V1 and the thick or pale stripes in V2. Thus, switching between achromatic orientation and achromatic contrast discrimination tasks might not produce a change in the average neural response pattern in either area. In comparison, for an isoluminant chromatic target, the model predicts that contrast discrimination is based on neurons in the blobs (V1) or thin stripes (V2), while in both areas, spatial orientation discrimination depends on cells in the intervening locations. Switching between the orientation and contrast detection tasks might therefore produce a differential response pattern at the spatial scale of the blob/interblob structures that could be detected using fMRI pattern classification techniques (Sumner et al., 2008; Brouwer and Heeger, 2009; Kriegeskorte et al., 2010; Seymour et al., 2010). If, on the other hand, there is little spatial segregation on the basis of spatiochromatic tuning in V1, selective attention cannot generate the types of voxel-level modulations that drive classification.

Materials and Methods

Subjects.

Six subjects (three males, mean age 32.5 years old) participated in our experiments. All subjects had normal or corrected-to-normal acuity and normal color vision (tested with Ishihara plates) and were experienced psychophysical observers. Fixation stability was tested outside the scanner and subjects could maintain fixation within a radius of 0.25° from the fixation better than 95% of the time. Four of the subjects were naive to the purpose of the experiment. Subjects were screened and consented in accordance with human subject protocols at the University of California, San Francisco (UCSF) and also at the Smith–Kettlewell Eye Research Institute.

Data collection and processing.

fMRI data were collected on a Siemens 3T Tim Trio system at the UCSF Neuroscience Imaging Center using a standard Siemens EPI sequence and 30 functional imaging planes with resolution of 1.7 × 1.7 × 2 mm collected each TR (2 s). Each run contained 156 TRs and a session consisted of at least seven functional runs. Additional T1-weighted “in-plane” anatomical scans were acquired with the same slice prescription as the T2* data to facilitate post hoc alignment to a high-resolution anatomical dataset collected on a separate occasion.

Anatomical segmentation.

High-resolution whole-head anatomical volumes were acquired on each subject to provide a canonical reference frame for subsequent functional datasets and to enable the restriction of functional data to the cortical sheet. Anatomical datasets were acquired using a T1-weighted MPRAGE sequence at an initial resolution of 0.9 × 0.9 × 0.9 mm. Segmentation of the white and gray matter was performed using the Freesurfer 4 “autorecon” script (http://surfer.nmr.mgh.harvard.edu/) followed by manual topology checking using the Stanford “VISTA” toolbox (http://white.stanford.edu/software/). The resulting gray and white matter volumes were used to generate cortical surface meshes for data visualization (Teo et al., 1997).

Functional data preprocessing.

T2* data were postprocessed to remove motion artifacts using the rigid body alignment routine “spm_coreg” from SPM5. Motion-corrected time series were then imported into the Stanford VISTA package and aligned to the high-resolution anatomy. BOLD signal changes were computed as percentage variations around the mean and normalized by the within-voxel SD for use in the classification analysis routines.

Stimulus presentation.

All visual stimuli were presented on a 19 inch LCD screen (NPD1954, Litemax) and viewed at the rear of the scanner bore via a mirror mounted on the headcoil. Gamma lookup tables and spectra for each color channel were calibrated using a spectroradiometer (USB2000, OceanOptics) to ensure output linearity (Brainard, 1989). Stimuli were generated using the Psychophysics toolbox (Brainard, 1997) on an Intel-based Apple “MacPro” system with a GEFORCE video card (Apple). The LCD screen subtended a visual angle of 15° horizontally. Subjects indicated the presence of target events by pressing a button on a magnet-safe button box connected to a fiber-optic response pad (FORP FIU-005, Current Designs).

Stimulus configuration.

The stimulus configuration is shown in Figure 1. Stimuli were annuli (outer, inner diameters = 10°, 4°) containing 1 cycle per degree (cpd) gratings defined by either achromatic or isoluminant red/green contrast. Cone isolating stimuli were computed using a “silent substitution” technique (Estévez and Spekreijse, 1982) based on the spectra of the display device and published measurements of the human cone photoreceptor absorption spectra in the central retina (Stockman et al., 1993). Because our stimuli extended beyond the fovea, and isoluminance differs from subject to subject, the precise ratio of L to M cones required to generate perceptual isoluminance for the chromatic gratings was established psychophysically for each subject using a separate minimum motion adjustment procedure (Anstis and Cavanagh, 1983). Subjects adjusted the ratio of L:M cone contrasts in a 1 cpd (L-M)-cone grating (1° to 5° eccentricity) drifting at 4°/s until it appeared stationary. The average of five settings of these cone ratios was used for the red/green (r/g) fMRI stimuli. The mean CIE coordinates of the endpoints of the isoluminant r/g axis were (r: x = 0.351, y = 0.4314, g: x = 0.280 y = 0.346), corresponding to a mean L:M cone ratio of almost exactly 1:2 (0.4929). The background was maintained at a constant mean gray (x = 0.314, y = 0.329) at a luminance of 31 cd/m² throughout the all scan sessions.

Root mean squared (RMS) stimulus contrasts of 1.8% [(L-M)-cone contrast] and 5% (luminance contrast) were used to generate half-saturating and equal responses in V1 (Engel et al., 1997a; Liu and Wandell, 2005). The absence of neural or hemodynamic response saturation was important since ceiling effects in responses to one or the other stimulus types could, potentially, explain the differences in classification performance that we observed.

Stimuli were presented in a balanced, randomized block design with 16 s of stimulus (1 s on, 1 s off) followed by 10 s of mean luminance blank screen with just a fixation point. Each fMRI run lasted 318 s, including a 6 s scanner “warm-up” period when no stimulus was present and an additional 12 s lead period to avoid stimulus onset transients. Four different stimulus conditions were presented within this period representing all the combinations of the two probe chromaticities (Ga, Gc), and the two tasks (detecting orientation changes or detecting changes in the amplitude of the RMS cone contrast along the stimulus color axis). Each condition appeared three times per run, and each subject completed a minimum of seven runs.

Gratings were presented for 1 s with a 1 s blank interval between each presentation in sequences of eight presentations. This temporal design minimized apparent motion cues at the beginning and end of the “orientation change” targets. Such changes could, potentially, have been detected through motion-sensitive mechanisms, meaning that signals arising in orientation tuned units in V1 might not be required to perform the task. The temporally sparse stimulus also minimized the potential effects of contrast adaptation (Jameson et al., 1979; Krauskopf et al., 1982).

Regardless of attentional task, blocks contained orientation change events and contrast change events at a constant frequency (p = 0.3 per presentation). Contrast and orientation modulation amplitudes were determined beforehand to yield ∼78% correct detection rates and the attentional demands of each task were therefore (1) significant and (2) approximately equal. Mean target modulation amplitudes were as follows: AO, 12.9°, AC: Ga = 1.5%, Gc = 0.65%.

AO and AC conditions were indicated by subtle changes in the small (0.14°) central fixation point as shown in Figure 1 (AO: red square; AC: blue circle). The average spatiotemporal characteristics of all conditions (spatial frequency, spatial extent, etc.) were identical.

Retinotopic mapping.

Each subject's retinotopic visual areas (V1, V2, and hV4) were defined using standard mapping techniques described previously (Engel et al., 1997b; Huk et al., 2002; Wade et al., 2002; Brewer et al., 2005). The visual areas (Fig. 2a) and average total sizes of the regions of interest defined in this study (Fig. 2b) are shown in Figure 2.

Figure 2.

a, Region activated by the annular stimulus rendered on an inflated cortex. b, V1, V2, and hV4 defined from separate retinotopic mapping experiments shown on a single subject's left hemisphere. c, Average sizes of analysis ROIs. ROIs were defined by intersecting visual area ROIs shown in b with localizers shown in a. Visual area sizes were measured on the 3D mesh. Error bars are 1 SEM.

Voxel selection.

Analysis was restricted to the retinotopic location of the stimulus annulus in visual cortex based on eccentricity mapping stimuli and independent localizers. Subsequent voxel selection was performed to reduce the possibility of contamination by stimulation from outside the region of interest—for example, from blood vessels draining from nearby locations (Gardner, 2010; Thompson et al., 2011). By restricting our analyses to voxels that intersect the gray matter, we excluded most large blood vessels. In addition, we excluded voxels with amplitude variations >2.5 of the local mean, as we find that high variance is a signature of large draining veins. For each region of interest, we then chose the 70 voxels with the strongest mean responses to the stimulus versus blank alternation (Serences and Boynton, 2007).

Classification analysis and statistics.

Our classification analysis procedures are similar to those reported in other recent papers (Serences and Boynton, 2007; Brouwer and Heeger, 2009). BOLD signal time series were extracted from all voxels in a given ROI on each trial and normalized by their SDs. Responses in a window 6–16 s after stimulus onset were averaged together to generate a scalar amplitude for each voxel and each trial. Classification analysis was done separately in a pairwise manner comparing the AC and AO attentional tasks for both achromatic or chromatic gratings. We applied a linear classifier, specifically, the Matlab (MathWorks) “classify” routine using the “diaglinear” option to specify that the data covariance matrix should be assumed to be diagonal. We confirmed this assumption in an independent analysis of a large V1 dataset containing both chromatic and achromatic trial events.

Accuracy was determined using a “leave-one-out” cross-validation procedure (Pereira et al., 2009) in which each trial in turn is omitted from the classifier and serves as a test. For each ROI of each participant, the mean classification accuracy of the test trials was determined (chance performance, assessed by repeating the classification with shuffled condition labels, was always 50%). Statistical significance for the cross-subject classification performance in each ROI was performed using a one-sample t test on the classification accuracies of the six participants.

Results

Classifications in V1

A univariate analysis on mean percentage signal changes during 6–16 s after stimulus onset revealed no significant response bias to particular chromatic/achromatic and AC/AO conditions in V1 (one-way ANOVA: p > 0.1, Fig. 3a). This shows that we were successful in choosing chromatic and achromatic contrasts that yielded indistinguishable fMRI responses from striate cortex.

Figure 3.

a, Mean responses to different stimulus conditions and attentional tasks. We found no significant univariate differences between conditions. b, Classification performance for AO versus AC using chromatic and achromatic gratings. Performance was highly significant for the chromatic condition but nonsignificant for the achromatic gratings. Overall performance for classifying chromatic versus achromatic gratings in V1 is also well above chance. Error bars are 1 SEM. Symbols are individual subject data.

However, when we asked whether pattern classification could discriminate between the two attentional conditions for (1) achromatic and (2) isoluminant chromatic stimuli, we found two striking results (Fig. 3b). First, we found that the classifier reliably discriminated attentional state (AC or AO) when the stimulus was a chromatic grating (p < 0.003). However, when we switched to an achromatic target, keeping all other stimulus parameters the same, discrimination performance was no better than chance (p > 0.2, ns). Furthermore, attentional state was decoded significantly better when the stimulus was a chromatic grating than an achromatic grating (p < 0.005).

Interestingly, the model of V1 chromatic and achromatic coding that we describe above predicts that classification of stimulus class (Gc vs Ga) should be both possible and similar in accuracy for the two task conditions. We also found this to be the case. Overall, in agreement with other reports (Haynes and Rees, 2005; Sumner et al., 2008; Brouwer and Heeger, 2009; Seymour et al., 2010), we were able to classify chromatic vs luminance stimuli accurately (p < 0.02) in V1, and there was no significant difference between classification performance in the AO (0.58 ± 0.02) and AC (0.6 ± 0.04, p > 0.6) conditions.

Classification was at chance when we used only voxels responding more to the Gc than Ga condition. This agrees with electrophysiological work showing that attention increases selectivity both by increasing responses in the neurons representing the attended stimulus and by suppressing responses to unattended features (Martinez-Trujillo and Treue, 2004).

Extrastriate visual areas

Classification was also possible in other visual areas (see Fig. 4). In V2, like V1 (Fig. 3b), we could discriminate AC versus AO for chromatic gratings (p < 0.03) but not achromatic gratings (p > 0.1). Overall, classification performance in V2 was similar to that in V1. Performance in hV4 was no better than chance (p > 0.05).

Figure 4.

Classification for AO/AC using chromatic and achromatic gratings in V2 and hV4. Conventions are the same as in Figure 3a.

Discussion

It is now clear that V1 contains many neurons sensitive to both chromatic and achromatic contrast. Many of these neurons also have spatially organized receptive field structures allowing them to respond selectively to features such as orientation and frequency (Livingstone and Hubel, 1984; Thorell et al., 1984; Lennie et al., 1990; Johnson et al., 2001, 2008; Conway and Livingstone, 2006; Horwitz et al., 2007; Conway et al., 2010). In addition, there is clear evidence from fMRI studies of neuronal populations in V1 sensitive to both orientation and color (Sumner et al., 2008; Seymour et al., 2010). However, it is also clear there exists a subpopulation of neurons with excellent chromatic tuning but little or no orientation sensitivity (Livingstone and Hubel, 1984; Conway and Livingstone, 2006; Johnson et al., 2008). Our results shed some light on the spatial arrangement of these different neural populations.

The classification performance we measure in V1 is consistent with a cortical organization in which chromatic contrast and the orientation of chromatic structure are coded in spatially distinct maps. Switching attentional tasks changes the responses in these different maps and this change can be detected using multivariate pattern classification algorithms. Attentional task switching generates much weaker changes in V1 population response patterns when luminance stimuli are used—either because the neurons that code luminance orientation and contrast are more finely intermixed or because there is less task-specific attentional modulation of these neurons, or both.

It is likely that the neural populations stimulated by our chromatic and achromatic stimuli contain considerable overlap. We do not claim that achromatic stimuli are represented solely in the interblob regions while isoluminant chromatic stimuli stimulate only the blobs. Instead, our data suggest that neurons that are most informative about the chromatic contrast of an isoluminant stimulus are spatially clumped and different from the neurons that are most informative about the orientation of an isoluminant stimulus. Recent work (Lu and Roe, 2008) supports the original reports that these chromatically sensitive but orientation-insensitive neurons lie within the cytochrome oxidase blobs (Livingstone and Hubel, 1984). It is also possible that clumping and segregation of this type also exists for achromatically sensitive neurons but at a much finer spatial scale.

Alternatively, there may be no spatial segregation at all for neurons sensitive to achromatic luminance contrast and orientation, and at least for the eccentricity and spatial frequencies used here, a similar group of neurons may be sensitive to changes in these two parameters. It would be interesting to examine the relationship between achromatic contrast sensitivity and spatial frequency and to test the possibility that neurons sensitive to achromatic high spatial frequency pattern orientation and contrast exhibit a greater degree of anatomical segregation.

Classification performance in V2 was comparable to V1 despite the fact that the V2 stripe structure has a slightly larger spatial scale. However, classification in V2 is complicated by the presence of domains within the thin stripes sensitive to low spatial frequency temporal luminance modulations (Wang et al., 2007). In addition, despite the apparently larger spatial extent of the stripes compared to the V1 blobs, careful analysis shows that the spatial periodicity of the V1 and V2 color domains is very similar (Ts'o et al., 2009). It is not clear, therefore, that classification performance in V2 should surpass that in V1.

Given its putative role in color vision, it is also surprising that we could not distinguish any condition pairs (p values >0.05) in area hV4. This result is particularly curious in the face of evidence suggesting that human and macaque V4 may be functionally homologous (Wade et al., 2008) and that macaque V4 contains relatively large domains of neurons with sharp chromatic tuning (Conway et al., 2007). We note that other researchers have reported only marginal chromatic discrimination performance in hV4 (Brouwer and Heeger, 2009). It is possible that performance in this region is limited both by the relatively small size of the area (Fig. 2b) and local MR signal distortion due to a large blood vessel on the ventral surface that often “eclipses” this region (Winawer et al., 2010). Alternatively, the functional microstructure of hV4 may differ in some way from that reported in macaque.