Hue Selectivity in Human Visual Cortex Revealed by Functional Magnetic Resonance Imaging

Abstract

The variability of color-selective neurons in human visual cortex is considered more diverse than cone-opponent mechanisms. We addressed this issue by deriving histograms of hue-selective voxels measured using fMRI with a novel stimulation paradigm, where the stimulus hue changed continuously. Despite the large between-subject difference in hue-selective histograms, individual voxels exhibited selectivity for intermediate hues, such as purple, cyan, and orange, in addition to those along cone-opponent axes. In order to rule the possibility out that the selectivity for intermediate hues emerged through spatial summation of activities of neurons selectively responding to cone-opponent signals, we further tested hue-selective adaptations in intermediate directions of cone-opponent axes, by measuring responses to 4 diagonal hues during concurrent adaptation to 1 of the 4 hues. The selective and unidirectional reduction in response to the adapted hue lends supports to our argument that cortical neurons respond selectively to intermediate hues.

Introduction

It is not well understood how color information is encoded even at the very early level in human visual cortex. Previous studies have suggested that color information is generated as the difference in photoreceptor excitations among 3 classes of cones, namely long-, medium- and short-wavelength-selective cones (L, M and S cones). Response differences among these photoreceptors originate from differences in their spectral sensitivity; for example, reddish light activates L cones predominantly, M cones secondarily, and S cones only slightly. The color selectivity in the lateral geniculate nucleus is derived mainly as differential inputs from different classes of cones (Derrington et al. 1984; Hanazawa et al. 2000). MacLeod and Boynton (1979) have introduced a color space, where color changes along one axis differentially co-vary L- and M-cone excitations under isoluminance while keeping S-cone response constant (L − M channel), and those along the other axis selectively vary S-cone responses alone (S channel). The third axis for the cone-opponent color space is achieved by the weighted sum of L- and M-cone responses (L + M or luminance channel). The 2 chromatic axes roughly correspond to red/green and blue/yellow components in color appearance, respectively. However, colors that selectively stimulate the 2 cone-opponent color channels do not exactly match perceptually pure colors of red, green, yellow, and blue, namely “unique hues” (e.g., De Valois et al. 1997).

The three-dimensional signal (L + M, L − M, and S channels) in macaque monkeys is not independently treated at the early level of visual cortex. Single-unit studies have shown that neurons in the primary visual cortex (V1) are selective for intermediate directions between the L − M axis and S axis (Lennie et al. 1990; Hanazawa et al. 2000; Wachtler et al. 2003). Some neurons receive signals from different axes as linear summations, whereas other neurons do so in nonlinear fashions. These properties, however, have not been precisely studied in humans. Several attempts have been made using psychophysical approaches to show the presence of multiple color-selective mechanisms in human visual cortex, but these efforts have not succeeded in specifying a detailed selectivity profile for such mechanisms (Krauskopf et al. 1986; Webster and Mollon 1991; Goda and Fujii 2001; Hansen and Gegenfurtner 2005; Kuriki 2007). Recently, several human fMRI studies with multivariate pattern analysis method have indicated that a portion of color-selective neurons are likely tuned to off-axis hues in the cone-opponent color space (Brouwer and Heeger 2009; Parkes et al. 2009; Goddard et al. 2010; Kuriki et al. 2011). However, it is impossible to explore the variability of hue selectivity in detail with pattern classifications alone. In addition, the number of hue classes that can be revealed is naturally limited by the variability of stimuli, which, up to now, has not exceeded 8. A recent study addressed this point by the reconstruction of responses to 12 color stimuli presented in a random order (Brouwer and Heeger 2013); they tried to reconstruct the hue selectivity of voxels by fitting the event-related responses with linear combinations of half-wave rectified responses of 6 cone-opponent mechanisms. Their study mainly focused on the clustering of neural representation for the test colors, and the variability of the most sensitive hues of individual voxels in early visual cortex was not explicitly explored. In short, the presence of neurons that respond selectively to hues in intermediate directions of cone-opponent axes, like those found in macaque V1 (Hanazawa et al. 2000; Wachtler et al. 2003; Hass and Horwitz 2013), is still controversial in humans, especially at the early stage of visual cortex.

Studies using combined single-unit recording and intrinsic optical imaging have revealed spatial arrangements of color-selective neurons (color maps) in macaque V1 (Xiao et al. 2007; their Fig. 4A), V2 (Wang et al. 2007; their Fig. 9), and V4 (Tanigawa et al. 2010; their Fig. 6e, bottom panel), where each color preferentially activates a specific cortical domain (such domains generally preserve the features of columns: Color preference changes gradually across cortical surface but is relatively stable across cortical layers). Neighboring domains overlap substantially, but their peaks shift across cortex systematically. These color maps, arranged at different spatial scales, demonstrate a continuous shift in color preference across cytochrome oxidase (CO) blobs in V1 (see Lu and Roe (2008) for the relationship between color domains and CO blobs), where color preference changes every tens of microns, CO thin stripes in V2, where the entire spectrum of colors is represented within ∼0.5 mm, and discrete color-selective regions in V4, where successive color domains are separated by >0.5 mm. It is thought that similar color maps also exist in human visual cortex, but at present, there is no means to directly visualize them. Though individual color domains in humans can be twice larger than those in monkeys (e.g., CO blobs are located in the centers of ocular dominance columns, and ocular dominance columns in humans are approximately twice larger than those in monkeys [cf. Adams et al. 2007; their Fig. 6C]), they are still much finer than the typical resolution that the current fMRI can offer. Nevertheless, we reasoned that a voxel in an fMRI study might still be selective for a color if the domains coding that color were excessively contained in the voxel. Accordingly, a biased color preference, including those off the cardinal axes, may be revealed using fMRI, at a conventional spatial resolution.

In the present fMRI study conducted with a spatial resolution of 2 × 2 × 3 mm³ and a novel stimulation paradigm that resembles the phase-encoding technique commonly used for the visual field mapping by fMRI and makes it possible to test the hues across the entire color space, we demonstrated that neurons selective for intermediate hues that are off the cardinal axes were indeed present in human early visual cortex. We also conducted a hue-selective adaptation experiment to confirm that selectivities for intermediate hues away from cone-opponent axes were not simply resulted from combined responses to cone-opponent mechanisms.

Methods

Subjects

Three subjects participated in the main fMRI experiment with written informed consents. They also took part in all or some of psychophysical tests. All experiments were approved by the RIKEN Functional MRI Safety and Ethics Committee. All subjects were healthy, without past history of psychiatric or neurological diseases, had normal or corrected-to-normal vision, and were confirmed to have normal color vision with Ishihara pseudo-isochromatic plates. Prior to each fMRI experiment, the subject went through procedures for heterochromatic flicker photometry and chromatic detection threshold measurements.

Definition of Color Space, Generation of Color Stimulus, and Measurement of Color Detection Thresholds

We used the basic structure of MacLeod and Boynton (1979) color space, based on cone responses, to define the color stimulus in this study. Cone fundamentals of Smith and Pokorny (1975) were used to calculate L-, M-, and S-cone responses from the spectrophotometric data of primary colors of an MRI-compatible artifact-free LCD projector (Silent Vision 6011, Avotec, Inc.), measured using a spectroradiometer SR-3N (Topcon). The horizontal coordinate of the color space is the cone-opponent axis for L- vs. M-cones; in short, L–M axis. One of the chromatic coordinates shows Weber contrast of L-cone response increments with respect to the gray background (L-cone response increment divided by L-cone response to the background, namely, ΔL divided by L-background; in short, ΔL/L_b), while M-cone response is uniquely defined from L-cone response under the isoluminant constraint (details are formulated below), and the other coordinate is similarly the Weber contrast for S-cone responses (ΔS/S_b). Figure 1A illustrates the schematic drawing of color space used in this study. To maximize stimulation to color-sensitive neurons, we used the highest possible color contrast available with the LCD projector. The radius of the hue circle varied among conditions (see below for more descriptions), but the basic setting was 0.08 (8%) and 0.80 (80%) for L- and S-cone contrast (ΔL/L_b and ΔS/S_b), respectively; hereafter, we refer to the radii as L_amp and S_amp. Because the apparent chromatic strength is closer between the 2 axes under the condition of L_amp = 0.08 and S_amp = 0.80, and this ratio is close to the multiples of color detection thresholds, all hue angles used in the present study were represented by the angle defined under this condition. This definition of hue angle is also compatible with the cardinal axis color space defined by Derrington et al. (1984), the so-called DKL color space. For each subject, we measured the color detection threshold along both axes and confirmed that the ratio of L- and S-cone contrast was roughly the same (i.e., ΔL/L_b:ΔS/S_b = 1:10; details are described below; see also Supplementary Material). This ratio of hue circle provided nearly equal chromatic saturation in appearance along the circular trajectory in the color space. The hue angle was defined by taking the L-cone increment direction as 0°, S-cone increment direction 90°, L-cone decrement direction 180°, and S-cone decrement direction 270° (Fig. 1A).

Figure 1.

Color stimulus, hue circle, and data-analysis method used in the main fMRI experiment. (A) The color space used in the present study. The horizontal (0°–180°) and vertical (90°–270°) axes in the isoluminant plane indicate the hue directions that selectively stimulate L and S cones, respectively. ΔL/L_b, Weber contrasts for L-cone responses; ΔS/S_b, Weber contrasts for S-cone responses; θ, the hue angle. (B) A schematic drawing of the stimulus, which was presented in the central region with a diameter of 6°. The checkers, whose color changed continuously, were 15% higher than the gray background in luminance. The heterochromatic flicker photometry was performed for individual subjects in 13 subregions (separated by dashed lines) to minimize positional differences in isoluminance across retinal positions. (C) Time courses of original BOLD responses (black trace) and estimated hemodynamic responses (red trace) from a voxel in a representative subject's V1 to continuous color changes whose cycle started from either red color (0°, Block A) or green color (180°, Block B). Note that in the actual experiment, Blocks A and B were each repeated 10 times. See also Supplementary Movie 1. (D) Estimated average hemodynamic responses of the voxel to Block A (red filled circles) and Block B (blue open circles). (E) The difference (black open circles) between the 2 hemodynamic responses shown in (D). The hue selectivity of the voxel was determined, after hemodynamic delay was corrected, by the phase of the peak position of a cosine function (red curve) used to fit the differential hemodynamic response in the final 24 s (indicated by the pair of blue dashed line segments) of the 36-s stimulation period, during which the color changed continuously. Numbers on the top of the plot indicate the stimulus phase in degrees for “Block A,” after the hemodynamic delay was corrected. The estimated peak response was at 236.1° in the hue angle and the amplitude ([peak response—trough response] divided by 2) was 1.24% for this sample voxel.

Each subject's sensitivity to luminance was measured by flicker photometry at 13 retinal positions covered by the hue-changing stimulus, including a central foveal region, 4 locations in eccentricities between 0.6° and 1.8°, and 8 between 1.8° and 3.0° in visual angle (sectors shown by dotted lines in Fig. 1B). The flicker photometry was conducted by alternately presenting a uniform test color and a reference color, subtending 1° × 1° in visual angle, at a temporal frequency of 16 or 20 Hz (depending on each subject's preference). Both reference and test colors were increments of red and green phosphors of a CRT display (GDM-F520, SONY), respectively, presented on a gray background (x = 0.333, y = 0.333, Y = 25.0 cd/m² in CIE XYZ (1931) coordinates). Each phosphor's L- and M-cone excitations were calculated for deriving the relative contribution of M-cone to the luminance channel by formulae described elsewhere (Ahn and MacLeod 1993; Kuriki and MacLeod 1998; Kuriki 2006), following the assumption that the response in the luminance channel is a weighted sum of L- and M-cone responses (Eisner and MacLeod 1980). The subject adjusted the green-phosphor intensity, while fixing the red-phosphor intensity at 10% increment with respect to the background in the Weber contrast, to achieve perceptually null or minimum flicker at each retinal position. Visual stimuli for the flicker photometry were generated at a resolution of 12 bits per gun using a graphic board (VSG 2/5, Cambridge Research Systems), controlled by a personal computer (Precision T3500, DELL). Ten measurements were made and the average in relative L- and M-cone contribution was calculated for each retinal position. The sensitivity of luminance channel differs between retinal locations, due mainly to the nonuniformity of ocular media such as macular pigment. Therefore, we restricted our stimulus to within the diameter of 6° in the parafoveal region. The relative M-cone weight at the central foveal region for individual subjects was as follows: Subject 1, 1.40; Subject 2, 2.75; and Subject 3, 0.785. The M-cone stimulation for each hue can be derived uniquely from the L-cone stimulation under the isoluminance constraint (Eisner and MacLeod 1980); the ΔM under an isoluminant constraint in each subject (L + ω_sub M = constant) with a given ΔL can be derived as follows:

$$\mathrm{\Delta }M=\frac{-\mathrm{\Delta }L}{{\omega }_{\mathrm{sub}}},$$ (1)

where ω_sub represents the relative contribution of M-cone to the luminance channel in each subject. The measurements and analyses for the relative M-cone weight were conducted immediately before each fMRI measurement, and the results were then applied to generating test colors in the stimulus. Hereafter, the 2 chromatic axes in the isoluminant plane are referred to L − M and S axes, and only L- and S-cone contrasts are considered when color stimuli are described.

Chromatic detection thresholds in L − M- and S-axis directions were psychophysically determined. The amount of increment in L − M- or S-axis direction was determined psychophysically by a staircase method with a two-down and one-up procedure. The details of the procedure for measuring chromatic detection thresholds are described in Supplementary Material. The means and standard deviations for the thresholds (θ) in Weber contrast (in %) for L − M- and S-axis directions were as follows: Subject 1, ΔL_θ/L_b = 0.32 ± 0.08, ΔS_θ/S_b = 3.23 ± 0.70; Subject 2, ΔL_θ/L_b = 0.13 ± 0.03, ΔS_θ/S_b = 1.89 ± 0.58; and Subject 3, ΔL_θ/L_b = 0.18 ± 0.06, ΔS_θ/S_b = 1.68 ± 0.70. On average, these measurements resulted in a ΔL_θ/L_b:ΔS_θ/S_b ratio of roughly 1:10.

During the fMRI experiment, visual stimulus was projected onto a frosted glass screen (subtending 20° by 15°) placed above the subject's head. The stimulus was a circular checkerboard pattern (6° in diameter), which was presented at the center of the screen (Fig. 1B). Each checker element measured 0.26° by 0.26° and was circumscribed with hairline black edges. The checkerboard pattern alternated in time between the test color and a gray background at a rate of 200 ms per frame (i.e., 2.5 Hz or 5 frames per second in square wave). The checkerboard pattern was chosen because it contained various spatial frequencies so that various hue-selective mechanisms with different spatial-frequency preferences could be probed more effectively. A cycle of complete hue change took 24 s, during which a total of 120 frames of images, 3° in hue difference between neighboring frames, were presented (see also Supplementary Movie 1). A fixation cross was presented at the center and changed its color occasionally for 0.5 s at random intervals. The subject, viewed the stimulus through a mirror mounted in front of the eyes, was instructed to report the onset of the color change by pressing a button. This task was introduced to ensure the subject's fixation at the center of the stimulus and to maintain the subject's level of arousal.

A similar hue-changing stimulation paradigm was previously used in a study on the selectivity of retinal ganglion cells in macaques (Sun et al. 2006), but additional efforts described in the next section were necessary for measuring hue selectivity of voxels in the human fMRI study.

Adding Luminance Pedestal to Color

In a preliminary study with the hue-changing stimulus that was isoluminant with the gray background, we could not constantly find voxels that show significant response as those shown in Figure 1C, D, due mainly to the signal decay over stimulation period. It was probably because the stimulation period was too long (>30 s) and the color change along the hue circle at the rate of 24 s per cycle was too slow, which allowed neurons to adapt to the continuous stimulation. To improve the signal-to-noise ratio in the fMRI measurement and to avoid blood oxygenation level-dependent (BOLD) (Ogawa et al. 1992) responses from luminance artifacts, the luminance of the color stimulus was increased by 15% with respect to that of the uniform gray background. A similar luminance pedestal was previously used in a study on the color selectivity of neurons in monkey V2 (Kiper et al. 1997) and V3 (Gegenfurtner et al. 1997). The amount of the luminance pedestal of 15% was empirically determined so that the BOLD signal modulation in terms of color changes would be in the range of 2–3%. The detailed definition of stimulus chromaticity is given by the formulas in Supplementary Material. The obtained cone responses were first transformed to CIE XYZ (1931) by applying the Smith–Pokorny's cone fundamentals (Smith and Pokorny 1975) and then to the intensities of R, G, and B primary colors to be delivered through the projector (Cowan 1983).

Different Combinations of L- and S-cone Contrasts

We tested stimulus color changes along a hue circle in the color space defined by S-cone and L-cone selective axes. The signals of S-cones and those of L- and M-cones are conveyed through relatively independent pathways through the retina and LGN. Thus, the aspect ratio between the 2 axes can be set arbitrarily. Previous fMRI studies, however, have reported that BOLD signal changes in response to L − M- and S-axis selective stimuli match those in response to different cone contrasts (Engel et al. 1997a) and to different detection thresholds as well (Mullen et al. 2007). To seek the optimal hue-circle trajectory, we tested various ratios of L − M- and S-axis amplitudes in L- and S-cone contrast. For this purpose, we fixed the L-cone amplitude to 0.08 (i.e., L_amp = 0.08 in ΔL/L_b) due to the limitation in the LCD projector's color-rendering area but varied the S-cone amplitude between 0.04 and 0.80 in the cone contrast, namely, S_amp = 0.04, 0.08, 0.16, 0.24, 0.40, 0.60, and 0.80 in ΔS/S_b, respectively. To present color stimuli under various S_amp conditions, the stimulus color in a particular S_amp condition was in fact modulated along the circumference of an ellipsoid, whose minor axis was determined by scaling the S-axis component while keeping the L − M-axis component (major axis) unchanged.

Imaging Parameters and Experimental Paradigm

All fMRI experiments were conducted on an Agilent 4 Tesla whole-body MRI system (Agilent, Inc.) equipped with a head gradients system. High-resolution (1 × 1 × 1 mm³) three-dimensional T₁-weighted anatomical MR images were acquired with a bird-cage radio-frequency (RF) coil, and a 5-inch transmit/receive quadrature RF surface coil, mounted on a bakelite support frame attached to the patient table, was used to acquire functional and co-registered anatomical images in functional experiments. During the experiment, the subject was made to lie supine on the patient table and rest the head on the surface coil. Rigid head motion was restricted using a bite-bar as well as foam rubber padding. The subject's heartbeat was monitored with a pulse oximeter, and respiration was monitored with a pressure sensor placed on the abdominal region. Both signals were recorded along with the timing of RF pulses for later corrections of physiological fluctuations using a retrospective estimation and correction method (Hu et al. 1995). Functional images were taken using a four-segment centric-ordered echo-planar imaging (EPI) pulse sequence with an in-plane resolution of 2 × 2 mm² and a slice thickness of 3 mm. Six slices were prescribed perpendicular to the calcarine sulcus on either the left or right hemisphere, and the most posterior slice was positioned near the occipital pole. The following scan parameters were used: volume TR = 1.5 s, TE = 25 ms, average flip angle = 45°, and field of view = 256 × 256 mm².

To probe the selectivity of cortical neurons for a wide spectrum of colors, we employed a continuous stimulation paradigm and a differential analysis method, which has been used recently for investigating the contextual contrast modulation (Tajima et al. 2010) and orientation selectivity (Sun et al. 2013) in human V1. Our stimuli consisted of 2 series of reversing checkerboard pattern (see above) whose color changed continuously. In one series, a color started from the positive direction in the L − M axis (+L − M), and as illustrated in Figure 1A, it went through, in counter-clockwise direction, the positive direction in the S- axis (+S), negative L − M axis (−L + M), and negative S-cone (−S) axis. In the other series, the color stimulus started from the negative direction in the L − M axis (−L + M), and the color then changed in the same direction and angular velocity as in the first series. For both series, a cycle of color change took 24 s, and we presented one and a half cycles of color changes (36 s), followed by a 24-s period with the uniform gray background, in a block. During the fMRI experiment, these 2 types of blocks were each presented 10 times in 5 repeats of an A–B–B–A sequence. Comparing the 2 series of color changes, a pair of opposite colors was always presented at any given latency from the onset throughout the 36-s stimulus presentation.

Based on previous single-unit findings in monkeys, it is reasonable to assume that a hue-selective neuron is the most responsive to a particular hue and the least to the hue in the opposite direction (Hanazawa et al. 2000; Gegenfurtner 2003; Wachtler et al. 2003). Thus, taking the difference of hemodynamic responses to the 2 series of stimuli should amplify the BOLD signal modulated by color stimuli by roughly a factor of 2. Even when rectification nonlinearities are considered in a hue-selective response profile, the maximal BOLD response is expected to emerge in response to the presentation of the most preferred hue in Block A, and no response to the opposite hue. Because the BOLD response to the stimulus in Block B, which is 180° different in phase, is subtracted away, the difference between the positive and negative “rectified” responses can constitute the peaks and troughs around the preferred hue and the hue in the opposite direction. Thus, a maximal response and a minimal response will emerge in the differential response, regardless of rectification or not. In the meantime, this differential approach should also help in canceling the BOLD responses common to the 2 stimulus series but irrelevant to the color per se, such as the responses to the onset and offset of the color stimulus and those to the slow adaptation to the prolonged stimulus presentation. To minimize the effect of the strong transient response to the onset of color stimulus, the differential hemodynamic response was derived from the cycle in the final 24 s of the 36 s series, in which colors changed continuously in opposite directions (Fig. 1D.E). After the linear trends were removed, the differential hemodynamic response was fit with a cosine curve, and the position of the peak response was then located by finding the phase lag in the cosine curve. To relate the peak for response to the hue selectivity for each voxel, a hemodynamic delay of 5.3–6 s was used. This delay was empirically estimated for each subject in a control fMRI experiment, in which the stimulus luminance, instead of color, was modulated with the same L-cone contrast (ΔL/L_b = 0.08) and the same temporal frequency (24 s per cycle) (see Supplementary Fig. 1).

It should be pointed out that choosing a resolution of 2 × 2 × 3 mm³ was an empirical decision in consideration of primarily 1) the unit time needed for scanning a reasonably large brain volume covering most of early visual areas (volume TR = 1.5 s), 2) the time needed for each scan (∼22.5 min), and 3) the total time needed for each functional session (within 2.5 h for preparation, multiple functional scans, and the anatomical scan for co-registration). Going for a higher spatial resolution, for example, reducing the in-plane resolution to 1 × 1 mm², would result in 2 major penalties; first the increased scan time per scan (∼3 times) would be prohibitively long unless we reduce the number of slices from 6 to 2, which is not desired for the coverage purpose, and second, the reduced voxel size would substantially lower the signal-to-noise ratio (SNR) of EPI images, which was empirically tested and proven to be detrimental to acquiring quality time courses.

Data Analysis

After physiological corrections on the k-space data, the images were reconstructed, and a band-pass filter, which had a 3-dB drop-off cutoff frequency at 0.0025 and 0.32 Hz for low- and high-pass frequencies, respectively, but retained the DC level, was used to suppress baseline signal drifts. No other spatial or temporal filtering was applied. All analyses described below were conducted on a voxel-by-voxel basis using in-house software packages written on MATLAB (The MathWorks). For each voxel, a hemodynamic response function was estimated from the time course of BOLD response using a deconvolution approach (Fig. 1C) (Dale 1999). The goodness-of-fit was evaluated by a procedure developed in a previous study (Gardner et al. 2005). Briefly, for each voxel, we calculated an r² value, which was defined as: r² = 1 − (residual variance after fit)/(variance of original BOLD response), where the residual variance after the fit refers to the difference between the time-locked BOLD response of the voxel and corresponding estimated hemodynamic response function. Across the voxels, to determine which of their r² values were higher than can be expected by chance, and thus significantly activated by our stimuli, we randomly shuffled stimulus times and recomputed r² values for all voxels in a scan (128 × 128 × 6 = 98 304 voxels), which resulted in a randomized distribution of r² values. The real distribution of r² values was then compared with this randomized distribution, and the voxels in the real distribution with r² values higher than r² values of top 2.5% in the randomized distribution were regarded as significantly activated voxels. In our experiment, the threshold for the r² value was ∼0.15 (two-tailed t-test, P < 0.05); only the voxels with r² values higher than this threshold were used for further analysis. In addition, we also excluded a small number of voxels that showed negative BOLD response, as well as those likely associated with large venous vessels, which exhibited >5% increase in the BOLD signal.

After the differential hemodynamic response was obtained from each of the voxels that passed the above-mentioned threshold and selection criteria, we assessed the statistical significance of its modulation by the colors that changed continuously. For this purpose, we calculated the SNR of the modulation in the time course of the differential hemodynamic response by taking the ratio of the amplitude at the hue-changing frequency (24 s per cycle) to the average of amplitudes at higher frequencies. A numerical simulation revealed that the level of the top 5% of voxels in the accumulated histogram had an SNR of 8.3 dB and above, which was then used as the threshold to select the voxels for further studying their selectivity for colors (Fig. 2A, left 2 panels, before thresholding; right 2 panels, after thresholding). The numbers of the voxels selected with these criteria from each visual area (V1, V2, V3, or V4) for each subject are summarized in Supplementary Table 1.

Figure 2.

Generation of hue-selective histograms. (A) An example hue-selective histogram from a representative subject before (left 2 panels) and after (right 2 panels) a signal-to-noise threshold of 8.3 dB was applied. The histogram was obtained with a condition of S_amp = 0.80. All voxels in areas V1 through V4 were pooled together. (B) Hue-selective histograms from a representative subject, obtained under various S_amp conditions. The abscissa represents the phase in a hue circle under each S_amp condition. The ordinate represents the proportion of voxels after normalized to the total number of voxels in each ΔS/S_b condition. (C–E) Hue-selective histograms in visual areas V1 through V4 in each subject under all ΔS/S_b conditions. The abscissa now represents the unified hue angle, defined for a hue circle with L_amp = 0.08, S_amp = 0.80, as illustrated in Figure 1A. See also Figure 3C and Supplementary Figures 1–3.

For comparison between 2 curves, a two-sample Kolmogorov–Smirnov test was applied with the null hypothesis that the 2 curves are identical. The number of samples for the Kolmogorov–Smirnov test was equated to 24 (i.e., sampling a hue circle at an interval of 15°). In addition, a correlation coefficient analysis was applied with the null hypothesis that 2 curves are uncorrelated. For multiple comparisons of adaptation indices, a Tukey–Kramer test was applied after the analysis of variance (ANOVA) on MATLAB.

Retinotopic Mapping

The voxels with statistically significant BOLD responses were found mainly in the visual cortex. Therefore, we defined visual areas as regions of interest (ROIs) based on the results from the meridian mapping and retinotopic mapping performed in separate sessions. The retinotopic mapping was performed following a conventional method (Engel et al. 1997b) with a rotating fan-shaped stimulus (in clockwise and counter-clockwise directions) and a contracting/expanding ring-shaped stimulus, both of which were made up of a black-and-white checkerboard pattern with a luminance contrast of 100% and a flickering frequency of 8 Hz. The boarders of multiple visual areas were demarcated as the convergence and divergence point in the phase map generated using the rotating fan-shaped stimulus. These borders were also confirmed with horizontal- and vertical-meridian representations in the visual field. We analyzed the data for the ROIs of V1, V2, V3, and V4, which could be clearly separated.

Results

We first determined the hue selectivity of individual voxels using a continuous stimulation paradigm and a differential analysis method (for stimulation paradigm and data analysis details, see Methods and Fig. 1). To assess the variability of hue selectivity, we generated hue-selective histograms for V1, V2, V3, and V4 in 3 subjects. Figure 2A illustrates the results from a representative subject, where each histogram was binned by 15° in the hue angle, and the histograms across all 4 areas were added up together. The shape of a histogram did not change significantly when different numbers of voxels, resulting from different SNR thresholds, were used (see Supplementary Fig. 2). These data were acquired with a basic set of Weber contrasts for L-cone responses (L_amp = 0.08) and S-cone responses (S_amp = 0.80), which were perceptually determined so that the chromatic modulation between colors along the L − M and S axes was roughly equaled in the multiples of chromatic detection thresholds. The voxels that were significantly modulated by different colors (Fig. 2A, rightmost panel), however, display 2 major peaks around S-cone selective directions in the hue cycle (90° and 270°) but are conspicuously absent around the L − M axis (0° and 180°). This observation cannot be explained if neurons are selectively tuned to multiple colors, at least along both the L − M and S axes, and indicates that there may exist a mismatch between neurons' color-selective responsiveness and perceptually determined equality in chromatic saturation. Since there are already several reports on the asymmetry in BOLD response strength between L − M- and S-axis mechanisms (Engel et al. 1997a; Mullen et al. 2007), we conducted the same experiment with different S-cone contrasts: S_amp = 0.04, 0.08, 0.16, 0.24, 0.40, 0.60, and 0.80 (same as in the original condition). The radius along L − M axis, L_amp, was kept at 0.08 in all S_amp conditions, and the sampling points in each condition was defined by simply scaling the extent of S-cone stimulation along the S axis (see Supplementary Table 1 for summary of results obtained with different S-cone contrasts).

Original histograms of hue-selective voxels in V1 through V4, obtained under 7 different S-cone contrast conditions from a representative subject, are summarized in Figure 2B. The abscissa in each histogram represents the phase in a cycle of hue changes. Because the hue angle in the color space depicted in Figure 1A was defined by the aspect ratio of an ellipsoid with (L_amp, S_amp) = (0.08, 0.80), for a given S_amp value, the phase in a cycle shown in the histogram obtained with that S_amp value represents the hues that are different from those shown in other histograms. The stimulus “phase in time ϕ” is described by θ: tan θ = (S_amp/0.80) × sin ϕ/cos ϕ.

Figure 2C shows 7 histograms from the same subject, but the hue angle in each histogram was now unified along the hue circle with L_amp = 0.08 and S_amp = 0.80. Similar results from the remaining 2 subjects are shown in Figure 2D,E. It is now possible to identify prominent peaks around the directions of the L − M axis (0° and 180° in the hue angle) when S-cone contrast was reduced, such as in the conditions of S_amp = 0.04 and 0.08, and the peaks around the S-cone axis gradually dominate as the S-cone contrast increased. Collectively, these results demonstrate neuron populations in early visual cortex that are selective for cone-opponent axes, such as +L − M, +S, −L + M, and −S axes, as well as those that respond optimally to the hues between cone-opponent axes. These results, thus, provide clear evidence for the presence of cortical neurons that are selective for intermediate hues.

Measurement of Histograms by fMRI under Optimal Hue-Sampling Condition

A prominent feature in Figure 2C–E is the apparent over-representation of the voxels around S-cone selective directions (90° and 270° in the hue angle). Because the analysis described above only quantifies the number of voxels that meet certain criteria, regardless of the amplitude of BOLD response, the essential shape of the histogram is not necessarily affected in proportion to the stimulus strength; the shape of the histogram, however, depends on the proportion of neurons selective for different hues (see Supplementary Fig. 4 for detailed assessment). The results from our control experiment, in which S_amp (the minor axis of the elliptic hue locus) was varied while keeping the hue-sampling density unchanged, showed that not only the amplitude but also sampling density along the hue angle were possible causes for the difference in the shape of the histogram among various S_amp conditions (Supplementary Fig. 4). If we assume that the sampling density in the hue angle under the condition of (L_amp, S_amp) = (0.08, 0.80) is too dense for hues around the S-cone axis, the resultant large population bias in the histogram can be explained. Thus, it is necessary to remove the bias in baseline population before the distribution in a hue-selective histogram is evaluated. We thus made an attempt to equate the strength of stimulation on both axes (see Supplementary Fig. 5 for detail). Based on the average equilibrium value obtained in the above experiment, we took L_amp = 0.08 and S_amp = 0.40 as being close to the optimal sampling density. The chromatic saturation was fixed along the circumference of the hue circle with L_amp = 0.08 and S_amp = 0.80 to obtain larger responses from individual voxels (see Supplementary Fig. 3 and 4).

For each subject, we acquired data from 4 scans using this near-optimal sampling density of L_amp = 0.08 and S_amp = 0.40; in 2 scans, the hue changed in the forward direction (i.e., the counter-clockwise direction in Fig. 1A) and in the remaining 2, the hue change was in the reverse direction. We then generated a hue-selective histogram for each area from V1 to V4. For V1 in all subjects, we confirmed that the similarity of 4 histograms obtained in the 4 scans by correlation coefficient analysis and Kolmogorov–Smirnov test; there was no significant difference between the 2 histograms obtained in either the forward or reverse hue-change direction (correlation coefficient analysis, all correlation coefficients > 0.36, P < 0.05; Kolmogorov–Smirnov test, all P > 0.05), and between those across the 2 hue-change directions (correlation coefficient analysis, all correlation coefficients >0.41, P < 0.05; Kolmogorov–Smirnov test, all P > 0.05; see Supplementary Table 2 for detail).

In each panel of Figure 3, blue and green lines with right- and left-pointed triangles are normalized histograms for 2 scans where the hue changed in the forward direction (counter-clockwise direction in Fig. 1A) and reverse direction (clockwise direction in Fig. 1A), respectively. Each histogram was individually normalized by the total number of voxels obtained in the scan that passed selection criteria, because this number varied from one scan to another. The criteria for voxel selection are the same as in the other experiments. The red curve in each panel indicates the grand mean histogram averaged from the 4 scans (2 in each direction), and the standard deviation is shown as the shaded area, together with the grand mean, in the polar plot in Figure 4. In addition, we assessed the voxel-by-voxel reproducibility of hue selectivity within a scan by splitting images in a scan into former and latter halves, each containing images acquired during 5 repetitions of Block A and Block B. For each half of the images, we used the same approach to identify the voxels that were selective for different hues and then estimated the reproducibility between the results obtained in the 2 halves. Although overall the numbers of hue-selective voxels in both halves decreased due to the reduced SNR (data points were halved), reasonable and significant voxel-by-voxel reproducibility was found in all 3 subjects (for detail, see Supplementary Fig. 6).

Figure 3.

Comparison and integration of histograms for forward and reverse order of hue presentations. (A–C) The results from V1 through V4 for 3 subjects. In each panel, broken lines with right-pointed and left-pointed triangles show normalized histograms where the hue changed in the forward direction (counter-clockwise direction in Fig. 1A) and reverse direction (clockwise direction in Fig. 1A), respectively, averaged across 2 runs for each direction. The criteria for voxel selection are the same as in the other experiments. See text for statistical comparisons between the 2 directions. The red solid line with circles in each panel indicates the grand mean histogram averaged from the 4 scans (2 in each direction), and the standard deviation is shown as the shaded area, together with the grand mean, in the corresponding polar plot in Figure 4.

Figure 4.

Hue-selective angular histograms. (A–C) The results for V1 through V4 from individual subjects, and (D) the averages across the subjects. All histograms were normalized to the total number of voxels that passed selection criteria and are depicted in the polar angle, similar to the color space shown in Figure 1A. Each red curve and accompanying shaded area with dotted line contours indicate the mean and the range of standard deviations across measurements in 4 scans. In all panels, black dotted lines indicate the directions of opponent-color axes. In (A–C), dashed radial lines in different colors indicate the directions of unique hues, measured for individual subjects (see Supplementary Fig. 9 for comparison between histograms and unique hues). In (D), the directions of unique hues were averaged across the subjects.

Thus, after normalizing individual histograms by the respective total number of voxels that passed selection criteria, we derived, for each area, an average hue-selective histogram from the 4 normalized histograms (Fig. 4A–C for individual subjects; Fig. 4D for average across subjects). Overall, among visually responsive voxels, only a fraction of voxels showed significant hue selectivity: on average, 21.3, 17.4, 24.1, and 15.3% for Subject 1; 28.2, 37.3, 25.9, and 29.1% for Subject 2; and 19.1, 11.9, 13.4, and 10.6% for Subject 3 in V1, V2, V3, and V4, respectively (for detail, see Supplementary Table 3, the row with bold letters; see also Supplementary Fig. 7 for direct comparisons between visually responsive voxels and hue-selective voxels). Inspection of these histograms reveals several features. First, it is clear that the spectrum of hue-selective neurons is already much broader than cone-opponent axes even at the level of V1. Second, there exists a strong “anisotropy” in all histograms; under-represented hues, spanning as large as a quadrant, were commonly observed in all subjects. Similar under-represented hues were also found in the histograms of hue-selective neurons in macaque monkey V1 (e.g., see Hanazawa et al. 2000, their Fig. 10; Wachtler et al. 2003, their Fig. 2A), which exhibit a prominent absence of neurons around 135°. Because the color coordinates and the definition of the hue angle in these monkey studies are close to those used in our study, the range of under-represented hues in their cases resemble the result from our Subject 2 (Fig. 4B). In Subjects 1 and 3, under-represented hues were mostly around the fourth and third quadrant, respectively (Fig. 4A,C).

Confirmation of Hue-Selective Responses in fMRI

Hue-Selective Adaptation Specific to Intermediate Hues

A major concern is that the selectivity for intermediate hues may emerge through spatial summation of the activities of neurons that selectively respond to cone-opponent signals, such as those from +L − M (0°), −L + M (180°), +S (90°), and −S (270°) channels. If so, it is expected that, for instance, the voxels that responded optimally to the hue around 45° in the hue circle would be evoked to some extent by 2 orthogonal hues around 0° and 90° as well. To address this concern, we conducted an fMRI adaptation experiment to ensure the presence of neurons selectively responding to intermediate hues.

The procedure for the hue-adaptation experiment was identical to that used in previous achromatic contrast-adaptation study (Gardner et al. 2005), except that both the adaptation stimulus and test stimuli were the hues selected from the hue circle. The configuration of each hue stimulus was similar to the one used in the main fMRI experiment, but the 15% luminance pedestal was not applied to color checkers. Adaptation to 1 of the 4 hues, located at 45°, 135°, 225°, or 315°, was studied as follows. After a 60-s adaptation with 1 of the 4 hues (e.g., the hue at 45°), each test hue (at 45°, 135°, 225°, or 315°) was randomly, transiently, and repeatedly presented for 4 s, followed by an 8–12 s top-up period with the adaptation hue. This procedure was repeated until adaptations to all 4 hues were probed for 20 times. As exemplified in Figure 5A–D for Subject 1, selective reductions in BOLD responses were observed only when the test hue was identical to the adaptation hue. To quantify the adaptation effect, we also replaced the adaptation hue with an isoluminant gray background (namely without hue adaptation, see Fig. 5E) and calculated an “adaptation index (AI)” for the mean amplitude of BOLD responses in each area: AI = 1.0 − (BOLD response with color adaptation/BOLD response without color adaptation). The BOLD response was defined as the ratio of the mean response taken 4–12 s after the onset of the test hue to that preceding the onset. The voxels used for generating AIs were a portion of voxels selected from the scan without color adaptation. For each ROI, a voxel was selected using a set of criteria similar to those used in the hue-selectivity experiment: The r² value was >0.15, the BOLD response was not negative, and the magnitude of BOLD response did not exceed 5%. Under each hue-adaptation condition, 4 AI values in each area, averaged across voxels, were obtained for each of the 4 test hues (Fig. 6A for results from Subject 1). If responses to intermediate hues are merely due to the mechanisms that combine responses to the hues on the 2 neighboring cardinal axes, the reduced response after adaptation should not be observed only for the test hue that was the same as the adaptation hue, but also for the 2 neighboring test hues, which share responses to the hues on either of the 2 cardinal axes. However, as demonstrated in Figure 6A, this was not the case; selective reduction (with a higher AI value) was observed for the test hue that was identical to the adaptation hue, except for the case with the adaptation hue at 315° in V1, V2, V3, and V4, which corresponds to the under-represented hue range for this subject (Subject 1, see Figs 3A and 4A) and for the case at 135° in V1. Similar results were also obtained from the remaining 2 subjects (Fig. 6B,C).

Figure 5.

Selective adaptation to intermediate hues in Subject 1. (A) A selective reduction, as a consequence of adaptation, was found only when the test color (45° in the hue angle) was the same as the adaptation color (45°). Note that there were no adaptation-induced decreases in BOLD responses when tested with colors of 135° and 315°. The hemodynamic responses were averaged across V1, V2, V3, and V4 voxels. Error bars represent standard errors. (B–D) BOLD responses for test hues under adaptation to 135°, 225°, and 315° in the same subject. (E) BOLD responses to 4 test stimuli without adaptation, in which voxels responded equally well to all 4 test colors.

Figure 6.

(A) Adaptation indices calculated for the 4 intermediate colors in response to different adaptation colors. The numbers of selected voxels in ROIs (V1, V2, V3, V4, total across 4 visual areas) and their proportions in respective ROIs were (98 [19.4%], 138 [33.8%], 126 [30.0%], 45 [15.2%], 407 [25%]) for Subject 1, (95 [19.8%], 147 [23.9%], 91 [12.7%], 75 [13.3%], 408 [17%]) for Subject 2, and (61 [6.96%], 50 [5.58%], 64 [10.7%], 43 [10.2%], 218 [7.8%]) for Subject 3, respectively. In each case, the greatest adaptation index (and thus the largest adaptation-induced reduction in BOLD response) was found for the color when the subject was adapted to the same color, except for the case with adaptation to the color of 315°, which was in the under-represented hue range for this subject (Subject 1). (B, C) Results of the same analysis on Subjects 2 and 3. (D) Average of adaptation indices calculated across 3 subjects. Error bars represent standard errors. Asterisks on the left side of symbols indicate that the adaptation indices in the adapted hue directions were significantly different from other conditions (*P < 0.05, **P < 0.01; one-way ANOVA with repeated measures). (E) The same plot as (D) but the AIs are aligned to adapted hues. (F) Summarized result of (E), averaged across adapted hues. See also Figure 7 for dependence of adaptation-induced response reduction on a voxel's color preference.

Adaptation indices averaged across 3 subjects are summarized in Figure 6D (see also Fig. 6E,F). Overall, the response reduction was largest and significant when tested with the adapted hue, manifesting a hue-selective adaptation effect, in particular in V1, V2, and V3. Similar trend was also observed in V4 for several conditions, but the effect was less significant, probably because the number of voxels available for this analysis was smaller in V4. Statistical significance was not observed for the 315° condition in all areas, primarily due to the poor adaptation effect in Subject 1, resulting, presumably, from the under-represented hue selectivity in this hue range in this subject (Figs 3A and 4A). Quantitatively, we classified AIs by the difference between the test and adapted stimuli, namely, the adapted hue (test and adapted hues were the same), adjacent hues (test and adapted hues differed by 90° on either side), and the opponent hue (differed by 180°), and performed a one-factor ANOVA with repeated measures in each visual area with test-stimulus conditions as the factor. The result revealed a significant effect of test-stimulus classes: F_3,44 = 12.3 in V1, F_3,44 = 19.9 in V2, F_3,44 = 16.2 in V3, and F_3,44 = 7.43 in V4; all P < 0.0005. A post-hoc test for the multiple comparisons between test-stimulus classes further showed that AIs for the tests with adapted hues were significantly larger than AIs for other test hues (all P < 0.05), AIs for nonadapted hues (i.e., tests with adjacent hues and the opponent hue) were not significantly different between them (all t₁₁ < 0.19; n.s.), and all AIs for nonadapted hues were not significantly different from zero (all t₁₁ < 0.98; n.s.).

We also performed a quantitative analysis for the difference between a “half” of the AI value for the adapted hue and AI values for the 2 adjacent test hues (+90° and −90° away from the adapted hue). If responses to the intermediate hues comprised a combination of mechanisms selective for cone-opponent axes, and if the cone-opponent mechanisms related to the adapted hue were fully adapted, responses to the 2 adjacent test hues (e.g., test hues of 135° and 315° with regard to the adapted hue of 45°) would be reduced by 50%, because 1 of the 2 cone-opponent mechanisms would be desensitized as a consequence of adaptation. A two-tailed t-test against 0.5, on AI values for the 2 adjacent hues as a priori test, confirmed that AI values for the 2 adjacent hues were significantly smaller than the “half” of the AI value for the adapted hue in all visual areas (all P < 0.05), except for 45° and 315°conditions in V4.

Finally, we tested whether a voxel's adaptation-induced response reduction depended on the voxel's color preference. For this purpose, we selected a portion of voxels that exhibited maximal responses to a hue (preferred hue) and minimal responses to the hue in the opposite direction and performed an analysis by directly linking a voxel's preferred hue to AI values when the voxel's preferred hue was adapted. The voxels entered into this analysis were chosen, from those used for generating AIs in Figure 6 (407, 408, and 218 voxels from Subjects 1, 2, and 3, respectively, across all 4 visual areas), based on the following criteria. First, a voxel's amplitudes of BOLD responses to the 4 test hues in the no-adaptation experiment were used to define the order of the voxel's hue preference. We then selected the voxels that responded maximally to a hue (preferred hue, e.g., 45°) and in the meantime minimally to the hue in the opposite direction (opposite hue, e.g., 225°). In total, 167 (40.9%), 130 (31.9%), and 99 (45.4%) voxels, pooled from all 4 visual areas, passed these criteria in Subjects 1, 2, and 3, respectively. Finally, we calculated the 2 adaptation indices (AIs) for a voxel when it was respectively adapted to the 2 colors and obtained an AI ratio for the voxel by dividing the AI for the preferred hue by that for the opposite hue. If the degree of adaptation of a voxel was affected by the voxel's hue preference, the AI ratio for the voxel would be >1.0 and the greatest when the adapted hue was identical to the voxel's preferred hue. As shown in Figure 7, this was indeed the case (solid curve). This selective adaptation effect disappeared when identical voxels were used, but the relationship between their preferred hues and adapted hues was randomized (dashed curve in Fig. 7). A two-factor (“voxel-selection method” and “preferred-hue direction”) ANOVA was applied. The main effects of “selection method” (F_1,33 = 6.56, P < 0.05) and “preferred hue” (F_3,33 = 11.57, P < 0.0001) were both significant, in addition to interactions between the factors (F_3,33 = 11.76, P < 0.0001). A post-hoc t-test on the AI difference between preferred hue angles for the solid curve in Figure 7 was conducted (indicated by asterisks), revealing that AI ratios were the highest when adapted hue was equal to the voxel's preferred hue, and decreased progressively with the angular distance from the adapted hue. This result suggests a systematic link between the adaptation-induced response reduction and a voxel's color preference.

Figure 7.

Dependence of adaptation-induced response reduction on a voxel's color preference. The abscissa represents the direction of preferred hue of each voxel with respect to the adapted hue, and the ordinate represents the ratio of adaptation indices (AIs). The results were averaged across subjects and sorted based on the adapted hue angle. Error bars indicate standard errors. The solid curve represents the result when the degree of adaptation of a voxel was calculated based on the voxel's hue preference, whereas the dashed curve represents the result when identical voxels were used but the relationship between their preferred hues and adapted hues was randomized.

The results from this adaptation experiment, thus, provide supporting evidence that the hue-selective histograms revealed in our study indeed represent the characteristics of neurons that are selective for individual, intermediate hues.

No Progressive Adaptation in Color Appearance

In the fMRI experiment, although stimulus colors were alternatively presented with the gray background at a frequency of 2.5 Hz, it was still possible that local adaptation to color stimuli took place during a 36-s trial of continuous stimulation. If adaptation had indeed taken place, responses to neighboring, similar hues in the hue circle would have been reduced, thus leading to the advancement in perceiving an apparent hue, and ultimately a mismatch between the color appearance and the shape of hue-selective histograms. In a psychophysical test, we confirmed whether there was the discrepancy between the color appearance and its corresponding hue angle as a consequence of adaptation to continuous color change in the stimulus. The circular checkerboard pattern used in the main fMRI experiment was split into the left and right halves in this test. Colors of checker elements along the vertical line separating the 2 half fields were the same as the gray background. The checkers in both the left and right fields were initially given a blue color (90° in the hue circle), then the checkers' colors in the 2 half fields changed in opposite directions along the hue circle, that is, they changed along the hue circle with the radii of L_amp = 0.08 and S_amp = 0.80, as illustrated in Figure 1A, in either clockwise or counter-clockwise direction. The other aspects of the stimulus were exactly the same as in the main fMRI experiment: The resolution of color change was 3° per step, the color and gray background alternated at a rate of 5 frames per second, the duration for one cycle of color change was 24 s, and one and a half cycles of color change (36 s) were presented, which was followed by a 24-s period with the gray background after the color change ceased. The subjects were asked to report the instance of color match between the 2 sides by pressing a button. The match could appear in 3 occasions, the first around 270° (yellow color), the second around 90° (blue), and the last one, toward the end of 1.5 cycles of color change, around 270° again. The subjects were instructed to concentrate on the first 2 instances. If the continuous change in color resulted in any progressive adaptation in the color appearance, this would affect the 2 half fields in opposite ways, thus leading to reported hue angles that were different from either 270° or 90°. The results from 2 subjects, who participated in 10 repetitions of measurements, showed no significant difference between reported hue angles and expected ones, that is, 270° and 90° (two-tailed t-test: t₉ = 1.19, P = 0.264, and t₉ = 1.51, P = 0.165 for the 2 subjects, respectively). Based on this observation, we conclude that with the stimulus in our main fMRI experiment, there was no significant effect of progressive adaptation on the hue angle and color appearance.

Discussion

Presence of Neurons Selective for Intermediate Hues in early Visual Cortex

In the present fMRI study, we used a continuous stimulation paradigm and a differential analysis method to uncover the hue selectivity in human visual cortex at the level of single voxels. Our results demonstrate that there existed a wide spectrum of hue selectivity in early visual cortex (as early as primary visual cortex), which included both the cone-opponent systems and the voxels preferring intermediate hues away from the L − M and S axes. This observation is consistent with single-unit recording findings in macaque monkeys (Hanazawa et al. 2000; Wachtler et al. 2003), thus confirming analogous characteristics between humans and nonhuman primates in terms of the hue selectivity. In addition, in a series of experiments with different S-cone contrasts, we established in a systematic way that when stimulations along the L − M and S axes were equated by the multiples of color detection thresholds (i.e., L_amp = 0.08 and S_amp = 0.80), there was an over-representation of the voxels preferring S-cone selective hue directions. The optimal ratio between the L-cone stimulation and S-cone stimulation (and thus altered sampling rates along L − M and S axes, respectively) for yielding a relatively flat distribution among hue-selective voxels was 1:5.66 (i.e., L_amp = 0.08 and S_amp = 0.453). Finally, we discovered that hue-selective histograms across early visual areas in individual subjects exhibited consistent anisotropies, even after the “oversampling” around the S-cone axis was compensated in fMRI measurements (Figs 3 and 4).

Previous single-unit recording and intrinsic optical imaging studies in macaques have revealed that color-selective neurons are modularly organized in early visual areas (Xiao et al. 2007; Wang et al. 2007; Tanigawa et al. 2010), forming color-specific domains (color maps), where color preference changes gradually across cortical surface, at a spatial scale of tens of microns to ∼0.5 mm. Similar color maps are thought to exist in human visual cortex as well, but there is no means to visualize them directly with currently available techniques, including fMRI; although individual color domains in humans can be larger than those in monkeys, they are still much finer than the typical resolution that the current fMRI can offer. With the spatial sampling scale employed in our study (i.e., 2 × 2 × 3 mm³), it may be natural to assume that multiple color domains are contained in a single voxel, rather than assuming that particular hue-selective neurons make large clusters in the voxel. If color domains for different hues were equally distributed in a voxel, the voxel would be expected to exhibit no selectivity for particular hues. Presumably, most of the voxels in visual cortex possess this kind of null responses, and thus, it may be reasonable to speculate that our hue-selective histograms represented the excess from the isotropic hue-selective population (Figs 2, 3 and 4). In this regard, we would stress that, while our approach allowed us to conclude that neurons selective for intermediate hues that are off the cardinal axes were indeed present in human visual cortex, it is not sufficient for us to address the general question as how color is represented in visual cortex.

It may also be argued that the hue-selective property observed in single fMRI voxels is due to the difference in spatial properties of neurons' receptive fields, as it has been documented previously that receptive-field shapes of color-selective cells are often asymmetric in the visual field (Conway and Livingstone 2006). The “receptive field” of an fMRI voxel in the visual field, however, can be modeled as the sum of overlapped receptive fields of neurons inside the voxel. Because the number of neurons contained in an fMRI voxel size is quite large, in the order of 10⁶ or even more, it is unlikely that the summed receptive field is still asymmetric. Instead, it is more natural to assume that, though the variability of spatial asymmetry of neurons' receptive fields is quite large, the overall average of receptive fields within a voxel is in effect rather uniform. Thus, we consider it extremely unlikely that hue-selective responses observed in single voxels were resulted from the spatial asymmetry of neurons' receptive fields contained in these voxels.

Another possible argument is that the isoluminance defined by minimum flicker at 16 or 20 Hz, depending on subject (see Methods section), did not equate stimulus strength for alternations of color checkers at 2.5 Hz (Wyszecki 1967), that is, alternations between the background gray and a test hue with 15% luminance increment. We have thus conducted a similar minimum-flicker experiment at 2.5 Hz and confirmed that the results of measurements with hues equated for minimum flicker at a higher (16 or 20 Hz) and a lower (2.5 Hz) temporal frequency did not differ significantly (see Fig. S8 for details). This may be partly due to the 15% luminance pedestal, which might have masked any residual difference in the stimulus strength across hues.

Our results suggest that it is possible to reveal much more voxels that code different colors, thereby allowing us to assess the spatial features of their distributions in more detail with fMRI at higher spatial resolutions in the future. While high-resolution fMRI has been demanding in the past, recent technological advancement in accelerated imaging methods, such as multiband acquisition and parallel imaging, together with the increasingly available ultra-high field (≥7 Tesla) MRI systems that provide improved SNRs, has made it feasible to pursue such high-resolution studies using the stimulation paradigm and analysis method developed in the present study.

Comparison with Other Color Studies using fMRI

Our results, by directly revealing the hue selectivity of single voxels across early visual areas V1 through V4, are broadly in agreement with, and lend support for, the results from several previous attempts using fMRI decoding approaches to investigate how colors are coded in human visual system (Brouwer and Heeger 2009; Parkes et al. 2009; Goddard et al. 2010; Kuriki et al. 2011). By decoding fMRI responses to a pair of colors in diagonal directions, such as 45° and 225° (appearing in magenta and lime green, respectively) and 135° and 315° (in cyan and orange), the presence of neurons responding selectively to the off-axis hues has been indirectly inferred (Goddard et al. 2010; Kuriki et al. 2011). Consistent with the finding by Goddard and colleagues (2010) that there is a systematic bias in the magnitude of BOLD responses to various hues, the population results from our study also exhibit a significant decrease of hue-selective voxels in either the fourth or the second quadrant, which includes the hues appearing in reddish yellow or cyan, respectively. The poverty of the voxels in a particular hue direction varied among subjects, but such a bias in the population of hue-selective voxels appears to be a general feature in the color-coding system.

One obvious limitation of using fMRI to study color selectivity in early visual cortex is the lack of a clear correspondence between color-selective BOLD responses and the supra-threshold performance in color perception. Categorical color perception is one of the supra-threshold color perceptions, but it represents colors in a symbolic and discrete manner. The colors we pictorially experience in our daily life, including unique hues, should be represented with a much finer precision. Therefore, mechanisms for these color perceptions may reside at different levels in human visual system. Results from previous studies (Engel et al. 1997a; Mullen et al. 2007), as well as our own results (Fig. S5), have pointed out mismatches between the detection threshold and BOLD signal equilibrium, in terms of “threshold” differences between L − M and S axes. However, threshold-level performances tend to reflect the characteristics of lower-level mechanisms (e.g., Derrington et al. 1984; Chaparro et al. 1993). It is well known that the supra-threshold characteristics are significantly different from performances at the threshold level in some cases (Krauskopf et al. 1986; Webster and Mollon 1991; Goda and Fujii 2001; Hansen and Gegenfurtner 2005; Kuriki 2007), and it is expected that the cortical activity reflects the neural representation of color appearance at the supra-threshold level. Unique hues (De Valois et al. 2000), for example, can be one of the candidates for investigation in human visual cortex. Attempts have been made in previous studies using fMRI decoding approaches to demonstrate that unique hues can be principal axes for the representation of colors in human V1 (Parkes et al. 2009), though a clear correspondence between hue-selective histograms and unique hues was not found in the present study (Figs 3 and 4; see also Supplementary Fig. 9 for further discussion). Future fMRI studies on color responses in early visual cortex, in our opinion, should seek any correspondence between BOLD responses to color stimuli and psychophysical performances in terms of color appearance, in which the subject's percept can be precisely accounted for by the brain activity.

Individual Differences and Cortical Color Representation

One prominent feature in the obtained hue-selective histograms is large individual differences. Anisotropic hue representation, as observed in our study, may be a genuine cortical mechanism; however, the approach used in our study, in particular, the spatial resolution, does not allow us to make such a decisive conclusion about the degree of anisotropy/uniformity. Future studies are warranted for addressing this potentially interesting question.

Nevertheless, despite large individual differences in the histogram shapes, there seem to be some trends within subjects and between areas V1, V2, and V3 that do not extend to V4. One possibility is that color representations in early visual areas (V1–V3) and V4 are different. It has been argued, for example, in the fMRI study by Brouwer and Heeger (2009), that the hue circle is better represented in V4 than that in earlier visual areas. The relatively diverse hue selectivity in V4 observed in our study lends support, to some extent, to their conclusion. In addition, the results from the hue-selective adaptation experiment also revealed a distinct feature in V4; the AIs for adjacent directions and the direction opposite to the adapted hue were smaller than zero (Fig. 6F). This implies that the response to the test hue was larger than that obtained under the no-adaptation condition. This might reflect a shift in the baseline of neural response, due to chromatic adaptation, similar to the phenomenon of the enhanced chromatic contrast in color appearance, namely, chromatic adaptation aftereffect, for which V4, but not earlier visual areas, is thought to be responsible (Sakai et al. 1995). In this context, color code in V1-V3 may be fed to and reorganized in V4 to form a neural signal that is more relevant to the higher-order color perception.

Supplementary Material

Supplementary material can be found at: http://www.cercor.oxfordjournals.org/.

Funding

This study was supported in part by Grant-in-Aid for Scientific Research (KAKENHI 21330165, 24330205, and 25245065 to I.K., and 20300114 and 25242079 to K.C.) from the Japan Society for the Promotion of Science (JSPS). Funding to pay the Open Access publication charges for this article was provided by the Japan Society for the Promotion of Sciences.