Effects of Luminance Contrast on the Color Selectivity of Neurons in the Macaque Area V4 and Inferior Temporal Cortex

Abstract

Appearance of a color stimulus is significantly affected by the contrast between its luminance and the luminance of the background. In the present study, we used stimuli evenly distributed on the CIE-xy chromaticity diagram to examine how luminance contrast affects neural representation of color in V4 and the anterior inferior temporal (AITC) and posterior inferior temporal (PITC) color areas (Banno et al., 2011). The activities of single neurons were recorded from monkeys performing a visual fixation task, and the effects of luminance contrast on the color selectivity of individual neurons and their population responses were systematically examined by comparing responses to color stimuli that were brighter or darker than the background. We found that the effects of luminance contrast differed considerably across V4 and the PITC and AITC. In both V4 and the PITC, the effects of luminance contrast on the population responses of color-selective neurons depended on color. In V4, the size of the effect was largest for blue and cyan, whereas in the PITC, the effect gradually increased as the saturation of the color stimulus was reduced, and was especially large with neutral colors (white, gray, black). The pattern observed in the PITC resembles the effect of luminance contrast on color appearance, suggesting PITC neurons are closely involved in the formation of the perceived appearance of color. By contrast, the color selectivities of AITC neurons were little affected by luminance contrast, indicating that hue and saturation of color stimuli are represented independently of luminance contrast in the AITC.

Introduction

The higher ventral cortical areas of macaque monkeys play critical roles in their color perception (Zeki, 1980; Horel, 1994; Heywood et al., 1995; Buckley et al., 1997; Komatsu, 1998; Huxlin et al., 2000; Cowey et al., 2001, 2010; Gegenfurtner, 2003), and many physiological studies have examined the properties of color-selective neurons in area V4 and the inferior temporal cortex (IT). However, one important problem that remains to be fully addressed is the relationship between color and luminance, as the appearance of a color stimulus changes with changes in the luminance contrast between the stimulus and its background. In human color perception, for example, when the luminance of stimulus becomes greater than the background, the appearance of the stimulus can be shifted from black to white or from brown to orange. It is not well understood where or how the interaction between color and luminance signals occurs in the ventral visual areas and forms the appearance of colors dependent on luminance contrast.

The first study to systematically investigate the impact of luminance contrast on color tuning was conducted by Conway et al. (2007), which examined the luminance contrast in the posterior IT (PIT) and V4 using color stimuli with high saturation that situated at the edge of the color gamut. In their study, responses to colors with low saturation were not tested though the effects of luminance contrast on color perception tend to be large for colors with low saturation. More recently, Bushnell et al. (2011) tested the responses of V4 neurons using stimuli that spanned the entire range of gamut and with varying luminance contrast. However, they did not quantitatively examine the effects of luminance contrast on the color selective responses. To fully understand the effect of luminance contrast on the responses of color-selective neurons, it should be important to use color stimuli that span as wide a range of chromaticity as possible and include colors with both high- and low-saturation.

In the present study, we aimed to systematically examine how the neural representation of color in V4 and IT is affected by the luminance contrast of the color stimuli. In the IT, we targeted subregions in the IT gyrus, where color-selective neurons are accumulated [PIT color area (PITC) and anterior IT color area (AITC); Banno et al., 2011; Lafer-Sousa and Conway, 2013]. We used color stimuli that were evenly distributed across the entire color gamut of the display, and examined whether color selectivity changes depending on the luminance contrast, and whether the population responses to individual colors change depending on the luminance contrast across V4, the PITC, and the AITC. We found that the effect of luminance contrast on the neural representation of color markedly differed among these three areas. Large effects were observed in V4 and PITC, whereas there was little effect in AITC.

Materials and Methods

Four adult male macaque monkeys (Macaca fuscata, weighing 5.1–7.7 kg) and two adult female macaque monkeys (weighing 5.0–6.3 kg) were used for these experiments. All procedures for animal care and experimentation were in accordance with the NIH Guide for the Care and Use of Laboratory Animals and were approved by Institutional Animal Care and Use Committee of the National Institute of Natural Sciences.

Surgery.

Before making electrophysiological recordings, sterile surgery was conducted under general anesthesia to attach a head holder (metal or plastic) and a plastic recording chamber to the skull of each monkey using dental cement and cortical screw. After surgery, the monkeys were allowed to recover for at least 1 week before electrophysiological recording was begun. During the week after the surgery, an antibiotic (cefazolin sodium, 10 mg/kg) was given every 12 h.

Recording sites.

In the present study, we determined the recording sites based on stereo coordinates and sulcal landmarks using MRI images as references. Before surgery, the positions of the lunate sulcus, superior temporal sulcus (STS), inferior occipital sulcus, posterior middle temporal sulcus (PMTS), and anterior middle temporal sulcus (AMTS) were identified on MRI images. We recorded neuronal activities in the AITC in two hemispheres of two monkeys (Monkeys CO and KM), in the PITC in two hemispheres of two monkeys (Monkeys KM and LW), and in area V4 in four hemispheres of three monkeys (Monkeys AL, SI, and SK) (Fig. 1A). The recording chamber for the AITC was placed above the position corresponding to posterior end of the AMTS, so that electrodes could be advanced vertically through the brain to reach the IT gyrus. The recording chamber for the PITC was placed on the lateral surface of the IT gyrus so that it covered the regions dorsal and ventral to the PMTS. The recording chamber for area V4 was placed so that it covered the positions corresponding to dorsal V4 in the prelunate gyrus. For the PITC and V4, electrodes were advanced directly into those areas through the dura.

Figure 1.

Recording sites and color stimuli. A, Approximate recording sites (red marks) in a lateral view of the right hemisphere of a monkey (top, V4; middle, PITC; bottom, AITC). B, Chromaticity coordinates of the stimuli in the color stimulus set plotted on the CIE-xy chromaticity diagram. Each color stimulus set contained 16 colors, including 15 chromatic colors whose chromaticity coordinates were evenly distributed on the chromaticity diagram (colors 1–15) and one achromatic color whose chromaticity coordinate was equal to the gray background (color 16). Fifteen stimuli (all except color 15) were presented at two different luminances (bright set, 20 cd/m²; dark set, 5cd/m²). Because the luminance of blue (color 15) in the bright set was different from the other colors, we omitted responses to that color elicited in both the bright and dark sets from the quantitative analysis. The white triangle indicates the gamut of displayed colors used for the experiments. C, Nineteen geometric shapes used to test the shape selectivity (V4, 19 or 11 shapes; PITC and AITC, 11 or 7 shapes). These shapes include, from top left to bottom right, square, oblique square, circle, star, cross, oblique cross, triangle, vertical bar, oblique bar in the clockwise direction, horizontal bar, oblique bar in the counterclockwise direction, narrow diamonds and broad diamonds (vertical diamond, oblique diamond in the clockwise direction, horizontal diamond and oblique diamond in the counterclockwise direction, respectively). Each of these shape stimuli was painted uniformly with a single color. LS, Lunate sulcus; IOS, inferior occipital sulcus.

To precisely locate the electrodes, we used plastic grids. Electrodes were advanced through a stainless steel guide tube situated within a plastic grid in which holes were placed at 1 mm intervals. We used two types of grids in which the positions of holes were shifted 0.5 mm vertically and horizontally with respect to one another so that a minimum interval of 0.7 mm between holes was attained.

The sites of V4 penetration were identified based on the grid coordinates on MRI images and the depth profiles of the electrode penetrations. The sites at which PITC neurons were recorded were confirmed by histological observation under a microscope after all recording sessions had been completed. Details of the histological procedures and recording sites in the PITC were described previously (Yasuda et al., 2010; Banno et al., 2011). In addition, some of the results of the PITC recordings were reported in an earlier paper (Yasuda et al., 2010). The recording sites in the AITC were located around the posterior part of the AMTS and slightly lateral to it, in the IT gyrus (Monkey CO: A13–A23, L16–L22; Monkey KM: A16–A24, L16–L22). The sites at which AITC neurons were recorded in monkey KM were checked by histological observation. We identified the recording sites in the AITC of monkey CO on MRI images based on the depth profile of the electrode tracks and by superimposing the x-ray image (Toshiba TR-80A-ES-L, 70 kV, 20 mA, and 0.4 s) of the electrode on the MRI image.

Visual stimuli.

Visual stimuli were generated using a graphics board (VSG2/3, Cambridge Research Systems) in a computer and displayed on a cathode-ray tube (CRT) monitor (Sony GDM-F500R, TOTOKU CV921X). The chromaticity coordinates and the luminance of the visual stimuli were calibrated using a spectrophotometer (PR650, Photo Research) or a colorimeter (CS200, Konica Minolta). Visual stimuli were presented on a neutral gray background (10 cd/m², x = 0.3127, y = 0.3290; Fig. 1B, color 16).

The main question addressed in this study is how does the polarity of the luminance contrast of a stimulus affect the responses of color-selective neurons. To study this problem, we used two color stimulus sets that were defined on the basis of the CIE 1931-xy chromaticity diagram (Fig. 1B) and whose luminance was either brighter (bright set) or darker (dark set) than the gray background. Both color stimulus sets contained 16 colors (Fig. 1B, colors 1–16): 15 chromatic colors whose chromaticity coordinates were evenly distributed on the chromaticity diagram (Fig. 1B, colors 1–15) and one achromatic color whose chromaticity coordinates were equal to the gray background (Fig. 1B, color 16). Each pair of the 16 colors in the bright set and dark set had the same chromaticity coordinates. In the bright set, all stimuli were brighter than the background (10 cd/m²), and all except a blue color (15; 11–12 cd/m²) had the same luminance (20 cd/m²). In the dark set, all 16 stimuli were darker than the background (10 cd/m²) and had the same luminance (5 cd/m²). The luminance contrasts of the bright and dark sets were equalized in terms of the Michelson contrast. Because the luminance of blue color (15) in the bright set differed from the other colors, we omitted the responses to that color from the quantitative analysis of both the bright and dark sets. We tested the bright and dark set in separate blocks.

In the recordings from the PITC and V4, we mapped the receptive fields (RFs) of each neuron by presenting the preferred stimulus at various positions in the visual field, and visual stimuli to test the color selectivity were presented in the RF. In the recordings from the AITC, visual stimuli of adequate size were presented at the foveal center. Stationary flash stimuli were used in all experiments.

When we assessed color selectivity, the shape of the stimuli was fixed and chosen from 7 to 19 geometric shapes (Fig. 1C; square, oblique square, circle, star, cross, oblique cross, triangle, vertical bar, and oblique bar in the clockwise direction; horizontal bar and oblique bar in the counterclockwise direction; narrow diamonds and broad diamonds: vertical diamond, oblique diamond in the clockwise direction, horizontal diamond and oblique diamond in the counterclockwise direction). Many V4 and PITC neurons exhibited selectivity for both color and shape, so after a single unit was isolated, we attempted to find the optimum combination of color and shape that elicited the strongest neuronal response. In V4, we used all 19 shapes (95 neurons) or 11 shapes other than diamonds (48 neurons). In the PITC, we used 11 shapes other than diamonds (82 neurons) or seven shapes other than diamonds and bars (2 neurons). In the AITC, we used 11 shapes other than diamonds (125 neurons) or seven shapes other than diamonds and bars (23 neurons). Each of these shape stimuli was painted homogeneously with a single color.

Behavioral task.

During the experiment, monkeys were seated on a primate chair and faced the CRT monitor at a distance of 56 cm. The monkeys were trained to fixate on a small white dot (0.1° in diameter) presented at the center of the monitor. The monkeys were required to maintain eye position within an eye window (1.5 × 1.5 to 3.0 × 3.0° for recording from the IT and 1.5 × 1.5 to 1.85 × 1.85° for recording from V4) throughout the trial. If the monkey maintained fixation until the fixation spot disappeared, a drop of water or juice was given as a reward. If the monkey's gaze deviated from the eye window, the trial was aborted, and an intertrial interval immediately started. Eye position was monitored using an eye coil or an infrared eye camera system (ISCAN).

A trial started when the fixation spot was turned on, and each color stimulus was presented 1–5 times for a period of 300 or 500 ms within a trial. When more than one stimulus was presented within a trial, the interstimulus interval was either 200 or 300 ms. When the visual stimulus was presented at the foveal center in the IT recordings, the fixation spot was turned off for a period extending from 300 or 350 ms before the onset of the visual stimulus until 260 or 300 ms after the offset of the stimulus, respectively. Otherwise, the fixation spot was on during the entire trial. A stimulus was chosen randomly from the stimulus set in each trial, regardless of the recording site.

Electrophysiology.

A varnish-coated tungsten microelectrode (200 μm in diameter, Frederick Haer) was advanced through a stainless steel guide tube fixed within a grid hole using a hydraulic microdrive (MO-951, Narishige), and neuronal activities were recorded. To record from the PITC or V4, the tip of a stainless guide tube was fixed in contact with the dura mater, and the electrode was advanced through the dura mater. To record from the AITC, we first systematically mapped regions where color selective neurons were clustered and then inserted guide tubes into the brain such that the tips of the tubes were positioned ∼5–10 mm above the targeted cortical regions. We then extensively sampled neuronal activities in the targeted regions using thinner tungsten microelectrodes (125 μm in diameter, Frederick Haer). Recordings were made through each guide tube inserted into the brain for up to 3 weeks.

Neuronal signals were amplified, sampled at 25 kHz, and stored on a computer for off-line analysis. Behavioral events were recorded at 1 kHz. To inspect the visual responses, neuronal signals were discriminated based on spike amplitude, converted to pulses, and displayed online as rasters and peristimulus time histograms (PSTHs). Neuronal signals and discriminated pulses were also fed to a speaker for audio monitoring.

Data analysis.

Off-line quantitative data analysis was conducted only for single neurons. We first used a template-matching algorithm to isolate spikes with a temporal resolution of 1 ms. We then computed the average firing rate of the isolated spikes during a period spanning 50–350 ms after stimulus onset, taking into account a response latency of 50 ms. From this average, we subtracted the firing rate before presentation of the first stimulus in the trial (300–0 ms before stimulus onset, baseline activity), and the resultant rate was taken as a measure of the neuronal response to the visual stimulus. Neural responses were analyzed only for correct trials, and the minimum number of repetitions of each stimulus accepted for analysis was five. Only neurons whose response to the optimal color was >10 spikes/s and whose discharge rates during presentation of the optimal color were significantly different from baseline (Student's t test, p < 0.05) were included in the sample for analysis. In the following text, “significant responses” means that the responses to a given stimulus set satisfied the above two criteria.

To quantify the strength of the color selectivity of each neuron, a selectivity index was calculated as 1 − (minimum response)/(maximum response). We also used one-way ANOVA to evaluate whether the variation in the responses to stimuli within a set of test stimuli was significant. When the selectivity index was larger than 0.6 (i.e., the maximum response was >2.5 times the minimum response) and response variation was significant (ANOVA, p < 0.05), the neuronal responses were regarded as stimulus selective. To quantify the sharpness of the stimulus selectivity, we calculated a sparseness index (Rolls and Tovee, 1995; Vinje and Gallant, 2000), which was defined as follows:

where r_i is the firing rate elicited by the i′th stimulus in the set of n stimuli. If r_i was a negative value, it was replaced with 0. This index indicates the degree to which responses are unevenly distributed across the set of stimuli. When all stimuli evoke the same response amplitude, the sparseness index is its minimum and has a value of 0. As the stimulus selectivity sharpens, the index becomes larger and approaches its maximum of 1.

Comparisons of the responses to the bright and dark sets were conducted for neurons in which both the bright and dark sets generated significant responses, and the selectivity index for at least the bright or dark set was >0.6. In the following text, these neurons will be referred to as “color-selective.” Neurons that exhibited a significant response to only one of the bright or dark set were not analyzed, even when they exhibited color selectivity to the responsive set. A color-selective neuron whose sparseness index was larger than 0.3 for at least one stimulus set was deemed to be “sharply color-selective.” On the other hand, a color-selective neuron whose sparseness index was smaller than 0.3 for both the bright and dark sets was deemed to be “broadly color-selective.” The sparseness index was calculated using responses to all colors except color 16, which distributed evenly on the chromaticity diagram.

Test of the effect of the luminance contrast.

We quantitatively examined the effect of changing the polarity of the luminance contrast on the response properties of color-selective neurons in several ways. First, to examine the effect of the luminance contrast of the stimuli on the color selectivity of each neuron, we calculated the Pearson's correlation coefficient between responses to 15 colors of the bright and dark sets (all colors except 15) for each neuron. Second, we computed four measures of the response properties, maximum response across a stimulus set, mean response across a stimulus set, selectivity index and sparseness index, for both the bright and dark sets and compared the distribution of each of the aforementioned measures across neurons in each area between the two stimulus sets (two-sample Wilcoxon signed-rank test). These measures were calculated for the bright and dark sets separately based on the responses to 14 chromatic colors except for blue (color 15) and an achromatic color (color 16).

To examine how the population responses of color-selective neurons varied with respect to the polarity of the luminance contrast, for each area we calculated Pearson's correlation coefficients (r) between the responses of a population of color-selective neurons to each color in the bright set and the responses to each color with the same chromaticity coordinates in the dark set. If a population of neurons exhibited the same pattern of responses to a color in the bright set and the same color in the dark set, the correlation coefficient would be 1. As the pattern of population responses elicited by a color became increasingly dissimilar between the bright and dark sets, the correlation coefficient would decrease.

Multidimensional scaling analysis.

To understand how the color-selective neurons in each area carry the information of color and luminance contrast, we conducted multidimensional scaling (MDS) analyses to quantitatively examine the dissimilarities of the population neural responses across the stimuli in the bright and dark sets. To do this, we first calculated Pearson's correlation coefficients (r) between the population responses of color-selective neurons to all possible pairs of stimuli in the bright and dark sets (a total of 30 stimuli except for blue, color 15). We then regarded 1 − r as the neural distance between two stimuli, and generated a distance matrix based on the neural distances across all pairs of stimuli. We next applied nonmetric MDS to the distance matrix, and the resultant dissimilarity between each stimulus was plotted on a two-dimensional plane.

Results

In the AITC, we recorded 155 well isolated visually responsive neurons, of which 107 showed visual responses to both the bright and dark sets (Monkeys CO: 60 neurons, KM: 47 neurons). Of those, 82 were classified as color-selective (Monkeys CO: 40, KM: 42; Table 1). In the PITC, we recorded 90 visually responsive neurons, of which 69 showed visual responses to both the bright and dark sets (Monkeys KM: 29, LW: 40). Of those, 58 were color-selective (Monkey KM: 29, LW: 29; Table 1). In V4, we recorded 149 visually responsive neurons, of which 108 showed visual responses to both the bright and dark sets (Monkeys AL: 59, SI: 30, SK: 39). Of those, 71 were color-selective (Monkeys AL: 40, SI: 19, SK: 12; Table 1). In the present study, the effect of luminance contrast was examined only in color-selective neurons that responded to both the bright and dark sets. There were also neurons that responded to either the bright or dark set (AITC, 48; PITC, 21; V4, 41; Table 1), and some of them showed color-selective responses (AITC, 32; PITC, 17; V4, 31; Table 1), but these neurons were not included in the subsequent analyses.

Table 1.

Neuron sample in each area

	No. of cells (%)
	AITC	PITC	V4
Visually responsive (total)	155 (100)	90 (100)	149 (100)
1. Visually responsive to both the bright and dark sets	107 (69.0)	69 (76.7)	108 (72.5)
1-1. Color-selective	82 (52.9)	58 (64.4)	71 (47.7)
1-1-1. Color-selective for both	68 (43.9)	44 (48.9)	46 (30.9)
1-1-2. Color-selective for either one	14 (9.0)	14 (15.6)	25 (16.8)
1-2. Not color-selective	25 (16.1)	11 (12.2)	37 (24.8)
2. Visually responsive to either the bright or dark set	48 (31.0)	21 (23.3)	41 (27.5)
2-1. Color-selective	32 (20.6)	17 (18.9)	31 (20.8)

Effects of luminance contrast on the color selectivity of individual neurons

The responses of three representative color-selective AITC neurons to the bright and dark sets are shown in Figure 2. Cell 1 was sharply tuned to red in both the bright (Fig. 2A, left) and dark (Fig. 2A, middle) sets. Responses to each color set are shown by rasters and PSTHs, as well as a bubble plot, in which response magnitudes to color stimuli are represented by the diameters of circles and are plotted at positions that correspond to their chromaticity coordinates. The pattern of color selectivity was similar between the bright and dark sets, though the response magnitudes differed: responses were stronger to the bright set than the dark set. The scatter plot shows the relationship between responses to the bright (horizontal axis) and dark (vertical axis) sets (Fig. 2A, right). The effect of luminance contrast on the color selectivity was quantified by calculating the Pearson's correlation coefficient between responses to the bright set and to the dark set. For cell 1 the correlation coefficient is large (r = 0.99), indicating that the color selectivity between responses to the bright and dark sets were highly similar for this neuron. Cell 2 showed similar color-selective responses to blue colors in both the bright and dark sets (Fig. 2B, left), which also exhibited strong correlation (r = 0.92; Fig. 2B, right). Cell 3 showed sharply color-selective responses that were strongest to green (color 1) in the bright set and to green-yellow (color 2) in the dark set. Color selectivity and response magnitudes were similar between the bright and dark sets (r = 0.81; Fig. 3C, right), although the preferred color was shifted slightly between them. As with these example neurons, the color selectivity of most AITC neurons was similar for the bright and dark sets.

Figure 2.

Responses of three example color-selective neurons recorded in the AITC. A, Responses of a neuron (Cell 1) to 15 stimuli in the bright set (left) and 15 stimuli in the dark set (middle). The response to each color is shown by rasters and PSTHs. The stimulus presentation period is indicated by a thick horizontal line below each histogram. The number at the bottom left of each histogram indicates the color ID that corresponds to the number of each color in Fig. 1B. Inset, The response magnitude to each color stimulus is represented by the diameter of a circle and is plotted at the position that corresponds to the chromaticity coordinate of that color (bubble plot). Open and solid circles indicate response increases and decreases, respectively. Contour lines in the bubble plots indicate 75%, 50%, 25%, and 0% of the maximum response, respectively. In the right panel, the scatter plot shows the relationship between the responses to the bright set (horizontal axis) and to the dark set (vertical axis). Each circle corresponds to one color. The correlation coefficient and the p-value between responses to the bright and dark sets are shown at the upper left of the plot. B, C, Responses of two other example neurons (B, Cell 2; C, Cell 3) to the bright and dark sets, and the relationship between the responses to the two sets (scatter plots). B, Bright set; D, dark set. Sparseness indices for the bright and dark sets were, respectively, 0.83 and 0.82 for Cell 1; 0.47 and 0.36 for Cell 2; and 0.63 and 0.67 for Cell 3.

Figure 3.

Responses of three example color-selective neurons recorded in the PITC. A–C, Responses of three neurons (A, Cell 4; B, Cell 5; C, Cell 6) are plotted using the same format as in Fig. 2. Sparseness indices for the bright and dark sets were, respectively, 0.73 and 0.69 for Cell 4; 0.38 and 0.28 for Cell 5; and 0.74 and 0.79 for Cell 6.

The responses of three representative color-selective PITC neurons are shown in Figure 3. Cell 4 was sharply tuned to colors ranging from green to blue in both the bright and dark sets (Fig. 3A). The response amplitudes and the strength and sharpness of the color selectivity observed with the bright set were analogous to those observed with the dark set. Cell 5 showed selective responses to red in both the bright and dark sets (Fig. 3B). The strength of the color selectivity and the maximum response amplitudes elicited with the bright set were similar to those elicited with the dark set, but the sharpness of the color selectivity differed greatly: the responses to the bright set were sharply tuned to red, whereas the selectivity for the dark set was broader. Although Cell 6 showed sharply color-selective responses to both the bright and dark sets, the responsive region in the chromaticity diagram clearly shifted depending on the luminance contrast (Fig. 3C). For the bright set, the largest response was to purple (color 14), but this neuron selectively responded to colors ranging from magenta to red (colors 9 and 12) in the dark set. Response magnitudes also differed between the two stimulus sets: the amplitude of the maximum response to the dark set was less than half of that to the bright set. Correlation coefficients between responses to the bright and dark sets for Cells 4, 5, and 6 were 0.97, 0.72, and 0.15, respectively.

The responses of three representative color-selective V4 neurons are shown in Figure 4. Cell 7 selectively responded to colors ranging from red to magenta in both the bright and dark sets, with the strongest response to magenta (color 12) in the bright set and to red (color 9) in the dark set (Fig. 4A); and the strength and sharpness of the color selectivity were similar between the bright and dark sets. Cell 8 responded to a range of colors from white to cyan to purple, with the strongest response to cyan (color 10) in the bright set and to more greenish colors around green-yellow with low saturation (color 7) in the dark set (Fig. 4B). In this neuron, both response amplitude and preferred color differed between the bright and dark sets. With Cell 9, the pattern of color selectivity also changed considerably between the bright and dark sets. This neuron showed broad and bimodal color-selective responses that peaked at green and purple in the bright set, whereas the strongest response elicited by the dark set was to colors around purple to red (Fig. 4C). Correlation coefficients between the responses to the two sets were 0.85, 0.48, and −0.17 for Cells 7, 8, and 9, respectively.

Figure 4.

Responses of three example color-selective neurons recorded in V4. A–C, Responses of three neurons (A, Cell 7; B, Cell 8; C, Cell 9) are plotted using the same format as in Fig. 2. Sparseness indices for the bright and dark sets were, respectively, 0.71 and 0.80 for Cell 7; 0.61 and 0.72 for Cell 8; and 0.15 and 0.21 for Cell 9.

As with the example neurons described above, the effect of luminance contrast on the color-selective responses varied across neurons, and it appears that the effect is smaller in the AITC than in other areas. To determine whether the effect of luminance contrast on color-selective responses differs across the three areas tested, the distributions of the correlation coefficients relating the bright and dark sets were compared for individual neurons (Fig. 5). In all three areas, the correlation coefficients for the neurons ranged from negative values to values close to 1, but the distributions differed across the three areas, with the smallest values tending to be in V4 and the largest tending to be in the AITC. Indeed, the distribution was skewed to values close to 1 in the AITC. The median correlation coefficients for V4, PITC, and AITC were 0.48, 0.56, and 0.78, respectively. The cumulative histogram of the correlation coefficients for the AITC was clearly shifted rightward compared with those for V4 and the PITC, and the cumulative histogram for the PITC appears to be shifted slightly rightward as compared with that for V4 (Fig. 5D). There was a significant difference between the distributions of correlation coefficients in the V4 and AITC (p < 0.001) and in PITC and AITC (p < 0.01; Mann–Whitney U test, Bonferroni corrected), whereas the difference between distribution in V4 and PITC was not significant. This confirms the impression that the effect of luminance contrast on the color-selective responses in the AITC was smaller than in V4 or the PITC.

Figure 5.

Distribution of the correlation coefficients between responses to the bright and dark sets. A–C, Bar graphs show the distribution of correlation coefficients between responses to the bright and dark sets for V4 (A), PITC (B), and AITC (C) neurons. Solid bars represent neurons classified as sharply color-selective and open bars represent neurons classified as broadly color-selective (see Materials and Methods). Triangles indicate the medians of the distributions. D–F, Shown are cumulative histograms of the correlation coefficients between responses to the bright and dark sets for all neurons (D), sharply color-selective neurons (E), and broadly color-selective neurons (F). Red, blue, and black lines represent V4, PITC, and AITC neurons, respectively. Triangles indicate the medians; **p < 0.001; *p < 0.01; ns, not significant (Mann–Whitney U test).

As was seen in the example neurons, some neurons had sharp color selectivity whereas others had only broad color tuning. This prompted us to examine whether the effect of luminance contrast differed between the more sharply tuned neurons and the more broadly tuned ones. To test this possibility, the same analysis was conducted separately for neurons classified as sharply color-selective and broadly color-selective based on the sparseness index (≥0.3, sharply color-selective neuron; <0.3, broadly color-selective neuron). In the AITC and PITC, the median correlation coefficients for the sharply color-selective neurons (0.83 in AITC, 0.61 in PITC; Fig. 5B,C, solid bar) were significantly larger than those for the broadly color-selective neurons (0.45 in AITC, 0.20 in PITC; Fig. 5B,C, open bar; Mann–Whitney U test, p < 0.01). This means that the luminance contrast had a greater effect on color selectivity in broadly color-selective neurons. Similarly, in V4 the median correlation coefficient was larger for sharply color-selective neurons (0.51; Fig. 5A, solid bar) than for broadly color-selective neurons (0.29; Fig. 5A, open bar), but the difference did not reach statistical significance (p > 0.1).

The distribution of correlation coefficients for sharply color-selective and broadly color-selective neurons was also separately compared across the three areas (Fig. 5E,F). For sharply color-selective neurons, the cumulative histogram for the AITC was shifted rightward as compared with V4 and the PITC, and there was a significant difference between the AITC and both the PITC and V4 (p < 0.01), but the difference was not significant between the PITC and V4 (p > 0.1; Fig. 5E; Mann–Whitney U test, Bonferroni corrected). For broadly color-selective neurons, although the cumulative histograms tended to be shifted increasingly rightward from the PITC to V4 to the AITC, the correlation coefficients did not significantly differ across the three areas (Fig. 5F; p > 0.05). These results indicate that the effect of luminance contrast on color selectivity was smaller in the AITC than the PITC or V4, and that this tendency was more obvious for sharply color-selective than broadly color-selective neurons.

There were significant differences in the baseline firing rate between V4 and other two areas (median: 1.6 spikes/s in V4, 5.2 spikes/s in PITC and 4.5 spikes/s in AITC, p < 0.01, Mann–Whitney U test, Bonferroni corrected). However, the results of Figure 5 did not change depending on whether the baseline firing rate was subtracted or not, indicating that the difference in the background firing rate did not affect the differences in correlation coefficients across areas.

Effect of luminance contrast on the response properties of neurons

As seen in the example neurons in Figures 2–4, the response magnitude and the strength (color selectivity index) and sharpness (color sparseness index) of the color selectivity of individual neurons may be affected by the luminance contrast. Therefore, to examine in more detail how these basic response properties were affected by the luminance contrast of stimuli, we compared the maximum response, mean response, color selectivity index, and sparseness index of each neuron in each area between the bright and dark sets. As a total population, there was no significant difference between the maximum response elicited by the bright and dark sets for all three areas (two-sample Wilcoxon signed-rank test, p > 0.1). There was also no significant difference in the mean responses between the bright and dark sets in the separate PITC and AITC (two-sample Wilcoxon signed-rank test, p > 0.1). On the other hand, in V4 we found that the mean responses elicited by the bright set were significantly larger (median = 14.2 spikes/s) than those elicited by the dark set (median = 9.3 spikes/s; two-sample Wilcoxon signed-rank test, p < 0.05).

To determine the color selectivity and sparseness indexes, comparisons between the bright and dark sets were only made for neurons that showed color selectivity for both the bright and dark sets (46 neurons in V4, 44 in the PITC, 68 in the AITC). As a population, there were no significant differences in either the selectivity index or sparseness index between the bright and dark sets in any of the three areas (two-sample Wilcoxon signed-rank test, p > 0.1). These results indicate that as populations, the basic response properties of color-selective neurons in each area were little affected by the luminance contrast of the visual stimuli.

Effect of luminance contrast on the representation of color

So far, the effect of luminance contrast was examined on the color selectivity of individual neurons. Another important issue is whether the effect of luminance contrast depends on color. This problem can be studied by comparing the population neural responses to a color stimulus brighter than the background with the responses to a stimulus of the same chromaticity but which is darker than the background. Therefore, for all 15 colors in the color set, we calculated Pearson's correlation coefficients between the population responses in each brain area to a color stimulus in the bright set (e.g., color 5) and the responses to the corresponding color stimulus in the dark set. We found that the correlation coefficient for the population responses varied depending on the color and the area recorded. Figure 6 shows the relationships between the population responses to stimuli in the bright set and those to stimuli in the dark set for four example colors (Fig. 6A, color 2: green-yellow with high saturation; B, color 5: red with high saturation; C, color 10: cyan; D, color 16: achromatic color). For colors 2 and 5, correlations of the population responses were moderate to high in all three brain areas: the correlation coefficients for color 2 were 0.53 in V4, 0.78 in the PITC, and 0.83 in the AITC, whereas those for color 5 were 0.60 in V4, 0.74 in the PITC and 0.86 in the AITC. For color 10, the correlation coefficients were significantly greater than zero (p < 0.001) in the PITC (r = 0.69) and AITC (r = 0.86), but not in V4 (r = 0.16, p > 0.1). For color 16, which is achromatic, correlation coefficients for the population responses of AITC neurons (r = 0.59) and V4 neurons (r = 0.26) were significant (AITC, p < 0.001; V4, p < 0.05), but those for the responses of PITC neurons (r = 0.05) were not significant (p > 0.1).

Figure 6.

Relationships between the population responses to a color stimulus in the bright set and the same color stimulus in the dark set for four example colors. The rows show the results in V4 (top), PITC (middle), and AITC (bottom). A, Scatter plots show the relationships between the population responses of color-selective neurons in each area to color 2 in the bright set (horizontal axis) and in the dark set (vertical axis). Each dot corresponds to one neuron. Diagonal lines connect the points where the responses to the stimuli in the bright and dark sets were identical. The correlation coefficient between the population responses is shown above each plot. B–D, Relationships between the population responses to color 5 (B), color 10 (C), and color 16 (D). The formats are the same as in A.

Figure 7 shows the correlation coefficients between the population responses elicited by a bright stimulus and those elicited by a dark stimulus for each of the 15 colors in the stimulus set in each area. The correlation coefficient for each color is indicated by the height of each bar in the top row, and the diameter of the circles plotted at the position corresponding to the chromaticity coordinates of each color in the bottom row (bubble plot). In V4, significant positive correlations between the population responses to the bright and dark stimuli were observed for 11 colors (colors 1–5, 7–9, 12, 14, 16), whereas four colors (colors 6, 10, 11, 13) did not show a significant correlation (p > 0.1; Fig. 7A). These four colors formed a contiguous region in the chromaticity diagram and, as can be seen in the bubble plot, the correlation coefficients tended to be lower in the cyan and blue regions, and gradually increased with colors such as magenta, red and orange.

Figure 7.

Correlation coefficients between the population responses to each bright stimulus and the corresponding dark stimulus for the 15 colors in the stimulus set. A–C, Top row, The height of each bar shows the correlation coefficient between the population responses to each of the 15 colors in the bright set and those to the same color in the dark set. The color ID is shown below each bar. Solid and open bars indicate significant and nonsignificant correlations, respectively. In the bottom row, the correlation coefficients are indicated by the diameters of the circles plotted at the positions corresponding to the chromaticity coordinates of each color (bubble plot). For scale, circles with diameters corresponding to r = 1.0 are shown as insets. Contour lines of decreasing thickness indicate where the correlation coefficients are 0.75, 0.50, and 0.25, respectively. The numbers of neurons included in the sample are shown at the bottom.

In the PITC, as in V4, the correlation coefficients between the population responses to bright and dark stimuli varied considerably across colors (Fig. 7B), but the pattern of variation markedly differed from V4. In the PITC, the correlation coefficients were particularly low for the neutral color (color 16) and for purple with very low saturation (color 11). Consequently, contour lines that reflect the tendency of the correlation coefficient across the chromaticity diagram formed concentric circles around the neutral color. In contrast to V4 and the PITC, in the AITC the correlation between the population responses to bright and dark stimuli was high for all colors (Fig. 7C, top), and there was little bias in the magnitude of the correlation coefficient across the entire range of the chromaticity diagram examined (Fig. 7C, bottom). Results of the analyses were largely consistent across individual monkeys. These results indicate that luminance contrast clearly affects the representation of color in the population activities of color-selective neurons in V4 and PITC, but has little effect on the representation of color in the AITC.

MDS analysis

The analysis so far indicated that neurons in V4 and the PITC carry information about both color and luminance contrast. If so, how do the population activities of color-selective neurons in these areas represent the color and luminance contrast information? One way to address this question is to determine which pairs of stimuli were well differentiated by the population neural responses and which pairs were not. This can be evaluated by computing the correlation coefficients between the population neural responses to one stimulus and those to another stimulus. As shown in Figures 6 and 7, for example, the correlation coefficient between the population responses of PITC neurons to color 16 in the bright set and those to color 16 in the dark color set was very low. This indicates that bright color 16 and dark color 16 are located separately in the space that represents the similarity of population responses of PITC neurons. By contrast, the correlation coefficient between the population neural responses to bright color 16 and those to bright color 7 is pretty high (0.88), indicating that bright colors 16 and 7 are located in close proximity within that space. We can regard 1 − r (correlation coefficient) as the neural distance between two stimuli and indicative of the dissimilarity of the pattern of population neural responses to two stimuli.

To visualize the dissimilarities of the population neural responses for all pairs of stimuli, correlation coefficients for the population neural responses in each area for all possible pairs of the 30 stimuli (15 colors in the bright and dark sets) were computed. A dissimilarity matrix was then constructed in which the neural distances for all pairs of stimuli were arranged as a matrix, and nonmetric MDS analysis was then applied. Thereafter, we used the results of the MDS analysis to visualize the dissimilarities of the population neural responses across the 30 colors in terms of distances on a two-dimensional plane (Fig. 8A–C). In these figures, population neural responses to color stimuli were projected onto two-dimensional planes such that the neural distances were preserved as accurately as possible in the arrangement of the 30 colored symbols. Circles and diamonds represent the colors in the bright and dark sets, respectively, and lines connect colors with the same chromaticity coordinates in the two sets. The number beside each circle corresponds to the color ID in the stimulus set (Fig. 1B), and the symbols are painted with the colors of the stimuli for visualization purposes. We also visualized the results of the MDS analysis in three-dimensional space, and the conclusion was basically the same. We chose two-dimensional plots here because the representation of color and luminance contrast is more clearly visible and the stress was reasonably small.

Figure 8.

Neural representation of bright and dark colors in the activities of color-selective neurons. Shown are two-dimensional plots of the results of MDS analyses conducted for neurons recorded in V4 (A, D, G), the PITC (B, E, H), and the AITC (C, F, I). A, Results of an MDS analysis based on a neural distance matrix computed from the raw (not normalized) responses of 71 V4 neurons to 30 color stimuli (15 bright and 15 dark stimuli) were projected onto a two-dimensional plane such that the neural distances (1 − r) were preserved as accurately as possible. Circles and diamonds represent the colors in the bright and dark sets, respectively. Lines connect colors with the same chromaticity coordinates in the two sets. The scree plot at the right shows the relationship between the number of dimensions and the stress in the MDS analysis. B, C, Results of MDS analyses based on the raw responses of populations of neurons in the PITC (B) and the AITC (C), respectively. Conventions are the same as in A. D–F, Results of MDS analyses based on the normalized responses of populations of neurons in V4 (D), the PITC (E), and the AITC (F) to 30 (15 bright and 15 dark) color stimuli. Conventions are the same as in A. G–I, Results of MDS analyses based on the normalized responses of populations of neurons in V4 (G), the PITC (H), and the AITC (I) to 15 chromaticity coordinates while disregarding the luminance contrast of stimuli. Colors with high saturation and those with low saturation are separately connected by lines.

In V4 (Fig. 8A), bright colors were clearly separated from the dark colors, even when two colors had the same chromaticity coordinates, and clustering of color stimuli with the same luminance was clearly observed. This indicates that information about the luminance contrast in the color stimuli was clearly represented in the population activity of color-selective neurons in V4. In addition, the spatial arrangement of different colors with the same luminance contrast was in accordance with the order in the hue circle, but the neural distances did not accurately reflect the structure of the hue circle and shrank in the direction connecting the green-yellow and purple colors. In other words, the distance between green-yellow (color 2) and purple (color 14) in Figure 10A was very small, while the distance between red (color 9) and cyan (color 10), which were located along the direction approximately perpendicular to the former pair of colors on the chromaticity diagram, was large. It is also noteworthy that neural distances between pairs of colors with the same luminance contrast tended to be larger for colors in the bright set than colors in the dark set.

Figure 10.

Simulated pattern of the differences in cone-related signals between bright and dark stimuli. A, Differences in pooled cone contrast between bright and dark stimuli across 15 chromaticities (see Discussion). B, Simulated pattern of the differences in the responses elicited by the bright and dark sets derived using the formula 1 − (difference in the pooled cone contrast) and shown as a bubble plot. C, Differences in examples of the cone-difference signals (2M − L) between bright and dark stimuli. D, Simulated pattern of the differences between the responses elicited by bright and dark sets derived using the formula 1 − [difference in the cone-difference signal (2M − L)].

Likewise, the population neural responses in the PITC were arranged in the order of hue for colors in both the bright and dark sets (Fig. 8B). As in V4, bright and dark colors were clearly separated, even when two colors had the same chromaticity coordinates, and a clear boundary existed between the bright and dark colors. Moreover, neural distance between bright color 16 and dark color 16 (neutral color: white or black), and that between bright color 11 and dark color 11 (purple with low saturation) were substantially greater than the neural distances between colors with high saturation.

The arrangement of color dissimilarities in the AITC neural population was dramatically different from those in the PITC and V4. In the population responses of AITC neurons, the effect of luminance contrast was smaller than in V4 or the PITC, and neural distances between two colors with the same chromaticity coordinates but different luminance contrasts were very small in the AITC (Fig. 8C). As a result, each pair of colors with the same chromaticity coordinates is located side-by-side. As in the PITC and V4, in the AITC colors with similar hues are located close to one another. Furthermore, neural distances between colors on this diagram faithfully reflect the distance on the chromaticity diagram, and the population responses to chromatic colors were arranged roughly in a circular array that resembled the hue circle. This indicates that population activities of color selective AITC neurons faithfully represent the order of hue in color space regardless of the luminance contrast.

In the MDS analysis described above, the neural distance matrix was based on the correlation coefficient computed using the raw (not normalized) firing rate. Because Pearson's correlation coefficient tends to be affected by samples with extreme values, it is possible that the results of the MDS analysis could be biased by the neurons showing very strong responses. To examine the dissimilarities of population neural responses across the 30 colors without the influence of differences in response amplitude across neurons, the response of each neuron was normalized to the maximum response among the 30 stimuli, and correlation coefficients were calculated using the resultant normalized responses. An MDS analysis was then conducted using the correlation coefficients obtained with the normalized responses (Fig. 8D–F).

Overall, the results of the MDS analysis using normalized responses plotted on a two-dimensional plane were very similar to those obtained using raw neural responses, but there were some interesting differences. One difference was that, in V4, dark colors were more clearly clustered in Figure 8A than D. This likely reflects the difference in the mean response magnitudes between the bright and dark colors in V4 as described above. That is, the mean response magnitudes were significantly smaller for dark colors than bright colors, which may have caused closer clustering of dark colors in the two-dimensional plot of the MDS analysis when raw responses were used. It was also observed that when normalized responses were used for MDS analysis in the AITC, neutral colors came to be plotted at about the center of the hue circle, and the color representation became more similar to the arrangement of colors in the chromaticity diagram (Fig. 8F).

The results of the MDS analyses shown in Figure 8A–F indicate that luminance contrast has a large effect on the population responses in V4 and the PITC but, at the same time, the systematic arrangement of hue suggests that hue representation also resides in the population responses of the color-selective neurons in V4 and the PITC. To examine this possibility more explicitly, we constructed a 15 × 15 distance matrix based on correlation coefficients computed using normalized population responses at the 15 chromaticity coordinates, after which MDS analysis was conducted using the resultant distance matrix. In other words, only the chromaticity of the stimuli was considered in this analysis; the luminance contrast of the stimulus was completely disregarded. To do this, the responses of a given neuron to a bright color and those to a dark color with the same chromaticity were separated and treated as different dataset. As a result, the number of dataset (e.g., n = 71 for V4) was doubled in this analysis (e.g., 2n = 142 for V4), whereas the number of colors was halved to 15, and MDS analysis was applied to the resultant 15 × 15 distance matrix. In other words, there were 2n dataset each of which recorded neural responses using only one of bright or dark set, and MDS analysis was conducted by computing distances between the responses to each pair of different chromaticities. The results of this MDS analysis show how the neural responses in each area represent the 15 chromaticities (Fig. 8G–I). In all three areas, the different chromaticities are arranged circularly in the order of hue in the two-dimension plane. In addition, colors with low saturation are located inside the hue circle. As a result, the observed color representation was analogous to the arrangement of colors with different hues and saturations on the chromaticity diagram.

The results summarized above indicate that information about both hue and color saturation resides in a systematic way in the population neural activities in all three areas. Nonetheless, differences across areas were observed. In V4 (Fig. 8G) and the PITC (Fig. 8H), the circular arrangement was compressed in the direction connecting green-yellow (color 2) and purple (color 14) so that the neural distance between those colors was shorter than the neural distance between red (color 9) and cyan (color 10). This confirms that the anisotropy seen in Figure 8A,B,D,E is not an artifact caused by plotting the results of MDS analyses in two dimensions. In contrast to V4 and the PITC, no clear anisotropy was observed in the arrangement of colors in the AITC, so hue was represented by a clear circle.

The results of the MDS analysis indicate that the population neural responses of color-selective neurons in V4 and the PITC carry information about the luminance contrast of the color stimuli, but at the same time, population responses of neurons in these areas and in the AITC also carry information about the hue and saturation of the color stimuli.

Effect of stimulus position

The results described above indicated that the effects of luminance contrast differed among V4, PITC and AITC. However, the RF positions of the neurons recorded in each area were not the same. When we recorded from the AITC, the stimulus was always presented at the foveal center, whereas the RF center of V4 neurons was situated at eccentricities of 2–12°, and the RF center of PITC neurons was situated at eccentricities of 0–13°. Therefore, we had to consider the possibility that the difference in the position of the stimulus in the visual field affected the results.

To determine whether the difference in the RF center where the stimuli were presented affected the differences in the effects of luminance contrast, the relationship between the eccentricity of the RF center and the magnitude of the effect of luminance contrast was evaluated in terms of the correlation coefficient between the responses to the stimuli in the bright and dark sets, in V4 and the PITC (Fig. 9A,B). We found that there was no significant correlation between the eccentricity of the RF center and the effect of luminance contrast in either V4 (r = 0.19, p = 0.123) or the PITC (r = −0.11, p = 0.357), indicating that there was no systematic relationship between the stimulus position in the visual field and the magnitude of the effect of luminance contrast.

Figure 9.

Relationship between stimulus position and the magnitude of the effect of luminance contrast. A, B, Relationships between the eccentricity of the RF center (horizontal axis) and the magnitude of the effect of luminance contrast (vertical axis) quantified in terms of the correlation coefficient between responses to the bright and dark sets in neurons recorded from V4 (A) and the PITC (B). C, D, Correlation coefficients between the population responses to a bright stimulus and a dark stimulus across colors for subpopulations of V4 (C) and PITC (D) neurons that had RFs with centers ranging between 5 and 10° in eccentricity. E, F, Correlation coefficients between the population responses to a bright stimulus and a dark stimulus across colors for a subpopulation of PITC neurons that had RFs whose centers ranged between 0 and 5° in eccentricity (E) and AITC neurons (F). The format for C–F is the same as Figure 7, bottom.

We next examined whether the effect of luminance contrast on the population responses of neurons to each color depended on the stimulus position in the visual field (Fig. 9C–F). First, comparisons were made between the effect of luminance contrast on the population responses of V4 (n = 51) and PITC (n = 22) neurons that were matched with respect to the range of eccentricity of RF center (5–10°; Fig. 9C,D). We found that there was a considerable difference between V4 and the PITC in the effect of luminance contrast on the population responses to colors, even when the eccentricity range of the RFs was matched. The correlation coefficient between the population responses to bright and dark stimuli tended to be low for colors from cyan to blue in V4 (Fig. 9C) and also tended to be low for the range of colors including neutral colors and magenta in the PITC (Fig. 9D). These patterns are similar to those obtained for the entire population of neurons in V4 and the PITC (Fig. 7A,B, bottom), and indicate that the difference in the response patterns between V4 and the PITC cannot be explained by the difference in the stimulus positions.

The effects of luminance contrast on the population responses to colors were also compared between the PITC and AITC using only PITC neurons having RFs with eccentricities of 0–5°, which is close to the stimulus position for AITC neurons (0° eccentricity; Fig. 9E,F). Again, there was a considerable difference in the effect of luminance contrast on the population responses to colors in the PITC and AITC, even though the range of RF eccentricities was similar. In the PITC, luminance contrast had a large effect on colors with low saturation (Fig. 9E), as was seen for the entire population of PITC neurons. These results indicate that differences in the stimulus positions cannot fully account for the difference in the effect of luminance contrast among V4, the PITC, and the AITC.

Discussion

In the present study, the effects of the luminance contrast on the color selectivity of individual neurons and population neural responses were systematically examined in V4, the PITC and the AITC. The magnitude of the effect on individual neurons tended to be smaller in the AITC than in V4 or the PITC. For population responses, the effect in V4 depended on color and was largest for blue and cyan, whereas in the PITC, the effect gradually increased with reduction in color saturation, and was especially large for neutral colors. By contrast, there was little effect of luminance contrast in the AITC, regardless of color.

Comparison with the previous studies

Conway et al. (2007) systematically studied the effect of luminance contrast on the tuning of color selective neurons in V4 and PIT, and found that there was little effect of luminance contrast on the “glob” neurons. They also noted some effects in “inter-glob” neurons. Likewise, other studies reported that color-selective V4 neurons often exhibited large effects of luminance contrast (Yoshioka et al., 1996; Bushnell et al., 2011), though the effect was not quantitatively analyzed. Our study is a significant extension of these previous studies by quantitatively analyzing the effect of luminance contrast using stimuli that spanned the entire range of gamut and comparing the responses in three areas including AITC that was not tested in previous studies. We did not examine responses to color stimuli equiluminant to the background because one of our interests was to examine the responses to stimuli with low saturation especially neutral colors. If stimuli equiluminant to the background were used, contrast to the background should become too low for such stimuli, and any reliable response will not be recorded.

In the present study, we found large effects of luminance contrast for colors with low saturation in PITC which was not reported in Conway et al. (2007). There are several methodological differences between their study and the present study that may have caused the differences. First, whereas we used stimuli ranging across the entire gamut of the display, they used only colors with high saturation. When we did the MDS analysis using only the colors at the perimeter of the gamut, two-dimensional embedding of the result in PITC exhibited clear tendency that colors were circularly arranged in the order of hue, and bright and dark colors were not clearly segregated. This indicates that the effect of luminance contrast observed in PITC in the present study is at least in part due to the use of stimuli with low saturation. There are also differences in the way neurons were sampled. In the present study, we did not preselect recording sites using fMRI as was done by Conway et al. (2007). Furthermore, the recording sites of Conway et al. (2007) within the glob in PIT seem to be concentrated in the region posterior and dorsal to STS (their Figs. 1–3, 8), whereas our recording sites in PIT were in the region ventral to STS around PMTS. These differences in the neural sampling may also have contributed to the different results in PIT in two studies.

Effects of background on perceived color and brightness

Luminance and color of the background affect the perceived brightness and color of a stimulus in various ways. Neuronal activities associated with brightness induction have been observed in the visual cortex (Rossi et al., 1996; Rossi and Paradiso, 1999; Kinoshita and Komatsu, 2001). These neuronal activities appear to discriminate between black and white, but the color selectivity of neurons was not analyzed in these studies. Responses of color-selective neurons in V1 (Wachtler et al., 2003) and V4 (Schein and Desimone, 1990; Kusunoki et al., 2006) have been shown to be affected by a change in background color. In those earlier studies, a change in background color modified the relative balance across the activities of different types of cones. In contrast, we used a gray background, which will not affect the relative balance across cone signals. Thus, the mechanism for the change in color selectivity observed in the present study must be quite different from that observed in earlier studies.

In a color naming experiment, a change in luminance contrast modified some of the chromatic color terms (e.g., brown and orange) as well as the achromatic colors (black, gray, white; Boynton and Olson, 1987). However, studies examining the basic color terms used by subjects with various cultural backgrounds showed that black and white constitute the most basic distinction followed by the chromatic color terms (Berlin and Kay, 1969). Thus, we can expect that the effect of luminance contrast is the largest for the neutral color and it gradually decays with increasing saturation of color. A concentric pattern of the effect of luminance contrast around the neutral color in the PITC suggests that this region is closely related to the perceived appearance of color.

Effects of cone-related signals

The effects of luminance contrast in the PITC suggest that effect depends on the color saturation, which is one of the perceptual color attributes. However, what is the factor determining the pattern observed in V4? Because neurons at early stages in the visual pathway would be more directly affected by changes in cone signals (Derrington et al., 1984; Lennie et al., 1990; Reid and Shapley, 1992; Johnson et al., 2004; Conway and Livingstone, 2006), one plausible factor is the difference in cone-related signals elicited by the bright and dark sets. Even when the chromaticities of the bright and dark stimuli are the same, cone-related signals can be different, and this may explain the pattern of the effect of luminance contrast observed in V4. We considered two types of physiologically important signals. The first was the cone contrast signal, defined as the ratio between the increment/decrement in cone activity for the stimulus relative to the background. The second is the cone-difference signal, which reflects the difference between the L cone and M cone signals and represents the color signal in early visual stages (for review, see Gegenfurtner, 2003; Solomon and Lennie, 2007; Conway et al., 2010). We did not consider the S cone signal because it does not contribute to the luminance (Cavanagh et al., 1987; Lee and Stromeyer, 1989).

Figure 10A shows the difference in pooled cone contrast defined as the square-root of the cone contrast energy (Kaiser and Boynton, 1996) calculated based on the cone fundamental (Stockman and Sharpe, 2000) for each of the 15 stimuli in the bright and dark sets. The differences tended to be larger for blue colors (colors 13, 14) and became smaller for other colors, such as green, yellow, and red. However, the simulated pattern of the effect of the luminance contrast calculated as 1 − (the difference in pooled cone contrast for each stimulus) (Fig. 10B) clearly differs from the actual pattern observed in V4.

To determine the cone-difference signal, we simply calculated the weighted difference between L cone and M cone excitations and plotted (1 − weighted difference in cone excitations) on the chromaticity diagram. We found that this yields a pattern similar to that observed in V4, and an example of the pattern (for the case of 2M − L) is shown in Figure 10C,D. This similarity suggests that the pattern observed in V4 is affected by the cone-difference signal. Presumably, the effect of the cone-difference signal is stronger in earlier stages of the visual pathway, but is still apparent in V4. Interestingly, Conway et al. (2007) observed that neurons in inter-glob tended to respond selectively to blue/cyan for stimuli equiluminant to the background that are similar to the colors we observed larger effects of luminance contrast in V4. This suggests that a common mechanism may have caused the observed biases in these two studies.

Color representation independent of luminance contrast in the AIT

We found that AITC neurons represent hue and saturation independently of luminance contrast. Information about the hue and saturation of objects, isolated from luminance, could be important cues for recognition of particular objects in the natural environment, which is crowded with numerous objects. In such an environment, the interaction between the spatial distribution of illumination and the three-dimensional shapes of objects leads to complicated variations in shading and shadow on objects' surfaces. As a result, even when the surface of an object is uniform in color, luminance may vary across its surface. On the other hand, hue and saturation would be less influenced and would remain roughly unchanged. Therefore, the properties of AITC neurons representing hue and saturation independent of luminance contrast should be useful for selection of objects in a natural environment, as AITC neurons can signal invariant color information specific to each object. AIT plays vital roles in object recognition (Mishkin et al., 1983; Tanaka, 1996), and the color selectivity of AIT neurons is thought to be involved in detecting color cues useful for object identification. Our findings provide new insight into how color-selective AITC neurons contribute to object recognition.