Mixing of Chromatic and Luminance Retinal Signals in Primate Area V1

Abstract

Vision emerges from activation of chromatic and achromatic retinal channels whose interaction in visual cortex is still poorly understood. To investigate this interaction, we recorded neuronal activity from retinal ganglion cells and V1 cortical cells in macaques and measured their visual responses to grating stimuli that had either luminance contrast (luminance grating), chromatic contrast (chromatic grating), or a combination of the two (compound grating). As with parvocellular or koniocellular retinal ganglion cells, some V1 cells responded mostly to the chromatic contrast of the compound grating. As with magnocellular retinal ganglion cells, other V1 cells responded mostly to the luminance contrast and generated a frequency-doubled response to equiluminant chromatic gratings. Unlike magnocellular and parvocellular retinal ganglion cells, V1 cells formed a unimodal distribution for luminance/color preference with a 2- to 4-fold bias toward luminance. V1 cells associated with positive local field potentials in deep layers showed the strongest combined responses to color and luminance and, as a population, V1 cells encoded a diverse combination of luminance/color edges that matched edge distributions of natural scenes. Taken together, these results suggest that the primary visual cortex combines magnocellular and parvocellular retinal inputs to increase cortical receptive field diversity and to optimize visual processing of our natural environment.

Introduction

Visual information is processed in parallel by 3 groups of retinal ganglion cells (magnocellular, parvocellular, and koniocellular) that differ in their cone inputs and preference for color stimuli. Magnocellular retinal ganglion cells, the parasol cells, receive summed input from long- (L) and middle- (M) wavelength-sensitive cones (Sun et al. 2006; Field et al. 2010) and appear to underlie an achromatic, luminance channel. Parvocellular retinal ganglion cells, the midget cells, receive opposed L and M cone inputs and transmit red–green chromatic information to the rest of the brain (Reid and Shapley 1992, 2002; Martin et al. 2001; Buzas et al. 2006; Field et al. 2010; Lee et al. 2011). Finally, most koniocellular retinal ganglion cells receive input from short-wavelength-sensitive (S) cones opposed to some combination of L and M cone inputs, which allow them to transmit blue–yellow chromatic information (Gouras 1968; De Monasterio and Gouras 1975; Lee et al. 1989b; Dacey and Lee 1994; Chichilnisky and Baylor 1999; Tailby et al. 2008; Cheong et al. 2011).

The 3 types of retinal ganglion cells project to separate layers and cells in the lateral geniculate nucleus (LGN; De Valois 1960, 1965; Wiesel and Hubel 1966; Hendry and Reid 2000; Chatterjee and Callaway 2003; Tailby et al. 2008; Roy et al. 2009) and, in turn, the 3 types of LGN neurons project to different layers in the primary visual cortex (Hubel and Wiesel 1972; Blasdel and Lund 1983; Hendry and Yoshioka 1994; Chatterjee and Callaway 2003). While there is general agreement that the 3 types of thalamic afferents segregate in the primary visual cortex, the extent to which cortical cells combine the thalamic inputs is still unclear (Hubel and Wiesel 1968; Dow 1974; Livingstone and Hubel 1984; Chatterjee and Callaway 2003; Horwitz et al. 2007; Economides et al. 2011; Shapley and Hawken 2011; Horwitz and Hass 2012). Here, we addressed this question by using a compound grating that has both luminance and chromatic components but with different spatial frequencies (Lee et al. 2011). Because luminance and color are intermingled in natural scenes, it is important to design stimuli to measure luminance–color interactions (Lee et al. 2011; Horwitz and Hass 2012), for example compound gratings. Here, we show that the responses of some V1 cells are dominated by the chromatic contrast of compound gratings, as is also the case for parvocellular retinal ganglion cells. Also, as in magnocellular retinal ganglion cells (Lee et al. 1988; Lee and Sun 2009), the responses of other V1 cells are dominated by the luminance contrast of compound gratings and show weak frequency-doubled responses to chromatic gratings. Unlike magnocellular and parvocellular retinal ganglion cells, the luminance/color preference of V1 neurons was distributed in a continuum that could be fit with a single Gaussian function, with V1 cells in deep cortical layers showing the strongest combined responses to color and luminance. We propose that the mixing of color/luminance signals in area V1 is optimized to process the diversity of edges found in natural scenes.

Materials and Methods

Surgery and Preparation for Retinal Recordings

Macaques (4 Macaca fascicularis, male 2.2–3.8 kg) were initially sedated with an intramuscular (IM) injection of ketamine (10 mg/kg). Anesthesia was induced with sodium thiopental (10 mg/kg) and maintained with inhaled isoflurane (0.2–2%) in a 70 : 30 N₂O–O₂ mixture. Local anesthetic was applied to points of surgical intervention. Electroencephalogram (EEG) and electrocardiogram (ECG) were monitored continuously to ensure animal health and adequate depth of anesthesia. Muscle relaxation was maintained by an infusion of gallamine triethiodide (5 mg/kg/h, IV) with accompanying dextrose Ringer solution (5 mL/kg/h). Careful monitoring of the EEG and ECG was carried out during recording and any increase in the heart rate or EEG frequency was controlled by an increase in the isoflurane level. Body temperature was kept close to 37.5 °C. End-tidal CO₂ was kept close to 4% by adjusting the rate and depth of respiration. At the termination of recording, the animals were euthanized with an overdose of sodium pentobarbital (120 mg/kg).

Neuronal activity was recorded directly from retinal ganglion cells by an electrode inserted through a cannula entering the eye behind the limbus. The details of the preparation can be found elsewhere (Crook et al. 1988). A gas-permeable contact lens of the appropriate power was used to bring stimuli into focus on the retina. All procedures strictly conformed to the National Institutes of Health Guide for the Care and Use of Laboratory Animals and were approved by the SUNY State College of Optometry Animal Care and Use Committee.

Visual Stimuli for Retinal Recordings

Visual stimuli were generated via a VSG series 2/3 graphic controller (Cambridge Research Systems, Rochester, UK) and presented on a cathode ray tube (CRT) monitor (SONY Trinitron GDM-F500, 150 Hz frame rate) 2.28 m away from the monkey. Stimuli were horizontal gratings of the following types presented in a 5° × 5° window: (1) A compound red–green grating (see below for a description of stimulus structure), (2) an equiluminant red–-green grating, and (3) a luminance grating. The 3 types of gratings were drifted downwards, with the luminance grating drifted at twice the temporal frequency of the others (so that frequency was matched to the luminance component of the compound grating). For each cell, a range of spatial and temporal frequencies and contrasts were investigated. At least in retinal ganglion cells, the ratio of responses to luminance and chromatic components of the compound grating was relatively independent of temporal and spatial frequency. To generate the red–green compound drifting–grating on a CRT monitor, we applied the following waveforms, W_R (eq. 1a) and W_G (eq. 1b), to the red and green guns respectively.

$$\begin{array}{ll}{W}_{\mathrm{R}}(\theta ,t)& =(1-C)R_{\max }/4+[C{R}_{\max }(\cos 2\pi (2f\theta +gt)+1)/2]Z_{\mathrm{R}}(\theta ,t)\\ Z_{\mathrm{R}}(\theta ,t)& =\left\{\begin{array}{cc}1& \cos \pi (2f\theta +gt)>0\\ 0& \cos \pi (2f\theta +gt)\le 0\end{array}\right.\end{array}$$ (1a)

$$\begin{array}{ll}{W}_{\mathrm{G}}(\theta ,t)& =(1-C)G_{\max }/4+[C{G}_{\max }(\cos 2\pi (2f\theta +gt)+1)/2]Z_{\mathrm{G}}(\theta ,t)\\ Z_{\mathrm{G}}(\theta ,t)& =\left\{\begin{array}{cc}0& \cos \pi (2f\theta +gt)>0\\ 1& \cos \pi (2f\theta +gt)\le 0,\end{array}\right.\end{array}$$ (1b)

where C is modulation contrast, R_max and G_max are peak red and green gun luminances, which are set to be equal, f is spatial frequency, g is temporal frequency, θ is phase, and t is time. Z_R and Z_G are 2 envelope functions whose frequency is half the frequency of W_R and W_G and whose value at each point in time is either 0 or 1. Z_R is 1 during the first W_R cycle, 0 during the second W_R cycle, and so on. Conversely, Z_G is 0 during the first W_G cycle, 1 during the second W_G cycle, and so on. Therefore, the first cycle of a compound grating is made of a red cycle (W_R cycle, Z_R = 1, Z_G = 0) followed by a green cycle (W_G cycle, Z_R = 0, Z_G = 1) with W_R and W_G having twice the frequency of the compound grating [e.g., cos 2π (2fθ + gt) for W_R versus cos π (2fθ + gt) for Z_R]. Note that the compound gratings are not a linear sum of luminance and chromatic gratings; the equations described above allowed exploring a larger contrast range than if luminance and chromatic waveforms were simply added together.

Luminance and chromatic gratings are generated in the standard manner, with a sinusoidal modulation of the red (W_R) and green (W_G) guns either in-phase (luminance grating) or out-of-phase of one another (chromatic grating), being W_R and W_G as described in equations 2a and 2b.

$$W_{\mathrm{R}}=R_{\max }(1+C\text{cos(}\theta \text{)})/2$$ (2a)

$$W_{\mathrm{G}}=G_{\max }(1+C\text{cos}(\theta +\pi ))/2,$$ (2b)

where θ represents spatial phase, such that θ = 2πfx, where f is spatial frequency and x spatial location.

Michelson luminance contrast was calculated as (L_max − L_min)/(L_max + L_min) for both the luminance grating and the luminance component of the compound grating. Michelson chromatic contrast was calculated as (G_max − G_min)/(G_max + G_min) for both the equiluminant grating and the chromatic component of the compound grating (note that G_max = R_max and G_min = R_min in both the equiluminant and the compound gratings). The root mean square (RMS) luminance contrasts were calculated as in equation 3.

$$C_{\mathrm{lum}}=\sqrt{\frac{1}{N}\sum _{i=0}^{N-1}\frac{{(L_i-\overline{L})}^2}{{\overline{L}}^2},}$$ (3)

where L_i is the luminance at each spatial position of the grating, $\overline{L}$ the mean luminance, and N the number of pixels.

The RMS chromatic contrasts were calculated as in equation 4a.

$$C_{\mathrm{chrom}}=\sqrt{\frac{1}{N}\sum _{i=0}^{N-1}\frac{{(R_i-G_i)}^2}{{(\overline{R}+\overline{G})}^2},}$$ (4a)

where N is the number of pixels, R_i and G_i are values of the red and green gun luminances, respectively, at each pixel after normalization so that R_max = G_max = 1, and $\overline{R}$ and $\overline{G}$ are the mean red and green luminances. The same values are obtained if RMS cone contrasts are calculated as in equation 4b.

$$C_{\mathrm{cone}}=\sqrt{\frac{1}{N}\sum _{i=0}^{N-1}\frac{{(L_i-M_i)}^2}{{(\overline{L}+\overline{M})}^2},}$$ (4b)

where L_i and M_i are cone excitations at the pixel i, each normalized to the mean cone excitations at the mean chromaticity, $\overline{L}$ and $\overline{M}$. A more detailed description of stimulus generation and definition of chromatic contrast is given in Lee et al. (2011). For measurements of parvocellular and magnocellular responses, the mean luminances of the red and green phosphors were set equal to give a mean luminance of 31.34 cd/m², with a chromaticity of (0.436, 0.476) in CIE x-, and y-coordinates.

Data Collection and Analysis for Retinal Recordings

We recorded responses of cells between 4° and 12° eccentricity. Cell identification was achieved through standard tests (Lee et al. 1989b). These included achromatic contrast sensitivity and responses to lights of different chromaticity. Additional tests (e.g., measuring responses to heterochromatically modulated lights; Smith et al. 1992) were employed in cases when identification was difficult. Parvocellular cells can generally be identified by their tonic responses and spectral opponency, and magnocellular cells by their phasic responses and lack of spectral opponency. For each cell, the locus of the receptive field center was determined, and the stimulus was centered on this point. Times of spike occurrence were recorded to an accuracy of 0.1 ms, and averaged histograms of spike trains were simultaneously accumulated with 64 bins per cycle of modulation. Responses were derived from the histograms by Fourier analysis to give the different Fourier components.

Surgery and Preparation for Cortical Recordings

We recorded V1 neurons from 2 awake male monkeys (Macaca mulatta) with a chronic array of 3–7 ultrathin electrodes, which were independently moved with chronically implanted microdrives (Swadlow et al. 2005). The electrodes were made of platinum-alloy core (90% platinum and 10% tungsten) covered with quartz (40 µm in diameter). The electrodes were pulled to a taper and sharpened to a fine tip. The monkeys were trained to fixate a small cross of 0.1° during 2–3 s while grating stimuli were presented within the receptive field of each V1 neuron. Eye movements were recorded with a scleral eye coil and trials with eye deviations >0.5–1° from the point of fixation were discarded. V1 cells were recorded at eccentricities ranging from 3 to 25° (mean: 13°). Surgeries were performed under fully aseptic and sterile conditions. Monkeys were initially sedated with an IM injection of chlorpromazine (1 mg/kg, IM), ketamine (5–15 mg/kg, IM), and atropine (0.05 mg/kg, IM). Anesthesia was induced with intravenous thiopental (5–15 mg/kg, IV) and maintained with a mixture of isoflurane (0.5–2%) and oxygen (30%). Analgesics and antibiotics were administrated before (buprenorphine: 0.01–0.02 mg/kg; cefazolin: 25 mg/kg, IM) and after the surgery (fentanyl patch: 3–6 µg/kg; Baytril: 5 mg/kg, IM for 3 consecutive days). All procedures were performed in accordance with the guidelines of the US Department of Agriculture and approved by the Institutional Animal Care and Use Committee at the State University of New York, State College of Optometry.

Visual Stimuli for Cortical Recordings

Visual stimuli were generated with a computer running Visionworks (Vision Research Graphics, Inc.) and presented on a GDM-F520 monitor (SONY Electronics, Inc., USA; refresh rate: 160 Hz, mean luminance: 55 cd/m², resolution: 640 × 480 pixels). The spatiotemporal receptive fields were mapped with Hartley stimuli (Ringach et al. 1997) and sparse noise (Jones and Palmer 1987). These methods allowed us to accurately measure the eccentricity of the receptive fields and to optimize the grating position, orientation, spatial frequency, and size. The Hartley stimulus consisted of 576 luminance gratings with different orientations, phases, and spatial frequencies randomly presented at 80 Hz. The size of the gratings in Hartley stimuli varies from 2.8° × 2.8° to 22.5° × 22.5° (0.05°/texel to 0.47°/texel). The sparse noise consisted of bright and dark squares (0.2–0.6°/side) randomly presented at 16 × 16 different positions at 25 Hz.

We used 3 types of drifting sinusoidal gratings to characterize the visual responses of V1 cells: luminance gratings, chromatic gratings, and compound gratings. In the luminance gratings, we varied the luminance contrast of the grating so that each cycle had a sinusoidal modulation between dark and light. In the color gratings, we varied the chromatic contrast of the grating so that each cycle had a sinusoidal modulation between equiluminant red and green (or blue and yellow). In the compound gratings, we varied both the luminance and the chromatic contrast of the grating as for the ganglion cell experiments mentioned above (Lee et al. 2011). The mean luminances of the red and green bars in the compound gratings were 26 cd/m² with a CIE chromaticity of (x: 0.475, y: 0.463), and the mean luminances of blue and yellow bars were 15.3 cd/m² with a chromaticity of (x: 0.348, y: 0.247). The RMS contrast was 71% for the luminance grating and 71% for the equiluminant chromatic grating. The RMS contrast of the compound grating was 71% for luminance and 122% for color. Therefore, the ratio between the chromatic and luminance contrasts of the compound gratings was 1.72. In red–green stimuli, the average L–M cone contrast was 18.85% (L-cone contrast: −0.105; M-cone contrast: 0.272). In blue–yellow stimuli, the average S −(L + M) cone contrast was 25.5% (S-cone contrast: 0.937, M-cone contrast: 0.37, and L-cone contrast: 0.057). Grating stimuli were optimized for each neuron in size, spatial frequency, and orientation using the receptive field estimated with Hartley stimuli and the tuning functions measured with luminance drifting gratings. The mean luminance was 19 cd/m² for the compound grating, 26 cd/m² for the equiluminant grating, and 55 cd/m² for the luminance grating and the background. Note that we used the same red and green luminances in chromatic and compound gratings. Therefore, because the compound grating has an additional dark bar, the mean luminance was lower in compound than in chromatic gratings. The size of the stimulus was optimized for each cell and usually ranged between 1° and 2.5°. Gratings were drifted at temporal frequencies of 2–4 Hz for luminance and half that frequency for chromatic and compound gratings (Fig. 1).

Figure 1.

Visual stimuli. Retinal and V1 neurons were stimulated with 3 different types of drifting gratings: luminance (A), chromatic (B), and compound (C). Spatially, the luminance and chromatic gratings had 2 bars, dark–light and equiluminant red–green, respectively. The compound grating had 3 bars, red-dark–green, red, and green being equiluminant. A compound grating drifting at 2 Hz for luminance (2 dark bars per second) will drift at 1 Hz for color (1 red/green bar per second).

Data Collection and Analysis for Cortical Recordings

Spike waveforms from each neuron were sampled at 40 kHz, filtered between 250 Hz and 8 kHz, and collected with a Plexon system (Plexon, Dallas, TX, USA). Repeated presentations of the same stimuli were used to generate peristimulus time histograms (PSTHs) with bin widths of 10 ms. We performed a Fast Fourier Transform of the PSTHs from 500 ms following the stimulus onset (to remove response transients) to the end of the grating drift. When using luminance gratings as stimuli, we measured the F₁ component of the response (luminance modulation frequency) and the F₀ component (mean rate). Then, the F₁/F₀ ratio was used to classify V1 cells as linear (F₁/F₀ ≥ 1) and nonlinear (F₁/F₀ < 1). It should be noted that our terms “linear” and “nonlinear” refer only to the linearity of spatial summation measured with luminance drifting gratings. Linear cells with F₁/F₀ ≥ 1 showed other nonlinearities, some of which are reported in this paper. When using equiluminant chromatic gratings as stimuli, we measured the F₁ component of the response (chromatic modulation frequency), the F₂ component (frequency-doubled response), and the F₀ (mean rate). When using compound gratings, we measured the F₁ component of the response (chromatic modulation frequency, C), the F₂ component (luminance modulation frequency, L), and the F₀ (mean rate). Then, the F₂/F₁ ratio was used to estimate the luminance/color (L/C) modulation ratio and the contribution from parvocellular and magnocellular retinal inputs to the V1 cell response. This L/C ratio is difficult to interpret in V1 cells with nonlinear spatial summation (F₁/F₀ < 1) that generate equally strong F₁ and F₂ responses to the luminance component of the compound grating. In cells with linear spatial summation (F₁/F₀ ≥ 1), we presume that the L/C ratio provides a better estimate of the contribution from parvocellular and magnocellular retinal inputs since the F₁ magnocellular response to the chromatic contrast is much weaker than the F₂ magnocellular response to the luminance contrast. To measure the L/C ratios, we used the optimal spatial frequency for each cortical cell (range: 0.1–1.5 cpd, average: 0.3 cpd) and 1 cpd for retinal neurons. In the retina, a range of spatial frequencies between 0.1 and 6 cpd were tested, and there were no major changes in the L/C ratio. Therefore, the spatial frequencies used were not much higher in the cortex than in the retina, and the higher L/C ratios that we report in the cortex are unlikely due to a difference in the range of spatial frequencies used. As indicated above, the L/C ratio measured in retinal ganglion cells was relatively independent of spatial and temporal frequencies. Also, because V1 neurons respond to lower spatial frequencies when stimulated with chromatic than luminance gratings, it seems reasonable to use a lower spatial frequency for the chromatic than luminance component of the compound grating. That being said, a technical limitation of our study is that all L/C ratios were measured at the same contrast level, same spatial frequency, and same temporal frequency. Therefore, it is possible that the L/C distribution for V1 neurons would be different if chromatic and luminance contrasts were equated across multiple spatial and temporal frequencies for each cell recorded. In the future, long-term recordings from the same neuron will be needed to measure responses to all possible combinations of the relevant stimulus dimensions.

Circular variance (CV; Ringach et al. 2002) was defined as CV = 1 − |R|, being R defined as in equation 5:

$$R=\frac{{\mathrm{\Sigma }}_k{r}_k{\text{e}}^{i\cdot 2\cdot {\theta }_k}}{{\mathrm{\Sigma }}_k{r}_k},$$ (5)

where r_k is the visual response to a drifting grating with an angle θ_k expressed in radians. The visual response was measured as the mean firing rate (F₀) in V1 nonlinear cells and as the response amplitude at the frequency of the grating (F₁) in V1 linear cells. To measure the orientation half-width at half-height (HWHH), we fit the orientation tuning curve with a sum of 2 von Mises functions, as described by equation 6 (Swindale et al. 2003):

$$R_{\phi }=A_1\text{exp{ }{k}_1(\cos (\phi -{\phi }_1)-1)\text{} }+A_2\text{exp{ }{k}_2\text{(}\cos (\phi -{\phi }_2)-1\text{)} ,}$$ (6)

where $R_{\phi }$ is the response to each stimulus orientation (φ), A₁ and A₂ are the response amplitudes for the 2 opposite directions of movement (φ₁ and φ₂, which are required to be 180° apart), and k₁ and k₂ are the inverses of the tuning widths. HWHH was measured around the preferred orientation in tuning curves with a reasonable goodness of fit (r² ≥ 0.7, n = 114). Orientation selectivity (OS) was defined as the ratio (R_max − R_min)/(R_max + R_min), where R_max and R_min are the maximum and minimum responses obtained across all orientations. Direction selectivity (DS) was measured at the preferred orientation of the cell as DS = 1 − R_NPD/R_PD, where R_PD and R_NPD are the responses to the preferred and nonpreferred directions. OS and DS were measured directly from the raw data for all the cells studied (n = 138). The spatial frequency tuning was fitted with a Gaussian function, as described by equation 7.

$$R_x=A\cdot {\text{e}}^{-{\displaystyle \frac{{(x-\mu )}^2}{2{\sigma }^2}}}+B,$$ (7)

where x is the spatial frequency, A the amplitude, B the baseline, µ the mean, and σ the standard deviation of the function. Both the spatial frequency peak and spatial frequency bandwidth (HWHH) were measured in cells with well-fit spatial frequency tunings (r² ≥ 0.7, n = 111). Orientation/direction tuning curves were measured with gratings drifting at 16 different directions of movement. Spatial frequency tuning curves were measured with 8 different spatial frequencies that ranged either from 0.01 to 2 cpd or from 0.01 to 5 cpd.

To estimate the depth of the cortical recordings, we measured the response of local field potentials (LFPs) to grating stimuli that were optimized for size and spatial frequency. LFPs were recorded with the same electrode used to isolate spikes, sampled at 5 kHz, and low-pass filtered at 500 Hz with a fourth-order Butterworth filter. We made PSTHs of the LFP response to the onset of all grating stimuli used to measure orientation tuning. Then, we selected the response between the stimulus onset and the following 300 ms and measured the time of the minimum (valley time) and maximum (peak time) LFP response. Both the valley and peak times were bimodally distributed (P < 0.001 for valley time and P = 0.008 for peak time, Hartigan test). The bimodal distribution for valley time was more pronounced and separated LFPs with positive and negative peaks. Therefore, LFPs were classified as positive and negative based on their valley time (longer or shorter than 70 ms). To measure the correlations between visual responses and eccentricity, we used both bimodal distributions and selected only cells associated with positive and negative LFPs with fast peak times (<100 ms). We then separated these cells as linear (F₁/F₀ ≥ 1) and nonlinear (F₁/F₀) and measured separate correlations for each group between eccentricity and each of the following parameters: L/C ratio, ratio between F₁ responses to luminance and chromatic gratings, F₁ response amplitude in luminance, chromatic, and compound gratings, and F₂ response amplitude in compound gratings. In total, we tested 28 correlations (2 groups based on the F₁/F₀ ratio, 2 groups based on LFP polarity, and 7 measurements of luminance/color response). We found 4 significant correlations (4 of 28) that we report in the Results section.

To investigate how luminance and chromatic signals combine, we fitted the responses to compound gratings with a weighted linear sum of responses to luminance and equiluminant gratings. First, we fitted multiple Gaussian functions to the PSTH measured with luminance (eq. 8a) and equiluminant red–green gratings (eq. 8b).

$$\begin{array}{ll}{R}_{\mathrm{L}}& =a_1\times \left({\text{e}}^{-{\displaystyle \frac{{(t-d_1)}^2}{2{{\sigma }_1}^2}}}+{\text{e}}^{-{\displaystyle \frac{{(t-0.5-d_1)}^2}{2{{\sigma }_1}^2}}}\right)\\ & +a_2\times \left(e^{-{\displaystyle \frac{{(t-d_2)}^2}{2{{\sigma }_2}^2}}}+{\text{e}}^{-{\displaystyle \frac{{(t-0.5-d_2)}^2}{2{{\sigma }_2}^2}}}\right)\end{array}$$ (8a)

$$R_{\mathrm{C}}=a_1\times {\text{e}}^{-{\displaystyle \frac{{(t-d_1)}^2}{2{{\sigma }_1}^2}}}+a_2\times {\text{e}}^{-{\displaystyle \frac{{(t-d_2)}^2}{2{{\sigma }_2}^2}}},$$ (8b)

where R_L is the response to luminance gratings, R_C is the response to chromatic gratings, t is time (from 0 to 1 s), a₁ and a₂ are the amplitudes of the 2 Gaussian functions, d₁ and d₂ are the temporal delays, and σ₁ and σ₂ are the widths of the functions. Because the luminance grating drifted at twice the frequency of the chromatic grating, we used 4 Gaussians in equation 8a and 2 Gaussians in equation 8b. However, the number of free parameters was the same and we used 2 Gaussians per drifting cycle in both equations. Most PSTHs could be well fit by this procedure (mean r² = 0.898 for luminance responses and 0.784 for chromatic responses, n = 138). V1 cells that were well fit for both luminance and chromatic responses (r² ≥ 0.6, n = 113) were selected to perform a weighted linear sum (L× R_L + C× R_C), where L and C are the weights given to the luminance and color responses in the fit, respectively. The weighted linear sum was then fit to the PSTH of the compound grating, and V1 cells with r² ≥ 0.6 (n = 80) were selected for further analysis. The average goodness of fit (r²) for these 80 V1 cells was 0.933 for responses to luminance gratings, 0.784 for responses to chromatic equiluminant gratings, and 0.807 for responses to compound gratings.

To measure the distribution of color and luminance edges in natural scenes, we selected 236 images from the McGill database collection of flowers, fruits, and textures (Olmos and Kingdom 2004). Local changes in luminance and color within the image were extracted with the imfilter function from Matlab Mathworks, Inc., Natick, MA using Sobel filters with 2 different orientations, such as vertical and horizontal. Examples of vertical luminance and color filters are shown below (the horizontal filters are the transposed matrices of the vertical filters):

Sobel luminance filter:

$$\begin{array}{ccc}Red\left[\begin{array}{ccc}-1& 0& 1\\ -2& 0& 2\\ -1& 0& 1\end{array}\right]& \text{Green}\left[\begin{array}{ccc}-1& 0& 1\\ -2& 0& 2\\ -1& 0& 1\end{array}\right]& \text{Blue}\left[\begin{array}{ccc}-1& 0& 1\\ -2& 0& 2\\ -1& 0& 1\end{array}\right]\end{array}$$

Sobel red/green chromatic filter:

$$\begin{array}{ccc}Red\left[\begin{array}{ccc}-1& 0& 1\\ -2& 0& 2\\ -1& 0& 1\end{array}\right]& \text{Green}\left[\begin{array}{ccc}1& 0& -1\\ 2& 0& -2\\ 1& 0& -1\end{array}\right]& \text{Blue}\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ 0& 0& 0\end{array}\right]\end{array}$$

Sobel filters were applied to each image by using the Matlab command imfilter (image, Sobel) and we obtained WxHx3 matrices as outputs. WxH is the image size (W: width, H: height) and the third dimension is the color intensity for Red, Green, and Blue (RGB).The WxHx3 matrices obtained with vertical and horizontal filters were truncated to a range from 0 to 255 (i.e., negative values were set to 0 and values larger than 255 were set to 255) and summed together to obtain 2 filter outputs for each image: luminance (when using Sobel luminance filters) and color (when using Sobel red/green filters). The truncating procedure resembled the response saturation and rectification of negative values observed in cortical neurons. To compare the outputs of luminance and color filters, we averaged the filter outputs across the third dimension (RGB) and obtained WxH matrices. Each value in these WxH matrices represents the intensity of a luminance edge or a color edge at each pixel location of an image. Note that the maximum filter output is obtained when a local region of the natural scene matches the orientation, phase, and color of the filter. However, as for V1 neurons, horizontal and vertical filters also “respond” to oblique orientations and unoriented spots. Moreover, as for V1 neurons, an output in the chromatic filter could be obtained not only from regions with local changes in color, but also from regions with local changes in luminance.

The frequency distribution of values from both filter outputs resembled a Poisson distribution. The tails of the Poisson distributions for color and luminance filters crossed at intermediate output values, as there were fewer pixels with a large filter output for color than for luminance. In this paper, we report a negative correlation between the average output across pixels obtained with luminance and color filters. This negative correlation was most pronounced when we selected a range of output values near the crossing point of the luminance and color distributions (e.g., r = −0.59, P < 0.0001, for range: 50–100), and it became weaker when the range was extended to all values, 0–255. However, the negative correlation was still significant for a wide range between 10 and 250 (r = −0.15, P = 0.02). Such negative correlation could not be demonstrated if we used random noise instead of natural scenes (236 images with RGB values randomly assigned, r = 0.07, P = 0.272 for the range with a lowest P-value, which was 50–100).

Results

We studied the visual responses of retinal ganglion and V1 cells in macaque monkeys with 3 different types of drifting gratings that had either luminance contrast (luminance gratings), chromatic equiluminant contrast (chromatic gratings), or luminance and chromatic contrasts with different spatial frequencies (compound gratings, Fig. 1). Equations describing such gratings can be found in the Methods section of this paper and elsewhere (Lee et al. 2011; Cooper et al. 2012). Luminance and chromatic gratings were of the standard form, with 2 bars per cycle, light–dark and equiluminant red–green, respectively. Instead, the novel compound grating had 4 bars per cycle (red-dark–green-dark), the red and green components being of the same luminance. As illustrated in Figure 1, the compound grating had a color cycle period (red–green) and a luminance cycle period (red-dark or green-dark) with different spatial frequencies that resulted, for a given drift speed, in different temporal frequencies. For example, when the compound grating drifted at 2 Hz for luminance (i.e., 2 dark–light bars per second), it drifted at 1 Hz for color (i.e., 1 red/green bar per second).

Responses of Retinal Ganglion Cells to Compound Gratings

Parvocellular retinal ganglion cells respond more strongly to equiluminant chromatic gratings (Fig. 2A, middle) than luminance gratings (Fig. 2A, left). Magnocellular cells respond more strongly to luminance than to chromatic gratings (Fig. 2B, left and middle). Moreover, while parvocellular cells respond consistently at the frequency of the equiluminant grating (Fig. 2A), many magnocellular cells respond at twice that frequency (Fig. 2B, cell 1) because of a chromatic nonlinearity in the receptive field (Lee et al. 1989a; Lee and Sun 2009). Consistently, when stimulated with compound gratings, magnocellular and parvocellular cells responded at different temporal frequencies (Lee et al. 2011). Parvocellular cells responded to the chromatic frequency of the compound grating (Fig. 2A), whereas magnocellular cells responded to the luminance frequency (Fig. 2B). Therefore, parvocellular cells responded to the compound grating as if it was a red–green equiluminant grating (compare middle and right columns in Fig. 2A), whereas magnocellular cells responded to the compound grating as if it was a luminance grating (compare left and right columns in Fig. 2B). These differences between parvocellular and magnocellular retinal ganglion cells could be quantified by calculating the ratio between the responses to the luminance (L) and the chromatic (C) component of the compound grating (L/C ratio). The luminance component is extracted from the F₂ response harmonic and the chromatic component from the F₁ harmonic. In the examples illustrated in Figure 2, the parvocellular cells had an L/C ratio of <1 because they responded most vigorously to the chromatic contrast of the compound grating, whereas magnocellular cells had an L/C ratio of >1 because they responded most vigorously to the luminance contrast.

Figure 2.

Examples of visual responses from parvocellular and magnocellular retinal ganglion cells to color contrast. Parvocellular cells responded to the chromatic component of the compound grating, and magnocellular cells responded to the luminance component. (A) Examples of visual responses from 2 parvocellular cells to the 3 gratings. Note that the responses to the compound grating (right) resembled more closely those to the chromatic equiluminant grating (middle) than those to the luminance grating (left). Consequently, the ratio of luminance (L) and chromatic (C) responses, L/C, was <1 (shown on the right). (B) Examples of visual responses from 2 magnocellular cells to the 3 gratings. The responses to the compound grating (right) resembled more closely those to the luminance grating (left) than the chromatic grating (middle). Consequently, the ratio of the responses at the luminance (L) and chromatic contrasts (C) of the compound grating, L/C, was >1 (shown on the right). In these and the following figures, rasters and PSTHs were arbitrarily centered at each response cycle for illustration purposes.

Responses of V1 Cells to Compound Gratings

As with retinal ganglion cells, some V1 cells responded mostly to the chromatic contrast of the compound gratings, while others responded mostly to the luminance contrast. For simplicity, we refer to V1 cells that are mostly driven by chromatic contrast as Parvo-like V1 cells and those that are mostly driven by luminance contrast as Magno-like V1 cells. Figure 3A illustrates 2 examples of Parvo-like V1 cells that responded similarly to compound and chromatic gratings (Fig. 3A, right and middle). An opposite pattern is shown in Figure 3B for 2 other examples of Magno-like V1 cells (Fig. 3B), which responded similarly to compound and luminance gratings (Fig. 3B, left, right). As in magnocellular retinal ganglion cells (Lee and Sun 2009), Magno-like V1 cells often generated a weak frequency-doubled response to red–green equiluminant gratings (Fig. 3B, middle), which has not been previously reported in the cortex. On average, Magno-like V1 cells (L/C ratio > 1.5) responded to equiluminant chromatic gratings with firing rates of 10–12 spk/s (F₂: 10.25 spk/s, F₁: 12.36 spk/s), which were similar to the responses of magnocellular retinal ganglion cells to the same stimulus (F₂: 12.34 spk/s, F₁: 11.12 spk/s).

Figure 3.

Examples of visual responses from V1 cortical cells to the 3 grating types. As in the retina, some cells responded to the chromatic component of the compound grating (Parvo-like V1 cells) and others to the luminance contrast (Magno-like V1 cells). However, unlike the retina, many V1 cells seem to have mixed parvocellular/magnocellular responses. (A) Two Parvo-like V1 cells that responded similarly to compound and chromatic gratings. (B) Two Magno-like V1 cells that responded similarly to compound and luminance gratings. (C) Two hybrid V1 cells that responded to the luminance contrast of the compound gratings but with different response strength to red and green bars. Spon: spontaneous activity.

While the results described above support the notion that parvocellular and magnocellular pathways remain well segregated at the level of some cells in area V1 (Blasdel and Lund 1983; Chatterjee and Callaway 2003), other V1 cells showed properties consistent with mixed parvocellular and magnocellular inputs. Figure 3C illustrates examples from 2 cells that responded to both the luminance and chromatic contrast of the compound grating. The cell illustrated at the top was strongly driven by both luminance and chromatic contrast, while the cell at the bottom was better driven by luminance than color (Fig. 3C, left and middle). While the 2 cells responded to the luminance frequency of the compound grating, they both responded with different strengths to the red and green color bars (Fig. 3C, right). Some V1 cells seemed to respond to different temporal phases with equiluminant and compound gratings; however, the recordings did not last long enough to investigate these differences in detail. Because most of the visual responses from hybrid cells were more strongly driven by luminance than chromatic contrast, their L/C ratio was >1.

The visual responses of some V1 neurons were strongly dominated by the chromatic contrast of blue–yellow compound gratings (L/C < 1). These V1 neurons responded poorly to luminance and red–green gratings as small-bistratified koniocellular retinal ganglion cells (Lee et al. 2011). Figure 4 illustrates an example of such a Konio-like V1 cell, which responded more strongly to blue–yellow gratings (Fig. 4B, middle and right) than luminance gratings (Fig. 4A, left). This V1 cell had very high spontaneous activity (∼50 spk/s), showed no OS, responded better to low spatial frequencies when using blue–yellow contrast than luminance contrast, and its preferred spatial frequency for luminance was ∼1 cycle/degree (Fig. 4B, left). Note that the high spontaneous activity of cell was completely abolished when red–green stimuli were presented in the receptive field (Fig. 4A, middle and right). Moreover, the red–green color seemed to have an aftereffect on the spontaneous activity of the cell, which increased to >50 spk/s in trial blocks with red–green stimulation (compare Fig. 4A middle-right with B middle-right), resembling an aftereffect reported in blue-on retinal ganglion cells (Gouras and Zrenner 1979). We studied 8 V1 cells with compound blue–yellow gratings, 3 of which had an L/C ratio of <1, as the example illustrated in Figure 4. In 5 of these cells, the spontaneous activity after stimulation with red–green gratings was twice that after stimulation with blue–yellow gratings (31.48 vs. 12.75 spk/s, P = 0.01, paired t-test). These results demonstrate that, as for Parvo-like V1 cells, Konio-like V1 cells respond more strongly to the chromatic contrast than the luminance contrast of the compound grating and have an L/C ratio of <1.

Figure 4.

Example of a Konio-like V1 cell that responded to blue–yellow color contrast. This V1 cell responded similarly to a retinal ganglion cell from the koniocellular pathway (Lee et al. 2011). The Konio-like V1 cell had high spontaneous activity (∼50 spk/s), responded weakly to luminance gratings, was completely turned off by red–green color gratings, and showed no OS. (A) Visual responses to luminance (left), equiluminant red–green color (middle), and red–green compound gratings (right). Note the high spontaneous activity (Spon) of the cell, which increases to >50 spk/s when using red–green color gratings. (B) The cell had no OS (left, top), and it showed band-pass spatial frequency tuning for luminance gratings (left, bottom, dark circles, and lines) and low-pass spatial frequency tuning for yellow–blue equiluminant gratings (left, bottom, blue circles, and lines). The strongest cell responses were obtained with an equiluminant (middle) and compound (right) yellow–blue color grating. The firing rate was measured as the first Fourier harmonic of the response to the drifting grating (F₁) to plot orientation tuning and spatial frequency (left). The firing rate was measured as the instantaneous rate (spikes in a 10-ms bin) to plot the histograms (middle and right).

Responses of V1 Cells to Luminance and Chromatic Contrasts

Retinal ganglion cells can be reliably classified into parvocellular and magnocellular types based on their chromatic opponency and luminance contrast sensitivity (Lee 2011). As illustrated in Figure 5, parvocellular and magnocellular retinal ganglion cells could also be classified based on the L/C ratios obtained from their responses to red–green compound gratings. Almost all parvocellular cells had an L/C ratio of <1 and almost all magnocellular cells had an L/C ratio of >2 (usually >4). The only exceptions were 2 parvocellular cells with an L/C ratio of 1.01 and 1.15 and 1 magnocellular cell with an L/C ratio of 1.96 (Fig. 5A). Parvocellular cells responded more strongly to chromatic gratings than magnocellular cells (43.2 vs. 29.35 spk/s, P = 0.017, Wilcoxon test, Fig. 5B), whereas magnocellular cells responded more strongly to luminance gratings than parvocellular cells (Fig. 5C, 37.38 vs. 24.83 spk/s, P = 0.005, Wilcoxon test). All retinal ganglion cells that responded to the chromatic component of compound gratings at rates >25 spk/s were parvocellular cells and those that responded at <25 spk/s were magnocellular cells (Fig. 5B). Note that the visual responses to the chromatic component of the compound grating provided a better separation between parvocellular and magnocellular retinal ganglion cells than those to equiluminant gratings (Fig. 5B). Our sample of retinal ganglion cells was not large enough (n = 40) to reveal a significant bimodal distribution for L/C ratio (Fig. 5A) or response strength to color contrast (Fig. 5B, top histogram) with a Hartigan test. However, the distribution did not appear unimodal and could not be fit with a single Gaussian function (Fig. 5A, r² = 0.19).

Figure 5.

Distribution of responses to color/luminance from retinal ganglion cells and V1 cells. In the retina, magnocellular and parvocellular cells responded strongly either to the luminance (L) or chromatic (C) contrast of the compound gratings. In contrast, in the visual cortex, the L/C distribution was unimodal. (A) Distribution of L/C ratios in retinal ganglion cells. (B) Scatter plots and histograms of the color responses from retinal ganglion cells to compound (x-axis) and equiluminant gratings (y-axis). (C) Scatter plots and histograms of the luminance responses from retinal ganglion cells to compound (x-axis) and luminance gratings (y-axis). (D–F) Same as A–C for V1 cells with linear spatial summation (F₁/F₀ ≥ 1). (G–I) Same as A–C for V1 cells with nonlinear spatial summation (F₁/F₀ < 1).

Unlike what is seen in retinal ganglion cells, the distribution of L/C ratios for V1 neurons could be accurately fit with a single Gaussian function (Fig. 5D,G, r² = 0.92 for cells with F₁/F₀ ≥ 1; r² = 0.97 for cells with F₁/F₀ < 1), and similar unimodal distributions could be demonstrated for their response amplitude to color contrast (Fig. 5E,H) and luminance contrast (Fig. 5F,I). By definition, V1 color neurons with an L/C ratio of <1 responded to compound gratings similarly to parvocellular retinal ganglion cells (compare Figs 2A and 3A). However, unlike retinal ganglion cells, Parvo-like V1 cells fell at the end of a continuum for the L/C ratio that included many hybrid cells responding to both chromatic and luminance contrasts. It should be noted that we only tested one contrast ratio with the compound gratings, as there was limited time to study V1 cell responses in awake primates. However, the same contrast ratio was used to measure visual responses in retinal ganglion cells and V1 cells to facilitate comparisons between the 2 neuronal populations.

Stimulus Selectivity of V1 Neurons That Respond to the Chromatic Contrast

V1 cells were classified based on the linearity of spatial summation measured with luminance gratings (Movshon et al. 1978; Skottun et al. 1991; Chen et al. 2009). We first calculated the ratio between the response modulation at the frequency of the luminance grating (F₁) and the mean firing rate (F₀). Then, V1 cells were classified as linear (F₁/F₀ ≥ 1) and nonlinear (F₁/F₀ < 1). As a population, V1 cells with linear spatial summation responded ∼4 times more robustly to the luminance than chromatic contrast of the compound grating (mean L/C = 4.31, Fig. 5D), and although 30 V1 cells had L/C ratios of <1, most V1 neurons had L/C ratios of ≥1. This preference of V1 cells for luminance contrast has been emphasized before (Hubel and Wiesel 1968), but it should be interpreted with caution. For example, while the RMS contrast was higher for the chromatic component than the luminance component of the compound gratings, the cone contrast of the chromatic component was lower than the luminance contrast. To estimate the L/C distribution for an RMS luminance contrast equal to the chromatic cone contrast, we measured the average contrast response function of 72 V1 neurons and calculated the ratio between the response to 71% contrast and 10–20% contrast (contrast response ratio = 2.38 for 10% and 1.56 for 20%). We then recalculated the L/C ratios after dividing the luminance response by the contrast response ratio. In neurons with F₁/F₀ ≥ 1, the mean of the L/C distribution was 2.76 for 20% luminance contrast and 1.81 for 10%. In neurons with F₁/F₀ < 1, the mean of the L/C distribution was 2.24 for 20% luminance contrast and 1.47 for 10% contrast. Therefore, we conclude that the mean of the L/C distribution would be reduced to ∼2 if the luminance component of the compound grating approached the cone contrast of the color pattern.

It should be noted that the L/C ratio was calculated by dividing the second Fourier harmonic by the first Fourier harmonic of the response to the compound grating. In V1 cells with linear spatial summation (Fig. 5D, mean L/C = 4.31), this F₂/F₁ measurement provides a reasonable estimate of the L/C ratio because the response to luminance is strongly dominated by the F₂ harmonic and those to color is strongly dominated by the F₁ harmonic (see Materials and Methods). The interpretation of the L/C ratio is more complicated in nonlinear cells (Fig. 5G, mean L/C = 3.49) because the responses to both luminance and color contain other Fourier harmonics. As shown in Figure 5, V1 cells with linear and nonlinear spatial summation had relatively similar distributions of L/C probably because what distinguishes linear from nonlinear cells is the ratio between the first Fourier harmonic (F₁) and the mean rate (F₀) rather than that between the first and second Fourier harmonics (F₂/F₁).

Interestingly, in V1 linear cells, luminance–color preference (L/C ratio) and OS were significantly correlated. This correlation was weak but it could be demonstrated for 3 different measures of orientation tuning (Fig. 6A–C): CV (r = 0.3, P = 0.01), HWHH (r = 0.34, P = 0.01), and OS (r = −0.35, P = 0.004). All 3 measures revealed a consistent tendency for linear V1 cells with poor OS to be less modulated by chromatic contrast than those with better OS. However, consistent with previous studies (Leventhal et al. 1995; Friedman et al. 2003; Gegenfurtner 2003), such correlations could not be demonstrated when measuring the luminance/color response ratio with separate equiluminant and luminance stimuli and they were not present in nonlinear cells (Fig. 6A–C). Because the correlations between the L/C ratio and OS were weak (r = 0.30–0.35), we performed a bootstrap test to measure the robustness of their significance. In this bootstrap test, we randomly removed 2 data values from each of the scatter plots illustrated in Figure 6A–C and then calculated the significance of the correlation. We performed this test 10 000 times for each scatter plot and calculated the mean of the distribution of P-values and the percentage of P-values that were <0.05. The correlations illustrated in Figure 6A and C had mean P-values of 0.015 and 0.005, respectively, and 100% of the P-values obtained in 10 000 tests were <0.05. In the correlation illustrated in Figure 6B, the mean P-value was 0.017 and 96% of the 10 000 P-values were <0.05.

Figure 6.

Relation between luminance/color preference and stimulus selectivity in linear and nonlinear V1 cells. Scatter plots showing how luminance/color preference (L/C ratio) relates to different response properties of linear (F₁/F₀ ≥ 1, solid circles) and nonlinear cells (F₁/F₀ < 1, open circles). Correlation (r) and significance (P) values are shown at the top of each plot. Significant correlations are highlighted in bold. (A)Significant correlation between L/C and CV in linear cells (r = 0.3, P = 0.01). (B)Significant correlation between L/C and orientation HWHH in linear cells (r = 0.34, P = 0.01). (C) Significant correlation between L/C and OS in linear cells (r = −0.35, P < 0.01). (D) No significant correlations between L/C and DS. (E) No significant correlations between L/C and the spatial frequency peak. (F) No significant correlations between L/C and spatial frequency HWHH.

On average, nonlinear V1 cells tended to have smaller L/C ratios than linear cells (nonlinear: 2, linear: 3.1, P = 0.03, n = 134, Fig. 6A–C), a difference that was more significant if we selected only V1 cells with spatial frequency tunings accurately fit with single Gaussian functions (r² ≥ 0.7) for both F₀ and F₁ responses (nonlinear: 2, linear: 3.47, P = 0.006, n = 111, Wilcoxon rank-sum test). These results indicate that, on average, nonlinear V1 cells amplify chromatic contrast more than linear V1 cells, when stimulated with patterns that have both luminance and chromatic components. Surprisingly, the L/C ratio was not significantly correlated with DS (Fig. 6D), as it would be expected if directional selective cells responded poorly to the chromatic contrast. In fact, many nonlinear V1 cells that were robustly driven by equiluminant red–green gratings had excellent orientation and direction selectivity (Fig. 7). Many of these nonlinear cells (cells 1, 3–5 in Fig. 7) also generated a frequency-doubled response to equiluminant chromatic gratings that resembled the frequency-doubled responses previously described in magnocellular retinal ganglion cells (Lee and Sun 2009). The positive LFPs associated with these cells (see Materials and Methods) suggest that they were located in the deep layers of the cortex. Consistent with previous studies (Zeki 1983; Livingstone and Hubel 1984; Yoshioka and Dow 1996; Conway 2001; Conway et al. 2002; Landisman and Ts'o 2002; Johnson et al. 2008), linear V1 cells with poor OS could be strongly modulated by the chromatic contrast (Fig. 8A, cell 1) or luminance contrast (Fig. 8A, cells 2–3). Conversely, linear V1 cells with sharp OS could be poorly driven by the chromatic contrast of equiluminant gratings (Fig. 8B, cells 1–4). Some V1 cells showed color response preference when driven by compound gratings (i.e., responded differently to the red and green bars of compound gratings), even if they responded weakly to equiluminant chromatic gratings (Fig. 8B, cells 3 and 4). For example, 23% of V1 cells responding weakly to red–green equiluminant gratings (<10 spk/s) were driven 50% more strongly by one color of the compound grating than the other (average: 35 ± 27% stronger responses, n = 34). This finding reveals another example of nonlinear summation in the combined responses of V1 neurons to color and luminance.

Figure 7.

Examples of nonlinear V1 cells that were orientation/direction selective and responded to chromatic contrast. The histograms and rasters on the left illustrate responses to luminance (left), equiluminant red–green (middle), and compound (right) gratings. Spon: spontaneous activity. L/C: luminance/color ratio. The histograms and plots on the right illustrate the responses at different orientation preferences (histograms on the left) and the orientation/direction tuning (right plots). The orientation/direction tuning was measured as the mean firing rate (F₀ in black) and as the F₁ response (F₁, red). The firing rate of the histograms was measured as the instantaneous rate (spikes in a 10-ms bin). The insets on the right at the top of the histograms show the average LFP, which was used to estimate the cortical depth of the recordings (see Materials and Methods).

Figure 8.

Examples of linear V1 cells with a diverse range of chromatic responsiveness and OS/DS. Same format as Figure 7. (A) Examples of 3 cells with poor OS. (B) Examples of 4 cells with sharp OS.

V1 Cells in Deep Cortical Layers Showed Strongest Combined Responses to Luminance and Color

Our chronic multielectrode arrays allowed us to independently move each electrode through the depth of the cortex in awake primates. Because we recorded both single units and LFPs from the same electrode tip, the cortical depth of the recording could be estimated from the polarity of the LFP (Fig. 9A). Deep layers of macaque area V1 are known to be associated with initially positive LFPs, whereas those recorded more superficially are associated with initially negative LFPs (Schroeder et al. 1998; Maier et al. 2011; Lashgari et al. 2012; see also Xing et al. 2012). Consistently, when the LFPs associated with our recorded cells were plotted as a function of their peak and valley times, we could reveal separate LFP clusters and bimodal distributions (Fig. 9B, P < 0.001 for valley time and P = 0.008 for peak time, Hartigan test). As shown in Figure 9C, V1 linear cells associated with positive LFPs responded more strongly to luminance and color stimuli, had higher spontaneous activity, and were less selective to stimulus orientation, direction, and spatial frequency than those associated with negative LFPs. The differences for nonlinear V1 cells were similar with the exception of DS (Fig. 9D). While positive LFPs are recorded in the deep layers of primary visual cortex (Schroeder et al. 1998; Maier et al. 2011; Lashgari et al. 2012), accurate layer identification can only be confirmed with careful histological recordings (Johnson et al. 2008). Therefore, it is possible that some neurons from layer 4C were associated with slightly positive LFPs, which would explain the higher spontaneous activity of this group.

Figure 9.

V1 cells associated with positive LFPs in deep cortical layers generate stronger combined responses to color and luminance. The laminar location of the each V1 cell was estimated based on the polarity of the LFP recorded from the same electrode tip. (A) LFPs generated by the onset of a grating stimulus presented at time 0 and measured with a single electrode that was moved through the depth of the cortex. As reported previously, the polarity of the LFP was negative at the superficial layers (top) and positive at the deep layers (bottom). (B) LFPs associated with all recorded V1 cells formed separate clusters when plotted as a function of their LFP peak (positive) and valley (negative) times and their distributions were bimodal (P < 0.001 for valley time and P = 0.008 for peak time, Hartigan test). The outliers in the distribution (e.g., peak time = 0) were flat LFPs usually recorded in the superficial layers of the cortex. (C) Response properties of V1 cells with linear spatial summation (F₁/F₀ ≥ 1). V1 cells associated with positive LFPs (light bars) generated stronger responses to chromatic and luminance stimuli had larger L/C ratios, broader tuning, and higher spontaneous activity than V1 cells associated with negative LFPs (dark bars). (D) Same as (C) but for nonlinear V1 cells (F₁/F₀ < 1). Spon: spontaneous activity; F₀: mean response to luminance gratings (Lum), red–green equiluminant gratings (RG), and compound gratings (CP); F₁: first harmonic of response; F₂ CP: second harmonic of response to compound gratings; CV: circular variance; ORb: orientation bandwidth, measured as HWHH; OS: orientation selectivity measured as the response to the preferred orientation divided by the response to the orthogonal orientation; DS: direction selectivity measured as 1 − (response to nonpreferred direction/response to preferred direction); SFp: peak of the spatial frequency tuning; SFb: spatial frequency bandwidth measured as HWHH; L/C: luminance/color ratio calculated from the F₂/F₁ ratio of the response to compound gratings; a.u.: arbitrary units. *P < 0.05; **P < 0.001.

In nonlinear V1 cells associated with positive LFPs, the eccentricity was correlated with the mean firing rate evoked by luminance gratings (r = 0.538, P = 0.003) and equiluminant chromatic gratings (r = 0.395, P = 0.031). This correlation indicates that nonlinear responses increase with visual eccentricity in the deep layers of area V1. Surprisingly, in linear V1 cells, the eccentricity was negatively correlated with the L/C ratio obtained from responses to compound gratings (r = −0.367, P = 0.002). Moreover, if the luminance/color ratio was calculated using separate luminance and equiluminant gratings, a similar negative correlation could be demonstrated for linear V1 cells associated with negative LFPs (r = −0.609, P = 0.006). These negative correlations indicate that linear responses in V1 increase more with visual eccentricity when driven by the chromatic than luminance contrast, which supports the notion that chromatic responses are well represented at high eccentricities in area V1.

V1 Luminance/Color Mixing May Provide a Mechanism to Process Edge Diversity in Natural Scenes

To further investigate how luminance and color signals combine in area V1, we fit the cell responses to luminance gratings (R_L) and chromatic gratings (R_C) with double Gaussian functions (see Materials and Methods). We then weighted the luminance response (L) and the chromatic response (C) and optimized the weighted linear sum of these fits (L× R_L + C× R_C) to match the responses to the compound grating. We selected 80 V1 cells with responses to compound gratings that could be accurately fit (r² ≥ 0.6) from this linear sum. Consistent with our previous results, some V1 cells were more strongly driven by equiluminant red/green contrast (Fig. 10A, top), others by luminance contrast (Fig. 10A, middle), and others by a combination of both (Fig. 10A, bottom). Also consistent with previous results, the compound gratings generated stronger color responses than equiluminant gratings (Fig. 10B, top), but most neurons showed weaker luminance responses to compound gratings than to luminance gratings (Fig. 10B, bottom). Most interestingly, this analysis revealed a negative correlation between the luminance weights (L) and chromatic weights (C) of the visual responses to compound gratings (R = −0.64, P < 0.001, Fig. 10C). This negative correlation could not be explained by a correlation between luminance and chromatic responses, which was much weaker (r = 0.28, P < 0.001 for luminance/color of compound gratings; r = 0.29, P < 0.001 for separate luminance and chromatic gratings).

Figure 10.

The V1 cell population is optimized to process the luminance/color diversity changes found in natural scenes. V1 cells responded to different combinations of luminance and chromatic contrasts. (A)Responses (red) and Gaussian fits (black) of 3 example V1 cells to luminance, chromatic, and compound gratings. V1 cell responses to luminance gratings (Lum, left) and chromatic equiluminant gratings (Col, middle) were fitted with double Gaussian functions (see Materials and methods). Then, we used a weighted sum of the luminance and chromatic fits (L × Lum + C× Col) to predict the responses of each V1 cell to compound gratings (right). (B) The compound gratings generated stronger color responses (F₁ component of the response) than equiluminant gratings. However, most neurons generated weaker luminance responses to compound gratings (F₂ component of the response) than to luminance gratings. Left, top: the number of neurons with responses to color enhanced (red) or suppressed (blue) by compound gratings. Left, bottom: the number of neurons with responses to luminance enhanced (red) or suppressed (blue) by compound gratings (*P < 0.05, **P < 0.001, χ² test). Right: magnitude of response enhancement and suppression for color (top) and luminance (bottom) (**P < 0.001, Wilcoxon rank-sum test). Cells were classified as enhanced if the response was larger for CP than RG (top) or BW (bottom), for any of the measurements on the x-axis (F₁, F₀, Amp). Cells were classified as suppressed if the response was lower for CP than RG (top) or BW (bottom). Amp: peak firing rate calculated from the Gaussian fit (bin size = 2 ms). (C) Scatter plot of the luminance (L) and chromatic (C) weights used to predict the response to compound gratings, showing a negative correlation (R = −0.64), with a slope of −1.1. (D) The normalized output of luminance and color Sobel filters applied to natural scenes are negatively correlated, as is also the case for V1 cells in C. Each circle represents an image defined by its normalized filter output for luminance and color, averaged across pixels with intermediate contrasts (see Materials and Methods for details). We selected 236 images from the McGill database collection of flowers, fruits, and textures (Olmos and Kingdom 2004). Example images are shown on the right.

To better understand the significance of the negative correlation between luminance and color responses, we measured the distribution of both the luminance and chromatic contrast in natural scenes (see also Hansen and Gegenfurtner 2009; Cecchi et al. 2010). We selected 236 images from the McGill database collection of flowers, fruits, and textures (Olmos and Kingdom 2004) and processed each image with Sobel luminance and chromatic filters (see Materials and Methods). As with V1 neurons, the Sobel luminance filters were more sensitive at detecting luminance contrast and the Sobel color filters were more sensitive at detecting color contrast; however, Sobel color filters could also detect luminance contrast (see Materials and Methods). The average output of these Sobel filters ranged from 0 to 255, for each pixel in an image (see Materials and Methods for details). Interestingly, at intermediate values of the filter response (50–100), which correspond to intermediate contrasts in the image (∼20–40%), the filter responses for luminance and color were negatively correlated (Fig. 10D, R = −0.59, slope: −0.9), closely resembling the correlation between the luminance and chromatic responses that we demonstrate in area V1 (R = −0.64, slope: −1.1). The negative correlation was still present if we extended the range of filter outputs to 10–250, but it was much weaker (r = −0.15, P = 0.02). Finally, the negative correlation disappeared if we used random noise instead of natural images (see Materials and Methods for more details). While a negative correlation between the luminance and color contrast could be demonstrated in both V1 responses and natural images, comparisons are complicated by several important differences between the 2 measurements. First, the distribution of color/luminance responses from V1 neurons (Fig. 10C) and Sobel filters (Fig. 10D) are not identical. Secondly, the range of color contrasts and background luminances is greater in natural scenes than in the stimuli used to drive V1 neurons. Thirdly, the statistics of natural images is very different from that of grating stimuli. In spite of these limitations, our results support the notion that the mixing of luminance and color signals in area V1 provides a mechanism to process the diversity of luminance and color contrast found in natural scenes.

Discussion

We have used a compound grating to investigate the visual responses of V1 neurons to the luminance and color contrast. Some V1 neurons responded to the compound grating as if their responses were strongly dominated by magnocellular, parvocellular, or koniocellular inputs (Blasdel and Lund 1983; Hendry and Reid 2000; Chatterjee and Callaway 2003), while others responded as if they were receiving mixed inputs. The ratio between the response amplitude to the luminance (L) and chromatic (C) components of the compound grating was unimodally distributed and pronouncedly biased towards luminance (mean L/C = 4.31 in V1 linear cells), indicating that V1 responds ∼4 times more robustly to the luminance than to chromatic contrast (∼2 times if the luminance contrast is equalized to the chromatic cone contrast). The luminance/color preference of V1 linear neurons was correlated with their OS, a finding that adds a novel perspective to an ongoing discussion on the neuronal encoding of color and orientation in the primary visual cortex (Livingstone and Hubel 1984; Johnson et al. 2001, 2008; Gegenfurtner 2003; Economides et al. 2011; Song et al. 2011). We also demonstrate that V1 cells associated with positive LFPs in deep cortical layers (the subcortical output of area V1) show stronger combined responses to luminance and color than other V1 cells. Finally, we show that, as a population, V1 cells respond to a diverse combination of stimuli that differ in luminance/color, orientation, direction of motion, and spatial frequency, a population that seems optimized to process the diversity of stimuli found in natural scenes.

The neuronal encoding of color and OS in area V1 remains a matter of debate (Conway et al. 2010). Some studies found that many V1 cells driven by chromatic contrast lacked OS (Zeki 1983; Livingstone and Hubel 1984; Yoshioka and Dow 1996; Conway 2001; Conway et al. 2002; Landisman and Ts'o 2002; Johnson et al. 2008) or showed coarse selectivity, sometimes specific for color stimuli (Conway 2001; Conway et al. 2002; Conway and Livingstone 2006). Others found evidence for V1 cells that encoded both stimulus attributes, color, and orientation (Poggio et al. 1975; Thorell et al. 1984; Lennie et al. 1990; Leventhal et al. 1995; Johnson et al. 2008). While some studies found a negative correlation between color and OS (Zeki 1983; Livingstone and Hubel 1984; Yoshioka and Dow 1996; Conway 2001; Conway et al. 2002; Landisman and Ts'o 2002; Economides et al. 2011), others found no correlation (Leventhal et al. 1995; Friedman et al. 2003; Gegenfurtner 2003) or a correlation with color bandwidth only (Friedman et al. 2003). Surprisingly, our results revealed a correlation between color and OS that was not negative but positive: Linear V1 cells with poor OS were less modulated by the chromatic contrast of compound gratings than those with higher OS. This positive correlation originates from a tendency of hybrid cortical cells to be better tuned to stimulus orientation than those responding to luminance contrast only. The functional meaning of this correlation is unclear, but it could reflect a greater need for neuronal resources that can perform accurate measurements of orientations defined by both the color and luminance contrast (e.g., red apple surrounded by green leaves) rather than orientations defined by the luminance contrast only (e.g., shade in the grass).

Our results also demonstrate that a large number of V1 neurons are driven by equiluminant chromatic gratings, a finding that is consistent with previous studies (Thorell et al. 1984; Lennie et al. 1990; Leventhal et al. 1995; Johnson et al. 2001, 2008). Moreover, we show that cells weakly driven by equiluminant chromatic gratings can still be modulated by the color of compound gratings. While the magnitude of the equiluminant response varied from cell to cell, our results show that many V1 cells respond robustly to equiluminant gratings, usually at twice their frequency, similarly to magnocellular retinal ganglion cells. This frequency-doubled response in magnocellular cells was initially thought to reflect a nonlinearity in M- and L-cone summation (Lee et al. 1989a), but later studies (Lee and Sun 2009) suggested that it was related to a rectified chromatic signal that could be used to enhance motion signals in equiluminant borders (Dobkins and Albright 1994). The frequency-doubled response that we demonstrate in V1 neurons adds an additional complication to the assessment of the linearity of cortical neurons when using chromatic stimuli (see also Horwitz and Hass 2012). Also, consistent with previous studies (Lennie et al. 1990), our results demonstrate that responses to compound gratings are well predicted from a weighted linear sum of responses to luminance and chromatic gratings (Fig. 10A,C). However, deviations from linearity were observed both in responses to color and luminance (Fig. 10B). Therefore, both linear and nonlinear interactions are likely to play an important role in the processing of color in visual cortex.

Finally, our study raises the question: What could be the advantage of mixing magnocellular and parvocellular pathways in area V1? While magnocellular and parvocellular inputs are clearly segregated in cortical layers that receive thalamic inputs, their mixing in other layers could help diversify the V1 receptive field array available for visual processing. Within the magnocellular pathway, ON and OFF channels are known to remain segregated in retina and thalamus but combine in the visual cortex to generate a diverse array of ON–OFF receptive fields that can be used to encode multiple combinations of orientations, spatial frequencies, and phases with different luminance contrasts. Similarly, magnocellular and parvocellular pathways may combine in area V1 to encode different arrangements of orientations, spatial frequencies, and phases that are defined by luminance contrast, chromatic contrast, or a combination of both. While having V1 cells strongly dominated by the parvocellular pathway may be needed to encode the color of a stimulus, an important role of area V1 is to encode stimulus orientation, independently of whether the contrast is chromatic, achromatic, or mixed. It seems that combining separate retinal inputs with different contrast polarities (ON and OFF), different color preferences (magnocellular and parvocellular), and different eyes of origin (left and right) is a better strategy to process local orientation and direction efficiently than keeping the channels cleanly segregated in area V1.

Funding

This work was supported by NIH grants EY02067901 (J.M.A.) and EY013112 (B.B.L.) and NSFC grant 91120304 (Y.C.).