Circuitry to explain how the relative number of L and M cones shapes color experience

Abstract

The wavelength of light that appears unique yellow is surprisingly consistent across people even though the ratio of middle (M) to long (L) wavelength sensitive cones is strikingly variable. This observation has been explained by normalization to the mean spectral distribution of our shared environment. Our purpose was to reconcile the nearly perfect alignment of everyone's unique yellow through a normalization process with the striking variability in unique green, which varies by as much as 60 nm between individuals. The spectral location of unique green was measured in a group of volunteers whose cone ratios were estimated with a technique that combined genetics and flicker photometric electroretinograms. In contrast to unique yellow, unique green was highly dependent upon relative cone numerosity. We hypothesized that the difference in neural architecture of the blue-yellow and red-green opponent systems in the presence of a normalization process creates the surprising dependence of unique green on cone ratio. We then compared the predictions of different theories of color vision processing that incorporate L and M cone ratio and a normalization process. The results of this analysis reveal that—contrary to prevailing notions--postretinal contributions may not be required to explain the phenomena of unique hues.

Introduction

Human color matching behavior can be sufficiently explained from the spectral sensitivity functions of the cone photoreceptors alone (Stockman & Sharpe, 1999). However, the discrimination and appearance of color cannot be directly related to the activity of the three classes of cones. Instead, the outputs of the three cones must be combined in postreceptoral circuitry (Hurvich & Jameson, 1957). A large body of literature has implicated parvocellular neurons, which compare L and M cone activity, as the basis of red-green discrimination performance (Derrington, Krauskopf, & Lennie, 1984; Krauskopf, Williams, & Heeley, 1982; Stockman & Brainard, 2010). The small bistratified ganglion cell with its S-ON/(L+M)-OFF spectral opponency (Crook et al., 2009; Dacey & Lee, 1994) is often regarded as the basis of blue-yellow (BY) color discrimination (Dacey, 2000; Mollon, 1999). Yet these subcortical channels, which align with psychophysical discrimination results, do not adequately predict the appearance of colors or the large variability in hue perception (Valberg, 2001).

Remarkably, while people agree about the appearance of some colors, there is surprising disagreement across people about others (Kuehni, 2004). For example, the wavelength of unique green varies so widely (from 490–555 nm) that pure green to one person can look pure blue to another. This striking fact of human color vision has never had an adequate biological explanation. M:L cone ratio is also strikingly variable, but, until recently, it has seemed unlikely to be the cause of individual differences in unique green which is the neutral point of the blue-yellow (BY) opponent system. This is because it has been demonstrated that unique yellow, the neutral point of the red-green (RG) system, is set by normalizing cone signals through experience (J. Neitz, Carroll, Yamauchi, Neitz, & Williams, 2002; Welbourne, Morland, & Wade, 2015). This normalization process results in remarkable consistency across people exposed to similar chromatic environments and is sufficient to compensate for large differences in cone ratio (Brainard et al., 2000; J. Neitz et al., 2002).

The relationship between cone ratio and unique green has not been as well investigated. Recent theoretical work indicates that the degree to which cone ratio influences unique green in the presence of a chromatic normalization process depends on the underlying circuitry (Schmidt, Neitz, & Neitz, 2014). For instance, the small bistratified ganglion cell is often implicated in the perception of blue and yellow (Dacey, 2000). Therefore, the neutral point of those cells should correspond to the sensation of pure green, i.e., a color in the middle-wavelength region of the spectrum that is neither bluish nor yellowish. However, the S – (L+M) signal carried by the small bistratified is constrained by its circuitry to fall between the deutan (∼500 nm) and protan (∼480 nm) spectral neutral points, which fails to capture the known range of unique green values (Schmidt et al., 2014). This observation suggests that the S versus L+M mechanism of the small bistratified cells cannot directly account for human BY color appearance. Consequently, numerous groups have recognized that blue-yellow color vision can only be explained by opponent circuitry in which M cones act synergistically with S cones to signal blueness (Abramov & Gordon, 1994; De Valois & De Valois, 1993; De Valois, De Valois, Switkes, & Mahon, 1997; Drum, 1989; Hofer, Singer, & Williams, 2005; Mollon, 2003; Mollon & Jordan, 1997; J. Neitz & Neitz, 2008; Stockman & Brainard, 2010; Webster, Miyahara, Malkoc, & Raker, 2000). Building upon these observations, we recently demonstrated that a theoretical BY mechanism, separate from the small bistratified cells, which combines S+M cone signals versus L could capture the known variability in unique green (Schmidt et al., 2014). Assuming the same environmental normalization process that accounts for the very limited variability in unique yellow (J. Neitz et al., 2002), this theoretical circuit predicts large changes in unique green with cone ratio.

Here, we began by extending our prior analysis of cone ratio and unique green to three additional models of color appearance: the classical LGN theory in which small bistratified ganglion cells are responsible for blue-yellow color vision and two versions of the De Valois and De Valois (1993) multistage model in which outputs of the small-bistratified and midget ganglion cells are recombined at a cortical stage. We then measured unique hues in volunteers for whom M/L cone ratios were estimated. Our results confirm that a mechanism summing S and M cones is necessary to account for the appearance of middle wavelength light and that L:M cone ratio predicts the spectral location of an individual's unique green. Finally, we considered the predictions of the four models in light of the current results, as well as published findings.

Materials and methods

Participants

Fourteen (eight male, six female) color normal subjects (assessed with a Nagel anomaloscope) were recruited from the University of Washington community. They ranged in age from 22 to 60 years with a mean of 37 (σ = 14.04) years. Three of these participants also enrolled in a study of hue cancellation, including one of the authors (S2). One individual (male, 33 years) with incomplete Congenital Stationary Night Blindness (CSNB1) participated in a separate study. Research on human subjects followed the tenets of the Declaration of Helsinki and was approved by the Human Research IRB at the University of Washington.

Estimates of M:L cone ratio

We estimated M:L cone ratio with a previously described technique that combines genetics and electroretinography (ERG) (Carroll, McMahon, Neitz, & Neitz, 2000; Carroll, Neitz, & Neitz, 2002; Hofer, Carroll, Neitz, Neitz, & Williams, 2005; McMahon, Carroll, Awua, Neitz, & Neitz, 2008). Briefly, each subjects' L and M genes were selectively amplified from DNA extracted from blood or spit samples. From that product, exons 2, 3, and 4 were amplified and sequenced. All of the procedures, primers, and thermal cycling parameters have been reported previously (Carroll et al., 2000; McMahon et al., 2008). ERG heterochromatic flicker photometry (30 Hz duty cycle) was then used to estimate the spectral sensitivity of each subject by adjusting the intensity of a test light until the ERG signal exactly matched that produced by a reference light. Details of the procedure have been described elsewhere (Carroll et al., 2000; Hofer, Carroll et al., 2005; McMahon et al., 2008). The resulting spectral sensitivity measurements were then fit with a weighted sum of L and M spectral sensitivity functions that were parameterized for each subject based on the results of the genetic analysis reported in Appendix 1. The %M values were computed from the L and M weights (M / (L + M) × 100). The %M was used here rather than %L for clarity, since the arguments presented are centered on the contribution of M cones to the BY system.

A limitation of ERG flicker photometry is that there is inevitably a change in chromatic adaptation as the target wavelength is changed, introducing some small systematic error in the estimate of relative L- and M-cone numerosity (Stockman, Jagle, Pirzer, & Sharpe, 2008). Evidence that the errors are small comes from the observation the mean value and distribution of cone ratios estimated with the present method line up well with relatively large samples obtained with other techniques, including estimates from quantitation of L and M messenger RNAs (M. Neitz, Balding, McMahon, Sjoberg, & Neitz, 2006), which are not affected by chromatic adaptation. What is most important for the work reported here is the technique produces close approximations to actual cone ratio values that are reliable and highly correlated with cone ratios measured from the same individuals using adaptive optics densitometry (Hofer, Carroll et al., 2005).

Light source

Monochromatic stimuli were produced with a Gooch and Housego OL 490 controllable light source (Crognale, Webster, & Fong, 2009). Light output was controlled with custom software written in C# (Microsoft Inc.) or MATLAB programming languages. For all experiments, spectra had a full bandwidth at half maximum of 10 nm. Output from the OL 490 was directed with a liquid light guide to either an integrating sphere or a Maxwellian view system. The light source was calibrated with a spectroradiometer (SpectroCal, Cambridge Research Systems) and an optometer (Gamma Scientific, flexOptometer).

Unique hue procedure

Unique yellow and green were determined with a randomly interleaved forced-choice, double-staircase procedure (J. Neitz et al., 2002). All experiments were carried out in the dark. Participants were positioned on a chin rest and freely viewed the circular 1.5-degree port of the integrating sphere. The mean luminance was 34 cd m⁻². Each light remained on until a selection was made; a one second pause separated each light. The program terminated after ten reversals. The step size was adaptive, decreasing to 1 nm after the third reversal. The final five reversals were averaged for each staircase. Each hue was tested three times per session. Therefore, a total of six staircases were averaged for each hue. Subjects completed at least two sessions. The first session was discarded for practice. The order of unique hue measurements was randomized for each session.

Macular pigment optical density

Macular pigment optical density was measured with an established psychophysical technique (Wooten, Hammond, Land, & Snodderly, 1999). The OL 490 produced a square wave 50% duty cycle alternation (at 14 Hz) between a 462 nm test light and a 575 nm reference. The integrating sphere port subtended 1.5°, and its output was superimposed on a blue adapting background (471 nm dominant wavelength) from an organic light emitting diode monitor with a beam-splitter. The luminance of the background adapting light and the reference were kept at 3.31 cd m⁻² and 1.78 cd m⁻², respectively. Participants controlled the intensity of the test light. Subjects viewed the test region with their right eye and were permitted to change the intensity of the test light for as long as needed. The starting intensity was randomly selected. Each measurement was an average of five trials. Each subject completed two sessions of the foveal and parafoveal (6°) condition. The first session for each condition was discarded as practice. Optical density was taken as the log of the ratio of the intensity of the test light in the foveal condition to the intensity in the parafoveal condition (Wooten et al., 1999).

Iris lightness

Iris lightness was measured following Welbourne and colleagues (Welbourne, Thompson, Wade, & Morland, 2013). A 14.2 megapixel Sony NEX-5 with an Advanced Photo System type-C (APS-C) sensor and an HVL-F7S flash (Sony) was used. The camera settings were 1/160 shutter speed, F5.6 aperture, and ISO800 sensitivity. The lens zoom was set to 35 mm, which is roughly equivalent to an 85 mm lens on a standard digital single lens reflex (SLR) camera.

Hue cancellation

Hue cancellation (Hurvich & Jameson, 1957) was performed with a single channel Maxwellian view optical system and the OL 490 light source, which simultaneously delivered test and cancellation stimuli. Test lights ranged from 484 to 582 nm in 7 nm steps with the order of presentation randomized. Cancellation lights consisted of the subject's unique blue or unique yellow (Werner & Wooten, 1979), as determined with the procedure outlined above. Retinal illuminance of each test light was set to 2.3 log trolands, assuming the luminous efficiency function of Sharpe, Stockman, Jagla, and Jägle (2005). The circular field subtended 3.4° on the retina and was presented to the right eye. Stimuli were presented for 1 s with a minimum of 3 s between presentations. Experiments were carried out in a dark room with no background adaptation. Two practice sessions were provided prior to data collection. Subjects were instructed to use a handheld controller to adjust the intensity of the cancellation light until the mixture appeared neither blue nor yellow. Subjects were allowed as many presentations as necessary within each trial. Sessions lasted between 10 and 20 min and no more than three sessions were completed in a single sitting. Each subject completed five sessions. The average relative radiance necessary to cancel each test stimulus was corrected for differences in lens filtering due to aging relative to a 20 year old standard lens (Pokorny, Smith, & Lutze, 1987).

The measured hue cancellation values were then arbitrarily scaled to minimize the least squares difference to the four models described below (see Models section). For each subject, the L cone photopigments were set to peak at the genetically determined maximum sensitivities. The S cone peak was assumed to be 421 nm for all subjects. The relative weightings of the L and M cones were allowed to vary in determining the best fits to the hue cancellation results. The best fitting relative M/L contributions were then compared to the M/L cone ratios determined experimentally.

Control analysis

To assess the potentially confounding effect of age in our unique hues analysis (Schefrin & Werner, 1990), we ran a multiple regression that included %M and age as factors. Age did not significantly add to the predictive power of either of the unique hues (data not shown). Therefore, we did not consider age as a factor in subsequent analyses.

Statistics

All statistical models were implemented in the open source programming language, Python, with the NumPy, SciPy and StatsModels statistical computing extensions. Error bars are represented as ± standard error of the mean.

Models

As explained in the Introduction, we considered how four biologically motivated models of color appearance are influenced by M:L cone ratio and an environmental normalization mechanism. Each model was based on proposed neural circuitry underlying primate color appearance. All models were constructed with the spectral sensitivity functions reported by Carroll et al. (2000, which can be downloaded from http://neitzvision.com/) and lens and macula density filters from Stockman and Sharpe (2000, downloaded at http://www.cvrl.org/). Nonlinearities in the chromatic pathways were not considered in this exercise.

LGN model

The LGN model (Figure 1A) implemented a classical S – (L+M) BY channel and a RG mechanism that differenced L and M cone signals. These opponent interactions are based on the commonly observed opponency in primate LGN (Derrington et al., 1984; Reid & Shapley, 2002; Tailby, Solomon, & Lennie, 2008) that have their retinal origins in the small bistratified (Dacey & Lee, 1994) and midget ganglion cells (Benardete & Kaplan, 1999; Calkins & Sterling, 1996), respectively. Each channel incorporated a normalization process to the mean environmental spectrum (equal energy white) to account for the known plasticity in hue perception after changes in spectral environment (Belmore & Shevell, 2008; Delahunt, Webster, Ma, & Werner, 2004; J. Neitz et al., 2002; Welbourne et al., 2015). The blue/yellow (BY) and red/green (RG) systems were defined as:

L(λ), M(λ) and S(λ) were the spectral sensitivity functions corrected for lens and macular pigment. For each equation, the environmental normalization constant, α, was adjusted such that the opponent response (RG(λ) and BY(λ)) integrated to zero across the visible spectrum. Therefore, an equal energy white produced no response from either system. The fraction of M cones (L + M = 1) was represented by m_frac.

Figure 1.

Predicted relationship between M:L cone ratio and unique hues depends on the underlying circuitry. The predicted influence of a normalization process in the presence of varying %M is shown for the spectral sensitivity of RG and BY chromatic mechanisms under different assumptions about the underlying circuitry. (A) The circuitry of a RG and BY system (left column) incorporating the predominant spectral opponency (middle column) of LGN does not predict large variation in unique hues with %M (right column). (B) The cortical model proposed by De Valois and De Valois anticipates substantial variation in unique green and yellow. ON pathways are represented by black lines; OFF pathways are denoted by gray lines. (C) A hybrid model, built through normalizing the LGN component of the De Valois' model (B), also demonstrates a substantial dependence upon cone numerosity. (D) Finally, a BY system based on a retinal circuit comparing S+M against L signals predicts large changes in unique green with changes in %M.

Figure 1A shows how the S – (L+M) cone opponent signals of small bistratified cells are expected to change as %M is varied over the range most commonly found in the population (Carroll et al., 2002). Though this mechanism predicts a small change in unique green, it could account for only a very small fraction of the wide range of wavelengths selected as unique green by color normal observers. The spectral location of unique yellow is also predicted to be largely unchanged as a function of cone ratio due to the normalizaton process.

De Valois model

The De Valois model (Figure 1B) is an implementation of the model described by De Valois and De Valois (De Valois & De Valois, 1993). The motivation for this model was to explain the transformation in opponent responses between LGN and cortex necessary to account for human hue perception. The model begins (first stage) by taking the cone fundamentals (L(λ), M(λ) and S(λ)), normalized to peak at unity, and opposes them (second stage) in a manner reflective of typical retinal/LGN neurons. This second stage produces three opponent mechanism L_o(λ), M_o(λ), and S_o(λ).

These opponent interactions are the same as in the LGN model, with the exception that the De Valois model did not include a normalization process in its original specification. The proportion of M cones is represented by m_frac.

In the third stage of the model, cortical color opponent cells are built by linearly combining second stage neurons from both the parvocellular (L_o and M_o) and koniocellular (S_o) pathways.

The proportion of M cones, m_frac, influences both the second and third stages of this model. In the original publication the M:L ratio was assumed to be 1:2, but here we permitted M:L cone ratio to vary in order to study the impact of that variable on the expected perception of color in individuals with differing M:L ratios. The predictions of this model are shown in Figure 1B. Unique yellow is expected to vary little across a large range of %M values only moving towards longer wavelengths in subjects with highly skewed cone ratios. Unique green is predicted to follow an inverted U-shaped function (Figure 1B, right column). At low %M, unique green approaches that of a deutan viewer (0%M). With increasing %M, unique green shifts to longer wavelengths, before plateauing at 518 nm between ∼ 55%M–65%M and finally sloping back down towards the protan (100%M) spectral neutral point (∼490 nm) at still greater %M values.

Hybrid model

Because a normalization to the environmental mean has been shown previously to be important in predicting unique yellow data (J. Neitz et al., 2002), we hypothesized that adding a normalization process to the De Valois model might improve predictions of color appearance data. Since the LGN model and the Midget Ganglion Cell models both incorporate a normalization step, a De Valois model with an added normalization step seems like a fairer comparison to the other two. Therefore, in this “hybrid” model, the L_o(λ), M_o(λ), and S_o(λ) mechanisms of the De Valois and De Valois (1993) model were normalized to integrate to zero. Again constant, α, was applied to one term in the equation and adjusted to produce a null to the environmental mean, which was assumed to be equal energy white for simplicity as described in LGN model.

The third stage of the model was mathematically identical to the De Valois model. The predictions of this model (Figure 1C) are similar to the De Valois model with more exaggerated unique yellow values at the extremes of the %M range, a broader plateau, and longer unique green values at all locations.

Midget ganglion model

Finally, we examined the predictions of a model of color appearance that we recently proposed based upon a newly discovered feedforward pathway from HII horizontal cells directly onto cone bipolar cells (Schmidt et al., 2014). Like the De Valois model, our theory results in a BY system with S+M versus L opponency and a RG system that opposes L+S against M signals. However, there are four major differences between this theory and theories that propose an added cortical stage to reconcile the physiological properties of midget and small bistratified cells with hue perception. According to our theory, (a) small bistratified cells do not contribute to conscious hue perception; (b) the majority (>90%) of midget ganglion cells do not contribute to conscious hue perception—rather they are only involved in producing conscious achromatic sensations in spite of their L/M opponency; (c) green and yellow hues are specifically relayed through ON pathways, whereas red and blue sensations are carried by OFF pathways; and (d) color and achromatic information is separated in the outer retina.

Our theory has been described in detail (Schmidt et al., 2014). Briefly, it proposes that human hue sensations are mediated by four different subclasses of midget ganglion cells which receive S cone input via a newly discovered feed-forward pathway from S cones via HII horizontal cells directly to midget bipolar cells (Puller, Haverkamp, Neitz, & Neitz, 2014; Puller, Manookin, Neitz, & Neitz, 2014). Specifically, we propose (a) that OFF-midget ganglion cells with S cone ON feed-forward (Puller, Manookin et al., 2014)) input and L cone centers serve the sensations of blue whereas (b) those with M centers serve sensations of red. And (c) ON-midgets with S cone OFF feed-forward input and L cones centers serve yellow whereas (d) those ON-midgets with S cone OFF feed-forward input and M centers are responsible for green sensations.

Although there are consistent reports in the literature of L/M opponent cells with S cone input—which we propose are partially the result of subset of midget ganglion/P cells receiving feedforward from HII horizontal cells—and these have been observed at different levels in the early visual pathway in the retina (de Monasterio, Gouras, & Tolhurst, 1975; de Monasterio, 1979), LGN (Derrington et al., 1984; Tailby et al., 2008; Valberg, Lee, & Tigwell, 1986), and primary visual cortex (Conway, 2001; Horwitz, Chichilnisky, & Albright, 2007), experimental evidence for functional input from S-cones to midget ganglion cells has been mixed. For example, Sun, Smithson, Zaidi, and Lee (2006a, 2006b) reported functional input from S cones to be undetectable in recordings from central macaque monkey retina. Part of the reason for the seeming discrepancies concerning S-cone input to midget ganglion cells may have to do with the nature of the proposed input. There can be no doubt that a large majority of L/M opponent cells in the retina and LGN have no measureable contribution from S-cones. This is because midget bipolar cells largely avoid contacting S-cones (Martin & Lee, 2014). Thus, surveys of midget ganglion cells consistently find that most have no functional S cone input.

What was not appreciated previously was the possibility of a second synaptic pathway via HII horizontal cell feedforward from S-cones to a small subset of midget bipolar cells (Puller, Haverkamp, Neitz, & Neitz, 2014; Puller, Manookin, Neitz, & Neitz, 2014). This could account for why midget ganglion cells with S-cone input occur in much smaller numbers and with much weaker S cone input than the small bistratified cells (Martin & Lee, 2014). The S cone input necessary to produce cells with spectral sensitivities proposed here is a modest 10%–20% as has been consistently found in a small fraction of subcortical parvocellar neurons. Further, the chromatic channels are known to have lower spatial resolution than the achromatic pathway (Mullen, 1985), which would be consistent with most parvocellular neurons serving high resolution black and white percepts, whereas only a small population of midget ganglion cells are devoted to signaling hue (Schmidt et al., 2014). To be clear, we are proposing that neurons responsible for sensations of black/white are first separated from those serving hue sensations at the level of the retina. Thus, the majority of midget ganglion mediate white-back sensations while they, at the same time, carry L/M opponent information.

We further emphasize that in this scenario, the L versus M spectral opponency carried by most parvocellular neurons still can serve detection of equiluminant red-green stimuli in experiments like those of Krauskopf, Williams, and Heeley (1982), explaining why such studies tend to reveal mechanisms tuned to the “cardinal direction” in color space along the L-M axis. In contrast, according to our hypothesis, paradigms that require subjects to describe the hue of stimuli only engage the small population of midget ganglion cells that receive S cone input representing the noncardinal “Hering primaries.” Whereas noncardinal representations of color are often believed to have a cortical origin, some groups, for example Bosten, Beer, and MacLeod (2015) and Danilova and Mollon (2012a, 2012b, 2014), have considered the possibility that retinal ganglion cells tuned to a noncardinal axis may underlie behavior under some stimulus conditions.

The midget ganglion model used here was constructed as described previously, with the exception that the S cone gain was slightly different than reported earlier (Schmidt et al., 2014) in which an equal quantum spectrum was employed (however, an equal energy spectrum was reported in error in the description). Changing from a quantal to energy spectrum does not change the prior or current results in a meaningful way except that the S cone weight is now set to one. L and M center ganglion cells (LC(λ) and MC(λ)) were modeled as

A family of ganglion cells was then constructed as m_frac and was iteratively changed from 0 to 1 in steps of 0.01. For each model cell, α was adjusted such that the response integrated to zero over the visible spectrum. Finally, RG and BY channels were created from L and M center ganglion cells by weighting each cell by its probability of occurrence and summing:

The probability of occurring (P(LC_m|m) and P(MC_m|m)) was determined by a binomial distribution parameterized by the assumed %M value, m. Previously, we have shown that this circuit predicts the mean spectral locations and variances of the unique hues reported in the prior literature (Schmidt et al., 2014).

Under this neural architecture, unique yellow is largely unchanged across a wide range of M:L ratios (Figure 1D). Unique green, on the other hand, is predicted to shift from 504 nm to 541 nm for retinas with 15 %M to retinas with 55 %M. The large shifts in unique green that are predicted to accompany changes in %M are the result of the characteristics of an opponent site in outer retina for the BY system and the large separation (of about 110 nm) in spectral peak of the S and M cones. According to this theory, +S-cone input is added to the center of a small subset of OFF-midget bipolar cells that have direct input from L cones. This produces a +S-L center. The excitatory input from M cones in the surround is the origin of the M cone input to blueness, and this predicts a large effect of M:L ratio on unique green. For example, in retinas with a high proportion of L cones, many midget ganglion cells with L cone centers will have all L cone surrounds; those with S cone feed-forward input are BY cells with +S-L opponency. The null point of +S-L cells is short-shifted, and the more cells like this a person has, the shorter his unique green. In contrast, in more M cone dominated retinas, many midget ganglion cells with L cone centers will have pure M cone surrounds producing more M biased (S+M) – L opponency contributing to longer shifted unique green.

Results

Hurvich and Jameson (1957) famously formulated theoretical equations assuming S versus L+M inputs to opponent processes that could reasonably fit measures of chromatic-opponent response functions from a hue cancellation task. Their work provided quantitative empirical support for multiprocess theories over the Helmholtzian view that separate neural pathways carrying signals from three cone types could directly account for color appearance. Here we examined the hypothesis that variation in unique green is the result of variation in cone ratio, and that differences in the underlying circuitry for BY and RG color vision are responsible for the differences in the variability of unique green and unique yellow. Since wide variability in unique green with cone ratio is predicted for opponent models, that difference S+M versus L (Figure 1B through D) but not the S versus M+L originally assumed by Hurvich and Jameson (Figure 1A), we first reexamined how well each model could account for chromatic response functions using modern measurements of the cone spectra derived from physiological measurements that were individualized for each volunteer from genetic data.

M cones contribute to blueness

We studied the appearance of monochromatic test lights in the middle wavelength portion of the visible spectrum in three subjects using a hue cancellation procedure (Hurvich & Jameson, 1957; Werner & Wooten, 1979). Three participants (U2, U3, and U12 in Appendix 1) were instructed to change the intensity of a canceling light until the appearance of blue and yellow was extinguished from the mixture. The intensity required to cancel the test light is plotted in Figure 2. In this portion of the spectrum the data behave monotonically, with decreasing blueness as the wavelength of the test light increases until reaching a null point after which yellow is increasingly sensed. We then fit the four models (introduced above) to the observed data.

Figure 2.

Hue cancellation reveals M cones contribute to blue. Three subjects made hue cancellation settings for wavelengths between 484 and 582 nm at a luminance of 200 Td. The relative intensity of the cancellation necessary to eliminate the sensation of blue and yellow is plotted for each subject. Four models of BY opponency were fit to each subject's data. The gray dashed lines demonstrate the best-fit LGN (S – (L+M)) system. Black dashed lines represent the De Valois (S+M) – L mechanism, whereas the solid colored lines are an (S+M) – L model with a normalization process (hybrid and midget models produce identical fits). In each case, the BY system is best fit with a system summing S and M cones and incorporating a normalization mechanism to the mean environmental distribution.

The gray dashed lines in Figure 2 represent the LGN model (S – (L+M)), which reflects cone inputs to small bistratified cells and the original mechanism proposed by Hurvich and Jameson (Figure 1A). In all three subjects, this model was unable to satisfactorily describe the data, corroborating prior findings under similar experimental conditions (Werner & Wooten, 1979). In comparison, the three channels carrying S+M signals opposed to L were more effective in capturing the features of our data. The De Valois model (black dashed line) was least successful of the three S+M versus L models. In the absence of a normalizing mechanism, the De Valois model does not integrate to zero, leaving the yellow side of their function with substantially more weight than the blue side, thus producing a poor fit to the blue data. The hybrid and midget ganglion cell models (both represented by the solid blue and yellow lines) produce identical fits to the data and were the most successful at describing our findings. The root mean squared errors (RMS) for each model are recorded in Table 1.

Table 1.

Model fits to hue cancellation data for three subjects. Notes: Each value is the RMS error of the model to the hue cancellation data.

A second interesting feature of our cancellation data is the prediction produced by each model for the relative number of M cones in each volunteer's retina. The LGN model predicted increasing %M would produce shorter unique greens (Figure 1A right column), which caused it to erroneously estimate all three of our subjects have 0 %M. In contrast, the three S+M versus L systems anticipate longer unique greens with increasing %M over much of the normal range of values observed in the population. However, the specifics of the relationship between the BY function and %M differ between these three models. In the three subjects studied here, the midget ganglion cell model predicted values of %M closer to the values measured with our ERG and genetic technique than the two cortical models (Table 2).

Table 2.

Estimated %M from ERG versus fitted models.

The relative number of M cones is correlated with unique green

The observation that M:L ratio estimated from fitting theoretical models roughly follows the objectively measured values of M:L ratio motivated us to further test whether the strength of M cone input to blue might be correlated with the relative number of M cones in a larger sample. Due to the difficulty in obtaining hue cancellation data in a large number of subjects, we measured only unique green and yellow in the larger sample.

The spectral locations of unique green and yellow in our subjects have been plotted as a function of the subject's relative number of M cones in Figure 3. As observed earlier (J. Neitz et al., 2002), unique yellow (μ = 574.7, σ = 3.7 nm), a spectral neutral point of the RG system, was not significantly correlated (r² = 0.156, p = 0.162) with the relative number of M and L cones. In striking contrast, the neutral point of the BY system, green (μ = 525.7, σ = 11.5 nm), was significantly correlated with %M (r² = 0.51, p = 0.004). These observations were well fit by our previously published predictions (solid lines) that are based upon an S+M versus L BY hue mechanism and an S+L versus M RG hue mechanism both incorporating cone ratio and an environmental normalization process (Schmidt et al., 2014). Increases in the wavelength of unique green (the null points in Figure 1C) can be explained by an increase in the strength of M cone input to blueness: Longer unique greens suggest a stronger input from M cones to a S+M versus L hue mechanism (Figure 1C). At the same time, increases in the strength of M cones to the sensation of green produces little change in the spectral location of unique yellow for a S+L versus M RG hue mechanism.

Figure 3.

The spectral locus of unique green is correlated with cone ratio as predicted by the midget ganglion cell model. The spectral position of unique green was determined for fourteen subjects. The unique green setting for each subject is represented by a green symbol. Filled circles are subjects with L cone λ_max = 559 nm; squares are subjects with L cone λ_max = 557.25 nm and triangles are subjects with L cone λ_max = 555.5 nm. The abscissa denotes %M (M / (L + M) × 100). For comparison, unique yellow settings are shown in yellow for the same subjects. Filled circles, squares and triangles represent L cone λ_max as above. The continuous lines are the predictions of the model that most accurately predicted our subjects' unique hue results which was the midget ganglion model (Figure 1D). The histograms on the right show the distribution of unique hue settings. Scale bar denotes number of subjects.

The search for biological correlates to explain hue perception has a long history (Kuehni, 2004) and individual differences in iris lightness, and macular pigment optical density (MPOD) have been weakly correlated with unique green (Jordan & Mollon, 1995; Welbourne et al., 2013). To assess the potentially confounding impact of these factors on our analysis, we also measured MPOD and iris lightness in our subjects (full data reported in Appendix 1). We found lighter irises had shorter values for unique green, similar to the previous studies. A linear regression indicated that this relationship was statistically significant for unique green (r² = 0.319, p = 0.035), but not for unique yellow (r² = 0.071, p = 0.358). Next we measured MPOD in our subjects and found qualitatively similar results to those of Welbourne et al., 2013. A multiple regression with MPOD, iris lightness and %M produced a significant prediction of unique green, F(3, 10) = 4.735, p = 0.026, capturing 58.7% of the variance. However, after controlling for all other predictors, %M was the only factor that contributed significantly to this prediction (t = −2.357, p = 0.04). In comparison to the results for green, the same regression against unique yellow captured only 19.5% of the variance and was not a significant predictor, F(3, 10) = 0.807, p = 0.518, of spectral location. Together these results support the hypothesis that M cones act synergistically with S cones to produce blue sensations and that the magnitude of L and M cone signals in opponent circuitry is set by their relative numerosity.

Predictions for each subject were generated for the four models of color appearance studied. As expected, the LGN model did not produce accurate predictions for any of our subjects (compare Figure 1A and Figure 3) and, thus, was excluded from further analysis. The RMS error for the three remaining models is shown in Table 3. For both unique yellow and green, the midget ganglion model produced the most accurate predictions for our 14 subjects. Pair-wise t tests revealed that the midget ganglion model significantly outperformed the De Valois model, but the performance of the midget ganglion model did not reach significance against the hybrid model (Table 3). Finally, it is worth noting that the mean SEM of unique green and yellow settings (Appendix 1) across the 14 subjects were 1.8 and 1.0 nm, respectively, which represents the limit on how well any model can predict our color appearance data.

Table 3.

Model fits to unique hue data. Notes: Mean RMS error and standard deviation (SD) are reported in nm.

The neural locus of opponency

The above findings do not conclusively resolve the neural locus at which S cone signals are injected into L-M opponent circuits. To further examine this question, we studied an individual with mutations in the genes encoding mGluR6, which disrupts the signal transmission between cones and ON-bipolar cells. Individuals with mGluR6 mutations experience complete Congenital Stationary Nightblindness (CSNB1). This disorder is clinically characterized by difficulty seeing at night and the absence of a b-wave in ERG recordings, as a result of a loss of direct signaling between cones and ON-bipolar cells (Dryja et al., 2005). Therefore, S-cone input to S-ON bipolar cells (Mariani, 1984) responsible for the traditional S − (L+M), mediated opponent pathway in these individuals is disrupted. The circuit diagram in Figure 4A demonstrates why a disruption of the S-cone input to small bistratified cells should dramatically alter the appearance of colors if the theory that noncardinal “Hering primaries” originate in cortex is correct. S-opponent cells are hypothesized to modulate the L- and M-opponent cells at the cortical level in order to produce the hue pathways and, therefore, disruption of the small bistratified circuit should seriously alter color perception.

Figure 4.

Absence of ON bipolar signal does not disrupt color appearance. (A) A mutation to mGluR6 predicts disrupted hue perception in a circuit that requires S-opponent ON ganglion cells (red lines) to modulate L- and M-opponent parvocellular neurons. (B) Unique hue results from a subject with CSNB1 (crosses) are plotted against mean values for normal subjects in the CIE diagram (circles represent mean, solid colored lines denote 1 standard deviation) (Kuehni, 2004). Cardinal axes are drawn following the conventions of Derrington et al. (1984). Note the unique hues do not align with the cardinal axes.

Previous studies have reported normal color discrimination in patients with CSNB1 (Bijveld et al., 2013; Dryja et al., 2005). Here we extend these findings and confirm normal color appearance behavior in a CSNB1 patient (Figure 1B). Like many CSNB subjects that we have interacted with, this subject had no discernable deficits in photopic vision as the result of the loss of cone input to ON bipolar cells. He had normal visual acuity and easily passed a battery of conventional color vision tests. The subject reported no difficulty completing the unique hue task. Comparison of his unique yellow and green to color normal subjects demonstrated the spectral lights chosen by our CSNB1 subject are indistinguishable from color normal subjects. His unique blue (488.1 nm) is also well within the range of normal subjects: 461–495 nm (Kuehni, 2004), falling 1.5 standard deviation from the population mean, 477.9 nm, 6.8 nm. These results confirm the persistence of undisrupted hue perception in CSNB1 individuals despite the absence of S-cone input to their small bistratified cells. This demonstrates that S-cone input to small bistratified cells is not required for normal hue perception. This finding is consistent with the hypothesis that ganglion cells, other than small bistratified cells, could be responsible for BY hue perception. The remarkably well preserved vision of these individuals could be explained by a direct feedforward pathway from S-cones to midget bipolar cells via HII cells which bypasses the defective cone to ON bipolar synapses.

Discussion

Our findings reveal a significant correlation between the relative number of L and M cones in a subject's retina and the character of sensations associated with middle wavelength lights. This solves a long-standing mystery of the physiological basis of the huge variability in color vision in the middle-wavelength part of the spectrum. A greater percentage of M cones bias the sensation of the purest green towards longer wavelengths. To account for these results, we compared the predictions of four potential neural circuits. Surprisingly, a circuit based in a small subset of midget ganglion cells that receive S cone input (Schmidt, Neitz, & Neitz, 2014) predicted the character of human hue sensations as well, or better, than cortical models, challenging the prevailing notion that the unique hues are constructed centrally.

Two conclusions are most clear from these experiments: (a) Human blue/yellow hue perception is best explained by neural circuits in which S+M cone inputs are differenced from L. The three models that incorporated an S+M versus L circuit made significantly more accurate predictions than hue perception based directly on an S – (M+L) circuitry. (b) Normalization to the mean spectral environment is required to explain hue perception. This seems to be an important general principal of opponent systems. Such systems operate efficiently when the relative weighting of the opposing inputs are adjusted to null in response to the average stimulus. Thus, only deviations away from the mean are transmitted, no energy is wasted in signaling the mean, and the operating range of the detector is perfectly aligned with the behaviorally relevant distribution of energies in the environment.

The remaining issue investigated here is the underlying biological substrate for the S+M versus L circuity required to explain BY color vision. Below we further consider the results presented here as well as additional results from the literature and suggest further tests to distinguish the possible underlying neural mechanisms.

The blue/yellow system and M:L ratio

The unique hues are thought to represent the null points of the two dichromatic color systems (Hurvich & Jameson, 1957) and thus serve as an abbreviated method of comparing color appearance mechanisms between individuals. The most surprising result from this series of experiments was the correlation observed between unique green and M:L cone ratio (Figure 3). After correcting for other factors known to influence the perception of unique green, iris lightness, and macular pigment optical density, we found that M:L cone ratio was still statistically predictive of the wavelength selected as purely green. In the same group of subjects, unique yellow was not dependent upon cone ratio, confirming previous findings (Jordan & Mollon, 1997; J. Neitz et al., 2002).

To account for these results, we examined the predictions for each of our subjects made by four neurobiologically motivated theories of color appearance that incorporate M:L cone ratio into their cone weights. All three models that opposed S+M against L signals captured the general trends in our data; over the range of %M values in our volunteers (23–53 %M), they all predicted relatively unchanging wavelengths of unique yellow and a steady decline in the wavelength of unique green with decreasing %M values. When the predictions of each model were directly compared, the midget ganglion model outperformed the two cortical models in both unique yellow and green predictions (Table 3). The midget ganglion model forms a BY circuit through an OFF midget pathway. In the central retina where midget centers draw excitatory input from a single cone, the hypothetical BY circuit would contain a pure OFF L cone center with direct S+ feedforward input differenced from a surround in which M cone excitation is opposed the L center and synergizes with the S+. Adding additional M cones increases the relative M cone contribution to the S+M versus L mechanism lengthening the wavelength of unique green. However, in the cortical models, the third stage of processing, which sums L/M opponent cells with S – (L+M) cells, the effect of adding more M cones is partially canceled out. Therefore, both cortical models, especially the original De Valois model, underestimate the effect of %M on unique green that we observed in our subjects. Similarly, in the case of unique yellow, decreasing the strength of M cones below 25% leads to rapidly increasing unique yellow settings in the two cortical models (Figure 1B and C). This is a clear failure of the cortical models because the unique yellows of individuals with cone ratios highly biased in the L direction do not differ as predicted (Jordan & Mollon, 1997; J. Neitz et al., 2002). In comparison, the midget ganglion theory, due largely to its retinal normalization mechanism, predicts largely unchanged values for unique yellow in this low %M region. Thus, considering the larger variability in unique green and the better fit for low M:L ratios, more accurate predictions from the midget model lends further support to the interpretation that the unique hues could plausibly be based on a small subset of midget ganglion cells, with no additional modification by postretinal processing.

ON pathway defects

Most modern theories of color vision circuitry propose that the small bistratified ganglion cell is the retinal basis for BY color vision. This includes the De Valois and hybrid models, which postulate a cortical stage combining the small bistratified outputs with L/M parvocellular neurons. Therefore, complete absence of S cone inputs to the small bistratified cells, as happens in people with mutations that interrupt signaling between S-cones and S-ON bipolar cells should have a profound effect upon on BY color vision. Contrary to this standard view, the midget ganglion model hypothesizes that blue hue perception is mediated by OFF midget bipolar cells with the +M component coming from the surround and the +S component coming from GABA mediated feedforward input and is unaffected by cone to ON bipolar signaling that is disabled in people with mGluR6 defects (Dryja et al., 2005; J. Neitz & Neitz, 2011; Schmidt et al., 2014). Previous reports have indicated that mGluR6 mutations are not associated with abnormalities in color vision as assessed with standard clinical tests (Bijveld et al., 2013; Dryja et al., 2005). Here we extend these findings and confirm normal color appearance in a subject with complete congenital stationary night blindness. The preservation of normal unique hue settings may be explainable by the retinal model and presents an additional challenge to cortical models.

Achromatic sensations

The separation of achromatic and chromatic sensations is a major challenge for the visual system. In the retina, the majority of midget ganglion cells carry both RG chromatic and achromatic spatial information. The prevailing theories of color vision propose that the midget system, with its intermingled chromatic and achromatic signals, performs “double duty,” serving both high-resolution spatial vision and chromatic vision. These theories then presume that the two channels are de-multiplexed in cortex through presently poorly defined circuitry (Boycott & Wässle, 1999; De Valois & De Valois, 1993; Ingling & Martinez, 1983; Lee, Martin, & Grünert, 2010; Shapley & Hawken, 2011; Shapley & Perry, 1986).

The midget ganglion theory posits L/M opponent midget ganglion cells mediate achromatic sensations, black, and white. The origins of this idea go back to Wiesel and Hubel (1966) who argued that the center-surround L/M spectrally opponent cells in the LGN are as good a candidate as the magnocellular neurons for the “mediation of black-white contrast mechanisms.” Although the idea that L/M opponent cells do double-duty has often been proposed, it has also been agued previously that they only mediate achromatic sensations (Calkins & Sterling, 1999; Rodieck, 1991). Thus, the midget ganglion theory argues that the vast majority of midget cells, probably >90%, only serve black and white sensations. Together these are the basis for high acuity achromatic vision and, in addition, the L/M opponent cells are capable of mediating detection of equiluminant red-green stimuli. A smaller subset of midget ganglion cells with HII feedforward S cone input mediate our conscious hue sensations.

Future predictions

It is now technically feasible, using adaptive optics, to confine a targeted spot of light to an individual cone (Harmening, Tuten, Roorda, & Sincich, 2014) and this can be done in retinas in which cones have been classified as L, M, and S. Contrary to double-duty models, the midget ganglion model predicts that a majority of L and M cones, even those surrounded by cones of the opposite type, will give rise to sensations of white, not color. Moreover, there should be clusters of cones that are nearly always associated with strong chromatic percepts. Finally, according to the theory, M cones in the surrounds of midgets with L cone centers serve blueness and M cones centers serve green sensations. Thus, it should be possible to illicit blue sensations or green sensations from a single cone depending on the background conditions.

Conclusion

Many aspects of human color vision cannot be explained by standard theories. These include a huge variability in unique green compared to unique yellow. Here we show that the majority of the variability in unique green is caused by individual differences in cone ratio and that this observation and many others can be explained by a theory in which separate subtypes of midget ganglion cells mediate the sensations of black, white, blue, yellow, red, and green respectively.

Appendix 1

Table A1.

Color appearance, genetic and optical data. Notes: L cone sensitivities were based on the sequence of each subjects L pigment. Amino acids reported: exon 2—65, 111, 116; exon 3—153, 171, 174, 180; and exon 4—230, 233, 236. For females with two L sequences, the peak sensitivity of both pigments was averaged to compute the reported peak L cone sensitivity. All subjects had M cone peak values of 530 nm. All subjects identified as Caucasian except subjects U12 and U13, who were of Asian ethnicity.