Human trichromacy revisited

Abstract

The presence of a photopigment (melanopsin) within certain retinal ganglion cells was a surprising and significant discovery. This pigment is routinely described as “nonvisual” to highlight its signaling role in pupil dilation and circadian rhythms. Here we asked whether light absorbed by melanopsin can be seen by healthy human subjects. To answer this requires delivering intense (above rod saturation), well-controlled lights using four independent primaries. We collected detection thresholds to many four-primary stimuli. Threshold measurements in the fovea are explained by trichromatic theory, with no need to invoke a fourth photopigment. In the periphery, where melanopsin is present, threshold measurements deviate from trichromatic theory; at high photopic levels, sensitivity is explained by absorptions in four, not three, photopigment classes. We consider a series of hypotheses to explain the tetrasensitivity at high photopic levels in the human peripheral field. The most likely hypothesis is that in healthy human subjects melanopsin absorptions influence visibility.

Transduction of light energy into neural signals in the primate nervous system was long thought to take place only in the photoreceptor layer of the retina. The presence of a photopigment (melanopsin) within certain retinal ganglion cells (mRGCs) was a surprising and significant discovery (Fig. 1) (1–3). Melanopsin is routinely described as a “nonvisual pigment” (4, 5) perhaps to highlight its role in functions like pupil dilation and circadian rhythms. However, there is no decisive evidence as to whether melanopsin absorptions can be seen by healthy human subjects. Mice born with no rods or cones can perform visual tasks, presumably mediated by a melanopsin-initiated pathway (6). Human subjects with no rods or cones due to retinal disease detect wavelengths of light around the peak of the melanopsin spectral sensitivity (7). And a class of mRGCs in macaques projects to the lateral geniculate nucleus, the thalamic relay to primary visual cortex (3). Finally, it appears that melanopsin-initiated signals influence brightness discrimination, although these results leave open the possibility that subjects fail to perceive signals arising from mRGCs (8). We describe direct tests of the hypothesis that sensitivity depends on absorptions in four receptor classes (tetrasensitivity). This differs from the hypothesis that color appearance is four-dimensional (tetrachromacy).

Fig. 1.

Do melanopsin absorptions contribute to light detection? Schematic illustration of retina in cross-section [modified from Field and Chichilnisky (57)]. In photopic viewing, L-, M-, and S-cone photopigments (red, green, and blue triangles) absorb light. Rhodopsin is bleached in high-intensity light so the rod system becomes saturated and ineffective (indicated by light gray shading). The cone signals are communicated to the output channels of the eye, retinal ganglion cells (RGCs), via a network of bipolar, horizontal, and amacrine cells. Multiple types of RGCs together are thought to represent three neural mechanisms for light detection: L+M, L−M, and S−(L+M). A small population of ganglion cells containing a new photopigment (melanopsin) was recently identified. We refer to these cells as melanopsin-containing retinal ganglion cells (mRGCs) (cyan circle). More recently, melanopsin-expressing cones were identified immunohistochemically in peripheral human retina (38) (cyan triangle). It is unknown how these cones contribute to retinal circuitry (indicated with dotted outline). Because melanopsin absorbs photons across a wide range of light levels, including photopic conditions, it is possible that four color channels contribute to photopic light perception. [Lower reprinted by permission from Macmillan Publishers Ltd: Nature (ref. 3), copyright 2005.]

This paper reports measurements and analyses that estimate the need to postulate a fourth class of photopigment to explain the visibility of lights presented in the healthy human. To analyze whether a fourth photopigment contributes to photopic visibility, it is necessary to deliver well-controlled light signals using at least four independent primaries. We built a display device capable of accurately delivering six independent primary lights (9). Furthermore, this device can deliver very intense light—an order of magnitude above the rod saturation level (10). We examined psychophysical evidence for a perceptible signal from a fourth photopigment at very high photopic levels.

In principle, a contribution from a specific pigment, such as melanopsin, could be assessed by delivering light stimulation that modulates only the melanopsin absorptions while leaving the cone absorptions unchanged. In practice, however, this level of stimulus control is not easily accomplished. The retinal irradiance reaches the cone photopigments and melanopsin only after passing through the cornea, lens, and inert pigments of the eye. Individual variability in the transmission through these structures makes it impossible to specify a light that is absorbed uniquely by melanopsin and not the cones; achieving this control with sufficient precision to convince a skeptical reviewer or ourselves is unlikely.

Hence, we used a different approach. We measured contrast thresholds along many directions in the four-dimensional space specified by the four primary lights. If visibility depends on the photons captured by three photopigments, there must be a combination of increments and decrements of the four primaries that is invisible. Further, if only cone photopigment absorptions contribute to detection—and melanopsin absorptions do not—then the invisible stimulus will be the combination of primaries that fails to produce absorptions in the cone photopigments; we refer to this as the cone-silent stimulus.

We analyze the threshold data to determine whether there is a plausible set of lens and pigment properties that can explain the threshold data and that depends on only absorptions by the three cone photopigments. We collected detection thresholds to many four-primary stimuli in two sets of experiments. In one set of experiments the stimuli were presented in the central fovea, and in a second set the stimuli were presented in the periphery. There are no retinal ganglion cells in the central fovea (11), so we expect that (a) the data will be explained by a model based on three photopigments and (b) the invisible four-primary stimulus will be cone silent. These measurements confirmed that the detection threshold in human fovea is explained accurately by Thomas Young’s trichromatic theory.

We find that the corresponding measurements in human peripheral retina, where there is melanopsin photopigment, deviate from the classic trichromatic theory. In the periphery, at high photopic levels, human sensitivity is not accurately explained by absorptions in only three types of cone photopigments. Threshold sensitivity in the visual periphery depends on absorptions in at least four photopigments (tetrasensitivity).

Color Threshold Theory

For about 100 y color threshold data have been modeled using line-element theory (12). The original line-element theory assumed that a threshold stimulus described as a change in cone absorptions from the background level, (ΔL, ΔM, ΔS), will satisfy the formula

where w indicates a scale factor of each cone class. Over the years, this original idea was generalized from weighted cone absorptions to weighted color-mechanism responses, reflecting an increasing understanding about the role of opponent colors (13–16). Over the last 40 y these quadratic line-element models have usually been implemented by transforming the cone absorptions into three theoretical opponent color mechanisms that are weighted sums of the cone absorption changes (17, 18):

The linear transformation from cone signals to mechanism responses is accepted as a good approximation for small, threshold-level signals in most threshold measurement conditions. In this formulation the line-element model becomes

This mathematical formulation has been adapted extensively in vision science, where it is commonly described using the term “energy model” (19, 20).

There are experimental conditions in which this model fails [e.g., Stromeyer et al. (21)]. We tested the model quantitatively for the conditions of our experiment and show that for our measurement conditions the model fits the data accurately (Fig. 2 and Fig. S1).

Fig. 2.

Quadratic model fitting to foveal measurements. (A) Threshold measurements and ellipses estimated by a quadratic model for subject 1 (S1). We fitted the sampled measurements, using a quadratic model with three mechanisms as defined by the row size of the opponent-mechanism matrix, V_3×4. The model (black solid line) fits the measurements well (black solid circles). Measurement points are shown only if they lie near the displayed plane (cosine of the angle between the point and the plane is more than 0.95). (Upper) Planes including the cone-silent axis (Z: zero-cone) and one of the L-, M-, or S-cone pigment axes. The photopigment densities are assumed to match the standard color observer (main text). Note that a subject could detect a cone-silent stimulus at 2% stimulus modulation. (Lower) Planes consisting of L-, M-, and S-cone pigments axes. The ellipses on the cone-pigment planes are in good agreement with the color-science literature. The threshold to detect L+M light is much higher than the L-M threshold and the threshold in the S-direction is lower than the L+M threshold. (B) Threshold ellipses and measurements estimated in subject 2 (S2). Thresholds are generally lower than those of S1. However, the shapes of the ellipses are similar to those of S1.

Trichromatic Theory

The classic formulation of trichromatic theory is the assertion that light is encoded exclusively by absorptions in three cone photopigments (22, 23). Color threshold theory makes the further assumption that cone signals are recombined into three opponent mechanisms (13–16). The second assertion (three opponent mechanisms) could hold even if the first (three photopigments) does not; signals from multiple photopigments can be combined into three opponent mechanisms (24). Hence in the following analyses we test the assertions separately. First, we assess the number of detection mechanisms, and second, we assess whether the data are consistent with absorptions only in the three cones.

Results

We first examine the trichromatic theory predictions for detection thresholds measured in the fovea. We then describe the corresponding measurements and analyses in the periphery. These analyses focus on the ability to detect relatively slow (pulse) test stimuli. In the final set of measurements, we describe the sensitivities of the neural mechanisms, using high temporal frequency test stimuli.

Foveal Sensitivity Is Explained by Three Opponent Mechanisms.

The quadratic model fits based on three opponent mechanisms are shown in Fig. 2 A and B. The thresholds are plotted in planar cross-sections through the four stimulus dimensions, corresponding to the three standard color-observer cone directions (L, M, and S), and the cone-silent direction (Z). Three planes include the cone-silent direction [(L, Z), (M, Z), and (S, Z)], and these are shown in Fig. 2 A and B, Upper for each subject. An additional three planes are shown in the cone planes [(L, M), (L, S), and (M, S)]. The quadratic model with three opponent mechanisms fits the threshold data well. We show that a model with a fourth mechanism does not significantly improve the fit (based on cross-validation) in the summary of the measurements at the end of Results.

Trichromatic Theory Explains Foveal Sensitivity.

The data in Fig. 2 deviate from the standard trichromatic theory because the observers both have some sensitivity to test lights in the cone-silent (Z) direction. However, the conditions of this experiment differed from the conditions used to define the standard color observer (25, 26). In particular, the mean illumination is significantly higher and somewhat bluish. We examined the parameters of the standard color observer to understand whether it is possible to predict the foveal data assuming that sensitivity is mediated entirely by cone absorptions.

First, we estimated the spectral power distribution of the invisible stimulus by fitting the threshold data multiple times, using a bootstrap procedure. Each bootstrap sample yields an invisible spectral power distribution and the range of these estimates is shown in Fig. 3A. We calculated the expected difference from the background of cone absorptions for the invisible stimulus, using the standard color-observer parameters, namely a macular pigment density of 0.28 and an (L, M, S) cone photopigment optical density of (0.5, 0.5, 0.4). For the standard observer model, the invisible stimulus produces a significant change in cone absorptions. These are shown as open circles in Fig. 3B.

Fig. 3.

Trichromatic theory can explain foveal measurements. A three-mechanism model (Fig. 2) must have an invisible direction in a four-primary display. (A) Spectral power distributions of the invisible stimuli at the fovea according to the model fits. The spectral patterns are similar for two subjects (S1, black outline; S2, gray). The shaded areas show the 80% confidence interval based on a bootstrap procedure: We fitted the model 1,000 times, each fit omitting 10% of the data selected at random. (B) Responses to the foveal stimulus in the invisible direction at 10% modulations are shown in LMS space for S1 according to two models of LMS spectral sensitivity: the standard color observer (gray) and models fit to the individual to account for pigment density in the lens, macula, and cone outer segments (black outline; see Fig. S2 for details). Assuming the standard color observer, the cloud of bootstrapped LMS responses is far from the origin, indicating that (according to this model) a 10% modulation of the invisible stimuli evokes about a 2% response in each of the L-, M-, and S-cones. However, after correction for the individual cone pigment densities, the LMS responses to the invisible stimuli for S1 lie near the origin, indicating that the invisible light is also the cone-silent light, as predicted by trichromatic theory. Histograms on the cone axes show the distribution of cone responses to the invisible stimuli according to the standard observer model (gray) or the individual observer (black outline). (C) Distribution of bootstrapped LMS responses to the invisible foveal stimuli in two subjects. Histograms show distances from the origin in 3D LMS space. In both S1 and S2, LMS responses are far from the origin when fitted using the standard observer’s pigment densities (median of S1 and S2 LMS responses: 4.5% and 3.3%, respectively). After pigment density correction, the invisible stimuli are also the cone-silent stimuli. (D) Threshold ellipses after pigment density correction. The cone-silent direction is invisible for both subjects, indicated by the holes in the Z-direction. Threshold ellipses in the six panels are similar for the two subjects.

Next, we reanalyzed the data, adjusting the cone, macular, and lens pigment densities. In the presence of an intense background, cone photopigment optical density is reduced and the spectral absorption can change (27). Hence, it is necessary to recompute the cone photopigment isomerizations, using pigment properties that are specific to the conditions and the observer. We also allowed the lens and macular pigment densities to vary within a plausible physiological range (Fig. S2). The full range of bootstrap estimates of change in cone absorptions for the adjusted values, along with an additional experiment to estimate subject (S)1’s macular pigment density (0.62), is described in SI Methods (Fig. S3).

With these corrected pigment density values, the invisible stimulus is aligned with the cone-silent direction (Fig. 3B, gray solid circles). Distances from the origin to estimated L-, M-, S-cone (LMS) responses are shown as histograms of bootstraps in Fig. 3C for both S1 and S2. The predicted visual thresholds of the trichromatic theory with the corrected standard observer parameters are shown in Fig. 3D. The foveal data are in excellent quantitative agreement with the trichromatic theory, and the invisible stimulus is aligned with the cone-silent direction.

Trichromacy Is Inconsistent with Peripheral Sensitivity.

Fig. 4 shows the fit based on three opponent mechanisms (gray ellipses) to the four-primary pulse stimulus threshold data when the stimuli are in the periphery. These data are plotted with respect to the adjusted cone parameters derived from the foveal measurements. Many of the measured points fall just outside of the plane. To show a more complete representation of the data, we plot the thresholds in standard observer color space in Fig. S1 (25, 26).

Fig. 4.

Trichromacy and tetrachromacy fitted to peripheral measurements. (A and B) Peripheral threshold measurements and ellipsoid fits after pigment correction in S1 (A) and S2 (B). A and B are drawn as in Fig. 2 A and B. Because the data are plotted after pigment correction, many of the data points do not lie exactly in any of the six planes. Hence, the number of visible dots in the six planes is lower than in Fig. 2. In fact, no points appear in three of the planes shown. Data were fitted using quadratic models with either three mechanisms (“trichromacy”) or four mechanisms (“tetrachromacy”), as defined by the row size of the opponent-mechanism matrix, V (Methods).

In contrast to the trichromatic fits in the fovea, the invisible directions according to the trichromatic fits in the periphery are not in the cone-silent (Z) direction. For S2 the invisible direction can be seen as a hole in the L vs. S and M vs. S planes; for S1 the invisible direction cannot be seen in the six cardinal planes. Also unlike the trichromacy model fits in the fovea, the fits in the periphery differ between the two subjects. For example, predicted thresholds near the S-cone direction are very high for S2, inconsistent with the data and the literature.

The tetrachromacy ellipses (black ellipses), unlike the trichromacy ellipses, are similar across the three subjects in all six planes. For the two subjects shown (S1 and S2) as well as for S3 (Fig. S4), the tetrachromacy model fits are better than the trichromacy model fits as assessed by the root mean-square error in cross-validated data.

A Fourth Photopigment Is Required to Explain Peripheral Sensitivity.

Next we asked whether by further modifications to the standard color observer parameters it is possible to explain the peripheral thresholds. We searched for cone photopigment and inert pigment parameters that would align the invisible direction predicted by the best-fitting three-pigment model with the cone-silent direction. For the peripheral data, unlike the foveal data, we could not find a set of pigment properties that accurately predicted the thresholds (Fig. 5). The vector length of the estimated cone absorptions does not decrease when we reduce the photopigment density, as would be expected in the periphery (Fig. 5C) (28, 29). We performed a systematic search and could find no plausible pigment parameters to align the predicted invisible stimulus from the model with the cone-silent direction. The search for photopigment optical density ranged from 0.01 to 0.5; the macular pigment was allowed to range from 0 to 0.1; the lens pigment density ranged from 0.5 to 1.5 (Fig. S3). Using a free search with no limitation on the parameters, the best fits often included impossible (negative) densities. Hence, we could not find a satisfactory model fit that excludes contributions from a fourth photopigment.

Fig. 5.

Peripheral measurements deviate from the classic trichromatic theory. (A) Spectral power distributions of invisible stimuli in the periphery estimated by the three-mechanism quadratic model. Thickness of the line indicates the 80% confidence interval as in Fig. 3A. (B) Estimated S1 cone responses to the peripheral invisible stimuli at 10% modulation in 3D LMS space. Assuming the standard color observer, the estimated invisible stimuli evoked large LMS responses. Similarly, assuming individual lens and optical density pigment densities based on the foveal experiments (“reduced pigment”) and no macular pigment, the estimated invisible stimuli also evoked robust LMS responses. Even if pigment densities are adjusted freely to include biologically implausible density levels, the invisible stimuli were absorbed by LMS cones at a high enough level to evoke light detection (not shown). Histograms on each cone axis show a distribution of cone responses to the invisible stimuli according to each of the three pigment models. LMS cone responses in S2 and S3 are shown in Fig. S8. (C) Distributions of estimated LMS responses to the peripheral invisible stimuli in three subjects. Histograms show distances from the origin to estimated responses to the stimulus in 3D LMS space. In all three subjects, pigment reduction does not reduce the estimated LMS response to zero. (D) Model error of the trichromatic theory fitted after pigment correction for foveal and peripheral experiments. The data are cross-validated: Models were fitted to a subset of data and predicted and measured thresholds were calculated for the left-out data. The “model error index” was calculated in each model by dividing the difference between the measured and predicted threshold by the predicted threshold. Medians of the model error index are binned by the projection of each color direction on to the cone-silent (Z) axis. Note that the model error index increases with the length of Z in the periphery (but not the fovea) in all subjects. This indicates that predicted thresholds to cone-silent stimuli are systematically higher than measured thresholds in the periphery, not the fovea.

Next, we examined the specific pattern of deviations of the trichromatic theory. If a fourth photopigment contributes significantly to peripheral sensitivity, the largest deviations from the model should occur when the stimulus contains a relatively large modulation in the cone-silent direction and relatively little modulation of the cone photopigment absorptions. We can operationalize this calculation as follows. Consider each unit length four-dimensional vector (L, M, S, Z). For these vectors, the Z-value measures the balance between the cones and a putative fourth photopigment. When the Z-value is large, the balance is weighted toward the fourth photopigment.

We analyzed the deviations from the trichromatic theory predictions for both the foveal and the peripheral data (Fig. 5D). For the foveal data, there is no systematic relationship between the prediction error and the value of Z. For the peripheral data, the error increases systematically with Z. Hence, the trichromatic theory systematically misestimates thresholds for cone-silent stimuli, precisely those stimuli expected to modulate the fourth photopigment strongly.

Thus, we could not find a set of photopigment and inert pigment parameters that align the data with the predicted cone-silent direction. The deviations from the model are systematic, with the largest deviations occurring when the stimuli are presented in a direction that should cause no cone absorptions. Hence, we conclude that the trichromatic model of sensitivity based on three cone photopigments fails in the periphery.

Summary Comparing Foveal and Peripheral Sensitivity Measurements.

The analyses in Fig. 6 summarize the three types of model fits in the fovea (Fig. 6A) and the periphery (Fig. 6B). The models differed in the numbers of mechanisms, as defined by the row size of the opponent-mechanism matrix, V (Methods). The three models are V_2×4 = dichromacy, V_3×4 = trichromacy, and V_4×4 = tetrachromacy. To evaluate the models we performed a cross-validation test. We sampled 70% of the measurements (with replacement) to create a simulated dataset and calculated the predicted thresholds for the data that were left out by the sampling procedure. We repeated this process 10,000 times to obtain a distribution of predictions for each point.

Fig. 6.

Cross-validation analysis of model accuracy. (A) Foveal data: scatter plot of the predicted and observed thresholds for “left-out’” trials in a cross-validation analysis. Shown are fits using trichromacy (Left) or tetrachromacy (Center). The open and solid symbols are predictions for the two subjects (S1, solid circles; S2, shaded triangles). The vertical error bars are 80% confidence intervals of the estimates. The horizontal bar (upper left) is the median confidence interval for the measured thresholds. (Right) Model accuracies are compared in the bar plot (80% confidence intervals on the bars are shown). The tetrachromacy fit is slightly better for both subjects. Comparing trichromacy to tetrachromacy, the root mean-square error (rmse) between observed and predicted thresholds is reduced from 0.060 to 0.050 (S1) and from 0.081 to 0.059 (S2). (B) Peripheral data. The quadratic models based on trichromacy (Left) and tetrachromacy (Center) are shown for S1, S2, and a third subject (S3). Comparing trichromacy to tetrachromacy, the rmse values decrease from 0.106 to 0.045 (S1), from 0.058 to 0.030 (S2), and from 0.061 to 0.044 (S3). Values are otherwise as in A.

The dichromacy model is generally poor in all cases. In the fovea, there is a small difference between trichromacy and tetrachromacy, with no meaningful difference for S1 and a very small difference for S2. In the periphery, the tetrachromatic model provides a better fit for all three subjects.

However, the differences in the predicted thresholds are small: One might not amend the two-century trichromatic theory on the basis of such a small effect alone. The principal reason for amending the theory arises from the additional observation that the best trichromatic model in the periphery predicts a cone-silent direction that is inconsistent with plausible biological estimates of cone photopigments and the inert pigments.

High Temporal Frequency Measurements in the Periphery Are Influenced by Noncone Absorptions.

Sensitivity to high temporal frequency (40 Hz) modulations in the periphery is well explained by a single visual mechanism (Fig. 7); the measured thresholds fall very near a one-dimensional subspace in the four-dimensional space (Fig. S5). Because the flicker data are fitted by a single mechanism, multiple color directions are invisible.

Fig. 7.

High temporal frequency thresholds in the periphery are influenced by cone-silent stimulus. Shown are estimated models at high temporal frequency (40 Hz) after pigment density correction in S1 and S3. Note that there is no hole in the Z-direction: Estimated thresholds in the cone-silent direction in both subjects are ∼10%, compared with no contribution of cone-silent stimulus for light detection in the foveal measurements (Fig. 3D). See Figs. S5 and S9 for further related measurements.

Surprisingly, the cone-silent direction is not invisible. We compared the estimated mechanisms from subjects S1 and S3 in cone coordinates corrected for the viewing conditions. The relative chromatic sensitivity, measured by the orientation of the lines throughout Fig. 7, is similar in these two subjects. The main difference is that S3 has a slightly lower sensitivity (distance of the lines from the origin). The model for S2 is not shown because the data obtained from this subject were insufficient to derive a reliable prediction at 40 Hz. This subject has very low sensitivity to high temporal frequency stimuli.

Again, no adjustment of the photopigment or inert pigment parameters produced a solution in which the cone-silent direction is invisible. We find it very surprising that a fourth photopigment contributes to sensitivity even at high temporal frequencies, and we discuss this result later.

Discussion

There are three main experimental findings. First, the trichromatic theory based on three cone pigments explains the foveal chromatic measurements. The trichromatic fit to the foveal thresholds is quantitatively consistent with two centuries of color science. Second, peripheral threshold measurements are inconsistent with the theory that only cone photopigment absorptions contribute to sensitivity. The measured thresholds to lights in the cone-silent direction are systematically lower than predicted by a trichromatic theory based on only cone photopigments. Third, sensitivity to rapidly flickering lights in the periphery can be explained by a single, linear, neural mechanism. Surprisingly, this mechanism is sensitive to stimuli in the cone-silent direction, that is, to stimuli that do not influence absorptions in the L-, M-, or S-cone photopigments. This finding suggests that a fourth photopigment contributes to the perception of rapidly flickering peripheral stimuli.

The data we present suggest that melanopsin-initiated absorptions can be detected. There are only a few studies that attempt to isolate the effect of melanopsin in healthy humans, and these are inconclusive about the role of melanopsin-initiated signals for visual perception (8). We describe the human measurements next. Then, we describe the neural circuitry data that suggest why melanopsin-initiated signals could influence visual perception. We then consider alternative hypotheses: The fourth photopigment we measure arises from rhodopsin in the rods or other biological features of the retina. Finally, we consider which of the several melanopsin pathways might be the source of the sensitivity described here.

Related Literature.

Behavior and melanopsin in healthy humans.

Tsujimura et al. (30) used a four-primary display to measure human pupil responses. Using the standard color observer, they designed a stimulus that modulates the melanopsin photopigment absorptions but that is invisible to the cone photopigments. They report that subjects do not perceive the melanopsin-isolated stimulus, but that the pupil size modulates in response to the melanopsin absorptions. The inhomogeneity of absorptions arising from the macular pigment distribution requires different spectral power distributions for metameric blacks in the fovea and periphery. In related work, Brown et al. (8) used a very large field, spanning both the fovea and the periphery, and concluded that melanopsin contrast influences brightness perception; using an annular spatial configuration that masked the central 5°, they concluded that melanopsin contrast does not influence color discrimination.

Vienot et al. (31) constructed a display apparatus with seven primaries, enabling them to generate cone-silent stimuli that modulate the rhodopsin and melanopsin pigments in human subjects. Basing their calibrations on the standard color observer from Stockman (25, 26), and using relatively low light levels, they report significant pupil responses in some individuals but not in others. This observation agrees with our results in that corrections for the individual photopigment characteristics of each observer and for specific light levels are probably required to achieve isolation. They make the interesting observation that two lights of equal luminance may produce different pupil apertures and thus different retinal illuminance. This finding may be significant for applications in lighting.

Behavior and melanopsin in a patient.

Zaidi et al. measured pupil sizes and visual awareness in two patients with very limited light perception (7). In one subject with long-standing cone–rod dystrophy and no light perception, a 10-s exposure to a 481-nm wavelength light (1.45 × 10²⁰ photons⋅m²⋅s⁻¹) produced a conscious percept that differed from a zero background. At other wavelengths the same photon flux did not produce a conscious experience of light. The 481-nm light is near the maximum sensitivity of the melanopsin pigment, and the authors conclude that the perception is melanopsin initiated.

The data we report in healthy controls also suggest that conscious percepts arise from melanopsin absorptions. The nature of our test stimuli—relatively brief contrast modulations with respect to a high mean background—is quite different from that used in ref. 7.

Melanopsin circuitry.

On the basis of careful behavioral measurements, several investigators proposed that there should be a “circadian” photoreceptor in the eye (32, 33). This hypothesis was convincingly demonstrated by Provencio et al., who described a novel retinal photopigment, melanopsin, expressed only in the inner retinal layers of the human (34). Berson et al. (1) further showed that retinal ganglion cells projecting to the suprachiasmatic nucleus in the hypothalamus contain the melanopsin photopigment.

Subsequent experiments in murine revealed at least three types of retinal ganglion cells containing melanopsin (M1, M2, and M3). An M1 cell monostratifies to inner plexiform off-layer, an M2 cell monostratifies to on-layer, and an M3 cell bistratifies to both on- and off-layers (35). Brn3b-positive M1 cells project to the olivary pretectal nucleus, Brn3b-negative M1 cells to the suprachiasmic nucleus, and non-M1 cells to the dorsal lateral geniculate nucleus (dLGN) (36). The multiplicity and basic architectures of melanopsin-containing retinal ganglion cells are confirmed by Dacey et al. (3) in a nonhuman primate. In human, melanopsin-containing ganglion cells were shown to be present in the ganglion cell layer and the inner plexiform layer (37).

Additionally, Dkhissi-Benyahya et al. (38) used immunohistochemistry to demonstrate melanopsin-containing cones within the human peripheral retina. The melanopsin cones are sparsely distributed (5–25 cones/mm²), are present only in peripheral retina (estimated as ∼20° from the foveal area), and contain only the melanopsin photopigment. Other investigators have also shown melanopsin labeling in mouse cones (6). There have not yet been demonstrations that these cones contribute a meaningful physiological signal.

The neural projections of the melanopsin-containing retinal ganglion cells are consistent with their role in circadian rhythms and pupil function. A variety of data show that rods and cones contribute to these functions as well. In addition to nonvisual functions, Dacey et al. (3) support the existence of anatomical circuits in macaque that carry the melanopsin signals to cortical regions essential for visual perception. There has been a debate about whether the circuitry from the melanopsin ganglion cells in mouse projects to the cortical regions used for light perception (39). In reviewing the literature, Nayak et al. (39) conclude that the mRGCs project to regions that are essential for visual perception.

Ecker et al. (6) reported behavioral measurements in mice in which melanopsin is the only functional photopigment. These mice could discriminate spatial patterns up to 0.16 cycles per degree of visual angle. The generalization from these animals to healthy humans is uncertain because of species differences as well as uncertainties concerning developmental neural plasticity in the absence of rod and cone function.

Alternative Explanations for Tetrasensitivity.

Rods.

A particular concern in the healthy human is whether the fourth photopigment might be rhodopsin rather than melanopsin. Including the optics, the estimated peak absorptions of rods and melanopsin-containing cells are 503 nm and 489 nm [using the Stockman–Sharpe photopigment template and lens pigment transmittance function (25); the peak melanopsin absorption without the optics in a nonhuman primate is 482 nm (3)]. Hence, it is virtually impossible to arrange the spectral characteristics of the test light to securely stimulate melanopsin without also stimulating rhodopsin.

To reduce the likelihood that signals are detected by rhodopsin in the rods, we presented the test modulations on a very intense background light (mean luminance 2,060 cd/m²). Psychophysical measurements suggest that rod vision has very little sensitivity above 300 cd/m² (10, 24, 40). Consistent with these data, Naarendorp et al. (41) recently reported thresholds in cone knockout mice and found that sensitivity loss is quite similar to that measured in a human rod monochromat. Sensitivity loss exceeding the classic Weber’s law relationship begins at 10⁴ isomerizations per rod per second. In our conditions this corresponds to a mean background of 20 cd/m² [calculated assuming a 3-mm pupil, inner rod diameter 2.22 µm (42), and rod peak absorbtance of 0.66 (11)]. Under our experimental conditions, we estimate 7.74 × 10⁵ isomerizations per rod per second. Extrapolating the existing data, it appears that rod thresholds at this background intensity would be at least 100% contrast (Fig. S6). The accumulated knowledge about rods and rhodopsin sensitivity under bright conditions makes it very unlikely that the fourth pigment that contributes to light detection in peripheral human retina is rhodopsin.

Further, we sought to reduce the likelihood of rod involvement by making measurements with rapidly flickering test lights (Fig. 7). It might be presumed that rod temporal sensitivity is too sluggish to carry the 40-Hz modulation. Interestingly, Conner and MacLeod (43) showed that under some conditions an apparent rod pathway detects rapid, high-contrast modulations (44). At the time of the psychophysical work, the presence of melanopsin was not known. It is possible that the psychophysical sensitivity they observed arises from a melanopsin-initiated pathway.

Shadow of the retinal blood vessels.

The cones in the deep shadow of the blood vessels do not signal to cortex (45, 46). The rate of cone photopigment isomerizations in the penumbra of the blood vessels could play a role at the detection threshold; our simulations show that there is enough information in the penumbral isomerization rates to detect the cone-silent signal. However, for this information to be useful, the nervous system must develop neurons that segregate the signals of penumbral cones from others. Further, preliminary measurements suggest that contrast sensitivity in these penumbral zones is low (47). This issue merits further investigation.

Photopigment density variations.

Estimates of photopigment optical density in the periphery differ (28, 29), but there is agreement that beyond 10° the optical density varies slowly and stays within the range of 0.25–0.30. Optical density differences change the spectral absorption of the photopigments; hence, such variation might provide an additional source of information.

Quantitative analyses suggest that this variation cannot be a significant factor. We calculated the cone-silent stimulus assuming an optical density of 0.25. We then calculated the Poisson-distributed isomerization rates, assuming cones with a 0.30 density. Because of the change in optical density, the stimulus is no longer cone silent. However, the difference caused by a 10% stimulus is less than 2 SD of the fluctuation in the background isomerization rate. Thus, there would be little chance that the stimulus would be visible. If the visual system does not segregate the cones by optical density, so that the cone population has some variance in the optical density and perhaps other sources of noise, there is even less chance that variations in optical density would enable a subject to detect the stimulus. In conclusion, the photopigment density variation is inconsistent with the quantitative simulations.

The penumbral cone explanation would force us to draw on two entirely different accounts to explain the patient and the healthy subjects. Thus, we consider the melanopsin hypothesis to be the simplest explanation at present. We acknowledge, of course, that further tests are desirable and we have shared our data and software with others who want to test their ideas (48).

Biological origin of the rapid flicker melanopsin signal.

At high temporal frequency (40 Hz), visual sensitivity is well modeled by a single mechanism (Fig. S5). Surprisingly, 40-Hz flicker subjects detect modulations in the cone-silent direction (Fig. 7). We could find no plausible values of inert pigment densities that exclude visibility of stimuli in the cone-silent direction at 40 Hz.

The temporal response of mRGCs is sluggish and sustained, apparently incapable of following a 40-Hz signal (1, 35). We carried out the measurements at 40 Hz with the goal of excluding the mRGCs as a possible pathway. What melanopsin pathway might signal the presence of such rapid flicker?

The melanopsin-containing cones reported in the human peripheral retina are one possibility (38). The stimuli used in our experiments cover a 20° diameter (∼6 × 6 mm² on the retina), which would cover between 200 and 900 such cones. However, there have been no convincing demonstrations that melanopsin-containing cones produce significant physiological signals (6).

Another possibility may be found in the biochemistry of melanopsin itself, which differs from that of rod and cone photopigments. The melanopsin photopigment is bistable with two states with peak wavelength sensitivities at 481 nm and 587 nm (49). Whereas the melanopsin signals may not follow the rapid flicker, the steady-state balance between these two states may depend on the flicker rate. On this hypothesis, the mRGC responses would not follow the 40-Hz signal, but the change in the balance between the states could influence the overall excitability of the mRGCs and produce a detectable signal.

The experimental design in this paper does not depend on having a model of melanopsin-initiated excitation. We have adopted an approach that depends only on showing that an absence of rod and cone photopigment modulations still produces a detectable signal. The bistability of melanopsin makes it difficult to estimate the number of visually effective absorptions because this depends on the relative proportion of molecules in the two states; but the analysis we perform does not depend on knowing the precise properties of melanopsin.

Summary

Historically, consideration of tetrachromacy in the retinal periphery has focused on the question of appearance: Does color matching require accounting for four types of receptors (rods and cones) under mesopic conditions in the periphery? There is no doubt that there are four classes of active receptors under these conditions, but there is no compelling evidence that color matching and appearance become four-dimensional (50). However, there is an interesting note in the literature concerning appearance and tetrachromacy. Bongard and Smirnov describe experiments in the periphery under photopic conditions in which they claim that five primaries are required to produce metameric matches (51). Such color-matching experiments are difficult to instrument and perform. Brindley reports having seen the phenomenon, but he allows that others tried to repeat the experiments and failed (ref. 52, p. 205).

Detection experiments are easier to instrument and perform than appearance, and thus the approach described here may be a simpler path for assessing tetrasensitivty in the periphery. Standard color theory predicts that under mesopic conditions detection experiments could reveal tetrasensitivity. Here we measured visual sensitivity in healthy human subjects under high photopic intensity conditions, far higher than the mesopic range. We tested the hypothesis that sensitivity can be explained by a model that begins exclusively with the encoding of light by three cone photopigments.

In the fovea, trichromatic theory explains visual sensitivity within the measurement error. On the other hand, detection measurements in the visual periphery are not well explained by trichromatic theory. Rather, measurements in the periphery support the hypothesis that a fourth photopigment, probably melanopsin, contributes to sensitivity under high photopic conditions.

The data we report here support a model of peripheral tetrasensitivity—four photopigments mediate sensitivity in the periphery. The data do not address the question of color appearance. If the signals initiated by the four photopigments are funneled into only three distinct neural populations that represent color appearance in the brain, trichromatic theory still serves to explain color appearance. Tetrasensitivity is a feature of the circuitry that determines peripheral light sensitivity.

Methods

To test whether melanopsin contributes to visibility in the healthy human periphery, several problems need to be considered: (i) The mRGC population is estimated to comprise only 3,000 cells with large receptive fields that tile the human retina; stimuli of large visual extent are the most likely to evoke a percept. (ii) Rods and melanopsin absorption curves are very similar, and it is impractical to create primaries that isolate melanopsin absorptions under mesopic conditions. To eliminate the likelihood of rod contributions, the experiments should be carried out on bright mean fields. (iii) To check for four visual pigments, at least four primary lights are necessary. (iv) Because of scattered light and spatial variation in the density of macular pigment density, two lights that are cone metamers in the periphery are not cone metamers in the fovea, and a spatially uniform scene can result in an apparent 2D Gaussian pattern in the central visual field (“Maxwell’s spot”). For these reasons it is best to place the stimulus beyond the range of the macular pigment. We introduce a unique apparatus and experimental procedures that are designed to solve these issues.

Subjects.

One male subject (age 33 y) and one female subject (age 30 y) participated in the foveal and the peripheral experiments (S1 and S2). Another male subject (age 26 y) participated in the peripheral experiments (S3). Additional data were collected from two male subjects (S4 and S5). All subjects had normal color vision according to the Ishihara pseudoisochromatic test (53). All subjects had normal visual fields and normal or corrected-to-normal visual acuity. S1, S2, and S5 used their usual corrective eyewear, clear soft contact lenses, during the experiments. Subjects S1 and S2, respectively, set 266 and 202 total stimulus color directions, which includes at least two staircase trials, at three different temporal frequencies (1, 20, and 30 Hz) in the foveal visual field. In the peripheral experiment, Subjects S1, S2, and S3, respectively, set 322, 368 and 324 total stimulus color directions at three different temporal frequencies (1, 20, and 40 Hz).

All studies were performed with the informed written consent of subjects. All procedures adhered to protocols based upon the world medical association declaration of Helsinki ethical principles for medical research involving human subjects, approved by the ethical committees of Stanford University.

Apparatus.

We designed and built a unique, uniform-field display apparatus (magnetic safe accurate rendering of color, msARC) suitable for both psychophysical and magnetic resonance (MR) imaging (9). The light source is the mixture of six high-intensity light-emitting diodes (LEDs) (LUXEON Star) with spectral peaks at 447.5 nm (royal blue), 470 nm (blue), 505 nm (cyan), 530 nm (green), 590 nm (amber), and 627 nm (red). Spectral power distributions of the six LEDs at 50% pulse-width modulation are shown in Fig. S7A.

The intensities and temporal waveforms of the primaries are managed by pulse-width modulation, using an Arduino Mega microcontroller board, using precise constant-current controllers (LuxDrive BuckPuck). The microcontroller runs a custom (open-source) firmware that receives simple commands from a host computer over a USB connection.

The light from the LEDs is delivered to the subject via optical fiber bundles and a MR-compatible eyepiece. The final image is made uniform by two round (25.4-mm diameter) diffusers in the eyepiece (LSD; Luminit). The subject observes a spatially uniform flickering stimulus through an aspheric lens attached to the eyepiece.

Stimulus Calibration.

The waveform play-out and pulse-width modulation (PWM) are controlled by the microcontroller’s 16-bit timers. The LED intensity is refreshed at about 2,000 Hz with 12-bit PWM intensity control. The device produces accurate sine wave flicker at temporal frequencies over 100 Hz.

Mean luminance (50% pulse-width modulation) of the six primaries is 2,060 cd/m², as seen through the eyepiece. At this high-intensity level, pupil diameter is stable and less than 3 mm for each of the subjects. Assuming a 3-mm pupil, the retinal illumination is over 14,000 Troland (Td); for a 2.5-mm pupil, the retinal illumination is over 10,000 Td.

In principle, it is possible to make threshold measurements by independently varying all six LED primaries. However, for the purpose of testing the three- and four-pigment hypotheses it is only necessary to measure using four primaries. Rather than excluding two LEDs, we decided to use four synthetic primaries, each one being a weighted sum of the six LEDs (Fig. S7B). The LED weights for the four primaries were chosen so that the modulation of one primary mainly influences one of the three cone photopigments or is cone silent. The display primary weights were calculated for a specific model: We assumed that the L-, M-, and S-cone fundamentals estimated by Stockman et al. were at 10° periphery (25, 26), and melanopsin absorption was at 482 nm (3). We further assumed the lens pigment transmittance and melanopsin absorbtance based on basis of the pigment template nomogram (25). Finally, we assumed a photopigment optical density of 0.5 (30). In SI Methods we provide detailed methods on how to calculate the display primary weights.

As we explain in the main text, these assumptions will not be met perfectly by any individual subject. Consequently, we do not assume that the primaries stimulate only a single photopigment type. Instead, we perform our experiments and analyses for a general four-primary system and then adjust the parameters using cone photopigment and inert pigment parameters that best explain the data for each individual subject.

Psychophysical Procedures.

We measured thresholds using a two-interval forced-choice (2IFC), staircase design. The onset of each 1-s interval was denoted by a brief tone. The subject indicated which of the two intervals contained a change from mean luminance. Subjects were provided auditory feedback on each trial.

In different conditions the stimulus waveform was adjusted to be either a slow pulse or a rapid temporal flicker. To efficiently program the Arduino microcontroller, the temporal waveform function was chosen to be sin(2πft) × (1 − cos(4πt)). Time t is in seconds and ranged from 0 to 0.5 s. In the slow pulse conditions f = 1, and both positive and negative modulations were used. The temporal frequency energy is mainly at 1–2 Hz and nearly all below 4 Hz. In the temporal flicker conditions, the frequency f was set to 20 Hz or 40 Hz at periphery and 20 Hz or 30 Hz at fovea (stimulus inspection showed that 40-Hz pulses in the fovea were not visible). In these conditions the temporal energy is centered at the respective frequency and nearly all of the energy is within a few hertz of the center frequency.

For the foveal stimulus experiment, the LED display was centered at a fixation point through a hole in a white board (Fig. S7C). Visual angle of the stimulus was 1° of diameter. The white board was exposed by studio light (ARRI T1 1000W Fresnel) with a blue filter (3203 three-quarter blue). Spectral power distributions of the white board were also measured (XYZ; 98, 101, and 96, respectively).

For the peripheral stimulus experiment, subjects fixated on a small dot; the LED display was centered at 30° horizontal eccentricity in the temporal visual field. The eyepiece has a large white plastic edge that defines the border of the flickering stimulus. The LED display spans a 20° diameter (Fig. S7D). The ambient light level in the room was 81 cd/m².

In some conditions light scattered from the peripheral flicker could be weakly detected in the fovea. To eliminate the possibility that such scattered light could be used for detection, we presented a masking stimulus that covered the central visual field (Fig. S7D, 20° wide, mean luminance 152 cd/m²). The masking stimulus consisted of a 2D Gaussian (FWHM: 5° diameter) flickering with 100% luminance and the same temporal profile as the test light. This mask was present in both intervals of the 2IFC, eliminating the possibility that flicker scattered into the fovea could provide a useful signal.

Color Theory Implementation.

The line-element (quadratic) model can be expressed compactly in matrix notation. Details of these calculations are provided in SI Methods.

The four primary lights are defined by four spectral power distributions D_i(λ) that specify their spectral radiance distribution at maximum intensity. The background is set to a middle intensity level, B(λ), and test lights are temporal modulations of the primary intensities around the background level. We describe the primary light modulations as 〈d_i〉, and the test stimulus is the sum of the background and these modulations . The contrasts in the three cones and the cone-silent direction contrast are computed by a linear transformation of the display primary intensities. We describe the contrast in these four directions by the vector, 〈c_i〉.

Finally, the opponent mechanism weights, v_i,j form a matrix, V and the line-element model can be expressed in matrix notation as

where Q = V^tV is a positive semidefinite quadratic form (0 < c^t Qc). The contrast vectors 〈c_i〉 that satisfy the quadratic (line-element) equation are predicted to be at threshold.

The advantage of matrix notation is that (a) the relationships are expressed in a way that is independent of the number of display primaries and the number of photopigments and (b) the equations can be programmed easily in modern languages.

Model Fitting.

The psychometric function is the relationship between stimulus strength ||c|| and the probability of correct detection, P. We approximate the psychometric function using the Weibull function

We estimate the psychometric function threshold α and slope β, using the following method. First, we fit each single color direction with a Weibull function to estimate α and β. The likelihood function used for the fitting procedure was defined by Watson (54). The log likelihood L is

where n is the number of presentations and x is the proportion of correct response.

The distribution of β-values peaks around 2, which is typical for such color stimuli (55) and slightly lower than the value measured with luminance patterns (56). Hence, we set β = 2 for subsequent Weibull estimates. The threshold α can be estimated from visibility matrix V, using

where d is the four-dimensional stimulus direction, which is unit length of c. We estimate the visibility matrix V, using an iterative search procedure with the Nelder–Mead simplex direct search algorithm,

To obtain confidence limits on the quadratic model parameters, we fitted the data 1,000 times with randomly resampled datasets (bootstrapping method).

Threshold detection data shown in this paper are supplied as a matlab data file on our laboratory’s Web site or on github (48). The data include the trial-by-trial stimulus specification for each staircase as well as the observer’s response on each trial.