Luminance and chromatic signals interact differently with melanopsin activation to control the pupil light response

Abstract

Intrinsically photosensitive retinal ganglion cells (ipRGCs) express the photopigment melanopsin. These cells receive afferent inputs from rods and cones, which provide inputs to the postreceptoral visual pathways. It is unknown, however, how melanopsin activation is integrated with postreceptoral signals to control the pupillary light reflex. This study reports human flicker pupillary responses measured using stimuli generated with a five-primary photostimulator that selectively modulated melanopsin, rod, S-, M-, and L-cone excitations in isolation, or in combination to produce postreceptoral signals. We first analyzed the light adaptation behavior of melanopsin activation and rod and cones signals. Second, we determined how melanopsin is integrated with postreceptoral signals by testing with cone luminance, chromatic blue-yellow, and chromatic red-green stimuli that were processed by magnocellular (MC), koniocellular (KC), and parvocellular (PC) pathways, respectively. A combined rod and melanopsin response was also measured. The relative phase of the postreceptoral signals was varied with respect to the melanopsin phase. The results showed that light adaptation behavior for all conditions was weaker than typical Weber adaptation. Melanopsin activation combined linearly with luminance, S-cone, and rod inputs, suggesting the locus of integration with MC and KC signals was retinal. The melanopsin contribution to phasic pupil responses was lower than luminance contributions, but much higher than S-cone contributions. Chromatic red-green modulation interacted with melanopsin activation nonlinearly as described by a “winner-takes-all” process, suggesting the integration with PC signals might be mediated by a postretinal site.

Introduction

The mammalian retina relies on three types of photoreceptors for phototransduction, including rods and cones in the outer retina and intrinsically photosensitive retinal ganglion cells (ipRGCs) in the inner retina. IpRGCs are important for several nonimage forming functions such as circadian photoentrainment and pupil reflexes (Berson, Dunn, & Takao, 2002; Dacey et al., 2005; Gamlin et al., 2007). These ganglion cells express melanopsin (Hattar, Liao, Takao, Berson, & Yau, 2002; Lucas et al., 2003; Panda et al., 2003; Provencio et al., 2000), which is a photopigment with a peak sensitivity at ∼ 480 nm (Dacey et al., 2005) to provide intrinsic photoresponses. IpRGCs also receive extrinsic rod and cone inputs (Dacey et al., 2005; Guler et al., 2008; Hattar et al., 2003; Wong, Dunn, Graham, & Berson, 2007), and their responses have displayed light adaptation characteristics in rodent studies (Do & Yau, 2013; Wong, Dunn, & Berson, 2005).

Several models have been proposed to explain the integration of intrinsic melanopsin activation and extrinsic rod/cone inputs in ipRGCs. In mammals, it was determined that sustained ipRGC responses were mainly driven by intrinsic melanopsin activation, while transient ipRGC responses were mainly mediated by cone contributions (Dacey et al., 2005; Wong et al., 2007). Therefore, it has been proposed that ipRGCs depend on cones to signal abrupt temporal changes in light intensity while melanopsin plays a dominant role in encoding steady-state illumination (Lucas, Lall, Allen, & Brown, 2012). This dual-role model was derived from ipRGC recordings with light pulses in darkness that explained circadian photoentrainment. Other models have proposed that the integration mechanisms depended on the relative strength or timing of intrinsic and extrinsic signals. In pupil responses, melanopsin, rod, and cone inputs can be combined in a “winner-takes-all” process with a pulsed light (Lall et al., 2010; McDougal & Gamlin, 2010) or a vector summation mechanism with a sinusoidally modulated light under a constant light adaptation condition (Barrionuevo et al., 2014). Both the winner-takes-all model and vector sum model assume that melanopsin, rod, and cone inputs work together in time and space to determine the combined function (in contrast with the dual-role model). However, these models only focused on the integration of luminance and melanopsin activation. Recent studies on the pupillary light reflex have shown opponency of S-cone signals relative to melanopsin activation (Cao, Nicandro, & Barrionuevo, 2015; Spitschan, Jain, Brainard, & Aguirre, 2014), suggesting melanopsin activation and chromatic signals might be integrated differently.

In primates, visual information is conveyed from the retina to the brain by three postreceptoral pathways, including the magnocellular (MC-), parvocellular (PC-), and koniocellular (KC-) pathways (reviewed by Lee, 2011). The MC-pathway consists of diffuse bipolar cells and parasol ganglion cells and combines L- and M-cone signals with the same sign to mediate luminance information. The PC-pathway includes midget bipolar cells and midget ganglion cells but combines L-cone and M-cone signals with opposite signs to provide red-green chromatic information. Finally, the KC-pathway involves S-cone bipolar cells and small bistratified ganglion cells and combines S-cone signals in opposition to L- and M-cone signals to provide blue-yellow chromatic information. Rod signals feed into the cone pathways through two circuits in the primate retina, including the rod- > rod bipolar - > AII amacrine- > cone bipolar pathway and the rod-cone gap junction pathway (reviewed by Zele & Cao, 2015). Therefore, rods also contribute to the three postreceptoral pathways, as confirmed by physiological and psychophysical studies (Cao, Lee, & Sun, 2010; Cao, Pokorny, & Smith, 2005; Cao, Pokorny, Smith, & Zele, 2008; Field et al., 2009; Lee, Smith, Pokorny, & Kremers, 1997). To date, there has been no evidence suggesting parasol, midget or small bistratified ganglion cells contribute to ipRGCs. Bipolar cells in the MC-, PC-, and KC-pathways, however, could provide postreceptoral signals to ipRGBs.

In this study, we first analyzed the light adaptation behavior of pupillary responses to melanopsin activation in comparison to rod and cones signaling. Second, we determined the mechanisms integrating melanopsin activation with postreceptoral signals. It is well known that the pupil responses can be driven by luminance and chromatic stimulations (Barbur, 2004; Kimura & Young, 1995, 1999; Tsujimura, Wolffsohn, & Gilmartin, 2001, 2006). However, it is unknown how melanopsin activation is integrated with the three (MC-, PC-, and KC-) postreceptoral signals to control pupil responses.

Materials and methods

Observers

Three male subjects took part in the study: S1 (20 years old), S2 (45 years old, second author), and S3 (34 years old, first author). All observers had normal color vision assessed by the Farnsworth-Munsell 100 Hue test and the Nagel anomaloscope. The study protocols were approved by the Institutional Review Board at the University of Illinois at Chicago and were in compliance with the Declaration of Helsinki.

Apparatus

We developed a five-primary Maxwellian-view photostimulator (Cao et al., 2015), which mixed lights from five bright LEDs in combination with narrow-band interference filters (Peak wavelength [FWHM]: 456 [10 nm] - blue, 488 [10 nm] -cyan, 540 [10 nm] - green, 592 [16 nm] - amber, 633 [17 nm] – red). The spectra of the five primaries used in this study can be found in https://dx.doi.org/10.6084/m9.figshare.3365005.v1. The lights from the five LEDs were combined through a custom-made fiber optics bundle and a homogenizer. A field lens with a 2 mm artificial pupil was used to create a Maxwellian view. Light attenuation was achieved by neutral density filters placed in front of the field lens. We developed Objective-C based software on a Mac computer to control the light outputs of the LEDs for instrument calibration, observer calibration, and experiments.

The five-primary photostimulator allowed independent control of the stimulations of five types of photoreceptors (S-cones, M-cones, L-cones, rods, and melanopsin-containing ipRGCs) in human retina, using a silent substitution method (Estévez & Spekreijse, 1982; Shapiro, Pokorny, & Smith, 1996). The cone excitations were computed based on the Smith–Pokorny cone fundamentals applied for the CIE 1964 10° Standard Observer (Smith & Pokorny, 1975). The rod excitation was computed based on the CIE 1951 scotopic luminosity function. The melanopsin-mediated ipRGC excitation was computed according to the melanopsin spectral sensitivity function (Enezi et al., 2011). The spectral sensitivity functions of L-cones, M-cones, S-cones, rods, and melanopsin-containing ipRGCs were normalized in a similar fashion as the relative Troland (Td) space (Smith & Pokorny, 1996) such that for an equal-energy-spectrum light at 1 Td, the excitations of rods, S-cones, and melanopsin-containing ipRGCs were all equal to 1 Td, while the excitations of L-cones were 0.667 Td and excitations of M-cones were 0.333 Td.

The instrument was calibrated in three steps, including (a) LED spectrum measurement using a PR-670 spectroradiometer; (b) LED linearization for 4096 digital levels using an International Light ILT1700 current meter; and (c) LED photopic illuminance measurement using an EG&G 550 Radiometer/photometer (Gaithersburg, MD). Details about the photostimulator calibration can be found in Cao et al. (2015).

After physical calibration, we conducted an observer calibration using heterochromatic flicker photometry (HFP) to account for individual differences in prereceptoral filtering and spectral sensitivity in the five-primary photostimulator. The spatial structure of HFP stimulus and artificial pupil were identical to the ones used in the pupil recording experiments. During HFP, the “cyan” LED at 100 Td was set as the reference and a second LED (“blue,” “green,” “amber,” or “red”) as the test. The “cyan” LED and the test LED were modulated out of phase with a squarewave at 15 Hz. We chose 100 Td as the reference illuminance for HFP, as chromatic adaptation at a high retinal illuminance may cause a change in the spectral-luminosity function, leading to a nonlinearity and a HFP failure (Pokorny, Jin, & Smith, 1993). Observers adjusted the retinal illuminance of the test LED to minimize flicker sensation. If the observer had the same photopic spectral luminosity function, V(λ), as the Standard Observer, then the ratios would be equal to one. However, due to differences in prereceptoral filtering and potentially small spectral sensitivity between the observer and Standard Observer, these ratios were different than one. In a block of trials, stimulus combinations were presented in a random order. Each observer repeated the block at least four times. The obtained retinal illuminance ratios of the test LED and the “cyan” LED (at 100 Td) were used in the experimental software to modify LED light presentation. For instance, if an observer set the “green” LED at 80 Td (based on Standard Observer luminosity function) to equate the 100 Td “cyan” LED, then during the stimulus presentation, the “green” LED needed to be scaled by a factor of 0.8 (80/100) for this observer. The same correction method was used for other LEDs based on HFP ratios.

Pupil recording

Observers used their right eye to view the stimuli. No dilation was used as we used a 2 mm artificial pupil, which was smaller than the smallest natural pupil sizes for the highest light levels we used (∼2.9 mm; see also Figure 1 inset). Consensual pupil diameters of the left eye were measured using an Eyelink II Eyetracker (SR Research Ltd., Ottawa, Ontario, Canada) at a sampling rate of 250 Hz (i.e., one sample every 4 ms) with a spatial resolution lower than 0.01 mm. At the beginning of a stimulus presentation, the computer sent a trigger signal to the Eyelink II Eyetracker to synchronize stimulus presentation and pupil recording. To establish the scaling factor between the recorded pupil size (in an arbitrary unit) and pupil diameter in millimeters, a calibration procedure was conducted to reference the pupil diameter to a 6 mm black circle (Barrionuevo et al., 2014). Observers placed their head on a chin rest to fix their head and eye positions.

Figure 1.

Representative pupil traces from 5 s to 15 s with the S-cone (blue line) and melanopsin (brown line) modulations (16% Michelson contrast, 3.3 log Td) recorded during the same session from one subject (S3) after removing the steady pupil size (∼3 mm). A sinusoidal trace in the same phase but an arbitrary amplitude as the stimulation is included for reference purposes (light gray line). Steady pupil size (averaged across subjects) for the five light levels is included in the inset. Traces are filtered minimizing components below 0.5 Hz and higher than 1.5 Hz.

Stimuli

A 30° circular field was generated by the five-primary photostimulator. The light in the central 10.5° diameter was blocked to minimize the potential effect of the macular pigment. A small hole (< 1 arcmin was created in the light blocker center to serve as the fixation therefore the fixation had the same light level as the annulus.

Using the silent substitution method (which can be thought roughly as a linear transformation between the primaries space and the photopigments space), we generated different kinds of sinusoidal modulations (1 Hz) in the five-primary system that stimulated rods, cones, and melanopsin in isolation or in combination (Table 1) at five mesopic-photopic light levels (0.3–0.43 log Td, Table 2). Further details about stimuli generation using the five-primary system can be found in Cao et al. (2015). The chromaticity of the baseline background was constant for all light levels with an orangish appearance [CIE chromaticity (x, y) = (0.45, 0.26); Relative photoreceptor excitation: L/(L+M) = 0.752; S/(L+M) = 0.105; R/(L+M) = 0.319; I/(L+M) = 0.235]. Photoreceptor isolation provided by the five-primary system was verified by measuring the spectral irradiance for the S, R, and I stimuli conditions. The difference between the measured and the desired photoreceptor contrasts (Δ) was less than 1% for the majority of the conditions (Table 3). Only the S-cone excitation (S) for the melanopsin stimulus (I) was 1.9%; however, this value would not be significant compared with the melanopsin contrast in physiological terms.

Table 1.

Photoreceptor modulation excitation contrasts produced for the isolated and combined stimulus conditions in adaptation (A) and integration (B) experiments. Notes: Because of gamut constraints, it was not possible to maintain in the integration experiment the values used in the adaptation experiment. For L+M and L/(L+M), the M-cones and L-cones were modulated. L/(L+M) represented the isoluminant condition.

Table 2.

Light levels used in this study. Notes: The values in log quanta/cm²/sec are expressed at the cornea.

Table 3.

Contrast differences (Δ) between desired and measured excitation contrast values for three stimuli conditions (S, R, and I) produced by the five-primary system.

Procedure

A session consisted of sequential presentations of various randomly chosen sinusoidal stimuli types, separated by a 30 s interstimuli interval (ISI) with a steady background. Each session started with a 15-min dark adaptation period, followed by a 2-min light adaptation period to the background light (see background light intensity in Table 2). The computer produced a sound signal 5 s before the presentation of each stimulus. During the sinusoidal stimulus presentation (40 s), the observers were asked to look at the fixation point using their right eye without blinking. If a blink happened anyway, the complete trial was removed when it was detected by the Eyelink software [long duration blink (>25ms), eye was completely closed] or the blinking period was replaced by average of neighboring regions (short duration blink, eye was not completely closed). Two or three sessions were tested each day, starting with the lowest light level and then higher light levels. Each observer repeated the same condition at least three times on different days. Depending on the experiment, a session consisted up to 13 stimulus conditions.

Analysis

The recorded pupil traces (see Figure 1 for representative pupil response traces that demonstrated opponent pupil responses for an S-cone and melanopsin stimuli at 3.3 log Td) were analyzed using the Discrete Fourier Transform in MATLAB (Mathworks Inc., Natick, MA) to obtain the amplitudes and phases of pupil responses (Barrionuevo et al., 2014). We estimated noise by averaging the amplitudes of neighboring frequencies (0.9 Hz and 1.1 Hz) of the stimulus frequency (1 Hz), and subtracted this noise from the derived amplitude at the stimulus frequency. If this subtraction led to a negative value, we deemed the trial as unmeasurable. Note that some results could be lower than the spatial resolution of the instrument due to this noise removal. A repeated-measures ANOVA was used to compare pupil responses among conditions in STATA 12.0 (StataCorp LP, College Station, TX).

Experiments

Two experiments were carried out. We first analyzed the adaptation characteristics of isolated photoreceptor-mediated pupillary responses, testing the effect of the background adapting light level on flicker responses. The stimuli conditions tested in this experiment were the isolated L, M, S, R, and I. Isochromatic luminance data (LMS) were also obtained for comparison purposes (Table 1).

To compare these flickering data with steady pupil light adaptation characteristics, we fitted the pupil amplitudes with Stevens' power functions (Stevens, 1957):

where R is the pupil amplitude, k is a constant of proportionality, T is the retinal illuminance of the background, and a is the exponent. Note exponent a = 1 represents a Weber adaptation behavior, which is typical for visual processing (Whittle & Challands, 1969).

In the second experiment, we first analyzed the integration of photoreceptor signals across different light levels. Then, to assess the nature of the integration mechanisms of melanopsin with isochromatic luminance (LMS, MC-mediated), chromatic blue-yellow (S-cone, KC-mediated) and red-green [L/(L+M), PC-mediated] signals, the relative phase of melanopsin (I) stimulation was varied from 0° to 315° in 45° steps while maintaining a constant phase for the second stimulation [LMS, S-cone, or L/(L+M)] condition. In order to obtain LED values from photoreceptor excitation, we applied the silent substitution method explained elsewhere (e.g., Cao et al, 2015). Phase transformations were easily achieved considering sinusoidal photoreceptor excitations as phasors (see Supplementary Appendix A). This phase paradigm allowed us to better understand the involvement of post-receptoral visual pathways in ipRGC processing for pupillary control, since the luminance signal is conveyed by the MC-pathway and chromatic information is conveyed by the PC- or KC-pathways (reviewed by Dacey, 2000, and Lee, 2011). If melanopsin activation is combined in a (linear) vector sum with any of the other signals, then the results will have patterns similar to the summation model predictions (Figure 2), either in-phase or out of phase, depending on the relative phase of the second signal with respect to melanopsin activation. On the other hand, if the combination follows the winner-takes-all rule, the amplitude would not change with the melanopsin phase. The pupil phase, however, would follow the melanopsin phase if the “winner” is melanopsin, or it will remain constant if the “winner” is the second stimulation.

Figure 2.

Predictions of the pupil response phase paradigm based on different mechanistic interactions.

To test vector summation, Equations 2 and 3 represent the melanopsin activation and the second postreceptoral input, respectively:

where I_M is the melanopsin activation, I_p is the second postreceptoral input, A_M and A_p are the amplitudes, φ_M is the intrinsic phase of the melanopsin activation, φ_V is the phase delay of the melanopsin activation with respect to the phase of the postreceptoral input, φ_p is the intrinsic phase of the post-receptoral input, f is the frequency of the stimulation, and t is time.

The summation of these contributions is proportional to the pupil response (R) by a factor k_I (Equation 4), that is,

The parameters in Equations 2–4 were searched simultaneously by minimizing the weighted total residual sum of squares (RSS_T) from amplitude and phase data,

where RSS_A and RSS_P were RSS from the amplitude and phase data respectively, and f_A and f_P were weights.

Results

Light adaptation

To understand light adaptation behavior driven by different photoreceptor types, pupil responses under isolated stimulus types were measured, together with the isochromatic luminance (LMS). For LMS stimulation, pupil amplitude increased monotonically with increasing light levels, while the phase was relatively stable at ∼ 15° (Figure 3, dark green symbols and lines). Melanopsin-induced pupil response amplitudes increased with light levels but were lower than the luminance-induced responses, F(1, 12) = 176.43, p < 0.001 (Figure 3A); and phases also increased monotonically with increasing light levels, F(1, 12) = 1.01, p = 0.37, suggesting that the intrinsic melanopsin response speeds up with increasing light levels (Barrionuevo et al., 2014). Note that at 0.3 log Td, melanopsin responses were not measurable, in agreement with previous reports (Barrionuevo et al., 2014; Dacey et al, 2005). Rod-induced responses were measurable only at low light levels (≤ 2.3 log Td), consistent with rod saturation threshold (Adelson, 1982; Aguilar & Stiles, 1954). Rod-induced responses had stable phases that were slightly lower than the luminance-induced phases at 0.3 log Td and 1.3 log Td, F(1, 11) = 5.7, p = 0.036, (Figure 3A), suggesting that rod-driven pupil responses are slower than cone-driven responses at these light levels (Barrionuevo et al., 2014). Both the M- and L-cone stimulations produced similar phases at all light levels (Figure 3B), but the L-cone-induced amplitudes were higher than the M-cone induced amplitudes at high light levels, F(1, 16) ≥ 6.28, p ≤ 0.023. On the other hand, the S-cone-induced responses were recordable only for light levels ≥ 2.3 log Td and were in opposite phases with respect to the luminance, L-cone, M-cone, or melanopsin-driven responses (Figures 1 and 3B), confirming S-cone opponency in ipRGCs, as shown in previous studies (Cao et al., 2015; Dacey et al., 2005; Spitschan et al., 2014).

Figure 3.

Light adaptation experiment results. Amplitude (upper panels) and phase (lower panels) values of pupillary recordings to 1 Hz sinusoidal stimulation, for six stimuli conditions. (A) Results for melanopsin excitation (l), isochromatic luminance stimulation (LMS) and rod excitations (R). (B) Results for isolated L-cone, M-cone, and S-cone excitations. LMS results are incorporated for comparison in panel B. Data points represent the average of the results for three subjects and error bars are SEM.

None of the pupil responses showed a Weber adaptation characteristic, as the exponents of Equation 4 were ≤ 0.25 (Table 4). We also analyzed the baseline steady pupil size data. From 0.3 log Td to 4.3 log Td, pupil diameter ranged from 5.7 mm to 3.5 mm in average (Figure 1, inset). Steady pupil sizes as a function of light level were modeled with Equation 1, resulting in an exponent of 0.47, which was much higher than any of the exponents from the flickering lights (Table 4).

Table 4.

Parameter values of Steven's function (Equation 1) for fits in Figure 1B.

Chromatic red-green modulation [L/(L+M), 4% contrast] showed a much delayed response with respect to luminance with no response at 0.3 log Td (Figure 4A). For this chromatic condition, we observed a large second harmonic (f2), as evidenced in Figure 4B. We computed the ratio between the second and first harmonics at 3.3 log Td (f2/f1, inset of Figure 4B). The ratio was ∼ 0.4 for the L/(L+M) stimulus and was ∼ 0.1 for the LMS stimulus (One-way ANOVA; df = 1, F = 7.22, p = 0.016, suggesting that bipolar cells in the magnocellular pathway may provide L and M-cone signals to ipRGCs (see Discussion for more details).

Figure 4.

(A) Pupil responses across light levels to chromatic red-green modulation [L/(L+M)] in comparison with luminance modulation (LMS). (B) Pupil frequency response for L/(L+M) stimulation for one recording of subject S3 at 3.3 log Td. Note the amplitude of the second harmonics (2 Hz) for this condition. The ratio of the second divided first harmonics of L/(L+M) condition compared with the LMS irradiance, S-cone (S), Rod (R), and melanopsin (I) modulation conditions ratio for the average of three subjects at 3.3 log Td (except for R, which was obtained at 2.3 log Td) are shown in the inset of panel B. Data points represent the average of the results for three subjects and error bars are SEM.

Integration

To assess the integration mechanisms between different photoreceptor inputs, we first measured pupil responses across different light levels. Isochromatic cone luminance stimulation (LMS) and L+M stimulation (Figure 5A, column I) produced similar amplitude and phase responses [amplitude: F(1, 14) = 0.07, p = 0.92; phase: F(1, 14) = 0.07, p = 0.79], suggesting that the S-cone contribution is either masked or cancelled by L-cone and M-cone contributions in the LMS stimulation. The combined melanopsin and luminance (LMS+I) and combined rod and luminance (LMS+R) produced amplitude with a similar monotonic pattern to the luminance stimuli, although with some difference in favor of the combined response LMS+I at light levels where melanopsin is activated [Figure 5A, columns II, LMS+I vs. LMS: F(1, 12) = 8.0, p = 0.047].

Figure 5.

Results of the integration experiment. (A) Pupil responses across light levels to combined stimuli: photopic luminance (L+M) condition, isochromatic luminance and melanopsin condition (LMS+I), and isochromatic luminance and rod condition (LMS+R). The LMS condition is shown in all panels for comparison purposes. (B) Phase paradigm results at 3.3 log Td and 2.3 log Td. Phase of melanopsin excitation was varied with the invariant phase of luminance stimulation (column I), S-cone excitation (column II), or chromatic red-green stimulation (column III) maintained at the same phase. Model fits based on the predictions (Figure 3) appear in each panel. Data points are the average of three subjects' results and the error bars are SEM.

The preponderance of combined L- and M-cone–induced pupil responses over other types of photoreceptor inputs that was shown for these data was in agreement with previous phasic data (Barrionuevo et al., 2014; Spitschan et al., 2014), suggesting strong inputs from the luminance channel in phasic pupil responses. Furthermore, responses to L/(L+M) and S-cone stimulations confirmed the role of chromatic signals in pupillary reflexes (Barbur, Harlow, & Sahraie, 1992; Tsujimura et al., 2001, 2006; Young, Han, & Wu, 1993).

We further tested integration using a phase paradigm at 2.3 and 3.3 log Td (see Figure 5B for the averaged results for three subjects, and Supplementary Appendix B for individual results at 3.3 log Td). For the combined melanopsin and isochromatic luminance (LMS) stimulations (16% Michelson contrast, Figure 5B, column I), pupil response amplitudes and phases followed the “in-phase summation” prediction. Further analysis indicated that melanopsin and luminance signals combined in a vector summation fashion (Figure 2). The model-fitting results indicated melanopsin contributions were relatively weak but with a similar phase when in combination with the isochromatic luminance contribution (Table 5), as shown in our measurements with the isolated photoreceptor stimuli (Figure 3). S-cone and melanopsin stimulation (Figure 5B column II; 16% Michelson contrast), followed the “out of phase summation” prediction (Figure 2), with melanopsin having a higher contribution than S-cones, which was consistent with the isolated responses at this light level (Figure 3). Figure 5B, column III shows the data for combined melanopsin and red-green L/(L+M) signaling. Although the contrast values were smaller [8% melanopsin and 2% L/(L+M) modulations due to gamut limitation], they were sufficient to elicit pupil responses (see Supplementary Appendix B) due to the high sensitivity of the pupillary system to red-green chromatic modulation (Tsujimura et al., 2006). Repeated-measures ANOVAs showed statistical differences with melanopsin phase for LMS+I, F(7, 28) = 5.17, p < 0.001, and S+I, F(7, 28) = 2.85, p = 0.02 mediate- amplitude responses; however, it was not the case for L/(L+M)+I mediated responses, F(7, 28) = 0.82, p = 0.58. This analysis indicated that L/(L+M)+I combination followed the winner-takes-all prediction with the melanopsin signal as the “winner” (“Winner melanopsin” in Figure 2). To evaluate the involvement of rods in ipRGC processing, Figure 6 shows the results for combined rod and melanopsin stimuli in the phase paradigm (9% contrast; 2.3 log Td). The results were consistent with the “in phase summation” prediction with a higher melanopsin contribution than rod contribution (Table 5). Phase paradigm results for LMS+I, S+I and L/(L+M)+I stimuli were obtained at 2.3 log Td (Figure 5B), with similar patterns and parameter values as with 3.3 log Td (Table 5).

Table 5.

Parameter values of the vector summation model for the integration experiment (average data from 3.3 log Td and 2.3 log Td pupil responses). Amplitude ratios and phase differences between both contributions are shown.

Figure 6.

Rod interaction with melanopsin at 2.3 log Td. The phase of melanopsin was varied while the rod phase was fixed. Model fits based on the predictions of Figure 3 appear in each panel. Data points are the average of three subjects' results and the error bars are SEM.

Control experiment to verify photoreceptor isolation

We conducted several control experiments to assure the reliability of our results. Since a recent study claimed that, at 1 Hz, the pupil responses due to melanopsin activation are very small (Spitschan et al., 2014), one of the subjects (S3) carried out the phase paradigm experiment at 0.25 Hz for the LMS+I and S+I conditions. We found that similar summation patterns at 0.25 Hz to 1 Hz (Figure 7A). Furthermore, for the same subject, we measured pupil responses between 0.125 Hz–2 Hz and demonstrated melanopsin showed a different pattern (low-pass) than LMS (band-pass; Figure 7B), suggesting cone intrusion in the melanopsin stimuli is minimal. Although optimal melanopsin-driven pupillary responses seem to be at lower frequencies (Figure 7B), these results showed that melanopsin could reliably track the 1 Hz sinusoidal flickering stimulation under light-adapted conditions.

Figure 7.

Testing photoreceptor isolation in the five-primary system. (A) Phase paradigm (0.25 Hz) for subject S3 when melanopsin activation was combined with luminance (green circles) or with S-cone excitation (blue circles). Both data sets can be approximated with a linear combination of photoreceptor inputs (solid lines). (B) Frequency responses of the melanopsin and LMS conditions (subject S3). (C) Computation of the contrast produced for penumbral L+M cones by the four stimulation types used in the phase paradigm experiment. The maximum variations in contrast (ΔC) are small for the penumbral cones as indicated in each panel. (D) Contrast responses of melanopsin activation for the average of the three subjects (3.3 log Td). Error bars are SEM.

A recent report indicated there is potential intrusion of penumbral cones for melanopsin stimulation generated by the silent substitution method (Spitschan, Aguirre, & Brainard, 2015). This report found that retinal blood vessels cast shadows over some cones (named penumbral cones), affecting the spectral sensitivity of these cones. Although the authors pointed out that penumbral cone intrusion did not affect pupil responses because of poor temporal resolution of the pupillary system (Spitschan et al., 2015), we evaluated whether our results could be affected by penumbral cone intrusion. We computed penumbral cone contrast for the phase paradigm stimuli, following the procedure provided by Spitschan and colleagues (2015, Figure 7C). The maximum range of modulation of penumbral cone contrast, when varying the melanopsin phase, was lower than 1.1% for all stimuli conditions. Additionally, a 16% melanopsin-isolating stimuli produced penumbral L+M cone contrast of only 0.49%. As the relative number of cones in the penumbra is very small compared with open-field cones and pupil response depends on the spatial integration of retinal signals over large areas, pupil response should not be influenced by small penumbral cone contrasts. We tested the contrast response of melanopsin (Figure 7D). With increasing melanopsin contrast, penumbral cone contrast increased accordingly. The intrusion of penumbral cones could be apparent by altering response phases at higher melanopsin contrasts due to differential response phases between melanopsin and cone signals. Our results, however, showed invariant pupil response phases for different melanopsin contrasts, F(4,14) = 1.09, p = 0.42 (Figure 7D), suggesting the intrusion of penumbral cones on the measurement of pupil responses with large field stimuli seems negligible.

Discussion

In this study, we measured human phasic pupil responses to assess melanopsin adaptational behavior and test the integration of melanopsin signals with luminance and chromatic signals. We found that melanopsin displayed a weaker adaptation than cone adaptation. Additionally, our results showed that melanopsin activation combines linearly (in a vector summation fashion) with luminance signals, S-cone inputs, and rod inputs but nonlinearly (in a winner-takes-all fashion) with chromatic red-green signal. The different integration mechanisms imply differential integration sites.

Light adaptation

Our results showed that light adaptation behavior in flickering conditions is weaker than typical Weber visual adaptation or steady pupil diameter changes with background levels. Since Weberian behavior is mediated only by the MC-pathway (Smith, Pokorny, Lee, & Dacey, 2008), it is surprising that LMS-induced flickering responses did not follow Weber's prediction. This adaptional behavior could be constrained by the range of pupil muscle elasticity (Ohba & Alpern, 1972). The results also showed that melanopsin has a weaker adaptation than cones (Table 4), probably due to lack of gain controls in ipRGCs, consistent with their photon counting properties (Dacey et al., 2005; Wong, 2012).

Linear integration

The phase paradigm allowed testing of the integration mechanisms between two signals. A similar approach was used in in vivo macaque ganglion cell recordings (Lee et al., 1997; Smith, Lee, Pokorny, Martin, & Valberg, 1992) and recently in human pupillary recordings (Barrionuevo et al., 2014). We observed that melanopsin activation is linearly combined with rods, S-cones, and isochromatic luminance inputs. Linear summation between melanopsin activation with luminance and rods is expected, as melanopsin activation is involved in the coding of brightness (Brown et al., 2012) and light adaptation (Allen et al., 2014; Wong et al., 2005). On the other hand, linearity between S-cone inputs opposed to melanopsin signals would be useful to code chromaticity variations for visual and nonvisual functions (Walmsley et al., 2015). The linear summation between melanopsin activation and luminance, S-cone or rod signals suggests the integration site could be retinal, likely in ipRGCs.

Nonlinear integration

Our results for the PC-mediated L/(L+M) signal combined with melanopsin follows the winner-takes-all prediction. A principal component analysis (PCA) of photoreceptor excitations in natural images revealed that melanopsin activation contributes to the PC-pathway but reduced the percentage of variance explained by the PC-pathway compared with the model without including melanopsin activation (Barrionuevo & Cao, 2014). Since PCA is a linear transformation, the results showed by Barrionuevo and Cao (2014) could be interpreted as a lack of linearity in the interaction of melanopsin inputs with the rod of cone imputs in the PC-pathway.

Pupil responses to light are largely mediated by the olivary pretectal nucleus (OPN) (Clarke & Ikeda, 1985; Gamlin, 2006; Gamlin, Zhang, & Clarke, 1995; Trejo & Cicerone, 1984; Young & Lund, 1994). Besides retinal inputs, the OPN is innervated by neurons from other brain areas (reviewed by Gamlin, 2006). It was shown that pupillary chromatic reflexes were abolished in rhesus monkeys after injury of the rostral temporal cortex (Heywood, Nicholas, LeMare, & Cowey, 1998). Furthermore, previous studies showed that red-green isoluminant phasic responses are delayed with respect to luminance (Barbur et al., 1992; Tsujimura et al., 2001), suggesting that red-green signals could be mediated by a visual cortical pathway before they reach OPN (Barbur, 2004). Consistent with this, we found a phase difference between L/(L+M) and LMS conditions. As such, a winner-takes-all mechanism could mediate the interaction of retinal melanopsin signals with cortical red-green isoluminant signals in the OPN. This speculation is supported by an evolutionary perspective because the development of specialized visual function driven by the parvocellular pathway is related to neocortical evolution (Barton, 1998). Therefore, it is thought that the PC pathway evolved later than the magnocellular and koniocellular pathways (Barton, 1998, 2004; Fornalé, Vaglio, Spiezio, & Previde, 2012; Regan et al., 2001).

Retinal circuitry

Which retinal circuitries convey rod and cone signals to ipRGCs to mediate phasic pupillary responses? Our results (Figure 4B) showed a frequency doubling component for L/(L+M) mediated response, which can provide clues about neurons involved in retinal processing. The OPN receives direct input from ipRGCs (Guler et al., 2008; Hattar et al., 2002; Viney et al., 2007). From the five types of ipRGCs detected in the rodent retina (Berson, 2014; Feigl & Zele, 2014), M1 cells disproportionally innervate the OPN (Hattar et al., 2006; Hattar et al., 2002). Furthermore, M1 cells are faster than the M2–M5 cells with regard to their intrinsic light responses (Schmidt & Kofuji, 2009; Zhao, Stafford, Godin, King, & Wong, 2014), suggesting that this type of ipRGCs is the main conduit of luminance and S-cone signals to drive phasic pupillary responses. In primate retinas, outer and inner stratifying ipRGCs are the counterpart of mice M1 and M2 cells, respectively (Liao et al, 2016). Outer cells have their dendrites in the OFF sublamina of the interplexiform layer near the inner nuclear layer and are innervated by bipolar and amacrine cells (Grünert, Jusuf, Lee, & Nguyen, 2011; Jusuf, Lee, Hannibal, & Grünert, 2007).

Diffuse bipolar cells convey L- and M-cone information to parasol ganglion cells in the primate magnocellular pathway (Lee, 2011), and these bipolar cells are thought to be responsible for a frequency doubling response to equiluminant red-green modulation, a signature of the MC pathway (Lee, Martin, & Valberg, 1989; Lee & Sun, 2009). Furthermore, previous studies suggested that cells from the MC pathway are involved in pupillary control (Tsujimura et al., 2001; Tsujimura, Wolffsohn, & Gilmartin, 2003). Based on the analysis of our results (Figure 4B), we can speculate that diffuse bipolar cell is a good candidate to convey L- and M-cone signals to ipRGCs and mediate excitatory pupil responses. In particular, direct excitatory input to outer cells is delivered from DB6 diffuse bipolar cells (Grünert et al., 2011). In addition, DB6 cells, although dominated by L and M cone inputs, also make minor but consistent connections with S-cones (Lee, Jusuf, & Grünert, 2004). This differential input suggests that, when stimulated in combination, the small S-cone contribution is masked by L and M signals, as the results between L+M and LMS stimulations showed. It has been suggested that DB6 could produce S-cone inhibitory responses and thus L+M ON and S OFF opponency, which is characteristic of ipRGCs and could be conveyed by DB6 cells (Dacey, Crook, & Packer, 2014). When excitation decreases with respect to the background adaptation level, a different group of retinal circuits is activated, namely the OFF pathway. Major inhibitory inputs were found for outer ipRGCs compared to inner ipRGCs (Neumann, Haverkamp, & Auferkorte, 2011), and dopaminergic amacrine cells that stratify in the same OFF sublamina than ipRGCs (Zhang et al., 2008) could mediate this inhibition (Neumann et al., 2011). These cells receive inputs from cones via ON and OFF bipolar cells (Newkirk, Hoon, Wong, & Detwiler, 2013). However, any explanation of the neurons involved in OFF response remains speculative because there is little data available on the communication between amacrine cells and ipRGCs.

On the other hand, rod signals could be conveyed through gap junctions with the abundant L- and M-cones, which contact DB6 cells. This pathway is active at mesopic light levels (Schneeweis & Schnapf, 1995; Sharpe & Stockman, 1999) and predominantly transmits rod signals due to saturation of the rod bipolar and AII amacrine cell pathway (Hornstein, Verweij, Li, & Schnapf, 2005; Pang, Gao, & Wu, 2004). This rod-cone gap junction pathway was shown to be involved in the ipRGC retinal circuitry in mice (Altimus et al., 2010). Furthermore, an anatomical report recently shown that rod bipolar cells do not provide inputs to ipRGCs in humans (Liao et al, 2016). Although several potential rod pathways could be involved (for a review, see Weng, Estevez & Berson, 2013), the most probable pathway that conveys ON rod and cone signals to ipRGCs to activate phasic pupillary movements is via rod-cone gap junctions- > DB6 cells. Inhibitory signals could be transmitted via dopaminergic amacrine cells, although alternative circuits cannot be ruled out. In fact the participation of more than one pathway could provide redundant information to ipRGCs, thereby reducing noise and improving efficiency.

Conclusions

This is the first study that assessed the integration mechanism of melanopsin activation with other postreceptoral visual signals in phasic pupil responses. Although there is a consensus about the sluggishness of melanopsin, our tests confirmed that melanopsin in light adapted conditions can respond to flicker stimulation in humans. Furthermore melanopsin activation combines linearly, possibly in ipRGCs with luminance and S-cone inputs. By contrast, melanopsin and isoluminant red-green signals seem to be combined nonlinearly, possibly in a postretinal site.