Pupil size as a window on neural substrates of cognition

Abstract

Cognitively driven pupil modulations reflect certain underlying brain functions. What do these reflections tell us? We review findings that have identified key roles for three neural systems: First, cortical modulation of the pretectal olivary nucleus (PON), which controls the pupillary light reflex; second, the superior colliculus (SC), which mediates orienting responses, including pupil changes to salient stimuli; and third, the locus coeruleus (LC)-norepinephrine (NE) neuromodulatory system, which mediates relationships between pupil-linked arousal and cognition. We discuss how these findings can inform the interpretation of pupil measurements in terms of activation of these neural systems. We also highlight caveats, open questions, and key directions for future experiments for improving these interpretations in terms of the underlying neural dynamics throughout the brain.

Changes in pupil size encode cognitive variables

The size of our pupils change continuously in response to variations in ambient light levels to regulate the amount of light entering our eyes. This process is known as the pupillary light reflex (PLR, [1,2]), but it is not purely reflexive. Instead, the PLR can be modulated by attention, high-level image perception, working memory, and other cognitive factors [3-9], Even under steady lighting conditions, and thus independent of the PLR, pupil size can be modulated by cognition [10-12] (also see Supplementary References). For example, as early as 1929 it was noted that: “deep emotions of pleasure as well as fear are commonly accompanied by pupillary dilation” [10,11]. Later it was recognized that these pupil modulations may provide a window into at least some aspects of the inner workings of the brain that are otherwise typically inaccessible; e.g., “the pupils… register directly certain activities of the nervous system, including, but not restricted to, the effects of visual stimulation” [11]. These findings, combined with the relative ease with which pupil size can be measured, have led to a growing use of pupil measurements to draw conclusions about specific features of neural processing that contribute to perception, decision-making, and other aspects of cognition [13]. However, these approaches also raise an important question: What, exactly, can we infer about cognitively relevant brain operations from measurements of pupil size?

The goal of this article is to review our current state of understanding of this “inverse problem.” That is, given what we know about neural mechanisms underlying cognition that also cause changes in pupil size, what can we infer about those mechanisms from measurements of pupil alone? We start with the forward problem and describe studies that have identified pupil modulations that result from, or are at least correlated with, activation of particular neural systems. We focus on three brainstem nuclei that are major points of convergence of descending cognitive signals and project to circuits that control pupil size: 1) the pretectal olivary nucleus (PON), which controls the PLR; 2) the superior colliculus (specifically the intermediate layers, or SC_i) in the midbrain; and 3) the locus coeruleus (LC) in the pontine brainstem. We highlight findings that have related neural activity in each of these systems to changes in pupil size, which provide compelling, albeit still incomplete, evidence for their contributions to cognitively driven pupil changes. We then consider directly the inverse problem of inferring neural activity in these systems from commonly used measures of baseline and event-driven changes in pupil size. We describe the strengths and weaknesses of our current state of knowledge that support these inferences. We also discuss the putative role of additional brain regions that are likely involved in pupil modulations via their effects on these systems. We conclude with a discussion of possible future directions, including experiments that would help to clarify the distinguishable contributions that these and other neural systems have on cognitively driven modulations of pupil size.

The forward problem: assessing relationships between brain activity and pupil size

Brain circuits that drive pupil dilation and constriction are located in the spinal cord and brainstem (Figure 1). These circuits operate in response to changes in light levels that are encoded by retinal signals that are transmitted directly to the PON in the brainstem, which controls pupil size primarily via its projections to the Edinger-Westphal nucleus (EWN; Box 1). These circuits also receive central descending inputs that can provide cognitively driven modulations of pupil size. Many of these inputs converge through three nuclei that each can modulate pupil size via their direct projections to the EWN (Box 2): 1) the PON, which receives top-down input from brain regions involved in high-order visual-saccadic processing [14-17]; 2) the SC, which is a specialized motor nucleus that integrates a wide range of sensory and associative signals to generate commands for orienting and moving the eyes [18]; and 3) the LC, which also receives a range of inputs from a number of brain regions [19-24] and is the primary source of central NE, via which it regulates global arousal [25]. Accordingly, cognitively mediated changes in pupil size are often interpreted in terms of activation of these three nuclei and the larger networks in which they operate [26]. Here we review the evidence that goes beyond these anatomical findings and shows more direct, functional relationships between neural activity in these systems and pupil size.

Figure 1. Pathways that control pupil constriction (left) and dilation (right).

Gray arrows indicate hemisphere-crossing connections at the level of brainstem. Blue arrows indicate LC output to brainstem and cortex.

Box 1: Autonomic circuits that control pupil size: the pupillary light response (PLR).

In the PLR, light-driven pupil constriction is driven mainly by parasympathetic activation, whereas in the dark response dilation is driven mainly by sympathetic activation. Specifically, changes in light level are conveyed by a class of melanopsin-containing intrinsically photosensitive retinal ganglion neurons that project to the pretectal olivary nucleus (PON) in the midbrain [111-113]. PON neurons project to the Edinger-Westphal nucleus (EWN), which is also located in the midbrain [114-116]. A subset of cholinergic EWN neurons provides parasympathetic innervation to the ciliary ganglion (CG), which is located just behind the globe [117-121]. Within the CG, the EWN cholinergic afferents contact neurons that, in turn, project to the eye and innervate the sphincter pupillae, which is a band of muscle encircling the pupil that causes constriction of the pupil when it is contracted [121,122]. Sympathetic input to the pupil arises from neurons in the intermediolateral (IML) cell column of the cervical and thoracic spinal cord [123-125]. This input drives the superior cervical ganglion (SCG) that, in turn, innervates directly the dilator pupillae, which is a band of radial muscle that contracts to dilate the pupil [126,127].

In addition to these relatively straightforward parasympathetic and sympathetic pathways to control pupil constriction and dilation, respectively, there are additional autonomic pathways that are not as well understood. For example, pupil constriction might involve not just parasympathetic control but also light-induced inhibition of sympathetic tone, leading to relaxation of the dilator pupillae [128-130]. Similarly, in addition to direct sympathetic activation of the dilator pupillae, withdrawal of parasympathetic tone in low-light levels leads to relaxation of the sphincter pupillae, which also facilitates dilation [128-130]. Moreover, the strength and dynamics of these factors appear to be controlled differently by each component of the autonomic system, with species-specific differences (reviewed in [103,131]). These effects illustrate the complexity of the circuits that control the pupil, even at the lowest levels in the brainstem, and thus the challenges inherent to drawing conclusions about if and how specific circuits are active based solely on observations of the pupil.

Box 2: Pathways for central modulation of pupil size.

Modulation of pupil size via the pretectal olivary nucleus (PON).

The PON is one of several nuclei in the pretectum that receives direct, ascending input from the retina, and of those it is the only one that also has prominent projections directly to the EWN [52,132]. The role of the PON in the PLR is well established, for example via studies showing that microstimulation of the PON results in pupil constriction [133]. However, the exact nature of the descending cortical signals that provide attentional and other cognitive modulations to the PON, and how those signals are used to govern changes in pupil size, are not as well understood and require further studies.

Modulation of pupil size via the superior colliculus (SC).

The SC control of pupil likely involves both direct and indirect pathways. The superficial layers of SC (SC_s) receive direct retinal ganglion neuron input and project to intermediate layers (SC_i). Certain SC_i neurons then project directly to the EWN and thus can influence the parasympathetic arm of pupil control [48,53,134]. A second, indirect pathway involves SC_i projections to the mesencephalic cuneiform nucleus (MCN), which, in turn, influences parasympathetic tone via projections to the EWN and sympathetic tone via projections to the thoracic and cervical spinal cord (although the latter projection has to date been shown only in rats) [135]. The strength and nature of these direct and indirect projections in terms of their effect on pupil control circuits are unknown, so more work is needed to determine if and how these pathways facilitate dilation, constriction, or both [132].

Modulation of pupil size via the locus coeruleus (LC).

The LC projects directly to the IML, promoting excitatory sympathetic drive viaα1 adrenergic receptors [47,103,136-139]. The LC also can have indirect sympathetic influence via reciprocal connectivity between the LC and the hypothalamus, which projects to the IML [103,140]. Moreover, the LC projects directly to the SC and thus could, in principle, modulate SC-mediated changes in pupil diameter [S16]. finally, the LC is thought to control pupil size via modulation of neural activity in the EWN. The anatomical basis for this modulation remains in debate and may involve direct projections between the LC and EWN, indirect pathways possibly involving common drive via other brainstem nuclei, or both, but is thought to result in inhibition of the EWN, mediated via α2 adrenergic receptors [41,103,109,141,142]. In this framework, LC activation should relax parasympathetic drive (by inhibition of EWN) and enhance sympathetic drive (by excitation of IML), resulting in pupil dilation [38,41,47,140,141].

Relationships between descending projections to the PON and pupil size

There is strong evidence for cognitively driven modulations of the PLR, based on a wide range of studies that have measured the PLR under different task demands and outcomes. For example, variations in whether and how a visual stimulus is perceived can affect the timing and/or magnitude of PLR, even when the physical properties of the stimulus (including its luminance) are unchanged [27,28]. As a striking example of this phenomenon, photographs or even abstract drawings of the sun can cause the pupils to constrict more than other pictures of equal luminance [7,29]. Other studies have shown modulations of the PLR by working-memory load [5,6], sensory context [7,29], spatial attention [8], and eye-movement planning and preparation [30,31].

Our understanding of the specific circuits that support these top-down modulations of the PLR is based largely, although not entirely, on indirect evidence. In general, lesion studies have implicated the involvement of cortical visual pathways [32-34], which are thought ultimately to send high-order information about visual perception and attention to the PON to affect the PLR. These pathways likely involve the frontal eye field (FEF) in prefrontal cortex and the lateral intraparietal area (LIP) in parietal cortex, both of which play key roles in guiding spatial attention and visually guided saccades and project to pretectal nuclei [14-16]. Consistent with this idea, a recent study showed that electrical microstimulation in the FEF can alter the magnitude of the PLR in a spatially specific manner [35]. More studies of this sort are needed to characterize in more detail the nature of FEF-pretectal interactions and possibly identify roles of other components of this visuo-saccadic network, including LIP, in controlling these high-order effects on the PLR under a broad range of conditions.

Relationships between SC_i activity and pupil size

The evidence for functional relationships between SC activation and pupil size, which can occur in the absence of the PLR, is more direct but also somewhat sparse. One prominent pair of studies examined these relationships in the context of the orienting reflex, which involves shifts of gaze and attention that are mediated in part by the SC. The first study showed that pupil dilations are evoked by electrical microstimulation in the SC_i, which receives sensory, motor, and cognitive information from cortex, but not in more superficial layers of the SC, which receives predominantly visual information [36]. These microstimulation effects, which tended to be larger in low-light conditions, were interpreted in terms of a possible role in enhancing visual sensitivity or attention. The second, follow-up study showed that both visual and auditory stimuli evoke pupil responses and changes in neural activity in SC_i that are modulated in a similar manner by contrast-based saliency, which is a key driver of shifts in gaze and attention [37]. Together these studies provide the strongest evidence that stimulus-driven pupil responses that occur in tandem with overt or covert orienting responses reflect functions of the SC_i.

These results are supported by another study that reported relationships between pupil diameter and both single-unit spiking and electrical microstimulation in the SC_i [38]. In that study, individual SC_i spikes were reliably associated with transient dilations that, in principle, could result from the direct projections from the SC_i to brainstem nuclei that control pupil diameter (see Figure 1 and Box 2). However, higher SC_i spiking activity measured over longer timescales (up to several seconds) tended to be associated with more constricted, not dilated, pupils. These results suggest the possibility of multiple pathways driving relationships between SC_i and either dilations or constrictions of the pupil under different conditions and different timescales. The functional roles of these pathways are currently being investigated and are likely linked with the established role of SC_i in conveying signals related to stimulus salience and spatial attention [37,93].

Relationships between LC activity and pupil size

The evidence for functional relationships between LC activation and pupil size, which also can occur in the absence of the PLR, is more extensive. There is indirect evidence for these relationships from measurements of global arousal using electroencephalography (EEG), which was found to covary with pupil size and separately with LC activity [39]. Pupil size has been related to event-related potentials (ERPs) also measured from the scalp, specifically to the P3 component of the ERP in humans performing an auditory oddball task [39]. The P3 is thought to reflect activation of the LC-NE system, and although current evidence for this relationship is largely indirect and somewhat contested [40], recent work has demonstrated a direct link under certain conditions [41]. Other indirect evidence for a pupil-LC link has come from studies that used pharmacological approaches to perturb the LC-NE system, resulting in pupil changes [42]. Together, these studies provided enough indirect evidence that even before extensive measurements of relationships between pupil size and LC neural activity were obtained, the two were generally assumed to covary. Consequently, pupil measurements have long been interpreted in terms of LC neural activation [26,39,40,43].

Recent studies have begun to provide more direct evidence in the form of simultaneous measurements of pupil and LC. In particular, several studies have shown that pupil size tends to covary with LC neural firing rates measured at the same time, under mainly passive task conditions [38,44,45]. First, fast dilations of the pupil were associated with phasic activity in LC-NE axons in the cortex of awake mice [45,46]. Second, LC single-neuron activity was shown to co-vary with pupil size over long (over several seconds) and short (associated with single LC spikes) timescales in monkeys performing a basic fixation task. Moreover, electrical microstimulation of LC in these monkeys reliably evoked pupil dilations [38], an observation that has since been also reported in rats [47]. Third, changes in pupil size that covary with BOLD activity were localized to LC in a pioneering imaging study of humans performing an oddball task [69], which has since been independently corroborated. Fourth, pupil dilation correlated with LC neuron firing rate in monkeys performing a task designed to assess neuromodulatory effects on effort and reward processing around the time of goal-directed actions, although that study did not describe the timing of pupil changes relative to LC neuron firing [44].

Caveats and open questions for interpreting pupil diameter in terms of neural activity

The findings summarized above provide compelling support for interpreting pupil diameter in terms of neural activation patterns that converge through the PON, SC_i and LC. However, several caveats and open questions also remain, including the following:

What are the sources of cognitively relevant signals in the PON, SC_i, and LC?

All three nuclei have strong interconnections with many brain regions, including different regions of cortex (Box 3) and each other. In particular, the PON receives ascending retinal input, descending cortical input, and is interconnected with numerous other subcortical structures including the SC [48-52]. The SC_i receives cortical and subcortical inputs and projects to other nuclei, particularly to nuclei that control gaze and to specific visuo-motor cortical regions [48,53,54]. The LC receives cortical and subcortical inputs and projects broadly throughout the brain [19-25]. Thus, in principle any covariation between pupil size and neural activity in any one of these structures likely also reflects activity in many other brain regions with which they interact. If and how pupil diameter depends on the specific, context-dependent forms of these interactions is not well understood. A useful first step would be simultaneous measurements of neural activity in relevant cortical regions and the PON, SC_i, and/or LC, along with pupil size, to better understand their respective relationships to changes in pupil size.

Box 3. Cortical inputs to PON, SC_i, and LC.

Cortical inputs to PON.

Visual and oculomotor cortical areas provide the dominant descending inputs to the PON [52]. These areas include visual cortex [15,132], inferior temporal cortex [143], lateral intraparietal area (LIP) [144], frontal eye field (FEF) [14,16], medial posterior parietal cortex [17], and supplementary eye field (SEF) [145]. These cortical inputs are thought to contribute to ipsilateral visual field responses of PON luminance-sensitive neurons [52,116] and possibly provide task and brain-state specific modulations of the PLR [35].

Cortical inputs to SC_i.

In the macaque, SC_i receives inputs from many cortical regions that subserve visual and oculomotor function, including secondary and higher visual cortex [24,146], LIP [146-148], FEF [14,54,149], and SEF [145,150], and other movement-related areas including the inferior parietal, ventral premotor, and ventrolateral prefrontal cortices [151]. Similar cortical connectivity has been described in other mammalian species including rats and cats [48,81,152]. However, little is known about the relationships between neural activity in these cortical regions, brain state, and changes in pupil size. One study showed that microstimulation of the FEF can modulate the PLR in a spatially specific manner, but it is not known if these effects are mediated via projections of the FEF to the SC_i, to the PON, or both [35]. More such studies are needed, along with simultaneous measurements of pupil size and neural activity in relevant cortical brain regions plus SC_i, to identify specific projections from cortex to SC_i that mediate changes in pupil size.

Cortical inputs to LC.

In the macaque, the strongest cortical projections to the LC appear to arise from dorsolateral and dorsomedial prefrontal cortices, with weaker projections from somatosensory, parietal, and temporal cortices [19]. In rats, pre- and infra-limbic divisions of the medial prefrontal cortex (mPFC) have strong projections to the core of the LC, with weaker projections from cingulate, orbital, and insular cortices [153]. Inputs from the anterior cingulate cortex (ACC) to the core of the LC have been reported to be relatively sparse [153]. However, other studies reported reliable ACC inputs to the LC, including a preferential input from ACC to the distal dendrites of LC neurons rather than to the LC core [154,155]. This result suggests that a closer examination of cortical inputs to the LC is needed, because there could be differential inputs from specific cortical regions to either the LC core or to the distal dendrites of LC neurons. This pattern could provide an anatomical substrate by which axons carrying distinct types of information, perhaps related to different brain states or behavioral conditions, could contact distinct receptor types on proximal somatic sites versus distal dendritic sites and thereby sculpt the responses of LC neurons. For example, previous work that did not dissect out such putative microarchitecture found both excitatory [22] and inhibitory [23] effects of PFC activation on LC. It is possible that such diverse effects reflect details of the local microarchitecture of LC.

Future directions.

Further work using modern tract-tracing techniques is beginning to better characterize and distinguish the detailed patterns connectivity between cortex and PON, SC_i, and LC. For example, visual area V4 was shown to have topographically organized bi-directional connections with SC_i but convergent, non-topographic bi-directional connections with the LC [24]. Such differences in connectivity patterns might represent an anatomical substrate for spatially specific versus non-specific modulation of activity in these cortical-subcortical loops. Future functional-anatomical studies are needed to continue to characterize these kinds of circuit details and provide a solid basis for designing behavioral tasks to better understand how neural activity in these regions contribute to changes in pupil size.

How to distinguish cause and effect?

Understanding these interactive effects also requires distinguishing cause (that is, is neural activity in a particular brain area causing changes in the PON, SC_i, or LC that are associated with pupil changes?) from effect (conversely, are changes in neural activity in a particular brain area a result of PON, SC_i, or LC activation that is associated with pupil changes?). For example, pupil size has been shown to covary with fluctuations in single-neuron membrane potentials, changes in correlated population activity, and sensory responses of neurons in sensory cortex [55-57]. Because these various neural dynamics were affected diffusely, these results were interpreted as an effect of NE release on cortex and not the cause of LC activation. Another recent study used the relative timing of relationships between pupil changes and neural activity changes in the LC and a number of other subcortical and cortical brain areas to arrive at a similar conclusion: a primary association between pupil and LC can correspond to secondary associations between pupil and targets of LC-NE neuromodulation [38]. Testing these ideas directly and more generally distinguishing causal influences on the pretectal nucleus, SC_i, and LC from effects of changes in their activity patterns will require, for example, both recording and manipulating activity in these brain areas while recording the activity of their neural targets and pupil diameter.

Do pupil changes reflect the activity of all LC neurons, or are they sensitive to the anatomical and functional heterogeneity of the LC?

Early descriptions of the LC emphasized its functional and anatomical homogeneity [58-61]. The persistence of these views has led to interpretations of pupil changes in terms of overall LC activation. However, recent studies have begun to identify anatomical and functional heterogeneity within the LC and in its connectivity patterns [46,62-66]. These findings suggest that there might also be heterogeneity in task- and neuron-specific relationships between LC activity and pupil diameter. In support of this idea, studies comparing LC activity to pupil size have shown neuron-specific differences in the strength and direction of relationships between the two [38,46]. further work is needed to identify the structural and functional characteristics of LC neurons that are responsible for these differences and if and how these relationships depend on the task context.

Do pupil changes reflect activity in multiple neuromodulatory systems?

The basal forebrain-acetylcholine (BF-ACh) system, which is associated with modulation of sensory as well as cognitive functions, including attention, has widespread projections throughout the brain [67,68]. Pupillometry has been used to index changes in the BF-ACh system in relation to attention, similar to the use of pupillometry as an index of LC-NE linked global arousal [3,4,26,39,40,69-71]. As in the case of the LC-NE system, direct, quantitative evidence for correlations between BF-ACh activation and changes in pupil size has come only very recently [45,72]. However, the circuit bases of these correlations remain unknown. One possibility involves NE effects via LC projections to the BF that can depolarize cholinergic neurons via α1-adrenergic receptors [68,73,74]. A direct comparison of the two systems provided indirect support for this mechanism by showing that LC-linked pupil changes preceded ACh-linked pupil changes by ~0.5 s [45].

The “inverse problem”: Inferring brain activity from measurements of pupil size

The relative ease of conducting pupillometry experiments compared to invasive brain measurements, along with a growing understanding of the neural substrates of cognitively mediated pupil modulations, have led to a rapid growth in the number of studies that use pupillometry to study a broad range of cognitive processes. To isolate cognitive from luminance-driven effects on pupil, these studies are often (but not always) conducted in isoluminant conditions, which in principle should minimize contributions of the PON but in practice might be more complicated [13]. Here we review the kinds of inferences that are and can be drawn from two kinds of pupil measurements typically used in these studies: 1) baseline pupil values that persist over relatively long timescales and are typically linked with overall brain state and its effects on arousal and attention; and 2) transient pupil responses to specific, task-relevant events. These measurements have a commonly cited but somewhat tenuous relationship to tonic and phasic firing modes of LC neurons, respectively, and under certain conditions may also reflect activity in the SC₁ and elsewhere.

Brain activation inferred from baseline pupil size

In the context of a number of human behavioral experiments, baseline pupil size has been measured during passive task epochs before the presentation of task-related stimuli. Task-relevant modulations of these baseline pupil values, which have been identified in the context of changes in exploration, belief uncertainty, engagement, effort (e.g., memory load), and other factors, in many cases have been interpreted in terms of cognitively relevant functions of the LC [4,26,70,75-77]. For example, an early study of the relationship between pupil size and memory load used a task in which subjects had to hear and then immediately recall three to seven digits in each trial. Baseline pupil size was found to increase with the number of digits that had to be held in memory [77]. Such links between pupil size and working memory are thought to be mediated, at least partly, by the LC-NE system [78].

Several recent studies have used baseline pupil dynamics to test specific theories of LC-NE function, such as its role in the cognitive control of adaptive behavior. For example, the activity of individual LC neurons in monkeys has been shown to be associated with performance on a visual oddball task [79]. Building on that finding, a subsequent study found a similar relationship between pupil diameter and performance on an auditory oddball task [26]. This study further identified modulations of baseline pupil diameter in terms of changes in costs and rewards on an auditory discrimination task, and in terms of diminishing utility and decisions to transition from an exploit to an explore state. The authors interpreted these baseline pupil modulations in terms of a previously proposed theory about LC-mediated regulation of adaptive state changes in the brain [61].

Baseline pupil size also can reflect sustained shifts in spatial attention. These attentional shifts typically involve changes in neural activity in cortical areas FEF and LIP [80], which can affect the PLR via their projections to the PON (Box 3). FEF and LIP are also heavily interconnected with the SC, whose role in top-down attention and other cognitive processes like decision-making are just beginning to be understood [81]. The SC’s role in spatial attention is modulated, in part, by the BF-ACh system [37,82-84], which suggests a neural substrate for relationships between pupil diameter and the BF-ACh system. Consistent with this idea, the activity of ACh axons imaged in awake mouse visual cortex has been shown to covary with pupil size over relatively long timescales associated with changes in locomotor state [45]. Pharmacological studies have also suggested a relationship between spatial attention, BF-ACh activity, and baseline pupil size, although such ACh-linked effects could in principle result from direct action of these pharmacological agents on the sphincter and dilator pupillae [85,86]. Direct, simultaneous measurements are still needed to assess whether and how the cholinergic BF and pupil size are related under particular cognitive conditions. It also will be useful to design tasks that manipulate factors that selectively engage the various components of these attention-related circuits, including FEF, LIP, SC_i, and BF, to determine their possibly distinct relationships to baseline pupil diameter.

Brain activation inferred from transient pupil responses

A host of factors have been shown to drive brief pupil dilations that are also linked with direct and indirect measures of LC and SC_i activation. Perhaps most prominently, surprising stimuli result in pupil dilations. Although these effects typically reflect bottom-up, stimulus-driven neural processing dynamics, they also often involve top-down, cognitive components (Box 4). For example, pupil dilations can occur in response to not just the presence of an unexpected stimulus but also the absence of an expected stimulus [160]. In humans, these surprise-driven pupil responses are related directly to the amplitude of the P3 component of the ERP and have been associated with LC activation [40,87-89].

BOX 4. Do surprise-driven pupil modulations reflect cognitive processing?

Transient, non-luminance-mediated changes in pupil diameter are often evoked using the oddball paradigm. In this paradigm, a sequence of stimuli is presented, consisting of a commonly presented “standard” stimulus and an infrequently presented (often ~20–40% of stimuli) “oddball” stimulus, in random order. Auditory tones of two different frequencies are often used for the two stimulus types to avoid visually driven pupil effects and because they tend to evoke relatively large pupil and brain responses. Both standard and oddball stimuli tend to cause transient pupil dilations, but the oddball-driven pupil response tends to be substantially larger and has been associated with both the P3 component of the ERP and activation of the LC [39,40].

This basic form of surprise-driven pupil response occurs in the absence of top-down attention and, in principle, can be computed using purely bottom-up mechanisms [9]. These mechanisms likely include stimulus-specific adaptation and neural selectivity for salient stimulus features (e.g., loudness or brightness) [156]. However, several lines of evidence suggest that surprise-driven pupil responses also can reflect the engagement of higher-order processing. First, oddball-driven pupil responses tend to be substantially larger when they are paired with a motor response that distinguishes between standard and oddball stimuli, versus when both are presented passively, which is consistent with a general sensitivity of pupil diameter to decision-making and task demands [9,70]. Second, requiring attentive processing also can lead to pupil dilations in response to more subtle and sophisticated violations of expectations, such as changes in complex stimulus patterns that were just learned [157]. Third, pupil responses also tend to be larger for novel versus rare-but-familiar (i.e., oddball) stimuli, implying a role for memory that, in principle, could be encoded in early sensory pathways for relatively simple stimuli but requires more sophisticated mechanisms for more complex or abstract stimuli [158].

Thus, it seems likely that under all but the simplest, passive conditions, surprise-driven pupil responses reflect both bottom-up and top-down mechanisms. Future work should focus on identifying distinguishable signatures of these two classes of mechanisms in the timing and/or magnitude of surprise-driven pupil responses to, for example, oddball sequences presented within versus without a task context. Such signatures could then be used to infer the relative contributions of bottom-up and top-down processing under a broad range of conditions, such as during adaptive decision-making that requires ongoing adjustments to those contributions to deal effectively with the kinds of dynamic and uncertain sources of information that we often encounter [108,159].

Task-relevant, evoked pupil dilations have also been associated with stimulus salience, stimulus value, attentive state, prior expectation, estimation error, decision biases, and changes in task utility [11,70,71,75,90,91]. Because many of these factors can also drive transient increases in LC neuron firing rates [61,79,92], transient pupil dilations are often interpreted in terms of such phasic LC firing. In addition, some of these factors including stimulus salience are associated with shifts of attention including the orienting response and thus are also likely associated with SC_i activation [36]. Consistent with this idea, when neural activity in SC_i was manipulated using electrical microstimulation and pharmacology, transient changes in pupil size were found to be modulated by stimulus features at an attended location [93]. When similar attentional shifts are associated with visual processing, particularly for stimuli whose real or perceived brightness might be affected by such shifts, these pupil effects are also likely to involve cortical modulations of the PLR via the PON [13,15].

Interpreting these kinds of task-driven changes in pupil diameter in terms of specific neural activity patterns requires identifying not just the proximal source(s) of those patterns (e.g., the PON, SC_i, or LC) but also the more distal, higher-order brain networks that provide the context-specific, descending signals that generate those activity patterns. The relevant computations are likely implemented in a number of brain regions whose contributions depend on the specific cognitive factors that are involved. For example, transient pupil modulations in response to cued shifts in spatial attention likely involve contributions from FEF, LIP, and SC [80], which then affect pupil diameter via their projections to the PON and SC_i.

In contrast, other cognitive operations that affect transient changes in pupil diameter via the LC likely involve contributions from other parts of cortex, particularly medial prefrontal regions including the anterior cingulate cortex (ACC) and other prefrontal regions including the orbitofrontal cortex (OFC; see Box 3 for more details) [61,94,96]. Relationships between neural activity in the ACC and pupil diameter are beginning to be understood. For example, numerous studies have shown that ACC neurons encode surprise, conflict, or other forms of uncertainty that also can modulate pupil diameter [96,97-99], with one study showing a direct relationship between conflict-driven ACC activity and pupil changes [100]. ACC neural activity has also been shown to covary with pupil modulations under passive viewing conditions, over both short (single spike) and longer (several seconds) timescales [38]. Less is known about if and how task-relevant neural responses in the OFC, which encodes value judgments and other forms of reward-related processing, relate to transient changes in pupil diameter [101]. A recent study showed that BOLD activity in OFC, along with not just the ACC but also parts of the thalamus, posterior cingulate cortex, and insular cortex, covaries with pupil size in humans performing a simple attention task [102]. However, more work is needed to distinguish cause from effect (see above) and identify the specific task conditions under which specific components of this brain network contribute to task-driven pupil changes. This work will undoubtedly benefit from emerging findings of anatomically distinct cortical circuits that are affected by, and can affect, the LC under different conditions [46,64], although it remains to be seen if and how such structural and functional heterogeneity of LC-related information processing can be inferred from different characteristics of the pupil response.

Caveats and open questions for interpreting baseline and evoked pupil measures

Despite the advances in our understanding of how to interpret baseline and pupil measures in terms of the underlying neural dynamics, a number of caveats and open questions remain, including the following:

How can pupil changes that are driven by task-related cognitive demands be distinguished from those driven by non-specific factors, such as changes in overall arousal state?

During goal-directed behaviors, the LC is driven by descending, task-relevant signals from cortex, causing changes in pupil size. However, task-independent modulation of LC activity, such as with fluctuations in drowsiness, can also result in changes in LC activity with similar effects on pupil [103]. Can LC-linked pupil changes that result from these distinct regimes be distinguished from each other? One possible answer is to characterize pupil separately in passive versus active behavioral epochs, using the former to characterize autonomic-related fluctuations that can be factored out when analyzing task-related fluctuations in the latter. Operationally, baseline autonomic tone can be estimated using passive fixation periods preceding presentation of task-related stimuli [26,70,71]. Conversely, baseline pupil measured during task-related epochs (such as stimulus presentation or the post stimulus decision period) might reflect a combination of autonomic tone and task-relevant factors.

How do phasic and tonic modes of LC activation relate to pupil measures?

The terms “phasic” and “tonic” have been proposed to distinguish between different modes of LC activation, not simply between transient and baseline LC activity, respectively [60,61]. Specifically, based primarily on data from rodent studies, LC activation patterns have been divided into: 1) “phasic” mode, which occurs primarily during periods of high task engagement and is dominated by robust, transient responses to task-relevant events that occur when baseline activity is relatively low in the LC; and 2) “tonic” mode, which occurs primarily during periods of high distractibility and involves higher baseline activity and weaker phasic responses in the LC [60,61,94,104]. However, the relevance of these distinct modes of LC activation to conditions that are used commonly to measure pupil diameter in humans, particularly in relation to cognitive processing, is not clear, because these conditions typically involve subjects who are almost always engaged in performing a task and thus likely are in a state akin to “phasic mode.” Exceptions to this idea include studies that have used tasks that were designed explicitly to test predictions based on phasic/tonic modes of LC firing [105]. Careful use and clear definitions of terminology can help to alleviate some of the confusion; e.g., distinguishing “tonic mode” (a specific form of LC activation pattern) from “tonic” or “sustained” activity (describing an ongoing neural or pupil signal) and “baseline” activity (a measured, typically sustained, neural or pupil signal that is independent of an experimental or task manipulation); and likewise “phasic mode” (a specific form of LC activation pattern) from “phasic” or “transient” activation (describing a brief, typically event-driven neural or pupil response).

How should relationships between baseline and evoked pupil magnitude be interpreted?

A key feature of the proposed phasic/tonic distinction for LC activity is that there is a fundamental trade-off between the magnitude of transient, event-driven activation and baseline levels. Specifically, it has been suggested that maximal transient responses occur at relatively moderate baseline levels (phasic mode) but are attenuated when LC is inactive and arousal is low or when LC is highly active and arousal is high (tonic mode) [61]. This trade-off has been used to interpret pupil measurements, as well, such that transient and baseline values that are inversely related to each other are assumed to reflect a similar relationship in the LC. One challenge to this interpretation is that, as noted above, many task conditions likely do not involve the dramatic changes in baseline arousal state that are needed to evoke the different modes of LC activation. Another challenge is that there is an additional, primary reason for an inverse relationship between transient and baseline pupil measurements that is independent of the LC: the structure of the eye, pupil musculature, and balance between the components of autonomic drive places physical limits on the degree to which the pupil can transiently dilate from a given baseline [38,106,107]. As a practical consequence, this relationship must be accounted for appropriately (e.g., by quantifying evoked pupil magnitudes in terms of residuals to the evoked-versus-baseline trend [71]; also see [26] for a systematic measurement of evoked relative to baseline pupil size) before interpreting the magnitude of an evoked pupil response in terms of a cognitive variable or the magnitude of a related neural response.

Which nuisance variations can affect measures of cognitively driven modulation of pupil size?

Baseline pupil size is often measured during a period of stable fixation at the beginning of each trial in an experimental task. These measurements can be affected by non-cognitive factors that can vary from session-to-session (e.g., the ambient lighting in the room, if it is not controlled carefully), trial-to-trial (e.g., the subject’s head position relative to the camera, if both are not securely fixed), and even moment-to-moment (e.g., the timing and precision of the subject’s visual fixation, relative to when the baseline pupil measurements are taken). Such factors must be carefully controlled to obtain reliable measures of cognitively driven changes in baseline pupil diameter, particularly within long experimental sessions or across sessions. These effects can also influence transient pupil responses, the magnitude of which depends on baseline pupil values.

Concluding remarks

Pupillometry is a useful, non-invasive technique for measuring changes in brain activity patterns that are associated with particular task-related factors, including arousal, salience, attention, surprise, and effort.

Compared to other techniques that measure brain activity like fMRI, pupillometry is convenient and inexpensive. However, pupillometry also has its limitations. Pupil size is affected by luminance and autonomic activity as well as cognitive factors, is an indirect marker of neural activity, and has limited temporal resolution for decoding the underlying neural signals. These limitations make it challenging to use pupillometry to draw conclusions about specific changes in the brain. The purpose of this review has been to describe the strengths and limitations of our current ability to draw such conclusions.

In the first part of this review, we focused on the anatomical and functional circuits that drive pupil changes. We described the PON, SC_i, and LC as primary hubs for mediating the effects of cognitive factors on the pupil. Several recent studies have used direct measures and manipulations of neural activity in these brain areas to establish a firm anatomical and functional basis for their relationships with pupil diameter. However, more work is needed to fully understand how these relationships depend on task conditions, autonomic state, and functional interactions between these hubs and the many other parts of the brain with which they are connected.

In the second part of this review, we addressed current approaches and caveats to interpreting changes in pupil size in terms of the underlying neural activity patterns. We argued that despite a diversity of conditions that affect pupil diameter, the convergence of cognitively relevant signals onto the three hubs of pupil circuitry, and a growing understanding of their distinguishable roles in different cognitive operations, makes it possible to interpret both baseline and evoked changes in pupil diameter in terms of activity in these hubs. For example, attention-related modulations that affect baseline pupil size likely reflect contributions of the SCi and possibly PON, whereas violations of expectations that case transient pupil dilations are likely encoded by the LC-NE system. Although we currently lack a complete understanding of how the roles of these hubs are similar and different from each other under different cognitively demanding conditions, ongoing and future work using tasks designed to selectively activate these specific regions or neuromodulatory systems will directly test for such expected relationships between neural activation and pupil (see “Outstanding Questions” Box) [108].

OUTSTANDING QUESTIONS.

1. What are the distinct and overlapping roles that the PON, LC, and SC_i play in controlling pupil size? For example, to what extent do LC effects on SC_i account for relationships between SC_i and pupil size?

2. What drives LC neurons, and do different sources of LC drive have different relationships with pupil changes? For example, can the effects of bottom-up autonomic inputs and top-down cortical inputs on the LC be distinguished from each other using pupil measurements alone?

3. What is the anatomical and functional basis for LC influence on the EWN? Some tracer studies show projections from LC to EWN, although if and how these projections affect EWN neurons that are related to pupil control remains unknown.

4. What is the anatomical and functional basis for relationships between SC_i neural activity and pupil size? In particular, what is the exact role of the MCN, which appears to be ideally positioned to provide a pathway for SC_i outputs to reach both the sympathetic and parasympathetic arms of the autonomic pupil control circuit? Moreover, why does SC_i microstimulation evoke pupil dilation, whereas under certain conditions SC mean firing rate is negatively correlated with baseline pupil size?

5. What is the relationship between PON- and SC_i-mediated changes in pupil size in the context of shifts of gaze and attention? Do the two circuits have distinguishable roles under different task conditions?

6. How is activity in different neuromodulatory systems related to changes in pupil size? Direct measurements under limited conditions have been made primarily in the LC-NE system, although there are intriguing findings from studies of the BF-ACh system in mice. We do not yet know whether and how the dopaminergic and serotonergic neuromodulatory systems are also linked with pupil fluctuations, and whether there might be species-specific differences in these links.

We conclude by proposing that it will be particularly fruitful to target different neuromodulatory systems in the context of specific cognitive operations that they are thought to mediate, possibly resulting in distinct relationships to pupil diameter. For example, phasic changes in global arousal are thought to be mechanism for rapid reorienting to a behaviorally relevant event. These mechanisms are generally associated with activity in the LC-NE system [79,92, 109,110]. In contrast, spatially specific attentional changes are more closely associated with activity in the BF-ACh system [82]. Well-established tasks can be used to separately manipulate arousal or attention and assess if and how each relates changes in pupil size and activity in these two systems. These studies can then be extended to include measures of neural activity in brain regions that project to or receive projections from these neuromodulatory systems, thereby providing a broad view of the patterns of changes in neural activity throughout the brain that can be inferred from pupillometry.

TRENDS BOX.

• Recent years have seen a surge in interest in using pupil size to gain insights into how the brain processes information in the context of a broad range of behavioral tasks; e.g., a search for “pupil” and “behavior” returned a list of ~3 studies/year from 1967–2008 and >15 studies/year from 2009–2019.

• A growing body of anatomical and functional work has identified key roles for three brain regions in funneling cognitively relevant information to brainstem circuits that control the pupil: the pretectal olivary nucleus (PON), intermediate layers of the superior colliculus (SC_i), and locus coeruleus (LC).

• Based on these findings, measures of baseline and evoked changes in pupil diameter can be interpreted in terms of activation of these regions and, in some cases, the cortical and subcortical sources of the cognitive signals that drive them.

• Emerging techniques are beginning to allow us to: 1) assess relationships between pupil size and neural activity measured simultaneously in multiple brain regions; 2) test for causal roles of circuits involving the PON, SC_i, and LC in controlling pupil; and 3) determine the context dependence of these roles by using carefully controlled behavioral tasks.