Ocular Measures During Associative Learning Predict Recall Accuracy

Abstract

The ability to form associations between stimuli and commit those associations to memory is a cornerstone of human cognition. Dopamine and noradrenaline are critical neuromodulators implicated in a range of cognitive functions, including learning and memory. Eye blink rate (EBR) and pupil diameter have been shown to index dopaminergic and noradrenergic activity. Here, we examined how these ocular measures relate to accuracy in a paired-associate learning task where participants (N=73) learned consistent object-location associations over eight trials consisting of pre-trial fixation, encoding, delay, and retrieval epochs. In order to examine how within-subject changes and between-subject changes in ocular metrics related to accuracy, we mean centered individual metric values on each trial based on within-person and across-subject means for each epoch. Within-participant variation in EBR was positively related to accuracy in both encoding and delay epochs: faster EBR within the individual predicted better retrieval. Differences in EBR across participants was negatively related to accuracy in the encoding epoch and in early trials of the pre-trial fixation: faster EBR, relative to other subjects, predicted poorer retrieval. Visual scanning behavior in pre-trial fixation and delay epochs was also positively related to accuracy in early trials: more scanning predicted better retrieval. We found no relationship between pupil diameter and accuracy. These results provide novel evidence supporting the utility of ocular metrics in illuminating cognitive and neurobiological mechanisms of paired-associate learning.

Introduction

Associative learning underlies a wide range of cognitive abilities, and typically involves linking two previously unrelated stimuli by encoding them, binding them in memory, and subsequently retrieving them. This binding is a fundamental cognitive process. Although the frontal-hippocampal circuits that support paired object-location learning have been described (Büchel, Coull, & Friston, 1999; Diwadkar et al., 2008; Law et al., 2005; Milner, Johnsrude, & Crane, 1997), the role of neuromodulatory systems has been less thoroughly explored. Neuromodulatory systems change the excitability and potentiation of cells within the hippocampus, effectively controlling how information within the frontal-hippocampal network is processed. Dopamine and noradrenaline are two such neuromodulators that affect excitatory signaling within the frontal-hippocampal network (Hansen, 2017; Jay, 2003; Kahnt & Tobler, 2016; Palacios-Filardo & Mellor, 2019). Understanding how these neuromodulatory systems relate to associative learning may be particularly important for uncovering the physiological mechanisms of inefficient associative learning in psychiatric disorders, most notably bipolar disorder and schizophrenia (Brambilla et al., 2011; Diwadkar et al., 2008). However, in vivo measurement of dopamine and noradrenaline is a methodological challenge. In this regard, two ocular measures – eye blink rate (EBR) and pupil diameter – assume importance because they are tractable behavioral measures that provide insight into neuromodulatory activity. These ocular measures index activity of dopamine and noradrenaline, respectively (Evinger et al., 1993; Jepma & Nieuwenhuis, 2010; Jongkees & Colzato, 2016; Koss, 1986; Wang & Munoz, 2015), and are modulated by cognitive processes that engage these neuromodulatory systems (Jongkees & Colzato, 2016; Peinkhofer, Knudsen, Moretti, & Kondziella, 2019). Here, to better understand how neuromodulatory systems relate to associative learning and memory, we examine how changes in pupil diameter and EBR relate to the accurate retrieval of object-location pairs across repeated presentations during an associative learning task.

Learning object-location associations is a cognitive process that engages a brain network that includes the medial temporal lobe and frontal cortex (Büchel et al., 1999; Diwadkar et al., 2008; Law et al., 2005; Milner et al., 1997). The importance of this network for associative learning has been highlighted in humans using fMRI (Banyai, Diwadkar, & Erdi, 2011; Bunge, Burrows, & Wagner, 2004; de Rover et al., 2011) and in animal studies (Gilbert & Kesner, 2002; Langston, Stevenson, Wilson, Saunders, & Wood, 2010; Talpos, Winters, Dias, Saksida, & Bussey, 2009). The neural mechanisms at play within this network are becoming more clearly understood using methods that do not rely on hemodynamic signals. For example, using a paired object-location learning task, proton functional magnetic resonance spectroscopy studies have demonstrated that efficient learners show increased glutamate modulation in the hippocampus early during learning, and this modulation decreases as a function of proficiency (Stanley et al., 2017). Excitatory dynamics in the hippocampus are influenced by neuromodulators including noradrenaline and dopamine (Harley, 1987; Puig, Antzoulatos, & Miller, 2014; Tully, Li, Tsvetkov, & Bolshakov, 2007). Modulatory effects on hippocampal excitability result from ventral tegmental area dopamine (Bethus, Tse, & Morris, 2010; Lisman & Grace, 2005) that affects synaptic plasticity (reviewed in Jay, 2003). Similarly, the locus coeruleus (the primary source of noradrenaline) densely innervates hippocampal structures, facilitating learning and memory through the modulation of synaptic plasticity (Harley, 1987, 1991; Kempadoo, Mosharov, Choi, Sulzer, & Kandel, 2016; McNamara & Dupret, 2017; Takeuchi et al., 2016). Ocular measures that index the activity of these neuromodulators may therefore provide insight into their role in associative learning.

Relationships between both pupil size and EBR and the activity of specific neuromodulators has been documented (Jongkees & Colzato, 2016; Peinkhofer et al., 2019). Pupil dilation is tightly coupled with increased activity of the noradrenergic neurons in the locus coeruleus. This coupling occurs via direct (Koss, 1986) or indirect effects of noradrenergic activity on the Edigner-Westphal nucleus, which controls muscles in the iris, or via shared pre-synaptic input (Nieuwenhuis, De Geus, & Aston-Jones, 2011). Similarly, EBR, and in particular resting EBR, relates to dopaminergic activity (reviewed in Jongkees & Colzato, 2016), especially in the striatum (Groman et al., 2014; Taylor et al., 1999). Dopamine influences EBR via modulation of the endogenous spontaneous blink generator through inputs from the basal ganglia (Evinger et al., 1993). This is thought to occur via basal ganglia connections to the superior colliculus and nucleus raphe magnus which in turn modulate inputs to the trigeminal complex (Basso & Evinger, 1996; Basso, Powers, & Evinger, 1996; C. Evinger et al., 1993; Kaminer, Powers, Horn, Hui, & Evinger, 2011), providing a mechanism for striatal dopamine to affect blink rate. Thus, both EBR and pupil diameter have established links to neuromodulatory systems. Given these connections, we would expect that engagement of cognitive processes reliant upon these neuromodulatory systems would be reflected in changes in EBR and pupil size.

In accordance with this expectation, EBR and pupil dilation have been widely linked to cognitive processes, in part because of their relationship to dopaminergic and noradrenergic activity. Pupil dilation – indexing increased noradrenergic activity – is broadly associated with increased arousal and alertness as baseline pupil sizes (Knapen et al., 2016) and phasic pupillary dilations (Hopstaken, van der Linden, Bakker, & Kompier, 2015) decrease with time on task as participants become fatigued. However, there is further nuance to the relationship between performance, noradrenergic activity, and pupillary dynamics. Two modes of locus coeruleus activity have been described: phasic and tonic, (Aston-Jones & Cohen, 2005; Usher, Cohen, Servan-Schreiber, Rajkowski, & Aston-Jones, 1999), which predict on and off task attention, respectively (Unsworth & Robison, 2016). Phasic firing occurs in response to external stimuli, and is associated with accurate task performance, low baseline noradrenergic activity (Aston-Jones, Rajkowski, Kubiak, & Alexinsky, 1994), small baseline (i.e. pre-trial) pupil diameter, and large task-evoked dilations (Gilzenrat, Nieuwenhuis, Jepma, & Cohen, 2010). Tonic mode, on the other hand, is characterized by higher baseline firing rate of noradrenergic locus coeruleus neurons, poor task performance (Aston-Jones et al., 1994), large baseline pupil diameter, and smaller task-evoked dilations (Gilzenrat et al., 2010). Taken together, pupil dynamics can index noradrenaline’s nuanced role in task engagement.

Similarly, between-person differences in resting EBR relate to cognitive processes that rely on dopaminergic modulation, such as attention (Maffei & Angrilli, 2018), inhibitory control (Den Daas, Häfner, & de Wit, 2013; Zhang et al., 2015), serial reaction time task learning (San Anton, Cleeremans, Destrebecqz, Peigneux, & Schmitz, 2018), and cognitive flexibility (Dreisbach et al., 2005; Müller et al., 2007; Tharp & Pickering, 2011; Zhang et al., 2015). Although less-studied, within-person differences in EBR across trials or in response to stimuli have also demonstrated associations with cognitive performance (Bacher, Retz, Lindon, & Bell, 2017; Drew, 1951; Oh, Jeong, & Jeong, 2012; Poulton & Gregory, 1952; Rac-Lubashevsky, Slagter, & Kessler, 2017; Siegle, Ichikawa, & Steinhauer, 2008). These effects have been characterized as tonic (differences between participants) and phasic (responses to tasks or stimuli) in the literature (Bacher & Allen, 2009; Jongkees & Colzato, 2016). As with pupil size, EBR also relates to fatigue: both drowsiness and time on task are associated with faster EBR (Barbato et al., 2007; Stern, Boyer, & Schroeder, 1994). These links between cognitive states that engage nueromodulatory systems, and ocular measures may provide insight into associative learning that compliments previous neuroimaging work.

Tentative links between both pupil diameter and EBR and associative learning and memory have been reported. In infants, phasic pupil dilation has been observed to a reward-predicting cue, and the extent of associative learning between the cue and reward was predicted by baseline EBR (Tummeltshammer, Feldman, & Amso, 2019). In adults, phasic pupil responses are linked with lexical paired-associate learning performance (Colman & Paivio, 1970; Pajkossy & Racsmány, 2020; van Rijn, Dalenberg, Borst, & Sprenger, 2012), and recognition memory more broadly (Goldinger & Papesh, 2012; Heaver & Hutton, 2011; Papesh, Goldinger, & Hout, 2012; Võ et al., 2008). EBR is associated with rapid associative binding of visual stimuli and actions (Aarts et al., 2012; Colzato, van Wouwe, & Hommel, 2007). Colzato and colleagues used a paradigm in which a first visual stimulus, which could vary on three dimensions, triggered a response based on a pre-cue (arrows of a particular direction). Subjects were then presented with a second stimulus that could share some featural properties of the first stimulus and were asked to make a response based on one of those features. The response could be the same or different to the first response. The reaction time costs of partial repetitions were taken as a measure of associative binding of visual features of the stimulus with the motor response. The authors found that resting EBR predicted individual differences in the association between stimulus features and response. Similarly, Aarts and colleagues used an intentional binding paradigm that measured the degree to which intentional actions and sensory inputs are bound and perceived as causal. They found that intentional binding was stronger after being primed with reward-related information, and that the strength of this binding was related to resting EBR.

These prior studies suggest that EBR and pupil diameter do indeed index processes related to binding stimuli in memory; however, this previous work (particularly for EBR) has focused on implicit, rather than explicit, associations. Here, we relate ocular metrics to performance on a task that measures explicit associative learning through repeated presentation and recall of object-location pairs – a paradigm increasingly characterized behaviorally (Diwadkar et al., 2016; Diwadkar et al., 2008), and through neuroimaging (Ravishankar et al., 2019; Stanley et al., 2017; Woodcock, Wadehra, & Diwadkar, 2016; Woodcock, White, & Diwadkar, 2015). In these paired object-location associative learning tasks, participants learned consistent pairings across repeated trials with four epochs – pre-trial fixation, encoding, delay, and retrieval. Behavioral results from this work demonstrate negatively accelerating exponential learning as participants commit object-location pairs to memory: with repeated presentation of the pairings, subjects recall object-location pairs more accurately. Neuroimaging using this task suggests an interplay between dorsolateral pre-frontal cortex, dorsal anterior cingulate, the basal ganglia, and the hippocampus. These network interactions differ between encoding, delay, and retrieval, and they change as object location pairs are learned (Ravishankar et al., 2019; Stanley et al., 2017; Woodcock et al., 2015). Here, we utilize ocular measures during paired object-location learning in order to provide further insight into the neurobiological underpinnings of associative learning and, more specifically, to index engagement of dopaminergic and noradrenergic neuromodulation during this task.

Given the established links between pupil diameter and noradrenaline and between EBR and dopamine, and given that these neuromodulators critically influence functional dynamics within brain networks known to be involved in associative learning, we expected both pupil diameter and EBR to relate to recall accuracy. Therefore, during a paired object-location associative learning task, we recorded pupil diameter over the final two seconds of pre-trial fixation and delay epochs, reflecting tonic pupil size. We also recorded EBR during all four epochs (pre-trial fixation, encoding, delay, and retrieval). We related between-participant and within-participant centered ocular measures to recall accuracy over repeated trials, while controlling for visual scanning (which can influence both pupil size, EBR, and spatial recall). We predicted that a decrease in noradrenaline, reflected by reduced pupil size relative to a participant’s own average, would impair associative learning due to reductions in arousal (Knapen et al., 2016; Unsworth & Robison, 2016). We predicted that pupil size during the delay period in particular would be positively related to recall accuracy, as this is the period during which information is retained before retrieval. With regard to EBR, we expected that increases in dopamine during the encoding and delay epochs – reflected by increased EBR relative to each individual’s own average – should result in facilitated memory given prior literature on dopaminergic facilitation of hippocampal plasticity (Jay, 2003; Jongkees & Colzato, 2016) known to subserve associative learning. However, increased baseline EBR between participants is also related to increased novelty bias and distractibility (Dreisbach et al., 2005; Tharp & Pickering, 2011), which could suggest that between-participant mean centered differences in EBR – especially during pre-trial fixation (akin to a baseline) and encoding epochs – could negatively impact accuracy.

Methods

Participants

Undergraduate students at Michigan State University (n=82) were recruited through the Department of Psychology Human Participation in Research pool and provided informed consent. The MSU IRB approved all study procedures. Participants were excluded for corrected visual acuity below 20/20, color blindness, not speaking English, self-reported mental illness, and use of psychotropic medications. Eight participants were determined to be noncompliant with the associative learning task and were excluded (see Data Analysis section below for specific criterion). One participant provided incomplete data due to eye tracker dysfunction. Thus, analyses were performed on a final sample of 73 participants (52 women; mean age of 21.4 yrs., range: 18–57; SD = 5.72; though the 57-year-old was an outlier in terms of age, corresponding ocular metrics and behavioral data were all within 3 standard deviations of their respective means).

Apparatus and stimuli

Participants were seated in a dimly lit room with their head stabilized on a chin rest 59 cm away from a computer screen. The paradigm was presented on a 39.3 × 29.2 cm monitor (spatial resolution 1024 by 960 pixels, vertical refresh rate 85 Hz), against a black background. During the task, eye position and pupil diameter were recorded using an EyeLink 1000 Plus (SR Research, Mississauga, ON, Canada). Stimulus presentation was controlled with MATLAB (The MathWorks, Natick, MA) using the Psychophysics (Brainard, 1997) and EyeLink (Cornelissen, Peters, & Palmer, 2002) toolboxes.

Associative learning task

The task was adapted from previous studies (Diwadkar et al., 2016; Diwadkar et al., 2008), and is depicted in Figure 1. Participants learned nine object-location associations over the course of eight trials. The object-location pairings were consistent across learning trials. Each trial consisted of four epochs: fixation, encoding, delay, and retrieval. Each trial began with a drift correction procedure where participants fixated an annulus and pressed the space bar. Successful drift correction initiated the onset of the fixation epoch. During the fixation epoch (9s), participants fixated on a white cross presented at the center of the screen. During the encoding epoch (27s), participants were sequentially presented with nine equi-familiar and mono-syllabic objects (Snodgrass & Vanderwart, 1980), each at a reserved location in a 3 × 3 grid (3s per object). A given object’s location on the grid did not change across trials, but the order in which objects were presented within each trial was pseudorandomized and consistent across participants. During the delay epoch (27s), participants were presented with an empty 3 × 3 grid to which they were asked to remain oriented. During the retrieval epoch (27s), participants were presented with the 3 × 3 grid, and sequentially cued to each location with a red square for 3s. During this cue, participants were asked to verbalize the object that was presented at that location. Participants responded with “no” if they were unable to recall or guess the cued object. Verbal responses were recorded by the experimenter. Prior to the advent of the first trial, participants were presented with a practice trial, in which the true task stimuli were replaced by color coded emoticons.

Figure 1. Associative Learning and Memory Paradigm.

(a) Schematic illustration portraying the trial structure, which consists of four discrete epochs, each 27s in duration: fixation, encoding, delay, and retrieval. During the encoding epoch, nine stimuli (e.g. pen, shoe) were sequentially presented for 3s in each location on the grid. During the retrieval epoch, participants were cued by a filled-square to verbalize their response at each location. There were eight total learning trials (b) Nine equi-familiar stimuli were shown during the encoding epoch of each of the 8 learning trials. The white number in the corner of the cell represents the order at which these stimuli were presented within a given learning trial. Object-location pairings remained consistent across trials; however, the sequence was pseudorandomized by trial and consistent across participants.

Data analysis

Performance on the associative learning task, pupil diameter, and blinks were analyzed using MATLAB (The MathWorks, Natick, MA). Statistical analyses were performed using R and the lme4 library (Bates, Sarkar, Bates, & Matrix, 2007; R development core team, 2013) using the lmer function and the LmerTest library (Kuznetsova, Brockhoff, & Christensen, 2015) to calculate p-values using Satterthwaite’s method.

Individual examination of associative learning

A single value for accuracy (percent correct) was computed for each participant for each of the eight learning trials, and this value was used for all further analyses of accuracy. Based on prior validation in associative learning studies using similar paradigms (Banyai et al., 2011; Buchel, Coull, & Friston, 1999; Diwadkar et al., 2016; Diwadkar et al., 2008; Ravishankar et al., 2019; Stanley et al., 2017; Woodcock et al., 2016; Woodcock et al., 2015) learning curves were iteratively fit to the accuracy data for each individual using a negatively accelerated response function of the form:

$$\left[\text{accuracy}\text{=}\text{a}-{\text{b}e}^{-\text{c}\left(\text{trial}\right)}\right],$$

Where a is the asymptote, b is the difference between the initial performance and the asymptote, c is the slope of the curve (larger c values indicate faster learning), and e is Euler’s number. These fitted curves were used as the basis for excluding individual participants on the basis of their performance. Namely, eight participants had poor fits (adjusted R² for the best fitting model was negative, suggesting non-incremental performance), and these participants were excluded from further analysis (resulting in a final sample of 73 participants).

Ocular measurements

Pupil diameter was measured during the fixation and delay epochs, while EBR was measured during all four epochs. Pupil diameter is confounded by eye position when using video-based eye trackers. Thus, samples where eye position fell beyond a two-degree visual angle square centered on central fixation as well as samples coinciding with blink and saccade events were removed from analysis. Pupil diameter from only the final two seconds from the fixation and delay epochs were averaged, similar to previous studies¹ (Gilzenrat et al., 2010). There were no subjects for whom mean EBR or mean pupil diameter across the eight trials exceeded three standard deviations from the group mean in any epoch. There were isolated instances within trials during which eye-tracker dysfunction produced missing samples for an entire epoch. Across participants and learning trials, pupil data was not viable in the final two seconds of twelve fixation epochs and one delay epoch, and eye blink data was not recorded or useable in two fixations epoch, one encoding epoch, one delay epoch, and four retrieval epochs.

Blinks were marked by the Eyelink online event parser using their proprietary algorithm based on several consecutive missing pupil samples. This algorithm has been independently shown to accurately identify blinks (Ehinger, Groß, Ibs, & König, 2019). To further ensure data quality in our blink measurements, we removed blinks outside of the normal range of blink duration (less than 80 ms or more than 900 ms in line with reported blink durations in Ehinger et al., 2019). After removing outliers, the mean blink duration was 220.21 ms (SD = 51.53 ms). EBR (blinks/min) was calculated for each epoch (fixation, encoding, delay, and retrieval) in each participant by dividing the total number of blinks in that epoch by the epoch duration (in minutes).

Blink rate and pupil size are sensitive to eye movements and, thus, visual scanning (Evinger et al., 1994; Fogarty & Stern, 1989; Mathôt, van der Linden, Grainger, & Vitu, 2015; Watanabe, Fujita, & Gyoba, 1980). Furthermore, visual scanning has been found to affect visual learning processes (Brandt & Stark, 1997; Brockmole & Irwin, 2005; Laeng & Teodorescu, 2002; Spivey & Geng, 2001). Thus, to ensure that putative effects of blinks or pupil size on accuracy were not due to differences in scanning behavior, we also computed a measure of visual scanning (see Fogarty & Stern, 1989), incorporating these values into our statistical models. Scanning behavior was operationalized as the summed distance in visual angle between fixations and was computed for each epoch within each learning trial.

Group level analysis relating ocular measures to associative learning

We used linear mixed effects models to predict accuracy over learning trials at the group level (as opposed to the negatively accelerating curves fit to individual participants). To assess how ocular measures within each epoch related to learning across trials, linear mixed effects models predicting accuracy from both trial and ocular measures were run. Data across learning trials from each epoch were analyzed separately with distinct models for pupil data and EBR data. Therefore, two models were run for pupil data (fixation and delay epochs), and four for EBR (all task epochs). Each of these models took a similar form:

$$\text{Accuracy}~{\text{t}}^2+{\text{wpO}}_{\text{e}}*\text{t}+{\text{gmO}}_{\text{e}}*\text{t}\text{+}{\text{wpD}}_{\text{e}}*\text{t}+{\text{gmD}}_{\text{e}}*\text{t}+\left(1+\text{t}|\text{ID}\right)+\left(1|\text{t}\right)$$

Where interactions (*) modeled both main effects and their interaction, t is the mean centered learning trial, t² is the squared, mean centered trial, wpO_e is the within-person mean centered ocular measure (EBR or pupil diameter) from the epoch (_e) of interest, gmO_e is the grand-mean centered ocular measure from the epoch of interest, wpD_e and gmD_e are the within-person mean and grand-mean centered distances between fixations for the given epoch. Both trial and the squared value of trial were included to account for the typical positive linear trend in accuracy with learning, and the eventual flattening out of that curve as maximal learning is achieved. Grand-mean centered measures were included to test whether differences between individuals during a particular epoch in the ocular measure of interest predict accuracy (e.g. do people who blink more, on average, during the delay period show greater accuracy?). Grand-mean centered ocular values represent the value for a participant on a trial in an epoch minus the average across all participants and trials in an epoch. Within-person centered ocular measures were included to test whether differences within an individual during a particular epoch predict accuracy (e.g. when people blink more during the delay period, relative to their own mean blink rate during the delay period, do they show greater accuracy?). Within-person centered values represent the value for a participant on a trial in an epoch minus the average value within the epoch for the participant across all trials. Interactions with trial were included to determine whether changes in ocular metrics over learning trials were related to corresponding changes in accuracy.

Random intercepts by participant were included in all models to account for between participant random variance. On epochs that could support it (following the procedure in Bates, Kliegl, Vasishth, & Baayen, 2015) random slopes across trials per participant were included to account for differences in learning rates across participants. Similarly, in models that could support it, random intercepts per trial were included. Where possible, all three random effects were included. In cases where models would not converge with the full set of random effects, the random intercepts by trial were first removed. If the model still did not converge then the slopes over trials per participant were removed and the trial level random intercepts were re-introduced. If the model still did not converge, only the participant level random intercepts were included. In assessing the variance accounted for by random effects, the participant level intercepts accounted for the majority of variance across all models.

Results

In the following section, we first describe behavioral performance during the associative learning task. We then present models predicting accuracy on the associative learning task based on pupil diameter and EBR.

Associative Learning

Accuracy over trials, averaged across participants and fitted with a negatively accelerated response function is depicted in Figure 2: participants became more accurate with repeated presentation of the object-location pairs.

Figure 2. Associative learning performance.

Mean percentage of correct responses per learning trial fitted by a negatively accelerated learning function.

Pupil Diameter

We examined whether pupil diameter in the fixation or delay epochs (Figure 3A) was related to learning while controlling for distance between fixations. In both epochs, trial showed significant (p < 0.001) linear and quadratic relationships with accuracy. Neither within-person, nor grand-mean centered pupil diameters, in either the fixation (Table 1A), or delay epoch (Table 1B) predicted accuracy (all ps > 0.210). Similarly, neither fixed effect showed significant interactions with trial (all ps > 0.549). Neither within-person, nor grand-mean centered scanning behavior related to accuracy in either fixation or delay epochs (all ps > 0.192), nor were there significant interactions with time (all ps > 0.149).

Figure 3.

Pupil diameter (A) and eye blink rate (B) across learning trials and averaged across participants for each analyzed epoch.

Table 1.

Percentage Correct Predicted by Change in Pupil Diameter.

A. Fixation Epoch
Random Effects	Variance	Std. Dev.	N.	Observations
Participant (Intercept)	208.81	14.45	73	572
Trial Slopes per Participant	5.04	2.25
Fixed Effects	Estimate	Std. Error	T-value	P-value
Intercept	80.54	1.78	45.28	< 0.001
Trial	17.17	1.11	15.48	< 0.001
Trial2	−1.24	0.12	−10.51	< 0.001
Within Person Centered Pupil Diameter	0.00	0.00	1.26	0.210
Within Person Centered Distance Between Fixations	0.04	0.06	0.79	0.431
Grand mean centered pupil diameter	0.00	0.00	0.55	0.585
Grand Mean Centered Distance Between Fixations	−0.03	0.05	−0.53	0.601
Grand Mean Centered Pupil Diameter*Trial	0.00	0.00	0.33	0.743
Grand Mean Centered Distance Between Fixations* Trial	0.01	0.01	1.18	0.240
Within Person Centered Pupil Diameter*Trial	0.00	0.00	0.60	0.549
Within Person Centered Distance Between Fixations*Trial	−0.02	0.01	−1.45	0.149
B. Delay Epoch
Random Effects	Variance	Std. Dev.	N.	Observations
Participant (Intercept)	198.90	14.10	73	580
Trial Slopes per Participant	4.55	2.13
Trial (Intercept)	1.60	1.27
Fixed Effects	Estimate	Std. Error	T-value	P-value
Intercept	80.74	1.81	44.55	< 0.001
Trial	16.31	1.49	10.95	< 0.001
Trial2	−1.14	0.16	−7.12	< 0.001
Within Person Centered Pupil Diameter	0.00	0.00	1.23	0.218
Within Person Centered Distance Between Fixations	−0.01	0.02	−0.42	0.675
Grand Mean Centered Pupil Diameter	−0.00	0.00	−0.07	0.945
Grand Mean Centered Distance Between Fixations	0.03	0.02	1.32	0.192
Grand Mean Centered Pupil Diameter*Trial	0.00	0.00	0.25	0.806
Grand Mean Centered Distance Between Fixations* Trial	−0.00	0.00	−0.81	0.418
Within Person Centered Pupil Diameter*Trial	0.00	0.00	0.13	0.896
Within Person Centered Distance Between Fixations*Trial	−0.01	0.01	−1.24	0.216

Eye Blink Rate

Descriptive statistics for EBR in each of the four task epochs are presented in Table 2 and displayed over learning trials in Figure 3B. We examined the relationship between EBR and accuracy in all four epochs, controlling for scanning behavior (see Table 3). In all four epochs we identified linear and quadratic effects of trial (all ps < 0.001).

Table 2.

Descriptive blink statistics in each epoch

Epoch	Mean (blinks/min)	Standard Deviation
Fixation	25.29	15.11
Encoding	24.55	12.97
Delay	32.68	23.28
Retrieval	31.73	13.20

Table 3.

Percentage Correct Predicted by Eye Blink Rate in the Four Epochs

A. Fixation Epoch
Random Effects	Variance	Std. Dev.	N.	Observations
Participant (Intercept)	198.38	14.09	73	582
Trial (Intercept)	3.30	1.82
Fixed Effects	Estimate	Std. Error	T-value	P-value
Intercept	80.65	1.86	48.94	< 0.001
Trial	17.12	1.74	5.05	< 0.001
Trial2	−1.24	0.19	5.04	0.001
Within Person Centered Distance Between Fixations	0.00	0.06	0.02	0.986
Grand Mean Centered Distance Between Fixations	0.01	0.06	0.09	0.929
Within Person Centered EBR	0.19	0.14	1.39	0.167
Grand mean centered EBR	−0.15	0.13	−1.20	0.235
Grand Mean Centered EBR*Trial	0.04	0.02	2.01	0.045
Grand Mean Centered Distance Between Fixations* Trial	0.01	0.01	1.42	0.156
Within Person Centered EBR*Trial	−0.01	0.04	−0.31	0.754
Within Person Centered Distance Between Fixations*Trial	−0.03	0.01	−2.45	0.015
B. Encoding Epoch
Random Effects	Variance	Std. Dev.	N.	Observations
Participant (Intercept)	186.08	13.64	73	583
Trial Slopes per Participant	4.59	2.14
Fixed Effects	Estimate	Std. Error	T-value	P-value
Intercept	80.64	1.69	47.75	< 0.001
Trial	16.52	1.21	13.63	< 0.001
Trial2	−1.17	0.13	−9.29	< 0.001
Within Person Centered Distance Between Fixations	0.02	0.03	0.59	0.554
Grand Mean Centered Distance Between Fixations	0.02	0.02	1.14	0.259
Within Person Centered EBR	0.04	0.17	2.41	0.017
Grand mean centered EBR	−0.34	0.13	−2.53	0.014
Grand Mean Centered EBR*Trial	0.00	0.03	0.25	0.805
Grand Mean Centered Distance Between Fixations* Trial	0.00	0.03	0.25	0.805
Within Person Centered EBR*Trial	−0.05	0.05	−1.00	0.320
Within Person Centered Distance Between Fixations*Trial	−0.01	0.00	−1.28	0.203
B. Delay Epoch
Random Effects	Variance	Std. Dev.	N.	Observations
Participant (Intercept)	188.24	13.72	73	583
Trial (Intercept)	0.41	0.64
Fixed Effects	Estimate	Std. Error	T-value	P-value
Intercept	80.70	1.72	46.95	< 0.001
Trial	15.63	1.28	12.22	< 0.001
Trial2	−1.08	0.14	−7.85	< 0.001
Within Person Centered Distance Between Fixations	−0.02	0.02	−0.97	0.332
Grand Mean Centered Distance Between Fixations	0.04	0.02	1.91	0.061
Within Person Centered EBR	0.33	0.12	2.82	0.005
Grand mean centered EBR	−0.13	0.08	−1.59	0.118
Grand Mean Centered EBR*Trial	−0.02	0.01	−1.34	0.181
Grand Mean Centered Distance Between Fixations* Trial	−0.00	0.00	−0.28	0.782
Within Person Centered EBR*Trial	0.00	0.04	0.10	0.917
Within Person Centered Distance Between Fixations*Trial	−0.01	0.01	−2.02	0.044
D. Retrieval Epoch
Random Effects	Variance	Std. Dev.	N.	Observations
Participant (Intercept)	196.78	14.03	73	580
Trial Slopes per Participant	4.64	2.15
Trial (Intercept)	3.18	1.78
Fixed Effects	Estimate	Std. Error	T-value	P-value
Intercept	80.60	1.84	43.83	< 0.001
Trial	17.40	1.70	10.25	< 0.001
Trial2	−1.27	0.18	−6.95	0.001
Within Person Centered Distance Between Fixations	−0.01	0.02	−0.48	0.635
Grand Mean Centered Distance Between Fixations	0.02	0.02	1.19	0.238
Within Person Centered EBR	0.24	0.16	1.52	0.131
Grand mean centered EBR	−0.21	0.13	−1.59	0.117
Grand Mean Centered EBR*Trial	0.02	0.03	0.75	0.455
Grand Mean Centered Distance Between Fixations* Trial	0.00	0.00	0.05	0.959
Within Person Centered EBR*Trial	0.02	0.03	0.40	0.689
Within Person Centered Distance Between Fixations*Trial	−0.00	0.01	−0.27	0.791

In the fixation epoch, we found two significant interaction effects. First, we observed an interaction between within-person centered distance between fixations and trial (p = 0.015; see Figure 4E). This interaction reflected a negative relationship between accuracy and within-person centered distance between fixation in early trials, which reversed direction in later trials. We additionally observed an interaction between grand-mean centered EBR and trial (p = 0.045; see Figure 4C). Grand-mean centered EBR was negatively related to accuracy, but this relationship disappeared in later learning trials. No other effects were significant.

Figure 4.

Significant fixed effects and their interaction with time for within-person mean centered EBR (top row), grand-mean centered EBR (middle row), and distance between fixations (bottom row). Estimates for main effects are indicated with a subscripted m and estimates for interactions with trial are indicated with a subscripted i.

In the encoding epoch, we found a significant effect of grand-mean centered EBR (p = 0.014; see Figure 4D), such that more blinks than average were associated with lower accuracy. We also observed an effect of within-person centered EBR (p = 0.017; see Figure 4A), but in the opposite direction. On trials where participants blinked more relative to their average blink rate across the encoding period, they showed increased accuracy. Critically, this was the case when including scanning behavior and trial in the model. No other effects were significant.

In the delay epoch, we found a significant effect of within-person centered EBR on accuracy (p = 0.005; see Figure 4B). On trials where participants blinked more relative to their average blink rate across the delay period, they showed increased accuracy. Interestingly, we also found a significant interaction between within-person centered distance between fixations (p = 0.044; see Figure 4F). Scanning behavior was positively related to accuracy early (on trial 1) but shifted to a negative relationship in later trials. This is noteworthy, because there were no stimuli on the screen during the delay epoch.

In the retrieval epoch, we found no significant effects.

To summarize, our findings in the encoding and delay epochs demonstrated that on trials where participants blinked more relative their own individual mean in each epoch, accuracy was better. Conversely, the finding during encoding that increased grand-mean centered EBR is related to decreased accuracy (in line with the pattern of results across trials in the fixation epoch) suggests that blinking more than average is related to poorer performance. These results reflect multiple tests; however, the effect of within-person centered blinks in the delay epoch remained even after conservative Bonferroni correction for the number of statistical tests performed when examining the effect of EBR on accuracy (corrected α =0.0125). Additionally, we demonstrated that during the fixation and delay epochs there was a relationship between scanning behavior and accuracy that changed over trials: early on, visual scanning when no stimuli were present predicted better accuracy. In later trials, increased visual scanning was related to lower accuracy or unrelated to accuracy.

Discussion

Our aim was to use ocular measures as a window into the functional mechanisms that subserve associative learning. We addressed this aim by recording pupil diameter and EBR during a paired object-location associative learning paradigm. We additionally examined visual scanning, given that it has been related to visual learning, EBR, and pupil size (Brockmole & Irwin, 2005; Evinger et al., 1994; Mathôt et al., 2015; Spivey & Geng, 2001; Watanabe et al., 1980). We hypothesized that within- and between-person differences in EBR would differentially affect learning. On one hand EBR is related to dopaminergic activity (Jongkees & Colzato, 2016; Taylor et al., 1999) that could facilitate hippocampal-mediated learning (Jay, 2003; Palacios-Filardo & Mellor, 2019), but increased EBR has also been related to increased distractibility (Dreisbach et al., 2005; Tharp & Pickering, 2011). We also hypothesized that pupil diameter would relate to accurate task performance given its relationship with noradrenaline-mediated arousal processes. These predictions were partially supported by our findings. First, we found that during both the encoding and delay periods, participants who blinked more than their own average blink rate across all eight encoding or delay epochs, showed increased recall accuracy. Second, participants who blinked more (relative to others) during the encoding epoch showed poorer recall accuracy—a finding that upon initial glance appears to be in conflict with the first finding. This same relationship appeared early in the fixation epoch but was reduced over time. These results were observed even after controlling for the extent of visual scanning, which can itself lead to changes in EBR. Third, participants who engaged in more visual scanning relative to their own mean in the fixation and delay epochs initially showed increased accuracy but across trials this relationship reversed. Finally, pupil size did not predict accuracy in either the pre-trial fixation period or the delay period. In the remainder of this report, we discuss the potential cognitive and neurobiological implications of these findings.

We found that increased within-person mean-centered EBR during the encoding and delay epochs predicted accurate recall. This result indicates that on trials in which participants blinked more relative to their average blink rate – averaged across the eight learning trials for the encoding and delay epochs separately – they were more accurate. Prior research on the relationship between EBR and learning has primarily focused on learning mediated by striatal D2 receptors (e.g. Slagter, Georgopoulou, & Frank, 2015). One possibility is that within-person increases in EBR on a trial-by-trial level may reflect phasic increases in dopaminergic activity that have the potential to influence hippocampal-based circuits that underlie this type of paired-associate learning (de Rover et al., 2011; Diwadkar et al., 2016; Kim, Heath, Kent, Bussey, & Saksida, 2015; Sommer, Rose, Weiller, & Büchel, 2005). Such an effect would be most likely during the encoding and delay phases when memory consolidation through binding of memoranda occurs.

Another possibility is that increased EBR relative to one’s own average blink rate during the encoding and delay periods—when information is being registered and maintained—reflects the engagement of working memory processes. Phasic bursts of dopamine provide gating signals for afferents to prefrontal cortex, enhancing representation of new information (Miller & Cohen, 2001; Müller et al., 2007) and working memory (Floresco & Phillips, 2001). These gating mechanisms determine what relevant information is allowed into working memory and have been proposed to rely on striatal dopamine (Frank, Loughry, & O’Reilly, 2001) that is essential for enhancement or inhibition of information within cortical-striatal-thalamic-cortical loops. Increased striatal activity during delay periods (Hikosaka, Sakamoto, & Usui, 1989) and striatal dopamine (Landau, Lal, O’Neil, Baker, & Jagust, 2009) have both been linked to working memory performance. Indeed, in a previous study, increases in event-based EBR were associated with increased working memory demands (Rac-Lubashevsky et al., 2017). Therefore, increased relative EBR – especially during the delay phase when participants are likely to be actively maintaining information – may reflect working memory processes that promote subsequent recall accuracy, given that visuo-spatial working memory is a crucial buffer for accessing and manipulating hippocampal representations (Axmacher et al., 2010; Treisman & Zhang, 2006). Given that our task required repeated study and testing of the same object-location associations, our suggested interaction of frontal and medial temporal systems aligns with prior findings on the testing effect (Wing, Marsh, & Cabeza, 2013) and with effective connectivity analyses in a comparable associative learning paradigm (Banyai et al., 2011).

In contrast to effects seen for within-person changes in EBR, increased grand-mean centered EBR during the encoding epoch (and in early trials in the fixation epoch) predicted poorer behavioral performance. That is, people who blinked more than the group average during the encoding epoch more frequently had lower recall accuracy. A comparable relationship was found in early trials for the fixation epoch. Although at face value, these results appear to conflict with findings that people who blink more relative to their own blinking rate have better recall accuracy (during encoding and delay periods), we consider here the multiple influences on and functions of blink rate. For example, prior studies have found that participants with higher EBR showed reduced cognitive stability or increased distractibility and therefore showed lower accuracy (Dreisbach et al., 2005; Tharp & Pickering, 2011). Relatedly, EBR is also correlated with drowsiness: as individuals become fatigued they blink more (Barbato et al., 2007; Stern et al., 1994); however, these effects are predominately shown by within participant changes. The interaction between grand-mean centered EBR in the fixation epoch and trial may result from increased homogeneity in accuracy across individuals over the course of the experiment. That is, towards the end of the experiment as learning asymptotes, there may not be sufficient variability for EBR to have predictive value. Alternatively, it could be the case that grand-mean centered EBR is indexing processes that are more detrimental to accuracy early in learning.

Inter-individual differences in EBR may also reflect tonic differences between participants in underlying dopaminergic signaling. In line with this interpretation, a prior study showed that increasing tonic dopaminergic signaling (via administration of a dopamine agonist) impaired associative learning (Breitenstein et al., 2006). Our results, therefore may suggest that overall higher levels of dopamine reduce long-term potentiation in the hippocampus by changing the balance between D1/D2 receptor modulation of AMPA channel insertion into glutamatergic neurons (Breitenstein et al., 2006; Sun, Zhao, & Wolf, 2005; Wolf, Mangiavacchi, & Sun, 2003). Furthermore, prior work using a delayed spatial memory task, similar to the task used in the current study, showed that increased EBR was associated with decreased working memory accuracy in a load dependent fashion (Lee et al., 2018). Assessing fatigue and collecting independent working memory assessments in future studies may help tease apart these competing interpretations.

Finally, increased visual scanning relative to individual’s own average in the fixation and delay epochs was associated with increased accuracy early in the experiment. These effects are reduced across trials with no effect or a negative relationship by the final trial. This result aligns with literature suggesting that visual rehearsal during delay periods is associated with improved performance on spatial memory tasks (Brandt & Stark, 1997; Brockmole & Irwin, 2005; Laeng & Teodorescu, 2002; Spivey & Geng, 2001). For example, Brockmole and Irwin (2005) identified differences in accuracy across participants in integrating information displayed in dynamic locations on the screen. These differences were related to participant’s eye movements during a delay, implying that differences in visuo-spatial rehearsal supported accurate performance. Our finding that distance between fixations during the delay period was related to accuracy – especially at the beginning of the experiment – raises the possibility that active rehearsal of the display during the delay period facilitated learning, an inference consistent with in vivo fMRI studies of functional connectivity during these “passive” consolidation epochs (Ravishankar et al., 2019). Increased visual scanning at the end of the experiment may occur more in participants struggling to maintain all location object associations who in turn have lower accuracy.

We did not observe a relationship between pupil diameter and associative learning performance despite prior literature documenting the relationship between pupil size and cognitive performance (Colman & Paivio, 1970; Goldinger & Papesh, 2012; Heaver & Hutton, 2011; Pajkossy & Racsmány, 2020; Papesh et al., 2012; van Rijn et al., 2012). This null finding may be explained by several possible factors. First, the relationship between pupil size and recall accuracy may be nonlinear, such that both very large and very small pupil diameters, reflecting very high and very low arousal, respectively, may be associated with performance decrements. We did not test for a quadratic relationship, which may explain the null finding. Alternatively, after partialling out the variance related to trial and the parallel change in accuracy, there may have been limited variance to be potentially explained by pupil size.

Some limitations constrain our interpretations. First, we were only able to reliably quantify pupil diameter during the pre-trial fixation and delay period when the eye was stable and centrally fixated. During the other task epochs, eye movements, which confound measures of pupil diameter when using video-based eye trackers, were directed across the screen. As a result, we are not able to capture phasic pupillary responses to task stimuli or look at pupil diameter during encoding or retrieval epochs. Although we removed pupil measures during blinks and saccades in the fixation and delay periods, the persistent effects of these ocular events over time may have obscured an effect related to cognition (Knapen et al., 2016). Second, our sample was not balanced across gender (with more women participants). Although we did not observe any significant gender differences in ocular measures, or any differences between women who were and were not using birth control², the imbalance could still be potentially problematic. First, there are reported gender differences in EBR, with women blinking more than men, however, more have found otherwise (Jongkees & Colzato, 2016). Furthermore, we did not account for women’s menstrual cycles or hormonal birth control either of which may affect EBR, given that D2 receptor availability covaries with estrogen levels (Yolton et al., 1994). Third, the link between EBR and dopamine activity remains an ongoing area of research. Two recent positron emission tomography (PET) studies have advised caution when inferring dopamine activity from resting EBR (Dang et al., 2017; Sescousse et al., 2018). Their results cast doubt on the relationship between EBR and striatal dopamine function at rest, rather than cases where dopaminergic signaling is induced by experimental demands, which consequently influences EBR. However, the link between such task-related EBR changes and dopamine is a claim largely reliant on behavioral rather than neural data. Fourth, the timeframe over which our blink rates were calculated (on the order of seconds) may have compromised the reliability of these measurements (Jongkees & Colzato, 2016). Finally, there is evidence for non-linear relationships between cognitive task performance and neuromodulators indexed by pupil diameter (Aston-Jones & Cohen, 2005) and EBR (Akbari Chermahini & Hommel, 2012; Cools & D’Esposito, 2011), such that intermediate values predict optimal cognitive performance under certain sets of demands. Our models unfortunately could not accommodate these quadratic effects, which may explain the lack of significant effects of pupil diameter on recall accuracy. Non-linear effects of ocular metrics on performance should be tested in future studies with larger sample sizes.

In conclusion, we found that eye blink rate and visual scanning behavior during specific task epochs predicted recall accuracy on a paired-associate learning task. Although blinking more relative to others, was related to worse accuracy, blinking more relative to one’s own blink rate (during encoding and delay periods) related to better accuracy. Further we found that visual scanning during the fixation and delay was related to accuracy early in the experiment suggesting that visuo-spatial rehearsal may benefit learning in this type of paired object-location associative learning. The link between eye blink rate and dopaminergic activity permits mechanistic explanations at the level of neuromodulation, and thus, our results provide novel insight into the temporal dynamics of dopamine during associative learning. These findings lay the framework for future study in neuropsychiatric populations to gain insight into the nature of associative learning dysfunction.

Highlights.

• Associative learning is fundamental to cognition.

• Eye blink rate and pupil diameter index dopamine and noradrenaline activity.

• Blinking more relative to others during pre-trial fixation and encoding predicts worse accuracy.

• Blinking more than one’s own average during the encoding and delay predicts better accuracy.

• Increased visual scanning during fixation and delay predicts better performance.