Interplay between components of pupil-linked phasic arousal and its role in driving behavioral choice in Go/No-Go perceptual decision-making

Abstract

In decision-making tasks, neural circuits involved in different aspects of information processing may activate the central arousal system, likely through their interconnection with brainstem arousal nuclei, collectively contributing to the observed pupil-linked phasic arousal. However, the individual components of the phasic arousal associated with different elements of information processing and their effects on behavior remain little known. In this study, we used machine learning techniques to decompose pupil-linked phasic arousal evoked by different components of information processing in rats performing a Go/No-Go perceptual decision-making task. We found that phasic arousal evoked by stimulus encoding was larger for the Go stimulus than the No-Go stimulus. For each session, the separation between distributions of phasic arousal evoked by the Go and by the No-Go stimulus was predictive of perceptual performance. The separation between distributions of decision-formation-evoked arousal on correct and incorrect trials was correlated with decision criterion but not perceptual performance. When a Go stimulus was presented, the action of go was primarily determined by the phasic arousal evoked by stimulus encoding. On the contrary, when a No-Go stimulus was presented, the action of go was determined by phasic arousal elicited by both stimulus encoding and decision formation. Drift diffusion modeling revealed that the four model parameters were better accounted for when phasic arousal elicited by both stimulus encoding and decision formation was considered. These results suggest that the interplay between phasic arousal evoked by both stimulus encoding and decision formation has important functional consequences on forming behavioral choice in perceptual decision-making.

AUTHOR SUMMARY

Pupil size has been used as a reliable, noninvasive index of arousal, able to account for variability in both neural activity and behavioral performance. Recent studies have shown that phasic arousal indexed by pupil size is driven throughout the decision-making process, modulating the perceptual interpretation of sensory input. However, this phasic arousal may reflect the summation of several, superimposed arousal events evoked by different information processing components during decision-making. To better understand the individual components of phasic arousal evoked by different elements of information processing and their functional consequences, we used machine learning to disentangle task-evoked pupil responses into their elementary components while simultaneously learning the contribution of each underlying process on a trial-by-trial basis. For the first time, we were able to look at the relationships between the strengths of the unique elementary components and behavioral performance on a trial-by-trial basis. We discovered that these components better explained behavioral variability and found complex interplay between phasic pupil-linked arousals evoked by stimulus encoding and decision formation drove behavioral choice. Taken together, our findings show that machine learning allowed us to unveil otherwise hidden aspects of the task-evoked pupil response. By deconvolving the task-evoked pupil response, we disentangled independent information streams that allowed us to account for more variability in the decision-making process. This work may lead to new technology for noninvasive measurement of activation of separate neural circuits which are responsible for processing different aspects of information during decision-making.

1. INTRODUCTION

Even within an unchanging, fully learned environment, skilled decision makers often execute different actions upon encountering identical stimuli. This behavioral variability may be partially accounted for by considering the fluctuating arousal levels of the decision maker and how these fluctuations influence sensory processing, perception, decision-making, and behavior (Harris & Thiele, 2011; S.-H. Lee & Dan, 2012; McGinley, Vinck, et al., 2015; Steriade, McCormick, & Sejnowski, 1993; Toader et al., 2019; Wekselblatt & Niell, 2015). Changes in arousal are associated with widespread changes in brain activity and modifications in behavior (Cano, Bezdudnaya, Swadlow, & Alonso, 2006; J. W. de Gee et al., 2017; Ebitz & Platt, 2015; Eldar, Cohen, & Niv, 2013; Niell & Stryker, 2010; Polack, Friedman, & Golshani, 2013; Poulet & Petersen, 2008; Rodenkirch, Liu, Schriver, & Wang, 2019; Rodenkirch & Wang, 2020; Schriver, Bagdasarov, & Wang, 2018), and have been shown to be largely regulated by neuromodulatory systems, including the locus coeruleus-norepinephrine (LC-NE) system (Aston-Jones & Cohen, 2005; Berridge & Waterhouse, 2003; Breton-Provencher & Sur, 2019; Carter et al., 2010; Liu, Rodenkirch, Moskowitz, Schriver, & Wang, 2017; Sara & Bouret, 2012) and the cholinergic systems (S.-H. Lee & Dan, 2012; Nelson & Mooney, 2016; J. Reimer et al., 2016).

Recent work has established the tight link between nonluminance-mediated changes in pupil size and the level of arousal (McGinley, David, & McCormick, 2015; Jacob Reimer et al., 2014; Vinck, Batista-Brito, Knoblich, & Cardin, 2015). For instance, Jacob Reimer et al. (2014) found that the intracellular membrane potential of cortical neurons was desynchronized and sensory evoked responses were increased during pupil dilation in rodents. The causal relationship between LC activation and pupil dilation has been demonstrated in several recent studies (Joshi, Li, Kalwani, & Gold, 2016; Liu et al., 2017; J. Reimer et al., 2016). In addition, LC activation also desynchronized cortical EEG, rendering the cortex in a more activated state (Liu et al., 2017; Vazey & Aston-Jones, 2014).

In a Go/No-Go perceptual decision-making task, information is processed at several layers of the hierarchy before behavioral action. The sensory stimulus is first processed at early stages of the sensory pathway before information is sent to higher order brain regions for cognitive processing to form a decision, a commitment to a plan of action (Brody & Hanks, 2016; Gold & Shadlen, 2007; Philiastides, Auksztulewicz, Heekeren, & Blankenburg, 2011). Motor related brain areas are subsequently engaged to execute or withhold a motor action according to the plan, to indicate choice (Roger Ratcliff, Huang-Pollock, & McKoon, 2016; Simmonds, Pekar, & Mostofsky, 2008). Due to the reciprocal nature of the connections between neuromodulatory nuclei and many brain regions, the activation of neural circuits responsible for processing the information may result in phasic arousal, indexed by rapid tasked-evoked pupil dilation as reported in literature (Aston-Jones & Cohen, 2005; Breton-Provencher & Sur, 2019; Jan Willem de Gee, Knapen, & Donner, 2014; Denison, Parker, & Carrasco, 2019; Hong, Walz, & Sajda, 2014; C. R. Lee & Margolis, 2016; Schriver et al., 2018; Schwarz & Luo, 2015). Phasic arousal elicited by the task has been reported to have functional consequences on cognitive processing (J. W. de Gee et al., 2019; Nassar et al., 2012). For example, J. W. de Gee et al. (2019) found that phasic arousal suppressed decision bias in both humans and mice. However, the individual contribution of the different components of information processing to the observed phasic arousal, and their functional consequence on behavior remain poorly understood.

In the present study, we trained head-fixed rats to perform a Go/No-Go perceptual decision-making task, while their left pupil was imaged throughout the tasks. We observed phasic pupil dilation in trials with all four possible behavioral outcomes (i.e. hit, miss, false alarm, and correct rejection), suggesting information processing evoked phasic arousal during the task. Because the task-evoked phasic arousal could result from processes of stimulus encoding, decision formation, motor execution, response inhibition, positive feedback, negative feedback, or a combination of these, we used machine learning techniques to decompose phasic arousal associated with these six information processing components in the perceptual decision-making task. In addition to the kernel of phasic arousal associated with the components, this method allowed us to learn the weight of the phasic arousal for each trial to account for fluctuations of phasic arousal on a trial-by-trial basis. We found that phasic arousal evoked by stimulus encoding was predictive of perceptual performance, while phasic arousal evoked by decision formation not correlated with perceptual performance, but was predictive of decision criterion. When a Go stimulus was presented, the action of go was primarily determined by the phasic arousal evoked to stimulus encoding. On the contrary, when a No-Go stimulus was presented, the action of go was determined by phasic arousal elicited by both stimulus encoding and decision formation. Drift diffusion modelling revealed that the four model parameters were better accounted for when phasic arousal in response to both stimulus encoding and decision formation was considered. Taken together, our results suggest that the interplay between phasic arousal evoked by both stimulus encoding and decision formation has important functional consequences on forming behavioral choice in perceptual decision-making tasks.

2. METHOD

2.1. Ethics statement

All experimental procedures involving animals were approved by the Columbia University Institutional Animal Care and Use Committee (protocol number: AAAP9636) and were conducted in compliance with NIH guidelines. Behavioral studies were conducted using 8 female albino rats (Sprague-Dawley, Charles River Laboratories, Wilmington, MA; ~225 – 275 g at time of implantation). Animals were single housed after implantation in a dedicated housing facility that maintained a twelve-hour light and dark cycle. All behavioral tasks were conducted during the light phase in a dedicated behavioral training chamber in which ambient luminance was kept constant during the task.

2.2. Surgical implantation and behavioral procedures

Implantation of a metal head-plate was similar to that described in detail in previous work (Bari, Ollerenshaw, Millard, Wang, & Stanley, 2013; D. R. Ollerenshaw et al., 2012; Douglas R. Ollerenshaw, Zheng, Millard, Wang, & Stanley, 2014; Schriver et al., 2018). In brief, following 7–10 days of habituation to experimenters, an aseptic surgery utilizing a ketamine/xylazine cocktail (80/5 mg/kg) or Isoflurane (1–5%) as anesthesia was conducted. A custom-made metal head-plate was bonded to stainless steel screws anchored in skull using dental cement.

Following 10 days of recovery from implantation surgery, which included administration of antibiotics (Baytril, 5 mg/kg) and extra analgesics (ketoprofen, 5 mg/kg) for the first 5 days, rats began water restriction. Rats were then systematically habituated to head-fixation using a custom-built restraint box that had two pneumatic cylinder actuators fixed with ball bearings on either side of the front of the box that fit into grooves in the implanted head-plate and allowed for sturdy quick fixation of an animal. The behavioral apparatus was fully contained within a standard sound and light attenuation chamber (Med Associated, St. Albans, VT).

Following habituation to head-fixation, rats were introduced to the task using a Go/No-Go discrimination paradigm (Figure 1B). The task consisted of whisker deflection direction discrimination indicated by Go/No-Go responses similar to previously described (Schriver et al., 2018). Briefly, an onset cue (6.0 kHz) initialized an interval of variable duration (1.5 – 3.5 s randomly selected from a uniform distribution) preceding presentation of a whisker deflection in either the dorsal or ventral direction. The whisker deflection stimuli were delivered by a multilayer piezoelectric bending actuator (PL140; Physik Instrumente, Karlsruhe, Germany) affixed with a short capillary tube (capillary glass pipette ~15.0 mm long with 1.0 mm outer diameter and 0.5 mm inner diameter; A-M Systems, Carlsborg, WA) bonded to the end of the piezo bending actuator (Zheng, Wang, & Stanley, 2015). For each animal we chose the thickest of the C2, C3, or D2 macro-vibrissae and used this same whisker for all sessions. The whisker was placed into the capillary tube and the end of the capillary tube was placed ~8 mm away from the right snout approximately perpendicular to the midline of the animal so as to not touch any other whiskers. To discourage impulsive licking, during the 1 s preceding stimulus presentation any lick initiated an additional 1.0 – 2.5 s waiting period that was randomly selected from a uniform distribution and that was added to the prestimulus period. Following presentation of the stimulus was a 1.3 s window of opportunity whereupon the animal could indicate a go response by licking a waterspout containing a piezoelectric force sensor. We randomly assigned deflections in the dorsal direction as Go stimuli (S+ stimuli, occurring randomly with 0.5 probability) and deflections in the ventral direction as No-Go stimuli (S− stimuli, occurring randomly with 0.5 probability). Therefore, if the animal decided to lick within 1.3 s following a dorsal direction whisker deflection (i.e. hit response), it was immediately met with a brief reward tone (300 ms, 3 kHz) accompanied by administration of Kool-Aid water (~60 uL) via the waterspout. If the rat licked in the 1.3 s following a ventral direction whisker deflection (i.e. false alarm; FA), a timeout tone (5 s; 16.5 kHz) sounded beginning a 10 s timeout period. When the rat fully withheld a response until the closure of the 1.3 s window of opportunity, given either a dorsal or ventral direction whisker deflection indicating a miss or correct rejection (CR) response outcome, no feedback was given. A 6 s inter-trial period followed administration of the reward in hit trials, the timeout period in FA trials, or the closure of the window of opportunity in miss or CR trials.

Figure 1.

Perceptual decision-making task. A) Experimental set up. B) The diagram of a Go/No-Go perceptual decision-making task that required animals to respond to a Go stimulus while withholding response to a No-Go stimulus. C) Response probability to the Go stimulus was significantly higher than that to the No-Go stimulus across sessions for the animals.

Animals were considered experts once they achieved higher hit rates (HR) than FA rate (FAR) for five sessions in a row. Response rates were computed as the total number of responses given either an S+ or S− stimuli divided by the total number of S+ and S− stimuli within a session respectively. Perceptual sensitivity (d’) and decision criterion (Criterion) were calculated from the observed behavioral responses as follows

$$\begin{gathered} d^{\prime }={\Psi }^{-1}(hitrate)-{\Psi }^{-1}(FArate) \\ Criterion=-\left[{\Psi }^{-1}(hitrate)+{\Psi }^{-1}(FArate)\right]/2 \end{gathered}$$ (1)

where Ψ⁻¹ is the inverse of the cumulative Gaussian distribution.

Pupillometry was measured on randomly selected sessions once animals were considered experts (N = 8 animals, n = 190 total sessions). Notably, performance varied from session to session and on rare occasions even expert rats performed poorly resulting in negative perceptual sensitivity (i.e HR was slightly lower than FAR). The presented figures and analyses include such sessions although results held if we excluded these sessions from analyses.

2.3. Pupillometry and pupillary dynamics

The left eye of the animals was imaged at a frequency of either 10 or 20 Hz using a custom-made pupillometry system. For pupillometry taken at 20 Hz, it was subsequently down sampled to 10 Hz. Therefore, all pupillometry data analyzed in this study were sampled at 10 Hz. For each video clip, a region of interest (ROI) was first manually assigned. The histogram of pixel intensity within the ROI was then calculated to estimate the optimal threshold for pupil segmentation (Liu et al., 2017). Pupil contour was segmented using the threshold and pupil size was defined as the area within the contour. Pupil size within periods of blinks was interpolated using pupil sizes just preceding and after the blinks (Nassar et al., 2012). Pupil size was band-pass filtered (0.01 – 3.75 Hz) before z-scoring for use in the decomposition model. Since baseline fluctuations were central to mediating the relationship between pupil baseline and reaction time, pupil size was z-scored but not band-pass filtered when pupil baseline was used to infer reaction times.

The first 20 trials of each session were excluded due to the time required to adjust the pupillometry camera. For each session pupil sizes were Z-scored by subtracting the mean and dividing by the standard deviation of the pupil size distribution for a given session. To estimate phasic pupil dilation following stimulus presentation, pupil size around stimulus presentation was baseline-corrected by subtracting the mean pupil size within the 0.5 s interval preceding stimulus onset. For correlational analyses, pupil baseline sizes were calculated using this same interval while pupil dilations were computed as the area under the curve in the 4 s following stimulus onset. For visualization these were averaged across all trials and then grand averages were taken across sessions. Additionally, any trials where animals licked preceding stimulus presentation or in trials where the animal licked after the window of opportunity closed were discarded (Schriver et al., 2018).

2.4. Data analysis

All data analyses were first conducted on individual sessions. Averages and standard errors of means were then calculated across sessions for analysis and presentation. The Kolmogorov-Smirnov test was used to assess the normality of the data. If the samples were normally distributed, a Student’s t-test was used. Otherwise, the Mann-Whitney U-test was used for unpaired samples or the Wilcoxon signed-rank test for paired samples. Bonferroni correction was implemented for multiple comparisons.

2.4.1. Inferring No-Go Reaction Times

As the kernel associated with decision formation ends at the reaction times (RT), and kernels associated with motor execution, response inhibition, positive feedback and negative feedback starts at the RT, RTs are required to learn these kernels. However, because the RT of trials on which animals omitted action could not be measured from behavior and these RTs were essential to decomposition of phasic arousal attributed to different information processing components, we used maximum a posteriori (MAP) estimation to infer the RT in trials where the animals fully withheld a response. To this end, we took advantage of the relationship between pupil baseline and RT that has previously been demonstrated (Schriver et al., 2018). For each rat separately we predicted RTs given pupil baseline using Bayesian inference computed from the full RT and pupil baseline given RTs distributions for an animal as (Bishop, 2006)

$$P(RT|PA)\propto P(RT)\ast P(PA|RT)$$ (2)

where RT is reaction time and PA is Z-scored pupil baseline size. For each animal a separate Bayesian model was fit to data across all sessions for computing probabilities based on baseline pupil size given RT. For computing probabilities based on RTs, all RTs were sorted into equally sized bins scaling the range of RTs with bin size of 0.1 seconds. This RT histogram was then smoothed by fitting a χ² probability density function (R. Ratcliff & McKoon, 2008). For computing probabilities of pupil baseline given RT (i.e. P(PA|RT)), a grid was created using the previously described RT bin boundaries as well as 50 equally sized bins scaling the pupil baseline range which single trials were sorted into to create a bivariate histogram. For every trial where the animal fully withheld a response, RT was inferred based on pupil baseline size, which maximized the posterior (RT|PA).

2.4.2. Task-evoked pupil response decomposition model

Previous studies have used one canonical pupil impulse response function (IRF) to represent various underlying cognitive processes. Observed pupil responses were modeled as linear combinations of regressors corresponding to transient events such as the onset of the decision interval, convolved with this singular IRF, which were then individually weighted before summation. The best fitting (beta) weights could then be calculated for each trial, under the assumption that each information processing component was associated with a weighted version of this same IRF.

Unlike these previous studies that have used one general IRF to represent each underlying information processing component, our model simultaneously learned a unique IRF for each of 6 defined information processing components (i.e. stimulus encoding, decision formation, motor execution, response inhibition, positive feedback, and negative feedback) as well as the associated beta weights for each individual trial. Each component regressor was represented as a vector of zeros with nonzero weights only at instances of the transient events, such that each individual trial was represented by the matrix below

$$X^{(n)}=\left[\begin{array}{l}{x}_1^{(n)}\hfill \\ x_2^{(n)}\hfill \\ x_3^{(n)}\hfill \\ x_4^{(n)}\hfill \\ x_5^{(n)}\hfill \\ x_6^{(n)}\hfill \end{array}\right]$$ (3)

where X⁽ⁿ⁾ is a matrix representing trial n of size 6 x T with each row representing one of the 6 components and the columns representing the time of stimulus onset and the following 4 s, sampled at 10 Hz to match the pupillometry sampling rate. The first row corresponded to stimulus encoding and had a nonzero weight time-locked to the moment of stimulus presentation (i.e. a scaled impulse). The second row corresponded to the decision formation component and was modeled as a scaled boxcar initiated at stimulus presentation and terminating at RT. These two components contained nonzero values in all trials regardless of outcome. The following four rows corresponded to the motor execution, response inhibition, positive feedback, and negative feedback respectively, with hit and FA trials having a nonzero weight time-locked to RT in the motor execution row as well as in the index following reaction time in their respective feedback row. RTs inferred from the Bayesian model were incorporated to time lock a nonzero weight in the response inhibition row for both CR and miss trials.

For any single trial, the task-evoked phasic pupil response was modeled as the summation of the time-locked weights that had been convolved with their corresponding kernel according to the equation below

$${\widehat{y}}^{(n)}={\sum }_{i=1}^6\left(x_i^{(n)}\ast k_i\right)$$ (4)

where ${\widehat{y}}^{(n)}$ is the predicted pupil response vector for trial n and k_i is the i^th general kernel vector.

2.4.3. Simultaneous learning of general impulse response functions and individual trial beta weights

To simultaneously learn the time-locked weights of each component for every trial as well as the general kernels they were convolved with for a single session, we minimized the following cost function

$$L\left(K,X^{(1)},X^{(2)},\dots ,X^{(N)}\right)=\Vert Y-\widehat{Y}{\Vert }_F^2$$ (5)

where $Y\in {ℝ}^{N\times T}$ and $\widehat{Y}\in {ℝ}^{N\times T}$ are matrices containing the N respective observed and predicted pupil responses each of length T, while $X^{(n)}\in {ℝ}^{6xT}$ and $K\in {ℝ}^{6xT}$ are the matrices of time-locked weights and kernels respectively. ‖·‖_F is the squared Frobenius norm.

To carry out this optimization and minimize the cost function we used stochastic gradient descent (SGD). This required computing the gradient of the cost function with respect to the time-locked weight matrix as below

$$\frac{\partial L}{\partial X^{(n)}}=2{\sum }_{n=1}^N{\sum }_{t=1}^T\left({\widehat{Y}}_{n,t}-Y_{n,t}\right){\sum }_{i=1}^6{\sum }_{j={\tau }_0(t)}^t{J}_{i,j}{K}_{i,t+1-j}$$ (6)

where $J_{i,j}\in {ℝ}^{6xt}$ is a matrix comprised of all zeros with a one in the i, j-th entry and τ₀(t) = max(t + 1 − T, 1). While the above gradient was used to optimize the time-locked weights, as we additionally wished to learn general IRFs, this also required computing the gradients of the cost function with respect to the kernel matrix as below

$$\frac{\partial L}{\partial K}=2{\sum }_{n=1}^N{\sum }_{t=1}^T\left({\widehat{Y}}_{n,t}-Y_{n,t}\right){\sum }_{i=1}^6{\sum }_{j={\tau }_0(t)}^t{X}_{i,j}^{(n)}{J}_{i,t+1-j}$$ (7)

We initialized all nonzero, time locked values as one and initialized the kernels by choosing each value randomly from a normal distribution with mean zero and standard deviation of 0.1. To implement SGD, we shuffled all trials and then for each iteration ran through all trials computing both gradients with respect to a single trial and subsequently updating the time-locked values of this single trial as well as the general kernel matrix. All values within the time locked value matrix other than the initial nonzero values were forced to remain zero, and after each update iteration the boxcar height was given a value equal to the average across its nonzero component values. Learning rates controlling how much values were updated each iteration were initially set at α_x = 0.05 for updating beta weights and α_k = 0.01 for kernels. Additionally, Nesterov-accelerated adaptive moment estimation algorithm (NADAM) was incorporated in updating learning rate for the kernels, but not the beta weights. Pairing NADAM with SGD was selected to avoid local minima.

To determine when the kernels and beta weights were adequately learned and to avoid overfitting, early stopping was implemented utilizing k-fold cross-validation (Fushiki, 2011; Mosteller & Tukey, 1968). For each session, the trial orders were randomized and then split into 3 partitions. One partition was labeled as a validation set, while two partitions were combined to create the training data set for the model to learn the kernels and weights. For one iteration we shuffled all training data trials and ran through all trials. After a full iteration, consisting of updating weights and kernels for each individual trial within the training data set, we fit the beta weights for the validation data using the kernels learned from the training data set to compute a least squares best fit solution to the equation

$$b=X^{-1}y$$ (8)

where b are the fitted weights, X is a matrix of ones and zeros corresponding to the observed timing of the information processing components and y is the observed pupil response. Additionally, the inverse was computed using the Moore-Penrose pseudoinverse. After each iteration, the best fit validation weights were computed, and then used with the current estimation of the kernels to compute the predicted pupil response for the validation set to compute the mean squared error (MSE) as

$$MSE=\frac{1}{N}\frac{1}{T}{\sum }_{n=1}^N{\sum }_{t=1}^T\Vert Y-\widehat{Y}{\Vert }_F^2$$ (9)

Each iteration the MSE was compared with the previously computed MSE and overfitting was indicated by an increase in MSE as this would imply a step away from a general solution (Yao, Rosasco, & Caponnetto, 2007). To ensure a small increase in MSE was not due to noise, we stopped fitting only after the MSE increased for 3 straight iterations, in which case the kernels and weights from the iteration with the lowest MSE were stored. We then moved onto using each of the other two partitions as a validation set, allowing each partition to have the weights simultaneously learned twice and the kernels learned a total of 3 times. This process was carried out a total of 10 times and the weights and kernels were averaged across them.

2.4.4. Beta weight analysis

The amount of phasic arousal corresponding to stimulus encoding and decision formation were quantified with their beta weights, and were compared across trials within the same session. Correlation analysis between beta weights and other variables were conducted by calculating the Pearson’s correlation coefficient for each session separately and averaging across sessions. To calculate normalized differences in distributions of beta weights associated with stimulus encoding for S+ and S− trials, for each session the normalized difference was calculated as

$$d_{D_1-D_2}^{\prime }=\frac{X_{S+}-X_{S-}}{\sqrt{\frac{{\sigma }_{S+}^2+{\sigma }_{S-}^2}{2}}}$$ (10)

where $d_{D_1-D_2}^{\prime }$ is the normalized difference in distributions, X_S+ and X_S− are the mean of the beta weights of stimulus encoding evoked phasic arousal on S+ and S− trials for a given session respectively, while ${\sigma }_{S+}^2$ and ${\sigma }_{S-}^2$ are the respective variances. Similarly, the normalized differences in distributions of beta weights associated decision formation on correct and incorrect (or S+ and S−) trials was calculated.

2.4.5. Discriminant analysis and bivariate distribution heat map visualization

To determine how to maximally separate responded or withheld trials given the presentation of S+ or S− stimuli (i.e. Hit vs. Miss and FA vs. CR, respectively), by their stimulus encoding and decision formation beta weights, we used linear discriminant analysis (LDA). For each session, we calculated a line which would give the maximum separation of our different response outcomes based on their paired stimulus encoding beta weights and decision formation beta weights when they were projected onto this line. This was achieved by maximizing the distance between the distributions projected means normalized by the within-class scatter of the projected samples calculated as

$$J(w)=\frac{{\left|w\cdot {\mu }_{Responded}-w\cdot {\mu }_{Withheld}\right|}^2}{w^{\prime }\cdot {\sigma }_{Responded}^2+w^{\prime }\cdot {\sigma }_{Withheld}^2}$$ (11)

where μ corresponds to mean and σ² corresponds to variance of the distributions. Allowing stimulus encoding beta weights to be placed along the x-axis and decision formation beta weights to be placed along the y-axis we computed the linear discriminant projection for each session and compared their slopes to a vertical discriminant projection.

To visualize difference between stimulus encoding and decision formation associated weights between behavioral outcomes, data from a single session was sorted into a 5 × 5 bivariate histogram created using equally spaced bin boundaries scaling from 5% quantile to 95% quantile value of the stimulus encoding beta weight distribution in the x direction and similarly for the decision formation associated beta weight distribution in the y direction. For each behavior outcome, we sorted all trials into the grid to create a separate histogram. We then divided the number in each bin by the total number of the respective responses to obtain a probability distribution, and finally subtracted the probability distributions for responded from the probability distributions for withheld trials given stimulus type (i.e. hit probability – miss probability and FA probability – CR probability).

2.4.6. Hierarchical drift-diffusion modeling (HDDM) of decision-making with phasic arousal regressors

To assess the relationship between the trial-by-trial fluctuations in pupil-linked phasic arousal evoked by stimulus encoding and decision formation and the internal components of decision-making, we fit the decision-making process with a hierarchical drift diffusion model (HDDM). To achieve this we fit the drift diffusion model to all of our sessions at once using the HDDM 0.6.0 package implemented in Python (Delis, Dmochowski, Sajda, & Wang, 2018; Wiecki, Sofer, & Frank, 2013). Due to the higher probability of responded than withheld behavioral outcomes we set our upper and lower bounds to be Go and No-Go respectively, and the internal component parameters we included in fitting our HDDM model were nondecision time (t), mean drift-rate (v), distance between decision boundaries (a), starting point (z), and drift bias (db).

Since stimulus encoding and decision formation occurred prior to the animals’ response, we used phasic arousal evoked by these two processes along with the interaction between them as regressors of the decision-making parameters as follows

$$\begin{gathered} a={\alpha }_0+{\alpha }_1\times {\beta }_{SE}\times {\beta }_{DF} \\ v={\gamma }_0+{\gamma }_1\times {\beta }_{SE} \\ db={\delta }_0+{\delta }_1\times {\beta }_{DF} \end{gathered}$$ (12)

Where a, v, and db are decision boundary, drift-rate, and decision bias, respectively, and α_i, γ_i, and δ_i are their respective intercepts (i = 0) and the coefficients (i = 1) weighting the influence of the stimulus encoding beta weight (β_SE), the decision formation beta weight (β_DF), or the interaction between them. We specifically tested models where drift rate, which is tied to the quality of the information extracted from the stimulus, included a regressor on the weight of the phasic arousal associated with stimulus encoding based on the correlation with stimulus identity and perceptual performance. We then created 6 models with regression to identify what the strongest relationships between beta weights of phasic arousal associated with decision formation and with the interaction between them had with decision boundary, starting point, and drift bias, to investigate the association with criterion. We allowed each parameter to only have one dependent to highlight what relationships were strongest and compared deviance information criterion to find the optimal model with the strongest relationships. We additionally, fit 3 models without regression to ensure that performance with regression was favorable validating their relationships. Two models were 4-parameter models, both with nondecision time, decision boundary, and drift rate, and either starting point or drift bias. Additionally, we fit one 5-parameter model containing each of these parameters, which we found favorable to either 4-parameter model but with less favorable DIC than the regression models.

We allowed the model to maintain different parent parameter distributions, for which child session parameter distributions were drawn from and subsequently observed trials were fit. However, only one set of coefficients relating to stimulus encoding and decision formation beta weights was fit for each parameter across all sessions for all subjects. In the Bayesian framework, the prior distributions of the model parameters were updated based on the likelihood of the data given the model and the posterior probability densities of each of these regression coefficients was estimated by having 5500 samples drawn from the posterior and then discarding the first 500 as a burn in and resampling or thinning the remaining data by a factor of 50 as previously described (Delis et al., 2018). The posterior probability densities were graphically represented as violin plots and significant effects were determined when >99% of the posterior density was greater or lower than 0, indicating a positive or negative relationship with the betta weight of a phasic arousal, respectively. Furthermore, we ran posterior predictive simulations from the fit HDDM to visualize similarities in the observed and simulated RT distributions to validate that the fitted model reproduced the observed RT and accuracy patterns.

2.4.7. Variance inflation factor calculation

The variance inflation factors (VIF) for each of our regressors corresponding to the other 6 components was computed as

$$VI{F}_j=\frac{1}{1-R_j^2}$$ (13)

Where $R_j^2$ is the multiple coefficient of determination of the least squares regression equation modelling the regressor j as a function of each of the other regressors, as shown below

$$X_j={\propto }_0+{\propto }_1{X}_1+\cdots +{\propto }_6{X}_6+\epsilon$$ (14)

Where X_j is the j^th regressor, ∝₀ is a constant, and is the error term. Therefore if $R_j^2$ is high, the VIF will be high, indicating that the j^th regressor is a strongly explained by a linear combination of any of the other regressors and thus high collinearity is present and may be a problem.

To explicitly compute the VIFs, for each session we calculated the correlation matrix for the 6 time-locked regressor vectors. We then calculated the inverse of this resulting square correlation matrix. The resulting diagonal elements of the inverse correlation matrix provided the VIFs for each of our 6 regressors, measuring to what extent each was a linear function of the others.

3. RESULTS

We trained head-fixed rats (N = 8) to perform a Go/No-Go direction discrimination task in which the animals were required to respond to whisker deflection in the dorsal direction and withhold responses to whisker deflection in the ventral direction (Figure 1A). On each trial, an onset cue initialized an interval randomly varying from 1.5 – 3.5 s, which concluded with stimulus presentation and the opening of the window of opportunity which lasted 1.3 s (Figure 1B). A Go response indicated by a lick within the window of opportunity following a deflection in the dorsal direction (i.e. Go or S+ stimulus) resulted in a water reward (i.e. hit trial), while a lick response within the window of opportunity following a deflection in the ventral direction (i.e. No-Go or S− stimulus) resulted in a time out period of 10 s (i.e. false alarm (FA) trial), beginning with a high frequency tone (16.5 kHz) lasting 5 s. After the animals became experts in the task, indicated by a higher hit rate than FA rate for 5 sessions in a row, we measured their pupil size during randomly selected sessions. In these sessions (n = 190), the animals maintained significantly higher hit rates than false alarm rates (Figure 1C) (Hit rate = 79 ± 1.2% vs. FA rate = 39 ± 1.5%; Mean ± S.E.M, computed across sessions, unless otherwise noted; p < 9.2 × 10⁻⁴⁵), resulting in a positive perceptual sensitivity of 1.3 ± 0.05. These behavioral results indicated proficiency in discriminating between deflection directions.

3.1. Decomposing phasic arousal evoked by different information processing components

During the task, pupil size of the animals fluctuated across the whole session (Figure 2A). Since we were interested in the task-evoked phasic activation of the pupil-linked arousal system, we subtracted the pupil baseline size (mean of the 0.5 s period prior to stimulus presentation) from the pupil size following stimulus presentation. When aligning pupil size with the onset of stimulus presentation, consistent with previous work (Jan Willem de Gee et al., 2014; C. R. Lee & Margolis, 2016; McGinley, David, et al., 2015; Schriver et al., 2018), we found that phasic pupil dilation was present on trials with all four possible behavioral outcomes (i.e. hit, FA, CR, and miss) (Figure 2B). During a simple Go/No-Go perceptual decision-making task, upon the presentation of the sensory stimulus, the brain initially processes task-relevant sensory information, then forms a decision to commit to either a Go or No-Go action (Carandini & Churchland, 2013; Gold & Shadlen, 2007; Smith & Ratcliff, 2004). For a Go decision, the motor system was activated to plan and execute motor activity to report the decision, while for a No-Go decision, neural circuits responsible for response inhibition were engaged to suppress motor actions (Rubia et al., 2001; Verbruggen & Logan, 2008). Following a Go action, either a positive (reward tone with water reward) or negative feedback (timeout tone followed by a time out period) was provided. All of these processes may have activated the pupil-linked arousal system, collectively contributing to the total phasic arousal measured by phasic pupil dilation that was observed in behavioral tasks.

Figure 2.

Decomposition of phasic arousal evoked by different information processing components. A) Example of phasic pupil dilation. B) Average phasic pupil dilation on trials given different behavioral outcomes. Shaded areas indicate S.E.M. C) Machine learning algorithm was able to simultaneously learn the kernel of phasic arousal associated with the six different information processing components and their beta weights for each trial.

Previous studies have shown that the pupil response can be modeled as a linear superposition of elementary components (Jan Willem de Gee et al., 2014). This led us to further break down and categorize phasic pupil dilation associated with the different components of underlying cognitive processes. However, unlike previous studies which model task-evoked pupil dilation using only one predefined impulse response function invoked by any of their cognitive processes, we simultaneously learned a unique impulse response function (i.e. kernel) for each of 6 defined information processing components (i.e. stimulus encoding, decision formation, motor execution, response inhibition, positive feedback, and negative feedback). Moreover, this method modeled the trial-by-trial variance of phasic arousal associated with the underlying information processing components with beta weights of each kernel for each individual trial (Figure 2C; see Methods). Each contribution was modeled as a weighted transient value. The contribution of stimulus encoding was time-locked to the moment of stimulus presentation, while the contributions of motor execution, response inhibition, and feedback were time-locked to the measured or inferred reaction time (RT; see following paragraph). The contribution of decision formation was modeled as a sustained boxcar initiated at stimulus presentation and terminating at RT (Jan Willem de Gee et al., 2014).

Since we used a Go/No-Go discrimination paradigm, RT could not be directly quantified by behavioral outputs on trials where subjects were required to withhold a response. However, as we assumed that there was a decision that the animal made to not go, similar to the decision to go, it is plausible to assume that there was an unobserved RT. Previous work has shown that RT covaried with baseline pupil size in a Go/No-Go discrimination task (McGinley, David, et al., 2015; Schriver et al., 2018). Therefore, it may be possible to infer this hidden RT from pupil baseline for both CR and miss trials. For each subject individually, we first estimated the likelihood P(Baseline pupil size | RT) and prior distribution P(RT) using data from hit and FA trials, then used a maximum a posteriori estimation to infer the RT from the measured baseline pupil size (see Methods).

To estimate the pupil response kernels associated with each of the six processes, stochastic gradient descent was employed to simultaneously learn from the task-evoked pupil dilations and behavioral outputs the kernels as well as the time-locked weights of the components for each trial (Figure 2C). For each session, this learning was implemented in two-thirds of randomly selected trials while the rest of trials were used to quantify the fitting errors of the learned kernels and beta weights to ensure overfitting did not occur (Fushiki, 2011; Lever, Krzywinski, & Altman, 2016; Mosteller & Tukey, 1968) (see Methods). The statistical modeling learned kernels with unique shapes associated with each information processing component, suggesting that the defined information processing components did not invoke the same response reflected in pupil size (Figure 3A). There were also differences in the shapes of the responses, with the stimulus encoding component having a quick, sharp response and the Go, No-Go, and both feedback components having longer, sustained responses. The decision formation component also had a somewhat sustained response that was smaller in amplitude than the other components, but this is at least in part due to the fact that this component was convolved with a boxcar in the model.

Figure 3.

Estimated phasic arousal evoked by different information processing components. A) Average kernels of phasic arousal evoked by the six information processing components during perceptual decision-making. B) Example of the measured and reconstructed pupil dilation on a hit, CR, miss, and FA trial. C) Pearson correlation coefficient between the measured and reconstructed pupil dilation for all trials given the four behavioral outcomes. D) Trial-by-trial correlation between pupil dilation and the beta weights of phasic arousals associated with the 6 information processing components. E) Pearson correlation coefficient between the measured and reconstructed pupil dilation using a single learned or canonical kernel for all information processing components. F) Variance inflation factor of the 6 regressors. G) Correlation matrix of phasic arousals associated with the 6 information processing components. **: p < 0.001. Error bar and shaded areas indicate S.E.M.

To validate that the learned kernels collectively contributed to the pupil phasic dilations, we first reconstructed pupil responses using learned kernels and corresponding beta weights, then calculated the similarity between the reconstructed pupil dilation and measured pupil dilation on a trial-by-trial basis (Figure 3B). We found that the weighted summation of kernels associated with different information processing components accounted for much of the variance in measured phasic pupil responses as the Pearson correlation coefficient between the reconstructed and measured pupil dilation was greater than .6 for all trials given the different behavioral outcomes (Figure 3C; hit: .79 ± 0.01, p < 1.4 × 10⁻³³; CR: .63 ± 0.01, p < 1.4 × 10⁻³³; FA: .72 ± 0.01, p < 1.4 × 10⁻³³; miss: .63 ± 0.01, p < 6.6 × 10⁻³³. rank-sum test). To further validate our machine learning algorithms, we performed a control analysis in which we used our algorithms to learn the 6 kernels and corresponding beta weights from reconstructed pupil dilation. The newly-learned kernels and corresponding beta weights were almost identical to the kernels and beta weights used to reconstructed pupil dilation with PCC > .97 ± 0.0029 (Supplementary figure 1). Moreover, the beta weights for 5 of 6 kernels were significantly either positively or negatively correlated with phasic pupil dilation (Figure 3D. Difference between 0: decision formation, p < 1.3 × 10⁻¹⁶; motor execution, p < 5.0 × 10⁻²³; response inhibition, p < 1.9 × 10⁻¹⁶; positive feedback, p < 3.4 × 10⁻²⁷; negative feedback, p < 3.9 × 10⁻⁹; rank-sum test). Together, these results suggested the machine learning algorithms were able to learn elements of phasic pupil responses associated with the 6 information processing components in the task.

We also compared our 6-kernel model with a previous model which used a single canonical kernel (Hoeks & Levelt, 1993), and found the correlation between measured and reconstructed pupil dilation was significantly lower for the canonical kernel model (p < 1.4 × 10⁻³³, Figure 3E). Additionally, we fit a model that for each session simultaneously learned a single kernel (i.e. the same used for each component), along with the trial by trial weights, and found the correlation between measured and reconstructed pupil dilation was higher than the canonical kernel model (p < 2.4 × 10⁻¹⁴), but still significantly lower than for the 6-kernel model (p < 1.4 × 10⁻³³), indicating our model better captures pupil dilation corresponding to different information processing components.

Since we used 6 regressors in our model and high collinearity between regressors can lead to highly variable model fitting, to ensure this was not the case, we first calculated the variance inflation factors (VIF) for each of our regressors (O’brien, 2007). We found that the average VIFs were far below the conservative threshold of 5 (Figure 3F. Stimulus encoding VIF = 1.3020 ± 0.0085, Decision formation VIF = 1.5934 ± 0.0168, Motor execution VIF = 1.1836 ± 0.0055, Response inhibition VIF = 1.1217 ± 0.0069, Positive feedback VIF = 1.0016 ± 0.00004, Negative feedback VIF = 1.0008 ± 0.00003). We further computed the correlation matrix of pupil dilations associated with the 6 information processing components. Consistent with VIF calculations, there was no significant correlation between pupil dilations associated with any two components (Figure 3G), suggesting the collinearity of regressors in our model was unlikely to confound the results presented here.

3.2. Phasic arousal evoked by stimulus encoding, but not decision formation, was correlated with perceptual performance

In each session, in addition to the unique kernels of phasic pupil dilation corresponding to each information processing component, our machine learning algorithm also learned the beta weights of these kernels for each trial. Importantly, this allowed us to compare the amount of evoked phasic arousal associated with each of the 6 information processing components between trials with different stimuli or behavioral outcomes. Consistent with previous work, tonic arousal, indexed by pupil baseline size, and phasic arousal in response to stimulus encoding and decision formation fluctuated throughout the entire session (Figure 4A) (Schriver et al., 2018). Pearson correlation analysis revealed that phasic arousal evoked by stimulus encoding was positively correlated with pupil baseline while phasic arousal evoked by decision formation was negatively correlated with pupil baseline (Figure 4B, p < 7.8 × 10⁻²², signed-rank test), indicating that tonic arousal had profound effects on phasic arousal elicited by processing of task-related information (Nassar et al., 2012; Schriver et al., 2018).

Figure 4.

Phasic arousal evoked by stimulus encoding was predictive of perceptual performance. A) Example of tonic arousal indexed by pupil baseline size and phasic arousal evoked by both stimulus encoding and decision formation that fluctuated throughout a session. B) Phasic arousal evoked by stimulus encoding was positively correlated with tonic arousal while phasic arousal evoked by decision formation was negatively correlated with tonic arousal. C) Phasic arousal evoked by stimulus encoding on trials with different behavioral outcomes and on responded and withheld trials. D) Phasic arousal evoked by stimulus encoding was higher for S+ trials than for S− trials. E) Example distribution of stimulus encoding evoked phasic arousal in response to S+ and S− stimuli in a session. F) Variances of stimulus encoding evoked phasic arousal in response to S+ and S− stimuli were not different. G) Separation between stimulus encoding evoked phasic arousal between S+ and S− stimuli was predictive of perceptual performance. H) Stimulus encoding evoked phasic arousal was slightly higher on correct trials than incorrect trials. I) Variances of stimulus encoding evoked phasic arousal on correct and incorrect trials were not different. J) Separation between stimulus encoding evoked phasic arousal between correct and incorrect trials was not correlated with perceptual performance. Error bars indicate S.E.M.

We also found phasic arousal evoked by stimulus encoding was higher on hit and FA trials than CR and miss trials, resulting in significantly higher arousal in responded trials than withheld trials (Figure 4C, 0.94 ± 6.3 × 10⁻³ vs.0.76 ± 9.6 × 10⁻³, p < 1.0 × 10⁻²⁹, signed-rank test). Interestingly, phasic arousal resulting from the Go stimulus was significantly higher than that evoked by the No-Go stimulus despite the fact that the only difference between the Go and No-Go stimuli was the direction, suggesting the central arousal circuitry was tuned for the target stimulus (Figure 4D, 0.93 ± 0.0064 vs. 0.81 ± 0.0074, p < 6.7 × 10⁻²⁵, signed-rank test). The variance of phasic arousal evoked by the Go stimulus and No-Go stimulus varied across sessions. However, there was no significant difference between the variance of phasic arousal evoked by the Go stimulus and No-Go stimulus for each session (Figure 4 E&F, 0.35 ± 0.02 vs. 0.37 ± 0.03, p = .93, signed-rank test). We further tested if the difference between phasic arousal evoked by target and nontarget stimuli was predictive of behavioral performance. To this end, we first quantified the difference between distributions of phasic arousal evoked by the Go and No-Go stimuli for each session by normalizing the difference between the means with averaged variances (see Methods). This normalized difference between the Go-stimulus-evoked arousal and No-Go-stimulus-evoked arousal distributions was found to be positively correlated with the animals’ perceptual sensitivity in the perceptual discrimination task (Figure 4G, p < 1.2 × 10⁻⁸), suggesting that phasic arousal elicited by stimulus encoding exerted strong influences on perceptual performance.

Since phasic arousal evoked by stimulus encoding may subsequently affect cognitive processing, we next examined the extent to which the difference between stimulus encoding evoked phasic arousal on correct and incorrect trials was predictive of perceptual performance. Phasic arousal in response to stimulus encoding on correct trials was slightly but statistically higher than that on incorrect trials (Figure 4H, 0.88 ± 0.0059 vs. 0.85 ± 0.0083, p = 1.0 × 10⁻⁵, signed-rank test) although there was no significant difference in its variance between the two types of trials (Figure 4I, 0.36 ± 0.02 vs. 0.37 ± 0.02, p =.35, signed-rank test test). However, the normalized difference between the distributions of stimulus encoding evoked phasic arousal on correct and incorrect trials was not correlated with perceptual performance (Figure 4J, p = .86).

Since in this perceptual task the animals needed to cognitively process information to form an optimal decision to maximize reward, we then assessed if phasic arousal evoked by decision formation modulated perceptual performance. The difference between decision formation evoked phasic arousal was relatively small across all behavioral conditions. The decision formation evoked phasic arousal was 0.91 ± 0.0048 vs. 0.91 ± 0.0082 for responded trials and withheld trials (Figure 5A, p = .29, signed-rank test), and 0.14 ± 0.0093 vs. 0.16 ± 0.01 for S+ and S− trials (Figure 5B, p < 4.62 × 10⁻⁸, signed-rank test). We also found that the normalized difference in decision formation evoked phasic arousal between S+ and S− trials was not predictive of perceptual sensitivity across all sessions (Figure 5C, p = .084). Although there was a small but significant difference in decision formation evoked phasic arousal between correct and incorrect trials (Figure 5D, 0.92 ± 0.0044 vs. 0.91 ± 0.0067, p = 6.3 × 10⁻³, signed-rank test), the normalized difference in decision formation evoked phasic arousal between correct and incorrect trials was not predictive of perceptual sensitivity across all sessions (Figure 5E, p = .08).

Figure 5.

Phasic arousal evoked by decision formation was not correlated with perceptual performance. A) Phasic arousal evoked by decision formation on trials with different behavioral outcomes and on responded and withheld trials. B) Phasic arousal evoked by decision formation was higher for S+ trials than S− trials. C) Separation between decision formation evoked phasic arousal between S+ and S− stimuli was not correlated with perceptual performance. D) Phasic arousal evoked by decision formation on correct trials and incorrect trials. E) Separation between decision formation evoked phasic arousal between correct and incorrect trials was not correlated with perceptual performance. Error bars indicate S.E.M.

Within the signal detection theory framework, the animals’ perceptual behavior was modeled by perceptual sensitivity, which is the discriminability between the Go and No-Go stimuli to the animal and thus a measure of perceptual performance, and decision criterion, which is an index of the tendency of the animal to choose one action versus the other. Although our data showed that the phasic arousal evoked by decision formation had no effect on perceptual sensitivity, it does not rule out the possibility that the phasic arousal evoked by decision formation modulated the decision criterion. To assess this possibility, we plotted the normalized difference in the phasic arousal evoked by decision formation between correct and incorrect trials against decision criterion for each session. We found that the separation of decision formation evoked phasic arousal between correct and incorrect trials was negatively correlated with the decision criterion, indicating that the greater the difference between the two types of trials, the more liberal the animal was in its decisions (Figure 6A). Interestingly, our data also demonstrated a negative correlation between the normalized difference in the phasic arousal evoked by stimulus encoding between S+ and S− trials against decision criterion, suggesting that phasic arousal in response to stimulus encoding contributed more to the animal’s perceptual behavior than that in response to decision formation (Figure 6B). To further evaluate this, we plotted the probability of action (i.e. animal deciding to lick) as a function of phasic arousals in response to stimulus encoding and decision formation. When S+ was presented, the animal’s action led to a hit while no-action resulted in a miss. Similarly, when S− was presented, the animal’s action led to a FA while no-action resulted in a CR. To quantify the contribution of phasic arousal in response to stimulus encoding and decision formation to the animal’s choice to respond when either S+ or S− was presented, we utilized linear discriminant analysis (LDA) to evaluate an optimal boundary to separate Go from No-Go actions (i.e. hit or miss when S+ was presented, or CR and FA when S− was presented) for each session (see Methods). Interestingly, we found that the angle of LDA boundaries was not significantly different from 90° when S+ was presented, suggesting that the animal’s decision to go was solely dependent upon phasic arousal evoked by stimulus encoding. On the contrary, when S− was presented, phasic arousal evoked by both stimulus encoding and decision formation contributed to the subject’s Go-action as the angles of the LDA boundary were slightly but significantly greater than 90°, suggesting that high arousal evoked by decision formation could make a small contribution to the animal’s decision to go. Taken together, our results suggest that the interplay between the two type of phasic arousal determined the animals’ behavioral choice in the perceptual task. s

Figure 6.

Interplay between phasic arousal evoked by stimulus encoding and decision formation in driving animals to respond. A) Separation between decision formation evoked phasic arousal between correct and incorrect trials was predictive of decision criterion. B) Separation between stimulus encoding evoked phasic arousal between S+ and S− stimuli was predictive of decision criterion. C) When an S+ stimulus was presented, animals’ response was determined by phasic arousal evoked by stimulus encoding. D) When an S− stimulus was presented, animals’ response was collectively determined by phasic arousal evoked by both stimulus encoding and decision formation. E) The angle of optimal LDA boundary separating animals’ behavior of go and no-go in panels C&D. Error bars indicate S.E.M.

3.3. HDDM modeling confirmed the interplay between phasic arousal associated with stimulus encoding and decision formation in all aspects of decision-making

To further examine this notion, we utilized hierarchical drift diffusion model (HDDM) analysis to probe the functional relationship between the phasic arousals and the constituent processes of decision-making in our perceptual decision-making tasks (Figure 7A). Our data demonstrated that the task performance data were fit well by the HDDM with trial-dependent nondecision time, decision boundary, drift rate, starting point bias, and drift bias, evidenced by the similarity between measured and model-simulated RT distributions (Figure 7B, Pearson correlation coefficient = .87), indicating that the HDDM model could explain behavior during the task. To evaluate if these processes bore any relation to the phasic arousal evoked by stimulus encoding, decision formation, and the interplay between them, we integrated the HDDM with a regression analysis that used the individual arousal components as predictors for the HDDM parameters. The HDDM framework therefore provided a principled approach to investigate whether the phasic arousal components drove decision-making in expected ways and allowed us to identify which processes may be predictive of behavior.

Figure 7.

Modeling the interplay between phasic arousal evoked by stimulus encoding and decision formation using a hierarchical drift diffusion model. A) Graphical model showing hierarchical estimation of drift diffusion model parameters with phasic arousal regressors. B) RT distributions were captured by the HDDM. Negative values in the time axis correspond to No-Go choices and positive values represent Go choices. C) Comparison with alternate models. A DIC_model -DIC_optimal value of >0 indicates that the model of choice achieved a better trade-off between goodness-of-fit and number of free parameters than alternate models. D-H) Violin plots showing the distribution of the regression coefficients of the two phasic arousals for the prediction of nondecision times, decision boundary, drift rate, starting point, and drift bias, respectively, on a trial-by-trial basis.

The HDDM model with phasic arousal regressors of the drift rate, decision boundary, and drift bias provided a better trade-off between goodness-of-fit and complexity (as assessed by the Deviance Information Criterion - DIC for model selection (Spiegelhalter, Best, Carlin, & van der Linde, 2002); see Methods) compared to alternative HDDM models (see Figure 7C for DIC comparisons). Specifically, we wanted to validate that drift rate, which is tied to the quality of the information extracted from the stimulus (R. Ratcliff & McKoon, 2008), was dependent on the weighted phasic arousal associated with stimulus encoding due to the to the previously examined correlation with stimulus identity and perceptual performance. Allowing drift rate to be dependent on the weighted phasic arousal associated with stimulus encoding, we then sought to find which combination of allowing decision boundary, starting point, or drift bias with a dependence on either the weighted phasic arousal associated with decision formation or the interaction between them provided the most favorable DIC, as this is in line with our previous findings of the decision formation being associated with criterion. We allowed each parameter to only have one dependent to highlight what relationships were strongest.

Specifically, in the model of choice, drift rate was dependent on the weighted phasic arousal associated with stimulus encoding, drift bias was dependent on the weighted phasic arousal associated with decision formation, and decision boundary was dependent on the interaction between them (see Eq. 12 in Methods). This model provided a better fit of the task performance data than a) a 4-parameter model with nondecision time, decision boundary, drift rate, and starting point that did not include regressors, b) a 4 parameter model with nondecision time, decision boundary, drift rate, and drift bias that did not include regressors, c) a 5 parameter model that did not include regressors, d) a model with drift rate dependent on the weighted phasic arousal associated with stimulus encoding, decision boundary dependent on the weighted phasic arousal associated with decision formation, and starting point dependent on the interaction between the two phasic arousals, e) a model with drift rate dependent on the weighted phasic arousal associated with stimulus encoding, decision boundary dependent on the weighted phasic arousal associated with decision formation, and drift bias dependent on the interaction between the two phasic arousals, f) a model with drift rate dependent on the weighted phasic arousal associated with stimulus encoding, starting point dependent on the weighted phasic arousal associated with decision formation, and decision boundary dependent on the interaction between the two phasic arousals (not shown as HDDM model did not converge), g) a model with drift rate dependent on the weighted phasic arousal associated with stimulus encoding, starting point dependent on the weighted phasic arousal associated with decision formation, and drift bias dependent on the interaction between the two phasic arousals (not shown as HDDM model did not converge), h) a model with drift rate dependent on the weighted phasic arousal associated with stimulus encoding, drift bias dependent on the weighted phasic arousal associated with decision formation, and starting point dependent on the interaction between the two phasic arousals. Therefore, we deduced that using phasic arousal as predictors of single-trial nondecision times and drift rates yielded better HDDM model performance.

Having modeled the perceptual decision-making process with HDDM, we found that all three phasic arousal regressors were significantly predictive of an HDDM model parameters, specifically drift rate was dependent on the weighted phasic arousal associated with stimulus encoding, drift bias was dependent on the weighted phasic arousal associated with decision formation, and starting point was dependent on the interaction between the two phasic arousals. Interestingly, our data illustrated a complex interplay between phasic arousal and decision formation, as decision boundary was positively correlated with phasic arousals evoked by the interaction between the two phasic arousals (0.026, p < .01) (Figure 7E). The drift rate was positively correlated with phasic arousal evoked by stimulus encoding (0.023, p < .01), indicating that the higher the phasic arousal associated with sensory encoding the faster the evidence accumulation during decision-making in the task (Figure 7F). The drift bias was positively correlated with phasic arousal evoked by decision formation (0.435, p <.01) (Figure 7H). As a positive drift bias is linked to the animals’ tendency to respond, this positive correlation indicated high phasic arousal associated with decision formation was likely to lead to a choice of action. Taken together, these modeling results further confirmed that the phasic arousals evoked by both stimulus encoding and decision formation contributed to the different processes of decision-making that led to the animals’ behavioral choice.

4. DISCUSSION

Phasic activation of the central arousal system, indicated by phasic pupil dilation time-locked to stimulus, has been reported in numerous behavioral tasks across species (Jan Willem de Gee et al., 2014; Denison et al., 2019; Hong et al., 2014; Krishnamurthy, Nassar, Sarode, & Gold, 2017; Murphy, Boonstra, & Nieuwenhuis, 2016; Schriver et al., 2018; Urai, Braun, & Donner, 2017; van den Brink, Murphy, & Nieuwenhuis, 2016). Phasic arousal has been suggested to have important effects on decision formation. These effects include adjusting decision bias, modifying internal models and functionally resetting neural networks mediating specific cognitive functions (Bouret & Sara, 2005; J. W. de Gee et al., 2019; Krishnamurthy et al., 2017; Murphy et al., 2016; Urai et al., 2017; Yu & Dayan, 2005). Although nonluminance-mediated changes in pupil size during behavioral tasks have long been thought to result from the activation of the LC (Aston-Jones & Cohen, 2005), it was not until recently that the causal relationship between LC activation and pupil dilation has been conclusively demonstrated (Joshi et al., 2016; Liu et al., 2017; J. Reimer et al., 2016). However, recent work has also revealed that the activity of other brain regions is correlated with changes in pupil size (Joshi et al., 2016; J. Reimer et al., 2016; Wang, Boehnke, White, & Munoz, 2012). In other words, activity in multiple brain regions, including the superior colliculus, inferior colliculus, and cingulate cortex, anticipates pupil dilation. For example, Joshi et al. reported that similar to the LC, spiking activity in the PFC reliably anticipated changes in pupil size (Joshi et al., 2016). The correlation between changes in pupil size and activity of these brain regions may be explained by the extensive connections between the LC and these regions (Aston-Jones & Cohen, 2005; Breton-Provencher & Sur, 2019). These lines of evidence suggest that the task-evoked changes in pupil size may reflect the activation of a distributed network of brain structures responsible for different information processing components in a perceptual decision-making task.

In the present study, we have used a data driven approach to recover kernels so as to disentangle the elements of pupil-linked phasic arousal resulting from different information processing components on a trial-by-trial basis. We simultaneously learned the kernel of phasic pupil dilation associated with each of the 6 components and their beta weight on each trial. As we attempted to estimate latent variables in a nonconvex optimization, it remained possible some models converged to a local minimum. However, due to the low variance between learned kernels across our ensembles for a single session with different randomized start points and randomly selected folds, we believe that learned kernels were unlikely to be local minima. Another possible pitfall of this approach was overfitting, which we took multiple steps to avoid (Lever et al., 2016). One of these was the implementation of early stopping, a robust method commonly employed in iterative machine learning methods to avoid overfitting by minimizing the generalization error (Yao et al., 2007). We specifically used k-fold cross validation (Fushiki, 2011; Mosteller & Tukey, 1968), by randomly assigning each trial from a session into either a training data set or a validation data set. The model was fit to the training data set and with each new update iteration to improve the fit to the training data, we also tested the general kernels’ ability to fit the validation set. We stopped additional fitting once the model’s ability to fit the validation set began to decrease, as this would imply a move away from a general solution and towards overfitting the training data.

We further employed an ensemble learning technique (Sollich & Krogh, 1995), bootstrap aggregating (bagging), to combine fits from multiple separate models, thus smoothing out their predictions, and thereby helping to avoid overfitting by reducing variance (Breiman, 1996; Petersen, Molinaro, Sinisi, & van der Laan, 2008). Specifically, we allowed each of our folds to play the role of validation once and furthermore carried out this process 10 times. We then averaged across these to estimate kernels and weights, thus making our approach more robust to the effects of a single instance of the model which could getting stuck in a locally optimal solution. Additionally, we used stochastic gradient descent for its ability to minimize solutions trapped by local minima, as well as to facilitate training speed and attain small generalization errors (Hardt, Recht, & Singer, 2016). Furthermore, the Nesterov-accelerated adaptive moment estimation algorithm (NADAM) was incorporated to update the learning rate for the kernels to avoid local minima, additionally speeding up model estimation (Dozat, 2016). While classical momentum accelerates gradient descent learning among stable but not oscillating dimensions (Polyak, 1964), Nesterov’s accelerated gradient (NAG) training further improves performance by applying the momentum vector prior to computing the gradient, thereby increasing stability (Ilya, James, George, & Geoffrey, 2013). For our algorithm, NAG was incorporated into the adaptive moment estimation algorithm (ADAM), which has been shown to further decrease both the number of iterations and the computation time, while improving overall convergence and maintaining a general solution (Kingma & Ba, 2015).

It is worth noting that the two components of phasic arousal that we focused our analyses on were associated with stimulus encoding response and decision formation. The activation of phasic arousal associated with stimulus encoding occurred with variable interval prior to the activation of other components of phasic arousal. The activation of phasic arousal associated with decision formation was of variable duration across trials (dependent on the RT of the animal). This variable timing should discourage the model from storing any arousal response in these kernels that was not actually associated with the corresponding components.

We found that phasic arousal evoked by both stimulus encoding and decision formation was generally higher on trials when the target stimulus (i.e. Go stimulus) was presented than when the nontarget stimulus (i.e. No-Go stimulus) was presented. This is consistent with previous findings that the target stimulus was selectively encoded in higher order brain regions (Fritz, David, Radtke-Schuller, Yin, & Shamma, 2010; Rainer, Asaad, & Miller, 1998) and the LC (Rajkowski, Majczynski, Clayton, & Aston-Jones, 2004). For example, neurons in the lateral prefrontal cortex were found to have a stronger response to target stimuli than nontarget stimuli (Rainer et al., 1998). Similarly, LC neurons exhibited a phasic response to target stimuli but little change in response to distractor stimuli in monkeys performing a detection task (Rajkowski et al., 2004). Interestingly, there is a heavy reciprocal connection between the LC and prefrontal cortex (Aston-Jones & Cohen, 2005; Berridge & Waterhouse, 2003). Therefore, the observed difference may be due to the interplay between the LC and the prefrontal cortex. However, the precise mechanism by which the target stimulus evoked higher phasic arousal warrants future investigations.

During the task, because correctly rejecting a No-Go stimulus did not result in water reward, during deliberation in decision formation based on a noisy representation of the Go and No-go stimulus, the animals should have been biased to choose a go. Indeed, we found that the animals tended to be liberal in their decision-making, indicated by a negative overall decision criterion (−0.33 ± 0.04, p < 1.7 × 10⁻³³). As we expected, the difference between phasic arousal elicited by decision formation in correct and incorrect trials was negatively correlated with decision criterion. However, we found that the difference between phasic arousal elicited by stimulus encoding in S+ and S− trials was also negatively correlated with decision criterion. Moreover, when the Go stimulus was presented, the animal’s decision to go was determined by phasic arousal resulting from stimulus encoding, while when S− was presented, the animal’s decision to go was determined collectively by phasic arousal resulting from both stimulus encoding and decision formation. This was probably because the phasic arousal evoked by the Go stimulus was sufficiently high as compared to that evoked by a No-Go stimulus leading to an optimal decision. This notion is consistent with a recent work in which phasic arousal was found to suppress deviation from optimal decision bias in both humans and mice (J. W. de Gee et al., 2019). This recent work focused on overall task evoked-pupil dilation, whereas here we learned individual contributions of time-locked information processing components to the total dilation. In both cases, increased pupil-linked arousals tended to drive the decision-makers to Go responses, suggesting this is a general principle across species (mice, rats, and humans). In line with total pupil dilation being related to reducing evidence accumulation bias, we found a strong positive coefficient of regression relating phasic arousal associated with decision formation and drift bias (Figure 7H). However, by decomposing pupil-linked phasic arousal, our data also found that this was only one of multiple relationships that phasic arousal had with aspects of the decision making process. Taken together, our results demonstrated the functional effects of phasic arousal evoked by perceptual processing and cognitive processing on forming animal’s behavioral choice.