The modulation of brain network integration and arousal during exploration

Nathan Tardiff; John D Medaglia; Danielle S Bassett; Sharon L Thompson-Schill

doi:10.1016/j.neuroimage.2021.118369

. Author manuscript; available in PMC: 2021 Oct 15.

Published in final edited form as: Neuroimage. 2021 Jul 6;240:118369. doi: 10.1016/j.neuroimage.2021.118369

The modulation of brain network integration and arousal during exploration

Nathan Tardiff ^a,^*, John D Medaglia ^b,^c,^d, Danielle S Bassett ^d,^e,^f,^g,^h,ⁱ, Sharon L Thompson-Schill ^a

PMCID: PMC8507424 NIHMSID: NIHMS1741054 PMID: 34242784

Abstract

There is growing interest in how neuromodulators shape brain networks. Recent neuroimaging studies provide evidence that brainstem arousal systems, such as the locus coeruleus-norepinephrine system (LC-NE), influence functional connectivity and brain network topology, suggesting they have a role in flexibly reconfiguring brain networks in order to adapt behavior and cognition to environmental demands. To date, however, the relationship between brainstem arousal systems and functional connectivity has not been assessed within the context of a task with an established relationship between arousal and behavior, with most prior studies relying on incidental variations in arousal or pharmacological manipulation and static brain networks constructed over long periods of time. These factors have likely contributed to a heterogeneity of effects across studies. To address these issues, we took advantage of the association between LC-NE-linked arousal and exploration to probe the relationships between exploratory choice, arousal—as measured indirectly via pupil diameter—and brain network dynamics. Exploration in a bandit task was associated with a shift toward fewer, more weakly connected modules that were more segregated in terms of connectivity and topology but more integrated with respect to the diversity of cognitive systems represented in each module. Functional connectivity strength decreased, and changes in connectivity were correlated with changes in pupil diameter, in line with the hypothesis that brainstem arousal systems influence the dynamic reorganization of brain networks. More broadly, we argue that carefully aligning dynamic network analyses with task designs can increase the temporal resolution at which behaviorally- and cognitively-relevant modulations can be identified, and offer these results as a proof of concept of this approach.

Keywords: Brainstem arousal systems, Dynamic functional connectivity, Exploration, Norepinephrine, Pupillometry, Time series analysis

1. Introduction

The brain has a remarkable capacity to adaptively shift processing to support a diverse array of behavioral goals, contextual demands, and environmental changes. This fact raises two fundamental questions: What neural mechanisms allow the brain to rapidly shift between states that form the substrates of different cognitive processes and behaviors, and how does the brain maintain a balance between the stability necessary to support ongoing behavior and the flexibility necessary to adapt to new exigencies? A number of theoretical proposals have pointed to a role for neuromodulatory systems in answering these questions, and in particular the neuromodulatory actions of norepinephrine (NE), a key component of physiological arousal (Arnsten et al., 2010; Aston-Jones and Cohen, 2005; Bouret and Sara, 2005; Yu and Dayan, 2005). The primary source of NE in the brain is the locus coeruleus (LC), a pontine nucleus that projects widely throughout the cortex (Berridge and Waterhouse, 2003). NE has complex effects at the single neuron level, but a common finding is that it increases the signal-to-noise ratio of neural responses, effectively modulating the gain of the neural response function (Berridge and Waterhouse, 2003; Hasselmo et al., 1997; Hurley et al., 2004), which will tend to amplify and propagate stronger neural activity while suppressing weaker activity (Aston-Jones and Cohen, 2005; Eldar et al., 2016; Mather et al., 2016). Simulations suggest these effects of gain modulation can collectively lead to changes in functional connectivity and network topology (Eldar et al., 2013; Shine et al., 2018a). These features make the LC-NE system well situated to effect large-scale changes in brain networks and cognitive function. Several prominent theories have ascribed this system such a role, suggesting that it resets functional brain networks in support of specific behaviors and cognitive states as dictated by environmental demands (Bouret and Sara, 2005), shifts the balance of information processing from top-down to bottom-up depending on the uncertainty of internal world models (Yu and Dayan, 2005), or shifts the brain between states of exploration and exploitation based on ongoing estimates of task utility (Aston-Jones and Cohen, 2005).

Recent studies have begun to explore the association between brainstem arousal systems and functional brain networks using functional neuroimaging. Neuromodulators including norepinephrine, dopamine, and acetylcholine have been implicated in coordinating brain network dynamics (Birn et al., 2019; Eldar et al., 2013; Guedj et al., 2017; Roffman et al., 2016; Shafiei et al., 2019; Turchi et al., 2018; van den Brink et al., 2018, 2016b; Záborszky et al., 2018; Zerbi et al., 2019; see van den Brink et al., 2019 for review). For example, utilizing the fact that activity in LC and other brainstem neuromodulatory nuclei leads to increases in pupil diameter (de Gee et al., 2017; Gilzenrat et al., 2010; Joshi et al., 2016; Reimer et al., 2016; Varazzani et al., 2015), studies have found that elevated pupil diameter—either baseline pupil during task or endogenous fluctuations during rest—is associated with stronger overall functional connectivity and greater clustering of functional connections (Eldar et al., 2013; van den Brink et al., 2016b; Warren et al., 2016), as well as an increase in the diversity of connectivity between functional modules, potentially indicating greater integration among cognitive systems (Shine et al., 2016). NE-linked changes in functional connectivity have also demonstrated spatial patterning consonant with specific catecholamine receptor distributions in humans (van den Brink et al., 2018) and mice (Zerbi et al., 2019). Pharmacological manipulation of NE with atomoxetine, a norepinephrine transporter blocker, has produced conflicting results, with resting-state studies finding decreased connectivity between networks (van den Brink et al., 2016b; see Guedj et al., 2017 for similar result in macaques), but increased connectivity between networks in a task-based study (Shine et al., 2018b).

The heterogenous results across studies likely stem from a number of factors, including differences in the methods used to construct and analyze brain networks, as well as differences in neural response between endogenous fluctuations of LC-NE activity and manipulation with atomoxetine, which influences LC firing in addition to increasing cortical NE levels (Bari and Aston-Jones, 2013). Importantly, the divergence between task and rest effects may stem from the inverted-U-shaped relationship between catecholamine levels and their neural and cognitive effects (Aston-Jones and Cohen, 2005; Berridge and Waterhouse, 2003; McGinley et al., 2015; Robbins and Arnsten, 2009), where the resting state is presumably associated with lower arousal levels. Furthermore, neural activity varies between rest and task, and between different tasks, which as outlined above will interact with neuromodulation of neural gain to amplify (or suppress) activity specific to that particular cognitive state at that particular moment in time. Given that the actions of LC-NE and other brainstem arousal systems depend on the underlying state of the system, it is critical to ask what the relationship between brain network organization, neuromodulatory activity, and task performance is for particular classes of behaviors and at behaviourally-relevant timescales. To date, however, the relationship between brainstem arousal systems and functional connectivity has not been assessed within the context of a task with an established relationship between these systems and behavior.

The role of the LC-NE system in mediating between exploration and exploitation provides a strong place to begin to form these links. It has been proposed that increases in tonic LC-NE activity promote disengagement from the current task (exploitation) in order to seek alternatives (exploration) (Aston-Jones and Cohen, 2005). Direct LC stimulation promotes patch leaving and general disengagement during foraging (Kane et al., 2017), and pupil diameter has been found to increase with exploratory choice (Jepma and Nieuwenhuis, 2011) and with decreases in task utility signaling the need to disengage from the current course of action (Gilzenrat et al., 2010). More broadly, elevated tonic LC activity and pupil diameter have been linked to distractibility (Aston-Jones and Cohen, 2005; Bouret and Sara, 2005; Ebitz and Platt, 2015; Unsworth and Robison, 2016; van den Brink et al., 2016a). A number of studies have found that performance in cognitively demanding tasks is supported by increased integration among functional brain networks, with poorer performance predicted by decreased integration (Braun et al., 2015; Ekman et al., 2012; Gießing et al., 2013; Shine et al., 2016; Vatansever et al., 2015). This pattern of results suggests a potential parallel between elevated LC-NE activity and brain network integration—namely, that elevated LC-NE activity may lead to decreased functional integration, which may in turn provide a substrate for exploration.

We tested this hypothesis in the present study. Subjects completed a two-armed bandit task while undergoing fMRI and pupillometry. In order to meet the goal of linking arousal, functional connectivity, and behavior, we examined dynamic functional connectivity (Calhoun et al., 2014; Fedorenko and Thompson-Schill, 2014; Khambhati et al., 2018b; Kopell et al., 2014; Medaglia et al., 2015), going beyond the static connectivity measures used in most prior studies in this domain to more tightly link arousal, connectivity changes, and behavior. Furthermore, by time-locking our analyses to exploratory choice and utilizing semi-parametric time series methods, we demonstrate the ability to uncover fine-grained fluctuations in functional network dynamics at a temporal resolution generally not achieved with dynamic functional connectivity. By addressing the above hypotheses at a timescale consonant with behavior, we seek to develop a more fundamental understanding of the neural bases of exploratory states and arousal-linked mechanisms of dynamic network reconfiguration.

2. Methods

2.1. Subjects

Forty subjects (22 female, M_age = 23.48 years) completed the study. Informed consent was obtained from each subject in accordance with the University of Pennsylvania IRB. All subjects in the final sample (1) were right-handed; (2) were between 18 and 35 years old; (3) had normal or corrected-to-normal vision; (4) had no known learning impairments or history of neurological or psychological disorders; and (5) were not currently taking any psychiatric medications or medications known to affect the autonomic nervous system. Two subjects were excluded because it was later determined they were taking medications that did not meet the inclusion criteria. Four additional subjects were excluded from the analyses for excessive head movement during scanning (average framewise displacement across runs > 0.2 mm), for a final sample of 34 (20 female, M_age = 22.82 years).

2.2. Materials and procedure

Subjects completed the Leapfrog bandit task (Knox et al., 2012). In this highly constrained two-armed bandit (Fig. 1A), the options are always 10 points apart in value; when the options are selected, they deliver payoffs deterministically. After every trial, with probability P(flip) the currently lesser-valued option may jump in value by 20 points to become the superior option. Which option is better thus alternates throughout the task, and subjects must balance (1) choosing the option that is the best according to their current knowledge (exploiting) and (2) sampling the other option to find out if it has improved (exploring). The constrained nature of this task is advantageous because trials can be classified as exploratory or exploitative solely on the basis of behavior, without recourse to model-based analyses necessitated by drifting bandits (Daw et al., 2006; Ebitz et al., 2018). Concretely, trials were classified as explore trials if subjects chose the option that was lesser-valued, based on previously observed outcomes (e.g., choosing the right option after having observed that the left option was now worth 120 points and prior choice of the left option had yielded 110 points).

Subjects completed four blocks of the task, with 80 trials per block (320 trials total). To minimize luminance-mediated changes in pupil diameter, task stimuli were luminance-matched grayscale images and were only modestly brighter than the background. Option changes were randomly generated per participant based on the underlying P(flip). P(flip) was fixed within blocks but alternated across blocks (low volatility: P(flip) = 0.05; high volatility: P(flip) = 0.20), with the order of alternation counterbalanced across subjects. At the start of block 1, the left and right options were set to a value of 100 and 110, respectively. In a separate behavioral session prior to the scan session, subjects were instructed about the structure of the task (including the initial option values), performed 8 practice trials to familiarize themselves with the controls and the task display, and then performed an identical version of the task to the scanner version, excepting that the stimuli were not luminance-controlled. While subjects received information about the volatility levels by completing the behavioral session, they were not told about the volatility changes. To minimize eye movements, subjects were instructed to fixate on the center of the task display, except during the ITI, when they were told to keep their gaze within a 189×179-pixel light gray rectangle in the center of the display. Subjects made their responses with the index and middle finger of their right hand. Because the increase in payoffs throughout the task could distort choice behavior, subjects were incentivized to choose the currently best option on all trials rather than maximize their payoffs (Otto et al., 2014). Subjects were paid $10/h for the behavioral session (length 1 h) and $20/h for the scan session (length 1.5–2 h) plus a bonus determined by $p \times \frac{b_{m a x}}{n_{t r i a l s}}$ , rounded to the nearest dollar, where p is the number of choices of the currently best option, b_max is the maximum possible bonus ($10 behavioral, $15 scan), and n_trials is the total number of trials. The task was written in PyPsyExp (https://gureckislab.org/pypsyexp/sphinx/) and run in the PsychoPy environment (Peirce, 2009).

The fMRI session began with eye tracker calibration, after which scans were run in the following order: Leapfrog block 1, B0 field map, Leapfrog block 2, Leapfrog block 3, T1, Leapfrog block 4. Subjects were reminded of the initial option values before the start of block 1. Because of the scans between task blocks, they were also reminded of the current option values before the start of blocks 2 and 4. Prior to the first Leapfrog block and after the last block, we also collected a five-minute resting-state scan, which we did not analyze here.

2.3. MRI data acquisition

Magnetic resonance images were collected using a Siemens Prisma 3T scanner (Siemens Medical Systems, Erlangen, Germany) with a 64-channel head coil. T1-weighted anatomical images were acquired (MPRAGE; repetition time [TR] = 1810 ms; echo time [TE] = 3.45 ms; flip angle [FA] = 9°; field of view [FOV] = 240 mm; matrix = 256×256; voxel size = 0.9×0.9×1.0 mm; 160 slices). During task runs, T2∗-weighted functional volumes were collected using multiband echo planar imaging (EPI; TR = 1000 ms; TE = 30 ms; FA = 60°; FOV = 208 mm; matrix = 104×104; voxel size = 2.0×2.0×2.0 mm; 72 slices; multiband acceleration factor = 6). A field map was also acquired for distortion correction of the EPI images (TR = 580 ms; TE 1 = 4.12 ms; TE 2 = 6.58 ms; flip angle = 45°; voxel size = 3.0 mm× 3.0 mm× 3.0 mm; FoV = 240 mm).

2.4. MRI preprocessing

Preprocessing was performed using FSL (Jenkinson et al., 2012) and FreeSurfer (Fischl, 2012). Cortical reconstruction and volumetric segmentation of the anatomical data were performed with FreeSurfer. Functional data were despiked by replacing values greater than 7 RMSE from a 1-degree polynomial fit to the time course of each voxel with the average value of the adjacent TRs. Motion correction parameters were computed by registering each volume of each run to the middle volume using a robust registration algorithm (mri_robust_register; Reuter, Rosas, & Fischl, 2010) and voxel shift maps for EPI distortion correction were calculated using PRELUDE and FUGUE (Jenkinson, 2004, 2003); the resulting transformations were combined and simultaneously applied to the functional images. Boundary-based registration between structural and functional images was performed with bbregister (Greve and Fischl, 2009). To account for motion and physiological noise, the following nuisance time series were regressed from the functional data: (1) 24 motion regressors (Friston et al., 1996); (2) the five first principal components of non-neural sources of noise (i.e., white matter, CSF), obtained with FreeSurfer segmentation tools (aCompCor; Behzadi et al., 2007); (3) cardiac and respiratory rhythms derived from pulse oximetry data collected during each scan (Verstynen and Deshpande, 2011; due to technical issues, pulse oximetry data were unavailable for two subjects); and (4) local noise, estimated as the average white matter signal within a 15 mm radius of each gray matter voxel (ANATICOR; Jo et al., 2010). The data were then high-pass filtered with a cutoff frequency of 0.009 Hz.

In the analyses reported in Section 3.6, to account for task-evoked activity, additional nuisance regressors were included corresponding to the choice and outcome phases of the task. The choice phase was modeled as a boxcar beginning at the onset of the choice signal (Fig. 1A) and lasting for 1.5 s. The outcome phase was modeled as a boxcar lasting for the duration of the 1.5 s outcome presentation. In blocks with missing responses, the signal for a missed trial (a question mark) was additionally modeled as a boxcar beginning at the onset of the question mark (the start of the wait phase) and lasting for the remaining 2.5 s of the trial. All task regressors were convolved with the canonical double-gamma hemodynamic response function prior to nuisance regression.

2.5. Network construction

The cortex was parcellated into 200 regions based on the Schaeffer 200-parcel atlas (Schaefer et al., 2018). To this we added 15 subcortical regions segmented by FreeSurfer (Fischl et al., 2002). The average BOLD time series was extracted from each region, and functional connectivity between all pairs of regions was estimated via continuous wavelet coherence in the range of 0.08–0.125 Hz (Grinsted et al., 2004). This frequency range has been previously shown to be sensitive to dynamic changes in task-based functional connectivity (Bassett et al., 2011; Braun et al., 2015; Craig et al., 2018; Gerraty et al., 2018; Sun et al., 2004). The continuous wavelet transform (CWT) was chosen over the more common discrete wavelet transform to provide additional sensitivity to time-varying changes around exploration. This procedure produces a connectivity value for each TR, sampled across the frequency range. Note that no windowing of the time series was performed prior to transformation, as the CWT is itself a sliding-window method (i.e., a convolution), and additional windowing would produce unwanted edge effects (Grinsted et al., 2004). We then averaged across the frequency range to produce a single time-varying connectivity measure between each region. Finally, given that the resultant signal was heavily over-sampled, the connectivity time series were then downsampled by a factor of 2 (Matlab decimate), providing a final sampling rate of 0.5 Hz, and yielding one 215×215×240 weighted adjacency matrix per task run.

2.6. Multislice modularity optimization

In order to identify changes in network architecture over time, the connectivity matrices were submitted to a Louvain-like locally greedy modularity maximization algorithm (Mucha et al., 2010) implemented in Matlab (Jeub et al., 2011). This method, which has been used extensively to estimate time-varying modular structure in functional brain networks (Bassett et al., 2015, 2011; Gerraty et al., 2018; Mattar et al., 2015), maximizes a multilayer quality function given by:

Q_{m u l t i s l i c e} = \frac{1}{2 μ} \sum_{i j s r} [(A_{i j s} - γ_{s} \frac{k_{i s} k_{j s}}{2 m_{s}} δ_{s r}) + δ_{i j} ω_{j s r}] δ (g_{i s}, g_{j r}),

(1)

where the adjacency matrix of layer s has components A_ijs, g_is gives the module assignment of node i in layer s, g_jr gives the module assignment of node j in layer r, k_js is the intralayer strength of node j in layer s, $c_{j s} = \sum_{r} ω_{j s r}$ is the interlayer strength of node j in layer s, κ_js = 𝑘_js + c_js is the strength of node j in layer s, and the total edge weight of the network is given by $μ = \frac{1}{2} κ_{j r}$ . The quantity $\frac{k_{i s} k_{j s}}{2 m_{s}}$ corresponds to the Newman-Girvan null model (Newman and Girvan, 2004), where $m_{s} = \frac{1}{2} \sum_{i j} A_{i j s}$ is the total edge weight in layer s. The structural resolution parameter γ_s of layer s and the interlayer coupling parameter ω_jsr from node j in layer s to node j in layer r tunes the size of the modules within each layer and the number of modules across layers (i.e., time), respectively. In this case, the structural resolution parameters were assumed to be constant across layers (γ_s = γ); the interlayer coupling parameters were set to a constant value ω for all s and r representing immediately adjacent layers and were set to 0 everywhere else, producing an ordered multilayer network.

The choice of γ and ω is not entirely straightforward. Often, they are left at a default value of 1 (Bassett et al., 2013). In other instances, they are selected to optimize some quantity, such as Q_multislice or other network measures of interest (Weir et al., 2017). Then, given the near degeneracy of the modularity landscape (Good et al., 2010), the modularity maximization algorithm is run a number of times (e.g., 100) at the selected parameter values. To avoid dependence of our results on a particular point in parameter space and to increase sensitivity to fluctuations in integration regardless of scale, here we repeated the modularity maximization procedure a single time across a range of parameter values (γ ∈ [1.14, 1.19] discretized by a step size of 0.01; ω ∈ [0.05, 0.85] discretized by a step size of 0.05) rather than multiple times at a single set of parameter values (see Vaiana et al., 2019 for a related approach). The range of γ was chosen such that on average the number of non-singleton modules in a layer approximated the number of non-singleton cognitive systems in our resting-state reference partition (see Section 2.7 below and Fig. 1C); the range of ω was chosen to optimize network flexibility, which quantifies how often nodes switch modules across layers (Bassett et al., 2011). We chose to optimize ω with respect to network flexibility to increase sensitivity to rapid changes in network architecture. The ranges used in the study were identified via a larger parameter sweep over γ and ω (Figure S1A,B), where the final range was chosen to meet the criteria above and yield ~100 repetitions of modularity maximization.

Maximizing the above multilayer modularity quality function is NP hard. To improve the quality of the multilayer partitions, we used a heuristic algorithm that at each step chooses moves (of a node into a module) probabilistically in proportion to the increase in the multilayer quality index (Bazzi et al., 2016; Jeub et al., 2011). In order to avoid local minima, after each run of the Louvain algorithm, module assignments were revised to maximize the persistence of modules across time without altering the intralayer modular structure (Bazzi et al., 2016; Jeub et al., 2011). Modifying the multilayer partition in this fashion is guaranteed to increase multilayer modularity and helps avoid large changes in module assignment across time that are not accompanied by prominent changes in intralayer modular structure (Bazzi et al., 2016). The resultant partition was then used as the starting point for an additional run of the Louvain algorithm, and this procedure was repeated until the output partition converged (Jeub et al., 2011). These steps were repeated across the parameter grid, yielding 102 time-varying networks per run.

2.7. Integration

At each time point, a module allegiance matrix P^t was constructed, with entries:

P_{i j}^{t} = \frac{1}{O} \sum_{o = 1}^{O} a_{i j}^{o t},

(2)

where O is the number of final output partitions (O = 102) and the allegiance value $a_{i j}^{o t}$ for nodes i and j is 1 if the nodes were placed in the same module at time t of partition o and 0 otherwise. Intuitively, $P_{i j}^{t}$ is the fraction of times that two nodes were placed in the same module, across the parameter space (see Braun et al., 2015 for a similar approach to computing network measures per time window). See Figure S1C for the standard deviation of modular allegiance values across the parameter space.

In order to then use the modular allegiance matrices to assess the interaction between brain regions across time, we assigned each network node to a resting-state cognitive system. All cortical nodes were previously assigned to one of seven resting-state systems identified from large-scale resting-state data (Schaefer et al., 2018; Yeo et al., 2011). All subcortical nodes were assigned to an eighth subcortical system with the following exceptions: bilateral amygdala and hippocampus were placed in the limbic system, while the brainstem was assigned to its own singleton system. See Figure S1D for the correspondence between average task connectivity and the resting-state cognitive systems.

The integration of brain region i in cognitive system s at time t can then be computed as:

I_{i}^{t s} = \frac{1}{N - n_{s}} \sum_{j \notin s} P_{i j}^{t},

(3)

where N is the total number of nodes (brain regions) and n_s is the number of nodes in system s (Mattar et al., 2015). Integration thus quantifies the probability at a given time that a node from a given cognitive system is placed into the same module as nodes from other cognitive systems. Averaging integration across nodes then provides a measure of the global level of integration in the brain at each time point.

Integration can also be computed for each system and between each pair of systems. The integration of system s with the rest of the brain (i.e., all systems not s) is:

I_{s}^{t} = \frac{1}{n_{s} (N - n_{s})} \sum_{i \in s} \sum_{j \notin s} P_{i j}^{t},

(4)

indicating the tendency for nodes from system s to be placed into modules with nodes from other systems at time t. Similarly, the integration between two systems k and l is given by:

I_{k l}^{t} = \frac{1}{n_{k} n_{l}} \sum_{i \in k} \sum_{j \in l} P_{i j}^{t},

(5)

where n_k is the number of nodes in system k and n_l is the number of nodes in system l. High integration between two systems at a given time indicates a departure from resting-state structure and is suggestive of strong functional interactions between cognitive systems.

2.7.1. Peri-explore integration analysis

Statistical analysis of change in the integration time course around exploration presents several methodological challenges. The time series is strongly autocorrelated due to the nature of the fMRI BOLD signal and the CWT, which increases the risk of type I error due to violation of the independence assumption of linear regression. The response to exploration is of an unknown functional form and possibly non-monotonic, making standard linear regression—even using polynomial terms—a potentially poor fit. Finally, there is no clear contrast or baseline because of the sluggishness of the signal relative to task timing (the ITI is only 1 s), so we cannot directly contrast explore with exploit time courses as individual exploit trials cannot be resolved; rather, explore trials occur against an effectively continuous background of exploit trials.

To address all these issues, we utilized generalized additive mixed models (GAMMs) in the peri-explore integration analyses. GAMMs are an extension of the regression framework that allow for the fitting of arbitrary (e.g., nonlinear, nonmonotonic) functions, including both linear and nonlinear random effects terms (Wood, 2017). These nonlinear functions, or smooths, are fit using maximum likelihood estimation using a weighted sum of basis functions. The basis functions are selected from families of penalized splines, where overfitting is mitigated and therefore smoothness is enforced by a penalty on basis function coefficients. The appropriate smoothness for a given data set is controlled via smoothing parameters that are estimated as part of the fitting procedure (see Baayen et al., 2017; Pedersen et al., 2019; and van Rij et al., 2019 for tutorials, and Wood, 2017 for additional technical and mathematical details).

Prior to model fitting, peri-explore integration time courses were extracted and processed as follows. After identifying the points in the integration time course that contained each exploratory choice, we extracted the time series immediately preceding (following) the choice window, up to the previous (next) exploratory choice. For this analysis, we did not include peri-explore epochs in which subjects explored immediately following a missed flip (i.e., subjects exploited and saw a change). We excluded these trials (median per subject: 1, range: 0–8) because they are rare and surprising, which might confound any subsequent physiological signature related to exploration. In order to isolate the effect of a single exploratory choice given the sluggishness of the integration time course, we further restricted the analysis to explore choices preceded by a minimum of 2 exploit trials and followed by a minimum of 4 exploit trials. Given limits on the amount of data per subject, we made the buffer asymmetric to maximize our sensitivity to the exploration-evoked response while including as much data as possible. We additionally excluded the first and last peri-explore periods of every block. The final analysis window was then restricted to encompass the 12 s prior to the explore time point extending to 18 s post-explore (median peri-explore periods per subject: 21.5, range: 2–37). We then downsampled the time series to 0.25 Hz (by dropping every other data point) as a first step in mitigating autocorrelation.

All GAMMs included a smooth for time as well as by-subject random smooths for time. Models also included by-time-course linear random intercepts and slopes in order to account for additional variance due to drifts in integration over time, which helps to further alleviate autocorrelation in the residuals (van Rij et al., 2019). Because model residuals were still autocorrelated, we also introduced an AR1 model to each GAMM. For analyses of global integration, the AR1 parameter that minimized AIC in a grid search (ρ ∈ [0.00, 0.99] in steps of 0.01) was selected for the final model (Wood, 2017). For by-system integration, residual autocorrelation was very similar in each system, so we selected the ρ that minimized AIC for the model with the median AR1 value. The same approach was used for between-system integration. Significance was assessed based on a Wald statistic (i.e., testing the null hypothesis that the smooth = 0), with p-values computed using the F-distribution with degrees of freedom based on the effective degrees of freedom of the smooth (Wood, 2017, 2013).

To ask whether there was overall evidence for significant differences between peri-explore time courses at the by-system and between-system levels, we conducted a model comparison procedure. We fit a full model to data from all systems (between-system interactions), which included each system (between-system interaction) as a separate smooth. We then fit a reduced model that assumed one global smooth for all data. Both the full and reduced models included by-system (between-system interaction) intercepts, since the focus was on differences in the form of the modulation rather than on baseline differences in the absolute level of integration. Because of the complexity of these models, we replaced the by-subject random smooths for time used in other models with parametric random effects terms: by-subject random intercepts and by-subject random effects of system and time. The models were then compared using AIC.

To confirm the results of the global integration GAMM, a permutation analysis was conducted. Within each block, the assignment of exploration time points to the integration time course was permuted 500 times, with the constraint that permuted time points must be ones in which subjects actually made a choice and the mean inter-explore interval of each permutation must be within one unit of the true mean interval for that block. Peri-explore time courses were then extracted and analyzed identically to the true time courses, resulting in a GAMM fit for each permutation. The significance of the true data was then assessed relative to this distribution. Note that we constructed our permutation distribution from the p-values of the smooths rather than the F-values. Unlike a standard parametric linear analysis, the number of degrees of freedom differs between models, owing primarily to differences in the roughness of the fit, and also to slight differences in the amount of data in each permutation as a result of preprocessing exclusions. Using F-values can thus produce conservative results, as smooths with fewer effective degrees of freedom may benefit from larger F-values. Because the p-value computation takes degrees of freedom into account (Wood, 2013), it is a more appropriate measure in this case.

2.8. Additional network measures

To better characterize the network dynamics surrounding exploration, we computed four additional measures, using the Brain Connectivity Toolbox (Rubinov and Sporns, 2010): strength, system segregation, modularity, and number of modules.

Changes in integration will be accompanied by changes in the underlying patterns or strength of functional connectivity. Therefore, the average strength, s, of node i at time t was computed as:

s_{i}^{t} = \frac{1}{N - 1} \sum_{j} A_{i j}^{t} .

(6)

By averaging node strength separately for within- and between-system connections across the whole brain, system segregation (Chan et al., 2014) was computed as:

s y s t e m s e g r e g a t i o n = \frac{{\bar{s}}_{w} - {\bar{s}}_{b}}{{\bar{s}}_{w}},

(7)

where ${\bar{s}}_{w}$ is the mean within-system strength and ${\bar{s}}_{b}$ is the mean between-system strength, across the whole brain. Unless otherwise noted, we computed system segregation relative to the Yeo cognitive systems (Yeo et al., 2011), to match our procedure for integration, rather than to the module assignments at each time point.

The single-layer modularity Q (Blondel et al., 2008) was computed at each time t using as input the module assignments derived from each run o of multilayer modularity (Eq. 1). Specifically:

Q_{o}^{t} = \frac{1}{2 m} \sum_{i j} [A_{i j} - \frac{k_{i} k_{j}}{2 m}] δ (c_{i}, c_{j}),

(8)

where $k_{i} = \sum_{j} A_{i j}$ and $m = \frac{1}{2} \sum_{i j} A_{i j}$ , and where o and t super/subscripts are omitted for clarity. The Q values were then averaged over all o runs to produce a single value at each time point. Both system segregation and modularity are considered to measure the extent to which cognitive systems are segregated (Chan et al., 2014; Cohen and D’Esposito, 2016; Rubinov and Sporns, 2010), meaning they have the potential to provide evidence convergent with integration as to the nature of the topological changes accompanying exploration. Note that Q_multislice (Eq. 1) was not used for these analyses because it yields a single value per run, reflecting overall modularity in time and space, whereas here the question is how modularity changes across time. Finally, because changes in integration may in part reflect modules coalescing or dividing, the number of modules was defined as the average number of modules present at each time point, averaged over runs of the GenLouvain algorithm.

As with integration, the significance of peri-explore modulation was assessed using GAMMs. Because the number of modules were heavily skewed, these data were fit with an inverse Gaussian regression (log link). A single AR1 parameter ρ was used for the strength-based measures (strength, system segregation), and a separate single ρ parameter was used for the modularity-based measures (Q, number of modules).

2.9. Pupillometry

Eye position and pupil diameter of the right eye were recorded during scanning at a sampling rate of 250 Hz with an EyeLink 1000 Plus (SR Research) equipped with the Long Range Mount. The PyGaze toolbox was used to interface with the eye tracker (Dalmaijer et al., 2014). Periods of missing data due to blinks or other artifacts were linearly interpolated after removing an additional 25 samples (100 ms) surrounding the blink on either side. Additional artifacts were identified by computing the difference between consecutive samples of the pupil time course. High velocity periods, defined as samples differing in diameter by more than 50 in absolute value (a.u.) from the preceding sample were removed, and for runs of high velocity > 4 samples we additionally removed 25 samples on either side of the run, identical to the procedure described for blinks. These censored periods were then linearly interpolated (median total proportion interpolated data per subject: 0.13, range 0.01–0.40). The pupil time course was then low-pass filtered with a 4 Hz cutoff. The data were then normalized by z-scoring within-subject across data from all functional runs. Gaze position data for time points missing or removed from the pupil time course were also interpolated. Blocks in which > 50% of the pupil data were missing or censored were not included in the analysis (two blocks from one subject).

2.9.1. Pupil analysis

Baseline pupil diameter was calculated as the average diameter in the last 500 ms of the fixation period at trial start. For trial-level analyses, data were downsampled to 50 Hz (all pupil downsampling was performed with Matlab decimate), and all models included vertical and horizontal gaze position as covariates. Analyses of the choice period also controlled for baseline pupil diameter at the start of the trial. Analyses for the outcome period instead controlled for average pupil diameter in the last 250 ms of the wait period between the end of the choice window and the onset of the outcome stimulus.

For the post-explore pupil analysis, pupil diameter was downsampled to 2 Hz, since the focus was on slower changes in diameter over a longer timescale. For pre- and post-explore pupil analyses, we used the same restrictions on the data as described for integration (Section 2.7.1), except we relaxed the minimum number of exploit trials post-explore to 2. For analyses of the post-explore peak/minimum, we identified peaks as the maximum dilation in the period from 0–12 s post-explore. The post-peak minimum was then identified in the period from the peak to 18 s post-explore.

2.10. Pupil–network relationships

To characterize the relationship between pupil-linked arousal and integration, we first downsampled pupil diameter to the sampling rate of the TR and then applied a low-pass filter by convolving it with a Gaussian with a standard deviation equal to that of the median wavelet scale used to compute wavelet coherence for the network analysis (9.80 s). Finally, we downsampled the filtered time course to the sampling rate of the integration time course (0.5 Hz). We then computed the cross-correlation between the pupil diameter and each network measure over the peri-explore period, using the same peri-explore criteria described for peri-explore integration (section 2.7.1). To plot the cross-correlation and compute within- and across-subject averages, we first Fisher z-transformed the correlations. Because the presence of autocorrelation biases the variance of sample correlations, we corrected the z-transformed correlations for this bias, using the method of Pyper and Peterman (1998), producing Z-scores (Afyouni et al., 2019). This procedure essentially weights each z value in proportion to its effective degrees of freedom. We then averaged the Z-scores within subject and assessed the significance of the correlation at the peak lag using a one-sample t-test against 0, across subjects.

2.11. Data analysis

Statistical analyses were performed in R (R R Core Team, 2019). Linear and logistic mixed effects models were implemented in the lme4 package (Bates et al., 2015b), except when an AR1 model was fit for the residuals, in which case nlme was used (Pinheiro et al., 2019). Where possible, models included random intercepts for subjects and random slopes for all within-subjects variables (i.e., the maximal model; Barr et al., 2013). In cases where the maximal model failed to converge or produced singular fits, we iteratively reduced the random effects structure until convergence, following steps outlined by Bates and colleagues (Bates et al., 2015a). Post-hoc comparisons were computed using the emmeans package (Lenth, 2016). GAMMs for the analysis of peri-explore integration time courses were implemented in the mgcv package (Wood, 2017). Where noted, significance levels were corrected for multiple comparisons using the Bonferroni-Holm method (Holm, 1979).

3. Results

We first characterized the pupil response to exploration in order to confirm that exploration leads to increases in pupil-linked arousal (Jepma and Nieuwenhuis, 2011). We also established the onset and time course of this response relative to choice behavior, which licenses inferences about the underlying neuromodulatory dynamics and their relationship to exploratory state. We then examined the dynamic modulation of global brain network integration around exploration, utilizing generalized additive mixed models (GAMMs), a semi-parametric regression approach (see Methods Section 2.7.1) that provides a principled method of uncovering nonmonotonic fluctuations in slowly-varying time series, obviating the need to pre-specify the functional form of the modulation (i.e., impose linearity) or to enforce arbitrary decisions related to time-averaging. We then asked whether there was heterogeneity in the modulation of integration across cognitive systems, which is relevant to assessing whether shifts into an exploratory state reflect a brain-wide phenomenon or possess anatomical specificity. We also examined additional measures of connectivity and topology to provide convergent evidence for the dynamic modulation of integration and to better characterize the form of this modulation. Finally, we utilized a cross-correlation approach to relate pupil dynamics and brain network dynamics, providing evidence for arousal-linked modulation of brain network dynamics at a relatively fine temporal scale. As a comparison with prior studies, we also examined block-level changes in pupil diameter and brain network dynamics. These results are reported in section S3 of the Supplementary Material.

3.1. Exploration modulates pupil dilation

Confirming our prediction, pupil dilation responses were higher for explore choices relative to exploit choices (Fig. 2A,B). The difference was reliable beginning 260 ms before the button press and continued to be reliable for the remainder of the choice period (all ps_corrected < 0.03). Elevated baseline pupil diameter has been previously found prior to exploration, distraction, and disengagement (Ebitz and Platt, 2015; Gilzenrat et al., 2010; Jepma and Nieuwenhuis, 2011). Given the results of these prior studies, we also examined baseline pupil diameter. While there was no overall difference in baseline diameter between explore and exploit trials (Fig. 2C, p > 0.31), baseline pupil diameter varied significantly among the three trials just prior to and including the explore trial (Fig. 2D; F(2, 5032) = 6.56, p = 0.001). This effect was driven primarily by a decrease in pupil diameter from the second to the first trial pre-explore (β = −0.06, 𝑡 (5032) = −3.62, p_corrected = 0.009), potentially reflecting in part the diminishing influence of the previous exploratory choice. Although pupil diameter rose on the explore trial relative to the immediately preceding trial, this rise was not significant (β = 0.03, 𝑡 (5032) = 1.80, p_corrected = 0.14), and baseline diameter on the explore trial was still numerically smaller than that of two trials previous (β = −0.03, 𝑡 (5032) = −1.81, p_corrected = 0.14). This finding indicates that in this task, increased post-explore pupil responses were driven by the explore choice itself and not by a gradual ramping of arousal.

Because pupil dilation is also modulated by outcomes, particularly if they are surprising (Alamia et al., 2019; Friedman et al., 1973; Lavín et al., 2014; Nassar et al., 2012; Preuschoff, 2011), we also examined pupil dilation in response to changes in payoffs. In the Leapfrog task, because payoffs are deterministic except for the stochastic jumps, outcomes will either be the same as when the option was last checked, or they will have jumped in value. Therefore, we divided trials into three classes, based on whether subjects explored and the payoff increased (explore–change), explored and the payoff was unchanged (explore–no change), or exploited and the payoff was unchanged (exploit–no change). Trials in which subjects exploited and the payoff increased (exploit–change) were excluded from the analysis as there were very few per subject (M = 5.94). Given the paucity of the exploit–change type, we contrasted the response to change within explore trials only.

Pupil dilation was slightly elevated in response to a change in outcome (Fig. 3A,B). This separation began to emerge in the averaged data around 500 ms after the outcome presentation but was only reliable in the last 100 ms of the outcome period (all ps_corrected < 0.047). This effect was much smaller in magnitude than the continued effect of exploration on the pupil response (contrast of explore trials with exploit–no change trials), which was reliable throughout the outcome period (Fig. 3A,C; all ps_corrected < 0.0001). Note that this effect is not simply due to the difference present at the end of the choice period, as these analyses controlled for average pupil diameter in the 250 ms prior to outcome presentation; rather, this effect appears to reflect an extended influence of exploration on post-choice arousal.

We next sought to characterize the duration of the arousal response (Fig. 4A). Pupil diameter was significantly elevated above the explore-trial baseline for 7.5 s post-choice, approximately the start of the outcome period of the subsequent trial (all ps_corrected < 0.015). This result held when controlling for gaze position (all ps_corrected < 0.039) and when additionally constraining the analysis to those epochs with minimal eye movements (< 50 pixels root mean squared; all ps_corrected < 0.028). The sustained duration of the effect also does not appear to be primarily attributable to an artifact of averaging over subjects with variable exploration responses (Fig. 4B). The median peak exploration response (median of within-subject medians) from 0–12 s post-explore occurred 4.0 s post-choice, which is very similar timing to the peak at 3.5 s in the time-averaged data. Furthermore, most individual subjects’ median peaks were not significantly different from the group median (31/34 subjects, sign test [corrected]). Similarly, the median minimum pupil dilation in the window from the post-explore peak to 18 s post-explore was 14.5 s, identical to the time-averaged minimum. The minimum in all subjects was consistent with this group median (34/34 subjects, sign test [corrected]). Nor was the time course significantly modulated by outcome type, although there was a small modulation that was significant at an uncorrected p < 0.05 level from 4–5.5 s post-choice, consistent with the effect seen at the trial level at the end of the outcome period and extending into the ITI and the start of the subsequent trial (Figure S3). The smearing out of the outcome effect by time-locking on choice, as well as the trial restrictions imposed on this analysis, may have made it more difficult to detect the small modulation by outcome found in the trial-level data.

Increases in pupil diameter with exploratory choice thus seem potentially explained as a transient increase in tonic arousal driven purely by the choice to shift from exploitation to exploration, rather than an artifact or a response to the outcome. Nor does increased arousal appear to be the cause of the exploratory choice, rather than its effect. However, an additional possibility is that pupil dilation in response to exploration is due to the greater uncertainty in the outcome on explore trials as compared to exploit trials. Indeed, the probability of observing a change in option value on explore trials is fairly uncertain (P(change | explore) = 0.41), while it is unlikely on exploit trials (P(change | exploit) = 0.13). If uncertainty were driving the response, it might be expected that the pupillary response to exploration would differ between volatility conditions, as P(change | explore) was higher in the high volatility blocks (P(change | explore,high) = 0.57; P(change | explore,low) = 0.24). This was not the case. There was no effect of volatility condition, nor any volatility x choice type interaction during the choice period (Figure S4; all ps_corrected = 1). Similarly, there was no effect of volatility condition on the post-explore time course (Figure S4; all ps_corrected > 0.62). Given subjects’ overall weak sensitivity to the volatility conditions (Figure S8A), these results do not completely rule out a role for uncertainty, but they raise the possibility that exploratory choice itself, isolated from effects of uncertainty or surprise, can drive shifts in arousal (see Section 4.3 for further discussion).

3.2. Exploration transiently modulates peri-explore integration

Integration was also significantly modulated around exploration (Fig. 5A.; F(3.32, 4551.90) = 4.03, p = 0.002). Integration appears to increase leading up to exploration, peak around the explore choice, and fall thereafter. To rule out the possibility that this result was reflective of some more general oscillation in the data, we refit the GAMM on data in which the location of explore trials was permuted within each block (500 permutations; see Methods Section 2.7.1). A permutation test suggested that the modulation was unique to exploration (p = 0.006).

To understand the factors driving this change in integration, it is important to answer two questions: (1) Which cognitive systems and their interactions contribute most to these dynamics? and (2) How do changes in integration relate to other global network properties?

3.3. Evidence for differential modulation of integration across cognitive systems

To answer the first question, we computed system-level integration, which we define as the integration of each cognitive system with all other systems (i.e., with the rest of the brain; see Methods Section 2.7, Eq. 4). While qualitatively there was some evidence of global modulation when examining each cognitive system individually, this effect was only significant for the dorsal attention, default, frontoparietal, and limbic systems (Fig. 6A.; dorsal attention: F(3.19, 4505.57) = 4.05, p_corrected = 0.018; limbic: F(3.32, 4623.70) = 3.53, p_corrected = 0.036; frontoparietal: F(3.30, 4554.87) = 3.40, p_corrected = 0.037; default: F(4.05, 4548.81) = 5.31, p_corrected = 0.0006). To provide stronger evidence for differences across systems, we conducted a model comparison procedure in which we asked whether the data were better fit by a model with separate time courses for each system (full model) or by model with a single time course for all systems (shared model). The full model failed to provide a sufficiently better fit to overcome the additional degrees of freedom required to fit separate time courses (Δdf = 14, ΔAIC = 11.02).

Fig. 6. — The modulation of peri-explore integration varies by cognitive system. A The integration of each cognitive system with all other systems (i.e., the rest of the brain). B Pairwise integration between cognitive systems that demonstrated a significant modulation around exploration. See Figure S5 for all between-system time courses. ∗ p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001.

We then asked whether any interactions between cognitive systems differentially contributed to the system-level changes by computing between-system integration, the integration of two cognitive systems with each other (Eq. 5). Qualitatively, only some between-system interactions appeared to change around integration (Figure S5), and significant modulation of between-system integration was found only for the dorsal attention–limbic, dorsal attention–default, and frontoparietal–default interactions (Fig. 6B; dorsal attention–limbic: F(3.36, 4466.03) = 5.36, p_corrected = 0.006; dorsal attention–default: F(3.57, 4428.86) = 5.11, p_corrected = 0.007; frontoparietal–default: F(4.03, 4368.86) = 5.19, p_corrected = 0.003). However, the model comparison procedure again failed to demonstrate a better fit for a full model compared to a shared model (Δdf = 54, ΔAIC = 19.51).

Together, the system-level and between-system results provide suggestive but inconclusive evidence for specificity in changes in integration. While it is not the case that integration was reliably modulated throughout the brain, these differences were not themselves reliable. For between-system interactions that were reliably modulated, the effects could be reflective of common neuromodulatory input (van den Brink et al., 2019), of interactions between these systems underlying decisions to explore, or of changes in interactions between these systems providing the substrate for exploratory states.

3.4. Exploration induces complex changes in connectivity and topology

Regarding the second question above, changes in integration between cognitive systems could be accompanied by other changes in the underlying connectivity and topology. For example, although integration is based on network topology and not directly on connectivity, intuitively increases in integration might reflect a shift toward increased functional connectivity strength. Contrary to this expectation, average node strength demonstrated an opposing profile to integration, reaching a minimum and plateauing close to the time of choice (Fig. 7A; F(3.80, 4297.55) = 7.64, p < 0.0001). To assess whether strength changed differentially within and between cognitive systems, potentially contributing to the change in integration, we computed a strength-based measure of system segregation—the difference in within- versus between-system connectivity, as a percentage of within-system connectivity (Chan et al., 2014; see Methods section 2.8, Eq. 7). Thus, increases in this quantity reflect an increase in the relative strength of within-system connectivity. While both within- and between-system connectivity demonstrated a qualitatively similar peri-explore profile (Figure S6), system segregation demonstrated a positive modulation in favor of within-system connectivity (Fig. 7B; F(3.18, 4337.78) = 4.79, p = 0.0007). This result was not driven by a mismatch between the assignment of nodes to cognitive systems relative to the dynamic modular structure of the network, as a similar pattern was obtained when computing system segregation relative to the module assignment at every time point (Figure S6; F(3.38, 4449.78) = 5.24, p = 0.0002).

Fig. 7. — A Average node strength, B system segregation, C number of modules, and D modularity all showed significant modulations in the peri-explore period.

This increase in system segregation, usually inferred to reflect a decrease in the integration of network modules, suggests that the positive modulation of integration may rather reflect a transient topological shift toward fewer modules. This was indeed the case (Fig. 7C; F(4.38, 4268.60) = 5.27, p < 0.0001). We then asked how these changes in connectivity and topology related to the (single-layer) modularity of the network (see Methods section 2.8, Eq. 8), which is also often considered a measure of segregation (Rubinov and Sporns, 2010). Because modularity is a measure of the extent to which intra-module strength is greater than expected, it might be predicted to positively associate with system segregation. Alternatively, it could be predicted to track with the number of modules, as fewer modules often accompany a less modular structure. Here, we found that modularity demonstrated a positive fluctuation during the peri-explore period, in line with the increase in system segregation (Fig. 7D; F(3.68, 4522.20) = 6.10, p < 0.0001). Finally, while the network measures demonstrated similar time courses around exploration and were moderately to strongly intercorrelated (0.39–0.86, in absolute value; Table S1), network measures computed over functional connectivity matrices in which the topological structure of the network was permuted demonstrate significant differences in correlation structure (Table S2), suggesting that the functional organization of the brain during the task strongly drove the relationships in the data.

In sum, around exploration, there was a temporary shift toward a smaller collection of more loosely connected modules that included nodes from a greater diversity of cognitive systems. This counterintuitively led to an increase in measures normally taken to measure segregation (modularity, system segregation), while at the same time increasing our measure of integration. While these results are consistent with our hypothesis that integration would be modulated around exploration, they are not entirely in line with the directionality of the hypothesis—that exploration would decrease integration. This inconsistency is due both to the heterogeneity across measures and to the fact that the integration results could be consistent with either a localized peak concomitant with exploration, or with an increase during exploitation followed by a decrease following exploration. Unfortunately, the temporal resolution of our analysis is not sufficient to fully disentangle these two possibilities. Notably, using wavelet analysis, the minimum size of an effect produced by a transient will be approximately the size of the wavelet’s “cone of influence” (COI), which is the central segment of the wavelet in which changes in the underlying signal have the greatest impact on wavelet power (Torrence and Compo, 1998; see Figure S7 for a visualization of the COIs in this study). Qualitatively, the integration and modularity time courses might be consistent with a transient, while the shifts in strength and the number of modules appear longer lasting and potentially indicative of more enduring changes to the network around exploration. We return to these topics in the discussion.

3.5. The relationship between pupil-linked arousal and network integration and segregation

Both pupil diameter and measures of network integration and segregation were modulated around exploration, raising the possibility that pupil-linked arousal systems, such as the LC-NE system, influence integration during exploration, as hypothesized. To more formally assess this possibility, we computed the cross-correlation between pupil diameter and our network measures (see Methods section 2.10). All measures demonstrated a peak at lag 0 (Fig. 8), so we therefore assessed the significance of the zero-lag correlation across subjects. This relationship was weak overall, with the only significant correlation occurring for pupil–strength (r_ave = 0.157, t(33) = 2.57, p = 0.015). However, the relationship was at a trend level for all other measures but integration (integration: r_ave = −0.035, t(33) = −0.88, p = 0.38; system segregation: r_ave = −0.098, t(33) = −1.77, p = 0.085; number of modules: r_ave = 0.086, t(33) = 2.01, p = 0.053; modularity: r_ave = −0.079, t(33) = −1.90, p = 0.066). This finding extends prior work demonstrating a positive association between pupil diameter and overall strength of functional connectivity at the block level (Eldar et al., 2013; Warren et al., 2016). Given that the other measures are all ultimately derived from connectivity strength, it may be that further noise introduced by those calculations—particularly those involving the computation of modularity—may have served to partially obscure these relationships. It may also be the case that the effect of brainstem arousal systems during exploration is best characterized as influencing overall connectivity strength, which then interacts with other factors to affect these other measures. Yet taken together, these results suggest a role for pupil-linked neuromodulatory activity in the complex changes in network connectivity and topology around exploration.

Fig. 8. — Pupil–network cross-correlations. The cross-correlation between each network measure and the downsampled and low-pass-filtered pupil time course during the peri-explore period. Average cross-correlations and SEMs were computed by first Fisher z-transforming the correlations at each lag, and then back-transforming for display.

3.6. Accounting for task-evoked confounds

A potential concern with the preceding analyses is that the functional interactions could reflect parallel responses to task events in the absence of genuine neural interactions (Cole et al., 2019; O’Reilly et al., 2012). In order to address this confound, we re-ran all network-based analyses, after additionally regressing out task events from the BOLD time series (see Methods section 2.4, Tables S3–S5). All principal results reported without task regression hold except: (1) The subcortical system now shows significant modulation of integration around exploration F(2.89, 4605.64) = 3.58, p_corrected = 0.043. (2) The modulation of integration between the dorsal attention and limbic systems fell to a trend level (F(2.96, 4465.94) = 4.24, p_corrected = 0.072). (3) The correlations for pupil–system segregation, pupil–modularity, and pupil–number of modules fell below trend-level (all ps > 0.13). Crucially, the pupil–strength correlation remained significant (r_ave = 0.154, t(33) = 2.60, p = 0.014). In short, task-evoked responses were not a strong driver of the exploration-induced modulations reported above.

4. Discussion

Here we assessed the relationship between changes in pupil diameter, brain network integration, and behavior in the context of exploratory choice. Consonant with our predictions and corroborating previous findings (Jepma and Nieuwenhuis, 2011), we found that exploration induced a reliable increase in pupil diameter. This increase is consistent with the adaptive gain theory of LC-NE function, which states that changes in tonic LC firing mediate between states of exploration and exploitation (Aston-Jones and Cohen, 2005). We also examined changes in brain network integration around exploration. While our hypothesis that integration would be modulated around exploration was confirmed, the simple directionality of the hypothesis was not. Rather than finding strictly reduced integration, exploration-linked alterations in functional network architecture across a range of measures were consistent with a shift toward fewer, more weakly connected modules that were both more segregated in terms of connectivity and topology but also more integrated with respect to the diversity of cognitive systems represented in each module. Importantly, overall functional connectivity strength decreased, and changes in connectivity were associated with changes in pupil diameter, in line with the hypothesis that changes in LC-NE or other pupil-linked neuromodulatory activity contribute to the dynamic reorganization of brain networks. These findings are the first to tightly link arousal, brain network dynamics, and behavior in human subjects, going beyond prior studies, which relied on incidental variations in arousal or pharmacological manipulation assayed over longer periods of time. In so doing, this study has pushed the temporal grain at which sliding-window network analyses have been applied, indicating the possibility of using these methods to uncover finer-timescale changes when carefully coupled to behaviors of interest.

4.1. Complex peri-explore network dynamics

We found that during the peri-explore period, when the brain has fewer, larger modules, cognitive systems are more weakly connected and less internally homogenous (lower connectivity strength), and the ratio of within- versus between- system strength is higher (system segregation). Having fewer modules in turn increases integration because regions from different cognitive systems become intermingled in these loose modules. This pattern of results suggests that when brain functional connectivity is relatively high, its topological structure tends to better respect the boundaries of the resting-state cognitive systems. When overall connectivity strength is lower, such as during exploration, the boundaries between cognitive systems tend to dissolve, favoring larger, looser modules. Despite this, system segregation increases because it is a relative measure, so connectivity within cognitive systems need only decrease less than connectivity between cognitive systems.

The overall decrease in connectivity strength during the peri-explore period may thus be particularly important in driving the present results. Closely mirroring our findings, in a model of coupled oscillators, global decreases in coupling strength can lead to decreases in synchronization both within and between modules, as well as increases in modularity (Zhao et al., 2010). Changes in coupling strength have also been a target of modeling the effect of neuromodulatory systems on brain networks, which can lead to nonlinear changes in the degree of integration in the network (Shine et al., 2018a).

However, the complex changes in functional network architecture during the peri-explore period contrast with some prior findings in the literature. For example, performing the cognitively demanding n-back task has been found to increase brain network integration as measured in the present study (Braun et al., 2015), as measured by the diversity of intermodular connections (participation coefficient; Shine et al., 2016), and as measured by the average path length between nodes (global efficiency; Cohen and D’Esposito, 2016). It has also been found to decrease modularity (Cohen and D’Esposito, 2016; Vatansever et al., 2015) and system segregation (Cohen and D’Esposito, 2016)—both taken as measures of segregation—and decrease the number of modules (Vatansever et al., 2015). In the n-back task, all measures converge on a depiction of brain networks that have become more integrated (less segregated) in their connectivity and topology. Indeed, while integration and segregation can be measured separately (Deco et al., 2015; Rubinov and Sporns, 2010), such measures display anticorrelations in both computational models (Deco et al., 2015) and empirical data (Cruzat et al., 2018), as is also implied by the findings from the n-back data across studies.

The divergence between these findings and the conflicting changes in integration and segregation found during exploration highlight the need to assess putative changes in integration across a range of tasks and measures. Network measures differ in whether they are based on topology or connectivity strength and differ in their focus on within- or between-network interactions. Therefore, different measures can be expected to show different sensitivities. For example, a neural network model trained on multiple measures of segregation and integration was better able to predict performance across a range of tasks than the individual measures alone, suggesting that each contributes unique information (Bertolero et al., 2018). Additionally, some network measures are relative, and so apparently contradictory changes may not be contradictory in relative terms, such as our finding of increased system segregation in the presence of increased integration. Moreover, as implied by our initial hypotheses, more integration—however defined—may not always be better. For example, performance in motor tasks has been shown to benefit from increased segregation of brain networks (Bassett et al., 2015; Cohen and D’Esposito, 2016). Indeed, it has been suggested that more modular brain networks are of benefit in simple tasks that rely on segregation of processing and relatively isolated cognitive systems, while less modular networks are better in more complex tasks that require integrated processing (Yue et al., 2017).

This discussion raises the following question: What is the benefit of modulating integration in the context of exploration, which is not well-captured by the distinction between simplicity and complexity? Indeed, these changes in state occur in the context of a single task. Modeling work suggests that networks constrained to be sparser and more modular in some cases are better at converging to the solution in a given task (Bernatskiy and Bongard, 2015) and better adapt to task changes (Clune et al., 2013). Importantly, structural brain networks are not only modular, but also small-world, characterized by high clustering and short path lengths (Bassett and Bullmore, 2006). While small-world networks need not be modular, this property of the brain has been proposed to balance the segregated processing afforded by modularity with the integrative processing afforded by more global connectivity (Bassett and Bullmore, 2006; Gallos et al., 2012). Interestingly, small-world topology has been shown to impact exploration and exploitation in the context of problem-solving networks. In such networks, agents attempt to find the best solution to a problem in parallel (e.g., guessing the number that yields the highest payoff), where individuals connected to each other in the network have access to one another’s answers. Networks of human subjects as well as simulated agents display more exploration of the problem space in less connected networks, as greater segregation of information promotes the coexistence of a greater variety of possible solutions (Lazer and Friedman, 2007; Mason et al., 2008). While fully connected networks excel in unimodal problem spaces, small-world networks excel in multimodal problem spaces (Mason et al., 2008). Notably, some of the same benefits of structural connectivity can be obtained by changing the dynamics, such that agents can only occasionally view the solutions of their network neighbors (Bernstein et al., 2018; Lazer and Friedman, 2007). As may be expected, these results are highly dependent on the type of problem to be solved (Mason and Watts, 2012; Shore et al., 2015), and they come from networks quite distinct from brain networks. However, they suggest the intriguing possibility that dynamically increasing segregation in the brain during exploration may increase its ability to flexibly adapt when exploring new problem spaces or environments. The fact that the overall number of modules decreased, contributing to an increase in integration of different cognitive systems, may serve to balance this segregation by increasing the diversity of processing within each module. Although these ideas are speculative by way of analogy to other networks, they suggest important areas for future research utilizing neural network models.

4.2. Specificity of network effects

While brainstem neuromodulatory nuclei project widely throughout the cortex, there appears to be some specificity in their effects on brain networks (van den Brink et al., 2019). To take the LC as an example, while some studies have suggested that LC-NE-linked modulation of network connectivity is relatively global, in keeping with the diffuse projections of LC (Eldar et al., 2013), others have uncovered heterogeneity in these effects and linked those heterogeneities to catecholamine receptor distributions (van den Brink et al., 2018, 2016b; Zerbi et al., 2019). Furthermore, recent work in rodents indicates that LC neuron projections and the interactions among LC ensembles are far more regionally specific with respect to their cortical targets than previously appreciated (Totah et al., 2019).

We also found suggestive evidence for anatomical specificity: modulation of integration around exploration was most reliable in the default, dorsal attention, limbic, and frontoparietal systems and their interactions. While the default mode system was initially defined based on its decreased activity during task (Raichle, 2015), a growing body of work suggests its relevance for task processing. In particular, it has been implicated in working memory (Vatansever et al., 2015), task switching (Crittenden et al., 2015), attentional shifting (Arsenault et al., 2018), and creative cognition (Beaty et al., 2016). Of particular relevance to the present study, neurons in posterior cingulate—a default mode area—have been implicated in performance monitoring (Heilbronner and Platt, 2013) and exploration (Pearson et al., 2009). There is also prior evidence of dynamic interactions between default, frontoparietal, and dorsal attention systems, with the frontoparietal system potentially regulating activity in the other two systems in order to adjust the balance between internally-generated (default) and externally-directed (dorsal attention) processing (Beaty et al., 2016; Dixon et al., 2018, 2017; Smallwood et al., 2012). Furthermore, interactions among limbic, attentional, and catecholamine systems appear to modulate attention, learning, and memory for salient or motivationally relevant events (Clewett and Murty, 2019; Gallagher and Holland, 1994; Mohanty et al., 2008). The Leapfrog task itself has been associated with both prefrontal function and arousal (Blanco et al., 2015; Otto et al., 2014). While we can only speculate about the role of these systems and their interactions in the present study, they may reflect the coordination of monitoring, decision-making, and attentional processes in service of flexibly shifting between exploitation and exploration based on ongoing estimates of the relative value of exploring.

These signs of specificity must be qualified by the fact that the differences among the time courses were not themselves reliable—model comparison did not favor GAMMs with separate time courses for each system or between-system interaction over one shared time course, though these are likely conservative tests due to the large number of additional degrees of freedom needed to fit individual time courses over a shared time course. This is particularly the case for the between-system analysis (28 pairwise interactions), where visual inspection suggests very little modulation in some interactions. Future work should continue to examine the role of the LC and other neuromodulatory structures in regulating brain network connectivity within specific contexts to provide further evidence for the existence of global versus task- and region-specific neuromodulatory effects. Such efforts would benefit from planned comparisons between regions or networks to increase the power to detect differential effects.

4.3. Pupillary response to exploratory state

While it was not a primary goal of the study, our results also bear strongly on the role of LC-NE-linked arousal in mediating between exploration and exploitation. Despite the long-standing hypothesis that tonic LC activity mediates between these states (Aston-Jones and Cohen, 2005), relatively few studies have examined this relationship, although most have found support for such a relationship (Gilzenrat et al., 2010; Hayes and Petrov, 2016; Jepma and Nieuwenhuis, 2011; Kane et al., 2017; cf. Jepma et al., 2010; Warren et al., 2017). Despite this, open questions remain about the nature of the relationship. Pupil diameter is sensitive to several non-luminance-mediated factors, including uncertainty and surprise (Alamia et al., 2019; Friedman et al., 1973; Jepma and Nieuwenhuis, 2011; Lavín et al., 2014; Nassar et al., 2012; Preuschoff, 2011; Qiyuan et al., 1985; Urai et al., 2017; Zénon, 2019), as well as mental load or task difficulty (Alnæs et al., 2014; Hess and Polt, 1964; Kahneman and Beatty, 1966; Wahn et al., 2016). Notably, past task designs used to test the relationship between LC-NE-linked activity and exploratory state do not always clearly differentiate states of exploration from these other factors. For example, a canonical study of exploration—operationalized as task disengagement—utilized increases in task difficulty to promote disengagement (Gilzenrat et al., 2010). It could thus be the case that pupil diameter in this study was more related to other variables than to exploration per se; indeed, it was argued to closely track expected utility, a putative signal of when to initiate exploration (Aston-Jones and Cohen, 2005). While it is an empirical question whether states of exploration reduce to states of uncertainty or low utility, the information gained by exploration has utility in and of itself, despite the opportunity costs associated with potentially lower payoffs (e.g., directed exploration; Gershman, 2018; Kaelbling et al., 1996; Knox et al., 2012; Wilson et al., 2014). Furthermore, mice demonstrate elevated pupil diameter during exploratory behaviors that are not associated with immediate payoffs (McGinley et al., 2015), and tonic LC stimulation induces disengagement and increased decision noise during patch foraging in rats, which are putative markers an exploratory state (Kane et al., 2017). Exploratory states would thus seem to be at least somewhat separable from these other factors and potentially heterogenous in nature. The simplified nature of the Leapfrog task partially mitigates these concerns; the option values change in a highly constrained way, meaning most of the uncertainty/difficulty lies in the decision of when to explore, given the rate of change in the environment (Knox et al., 2012).

Another question concerns the timing of arousal fluctuations relative to exploratory actions. A key prediction of the adaptive gain theory is that tonic LC activity and baseline pupil diameter should increase preceding exploration or disengagement and then fall when transitioning to exploitation, based on the proposal that tonic LC activity tracks expected utility (Aston-Jones and Cohen, 2005; Gilzenrat et al., 2010; Jepma and Nieuwenhuis, 2011). Crucially, we found no anticipatory increase in pupil diameter leading up to the explore trial. This suggests that increased arousal was a consequence of the decision to explore, rather than its cause. This is partially at odds with a prior report that found elevated baseline pupil diameter prior to explore trials—though not on exploit trials leading up to exploration (Jepma and Nieuwenhuis, 2011), and might therefore be taken as support for the supposition that baseline pupil was tracking a quantity such as utility in prior studies but not here. While the numerical value of the exploit option does not diminish in the Leapfrog task, utility does diminish, as a normative account of the task demonstrates that the relative value of choosing the exploit over the explore option decreases over time, due to the increasing possibility that the explore option has become the better option; human subjects—including in this study (Figure S8B)—demonstrate behavioral signatures at least partially consistent with this model (Knox et al., 2012). Pupil diameter may thus be more sensitive to explicitly decreasing payoffs than to more abstract computations of utility. Task timing might also be a factor. Studies that found anticipatory arousal with either drifting bandits or the Leapfrog task allotted more time before subjects could make a decision, so pre-choice arousal levels might be more reflective of the anticipation of making an exploratory choice (Jepma and Nieuwenhuis, 2011; Otto et al., 2014), whereas our task timing may not have provided sufficient time for these signals to develop.

Following the explore choice, pupil diameter remained elevated for several seconds. Importantly, response to a change in outcome did not drive this effect, making a role for surprise unlikely, and the explore response was not sensitive to volatility condition, suggesting it also was not due to greater uncertainty in the outcome of explore choices. This conclusion must be qualified, however, by the relatively weak sensitivity of subject behavior to volatility level. Given that the pupillary response has been shown to be modulated by probabilities and at least qualitatively demonstrates more extended responses to low probability events (Alamia et al., 2019; Qiyuan et al., 1985; Zénon, 2019), we cannot completely rule out this possibility. Nor can we rule out the possibility that the difference between explore and exploit response was due to anticipated outcome, which prior studies have demonstrated positively modulates pupil diameter (Cash-Padgett et al., 2018; Van Slooten et al., 2018; Varazzani et al., 2015). In this task, we cannot decouple outcome anticipation from outcome uncertainty due to the constrained nature of value changes in this task. Exploration in drifting bandits is associated with lower expected outcomes (Jepma and Nieuwenhuis, 2011), and if subjects in our study were sensitive to the fact that explore choices resulted in improved outcomes on only 41% of trials, they should not have anticipated higher outcomes. On the other hand, if subjects were exploring using a normative strategy, they should have a subjective belief that the explore option is higher-valued before exploring (Knox et al., 2012). However, outcome-related responses are very sensitive to ongoing task conditions (Cash-Padgett et al., 2018; Van Slooten et al., 2018), so there is no reason to believe such anticipatory responses would persist well into the following trial. Based on this pattern of results and the absence of baseline pupil increases prior to choice, we tentatively propose that a shift into an exploratory state is accompanied by an increase in arousal that is independent from the decision variables contributing to the decision to explore. This conclusion is supported by prior work demonstrating that differences in pupil diameter on explore vs. exploit trials survive controlling for numerous other factors related to expected utility, including expected payoff and the entropy of the option values (Jepma and Nieuwenhuis, 2011). Such increases in arousal may have adaptive benefits that are separate from merely encouraging further overt exploratory choices, such as increasing learning rates (Ebitz et al., 2018). While in our study the increase in arousal was only on the order of seconds, we predict it would be more extended in contexts requiring more extended bouts of exploration.

Though it is challenging to draw inferences about neural activity from pupil diameter (Joshi and Gold, 2020), the timing and duration of the pupil response provide some constraints. The canonical pupillary response function has an approximately one second lag to peak and returns to baseline after about two seconds (Hoeks and Levelt, 1993). Pupil responses of similar latency and duration are evoked by single LC spikes and LC microstimulation (Joshi et al., 2016), and response-related LC activity tends to be quite sparse, on the order of single spikes (Kalwani et al., 2014; Varazzani et al., 2015). Pupil diameter on exploit trials closely followed the canonical pattern (Fig. 2A), and is thus suggestive of phasic LC-NE activity. Pupil diameter on explore trials remained elevated for several seconds following the explore choice. This might indicate a brief elevation of tonic LC-NE activity, given the proposed link between the tonic mode of LC firing and exploration (Aston-Jones and Cohen, 2005). Pupil response scales with LC stimulation frequency (Liu et al., 2017), so it is also conceivable the extended response results from a phasic LC response that is much larger than for exploit choices. Finally, the prolonged response could be due to other neuromodulatory influences. In particular, acetylcholine axon activity has been found to more closely track extended pupil dilations and their slow decay, while NE activity correlates more strongly with rapid changes in pupil diameter (Reimer et al., 2016).

4.4. Limitations and future directions

While this study identified exploration-induced modulation of brain network connectivity on a fairly fine temporal scale, there are a few caveats that warrant consideration. First, the low-frequency nature of the continuous wavelet coherence analysis makes it difficult to infer the exact nature of the underlying neural activity. Indeed, filtering, including the use of wavelets, can distort the timing of the underlying signals (de Cheveigné and Nelken, 2019; Yael et al., 2018). Thus, while our analyses provide evidence of an exploration and arousal-linked modulation, the exact nature of the modulation—its timing and directionality—may be quite different than that uncovered here. On the other hand, wavelet analysis has benefits over correlation-based methods in robustness to noise and temporal autocorrelation (Zhang et al., 2016).

Relatedly, we took substantial steps to address temporal autocorrelation in our analyses, including the use of GAMMs, AR1 error models, and corrected correlation Z-scores. Although the impact of temporal autocorrelation—particularly in nonstationary time series—has long been recognized in fields such as economics and statistics (Granger and Newbold, 1974; Johansen, 2012; Phillips, 1986; Yule, 1926), and univariate analyses of fMRI data correct for non-independence in the residuals of fMRI GLM analyses due to autocorrelation (Monti, 2011), autocorrelation has not always been taken into account in psychological and neuro-scientific analyses, including in analyses of pupil–network relationships. This potentially threatens not only statistical inference (i.e., inflated Type I error rate), but also in some cases the validity of the parameter estimates themselves (i.e., inducing spurious correlations). That said, there has been disagreement as to the severity of the autocorrelation problem, likely owing to differences in the underlying signals, the length of the time series, and the assumptions made about the autoregressive processes (Afyouni et al., 2019; Arbabshirani et al., 2014; Baayen et al., 2017; Dean and Dunsmuir, 2016; Elber-Dorozko and Loewenstein, 2018; Honari et al., 2019; Leonardi and Van De Ville, 2015). We have chosen to take this problem seriously, although other solutions, such as pre-whitening or the use of ARIMA models, could have been used, as is recommended by some of these authors. We did not use these methods here because we did not want to eliminate low-frequency signal components (Afyouni et al., 2019; Pyper and Peterman, 1998), but future work could usefully assess the impact of various mitigation strategies not only on functional connectivity itself, but also on its relation to other signals of interest such as pupil diameter. It may also be worth investigating the use of clustering (Khambhati et al., 2018a; Liu et al., 2018; Medaglia et al., 2018) or deconvolution (Karahanoğlu et al., 2013; Wierda et al., 2012) techniques to aid in addressing both issues of temporal precision and autocorrelation.

While we have attributed the peri-explore modulation to exploration, this assumption must be examined in more detail in future studies. Given our task design and limits on the amount of explore trials per subject, we cannot completely disentangle effects of exploration from effects of change, uncertainty, and overall volatility, although we made several attempts to do so. Furthermore, in the Leapfrog paradigm bouts of exploration are usually on the order of a single trial, and we restricted our network analyses to explore choices surrounded by several exploit choices to clearly identify the effects of these individual explore choices. Designs that provoke more extended exploratory states may help to overcome issues related to temporally isolating the effects of exploration, and future work should assess the impact of the frequency and duration of exploratory states on network dynamics. Additionally, we cannot separate effects of exploration from more general effects of attentional shifting. While LC-NE-linked effects on attentional processes are well-known and in some sense are partly constitutive of its influence on exploratory state (Aston-Jones and Cohen, 2005; Corbetta et al., 2008; McGinley et al., 2015; Sara and Bouret, 2012), exploration has been isolated from switching at the single-neuron level (Pearson et al., 2009), so it will be important to better delineate the boundaries of these different processes and states in the future.

Though we have focused on the LC due to its role in regulating both pupil diameter and exploration, other neuromodulators, such as dopamine and acetylcholine, have also been implicated in coordinating brain network dynamics (Birn et al., 2019; Roffman et al., 2016; Shafiei et al., 2019; Turchi et al., 2018; Záborszky et al., 2018) and have been implicated in uncertainty and exploration (Beeler et al., 2010; Fiorillo et al., 2003; Yu and Dayan, 2005). Acetylcholine in particular also influences pupil diameter (Reimer et al., 2016), and therefore we cannot rule out its impact in the present results. Finally, other mechanisms, such as thalamic regulation, have been linked to the control of cortical connectivity (Halassa and Kastner, 2017), which highlights the need look beyond neuromodulators for other mechanisms of brain network reconfiguration.

In sum, we have demonstrated a relationship between exploration, pupil-linked arousal, and brain network dynamics. We argue that forming linkages between functional connectivity, behavior, and physiological markers such as pupil diameter represents a promising path forward for understanding the impact of neuromodulatory actions on brain network dynamics and cognitive processing. More generally, we suggest that carefully aligning dynamic network analyses with task designs can increase the temporal resolution at which behaviourally- and cognitively-relevant modulations can be identified.

Supplementary Material

NIHMS1741054-supplement-1.docx^{(19MB, docx)}

Acknowledgments

We would like to thank Kathryn Graves for help with data collection. We would also like to thank Bradley Love and Nathaniel Blanco for generously sharing the Leapfrog task code.

Funding

This work was supported by a National Institutes of Health Grant [R01DC009209] to ST-S and by a Department of Defense/Army Research Office National Defense Science and Engineering Graduate (ND-SEG) Fellowship [32 CFR 168a] to NT.

Footnotes

Declaration of Competing Interest

The authors declare that they have no competing interests.

Supplementary materials

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.neuroimage.2021.118369.

Data and code availability statement

The MRI preprocessing pipeline code is available at: https://github.com/ntardiff/MRklar. Data and analysis code are available from the corresponding author upon reasonable request.

References

Afyouni S, Smith SM, Nichols TE, 2019. Effective degrees of freedom of the Pearson’s correlation coefficient under autocorrelation. Neuroimage 199, 609–625. doi: 10.1016/j.neuroimage.2019.05.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
Alamia A, VanRullen R, Pasqualotto E, Mouraux A, Zenon A, 2019. Pupil-linked arousal responds to unconscious surprisal. J. Neurosci. 39, 5369–5376. doi: 10.1523/JNEUROSCI.3010-18.2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
Alnæs D, Sneve MH, Espeseth T, Endestad T, van de Pavert SHP, Laeng B, 2014. Pupil size signals mental effort deployed during multiple object tracking and predicts brain activity in the dorsal attention network and the locus coeruleus. J. Vis. 14. doi: 10.1167/14.4.1, 1–1. [DOI] [PubMed] [Google Scholar]
Arbabshirani MR, Damaraju E, Phlypo R, Plis S, Allen E, Ma S, Mathalon D, Preda A, Vaidya JG, Adali T, Calhoun VD, 2014. Impact of autocorrelation on functional connectivity. Neuroimage 102, 294–308. doi: 10.1016/j.neuroimage.2014.07.045. [DOI] [PMC free article] [PubMed] [Google Scholar]
Arnsten AFT, Paspalas CD, Gamo NJ, Yang Y, Wang M, 2010. Dynamic network connectivity: a new form of neuroplasticity. Trends Cogn. Sci. 14, 365–375. doi: 10.1016/j.tics.2010.05.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
Arsenault JT, Caspari N, Vandenberghe R, Vanduffel W, 2018. Attention shifts recruit the monkey default ode network. J. Neurosci. 38, 1202–1217. doi: 10.1523/JNEUROSCI.1111-17.2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
Aston-Jones G, Cohen JD, 2005. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 28, 403–450. doi: 10.1146/annurev.neuro.28.061604.135709. [DOI] [PubMed] [Google Scholar]
Baayen H, Vasishth S, Kliegl R, Bates D, 2017. The cave of shadows: addressing the human factor with generalized additive mixed models. J. Mem. Lang. 94, 206–234. doi: 10.1016/j.jml.2016.11.006. [DOI] [Google Scholar]
Bari A, Aston-Jones G, 2013. Atomoxetine modulates spontaneous and sensory-evoked discharge of locus coeruleus noradrenergic neurons. Neuropharmacology 64, 53–64. doi: 10.1016/j.neuropharm.2012.07.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
Barr DJ, Levy R, Scheepers C, Tily HJ, 2013. Random effects structure for confirmatory hypothesis testing: keep it maximal. J. Mem. Lang. 68, 255–278. doi: 10.1016/j.jml.2012.11.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
Bassett DS, Bullmore E, 2006. Small-world brain networks. Neuroscientist 12, 512–523. doi: 10.1177/1073858406293182. [DOI] [PubMed] [Google Scholar]
Bassett DS, Porter MA, Wymbs NF, Grafton ST, Carlson JM, Mucha PJ, 2013. Robust detection of dynamic community structure in networks. Chaos An Interdiscip. J. Nonlinear Sci. 23, 013142. doi: 10.1063/1.4790830. [DOI] [PMC free article] [PubMed] [Google Scholar]
Bassett DS, Wymbs NF, Porter MA, Mucha PJ, Carlson JM, Grafton ST, 2011. Dynamic reconfiguration of human brain networks during learning. Proc. Natl. Acad. Sci. U. S. A. 108, 7641–7646. doi: 10.1073/pnas.1018985108. [DOI] [PMC free article] [PubMed] [Google Scholar]
Bassett DS, Yang M, Wymbs NF, Grafton ST, 2015. Learning-induced autonomy of sensorimotor systems. Nat. Neurosci. 18, 744–751. doi: 10.1038/nn.3993. [DOI] [PMC free article] [PubMed] [Google Scholar]
Bates D, Kliegl R, Vasishth S, Baayen H, 2015a. Parsimonious Mixed Models.
Bates D, Mächler M, Bolker B, Walker S, 2015b. Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67, 1–48. doi: 10.18637/jss.v067.i01. [DOI] [Google Scholar]
Bazzi M, Porter MA, Williams S, McDonald M, Fenn DJ, Howison SD, 2016. Community detection in temporal multilayer networks, with an application to correlation networks. Multiscale Model. Simul. 14, 1–41. doi: 10.1137/15M1009615. [DOI] [Google Scholar]
Beaty RE, Benedek M, Silvia PJ, Schacter DL, 2016. Creative cognition and brain network dynamics. Trends Cogn. Sci. 20, 87–95. doi: 10.1016/j.tics.2015.10.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
Beeler JA, Daw N, Frazier CRM, Zhuang X, 2010. Tonic dopamine modulates exploitation of reward learning. Front. Behav. Neurosci. 4, 170. doi: 10.3389/fnbeh.2010.00170. [DOI] [PMC free article] [PubMed] [Google Scholar]
Behzadi Y, Restom K, Liau J, Liu TT, 2007. A component based noise correction method (CompCor) for BOLD and perfusion based fMRI. Neuroimage 37, 90–101. doi: 10.1016/j.neuroimage.2007.04.042. [DOI] [PMC free article] [PubMed] [Google Scholar]
Bernatskiy A, Bongard JC, 2015. Exploiting the relationship between structural modularity and sparsity for faster network evolution. In: GECCO 2015 - Companion Publication of the 2015 Genetic and Evolutionary Computation Conference. Association for Computing Machinery, Inc, pp. 1173–1176. doi: 10.1145/2739482.2768472. [DOI] [Google Scholar]
Bernstein E, Shore J, Lazer D, 2018. How intermittent breaks in interaction improve collective intelligence. Proc. Natl. Acad. Sci. USA 115, 8734–8739. doi: 10.1073/pnas.1802407115. [DOI] [PMC free article] [PubMed] [Google Scholar]
Berridge CW, Waterhouse BD, 2003. The locus coeruleus–noradrenergic system: modulation of behavioral state and state-dependent cognitive processes. Brain Res. Rev. 42, 33–84. doi: 10.1016/S0165-0173(03)00143-7. [DOI] [PubMed] [Google Scholar]
Bertolero MA, Yeo BTT, Bassett DS, D’Esposito M, 2018. A mechanistic model of connector hubs, modularity and cognition. Nat. Hum. Behav. 2, 765–777. doi: 10.1038/s41562-018-0420-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
Birn RM, Converse AK, Rajala AZ, Alexander AL, Block WF, McMillan AB, Christian BT, Filla CN, Murali D, Hurley SA, Jenison RL, Populin LC, 2019. Changes in endogenous dopamine induced by methylphenidate predict functional connectivity in nonhuman primates. J. Neurosci. 39, 1436–1444. doi: 10.1523/JNEUROSCI.2513-18.2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
Blanco NJ, Love BC, Cooper JA, McGeary JE, Knopik VS, Maddox WT, 2015. A frontal dopamine system for reflective exploratory behavior. Neurobiol. Learn. Mem. 123, 84–91. doi: 10.1016/j.nlm.2015.05.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
Blondel VD, Guillaume JL, Lambiotte R, Lefebvre E, 2008. Fast unfolding of communities in large networks. J. Stat. Mech. Theory Exp. 2008, P10008. doi: 10.1088/1742-5468/2008/10/P10008. [DOI] [Google Scholar]
Bouret S, Sara SJ, 2005. Network reset: A simplified overarching theory of locus coeruleus noradrenaline function. Trends Neurosci. 28, 574–582. doi: 10.1016/j.tins.2005.09.002. [DOI] [PubMed] [Google Scholar]
Braun U, Schäfer A, Walter H, Erk S, Romanczuk-Seiferth N, Haddad L, Schweiger JI, Grimm O, Heinz A, Tost H, Meyer-Lindenberg A, Bassett DS, 2015. Dynamic reconfiguration of frontal brain networks during executive cognition in humans. Proc. Natl. Acad. Sci. USA 112, 11678–11683. doi: 10.1073/pnas.1422487112. [DOI] [PMC free article] [PubMed] [Google Scholar]
Calhoun VD, Miller R, Pearlson G, Adali T, 2014. The chronnectome: time-varying connectivity networks as the next frontier in fMRI data discovery. Neuron 84, 262–274. doi: 10.1016/j.neuron.2014.10.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
Cash-Padgett T, Azab H, Yoo SBM, Hayden BY, 2018. Opposing pupil responses to offered and anticipated reward values. Anim. Cogn. 21, 671–684. doi: 10.1007/s10071-018-1202-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
Chan MY, Park DC, Savalia NK, Petersen SE, Wig GS, 2014. Decreased segregation of brain systems across the healthy adult lifespan. Proc. Natl. Acad. Sci. USA 111, E4997–E5006. doi: 10.1073/pnas.1415122111. [DOI] [PMC free article] [PubMed] [Google Scholar]
Clewett D, Murty VP, 2019. Echoes of emotions past: how neuromodulators determine what we recollect. eNeuro 6, ENEURO.0108–18.2019. 10.1523/ENEURO.0108-18.2019 [DOI] [PMC free article] [PubMed] [Google Scholar]
Clune J, Mouret JB, Lipson H, 2013. The evolutionary origins of modularity. Proc. R. Soc. B Biol. Sci. 280, 20122863. doi: 10.1098/rspb.2012.2863. [DOI] [PMC free article] [PubMed] [Google Scholar]
Cohen JR, D’Esposito M, 2016. The segregation and integration of distinct brain networks and their relationship to cognition. J. Neurosci. 36, 12083–12094. doi: 10.1523/JNEUROSCI.2965-15.2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
Cole MW, Ito T, Schultz D, Mill R, Chen R, Cocuzza C, 2019. Task activations produce spurious but systematic inflation of task functional connectivity estimates. Neuroimage 189, 1–18. doi: 10.1016/j.neuroimage.2018.12.054. [DOI] [PMC free article] [PubMed] [Google Scholar]
Corbetta M, Patel G, Shulman GL, 2008. The reorienting system of the human brain: from environment to theory of mind. Neuron 58, 306–324. doi: 10.1016/j.neuron.2008.04.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
Craig MM, Manktelow AE, Sahakian BJ, Menon DK, Stamatakis EA, 2018. Spectral diversity in default mode network connectivity reflects behavioral state. J. Cogn. Neurosci. 30, 526–539. doi: 10.1162/jocn_a_01213. [DOI] [PubMed] [Google Scholar]
Crittenden BM, Mitchell DJ, Duncan J, 2015. Recruitment of the default mode network during a demanding act of executive control. Elife 2015, e06481. doi: 10.7554/eLife.06481. [DOI] [PMC free article] [PubMed] [Google Scholar]
Cruzat J, Deco G, Tauste-Campo A, Principe A, Costa A, Kringelbach ML, Rocamora R, 2018. The dynamics of human cognition: increasing global integration coupled with decreasing segregation found using iEEG. Neuroimage 172, 492–505. doi: 10.1016/j.neuroimage.2018.01.064. [DOI] [PubMed] [Google Scholar]
Dalmaijer ES, Mathôt S, Van der Stigchel S, 2014. PyGaze: an open-source, cross-platform toolbox for minimal-effort programming of eyetracking experiments. Behav. Res. Methods 46, 913–921. doi: 10.3758/s13428-013-0422-2. [DOI] [PubMed] [Google Scholar]
Daw ND, O’Doherty JP, Dayan P, Seymour B, Dolan RJ, 2006. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879. doi: 10.1038/nature04766. [DOI] [PMC free article] [PubMed] [Google Scholar]
de Cheveigné A, Nelken I, 2019. Filters: when, why, and how (not) to use them. Neuron 102, 280–293. doi: 10.1016/j.neuron.2019.02.039. [DOI] [PubMed] [Google Scholar]
de Gee JW, Colizoli O, Kloosterman NA, Knapen T, Nieuwenhuis S, Donner TH, Noorden M.van, Bosch J, Wilson R, Cohen J, Nieuwenhuis S, Saunders J, Vickers J, Zhang Y, Stefano N.De, Brady J, Matthews P, 2017. Dynamic modulation of decision biases by brainstem arousal systems. Elife 6, 1127–1141. doi: 10.7554/eLife.23232. [DOI] [PMC free article] [PubMed] [Google Scholar]
Dean RT, Dunsmuir WTM, 2016. Dangers and uses of cross-correlation in analyzing time series in perception, performance, movement, and neuroscience: the importance of constructing transfer function autoregressive models. Behav. Res. Methods 48, 783–802. doi: 10.3758/s13428-015-0611-2. [DOI] [PubMed] [Google Scholar]
Deco G, Tononi G, Boly M, Kringelbach ML, 2015. Rethinking segregation and integration: contributions of whole-brain modelling. Nat. Rev. Neurosci. 16, 430–439. doi: 10.1038/nrn3963. [DOI] [PubMed] [Google Scholar]
Dixon ML, Andrews-Hanna JR, Spreng RN, Irving ZC, Mills C, Girn M, Christoff K, 2017. Interactions between the default network and dorsal attention network vary across default subsystems, time, and cognitive states. Neuroimage 147, 632–649. doi: 10.1016/j.neuroimage.2016.12.073. [DOI] [PubMed] [Google Scholar]
Dixon ML, De La Vega A, Mills C, Andrews-Hanna J, Spreng RN, Cole MW, Christoff K, 2018. Heterogeneity within the frontoparietal control network and its relationship to the default and dorsal attention networks. Proc. Natl. Acad. Sci. USA 115, E1598–E1607. doi: 10.1073/pnas.1715766115. [DOI] [PMC free article] [PubMed] [Google Scholar]
Ebitz RB, Albarran E, Moore T, 2018. Exploration disrupts choice-predictive signals and alters dynamics in prefrontal cortex. Neuron 97, 450–461. doi: 10.1016/j.neuron.2017.12.007, e9. [DOI] [PMC free article] [PubMed] [Google Scholar]
Ebitz RB, Platt ML, 2015. Neuronal activity in primate dorsal anterior cingulate cortex signals task conflict and predicts adjustments in pupil-linked arousal. Neuron 85, 628–640. doi: 10.1016/j.neuron.2014.12.053. [DOI] [PMC free article] [PubMed] [Google Scholar]
Ekman M, Derrfuss J, Tittgemeyer M, Fiebach CJ, 2012. Predicting errors from reconfiguration patterns in human brain networks. Proc. Natl. Acad. Sci. USA 109, 16714–16719. doi: 10.1073/pnas.1207523109. [DOI] [PMC free article] [PubMed] [Google Scholar]
Elber-Dorozko L, Loewenstein Y, 2018. Striatal action-value neurons reconsidered. Elife 7. doi: 10.7554/eLife.34248. [DOI] [PMC free article] [PubMed] [Google Scholar]
Eldar E, Cohen JD, Niv Y, 2013. The effects of neural gain on attention and learning. Nat. Neurosci. 16, 1146–1153. doi: 10.1038/nn.3428. [DOI] [PMC free article] [PubMed] [Google Scholar]
Eldar E, Niv Y, Cohen JD, 2016. Do you see the forest or the tree? Neural gain and breadth versus focus in perceptual processing. Psychol. Sci. 27, 1632–1643. doi: 10.1177/0956797616665578. [DOI] [PubMed] [Google Scholar]
Fedorenko E, Thompson-Schill SL, 2014. Reworking the language network. Trends Cogn. Sci. 18, 120–126. doi: 10.1016/j.tics.2013.12.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
Fiorillo CD, Tobler PN, Schultz W, 2003. Discrete coding of reward probability and uncertainty by dopamine neurons. Science (80-.) 299, 1898–1902. doi: 10.1126/science.1077349. [DOI] [PubMed] [Google Scholar]
Fischl B, 2012. FreeSurfer. Neuroimage 62, 774–781. doi: 10.1016/j.neuroimage.2012.01.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
Fischl B, Salat DH, Busa E, Albert M, Dieterich M, Haselgrove C, Van Der Kouwe A, Killiany R, Kennedy D, Klaveness S, Montillo A, Makris N, Rosen B, Dale AM, 2002. Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron 33, 341–355. doi: 10.1016/S0896-6273(02)00569-X. [DOI] [PubMed] [Google Scholar]
Friedman D, Hakerem G, Sutton S, Fleiss JL, 1973. Effect of stimulus uncertainty on the pupillary dilation response and the vertex evoked potential. Electroencephalogr. Clin. Neurophysiol. 34, 475–484. doi: 10.1016/0013-4694(73)90065-5. [DOI] [PubMed] [Google Scholar]
Friston KJ, Williams S, Howard R, Frackowiak RSJ, Turner R, 1996. Movement-related effects in fMRI time-series. Magn. Reson. Med. 35, 346–355. doi: 10.1002/mrm.1910350312. [DOI] [PubMed] [Google Scholar]
Gallagher M, Holland PC, 1994. The amygdala complex: multiple roles in associative learning and attention. Proc. Natl. Acad. Sci. USA 91, 11771–11776. doi: 10.1073/pnas.91.25.11771. [DOI] [PMC free article] [PubMed] [Google Scholar]
Gallos LK, Makse HA, Sigman M, 2012. A small world of weak ties provides optimal global integration of self-similar modules in functional brain networks. Proc. Natl. Acad. Sci. USA 109, 2825–2830. doi: 10.1073/pnas.1106612109. [DOI] [PMC free article] [PubMed] [Google Scholar]
Gerraty RT, Davidow JY, Foerde K, Galvan A, Bassett DS, Shohamy D, 2018. Dynamic flexibility in striatal-cortical circuits supports reinforcement learning. J. Neurosci. 38, 2442–2453. doi: 10.1523/JNEUROSCI.2084-17.2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
Gershman SJ, 2018. Deconstructing the human algorithms for exploration. Cognition 173, 34–42. doi: 10.1016/j.cognition.2017.12.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
Gießing C, Thiel CM, Alexander-Bloch AF, Patel AX, Bullmore ET, 2013. Human brain functional network changes associated with enhanced and impaired attentional task performance. J. Neurosci. 33, 5903–5914. doi: 10.1523/JNEUROSCI.4854-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
Gilzenrat MS, Nieuwenhuis S, Jepma M, Cohen JD, 2010. Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function. Cogn. Affect. Behav. Neurosci. 10, 252–269. doi: 10.3758/CABN.10.2.252. [DOI] [PMC free article] [PubMed] [Google Scholar]
Good BH, de Montjoye Y-A, Clauset A, 2010. Performance of modularity maximization in practical contexts. Phys. Rev. E 81, 046106. doi: 10.1103/Phys-RevE.81.046106. [DOI] [PubMed] [Google Scholar]
Granger CWJ, Newbold P, 1974. Spurious regressions in econometrics. J. Econom. 2, 111–120. doi: 10.1016/0304-4076(74)90034-7. [DOI] [Google Scholar]
Greve DN, Fischl B, 2009. Accurate and robust brain image alignment using boundary-based registration. Neuroimage 48, 63–72. doi: 10.1016/j.neuroimage.2009.06.060. [DOI] [PMC free article] [PubMed] [Google Scholar]
Grinsted A, Moore JC, Jevrejeva S, 2004. Application of the cross wavelet transform and wavelet coherence to geophysical time series. Nonlinear Process. Geophys. 11, 561–566. doi: 10.5194/npg-11-561-2004. [DOI] [Google Scholar]
Guedj C, Monfardini E, Reynaud AJ, Farnè A, Meunier M, Hadj-Bouziane F, 2017. Boosting norepinephrine transmission triggers flexible reconfiguration of brain networks at rest. Cereb. Cortex 27, 4691–4700. doi: 10.1093/cercor/bhw262. [DOI] [PubMed] [Google Scholar]
Halassa MM, Kastner S, 2017. Thalamic functions in distributed cognitive control. Nat. Neurosci. 20, 1669–1679. doi: 10.1038/s41593-017-0020-1. [DOI] [PubMed] [Google Scholar]
Hasselmo ME, Linster C, Patil M, Ma D, Cekic M, 1997. Noradrenergic suppression of synaptic transmission may influence cortical signal-to-noise ratio. J. Neurophysiol. 77, 3326–3339. doi: 10.1152/jn.1997.77.6.3326. [DOI] [PubMed] [Google Scholar]
Hayes TR, Petrov AA, 2016. Pupil diameter tracks the exploration–exploitation trade-off during analogical reasoning and explains individual differences in fluid intelligence. J. Cogn. Neurosci. 28, 308–318. doi: 10.1162/jocn_a_00895. [DOI] [PubMed] [Google Scholar]
Heilbronner SR, Platt ML, 2013. Causal evidence of performance monitoring by neurons in posterior cingulate cortex during learning. Neuron 80, 1384–1391. doi: 10.1016/j.neuron.2013.09.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
Hess EH, Polt JM, 1964. Pupil size in relation to mental activity during simple problem-solving. Science (80-.) 143, 1190–1192. doi: 10.1126/science.143.3611.1190. [DOI] [PubMed] [Google Scholar]
Highly Accurate Inverse Consistent Registration: A Robust Approach,Reuter M, Rosas HD, Fischl B NeuroImage 53(4), pp. 1181–1196, 2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
Hoeks B, Levelt WJM, 1993. Pupillary dilation as a measure of attention: a quantitative system analysis. Behav. Res. Methods, Instrum., Comput. 25, 16–26. doi: 10.3758/BF03204445. [DOI] [Google Scholar]
Holm S, 1979. A simple sequentially rejective multiple test procedure. Scand. J. Stat. 6, 65–70. doi: 10.2307/4615733. [DOI] [Google Scholar]
Honari H, Choe AS, Pekar JJ, Lindquist MA, 2019. Investigating the impact of autocorrelation on time-varying connectivity. Neuroimage 197, 37–48. doi: 10.1016/j.neuroimage.2019.04.042. [DOI] [PMC free article] [PubMed] [Google Scholar]
Hurley LM, Devilbiss DM, Waterhouse BD, 2004. A matter of focus: monoaminergic modulation of stimulus coding in mammalian sensory networks. Curr. Opin. Neurobiol. 14, 488–495. doi: 10.1016/j.conb.2004.06.007. [DOI] [PubMed] [Google Scholar]
Jenkinson M, 2004. Improving the registration of B0-disorted EPI images using calculated cost function weights. Tenth International Conference on Functional Mapping of the Human Brain. [Google Scholar]
Jenkinson M, 2003. Fast, automated, N-dimensional phase-unwrapping algorithm. Magn. Reson. Med. 49, 193–197. doi: 10.1002/mrm.10354. [DOI] [PubMed] [Google Scholar]
Jenkinson M, Beckmann CF, Behrens TEJ, Woolrich MW, Smith SM, 2012. FSL. Neuroimage 62, 782–790. doi: 10.1016/j.neuroimage.2011.09.015. [DOI] [PubMed] [Google Scholar]
Jepma M, Nieuwenhuis S, 2011. Pupil diameter predicts changes in the exploration-exploitation trade-off: evidence for the adaptive gain theory. J. Cogn. Neurosci. 23, 1587–1596. doi: 10.1162/jocn.2010.21548. [DOI] [PubMed] [Google Scholar]
Jepma M, te Beek ET, Wagenmakers EJ, van Gerven JMA, Nieuwenhuis S, 2010. The role of the noradrenergic system in the exploration-exploitation trade-off: a psychopharmacological study. Front. Hum. Neurosci. 4, 170. doi: 10.3389/fnhum.2010.00170. [DOI] [PMC free article] [PubMed] [Google Scholar]
Jeub LGS, Bazzi M, Jutla IS, Mucha PJ, 2011. A generalized Louvain method for community detection implemented in MATLAB.
Jo HJ, Saad ZS, Simmons WK, Milbury LA, Cox RW, 2010. Mapping sources of correlation in resting state FMRI, with artifact detection and removal. Neuroimage 52, 571–582. doi: 10.1016/j.neuroimage.2010.04.246. [DOI] [PMC free article] [PubMed] [Google Scholar]
Johansen S, 2012. The analysis of nonstationary time series using regression, correlation and cointegration. Contemp. Econ. 6, 40–57. doi: 10.5709/ce.1897-9254.39. [DOI] [Google Scholar]
Joshi S, Gold JI, 2020. Pupil size as a window on neural substrates of cognition. Trends Cogn. Sci. 24, 466–480. doi: 10.1016/j.tics.2020.03.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
Joshi S, Li Y, Kalwani RM, Gold JI, 2016. Relationships between pupil diameter and neuronal activity in the locus coeruleus, colliculi, and cingulate cortex. Neuron 89, 221–234. doi: 10.1016/j.neuron.2015.11.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
Kaelbling LP, Littman ML, Moore AW, 1996. Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285. doi: 10.1613/jair.301. [DOI] [Google Scholar]
Kahneman D, Beatty J, 1966. Pupil diameter and load on memory. Science (80-.). 154, 1583–1585. doi: 10.1126/science.154.3756.1583. [DOI] [PubMed] [Google Scholar]
Kalwani RM, Joshi S, Gold JI, 2014. Phasic activation of individual neurons in the locus ceruleus/subceruleus complex of monkeys reflects rewarded decisions to go but not stop. J. Neurosci. 34, 13656–13669. doi: 10.1523/JNEUROSCI.2566-14.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
Kane GA, Vazey EM, Wilson RC, Shenhav A, Daw ND, Aston-Jones G, Cohen JD, 2017. Increased locus coeruleus tonic activity causes disengagement from a patch-foraging task. Cogn. Affect. Behav. Neurosci. 17, 1073–1083. doi: 10.3758/s13415-017-0531-y. [DOI] [PubMed] [Google Scholar]
Karahanoğlu FI, Caballero-Gaudes C, Lazeyras F, Van De Ville D, 2013. Total activation: fMRI deconvolution through spatio-temporal regularization. Neuroimage 73, 121–134. doi: 10.1016/j.neuroimage.2013.01.067. [DOI] [PubMed] [Google Scholar]
Khambhati AN, Mattar MG, Wymbs NF, Grafton ST, Bassett DS, 2018a. Beyond modularity: fine-scale mechanisms and rules for brain network reconfiguration. Neuroimage 166, 385–399. doi: 10.1016/j.neuroimage.2017.11.015. [DOI] [PubMed] [Google Scholar]
Khambhati AN, Sizemore AE, Betzel RF, Bassett DS, 2018b. Modeling and interpreting mesoscale network dynamics. Neuroimage 180, 337–349. doi: 10.1016/j.neuroimage.2017.06.029. [DOI] [PMC free article] [PubMed] [Google Scholar]
Knox WB, Otto AR, Stone P, Love BC, 2012. The nature of belief-directed exploratory choice in human decision-making. Front. Psychol. 2, 398. doi: 10.3389/fpsyg.2011.00398. [DOI] [PMC free article] [PubMed] [Google Scholar]
Kopell NJ, Gritton HJ, Whittington MA, Kramer MA, 2014. Beyond the connectome: the dynome. Neuron 83, 1319–1328. doi: 10.1016/j.neuron.2014.08.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
Lavín C, San Martín R, Rosales Jubal E, 2014. Pupil dilation signals uncertainty and surprise in a learning gambling task. Front. Behav. Neurosci. 7. doi: 10.3389/fnbeh.2013.00218. [DOI] [PMC free article] [PubMed] [Google Scholar]
Lazer D, Friedman A, 2007. The network structure of exploration and exploitation. Adm. Sci. Q. 52, 667–694. doi: 10.2189/asqu.52.4.667. [DOI] [Google Scholar]
Lenth RV, 2016. Least-squares means: the R package lsmeans. J. Stat. Softw. 69, 1–33. doi: 10.18637/jss.v069.i01. [DOI] [Google Scholar]
Leonardi N, Van De Ville D, 2015. On spurious and real fluctuations of dynamic functional connectivity during rest. Neuroimage 104, 430–436. doi: 10.1016/j.neuroimage.2014.09.007. [DOI] [PubMed] [Google Scholar]
Liu X, Zhang N, Chang C, Duyn JH, 2018. Co-activation patterns in resting-state fMRI signals. Neuroimage 180, 485–494. doi: 10.1016/j.neuroimage.2018.01.041. [DOI] [PMC free article] [PubMed] [Google Scholar]
Liu Y, Rodenkirch C, Moskowitz N, Schriver B, Wang Q, 2017. Dynamic lateralization of pupil dilation evoked by locus coeruleus activation results from sympathetic, not parasympathetic, contributions. Cell Rep. 20, 3099–3112. doi: 10.1016/j.celrep.2017.08.094. [DOI] [PMC free article] [PubMed] [Google Scholar]
Mason W, Watts DJ, 2012. Collaborative learning in networks. Proc. Natl. Acad. Sci. 109, 764–769. doi: 10.1073/pnas.1110069108. [DOI] [PMC free article] [PubMed] [Google Scholar]
Mason WA, Jones A, Goldstone RL, 2008. Propagation of innovations in networked groups. J. Exp. Psychol. Gen. 137, 422–433. doi: 10.1037/a0012798. [DOI] [PubMed] [Google Scholar]
Mather M, Clewett D, Sakaki M, Harley CW, 2016. Norepinephrine ignites local hotspots of neuronal excitation: how arousal amplifies selectivity in perception and memory. Behav. Brain Sci. 39, 1–100. doi:. [DOI] [PMC free article] [PubMed] [Google Scholar]
Mattar MG, Cole MW, Thompson-Schill SL, Bassett DS, 2015. A functional cartography of cognitive systems. PLoS Comput. Biol. 11, e1004533. doi: 10.1371/journal.pcbi.1004533. [DOI] [PMC free article] [PubMed] [Google Scholar]
McGinley MJ, Vinck M, Reimer J, Batista-Brito R, Zagha E, Cadwell CR, Tolias AS, Cardin JA, McCormick DA, 2015. Waking state: rapid variations modulate neural and behavioral responses. Neuron 87, 1143–1161. doi: 10.1016/j.neuron.2015.09.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
Medaglia JD, Lynall ME, Bassett DS, 2015. Cognitive network neuroscience. J. Cogn. Neurosci. 27, 1471–1491. doi: 10.1162/jocn_a_00810. [DOI] [PMC free article] [PubMed] [Google Scholar]
Medaglia JD, Satterthwaite TD, Kelkar A, Ciric R, Moore TM, Ruparel K, Gur RC, Gur RE, Bassett DS, 2018. Brain state expression and transitions are related to complex executive cognition in normative neurodevelopment. Neuroimage 166, 293–306. doi: 10.1016/j.neuroimage.2017.10.048. [DOI] [PMC free article] [PubMed] [Google Scholar]
Mohanty A, Gitelman DR, Small DM, Mesulam MM, 2008. The spatial attention network interacts with limbic and monoaminergic systems to modulate motivation-induced attention shifts. Cereb. Cortex 18, 2604–2613. doi: 10.1093/cercor/bhn021. [DOI] [PMC free article] [PubMed] [Google Scholar]
Monti MM, 2011. Statistical analysis of fMRI time-series: a critical review of the GLM approach. Front. Hum. Neurosci. 5, 28. doi: 10.3389/fnhum.2011.00028. [DOI] [PMC free article] [PubMed] [Google Scholar]
Mucha PJ, Richardson T, Macon K, Porter MA, Onnela JP, 2010. Community structure in time-dependent, multiscale, and multiplex networks. Science (80-.) 328, 876–878. doi: 10.1126/science.1184819. [DOI] [PubMed] [Google Scholar]
Nassar MR, Rumsey KM, Wilson RC, Parikh K, Heasly B, Gold JI, 2012. Rational regulation of learning dynamics by pupil-linked arousal systems. Nat. Neurosci. 15, 1040–1046. doi: 10.1038/nn.3130. [DOI] [PMC free article] [PubMed] [Google Scholar]
Newman MEJ, Girvan M, 2004. Finding and evaluating community structure in networks. Phys. Rev. E 69, 026113. doi: 10.1103/PhysRevE.69.026113. [DOI] [PubMed] [Google Scholar]
O’Reilly JX, Woolrich MW, Behrens TEJ, Smith SM, Johansen-Berg H, 2012. Tools of the trade: psychophysiological interactions and functional connectivity. Soc. Cogn. Affect. Neurosci. 7, 604–609. doi: 10.1093/scan/nss055. [DOI] [PMC free article] [PubMed] [Google Scholar]
Otto AR, Knox WB, Markman AB, Love BC, 2014. Physiological and behavioral signatures of reflective exploratory choice. Cogn. Affect. Behav. Neurosci. 14, 1167–1183. doi: 10.3758/s13415-014-0260-4. [DOI] [PubMed] [Google Scholar]
Pearson JM, Hayden BY, Raghavachari S, Platt ML, 2009. Neurons in posterior cingulate cortex signal exploratory decisions in a dynamic multioption choice task. Curr. Biol. 19, 1532–1537. doi: 10.1016/j.cub.2009.07.048. [DOI] [PMC free article] [PubMed] [Google Scholar]
Pedersen EJ, Miller DL, Simpson GL, Ross N, 2019. Hierarchical generalized additive models in ecology: an introduction with mgcv. PeerJ 7, e6876. doi: 10.7717/peerj.6876. [DOI] [PMC free article] [PubMed] [Google Scholar]
Peirce JW, 2009. Generating stimuli for neuroscience using PsychoPy. Front. Neuroinform. 2, 10. doi: 10.3389/neuro.11.010.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
Phillips PCB, 1986. Understanding spurious regressions in econometrics. J. Econom. 33, 311–340. doi: 10.1016/0304-4076(86)90001-1. [DOI] [Google Scholar]
Pinheiro J, Bates D, DebRoy S, Sarkar D, R Core Team, 2019. {nlme}: Linear and Nonlinear Mixed Effects Models.
Preuschoff K, 2011. Pupil dilation signals surprise: evidence for noradrenaline’s role in decision making. Front. Neurosci. 5, 1–12. doi: 10.3389/fnins.2011.00115. [DOI] [PMC free article] [PubMed] [Google Scholar]
Pyper BJ, Peterman RM, 1998. Comparison of methods to account for autocorrelation in correlation analyses of fish data. Can. J. Fish. Aquat. Sci. 55, 2127–2140. doi: 10.1139/cjfas-55-9-2127. [DOI] [Google Scholar]
Qiyuan J, Richer F, Wagoner BL, Beatty J, 1985. The pupil and stimulus probability. Psychophysiology 22, 530–534. doi: 10.1111/j.1469-8986.1985.tb01645.x. [DOI] [PubMed] [Google Scholar]
R Core Team, 2019. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. [Google Scholar]
Raichle ME, 2015. The brain’s default mode network. Annu. Rev. Neurosci. 38, 433–447. doi: 10.1146/annurev-neuro-071013-014030. [DOI] [PubMed] [Google Scholar]
Reimer J, McGinley MJ, Liu Y, Rodenkirch C, Wang Q, McCormick DA, Tolias AS, 2016. Pupil fluctuations track rapid changes in adrenergic and cholinergic activity in cortex. Nat. Commun. 7, 13289. doi: 10.1038/ncomms13289. [DOI] [PMC free article] [PubMed] [Google Scholar]
Robbins TW, Arnsten AFT, 2009. The neuropsychopharmacology of fronto-executive function: monoaminergic modulation. Annu. Rev. Neurosci. 32, 267–287. doi: 10.1146/annurev.neuro.051508.135535. [DOI] [PMC free article] [PubMed] [Google Scholar]
Roffman JL, Tanner AS, Eryilmaz H, Rodriguez-Thompson A, Silverstein NJ, Ho NF, Nitenson AZ, Chonde DB, Greve DN, Abi-Dargham A, Buckner RL, Manoach DS, Rosen BR, Hooker JM, Catana C, 2016. Dopamine D₁ signaling organizes network dynamics underlying working memory. Sci. Adv. 2, e1501672. doi: 10.1126/sciadv.1501672. [DOI] [PMC free article] [PubMed] [Google Scholar]
Rubinov M, Sporns O, 2010. Complex network measures of brain connectivity: uses and interpretations. Neuroimage 52, 1059–1069. doi: 10.1016/j.neuroimage.2009.10.003. [DOI] [PubMed] [Google Scholar]
Sara SJ, Bouret S, 2012. Orienting and reorienting: the locus coeruleus mediates cognition through arousal. Neuron 76, 130–141. doi: 10.1016/j.neuron.2012.09.011. [DOI] [PubMed] [Google Scholar]
Schaefer A, Kong R, Gordon EM, Laumann TO, Zuo X-N, Holmes AJ, Eickhoff SB, Yeo BTT, 2018. Local-global parcellation of the human cerebral cortex from intrinsic functional connectivity MRI. Cereb. Cortex 28, 3095–3114. doi: 10.1093/cercor/bhx179. [DOI] [PMC free article] [PubMed] [Google Scholar]
Shafiei G, Zeighami Y, Clark CA, Coull JT, Nagano-Saito A, Leyton M, Dagher A, Mišić B, 2019. Dopamine signaling modulates the stability and integration of intrinsic brain networks. Cereb. Cortex 29, 397–409. doi: 10.1093/cercor/bhy264. [DOI] [PMC free article] [PubMed] [Google Scholar]
Shine JM, Aburn MJ, Breakspear M, Poldrack RA, 2018a. The modulation of neural gain facilitates a transition between functional segregation and integration in the brain. Elife 7, e31130. doi: 10.7554/eLife.31130. [DOI] [PMC free article] [PubMed] [Google Scholar]
Shine JM, Bissett PG, Bell PT, Koyejo O, Balsters JH, Gorgolewski KJ, Moodie CA, Poldrack RA, 2016. The dynamics of functional brain networks: integrated network states during cognitive task performance. Neuron 92, 544–554. doi: 10.1016/j.neuron.2016.09.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
Shine JM, van den Brink RL, Hernaus D, Nieuwenhuis S, Poldrack RA, 2018b. Catecholaminergic manipulation alters dynamic network topology across cognitive states. Netw. Neurosci. 2, 381–396. doi: 10.1162/netn_a_00042. [DOI] [PMC free article] [PubMed] [Google Scholar]
Shore J, Bernstein E, Lazer D, 2015. Facts and figuring: an experimental investigation of network structure and performance in information and solution spaces. Organ. Sci. 26, 1432–1446. doi: 10.1287/orsc.2015.0980. [DOI] [Google Scholar]
Smallwood J, Brown K, Baird B, Schooler JW, 2012. Cooperation between the default mode network and the frontal–parietal network in the production of an internal train of thought. Brain Res 1428, 60–70. doi: 10.1016/j.brainres.2011.03.072. [DOI] [PubMed] [Google Scholar]
Sun FT, Miller LM, D’Esposito M, 2004. Measuring interregional functional connectivity using coherence and partial coherence analyses of fMRI data. Neuroimage 21, 647–658. doi: 10.1016/j.neuroimage.2003.09.056. [DOI] [PubMed] [Google Scholar]
Torrence C, Compo GP, 1998. A practical guide to wavelet analysis. Bull. Am. Meteorol. Soc. 79, 61–78 . [DOI] [Google Scholar]
Totah NKB, Logothetis NK, Eschenko O, 2019. Noradrenergic ensemble-based modulation of cognition over multiple timescales. Brain Res. 1709, 50–66. doi: 10.1016/j.brainres.2018.12.031. [DOI] [PubMed] [Google Scholar]
Turchi J, Chang C, Ye FQ, Russ BE, Yu DK, Cortes CR, Monosov IE, Duyn JH, Leopold DA, 2018. The basal forebrain regulates global resting-state fMRI fluctuations. Neuron 97, 940–952. doi: 10.1016/j.neuron.2018.01.032, e4. [DOI] [PMC free article] [PubMed] [Google Scholar]
Unsworth N, Robison MK, 2016. Pupillary correlates of lapses of sustained attention. Cogn. Affect. Behav. Neurosci. 16, 601–615. doi: 10.3758/s13415-016-0417-4. [DOI] [PubMed] [Google Scholar]
Urai AE, Braun A, Donner TH, 2017. Pupil-linked arousal is driven by decision uncertainty and alters serial choice bias. Nat. Commun. 8, 14637. doi: 10.1038/ncomms14637. [DOI] [PMC free article] [PubMed] [Google Scholar]
Vaiana M, Goldberg EM, Muldoon SF, 2019. Optimizing state change detection in functional temporal networks through dynamic community detection. J. Complex Netw 7, 529–553. doi: 10.1093/comnet/cny030. [DOI] [Google Scholar]
van den Brink RL, Murphy PR, Nieuwenhuis S, 2016a. Pupil diameter tracks lapses of attention. PLoS One 11, e0165274. doi: 10.1371/journal.pone.0165274. [DOI] [PMC free article] [PubMed] [Google Scholar]
van den Brink RL, Nieuwenhuis S, Donner TH, 2018. Amplification and suppression of distinct brainwide activity patterns by catecholamines. J. Neurosci. 38, 7476–7491. doi: 10.1523/JNEUROSCI.0514-18.2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
van den Brink RL, Pfeffer T, Donner TH, 2019. Brainstem modulation of large-scale intrinsic cortical activity correlations. Front. Hum. Neurosci. 13, 340. doi: 10.3389/fnhum.2019.00340. [DOI] [PMC free article] [PubMed] [Google Scholar]
van den Brink RL, Pfeffer T, Warren CM, Murphy PR, Tona KD, van der Wee NJA, Giltay E, Van Noorden MS, Rombouts SARB, Donner TH, Nieuwenhuis S, 2016b. Catecholaminergic neuromodulation shapes intrinsic MRI functional connectivity in the human brain. J. Neurosci. 36, 7865–7876. doi: 10.1523/JNEUROSCI.0744-16.2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
van Rij J, Hendriks P, van Rijn H, Baayen RH, Wood SN, 2019. Analyzing the time course of pupillometric data. Trends Hear. 23, 233121651983248. doi: 10.1177/2331216519832483. [DOI] [PMC free article] [PubMed] [Google Scholar]
Van Slooten JC, Jahfari S, Knapen T, Theeuwes J, 2018. How pupil responses track value-based decision-making during and after reinforcement learning. PLoS Comput. Biol. 14, e1006632. doi: 10.1371/journal.pcbi.1006632. [DOI] [PMC free article] [PubMed] [Google Scholar]
Varazzani C, San-Galli A, Gilardeau S, Bouret S, 2015. Noradrenaline and dopamine neurons in the reward/effort trade-off: a direct electrophysiological comparison in behaving monkeys. J. Neurosci. 35, 7866–7877. doi: 10.1523/JNEUROSCI.0454-15.2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
Vatansever D, Menon DK, Manktelow AE, Sahakian BJ, Stamatakis EA, 2015. Default mode dynamics for global functional integration. J. Neurosci. 35, 15254–15262. doi: 10.1523/JNEUROSCI.2135-15.2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
Verstynen TD, Deshpande V, 2011. Using pulse oximetry to account for high and low frequency physiological artifacts in the BOLD signal. Neuroimage 55, 1633–1644. doi: 10.1016/j.neuroimage.2010.11.090. [DOI] [PubMed] [Google Scholar]
Wahn B, Ferris DP, Hairston WD, König P, 2016. Pupil sizes scale with attentional load and task experience in a multiple object tracking task. PLoS One 11, e0168087. doi: 10.1371/journal.pone.0168087. [DOI] [PMC free article] [PubMed] [Google Scholar]
Warren CM, Eldar E, van den Brink RL, Tona KD, van der Wee NJ, Giltay EJ, Van Noorden MS, Bosch JA, Wilson RC, Cohen JD, Nieuwenhuis S, 2016. Catecholamine-mediated increases in gain enhance the precision of cortical representations. J. Neurosci. 36, 5699–5708. doi: 10.1523/JNEUROSCI.3475-15.2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
Warren CM, Wilson RC, Van Der Wee NJ, Giltay EJ, Van Noorden MS, Cohen JD, Nieuwenhuis S, 2017. The effect of atomoxetine on random and directed exploration in humans. PLoS One 12, e0176034. doi: 10.1371/journal.pone.0176034. [DOI] [PMC free article] [PubMed] [Google Scholar]
Weir WH, Emmons S, Gibson R, Taylor D, Mucha PJ, 2017. Post-processing partitions to identify domains of modularity optimization. Algorithms 10, 93. doi: 10.3390/a10030093. [DOI] [PMC free article] [PubMed] [Google Scholar]
Wierda SM, van Rijn H, Taatgen NA, Martens S, 2012. Pupil dilation deconvolution reveals the dynamics of attention at high temporal resolution. Proc. Natl. Acad. Sci. 109, 8456–8460. doi: 10.1073/pnas.1201858109. [DOI] [PMC free article] [PubMed] [Google Scholar]
Wilson RC, Geana A, White JM, Ludvig EA, Cohen JD, 2014. Humans use directed and random exploration to solve the explore–exploit dilemma. J. Exp. Psychol. Gen. 143, 2074–2081. doi: 10.1037/a0038199. [DOI] [PMC free article] [PubMed] [Google Scholar]
Wood SN, 2017. Generalized Additive Models: An Introduction with R, second edition Chapman and Hall/CRC; doi: 10.1201/9781315370279. [DOI] [Google Scholar]
Wood SN, 2013. On p-values for smooth components of an extended generalized additive model. Biometrika 100, 221–228. doi: 10.1093/biomet/ass048. [DOI] [Google Scholar]
Xia M, Wang J, He Y, 2013. BrainNet viewer: a network visualization tool for human brain connectomics. PLoS One 8, e68910. doi: 10.1371/journal.pone.0068910. [DOI] [PMC free article] [PubMed] [Google Scholar]
Yael D, Vecht JJ, Bar-Gad I, 2018. Filter-based phase shifts distort neuronal timing information. eNeuro 5. doi: 10.1523/ENEURO.0261-17.2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
Yeo BTT, Krienen FM, Sepulcre J, Sabuncu MR, Lashkari D, Hollinshead M, Roffman JL, Smoller JW, Zöllei L, Polimeni JR, Fischl B, Liu H, Buckner RL, 2011. The organization of the human cerebral cortex estimated by intrinsic functional connectivity. J. Neurophysiol. 106, 1125–1165. doi: 10.1152/jn.00338.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
Yu AJ, Dayan P, 2005. Uncertainty, neuromodulation, and attention. Neuron 46, 681–692. doi: 10.1016/j.neuron.2005.04.026. [DOI] [PubMed] [Google Scholar]
Yue Q, Martin RC, Fischer-Baum S, Ramos-Nuñez AI, Ye F, Deem MW, 2017. Brain modularity mediates the relation between task complexity and performance. J. Cogn. Neurosci. 29, 1532–1546. doi: 10.1162/jocn_a_01142. [DOI] [PubMed] [Google Scholar]
Yule GU, 1926. Why do we sometimes get nonsense-correlations between time-series?–a study in sampling and the nature of time-series. J. R. Stat. Soc. 89, 1. doi: 10.2307/2341482. [DOI] [Google Scholar]
Záborszky L, Gombkoto P, Varsanyi P, Gielow MR, Poe G, Role LW, Ananth M, Rajebhosale P, Talmage DA, Hasselmo ME, Dannenberg H, Minces VH, Chiba AA, 2018. Specific basal forebrain–cortical cholinergic circuits coordinate cognitive operations. J. Neurosci. 38, 9446–9458. doi: 10.1523/JNEUROSCI.1676-18.2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
Zénon A, 2019. Eye pupil signals information gain. Proc. R. Soc. B Biol. Sci. doi: 10.1098/rspb.2019.1593. [DOI] [PMC free article] [PubMed] [Google Scholar]
Zerbi V, Floriou-Servou A, Markicevic M, Vermeiren Y, Sturman O, Privitera M, von Ziegler L, Ferrari KD, Weber B, De Deyn PP, Wenderoth N, Bohacek J, 2019. Rapid reconfiguration of the functional connectome after chemogenetic locus coeruleus activation. Neuron 103, 702–718. doi: 10.1016/j.neuron.2019.05.034, e5. [DOI] [PubMed] [Google Scholar]
Zhang Z, Telesford QK, Giusti C, Lim KO, Bassett DS, 2016. Choosing wavelet methods, filters, and lengths for functional brain network construction. PLoS One 11, e0157243. doi: 10.1371/journal.pone.0157243. [DOI] [PMC free article] [PubMed] [Google Scholar]
Zhao M, Zhou C, Chen Y, Hu B, Wang B-H, 2010. Complexity versus modularity and heterogeneity in oscillatory networks: combining segregation and integration in neural systems. Phys. Rev. E 82, 046225. doi: 10.1103/PhysRevE.82.046225. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

NIHMS1741054-supplement-1.docx^{(19MB, docx)}

Data Availability Statement

The MRI preprocessing pipeline code is available at: https://github.com/ntardiff/MRklar. Data and analysis code are available from the corresponding author upon reasonable request.

[R1] Afyouni S, Smith SM, Nichols TE, 2019. Effective degrees of freedom of the Pearson’s correlation coefficient under autocorrelation. Neuroimage 199, 609–625. doi: 10.1016/j.neuroimage.2019.05.011. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R2] Alamia A, VanRullen R, Pasqualotto E, Mouraux A, Zenon A, 2019. Pupil-linked arousal responds to unconscious surprisal. J. Neurosci. 39, 5369–5376. doi: 10.1523/JNEUROSCI.3010-18.2019. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R3] Alnæs D, Sneve MH, Espeseth T, Endestad T, van de Pavert SHP, Laeng B, 2014. Pupil size signals mental effort deployed during multiple object tracking and predicts brain activity in the dorsal attention network and the locus coeruleus. J. Vis. 14. doi: 10.1167/14.4.1, 1–1. [DOI] [PubMed] [Google Scholar]

[R4] Arbabshirani MR, Damaraju E, Phlypo R, Plis S, Allen E, Ma S, Mathalon D, Preda A, Vaidya JG, Adali T, Calhoun VD, 2014. Impact of autocorrelation on functional connectivity. Neuroimage 102, 294–308. doi: 10.1016/j.neuroimage.2014.07.045. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R5] Arnsten AFT, Paspalas CD, Gamo NJ, Yang Y, Wang M, 2010. Dynamic network connectivity: a new form of neuroplasticity. Trends Cogn. Sci. 14, 365–375. doi: 10.1016/j.tics.2010.05.003. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R6] Arsenault JT, Caspari N, Vandenberghe R, Vanduffel W, 2018. Attention shifts recruit the monkey default ode network. J. Neurosci. 38, 1202–1217. doi: 10.1523/JNEUROSCI.1111-17.2017. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R7] Aston-Jones G, Cohen JD, 2005. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 28, 403–450. doi: 10.1146/annurev.neuro.28.061604.135709. [DOI] [PubMed] [Google Scholar]

[R8] Baayen H, Vasishth S, Kliegl R, Bates D, 2017. The cave of shadows: addressing the human factor with generalized additive mixed models. J. Mem. Lang. 94, 206–234. doi: 10.1016/j.jml.2016.11.006. [DOI] [Google Scholar]

[R9] Bari A, Aston-Jones G, 2013. Atomoxetine modulates spontaneous and sensory-evoked discharge of locus coeruleus noradrenergic neurons. Neuropharmacology 64, 53–64. doi: 10.1016/j.neuropharm.2012.07.020. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R10] Barr DJ, Levy R, Scheepers C, Tily HJ, 2013. Random effects structure for confirmatory hypothesis testing: keep it maximal. J. Mem. Lang. 68, 255–278. doi: 10.1016/j.jml.2012.11.001. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R11] Bassett DS, Bullmore E, 2006. Small-world brain networks. Neuroscientist 12, 512–523. doi: 10.1177/1073858406293182. [DOI] [PubMed] [Google Scholar]

[R12] Bassett DS, Porter MA, Wymbs NF, Grafton ST, Carlson JM, Mucha PJ, 2013. Robust detection of dynamic community structure in networks. Chaos An Interdiscip. J. Nonlinear Sci. 23, 013142. doi: 10.1063/1.4790830. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R13] Bassett DS, Wymbs NF, Porter MA, Mucha PJ, Carlson JM, Grafton ST, 2011. Dynamic reconfiguration of human brain networks during learning. Proc. Natl. Acad. Sci. U. S. A. 108, 7641–7646. doi: 10.1073/pnas.1018985108. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R14] Bassett DS, Yang M, Wymbs NF, Grafton ST, 2015. Learning-induced autonomy of sensorimotor systems. Nat. Neurosci. 18, 744–751. doi: 10.1038/nn.3993. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R15] Bates D, Kliegl R, Vasishth S, Baayen H, 2015a. Parsimonious Mixed Models.

[R16] Bates D, Mächler M, Bolker B, Walker S, 2015b. Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67, 1–48. doi: 10.18637/jss.v067.i01. [DOI] [Google Scholar]

[R17] Bazzi M, Porter MA, Williams S, McDonald M, Fenn DJ, Howison SD, 2016. Community detection in temporal multilayer networks, with an application to correlation networks. Multiscale Model. Simul. 14, 1–41. doi: 10.1137/15M1009615. [DOI] [Google Scholar]

[R18] Beaty RE, Benedek M, Silvia PJ, Schacter DL, 2016. Creative cognition and brain network dynamics. Trends Cogn. Sci. 20, 87–95. doi: 10.1016/j.tics.2015.10.004. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R19] Beeler JA, Daw N, Frazier CRM, Zhuang X, 2010. Tonic dopamine modulates exploitation of reward learning. Front. Behav. Neurosci. 4, 170. doi: 10.3389/fnbeh.2010.00170. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R20] Behzadi Y, Restom K, Liau J, Liu TT, 2007. A component based noise correction method (CompCor) for BOLD and perfusion based fMRI. Neuroimage 37, 90–101. doi: 10.1016/j.neuroimage.2007.04.042. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R21] Bernatskiy A, Bongard JC, 2015. Exploiting the relationship between structural modularity and sparsity for faster network evolution. In: GECCO 2015 - Companion Publication of the 2015 Genetic and Evolutionary Computation Conference. Association for Computing Machinery, Inc, pp. 1173–1176. doi: 10.1145/2739482.2768472. [DOI] [Google Scholar]

[R22] Bernstein E, Shore J, Lazer D, 2018. How intermittent breaks in interaction improve collective intelligence. Proc. Natl. Acad. Sci. USA 115, 8734–8739. doi: 10.1073/pnas.1802407115. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R23] Berridge CW, Waterhouse BD, 2003. The locus coeruleus–noradrenergic system: modulation of behavioral state and state-dependent cognitive processes. Brain Res. Rev. 42, 33–84. doi: 10.1016/S0165-0173(03)00143-7. [DOI] [PubMed] [Google Scholar]

[R24] Bertolero MA, Yeo BTT, Bassett DS, D’Esposito M, 2018. A mechanistic model of connector hubs, modularity and cognition. Nat. Hum. Behav. 2, 765–777. doi: 10.1038/s41562-018-0420-6. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R25] Birn RM, Converse AK, Rajala AZ, Alexander AL, Block WF, McMillan AB, Christian BT, Filla CN, Murali D, Hurley SA, Jenison RL, Populin LC, 2019. Changes in endogenous dopamine induced by methylphenidate predict functional connectivity in nonhuman primates. J. Neurosci. 39, 1436–1444. doi: 10.1523/JNEUROSCI.2513-18.2018. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R26] Blanco NJ, Love BC, Cooper JA, McGeary JE, Knopik VS, Maddox WT, 2015. A frontal dopamine system for reflective exploratory behavior. Neurobiol. Learn. Mem. 123, 84–91. doi: 10.1016/j.nlm.2015.05.004. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R27] Blondel VD, Guillaume JL, Lambiotte R, Lefebvre E, 2008. Fast unfolding of communities in large networks. J. Stat. Mech. Theory Exp. 2008, P10008. doi: 10.1088/1742-5468/2008/10/P10008. [DOI] [Google Scholar]

[R28] Bouret S, Sara SJ, 2005. Network reset: A simplified overarching theory of locus coeruleus noradrenaline function. Trends Neurosci. 28, 574–582. doi: 10.1016/j.tins.2005.09.002. [DOI] [PubMed] [Google Scholar]

[R29] Braun U, Schäfer A, Walter H, Erk S, Romanczuk-Seiferth N, Haddad L, Schweiger JI, Grimm O, Heinz A, Tost H, Meyer-Lindenberg A, Bassett DS, 2015. Dynamic reconfiguration of frontal brain networks during executive cognition in humans. Proc. Natl. Acad. Sci. USA 112, 11678–11683. doi: 10.1073/pnas.1422487112. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R30] Calhoun VD, Miller R, Pearlson G, Adali T, 2014. The chronnectome: time-varying connectivity networks as the next frontier in fMRI data discovery. Neuron 84, 262–274. doi: 10.1016/j.neuron.2014.10.015. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R31] Cash-Padgett T, Azab H, Yoo SBM, Hayden BY, 2018. Opposing pupil responses to offered and anticipated reward values. Anim. Cogn. 21, 671–684. doi: 10.1007/s10071-018-1202-2. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R32] Chan MY, Park DC, Savalia NK, Petersen SE, Wig GS, 2014. Decreased segregation of brain systems across the healthy adult lifespan. Proc. Natl. Acad. Sci. USA 111, E4997–E5006. doi: 10.1073/pnas.1415122111. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R33] Clewett D, Murty VP, 2019. Echoes of emotions past: how neuromodulators determine what we recollect. eNeuro 6, ENEURO.0108–18.2019. 10.1523/ENEURO.0108-18.2019 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R34] Clune J, Mouret JB, Lipson H, 2013. The evolutionary origins of modularity. Proc. R. Soc. B Biol. Sci. 280, 20122863. doi: 10.1098/rspb.2012.2863. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R35] Cohen JR, D’Esposito M, 2016. The segregation and integration of distinct brain networks and their relationship to cognition. J. Neurosci. 36, 12083–12094. doi: 10.1523/JNEUROSCI.2965-15.2016. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R36] Cole MW, Ito T, Schultz D, Mill R, Chen R, Cocuzza C, 2019. Task activations produce spurious but systematic inflation of task functional connectivity estimates. Neuroimage 189, 1–18. doi: 10.1016/j.neuroimage.2018.12.054. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R37] Corbetta M, Patel G, Shulman GL, 2008. The reorienting system of the human brain: from environment to theory of mind. Neuron 58, 306–324. doi: 10.1016/j.neuron.2008.04.017. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R38] Craig MM, Manktelow AE, Sahakian BJ, Menon DK, Stamatakis EA, 2018. Spectral diversity in default mode network connectivity reflects behavioral state. J. Cogn. Neurosci. 30, 526–539. doi: 10.1162/jocn_a_01213. [DOI] [PubMed] [Google Scholar]

[R39] Crittenden BM, Mitchell DJ, Duncan J, 2015. Recruitment of the default mode network during a demanding act of executive control. Elife 2015, e06481. doi: 10.7554/eLife.06481. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R40] Cruzat J, Deco G, Tauste-Campo A, Principe A, Costa A, Kringelbach ML, Rocamora R, 2018. The dynamics of human cognition: increasing global integration coupled with decreasing segregation found using iEEG. Neuroimage 172, 492–505. doi: 10.1016/j.neuroimage.2018.01.064. [DOI] [PubMed] [Google Scholar]

[R41] Dalmaijer ES, Mathôt S, Van der Stigchel S, 2014. PyGaze: an open-source, cross-platform toolbox for minimal-effort programming of eyetracking experiments. Behav. Res. Methods 46, 913–921. doi: 10.3758/s13428-013-0422-2. [DOI] [PubMed] [Google Scholar]

[R42] Daw ND, O’Doherty JP, Dayan P, Seymour B, Dolan RJ, 2006. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879. doi: 10.1038/nature04766. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R43] de Cheveigné A, Nelken I, 2019. Filters: when, why, and how (not) to use them. Neuron 102, 280–293. doi: 10.1016/j.neuron.2019.02.039. [DOI] [PubMed] [Google Scholar]

[R44] de Gee JW, Colizoli O, Kloosterman NA, Knapen T, Nieuwenhuis S, Donner TH, Noorden M.van, Bosch J, Wilson R, Cohen J, Nieuwenhuis S, Saunders J, Vickers J, Zhang Y, Stefano N.De, Brady J, Matthews P, 2017. Dynamic modulation of decision biases by brainstem arousal systems. Elife 6, 1127–1141. doi: 10.7554/eLife.23232. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R45] Dean RT, Dunsmuir WTM, 2016. Dangers and uses of cross-correlation in analyzing time series in perception, performance, movement, and neuroscience: the importance of constructing transfer function autoregressive models. Behav. Res. Methods 48, 783–802. doi: 10.3758/s13428-015-0611-2. [DOI] [PubMed] [Google Scholar]

[R46] Deco G, Tononi G, Boly M, Kringelbach ML, 2015. Rethinking segregation and integration: contributions of whole-brain modelling. Nat. Rev. Neurosci. 16, 430–439. doi: 10.1038/nrn3963. [DOI] [PubMed] [Google Scholar]

[R47] Dixon ML, Andrews-Hanna JR, Spreng RN, Irving ZC, Mills C, Girn M, Christoff K, 2017. Interactions between the default network and dorsal attention network vary across default subsystems, time, and cognitive states. Neuroimage 147, 632–649. doi: 10.1016/j.neuroimage.2016.12.073. [DOI] [PubMed] [Google Scholar]

[R48] Dixon ML, De La Vega A, Mills C, Andrews-Hanna J, Spreng RN, Cole MW, Christoff K, 2018. Heterogeneity within the frontoparietal control network and its relationship to the default and dorsal attention networks. Proc. Natl. Acad. Sci. USA 115, E1598–E1607. doi: 10.1073/pnas.1715766115. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R49] Ebitz RB, Albarran E, Moore T, 2018. Exploration disrupts choice-predictive signals and alters dynamics in prefrontal cortex. Neuron 97, 450–461. doi: 10.1016/j.neuron.2017.12.007, e9. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R50] Ebitz RB, Platt ML, 2015. Neuronal activity in primate dorsal anterior cingulate cortex signals task conflict and predicts adjustments in pupil-linked arousal. Neuron 85, 628–640. doi: 10.1016/j.neuron.2014.12.053. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R51] Ekman M, Derrfuss J, Tittgemeyer M, Fiebach CJ, 2012. Predicting errors from reconfiguration patterns in human brain networks. Proc. Natl. Acad. Sci. USA 109, 16714–16719. doi: 10.1073/pnas.1207523109. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R52] Elber-Dorozko L, Loewenstein Y, 2018. Striatal action-value neurons reconsidered. Elife 7. doi: 10.7554/eLife.34248. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R53] Eldar E, Cohen JD, Niv Y, 2013. The effects of neural gain on attention and learning. Nat. Neurosci. 16, 1146–1153. doi: 10.1038/nn.3428. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R54] Eldar E, Niv Y, Cohen JD, 2016. Do you see the forest or the tree? Neural gain and breadth versus focus in perceptual processing. Psychol. Sci. 27, 1632–1643. doi: 10.1177/0956797616665578. [DOI] [PubMed] [Google Scholar]

[R55] Fedorenko E, Thompson-Schill SL, 2014. Reworking the language network. Trends Cogn. Sci. 18, 120–126. doi: 10.1016/j.tics.2013.12.006. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R56] Fiorillo CD, Tobler PN, Schultz W, 2003. Discrete coding of reward probability and uncertainty by dopamine neurons. Science (80-.) 299, 1898–1902. doi: 10.1126/science.1077349. [DOI] [PubMed] [Google Scholar]

[R57] Fischl B, 2012. FreeSurfer. Neuroimage 62, 774–781. doi: 10.1016/j.neuroimage.2012.01.021. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R58] Fischl B, Salat DH, Busa E, Albert M, Dieterich M, Haselgrove C, Van Der Kouwe A, Killiany R, Kennedy D, Klaveness S, Montillo A, Makris N, Rosen B, Dale AM, 2002. Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron 33, 341–355. doi: 10.1016/S0896-6273(02)00569-X. [DOI] [PubMed] [Google Scholar]

[R59] Friedman D, Hakerem G, Sutton S, Fleiss JL, 1973. Effect of stimulus uncertainty on the pupillary dilation response and the vertex evoked potential. Electroencephalogr. Clin. Neurophysiol. 34, 475–484. doi: 10.1016/0013-4694(73)90065-5. [DOI] [PubMed] [Google Scholar]

[R60] Friston KJ, Williams S, Howard R, Frackowiak RSJ, Turner R, 1996. Movement-related effects in fMRI time-series. Magn. Reson. Med. 35, 346–355. doi: 10.1002/mrm.1910350312. [DOI] [PubMed] [Google Scholar]

[R61] Gallagher M, Holland PC, 1994. The amygdala complex: multiple roles in associative learning and attention. Proc. Natl. Acad. Sci. USA 91, 11771–11776. doi: 10.1073/pnas.91.25.11771. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R62] Gallos LK, Makse HA, Sigman M, 2012. A small world of weak ties provides optimal global integration of self-similar modules in functional brain networks. Proc. Natl. Acad. Sci. USA 109, 2825–2830. doi: 10.1073/pnas.1106612109. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R63] Gerraty RT, Davidow JY, Foerde K, Galvan A, Bassett DS, Shohamy D, 2018. Dynamic flexibility in striatal-cortical circuits supports reinforcement learning. J. Neurosci. 38, 2442–2453. doi: 10.1523/JNEUROSCI.2084-17.2018. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R64] Gershman SJ, 2018. Deconstructing the human algorithms for exploration. Cognition 173, 34–42. doi: 10.1016/j.cognition.2017.12.014. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R65] Gießing C, Thiel CM, Alexander-Bloch AF, Patel AX, Bullmore ET, 2013. Human brain functional network changes associated with enhanced and impaired attentional task performance. J. Neurosci. 33, 5903–5914. doi: 10.1523/JNEUROSCI.4854-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R66] Gilzenrat MS, Nieuwenhuis S, Jepma M, Cohen JD, 2010. Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function. Cogn. Affect. Behav. Neurosci. 10, 252–269. doi: 10.3758/CABN.10.2.252. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R67] Good BH, de Montjoye Y-A, Clauset A, 2010. Performance of modularity maximization in practical contexts. Phys. Rev. E 81, 046106. doi: 10.1103/Phys-RevE.81.046106. [DOI] [PubMed] [Google Scholar]

[R68] Granger CWJ, Newbold P, 1974. Spurious regressions in econometrics. J. Econom. 2, 111–120. doi: 10.1016/0304-4076(74)90034-7. [DOI] [Google Scholar]

[R69] Greve DN, Fischl B, 2009. Accurate and robust brain image alignment using boundary-based registration. Neuroimage 48, 63–72. doi: 10.1016/j.neuroimage.2009.06.060. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R70] Grinsted A, Moore JC, Jevrejeva S, 2004. Application of the cross wavelet transform and wavelet coherence to geophysical time series. Nonlinear Process. Geophys. 11, 561–566. doi: 10.5194/npg-11-561-2004. [DOI] [Google Scholar]

[R71] Guedj C, Monfardini E, Reynaud AJ, Farnè A, Meunier M, Hadj-Bouziane F, 2017. Boosting norepinephrine transmission triggers flexible reconfiguration of brain networks at rest. Cereb. Cortex 27, 4691–4700. doi: 10.1093/cercor/bhw262. [DOI] [PubMed] [Google Scholar]

[R72] Halassa MM, Kastner S, 2017. Thalamic functions in distributed cognitive control. Nat. Neurosci. 20, 1669–1679. doi: 10.1038/s41593-017-0020-1. [DOI] [PubMed] [Google Scholar]

[R73] Hasselmo ME, Linster C, Patil M, Ma D, Cekic M, 1997. Noradrenergic suppression of synaptic transmission may influence cortical signal-to-noise ratio. J. Neurophysiol. 77, 3326–3339. doi: 10.1152/jn.1997.77.6.3326. [DOI] [PubMed] [Google Scholar]

[R74] Hayes TR, Petrov AA, 2016. Pupil diameter tracks the exploration–exploitation trade-off during analogical reasoning and explains individual differences in fluid intelligence. J. Cogn. Neurosci. 28, 308–318. doi: 10.1162/jocn_a_00895. [DOI] [PubMed] [Google Scholar]

[R75] Heilbronner SR, Platt ML, 2013. Causal evidence of performance monitoring by neurons in posterior cingulate cortex during learning. Neuron 80, 1384–1391. doi: 10.1016/j.neuron.2013.09.028. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R76] Hess EH, Polt JM, 1964. Pupil size in relation to mental activity during simple problem-solving. Science (80-.) 143, 1190–1192. doi: 10.1126/science.143.3611.1190. [DOI] [PubMed] [Google Scholar]

[R77] Highly Accurate Inverse Consistent Registration: A Robust Approach,Reuter M, Rosas HD, Fischl B NeuroImage 53(4), pp. 1181–1196, 2010. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R78] Hoeks B, Levelt WJM, 1993. Pupillary dilation as a measure of attention: a quantitative system analysis. Behav. Res. Methods, Instrum., Comput. 25, 16–26. doi: 10.3758/BF03204445. [DOI] [Google Scholar]

[R79] Holm S, 1979. A simple sequentially rejective multiple test procedure. Scand. J. Stat. 6, 65–70. doi: 10.2307/4615733. [DOI] [Google Scholar]

[R80] Honari H, Choe AS, Pekar JJ, Lindquist MA, 2019. Investigating the impact of autocorrelation on time-varying connectivity. Neuroimage 197, 37–48. doi: 10.1016/j.neuroimage.2019.04.042. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R81] Hurley LM, Devilbiss DM, Waterhouse BD, 2004. A matter of focus: monoaminergic modulation of stimulus coding in mammalian sensory networks. Curr. Opin. Neurobiol. 14, 488–495. doi: 10.1016/j.conb.2004.06.007. [DOI] [PubMed] [Google Scholar]

[R82] Jenkinson M, 2004. Improving the registration of B0-disorted EPI images using calculated cost function weights. Tenth International Conference on Functional Mapping of the Human Brain. [Google Scholar]

[R83] Jenkinson M, 2003. Fast, automated, N-dimensional phase-unwrapping algorithm. Magn. Reson. Med. 49, 193–197. doi: 10.1002/mrm.10354. [DOI] [PubMed] [Google Scholar]

[R84] Jenkinson M, Beckmann CF, Behrens TEJ, Woolrich MW, Smith SM, 2012. FSL. Neuroimage 62, 782–790. doi: 10.1016/j.neuroimage.2011.09.015. [DOI] [PubMed] [Google Scholar]

[R85] Jepma M, Nieuwenhuis S, 2011. Pupil diameter predicts changes in the exploration-exploitation trade-off: evidence for the adaptive gain theory. J. Cogn. Neurosci. 23, 1587–1596. doi: 10.1162/jocn.2010.21548. [DOI] [PubMed] [Google Scholar]

[R86] Jepma M, te Beek ET, Wagenmakers EJ, van Gerven JMA, Nieuwenhuis S, 2010. The role of the noradrenergic system in the exploration-exploitation trade-off: a psychopharmacological study. Front. Hum. Neurosci. 4, 170. doi: 10.3389/fnhum.2010.00170. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R87] Jeub LGS, Bazzi M, Jutla IS, Mucha PJ, 2011. A generalized Louvain method for community detection implemented in MATLAB.

[R88] Jo HJ, Saad ZS, Simmons WK, Milbury LA, Cox RW, 2010. Mapping sources of correlation in resting state FMRI, with artifact detection and removal. Neuroimage 52, 571–582. doi: 10.1016/j.neuroimage.2010.04.246. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R89] Johansen S, 2012. The analysis of nonstationary time series using regression, correlation and cointegration. Contemp. Econ. 6, 40–57. doi: 10.5709/ce.1897-9254.39. [DOI] [Google Scholar]

[R90] Joshi S, Gold JI, 2020. Pupil size as a window on neural substrates of cognition. Trends Cogn. Sci. 24, 466–480. doi: 10.1016/j.tics.2020.03.005. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R91] Joshi S, Li Y, Kalwani RM, Gold JI, 2016. Relationships between pupil diameter and neuronal activity in the locus coeruleus, colliculi, and cingulate cortex. Neuron 89, 221–234. doi: 10.1016/j.neuron.2015.11.028. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R92] Kaelbling LP, Littman ML, Moore AW, 1996. Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285. doi: 10.1613/jair.301. [DOI] [Google Scholar]

[R93] Kahneman D, Beatty J, 1966. Pupil diameter and load on memory. Science (80-.). 154, 1583–1585. doi: 10.1126/science.154.3756.1583. [DOI] [PubMed] [Google Scholar]

[R94] Kalwani RM, Joshi S, Gold JI, 2014. Phasic activation of individual neurons in the locus ceruleus/subceruleus complex of monkeys reflects rewarded decisions to go but not stop. J. Neurosci. 34, 13656–13669. doi: 10.1523/JNEUROSCI.2566-14.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R95] Kane GA, Vazey EM, Wilson RC, Shenhav A, Daw ND, Aston-Jones G, Cohen JD, 2017. Increased locus coeruleus tonic activity causes disengagement from a patch-foraging task. Cogn. Affect. Behav. Neurosci. 17, 1073–1083. doi: 10.3758/s13415-017-0531-y. [DOI] [PubMed] [Google Scholar]

[R96] Karahanoğlu FI, Caballero-Gaudes C, Lazeyras F, Van De Ville D, 2013. Total activation: fMRI deconvolution through spatio-temporal regularization. Neuroimage 73, 121–134. doi: 10.1016/j.neuroimage.2013.01.067. [DOI] [PubMed] [Google Scholar]

[R97] Khambhati AN, Mattar MG, Wymbs NF, Grafton ST, Bassett DS, 2018a. Beyond modularity: fine-scale mechanisms and rules for brain network reconfiguration. Neuroimage 166, 385–399. doi: 10.1016/j.neuroimage.2017.11.015. [DOI] [PubMed] [Google Scholar]

[R98] Khambhati AN, Sizemore AE, Betzel RF, Bassett DS, 2018b. Modeling and interpreting mesoscale network dynamics. Neuroimage 180, 337–349. doi: 10.1016/j.neuroimage.2017.06.029. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R99] Knox WB, Otto AR, Stone P, Love BC, 2012. The nature of belief-directed exploratory choice in human decision-making. Front. Psychol. 2, 398. doi: 10.3389/fpsyg.2011.00398. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R100] Kopell NJ, Gritton HJ, Whittington MA, Kramer MA, 2014. Beyond the connectome: the dynome. Neuron 83, 1319–1328. doi: 10.1016/j.neuron.2014.08.016. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R101] Lavín C, San Martín R, Rosales Jubal E, 2014. Pupil dilation signals uncertainty and surprise in a learning gambling task. Front. Behav. Neurosci. 7. doi: 10.3389/fnbeh.2013.00218. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R102] Lazer D, Friedman A, 2007. The network structure of exploration and exploitation. Adm. Sci. Q. 52, 667–694. doi: 10.2189/asqu.52.4.667. [DOI] [Google Scholar]

[R103] Lenth RV, 2016. Least-squares means: the R package lsmeans. J. Stat. Softw. 69, 1–33. doi: 10.18637/jss.v069.i01. [DOI] [Google Scholar]

[R104] Leonardi N, Van De Ville D, 2015. On spurious and real fluctuations of dynamic functional connectivity during rest. Neuroimage 104, 430–436. doi: 10.1016/j.neuroimage.2014.09.007. [DOI] [PubMed] [Google Scholar]

[R105] Liu X, Zhang N, Chang C, Duyn JH, 2018. Co-activation patterns in resting-state fMRI signals. Neuroimage 180, 485–494. doi: 10.1016/j.neuroimage.2018.01.041. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R106] Liu Y, Rodenkirch C, Moskowitz N, Schriver B, Wang Q, 2017. Dynamic lateralization of pupil dilation evoked by locus coeruleus activation results from sympathetic, not parasympathetic, contributions. Cell Rep. 20, 3099–3112. doi: 10.1016/j.celrep.2017.08.094. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R107] Mason W, Watts DJ, 2012. Collaborative learning in networks. Proc. Natl. Acad. Sci. 109, 764–769. doi: 10.1073/pnas.1110069108. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R108] Mason WA, Jones A, Goldstone RL, 2008. Propagation of innovations in networked groups. J. Exp. Psychol. Gen. 137, 422–433. doi: 10.1037/a0012798. [DOI] [PubMed] [Google Scholar]

[R109] Mather M, Clewett D, Sakaki M, Harley CW, 2016. Norepinephrine ignites local hotspots of neuronal excitation: how arousal amplifies selectivity in perception and memory. Behav. Brain Sci. 39, 1–100. doi:. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R110] Mattar MG, Cole MW, Thompson-Schill SL, Bassett DS, 2015. A functional cartography of cognitive systems. PLoS Comput. Biol. 11, e1004533. doi: 10.1371/journal.pcbi.1004533. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R111] McGinley MJ, Vinck M, Reimer J, Batista-Brito R, Zagha E, Cadwell CR, Tolias AS, Cardin JA, McCormick DA, 2015. Waking state: rapid variations modulate neural and behavioral responses. Neuron 87, 1143–1161. doi: 10.1016/j.neuron.2015.09.012. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R112] Medaglia JD, Lynall ME, Bassett DS, 2015. Cognitive network neuroscience. J. Cogn. Neurosci. 27, 1471–1491. doi: 10.1162/jocn_a_00810. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R113] Medaglia JD, Satterthwaite TD, Kelkar A, Ciric R, Moore TM, Ruparel K, Gur RC, Gur RE, Bassett DS, 2018. Brain state expression and transitions are related to complex executive cognition in normative neurodevelopment. Neuroimage 166, 293–306. doi: 10.1016/j.neuroimage.2017.10.048. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R114] Mohanty A, Gitelman DR, Small DM, Mesulam MM, 2008. The spatial attention network interacts with limbic and monoaminergic systems to modulate motivation-induced attention shifts. Cereb. Cortex 18, 2604–2613. doi: 10.1093/cercor/bhn021. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R115] Monti MM, 2011. Statistical analysis of fMRI time-series: a critical review of the GLM approach. Front. Hum. Neurosci. 5, 28. doi: 10.3389/fnhum.2011.00028. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R116] Mucha PJ, Richardson T, Macon K, Porter MA, Onnela JP, 2010. Community structure in time-dependent, multiscale, and multiplex networks. Science (80-.) 328, 876–878. doi: 10.1126/science.1184819. [DOI] [PubMed] [Google Scholar]

[R117] Nassar MR, Rumsey KM, Wilson RC, Parikh K, Heasly B, Gold JI, 2012. Rational regulation of learning dynamics by pupil-linked arousal systems. Nat. Neurosci. 15, 1040–1046. doi: 10.1038/nn.3130. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R118] Newman MEJ, Girvan M, 2004. Finding and evaluating community structure in networks. Phys. Rev. E 69, 026113. doi: 10.1103/PhysRevE.69.026113. [DOI] [PubMed] [Google Scholar]

[R119] O’Reilly JX, Woolrich MW, Behrens TEJ, Smith SM, Johansen-Berg H, 2012. Tools of the trade: psychophysiological interactions and functional connectivity. Soc. Cogn. Affect. Neurosci. 7, 604–609. doi: 10.1093/scan/nss055. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R120] Otto AR, Knox WB, Markman AB, Love BC, 2014. Physiological and behavioral signatures of reflective exploratory choice. Cogn. Affect. Behav. Neurosci. 14, 1167–1183. doi: 10.3758/s13415-014-0260-4. [DOI] [PubMed] [Google Scholar]

[R121] Pearson JM, Hayden BY, Raghavachari S, Platt ML, 2009. Neurons in posterior cingulate cortex signal exploratory decisions in a dynamic multioption choice task. Curr. Biol. 19, 1532–1537. doi: 10.1016/j.cub.2009.07.048. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R122] Pedersen EJ, Miller DL, Simpson GL, Ross N, 2019. Hierarchical generalized additive models in ecology: an introduction with mgcv. PeerJ 7, e6876. doi: 10.7717/peerj.6876. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R123] Peirce JW, 2009. Generating stimuli for neuroscience using PsychoPy. Front. Neuroinform. 2, 10. doi: 10.3389/neuro.11.010.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R124] Phillips PCB, 1986. Understanding spurious regressions in econometrics. J. Econom. 33, 311–340. doi: 10.1016/0304-4076(86)90001-1. [DOI] [Google Scholar]

[R125] Pinheiro J, Bates D, DebRoy S, Sarkar D, R Core Team, 2019. {nlme}: Linear and Nonlinear Mixed Effects Models.

[R126] Preuschoff K, 2011. Pupil dilation signals surprise: evidence for noradrenaline’s role in decision making. Front. Neurosci. 5, 1–12. doi: 10.3389/fnins.2011.00115. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R127] Pyper BJ, Peterman RM, 1998. Comparison of methods to account for autocorrelation in correlation analyses of fish data. Can. J. Fish. Aquat. Sci. 55, 2127–2140. doi: 10.1139/cjfas-55-9-2127. [DOI] [Google Scholar]

[R128] Qiyuan J, Richer F, Wagoner BL, Beatty J, 1985. The pupil and stimulus probability. Psychophysiology 22, 530–534. doi: 10.1111/j.1469-8986.1985.tb01645.x. [DOI] [PubMed] [Google Scholar]

[R129] R Core Team, 2019. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. [Google Scholar]

[R130] Raichle ME, 2015. The brain’s default mode network. Annu. Rev. Neurosci. 38, 433–447. doi: 10.1146/annurev-neuro-071013-014030. [DOI] [PubMed] [Google Scholar]

[R131] Reimer J, McGinley MJ, Liu Y, Rodenkirch C, Wang Q, McCormick DA, Tolias AS, 2016. Pupil fluctuations track rapid changes in adrenergic and cholinergic activity in cortex. Nat. Commun. 7, 13289. doi: 10.1038/ncomms13289. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R132] Robbins TW, Arnsten AFT, 2009. The neuropsychopharmacology of fronto-executive function: monoaminergic modulation. Annu. Rev. Neurosci. 32, 267–287. doi: 10.1146/annurev.neuro.051508.135535. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R133] Roffman JL, Tanner AS, Eryilmaz H, Rodriguez-Thompson A, Silverstein NJ, Ho NF, Nitenson AZ, Chonde DB, Greve DN, Abi-Dargham A, Buckner RL, Manoach DS, Rosen BR, Hooker JM, Catana C, 2016. Dopamine D₁ signaling organizes network dynamics underlying working memory. Sci. Adv. 2, e1501672. doi: 10.1126/sciadv.1501672. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R134] Rubinov M, Sporns O, 2010. Complex network measures of brain connectivity: uses and interpretations. Neuroimage 52, 1059–1069. doi: 10.1016/j.neuroimage.2009.10.003. [DOI] [PubMed] [Google Scholar]

[R135] Sara SJ, Bouret S, 2012. Orienting and reorienting: the locus coeruleus mediates cognition through arousal. Neuron 76, 130–141. doi: 10.1016/j.neuron.2012.09.011. [DOI] [PubMed] [Google Scholar]

[R136] Schaefer A, Kong R, Gordon EM, Laumann TO, Zuo X-N, Holmes AJ, Eickhoff SB, Yeo BTT, 2018. Local-global parcellation of the human cerebral cortex from intrinsic functional connectivity MRI. Cereb. Cortex 28, 3095–3114. doi: 10.1093/cercor/bhx179. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R137] Shafiei G, Zeighami Y, Clark CA, Coull JT, Nagano-Saito A, Leyton M, Dagher A, Mišić B, 2019. Dopamine signaling modulates the stability and integration of intrinsic brain networks. Cereb. Cortex 29, 397–409. doi: 10.1093/cercor/bhy264. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R138] Shine JM, Aburn MJ, Breakspear M, Poldrack RA, 2018a. The modulation of neural gain facilitates a transition between functional segregation and integration in the brain. Elife 7, e31130. doi: 10.7554/eLife.31130. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R139] Shine JM, Bissett PG, Bell PT, Koyejo O, Balsters JH, Gorgolewski KJ, Moodie CA, Poldrack RA, 2016. The dynamics of functional brain networks: integrated network states during cognitive task performance. Neuron 92, 544–554. doi: 10.1016/j.neuron.2016.09.018. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R140] Shine JM, van den Brink RL, Hernaus D, Nieuwenhuis S, Poldrack RA, 2018b. Catecholaminergic manipulation alters dynamic network topology across cognitive states. Netw. Neurosci. 2, 381–396. doi: 10.1162/netn_a_00042. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R141] Shore J, Bernstein E, Lazer D, 2015. Facts and figuring: an experimental investigation of network structure and performance in information and solution spaces. Organ. Sci. 26, 1432–1446. doi: 10.1287/orsc.2015.0980. [DOI] [Google Scholar]

[R142] Smallwood J, Brown K, Baird B, Schooler JW, 2012. Cooperation between the default mode network and the frontal–parietal network in the production of an internal train of thought. Brain Res 1428, 60–70. doi: 10.1016/j.brainres.2011.03.072. [DOI] [PubMed] [Google Scholar]

[R143] Sun FT, Miller LM, D’Esposito M, 2004. Measuring interregional functional connectivity using coherence and partial coherence analyses of fMRI data. Neuroimage 21, 647–658. doi: 10.1016/j.neuroimage.2003.09.056. [DOI] [PubMed] [Google Scholar]

[R144] Torrence C, Compo GP, 1998. A practical guide to wavelet analysis. Bull. Am. Meteorol. Soc. 79, 61–78 . [DOI] [Google Scholar]

[R145] Totah NKB, Logothetis NK, Eschenko O, 2019. Noradrenergic ensemble-based modulation of cognition over multiple timescales. Brain Res. 1709, 50–66. doi: 10.1016/j.brainres.2018.12.031. [DOI] [PubMed] [Google Scholar]

[R146] Turchi J, Chang C, Ye FQ, Russ BE, Yu DK, Cortes CR, Monosov IE, Duyn JH, Leopold DA, 2018. The basal forebrain regulates global resting-state fMRI fluctuations. Neuron 97, 940–952. doi: 10.1016/j.neuron.2018.01.032, e4. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R147] Unsworth N, Robison MK, 2016. Pupillary correlates of lapses of sustained attention. Cogn. Affect. Behav. Neurosci. 16, 601–615. doi: 10.3758/s13415-016-0417-4. [DOI] [PubMed] [Google Scholar]

[R148] Urai AE, Braun A, Donner TH, 2017. Pupil-linked arousal is driven by decision uncertainty and alters serial choice bias. Nat. Commun. 8, 14637. doi: 10.1038/ncomms14637. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R149] Vaiana M, Goldberg EM, Muldoon SF, 2019. Optimizing state change detection in functional temporal networks through dynamic community detection. J. Complex Netw 7, 529–553. doi: 10.1093/comnet/cny030. [DOI] [Google Scholar]

[R150] van den Brink RL, Murphy PR, Nieuwenhuis S, 2016a. Pupil diameter tracks lapses of attention. PLoS One 11, e0165274. doi: 10.1371/journal.pone.0165274. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R151] van den Brink RL, Nieuwenhuis S, Donner TH, 2018. Amplification and suppression of distinct brainwide activity patterns by catecholamines. J. Neurosci. 38, 7476–7491. doi: 10.1523/JNEUROSCI.0514-18.2018. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R152] van den Brink RL, Pfeffer T, Donner TH, 2019. Brainstem modulation of large-scale intrinsic cortical activity correlations. Front. Hum. Neurosci. 13, 340. doi: 10.3389/fnhum.2019.00340. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R153] van den Brink RL, Pfeffer T, Warren CM, Murphy PR, Tona KD, van der Wee NJA, Giltay E, Van Noorden MS, Rombouts SARB, Donner TH, Nieuwenhuis S, 2016b. Catecholaminergic neuromodulation shapes intrinsic MRI functional connectivity in the human brain. J. Neurosci. 36, 7865–7876. doi: 10.1523/JNEUROSCI.0744-16.2016. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R154] van Rij J, Hendriks P, van Rijn H, Baayen RH, Wood SN, 2019. Analyzing the time course of pupillometric data. Trends Hear. 23, 233121651983248. doi: 10.1177/2331216519832483. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R155] Van Slooten JC, Jahfari S, Knapen T, Theeuwes J, 2018. How pupil responses track value-based decision-making during and after reinforcement learning. PLoS Comput. Biol. 14, e1006632. doi: 10.1371/journal.pcbi.1006632. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R156] Varazzani C, San-Galli A, Gilardeau S, Bouret S, 2015. Noradrenaline and dopamine neurons in the reward/effort trade-off: a direct electrophysiological comparison in behaving monkeys. J. Neurosci. 35, 7866–7877. doi: 10.1523/JNEUROSCI.0454-15.2015. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R157] Vatansever D, Menon DK, Manktelow AE, Sahakian BJ, Stamatakis EA, 2015. Default mode dynamics for global functional integration. J. Neurosci. 35, 15254–15262. doi: 10.1523/JNEUROSCI.2135-15.2015. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R158] Verstynen TD, Deshpande V, 2011. Using pulse oximetry to account for high and low frequency physiological artifacts in the BOLD signal. Neuroimage 55, 1633–1644. doi: 10.1016/j.neuroimage.2010.11.090. [DOI] [PubMed] [Google Scholar]

[R159] Wahn B, Ferris DP, Hairston WD, König P, 2016. Pupil sizes scale with attentional load and task experience in a multiple object tracking task. PLoS One 11, e0168087. doi: 10.1371/journal.pone.0168087. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R160] Warren CM, Eldar E, van den Brink RL, Tona KD, van der Wee NJ, Giltay EJ, Van Noorden MS, Bosch JA, Wilson RC, Cohen JD, Nieuwenhuis S, 2016. Catecholamine-mediated increases in gain enhance the precision of cortical representations. J. Neurosci. 36, 5699–5708. doi: 10.1523/JNEUROSCI.3475-15.2016. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R161] Warren CM, Wilson RC, Van Der Wee NJ, Giltay EJ, Van Noorden MS, Cohen JD, Nieuwenhuis S, 2017. The effect of atomoxetine on random and directed exploration in humans. PLoS One 12, e0176034. doi: 10.1371/journal.pone.0176034. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R162] Weir WH, Emmons S, Gibson R, Taylor D, Mucha PJ, 2017. Post-processing partitions to identify domains of modularity optimization. Algorithms 10, 93. doi: 10.3390/a10030093. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R163] Wierda SM, van Rijn H, Taatgen NA, Martens S, 2012. Pupil dilation deconvolution reveals the dynamics of attention at high temporal resolution. Proc. Natl. Acad. Sci. 109, 8456–8460. doi: 10.1073/pnas.1201858109. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R164] Wilson RC, Geana A, White JM, Ludvig EA, Cohen JD, 2014. Humans use directed and random exploration to solve the explore–exploit dilemma. J. Exp. Psychol. Gen. 143, 2074–2081. doi: 10.1037/a0038199. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R165] Wood SN, 2017. Generalized Additive Models: An Introduction with R, second edition Chapman and Hall/CRC; doi: 10.1201/9781315370279. [DOI] [Google Scholar]

[R166] Wood SN, 2013. On p-values for smooth components of an extended generalized additive model. Biometrika 100, 221–228. doi: 10.1093/biomet/ass048. [DOI] [Google Scholar]

[R167] Xia M, Wang J, He Y, 2013. BrainNet viewer: a network visualization tool for human brain connectomics. PLoS One 8, e68910. doi: 10.1371/journal.pone.0068910. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R168] Yael D, Vecht JJ, Bar-Gad I, 2018. Filter-based phase shifts distort neuronal timing information. eNeuro 5. doi: 10.1523/ENEURO.0261-17.2018. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R169] Yeo BTT, Krienen FM, Sepulcre J, Sabuncu MR, Lashkari D, Hollinshead M, Roffman JL, Smoller JW, Zöllei L, Polimeni JR, Fischl B, Liu H, Buckner RL, 2011. The organization of the human cerebral cortex estimated by intrinsic functional connectivity. J. Neurophysiol. 106, 1125–1165. doi: 10.1152/jn.00338.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R170] Yu AJ, Dayan P, 2005. Uncertainty, neuromodulation, and attention. Neuron 46, 681–692. doi: 10.1016/j.neuron.2005.04.026. [DOI] [PubMed] [Google Scholar]

[R171] Yue Q, Martin RC, Fischer-Baum S, Ramos-Nuñez AI, Ye F, Deem MW, 2017. Brain modularity mediates the relation between task complexity and performance. J. Cogn. Neurosci. 29, 1532–1546. doi: 10.1162/jocn_a_01142. [DOI] [PubMed] [Google Scholar]

[R172] Yule GU, 1926. Why do we sometimes get nonsense-correlations between time-series?–a study in sampling and the nature of time-series. J. R. Stat. Soc. 89, 1. doi: 10.2307/2341482. [DOI] [Google Scholar]

[R173] Záborszky L, Gombkoto P, Varsanyi P, Gielow MR, Poe G, Role LW, Ananth M, Rajebhosale P, Talmage DA, Hasselmo ME, Dannenberg H, Minces VH, Chiba AA, 2018. Specific basal forebrain–cortical cholinergic circuits coordinate cognitive operations. J. Neurosci. 38, 9446–9458. doi: 10.1523/JNEUROSCI.1676-18.2018. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R174] Zénon A, 2019. Eye pupil signals information gain. Proc. R. Soc. B Biol. Sci. doi: 10.1098/rspb.2019.1593. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R175] Zerbi V, Floriou-Servou A, Markicevic M, Vermeiren Y, Sturman O, Privitera M, von Ziegler L, Ferrari KD, Weber B, De Deyn PP, Wenderoth N, Bohacek J, 2019. Rapid reconfiguration of the functional connectome after chemogenetic locus coeruleus activation. Neuron 103, 702–718. doi: 10.1016/j.neuron.2019.05.034, e5. [DOI] [PubMed] [Google Scholar]

[R176] Zhang Z, Telesford QK, Giusti C, Lim KO, Bassett DS, 2016. Choosing wavelet methods, filters, and lengths for functional brain network construction. PLoS One 11, e0157243. doi: 10.1371/journal.pone.0157243. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R177] Zhao M, Zhou C, Chen Y, Hu B, Wang B-H, 2010. Complexity versus modularity and heterogeneity in oscillatory networks: combining segregation and integration in neural systems. Phys. Rev. E 82, 046225. doi: 10.1103/PhysRevE.82.046225. [DOI] [PubMed] [Google Scholar]

PERMALINK

The modulation of brain network integration and arousal during exploration

Nathan Tardiff

John D Medaglia

Danielle S Bassett

Sharon L Thompson-Schill

Abstract

1. Introduction

2. Methods

2.1. Subjects

2.2. Materials and procedure

Fig. 1.

2.3. MRI data acquisition

2.4. MRI preprocessing

2.5. Network construction

2.6. Multislice modularity optimization

2.7. Integration

2.7.1. Peri-explore integration analysis

2.8. Additional network measures

2.9. Pupillometry

2.9.1. Pupil analysis

2.10. Pupil–network relationships

2.11. Data analysis

3. Results

3.1. Exploration modulates pupil dilation

Fig. 2.

Fig. 3.

Fig. 4.

3.2. Exploration transiently modulates peri-explore integration

Fig. 5.

3.3. Evidence for differential modulation of integration across cognitive systems

Fig. 6.

3.4. Exploration induces complex changes in connectivity and topology

Fig. 7.

3.5. The relationship between pupil-linked arousal and network integration and segregation

Fig. 8.

3.6. Accounting for task-evoked confounds

4. Discussion

4.1. Complex peri-explore network dynamics

4.2. Specificity of network effects

4.3. Pupillary response to exploratory state

4.4. Limitations and future directions

Supplementary Material

Acknowledgments

Funding

Footnotes

Data and code availability statement

References

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases