A Basal Forebrain-Cingulate Circuit in Macaques Decides It Is Time to Act

Summary

The medial frontal cortex has been linked to voluntary action, but an explanation of why decisions to act emerge at particular points in time has been lacking. We show that, in macaques, decisions about whether and when to act are predicted by a set of features defining the animal’s current and past context; for example, respectively, cues indicating the current average rate of reward and recent previous voluntary action decisions. We show that activity in two brain areas—the anterior cingulate cortex and basal forebrain—tracks these contextual factors and mediates their effects on behavior in distinct ways. We use focused transcranial ultrasound to selectively and effectively stimulate deep in the brain, even as deep as the basal forebrain, and demonstrate that alteration of activity in the two areas changes decisions about when to act.

Highlights

• •Likelihood and timing of voluntary action in macaques can be partially predicted
• •Recent experience and present context influence when voluntary action occurs
• •A basal forebrain-cingulate circuit mediated effects of these factors on behavior
• •Stimulation of this circuit by ultrasound changed decisions about when to act

Deciding when to act is crucial for animals’ survival. Khalighinejad et al. used neuroimaging and ultrasound to identify a basal forebrain-cingulate circuit that tracks contextual factors in animals’ environments and mediates their effects on animals’ decisions about when to act.

Introduction

Leopards are expert stalkers. When they are in close proximity of prey, they ambush and wait for the right moment. If they charge too early, then the element of surprise is lost, and the prey will run away. If they charge too late, then they risk being detected. Deciding when to lunge is crucial for their survival. This decision not only comprises when to act but entails a decision regarding whether it is worth acting at all; there may be situations in which the prey is simply not worth pursuing or times when it is necessary to repress the urge to act on immediate desires. Deciding requires integrating information from the surrounding environment with past hunting experiences and internal state. In the current experiment, we examine precisely this question: how the current environmental context and recently experienced past contexts combine to influence “voluntary” decisions about when to act. We identify and record neural activity mediating decisions about when to act and examine the effect of manipulating the activity using ultrasound stimulation.

We introduced a new paradigm to investigate, in the macaque, how contextual factors and internal state, shaped by the present and past environment, are integrated to influence whether and when to act. Four macaques were trained to track the number of dots on a screen while in the MRI scanner. Dots appeared one at a time on a screen, and animals could decide to make a response, at a time of their choice, by tapping on a response pad in front of them (Figures 1A and 1B). The number of dots on the screen at the time of response determined the probability of reward. Reward probability was drawn from a sigmoid function; the longer the animals waited before responding, the more dots appeared on the screen, and the higher was the probability of reward (Figure 1D). Impulsive responses were unlikely to yield a reward. The probability distribution remained constant across the trials and sessions. Three features determined the present context: reward magnitude, the speed of the sequential appearance of the dots, and the inter-trial interval (ITI). Different levels of reward magnitude and ITI were associated with different dot colors and patterns, respectively (Figure 1C). These “present contextual factors” were varied independently of one another and in a pseudo-randomized order (Figure 1E). The reward magnitude and dot speed varied from trial to trial, and the ITI varied in blocks of 30 trials. In addition to the present context, the past context also varied. The past context was defined by the animal’s own recent behavior and recent reward experience—the outcomes and action times of recent past trials (Figure 1C).

Figure 1.

Experimental Task

(A) On each trial, animals tracked the number of dots on the screen (maximum number of dots = 25). They could make a response, at a time of their own choice, by touching a response pad in front of them. If they responded, then they received drops of juice or no juice as a function of the reward probability distribution at the time of response. There was a 4-s delay between the response and the outcome (AO-delay). Successful and unsuccessful outcomes are indicated by an upward- and downward-pointing triangle, respectively. The triangle remained on the screen for 2 s. If rewarded, then drops of juice were delivered by a spout near the animal’s mouth. The outcome phase was followed by an inter-trial interval (ITI; 3–7 s). If animals did not respond by 300 ms after the last dot, then the frame disappeared, and they had to wait for an AO-delay + ITI before the next trial (trial t+1) started. The color of the frame and the dots represent the potential reward magnitude on that trial. The patterns on each side of the screen represent the duration of the ITI.

(B) Timeline of one trial. At the beginning of each trial, an empty frame appeared on the left or right side of the screen. The frame was gradually filled with dots emerging from top to bottom. The dots appeared every 100, 200, or 300 ms, depending on trial type (in the example shown, a new dot emerges every 200 ms).

(C) Contextual factors from current and past trials were used to predict animals’ actTime. Present contextual factors consisted of reward magnitude (three levels), dot speed (three levels), and ITI (three levels). They were varied independently of one another and in a pseudo-randomized order. Reward and dot speed changed from trial to trial. The ITI changed in blocks of 30 trials. Past contextual factors consisted of reward outcome (four levels) and actTime on the past trial (continuous variable).

(D) The probability of getting a reward increased as more dots appeared on the screen, following a sigmoid curve. The probability distribution was constant across the trials and sessions.

(E) Correlation matrix of regressors from fMRI design.

First, behavioral analyses demonstrated that both types of contextual information influenced decisions about whether and when to act. A large proportion of variance in decisions about when to act could be explained by a quantitative model that deduced a deterministic component of time to act by identifying features of the environment relating to both the current context and the recent past context. Second, we used fMRI to look for brain activity that is parametrically related to the factors that change the likelihood of action rather than action initiation per se. We identified two areas—the anterior cingulate cortex (ACC) and basal forebrain (BF)—that tracked these contextual factors and mediated their effect on behavior in distinct ways. Third, we used focused transcranial ultrasound stimulation (TUS) to modulate activity in these brain areas. We simulated the acoustic wave propagation and obtained offline resting-state fMRI (rs-fMRI) to show that it is possible to manipulate activity in an area deep in the brain, such as the BF, using TUS. Fourth, we showed that alteration of activity in the ACC and BF by TUS changed decisions about when to act.

Results

Animals Used Contextual Factors to Decide Whether and When to Act

We used a hierarchical linear model to test whether animals use contextual factors to decide when to act. Time to act (actTime) was indexed by the number of dots on the screen at the time of response. Note that there was a correspondence (a sigmoid relationship; Figure 1D) between dot number and reward probability and that experimental manipulation of “dot speed” ensured that actTime as indexed by the number of dots was decorrelated from time measured in seconds. This facilitated interpretation of the results; changes in actTime, as measured by number of dots, therefore signified a deliberate decision process rather than merely passage of time. On average, animals waited for 14 ± 4 dots before responding (Figure S1A), which was associated with a 73% chance of reward. Given the sigmoid distribution of reward function, there is little to gain by employing an actTime of longer than 17 dots (92% chance of reward; Figure 1D). The task finished after 40 min regardless of the number of trials performed. That means that animals might collect fewer rewards overall across the whole session when they waited longer before responding on any given trial. A multilevel ANOVA (STAR Methods) showed that all aspects of present context (reward magnitude, χ²(2) = 284, p < 0.001; dot speed, χ²(2) = 1,465, p < 0.001; and ITI, χ²(2) = 44, p < 0.001) and past context (reward outcome on past trial, χ²(3) = 144, p < 0.001; and actTime on past trial, χ²(1) = 25, p < 0.001) influenced animals’ actTime. actTime was longer during long compared with short ITI blocks (Tukey’s honestly significant difference [HSD]; β = 0.16 ± 0.02, Z = 6, p < 0.001) (Figure 2A), under fast compared with slow dot speed conditions (β = 1.06 ± 0.02, Z = 42, p < 0.001) (Figure 2B), when offered a large compared with a medium reward (β = 0.22 ± 0.03, Z = 9, p < 0.001), and when offered a small compared with a medium reward (β = 0.44 ± 0.03, Z = 17, p < 0.001), giving rise to a U-shaped effect of reward magnitude on actTime (Figure 2C; see Figure S1 for further discussion). actTime was shorter when they had received a large reward compared with no reward on the past trial (β = −0.36 ± 0.03, Z = −11, p < 0.001; Figure 2D), and it was longer when they had already delayed actTime on the past trial (β = 0.07 ± 0.01; Figure 2E).

Figure 2.

Animals Used Contextual Factors to Decide Whether and When to Act

(A–E) The effect of present and past contextual factors on actTime (left y axis). The corresponding reward probability at response is displayed on the right y axis. actTime was longer in long (7 s) and medium (5 s) compared with short (3 s) ITI blocks (A); in fast compared with slow trials (B); when they were offered a large compared with a medium reward and when offered a small compared with a medium reward, giving rise to a U-shaped effect of reward magnitude on actTime (C); when a previous trial was not rewarded (D); and when they had already delayed actTime on the past trial (E). For illustrative purposes, actTime in the immediate past trial is binned into three groups. Each color represents one animal, and each ring is one testing session. The gray columns illustrate the group mean across all observations. Multilevel ANOVA followed by Tukey’s HSD.

(F) The log of the odds of responding during the trial for different levels of present and past contextual factors.

^∗p < 0.05, ^∗∗p < 0.01, ^∗∗∗p < 0.001.

See also Figure S1.

Having found an effect of contextual factors on decisions about when to act, we asked whether the same factors influence whether it is worth acting as well as how quickly to do so. We used hierarchical logistic regression to predict the odds of responding from present and past contextual factors and showed that this was indeed the case, confirming the importance of these factors in influencing self-initiated action (Figure 2F). Across all testing sessions, animals refrained from responding in 16.38% ± 11.42% of the trials. They were more likely to respond in large and medium compared with small reward trials (large versus small, odds ratio [OR] = 12.82, Z = 17.92, p < 0.001; medium versus small, OR = 6.94, Z = 16.58, p < 0.001) and when they had been rewarded on the past trial (large versus no reward, OR = 3.27, Z = 7.58, p < 0.001; medium versus no reward, OR = 2.10, Z = 4.93, p < 0.001; small versus no reward, OR = 2.57, Z = 5.66, p < 0.001). On the other hand, they were more likely to refrain from responding in long ITI blocks (long versus short, OR = 0.48, Z = −6.23, p < 0.001) and when they had waited for a long time on the past trial (OR = 0.86, Z = −2.55, p = 0.01). These results suggest that animals were less likely to respond when the average reward rate of the environment was low.

It is not clear why animals sometimes refrain from responding in an experimental task when any task-related response is usually likely to increase the likelihood of reward. Nevertheless, it is well known that they often do so (San-Galli et al., 2018, Stoll et al., 2016a). One way to interpret the results is that the animals doing so are avoiding paying a cost that is entailed by performing the trial. Performing a trial requires more than just the motor act of responding and also entails cognitive demands related to stimulus attention and response withholding (Manohar et al., 2015). This might mean that animals may judge the effort of engaging with a trial as not worth the reward that could be obtained for doing so when the reward rate is low.

Contextual Factors Explained a Large Proportion of Variance in Time to Act

To determine the fraction of actTime that was explainable by present and past contextual factors, we used a Cox proportional hazards model. Cox regressions are a class of survival models suitable for relating the time that passes before a specific event to one or more covariates. This model has been used previously to estimate action time in rats (Murakami et al., 2017), although here we expand the approach to consider the influence of a wider range of factors that reflect both past and present context on actTime (STAR Methods).

First we asked whether present and past contextual factors significantly contribute to the model (all subsequent tests are corrected for multiple comparisons; Figure 3A). The Cox regression coefficients were significantly negative for dot speed (one-sample t test; t(44) = −15.5, p < 0.001), ITI (t(44) = −5.5, p < 0.001), and actTime on past trials (significant in 4 of 10 past trials; t(44) < −3.72, p < 0.02), suggesting that slower dot speed and longer ITI on the current trial and longer actTime on past trials lengthen actTime on the current trial. Cox regression coefficients were significantly positive for expected reward magnitude on the current trial (t(44) = 8, p < 0.001) and reward outcome on the past trials (significant for the trial immediately preceding the current trial; t(44) = 12.38, p < 0.001); a larger potential reward on the current trial and a larger reward outcome on the past trial shorten actTime on the current trial. We then used the Cox regression coefficients from the current and immediately preceding trial (only coefficients from the immediately preceding trial were used because only these had been significant for both past reward outcome and past actTime) to estimate the expected actTime at each trial (STAR Methods; Figure 3B). We termed this estimate deterministic actTime_{present + past context}, which is defined as the number of dots, given present and past context, at which an animal is expected to respond on any given trial. Deterministic actTime was also later used for model-based fMRI analysis.

Figure 3.

Contextual Factors Explained a Large Proportion of Variance in Time to Act

(A) Cox regression coefficients for present (left panel) and past (middle and right panels) contextual factors. A negative Cox coefficient means a negative effect of predictor on the probability of responding.

(B–D) Cox regression coefficients were used to estimate deterministic actTime on each trial. Deterministic actTime is superimposed on observed actTime for comparison. Deterministic actTime was estimated from both present and past contextual factors (B), from present contextual factors alone (C), and from past contextual factors alone (D).

(E) Proportion of variance (PEV) in actTime explained by the Cox regression model. PEV is estimated separately from present and past, present, and past contextual factors. Each color represents one animal, and each ring is one testing session. The gray columns illustrate the group mean.

One-sample t tests; ^∗p < 0.05, ^∗∗p < 0.01, ^∗∗∗p < 0.001, corrected for multiple comparisons.

Subsequently, we estimated the Cox regression coefficients separately from their present and past components. We then used the Cox coefficients relating to either the present or the immediately preceding trial to derive two separate actTime estimates. These new estimates were termed deterministic actTime_{present context} (Figure 3C) and deterministic actTime_{past context} (Figure 3D), respectively. Finally, we asked what percentage of variability in observed actTime could be explained by present, past, or a combination of both contexts (STAR Methods). On average, present and past contextual factors together explained 36% ± 9% of actTime variance. Of these, 25% ± 10% and 12% ± 5% were explained by present and past contextual factors, respectively (Figure 3E).

ACC and BF Activity Is Correlated with Time to Act

Having shown that animals use contextual factors to decide when to act, we used fMRI to identify potential brain mechanisms mediating this behavior. We first used a generalized linear model (GLM; STAR Methods, GLM.1) to look for brain areas in which activity reflected parametric variation in the empirically observed actTime, as indexed by dot number at the time of response, and then asked whether the same areas integrated contextual factors to compute deterministic actTime derived from the Cox model. The first analysis revealed two main bilateral/midline regions in which a blood-oxygen-level-dependent (BOLD) signal was modulated by observed actTime (whole-brain cluster-based correction, Z > 2.3, p < 0.01; Table S1): (1) ACC (peak Z = 3.88, Caret-F99 Atlas [F99]: x = 0.5, y = 20.5, z = 12.5), (2) BF. The activity in the BF extends from the anterior-medial BF (amBF) containing the medial septum/diagonal band of Broca (peak Z = 4.34, F99: x = −0.5, y = 4.0, z = 1.0) to the posterior-lateral BF (plBF) containing the nucleus basalis of Meynert (peak Z = 4.49, F99: x = 4.5, y = 2.0, z = −2.0) (Figure 4A; see Figure S2A for an alternative analysis). Corresponding BF sub-regions have been described in the human BF (Fritz et al., 2019, Markello et al., 2018).

Figure 4.

The ACC and BF Encode Time to Act

(A) Whole-brain analysis showing voxels where activity reflected parametric variation in the empirically observed actTime. Here we focused on areas with bilateral/midline activity: the ACC (top panel) and BF (middle [amBF] and bottom [plBF] panels). Whole-brain cluster-based correction, Z > 2.3.

(B) ROI time course analysis of the ACC (top panel), amBF (middle panel), and plBF (bottom panel), showing the relationship between BOLD and actTime. The lines and shadings show the mean and standard error (SE) of the β weights across the sessions, respectively. Time zero is the response time. Note that, because of delay in the BOLD hemodynamic response function, the BOLD signal time course peaks 3 s after neural activity. When the delay in BOLD response is taken into account, it is clear that BOLD activity reflects neural events occurring before the response onset.

(C) No significant difference in actTime encoding was observed between the ACC, amBF, and plBF. Each color represents one animal, and each ring is the peak beta-weight of one testing session. The gray columns illustrate the group mean.

See also Figures S2 and S3 and Table S1.

To illustrate the timing of encoding of observed actTime in the ACC, amBF, and plBF, we extracted and averaged the BOLD time course of each voxel within each region of interest (ROI) with respect to response onset. ROIs were defined as spheres centered on the peak of the activations (Figure 4B; Figure S3 illustrates the time course of each contextual factor). We found no significant difference in activity between these three areas (multilevel ANOVA, χ²(2) = 0.77, p = 0.68) (Figure 4C). Given the delay in the hemodynamic response (BOLD signals take approximately 3 s to peak in the monkey; Chau et al., 2015), it is clear that the activity in all three areas begins in advance of the response.

Trial-by-trial variation in dot speed decorrelates actTime, measured in number of dots, from passage of time, measured in seconds (r = 0.36 ± 0.28, across all sessions, where r is the correlation coefficient). However, to make sure that the relationship between BOLD and observed actTime is not simply explained by passage of time, we ran a new GLM (STAR Methods, GLM2.2) but this time also added time to act in seconds to the model as a covariate. The result of the new model was similar to that obtained previously (compare Figures 4B and 4C and Figures S2B and S2C).

BF Integrates Contextual Factors to Construct the Deterministic Component of Time to Act

Whole-brain analyses suggested that ACC, amBF, and plBF activity is correlated with observed actTime. We next asked whether the same areas integrated contextual factors to compute the deterministic component of actTime, as estimated by the Cox regression model. We added deterministic actTime to the previous time series GLM as the variable of interest and the observed actTime (measured in both number of dots and seconds) as covariates (trial-by-trial correlation between deterministic and observed actTime as measured in number of dots and seconds are r = 0.24 ± 0.23 and r = 0.11 ± 0.27 across sessions, respectively) (STAR Methods, GLM2.3; Figure 5A). We found that deterministic actTime explained BOLD activity in the amBF (one-sample t test; t(44) = 2.78, p = 0.008, d = 0.41, where d is the effect size) and plBF (t(44) = 3.59, p < 0.001, d = 0.54). We found no significant relationship between deterministic actTime and BOLD in the ACC (t(44) = −1.15, p = 0.26) (Figure 5B; leave-one-out procedure for peak selection). Multilevel ANOVA showed a significant main effect of ROI (χ²(2) = 14.65, p = 0.0006); deterministic actTime was a better predictor of BOLD signal in the plBF compared with the ACC (Tukey’s HSD; β = 0.07 ± 0.02, Z = 3.82, p = 0.0004) and in the amBF compared with the ACC (β = 0.05 ± 0.02, Z = 2.93, p = 0.0095). We found no significant difference between plBF and amBF (β = 0.02 ± 0.02, Z = 0.88, p = 0.65). This suggests that the BF is more strongly involved in integrating present and past contextual information to construct the deterministic component of actTime, compared with the ACC. A model-based whole-brain analysis confirmed the importance of the BF (Figure S4).

Figure 5.

The BF Encodes the Deterministic Component of Time to Act

(A) ROI time course analysis of the ACC, amBF, and plBF, showing the relationship between BOLD activity and deterministic actTime estimated from present and past context. Format is as in Figure 4B.

(B) The relationship between deterministic actTime and BOLD signal was significantly stronger in the plBF and amBF compared with the ACC. Format is as in Figure 4C.

(C) ROI time course analysis of the amBF and plBF, showing the relationship between BOLD activity and deterministic actTime as estimated from present and past, present, and past contextual factors.

(D) The relationship between deterministic actTime, as estimated from past contextual factors, and BOLD signal was significantly stronger in the amBF compared with the plBF. In boxplots, the central line indicates the median, and the bottom and top edges of the box indicate the 25^th and 75^th percentiles, respectively. Whiskers extend to the most extreme data points not considered outliers.

(E) PPI analysis between the BOLD signal in the plBF and amBF, with deterministic actTime as the psychological factor. Trial-by-trial variation in the activity in the plBF was significantly related to trial-by-trial variation in the activity in the amBF as a function of deterministic actTime.

One-sample t tests and multilevel ANOVA followed by pairwise t test. ^∗p < 0.05, ^∗∗p < 0.01, ^∗∗∗p < 0.001. See also Figure S4.

Next we asked whether present and past contextual factors contribute equally to encoding of deterministic actTime in the BF. Deterministic actTime_{present context} and deterministic actTime_{past context} were used in a time-series GLM (STAR Methods, GLM2.4), with the observed actTime (measured in both number of dots and seconds) as covariate (Figure 5C). BOLD activity in the plBF was related with trial-by-trial variation in deterministic actTime_{present context} (one-sample t test; t(44) = 3.16, p = 0.003, d = 0.47) and deterministic actTime_{past context} (t(44) = 2.47, p = 0.017, d = 0.37). However, BOLD activity in the amBF was only related with deterministic actTime_{past context} (t(44) = 4.04, p = 0.0002, d = 0.60), but its relationship with actTime_{present context} was not significant (t(44) = 0.85, p = 0.40; Figure 5D). Importantly, we found a significant interaction between deterministic actTime (present and past versus present versus past) and BF (amBF versus plBF) (multilevel ANOVA; χ²(2) = 7.78, p = 0.02); the relationship between deterministic actTime, when estimated from past as opposed to present context, and BOLD was significantly stronger in the amBF compared with the plBF (β = 0.1 ± 0.04, t(220) = 2.64, p = 0.009) (Figure 5D), suggesting that the amBF mostly employs past contextual factors to construct actTime. Its estimate of actTime_{past context} may then be passed to the plBF and integrated with actTime_{present context} to estimate time to act. Thus, we should predict that functional connectivity between the plBF and amBF is moderated by deterministic actTime. A psychophysiological interaction (PPI) analysis (O’Reilly et al., 2012; STAR Methods, GLM2.5) confirmed that this was the case (t(44) = 3.53, p = 0.001, d = 0.52) (Figure 5E). Although PPI cannot reveal the direction of influence, it is clear that the deterministic component of actTime is constructed within a circuit comprising both BF subdivisions. Finally, although timing differences in BOLD signals must be interpreted with care, it is noteworthy that, despite the proximity and similar nature of the two structures, amBF activity precedes plBF activity, and when the BOLD hemodynamic lag is considered, it is clear that amBF signaled actTime_{past context} even during the previous ITI prior to trial onset (Figure 5C). In contrast, actTime_{present context} is only encoded after trial onset, when the factors determining it are observable.

The Effect of Expected Reward on Time to Act Is Mediated by the ACC

Our findings so far suggest that the BF integrates present and past contextual factors to construct the deterministic component of actTime. However, although ACC’s activity is correlated with actTime observed on any trial (Figure 4), its activity does not reflect what the actTime ought to be on any trial—the deterministic actTime—in a simple manner (Figure 5). We hypothesized that the ACC might mediate the effect of a specific element among contextual factors on actTime rather than a compound effect, as is the case with the BF. More specifically, given its known role in reward-guided decision-making (Kolling et al., 2016, Wittmann et al., 2016), we predicted that the observed relationship between reward magnitude and actTime (Figure 2C) might be mediated by the ACC. One method to test this hypothesis is mediation analysis (STAR Methods, Figure 6A). Mediation occurs when the direct effect of contextual factors on actTime (path c) can be explained by an indirect pathway through a brain area (path a × b). Note demonstrating mediation requires only that there be a significant indirect effect (path a × b) (Hayes, 2018, Zhao et al., 2010). This was indeed the case; the quadratic relationship between expected reward magnitude and actTime was mediated by an indirect effect through the ACC (bootstrapped p = 0.01). However, we did not find a mediation effect for the amBF (bootstrapped p = 0.91) or plBF (bootstrapped p = 0.24) (Figure 6B) or between other elements among contextual factors (dot speed, ITI, past reward; however, see the legend of Figure 6, past actTime) and actTime (all bootstrapped p > 0.2). This suggests that the ACC mediates the observed relationship between reward magnitude and actTime (Figure 2C). This reward-guided actTime could then be passed to the plBF to be used in construction of actTime_{present context} along with actTime_{past context} from the amBF. This would be consistent with the earlier timing of the ACC effect in relation to the plBF effect (Figure 4B), but it can be difficult to interpret timing differences in the BOLD signal, especially when they occur in spatially distant brain areas such as the ACC and plBF. A PPI analysis, however, was also consistent with this hypothesis; functional connectivity between the ACC and plBF (t(44) = −2.07, p = 0.044, d = 0.31) was moderated by quadratic reward magnitude (STAR Methods, GLM2.6). We did not find a moderation effect of reward magnitude on the functional connectivity between the ACC and amBF (t(44) = 0.81, p = 0.42). This suggests that, by encoding the relationship between expected reward and actTime, the ACC contributes to construction of actTime mainly through connectivity with the plBF rather than the amBF (Wilcoxon signed-rank test; Z = 2.11, p = 0.035, r = 0.22) (Figure 6C). In summary, the ACC is modulated by reward magnitude (Figure S3A) and by the relationship between reward magnitude and action time (Figure 6), but unlike the BF (Figure 5), it does not integrate contextual factors from the present and past to determine how long animals should wait before responding at each trial.

Figure 6.

Mediation Analysis Can Explain the Relationship between Reward Magnitude and actTime

(A) Mediation occurs when the direct effect of contextual factors on actTime (path c) can be explained by an indirect pathway through a brain area (path a × b).

(B) The ACC mediated the quadratic relationship between reward magnitude and actTime. This was not the case for plBF or amBF or between other elements among contextual factors. However, after controlling for the effect of reward magnitude on the current trial, we found a small mediation effect of the ACC for the influence of past reward on actTime (bootstrapped p = 0.06). The numbers are the coefficients of each path. Significant paths are displayed with thick arrows.

(C) PPI analysis between the BOLD signal in the ACC and plBF (top panel) and amBF (bottom panel), with quadratic reward magnitude as the psychological factor. Trial-by-trial variation in the activity in the ACC was more strongly related with trial-by-trial variation in the activity in the plBF as a function of reward magnitude compared with the amBF (Wilcoxon signed-rank test; Z = 2.11, p = 0.035, r = 0.22).

Ultrasound Stimulation Selectively and Effectively Modulates Activity in Deep Brain Areas

It has been shown that 40 s sonification at 250 kHz reaches deep cortical areas such as the ACC and subcortical brain areas such as the amygdala and does so in a relatively focal manner, having less effect on adjacent, overlying brain areas (Folloni et al., 2019). We therefore first examined whether TUS could also be used to manipulate activity in an even deeper structure, the BF. To verify whether TUS effectively modulates BF activity, we first simulated the acoustic wave propagation and its thermal effect in a whole-head, finite-element model based on a high-resolution monkey computed tomography (CT) scan (Figure S5). The resulting “impact probability map” showed that the peak of the expected neuromodulatory effect was at the BF target (Figure 7A). We then performed offline rs-fMRI under anesthesia immediately after BF TUS in three animals (STAR Methods) and compared the data with rs-fMRI from three anesthetized control animals without TUS (Verhagen et al., 2019). Previous investigations of the neural effects of TUS have found that activity coupling between the targeted area and anatomically interconnected areas is altered (Folloni et al., 2019, Verhagen et al., 2019). The activity coupling of other brain areas, however, is unaffected. We might therefore expect altered coupling after BF TUS that is restricted to BF, but possibly extending to strongly connected regions such as the ACC. However, the BF projects broadly across many brain areas, and it has recently been reported that global rs-fMRI fluctuations across the whole brain are suppressed after BF perturbation (Turchi et al., 2018). Therefore, it is also possible that there will be widespread suppression of coupling elsewhere in the brain after BF TUS. We can test these predictions by quantifying the effect of TUS on brain activation by regressing the whole-brain connectivity profile of a seed area under the control condition against that observed after BF TUS. Importantly, we can repeat this regression analysis seeded for every point in the brain and report a map with scaling factors indicative of the TUS effect across the whole brain (Figure 7B). Indeed, we observed enhanced coupling in the BF target site and selectively connected regions, such as the ACC, but not elsewhere. Although the BF projects to many cortical areas, its connections with the ACC are notable because the ACC also projects to the BF (Ghashghaei and Barbas, 2001). This is, of course, also consistent with the evidence that we have already presented that the BF and ACC act in concert to influence actTime. In contrast, there was widespread suppression of coupling elsewhere in the brain after BF TUS compared with the control (Figure 7B). These effects of TUS were specific to stimulation of the BF and not observed after TUS of another region, as evidenced by seed-based correlation analyses following TUS over the supplementary motor area (SMA) (Verhagen et al., 2019; Figure S6).

Figure 7.

Ultrasound Stimulation of the ACC and BF Modulates Time to Act

(A) Probability map of the combined neural impact of sonications targeted at the BF. This is calculated as the average stimulation intensity (I_sppa) map across the two consecutive stimulations delivered over the left and right hemisphere, respectively (Figure S5). The combined impact probability map is overlaid on a standard F99 brain. The low impact probability level corresponds to 10 W/cm², in correspondence with previous work (Verhagen et al., 2019) and Figure S5. As the color changes from red to bright yellow, the probability of neuromodulation from bilateral BF TUS increases.

(B) The result of a regression analysis comparing, for each point in the brain, its whole-brain coupling map (“connectome”) in the no-stimulation state against its coupling map observed after TUS targeted at the BF. The hot colors indicate points in the brain with enhanced whole-brain coupling strength after BF TUS compared with no TUS, whereas the cool colors indicate reduced whole-brain coupling strength after BF TUS compared with no TUS. Compared with no TUS, BF TUS showed a clear enhancement in activity coupling within the BF and between the BF and ACC and superior temporal area (Ts2).

(C) Sites where TUS was applied for each animal (S1, S2, S3, and S4) to assess its effect on actTime. The TUS transducer was set at a resonance frequency of 250 kHz and concentrated ultrasound in a cigar-shaped focal spot in the ACC, BF, and POp. For the sham control, the transducer was positioned on the skull but without sonication.

(D) Animals acted more quickly after ACC than after BF, POp, or sham TUS when offered medium or large compared with small rewards. actTime is indexed as the number of dots at response (D_i) and the corresponding reward probability (D_ii).

(E) BF TUS reduced the bias between observed and deterministic actTime compared with POp and sham TUS.

Error bars show SEM across observations. Multilevel ANOVA followed by pairwise t test. ^∗p < 0.05, ^∗∗p < 0.01, ^∗∗∗p < 0.001.

See also Figures S5–S7.

Ultrasound Stimulation of the ACC and BF Modulates Time to Act

If the effect of contextual factors on actTime is mediated by the BF and ACC, then causal manipulation of these areas should modulate the relationship between contextual factors and actTime. We collected a new dataset outside of the scanner, but this time we used TUS to modulate ACC and BF activity immediately before animals performed the task (STAR Methods). Each animal underwent four stimulation conditions, including two conditions of interest and two control conditions (Figure 7C). Our two primary conditions were bilateral ACC TUS and bilateral BF TUS. We also chose two control conditions: passive bilateral sham control with the transducer positioned on the skull but without sonication and one active control with TUS targeted at the bilateral parietal operculum (POp), a region distinct from and anterior to intraparietal areas linked to sensorimotor decision-making (Shadlen and Kiani, 2013), in which activity was unmodulated by our contrast of interest or any other task event. Each condition was repeated five times on separate days in a pseudo-randomized order for each animal (20 sessions per animal).

Based on the findings from the mediation analysis, we first predicted that ACC TUS should modulate the relationship between reward magnitude and actTime compared with control conditions. Multilevel ANOVA (GLM3.1) showed a significant interaction between TUS and reward magnitude (χ²(6) = 26.89, p = 0.0002). Planned contrasts (STAR Methods) revealed that animals acted quicker after ACC TUS than after BF, control POp, or sham TUS but only when offered a medium reward (compared with a small one) (ACC versus BF, β = −1.35 ± 0.31, t(13,068) = −4.34, p < 0.001; ACC versus POp, β = −1.13 ± 0.31, t(13,068) = −3.62, p < 0.001; ACC versus sham, β = −0.66 ± 0.31, t(13,068) = −2.11, p = 0.035) or a large reward (compared with a small one) (ACC versus BF, β = −1.16 ± 0.30, t(13,068) = −3.78, p < 0.001; ACC versus POp, β = −0.65 ± 0.30, t(13068) = −2.13, p = 0.033; ACC versus sham did not reach significance) (Figure 7D_i; see Figure S7 for individual animal data). This pattern of results confirms the interpretation of the fMRI results and suggests that the ACC causally mediates the relationship between reward magnitude and actTime. Interestingly, the effect was observed for large and medium levels of reward. Although the higher reward prospects on such trials make them enticing, it is important to wait before responding to increase the likelihood of success. Because ACC TUS compromised the ability to delay responding on just such trials, it led to a reduced likelihood of receipt of large rewards (Tukey’s HSD; ACC versus sham, β = −0.05 ± 0.01, Z = −4.04, p < 0.001; ACC versus POp, β = −0.04 ± 0.01, Z = −2.87, p = 0.022; ACC versus BF, β = −0.04 ± 0.01, Z = −3.21, p = 0.007; BF versus sham, Z = −0.80, p = 0.85; BF versus POp, Z = 0.004, p = 0.99) (Figure 7D_ii).

Previous studies showed that the TUS effect starts to diminish after approximately 1 h but is still evident up to 2 h after the end of stimulation (Folloni et al., 2019, Verhagen et al., 2019). In our experiment, the task was performed within 40 min of stimulation, well within the 1-h peak effect of TUS. Therefore, we do not expect to see a diminishing effect of stimulation on the behavior recorded during the task period. Nevertheless, we repeated the same analyses as before, but this time added “time passed since beginning of the session” as a covariate. This did not influence the results. Finally, we ran a control analysis by including all present and past contextual factors to check whether modulation of ACC activity could influence the relationship between other contextual factors (dot speed, ITI, past reward outcome, and past actTime) and actTime. We found no interaction between contextual factors and stimulation conditions (all p > 0.16) other than with reward magnitude (χ²(6) = 37.30, p < 0.001).

Having shown that TUS could be used to target the BF, we next investigated its behavioral effect. We hypothesized, based on our previous findings, that causal manipulation of the BF might have a stronger effect on the relationship between deterministic and observed actTime on a trial-by-trial basis compared with ACC, control POp, or sham stimulation. To quantify this relationship, we derived a measure of trial-by-trial actTime bias by subtracting each trial’s deterministic actTime from the observed actTime and took the absolute value under each stimulation condition (STAR Methods, GLM3.2). Multilevel ANOVA showed a significant main effect of stimulation condition on actTime bias (χ²(3) = 12.32, p = 0.006); however, this effect was not observed across all animals (Figure S7). Planned contrasts (STAR Methods) showed that BF TUS reduced the bias between observed and deterministic actTime compared with both control POp (β = 0.28 ± 0.11, t(12,430) = 2.45, p = 0.014) and sham (β = 0.36 ± 0.11, t(12,430) = 3.18, p = 0.001) TUS (Figure 7E; Figure S7). The differences between the effects of BF and ACC TUS on this measure, when compared directly, were not significant (although, notably, post hoc tests showed no significant difference between ACC and POp or sham TUS, either). This overall pattern of behavior change after TUS is consistent with the recording data that suggested that, although the BF had the key role in the encoding of actTime, the BF and ACC are strongly connected and interact (Figures 6 and 7B; Figure S6).

Discussion

Although a number of studies have considered the important question of how an action is initiated, here we identified factors that influence when (Figures 2A–2E) and even whether (Figure 2F) an action should be initiated. These factors relate to both the current context, signaled to the animal by cues in the environment, and the recent past context, such as recent rewards and the timing of recent previous decisions (Figure 3). Even though a large fraction of variance in animals’ behavior remains unexplained, by careful and controlled manipulation of identifiable features of the environment, we managed to explain a considerable proportion of variance in their decision time to act (Figure 3E).

Few theoretical accounts exist to explain why actions might be made at one time rather than another, and none is quite adequate to explain all of the observations made in the current study. Potentially marginal value theorem (MVT), which predicts many aspects of decision-making (Pearson et al., 2014), might be used to predict when actions will be made. In the fMRI experiment, macaques responded more deliberately and slowly when there was a possibility of large reward as opposed to a medium reward (Figure S1). Careful long actTimes ensure that opportunities to obtain a large reward are not wasted before transitioning to the next trial when the possibility of another large reward is lower and equal to the possibility of a small or medium reward. However, many aspects of the current findings are not explained easily by MVT because it predicts that animals should respond faster when their intake rate diminishes to the average intake rate of the environment (Charnov, 1976, Krebs, 1987). In contrast, we found that on small reward trials actions were made more slowly rather than more quickly. These aspects of deciding when to act were consistent with an alternative notion that vigor and speed of responding increase as the average reward rate increases; actions are made quickly so that reward opportunities are not lost, whereas slow actions entail an opportunity cost (Niv et al., 2007). Finally, in line with observations made in rodents (Murakami et al., 2014, Murakami et al., 2017), we observed that an additional important predictor of the timing of the next action is the timing of recent past actions.

We used fMRI to look for brain activity that is parametrically related to the factors that change the likelihood of action. Note that we are not looking for brain areas that initiate the action but, rather, those that encode the current and recent past factors that influence the right time to make the action. We identified two areas (Figure 4): the ACC and BF. However, the influence of present and past context on when an action will emerge was more strongly encoded by the BF compared with the ACC; BOLD activity in the BF could be explained by trial-by-trial variation in deterministic actTime, which is the predicted actTime given present and past contextual factors. Moreover, compared with control POp TUS and sham TUS, manipulation of the BF with TUS significantly altered the closeness of the relationship between deterministic actTime and the actually observed actTime (Figure 7E). In contrast, ACC TUS did not change this relationship in comparison with control POp TUS and sham TUS.

BF activity extended across both the amBF (approximately the medial septum/diagonal band of Broca) and plBF (approximately the nucleus basalis of Meynert) (Figure 4). Although plBF activity was significantly correlated with deterministic actTime predicted from both present and past contextual factors, amBF activity was only correlated with deterministic actTime predicted from past contextual factors. The animal’s internal state (shaped by past trials) may be passed from the amBF to the plBF and integrated with present contextual factors to inform time to act in the plBF. Some aspects of the influence of present contextual factors on construction of deterministic actTime involve the ACC; the ACC mediated the effect of reward magnitude on actTime. PPI analyses were consistent with a circuit comprising both the BF and ACC in which the component parts of the BF, the amBF and plBF, were connected as a function of deterministic actTime, whereas current reward magnitude influenced the functional connectivity between the ACC and plBF (Figure 8).

Figure 8.

A Schematic View of the Main Findings

A BF-cingulate circuit in macaques decides it is time to act by integrating identifiable features of both the current context and the recent past context.

Within the cortex, voluntary action has been especially linked to medial frontal areas such as the ACC (Heilbronner and Hayden, 2016, Thaler et al., 1995) or closely adjacent areas such as the SMA and pre-SMA (Lara et al., 2018). Although investigations of such areas have made clear that the medial frontal cortex is important when action is initiated voluntarily, the precise role has been less clear. The present results demonstrate that it is the influence of the prospect of reward on the decision about when to act that is mediated by the ACC. This is consistent with the fact that the ACC carries multiple value signals (Kolling et al., 2016, Meder et al., 2017) but, at the same time, has also been associated with determining the balance between persistence with a given manner of responding or changing to an alternative mode (Ebitz et al., 2018, Karlsson et al., 2012, Parvizi et al., 2013, Stoll et al., 2016b, Wittmann et al., 2016), when to leave a depleting patch in a foraging task (Hayden et al., 2011), and signaling initiation of an action plan by integrating evidence toward a decision bound (Hunt et al., 2018). We did not find a clear mediation effect for the influence of past reward on action time (Figure 6). This was unexpected, given previous studies showing the role of the ACC in encoding of reward history (Kennerley et al., 2006, Seo and Lee, 2007). It may reflect the limited influence of this one factor on actTime, given the many other factors that also affected actTime. Nevertheless, we found a direct effect of past trial reward outcome on the ACC, irrespective of its effect on actTime (Figure S3D), and there was a suggestion, from a marginally significant result, of a degree of mediation of the influence of the past reward on actTime (see the legend of Figure 6).

The prospect of reward may have a complex effect on the determination of when to act. On one hand, when a reward is available, it may be adaptive to increase the response rate (Niv et al., 2007). On the other hand, if there is a particularly large reward available right now on the current trial (e.g., high-reward trials in the current task) and there is only a small chance that the same level of reward will be available on the next trial (there is only a 0.33 probability that the next trial will be a high-reward trial), then care should be taken in how the response is made so that the current high-reward opportunity is not wasted. It was notable that ACC disruption by TUS particularly led to faster responses on medium- and high-reward trials (Figure 7D_i), which meant that animals were less likely to actually receive the medium and high rewards (Figure 7D_ii). In some cases (e.g., Figure S1), animals produced carefully controlled, slow responses on high-reward trials. Danielmeier et al., 2015 have also inferred, from recordings of ACC activity and cholinergic pharmacological investigation, that the ACC might mediate controlled response strategies via the BF. This is consistent with the negative direction of the PPI effect (Figure 6C), consistent with an inhibitory influence between the ACC and plBF when carefully controlled responding was needed on higher reward expectation trials.

Although many aspects of TUS, such as its ability to stimulate deep in the brain while leaving overlying areas unaffected, compare favorably with other minimally invasive stimulation techniques, there are still limits to its precision (in the present study, BF TUS spreads to both active sub-regions). However, although the precision of TUS is likely to improve to allow more specific targeting in the future, it will also be important to relate the findings to activity patterns recorded in specific neuron populations. BF is a major neuromodulatory hub. It is the major source of cholinergic projection neurons to the cortex (Mesulam et al., 1983). Although BF and acetylcholine has been linked to affect, attention, motivation, and memory (Danielmeier et al., 2015, Marshall et al., 2016, Záborszky et al., 2018), our demonstration that BF mediates the influence of past and present context on the emergence of a decision of when to act might seem surprising, especially given that the nigrostriatal dopaminergic pathway has been linked to self-paced action initiation (Howe and Dombeck, 2016, da Silva et al., 2018). We cannot completely rule out the influence of BF TUS on adjacent basal ganglion regions. However, a recent study shows that dopamine and acetylcholine may play independent and complementary roles in initiation of self-paced actions (Howe et al., 2019), and other studies indicate that BF activity is linked to response initiation and suppression (Avila and Lin, 2014, Mayse et al., 2015). Moreover, neurons in the medial BF of monkeys combine various contextual factors, such as reward size and uncertainty (Monosov et al., 2015). Some of these neurons have ramping activity that anticipates the timing of external events (Zhang et al., 2019). Here we suggest a new function for a BF-cingulate circuit in combining contextual factors with internal state to guide decisions about when to act or, equally, when not to act. This finding could be of potential clinical importance, given the involvement of the BF cholinergic system in Parkinson’s disease (Ballinger et al., 2016). Future studies should assess the possible distinctive role of cholinergic and noncholinergic BF neurons in decisions about when to act.

Consideration of previous investigations of nigrostriatal activity in tandem with the current results suggests the hypothesis that the BF integrates past and present contextual information that will influence the decision about when an action should be made and communicates this information to nigrostriatal circuits via direct or indirect pathways known to exist between them (Hikosaka, 2010). It is in the nigrostriatal circuit, or in one of the interconnecting linking regions such as the habenula (Matsumoto and Hikosaka, 2007), that action initiation per se begins.

STAR★Methods

Key Resources Table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Chemicals, Peptides, and Recombinant Proteins
Isoflurane – ISOFLO 250ml	Centaur	30135687
Ketamine – Narketan 10% 10ml INJ CD(SCH4)1 1-MCD	Centaur	03120257
Midazolam – Hypnoval amps 10mg/2ml	Centaur	23191407
Atropine – Atrocare INJ 25ml	Centaur	02500456
Meloxicam – Metacam INJ 10ml 5mg/ml DOGS/CATS	Centaur	02500456
Ranitidine 50mg/2ml x5 INJ	Centaur	30294115
Saline	DPAG, University of Oxford	N/A
Formalin	DPAG, University of Oxford	N/A
SignaGel Electrode Gel	Parker Laboratories	#15-25
Experimental Models: Organisms/Strains
Macaca mulatta, 4 males, between 4-6 years old, between 11.6-14.2 kg, socially housed	MRC, Centre for Macaques	NCBITaxon:9544
Software and Algorithms
MATLAB 2017a	Mathworks	N/A
Presentation	Neurobehavioral systems	N/A
FMRIB Software Library v5.0	FMRIB, WIN, Oxford, UK	N/A
Advanced Normalization Tools	Tustison and Avants, 2013	N/A
Connectome Workbench	The Human Connectome Project and Connectome Coordination Facility	N/A
Magnetic Resonance Comparative Anatomy Toolbox	Neuroecology Lab	https://github.com/neuroecology/MrCat
Offline_SENSE	Windmiller Kolster Scientific	N/A
R	The R Foundation	N/A
Mediation Toolbox	Cognitive Affective Neuroscience Laboratory	https://wagerlab.colorado.edu/tools
Other
Transducer H-115MR 250kHz SN:018	Sonic Concepts	http://sonicconcepts.com
Transducer H-115MR 250kHz SN:017	Sonic Concepts	http://sonicconcepts.com
Amplifier Model 75A250A – 75Watts – 10khz 250MHz	Amplifier Research	https://www.arworld.us/
Tie Pie Handyscope HS5 SN: 32239	Tie Pie	https://www.tiepie.com/en
Brainsight frameless stereotaxic neuronavigation system	Rogue Research	N/A
MRI compatible frame	Crist Instruments	http://www.cristinstrument.com/products/stereotax/stereotax-primate
Four-channel phased-array coil	Windmiller Kolster Scientific	https://www.wkscientific.com/#mri-coils

Lead Contact and Materials Availability

Further information and requests for resources should be directed to and will be fulfilled by the Lead Contact, Nima Khalighinejad (nima.khalighinejad@psy.ox.ac.uk)

Experimental Model and Subject Details

Four male rhesus monkeys (Macaca mulatta) were involved in the experiment. They weighed 11.6–14.2 kg and were 4-6 years of age. They were group housed and kept on a 12 hr light dark cycle, with access to water 12–16 hr on testing days and with free water access on non-testing days. All procedures were conducted under licenses from the United Kingdom (UK) Home Office in accordance with the UK The Animals (Scientific Procedures) Act 1986 and with the European Union guidelines (EU Directive 2010/63/EU).

Method Details

Behavioral Training

Prior to the data acquisition, all animals were trained to work in an MRI compatible chair in a sphinx position that was placed inside a custom mock scanner simulating the MRI scanning environment. In the first stage of training, animals were trained to use custom-made infra-red touch sensors to respond to an image of a frame filled with dots that was presented either on the left or right side of the screen, with their left or right hand, respectively. They got a reward for touching the sensor on the side corresponding to the image within 4 s. The color of the image changed from trial-to-trial and animals learnt that different colors were associated with different levels of reward (drops of juice). In the second stage of training dots were presented gradually, one at a time. Animals learned to withhold their response for a few seconds before responding to increase their chance of getting a reward, given the probabilistic nature of the task. In the last stage of training animals learned to tolerate long action-outcome delays; given the hemodynamic lag in macaques (Chau et al., 2015) this delay allowed disambiguation of neural activity occurring at the time of decision making and at the time of decision outcome. The delay gradually increased from 0.5 s over several sessions to 4 s. The animals underwent aseptic surgery to implant an MRI compatible head post (Rogue Research, Mtl, CA). After a recovery period of at least 4 weeks, the animals were trained to perform the task inside the actual MRI scanner under head fixation. The imaging data acquisition started once they were receiving a reward on more than 50% of the trials (i.e., passing the midpoint of the probability distribution in more than half of the trials), for at least three consecutive sessions in the scanner.

Experimental task

At the beginning of each trial an empty frame (8 × 26 cm) appeared on the left or right side of the screen. The frame gradually filled with dots (round circles, r = 0.3 cm, max number of dots = 25) emerging from top to bottom (Figure 1B). Animals could terminate the trial, at a time of their own choice, by touching a custom-made infra-red touch sensor, on the side corresponding to the image. The trial continued if they touched the opposite side. The probability of getting reward increased as more dots appeared on the screen, following a sigmoid curve (Figure 1D). The probability distribution was drawn from a sigmoid function. The input to the function was a vector corresponding to the number of dots from 1 to 25. The midpoint of the curve was at dot #12 (50% chance of getting reward) with the steepness of 0.5. The probability distribution was constant across the trials and the sessions. The color of the frame and dots varied from trial to trial but remained constant within a trial. The color indicated potential reward magnitude and could be red, green or blue, indicating one, two or three drops of juice, respectively. In addition to the color, the speed of the dots appearance also varied from trial to trial. A new dot appeared every 100, 200 or 300 ms. Animals had the option to respond, any time from the beginning of the trial (appearance of the empty frame) to 300 ms after the frame was filled (appearance of the last dot). If they responded, they were offered drops of juice or no juice, based on the probability distribution at the time of response. There was a delay of 4 s between response and outcome (action-outcome delay). Successful and unsuccessful outcomes were indicated by an upward and downward pointing triangle, respectively. The triangle remained on the screen for 2 s. If rewarded, drops of blackcurrant juice were delivered by a spout placed near the animal’s mouth during scanning. Each drop was composed of 1 mL blackcurrant juice. No juice was delivered when the trial was not rewarded. After the outcome phase, they proceeded to the next trial after a 3, 5 or 7 s inter-trial interval (ITI). ITI varied in blocks of 30 trials in a pseudo-randomized order. Specific patterns on the left and right side of the screen indicated the ITI block (Figure 1C). If animals did not respond by 300ms after the emergence of the last dot, the frame disappeared, and they had to wait for 4 s (equivalent to action-outcome delay) + 3, 5 or 7 s (ITI) for the next trial to start. Animals were given 40min to perform the task at each session. The task finished after 40 min, regardless of the number of trials done. Each animal performed ten to twelve sessions in the MRI scanner. The experiment was controlled by Presentation software (Neurobehavioral Systems Inc., Albany, CA).

Cox regression model

To estimate the deterministic component of actTime we used a specific class of survival models called the Cox proportional hazard model (Murakami et al., 2017). The model predicts time-to-event (actTime) on the current trial from present and past contextual factors. Specifically, the predictors (covariates) included reward magnitude, dot speed and ITI of the current trial, and the actual reward and actTime on the past 10 trials. Importantly, actTime in trials that monkeys decided not to respond are labeled as ‘censored’ data because in those trials monkeys might have responded if the trials were to continue for longer. The model is described as:

$$\lambda \left(t\right)={\lambda }_0\left(t\right).\exp \left(\beta \mathbf{x}\right),$$

where $\lambda \left(t\right)$represents a hazard function (hazard rate of responding), ${\lambda }_0\left(t\right)$represents a baseline hazard function, that is a hazard function when all the covariates are 0, β is a row vector with 23 elements (3 present contextual factors + 10 past rewards + 10 past actTimes) representing Cox coefficients for each covariate and x is a 23 element column vector representing covariates, present contextual factors and contextual factors of the past 10 trials. The coefficients were estimated for each testing session by using the ‘coxphfit’ function in MATLAB. We also investigated an alternative model where we introduced ‘trial number’ as a covariate to account for a potential effect of satiety on actTime; however, the Cox coefficient for trial number was not significantly different from zero. Moreover, its inclusion had a negligible impact on the proportion of variance in actTime explained by the other factors. This may partly reflect the relatively short duration of each testing session.

A detailed method for obtaining Cox coefficients has been previously described (Murakami et al., 2017). The estimated Cox coefficients $\left(\hat{\beta }\right)$ from present and past contextual factors (only coefficients from the immediately preceding trial were used, as only these had been significant for both past reward outcome and past actTime) were used to obtain the expected actTime from the given predictors by the following method:

First, the cumulative hazard function, ${\widehat{\text{Λ}}}_x\left(t\right)$, of each trial was estimated given the baseline cumulative hazard function, ${\widehat{\text{Λ}}}_0\left(t\right)$, and the covariates:

$${\widehat{\text{Λ}}}_x\left(t\right)={\widehat{\text{Λ}}}_0\left(t\right).\exp \left(\hat{\beta }\mathbf{x}\right),$$

The cumulative hazard function of each trial was then used to estimate the survival function of each trial, S(t):

$${\hat{S}}_x\left(t\right)=\exp \left(-{\widehat{\text{Λ}}}_x\left(t\right)\right),$$

The deterministic actTime is estimated by:

$$\left[actTime\right]=\underset{0}{\overset{\infty }{\int }}{\hat{S}}_x\left(t\right),$$

We also separately assessed the contribution of past and present context to deterministic actTime. The original model was split into two, estimating the Cox regression coefficients separately from its present and past components. We then used the Cox coefficients relating to either the present or the immediately preceding trial (similar to the original model) to derive two separate actTime estimates. These new estimates were termed deterministic actTime_{present context} and deterministic actTime_{past context}.

Finally, to measure the proportion of variance explained by the Cox regression model, we used Schemper’s V (Schemper and Henderson, 2000), which is defined as:

$$V=\frac{\left(\hat{D}-{\hat{D}}_x\right)}{\hat{D}},$$

Where $\hat{D}$ is the distance between survival functions of individual trials S_i(t) and a survival function estimated from all the trials without taking into account covariates $\hat{S}\left(t\right)$, by using Kaplan–Meier estimator.${\hat{D}}_x$ is calculated in the same way as $\hat{D}$, but is the distance between survival functions of individual trials S_i(t), and an estimated conditional survival function given covariates x, ${\hat{S}}_x\left(t\right)$. The equations to calculate $\hat{D}$ are previously described in detail (Murakami et al., 2017).

Imaging data acquisition

Awake-animals were head-fixed in a sphinx position in an MRI-compatible chair (Rogue Research, MTL, CA). MRI was collected using a 3T horizontal bore MRI clinical scanner and a four-channel phased array receive coil in conjunction with a radial transmission coil (Windmiller Kolster Scientific Fresno, CA). Each loop of the coil had an 8cm diameter which ensures a good coverage of the animal’s head. Similar coils have been previously used for awake fMRI studies in primates (Chau et al., 2015, Kolster et al., 2014, Papageorgiou et al., 2017). The chair was positioned on the sliding bed of the scanner. The receiver coils were placed on the side of the animal’s head with the transmitter placed on top. The touch sensors and the juice delivery system were the same as the one used in the mock scanner. An MRI-compatible screen (MRC, Cambridge) was placed 30cm in front of the animal and the image was projected on the screen by a LX400 projector (Christie Digital Systems). Functional data were acquired using a gradient-echo T2^∗ echo planar imaging (EPI) sequence with a 1.5 × 1.5 × 1.5 mm resolution, repetition time (TR) 2.28 s, echo time (TE) 30 ms and flip angle 90°. At the end of each session, proton-density-weighted images were acquired using a gradient-refocused echo (GRE) sequence with a 1.5 × 1.5 × 1.5 mm resolution, TR 10 ms, TE 2.52 ms, and flip angle 25°. These images were later used for offline MRI reconstruction. T1-weighted MP-RAGE images with a resolution of 0.5 × 0.5 × 0.5 mm, TR 2.5 s, TE 4.04 ms, inversion pulse time (TI) 1.1 s, and flip angle 8°, were acquired in separate sessions under general anesthesia. Anaesthesia was induced by intramuscular injection of 10 mg/kg ketamine, 0.125-0.25 mg/kg xylazine, and 0.1 mg/kg midazolam and maintained with isoflurane (for details see Sallet et al., 2013). Anaesthesia was only used for collecting T1-weighted structural images.

fMRI data preprocessing

Preprocessing was performed using tools from FMRIB Software Library (FSL) (Jenkinson et al., 2012), Advanced Normalization Tools (ANTs; http://stnava.github.io/ANTs) (Tustison and Avants, 2013), Human Connectome Project Workbench (Glasser et al., 2013) (https://www.humanconnectome.org/software/connectome-workbench), and the Magnetic Resonance Comparative Anatomy Toolbox (MrCat; https://github.com/neuroecology/MrCat). First, T2^∗ EPI images acquired during task performance were reconstructed by an offline-SENSE method that achieved higher signal-to-noise and lower ghost levels than conventional online reconstruction (Kolster et al., 2009) (Offline_SENSE GUI, Windmiller Kolster Scientific, Fresno, CA). A low-noise EPI reference image was created for each session, to which all volumes were non-linearly registered on a slice-by-slice basis along the phase-encoding direction to correct for time-varying distortions in the main magnetic field due to body and limb motion. The aligned and distortion-corrected functional images were then non-linearly registered to each animal’s high-resolution structural images. A group specific template was constructed by registering each animal’s structural image to the CARET macaque F99 space (Kolster et al., 2009). Finally, the functional images were temporally filtered (high-pass temporal filtering, 3-dB cutoff of 100 s) and spatially smoothed (Gaussian spatial smoothing, full-width half maximum of 3mm).

Transcranial Ultrasound Stimulation (TUS)

Ultrasound stimulation was performed using a single element ultrasound transducer with 63.2 mm radius of curvature (H115-MR, diameter 64 mm, Sonic Concept, Bothell, WA, USA). The transducer was coupled with a coupling cone and was filled with degassed water and sealed with a latex membrane (Durex). The ultrasound wave frequency was set to 250 kHz resonance frequency. 30 ms bursts of ultrasound were generated every 100 ms with a digital function generator (Handyscope HS5, TiePie engineering, Sneek, the Netherlands) for a total duration of 40 s. A 75-Watt amplifier (75A250A, Amplifier Research, Souderton, PA) was used to deliver the required power to the transducer. A TiePie probe connected to an oscilloscope was used to monitor the output voltage. The recorded peak-to-peak voltage was constant throughout the stimulation session and ranged from 128 to 136 V. This corresponds to a peak negative pressure of 1.152 to 1.292 MP, respectively, measured in water with an in-house heterodyne interferometer (see reference Constans et al., 2017 for more details about the calibration protocol).

At the beginning of each stimulation session the animal’s skull was shaved and a conductive gel (SignaGel Electrode; Parker Laboratories Inc.) was applied to the skin. The water-filled coupling cone and the gel was used to ensure ultrasonic coupling between the transducer and the animal’s head. Next, the ultrasound transducer / coupling cone montage was placed on the skull and a Brainsight Neuronavigation System (Rogue Research, Montreal, CA) was used to position the montage so that the focal spot would be centered on the targeted brain region. Ad hoc coupling cone were used for each target of interest. All targets were sonicated bilaterally for 80 s in total, with 40 s of stimulation applied to a target in each hemisphere. Sonication of the target in one hemisphere was immediately followed by sonication of a homologous target in the contralateral hemisphere. Hemispheres were sonicated in a pseudo-random order. After stimulation, monkeys were immediately moved to a testing room for behavioral data collection. There were four stimulation conditions (Figure 7C): ACC, BF, parietal cortex and sham. Left and right ACC and BF targets were defined based on the whole-brain peak activity for actTime contrast, projected on each individual monkey’s structural image. We targeted amBF where we had strong bilateral activity; however, as explained in the Results, the stimulation was also associated with activity change in adjacent plBF. Left and right posterior parietal operculum (POp) targets were used as active control stimulation sites. A sham condition was also implemented as a non-stimulation passive control. The sham condition completely matched a typical stimulation session (setting, stimulation procedure, neuro-navigation, targeting, transducer preparation and timing of its bilateral application to the shaved skin on the head of the animal) except that sonication was not triggered. During the sham session the montage was pseudo-randomly positioned to target ACC, BF or POp. Each stimulation condition was repeated five times, on separate days, and the order of the stimulation sessions was pseudo-randomized for each animal. For example, the stimulation schedule for monkey W was POp TUS – sham TUS – ACC TUS – BF TUS, repeated five times, over 40 days. The stimulation was always performed at the same time of the day and there was always a 48 hours gap between each session, regardless of it being a real or sham stimulation session.

The TUS procedure used here, in which a short train of TUS was delivered, has a short-term impact on neural activity and behavior that lasts many minutes to a few hours (Folloni et al., 2019, Fouragnan et al., 2019, Verhagen et al., 2019). This ensures that neither the neural nor the behavioral effects of TUS found here can be attributed to the auditory stimulation that accompanies TUS delivery (Airan and Butts Pauly, 2018, Mohammadjavadi et al., 2019).

Acoustic and thermal modeling

The acoustic wave propagation of our focused ultrasound protocol was simulated using a k-space pseudospectral method-based solver, k-Wave (Cox et al., 2007) to obtain estimates for the pressure amplitude, peak intensity, spatial distribution, and thermal impact at steady state. 3D maps of the skull were extracted from a monkey CT scan (monkey L (Constans et al., 2017), 0.14 mm slice resolution, 0.33 mm slice distance). Soft tissues were assumed to be homogeneous, with acoustic values of water $\left({\rho }_{tissue}=1000\text{kg}.{\text{m}}^{-3}\text{and}{\text{c}}_{tissue}=1500\text{m}.{\text{s}}^{-1}\right)$. In the bone, a linear relationship between the Hounsfield Units (HU) from the CT scan and the sound speed, as well as the density, was used. The power law model for attenuation is ${\text{α}}_{att}={\text{α}}_{min}+{\text{α}}_{max}\ast {\text{Ф}}^{\beta }$ where the porosity $\text{Ф}$ is defined by $\text{Ф}=\left({\rho }_{max}-\rho /{\rho }_{max}-{\rho }_{tissue}\right)$in the skull (Aubry et al., 2003). The attenuation coefficients for the acoustic propagation ${\text{α}}_{min}$ and ${\text{α}}_{max}$depend on the frequency: ${\text{α}}_{min}={\text{α}}_{min0}{f}^{\text{b}}$with ${\text{α}}_{min0}=0.2\text{dB}.{\text{cm}}^{-1}.{\text{MHz}}^{-\text{b}}$ and ${\text{α}}_{max}={\text{α}}_{max0}{f}^{\text{b}}$with ${\text{α}}_{max0}=8\text{dB}.{\text{cm}}^{-1}.{\text{MHz}}^{-\text{b}}$ (Aubry et al., 2003). We set the parameters to ${\rho }_{max}=2200\text{kg}.{\text{m}}^{-3}$, ${\text{c}}_{max}=3100\text{m}.{\text{s}}^{-1}$(Constans et al., 2017), $\beta =0.5$ (Aubry et al., 2003), $\text{b}=1.1$ (Constans et al., 2018). The attenuation coefficient in bone accounts for both absorption and scattering (Pinton et al., 2012).

The propagation simulation was performed at 250 kHz with a 150 μs-long pulse signal (enough to reach a steady state). The transducer was modeled as a spherical section (63.2 mm radius of curvature and 64 mm active diameter). The simulated pulses were spatially apodized (r = 0.35) on the spherical section. Ultrasound propagates first through water before entering the skull cavity with the geometrical focal point located below the surface, inside the brain. Simulations were performed in free water, and the maximum amplitude obtained was used to rescale the results in skull. The thermal modeling is based on the bio-heat equation (Pennes, 1948):

$$\rho \text{C}\frac{\partial \text{T}}{\partial \text{t}}=\text{κ}{\nabla }^2\text{T}+\text{q}+\text{w}{\rho }_b{\text{C}}_{\text{b}}\left(\text{T}-{\text{T}}_{\text{a}}\right)$$

where T, ρ, C, κ and q are the temperature, density, specific heat, thermal conductivity and rate of heat production respectively. Heat production is defined as $\text{q}={\text{α}}_{\text{abs}}\left(\text{P}²/2\text{ρC}\right)$, ${\text{α}}_{\text{abs}}$ being the absorption coefficient and P the peak negative pressure. κ is set to 0.528 W.m^-1.K^-1 in soft tissue and 0.4 W.m^-1.K^-1 in the skull; C is set to 3600 J.kg^-1.K^-1 in soft tissue and 1300 J.kg^-1.K^-1 in the skull (Duck, 2012). In the tissue, the absorption coefficient was set to${\text{α}}_{abstissue}=0.21\text{dB}.{\text{MHz}}^{-\text{b}}$ (Goss et al., 1979). In the skull the longitudinal absorption coefficient is proportional to the density with ${\text{α}}_{absmax}=\left({\text{α}}_0/3\right)=2.7\text{dB}.\text{cm}.{\text{MHz}}^{-\text{b}}$ (Pinton et al., 2012). The last term corresponds to the perfusion process: $\text{w},{\rho }_{b,}{\text{C}}_{\text{b}}$ and ${\text{T}}_{\text{a}}$ correspond to the blood perfusion rate, blood density, blood specific heat and blood ambient temperature respectively. These parameters are assumed homogeneous over the brain, although a more detailed description of the brain cooling processes can be found in the literature (Wang et al., 2016). The perfusion parameters are based on previous reports (Pulkkinen et al., 2011): w = 0.008 s^-1; ρ_b = 1030 kg.m^-3; C_b = 3620 J.kg^-1.K^-1 and T_a = 37°C.

The bioheat equation is solved by using a 3D finite-difference scheme in MATLAB (Mathworks, Natick, USA) with Dirichlet boundary conditions. Initial temperature conditions were 37°C in the brain, skull and tissue, and 24°C in the water coupling cone. Simulations were run over 1 minute pre-sonication, followed by 40 s of sonication, and 5 minutes post-sonication, closely following the experimental procedure.

Resting-state imaging data acquisition, pre-processing, and analysis

The detailed procedure for rs-fMRI acquisition, pre-processing, and analysis has been reported elsewhere (Folloni et al., 2019, Verhagen et al., 2019). In summary, rs-fMRI was collected under inhalational isoflurane gas anesthesia for three monkeys after BF TUS, SMA TUS (Verhagen et al., 2019), and a control state (Verhagen et al., 2019). During the acquisition of rs-fMRI data the mean expired isoflurane concentration was around 1%. Isoflurane was chosen to maintain anesthesia as it has been previously demonstrated to preserve rs-fMRI networks (Mars et al., 2013, Neubert et al., 2015, Sallet et al., 2013, Vincent et al., 2007). Moreover, the impact of the transcranial ultrasound stimulation (TUS) was established by comparing rs-fMRI sessions collected after TUS with sessions without prior TUS but in both cases data were collected under identical conditions of isoflurane anesthesia. Because animals were anaesthetised in both cases it is therefore possible to compare the impact of TUS stimulation on functional connectivity of the stimulated areas. After pre-processing, we used a seed-based correlation analysis approach (Neubert et al., 2015, Sallet et al., 2013) to report the whole-brain functional connectivity of the stimulation site (BF) with and without TUS. Additionally, we reported whole-brain functional connectivity for SMA and POp with BF TUS, SMA TUS and no stimulation (Figure S6). The impact of BF TUS was quantified by regressing, for each point in the brain, the seed-based correlation map observed after BF TUS against the seed-based correlation map observed in the control state (Figure 7B).

Quantification and Statistical Analysis

Behavioral analysis

Time to act (actTime) was defined as the number of dots on the screen at the time of response. We used a multilevel generalized linear model (GLM) to predict actTime from present and past contextual factors, with ‘trial data’ as level one variable, ‘testing session’ as level two and ‘animal’ as level three. The variables ‘testing session’ and ‘animal’ were assigned as the random part of the model, with the variable ‘testing session’ nested within the variable ‘animal’. Note that by-subject random slopes were not included in the multilevel models, because it is suggested that a random variable should have at least 10 levels before one can include random effects for it (sample size in this study was four) (Raudenbush and Bryk, 2001). Maximum likelihood method was used for model estimation and pairwise t test and Tukey’s HSD for post hoc comparisons. We examined the impact of both present and past contextual factors on actTime. Present contextual factors consisted of potential reward magnitude, speed of dots and ITI on the current trial. Past contextual factors consisted of actual reward outcome and actTime on the past trial.

$$actTim{e}_t={\beta }_0+{\beta }_{1}rewar{d}_t+{\beta }_{2}dotSpee{d}_t+{\beta }_{3}IT{I}_t+{\beta }_4\left(rewar{d}_t\ast dotSpee{d}_t\right)+{\beta }_5\left(rewar{d}_t\ast IT{I}_t\right)+{\beta }_6\left(dotSpee{d}_t\ast IT{I}_t\right)+{\beta }_7\left(rewar{d}_t\ast dotSpee{d}_t\ast IT{I}_t\right)+{\beta }_{8}rewardOutcom{e}_{t-1}+{\beta }_{9}actTim{e}_{t-1}$$

Likewise, we used a hierarchical logistic regression to predict the odds of responding at a given trial from contextual factors.

$$logit\left(respons{e}_t\right)={\beta }_0+{\beta }_{1}rewar{d}_t+{\beta }_{2}dotSpee{d}_t+{\beta }_{3}IT{I}_t+{\beta }_{4}rewardOutcom{e}_{t-1}+{\beta }_{5}actTim{e}_{t-1}$$

The modeling was performed by ‘nlme’ and ‘lme4’ packages in R (Bates et al., 2018, Pinheiro et al., 2018).

fMRI data analysis

To perform whole-brain statistical analyses we used a univariate generalized linear model (GLM) framework as implemented in FSL FEAT (Woolrich et al., 2001). At the first level, we constructed a GLM to compute the parameter estimates (PEs) for each regressor. The GLM was constructed based on the linear model we previously used for behavioral analyses:

$$GLM1:BOLD={\beta }_0+{\beta }_{1}resp+{\beta }_{2}reward+{\beta }_{3}dotSpeed+{\beta }_{4}ITI+{\beta }_{5}pastRew+{\beta }_{6}pastactTime+{\beta }_{7}actTime+{\beta }_{8}time+{\beta }_{9}rightconv+{\beta }_{10}leftconv+{\beta }_{11}mainOut+{\beta }_{12}levelOut+{\beta }_{13}rightunconv+{\beta }_{14}leftunconv+{\beta }_{15}mouth,$$

where $BOLD$ is a t x 1 (t time samples) column vector containing the times series data for a given voxel. Trials where animals made no response were left out of the analysis (16.38% ± 11.42% of the trials, across all sessions). Regressors 1 to 7 are task-related regressors of interest: resp is an unmodulated regressor representing the main effect of stimulus presentation in responded trials (all event amplitudes set to one). reward, dotSpeed and ITI are parametric regressors with three levels, which represent reward magnitude, speed of dots, and inter-trial-interval on the current trial, respectively. pastRew is a parametric regressor with four levels representing the reward outcome on the past trial. pastactTime is also parametric and represents actTime on the past trial. pastRew and pastactTime were both weighted by their influence on actTime on the current trial (multiplied by their coefficients from behavioral GLM). actTime represents time-to-act (number of dots at response) on the current trial. Regressors 1 to 7 were all boxcar regressors with a duration of 500 ms that were convolved with a hemodynamic response function (HRF) specific for monkey brains (Kagan et al., 2010, Nakahara et al., 2002). Regressors 1-6 were all time-locked to the onset of the trial. Regressor 7 started 500 ms before animals made a response by cutting the infra-red touch sensor and continued for 500 ms.

Regressors 8-15 were task-related confound regressors. time is a parametric regressor representing the time passed since the beginning of the scanning session and is locked to the trial onset. leftconv and rightconv are unmodulated regressors (all event amplitudes set to one), locked to 500 ms prior to response, representing the response with the left and right hand, respectively. mainOut is an unmodulated regressor representing the main effect of outcome (all event amplitudes set to one). levelOut is a parametric regressor with four levels representing the reward outcome on the current trial. Regressors 11-12 were locked to the onset of outcome (juice) delivery. Regressors 13-15 modeled instant signal distortions due to changes in the magnetic field caused by movement. These regressors were therefore not convolved with HRF. rightunconv and leftunconv represented distortion due to right and left-hand responses. They started at the beginning of the TR when the response was recorded and had a duration of one TR (2.28 s). mouth represented distortion due to mouth movements. It started at the beginning of the TR when the juice delivery started and terminated at the end of the TR when the juice delivery ended. To further reduce variance and noise in the BOLD signal, we also added task-unrelated confounds which included 13 parametric PCA components that describe, for each volume, the warping from that volume to the mean reference image when correcting motion artifacts.

First level analysis was performed on each scanning session (45 sessions in total). The contrast of parameter estimates (COPEs) and variance estimates (VARCOPEs) from each scanning session were then combined in a second-level mixed-effects analysis (FLAME 1+2) treating sessions as random effects. Time series statistical analysis was carried out using FMRIB’s improved linear model with local autocorrelation correction.

ROI analysis

Regions of interest (ROI) were defined as spheres of 3 mm radius, centered at the peak of the activation of a contrast. For time-series analyses, the filtered time-series of each voxel within each ROI was averaged, normalized and up-sampled. The up-sampled data was then epoched in 8 s windows, starting from 2 s before to 6 s after the response time. Time-series GLMs were then fit at each time step of the epoched data, in responded trials, using ordinary least-squares (OLS). We ran the following GLMs:

GLM2.1

$$BOLD={\beta }_0+{\beta }_{1}actTim{e}_{numDot}+{\beta }_{2}time+{\beta }_{3}nontask,$$

where $BOLD$ is a i x t (i trial, t time samples) matrix containing the times series data for a given ROI. actTime_numDot is the number of dots at response. time is the time passed since the beginning of the testing session. nontask is a task-unrelated unmodulated constant regressor.

GLM2.2

$$BOLD={\beta }_0+{\beta }_{1}actTim{e}_{numDot}+{\beta }_{2}actTim{e}_{second}+{\beta }_{3}time+{\beta }_{4}nontask,$$

where actTime_seconds is the log time-to-act in seconds.

GLM2.3

$$BOLD={\beta }_0+{\beta }_{1}deterministic\_actTim{e}_{present+past}+{\beta }_{2}actTim{e}_{numDot}+{\beta }_{3}actTim{e}_{second}+{\beta }_{4}time+{\beta }_{5}nontask,$$

where deterministic_actTime_present+past is the number of dots at response as predicted by the Cox regression model from present and past contextual factors.

GLM2.4

$$BOLD={\beta }_0+{\beta }_{1}deterministic\_actTim{e}_{present}+{\beta }_{2}deterministic\_actTim{e}_{past}+{\beta }_{3}actTim{e}_{numDot}+{\beta }_{4}actTim{e}_{second}+{\beta }_{5}time+{\beta }_{6}nontask,$$

where deterministic_actTime_present is the number of dots at response as predicted by the Cox regression model from present contextual factors and deterministic_actTime_past is the number of dots at response as predicted by the Cox regression model from past contextual factors.

GLM2.5

$$BOL{D}_{ROI}={\beta }_0+{\beta }_{1}BOL{D}_{seed}+{\beta }_{2}deterministic\_actTim{e}_{present+past}+{\beta }_{3}PPI+{\beta }_{4}actTim{e}_{numDot}+{\beta }_{5}actTim{e}_{second}+{\beta }_{6}time+{\beta }_{7}nontask,$$

where $BOL{D}_{ROI}$ is BOLD activity at amBF, $BOL{D}_{seed}$ is BOLD activity at plBF, and PPI is the interaction between $BOL{D}_{seed}$ and deterministic_actTime_present+past.

GLM2.6

$$BOL{D}_{ROI}={\beta }_0+{\beta }_{1}BOL{D}_{seed}+{\beta }_{2}quadReward+{\beta }_{3}PPI+{\beta }_{4}time+{\beta }_{5}nontask,$$

where $BOL{D}_{ROI}$ is BOLD activity at ACC, $BOL{D}_{seed}$ is BOLD activity at plBF, $quadReward$ is the quadratic reward magnitude, and PPI is the interaction between $BOL{D}_{seed}$ and $quadReward$.

Leave one out on time-series group peak signal

Significance testing on time-course data was performed by using a leave-one-out procedure on the group peak signal to avoid potential temporal selection biases. For every session, we calculated the time course of the group mean beta weights of the relevant regressor based on the remaining 44 sessions. We then identified the (positive or negative) group peak of the regressor of interest within the full width of the epoched time course: from 2 s before to 6 s after the response. Next, we took the beta weight of the remaining session at the time of the group peak. We repeated this for all sessions. Therefore, the resulting 45 peak beta weights were selected independently from the time course of each single session. We assessed significance using multilevel ANOVA and t tests on the resulting beta weights. The variables ‘testing session’ and ‘animal’ were assigned as the random part of the model, with the variable ‘testing session’ nested within the variable ‘animal’. Note that to avoid any circularity in analyses, the regressors that first used to identify the locations of ROIs were always used as covariates in subsequent analyses of other signals extracted from those ROIs.

Mediation analysis

To investigate the mechanisms that underlie the relationship between contextual factors and actTime we performed multi-level mediation analysis using the Mediation Toolbox (https://wagerlab.colorado.edu/tools). We asked whether covariance between an initial variable (contextual factors) and an outcome variable (actTime as measured by number of dots) can be explained by a mediator variable (BOLD activity from ROI). Mediation analysis extends the previous univariate model and would allow us to test four effects (Figure 6A): the direct effect of initial variable (e.g., reward) on outcome variable (actTime) (path c); impact of initial variable on brain activity (path a); the impact of brain activity on outcome variable, controlling for initial variable (path b); and mediation of initial variable on outcome variable by a potential mediator (BOLD signal) (path a x b). The mediator variable was defined as the positive peak of the BOLD signal within the full width of the epoched time course for each ROI.

Significance estimates for paths a, b, c, and the mediation path were computed by bootstrapping using 10000 samples with replacement. The only requirement to demonstrate mediation, when using bootstrapping, is a significant indirect effect (path a x b) (Hayes, 2018, Zhao et al., 2010). A significant mediation suggests that including the brain activity in the path between contextual factors and actTime, significantly alters the slope of their relationship. P values were calculated for each ROI from the bootstrap confidence intervals. As in previous analyses, the log time-to-act in seconds and the time passed since the beginning of the session were added to the model as covariates of no interest.

TUS behavioral data analysis

Based on our previous findings, we designed two hypothesis-driven multilevel GLMs to investigate the effect of ACC TUS on the relationship between reward magnitude and observed actTime (GLM3.1), and the effect of BF TUS on the relationship between deterministic and observed actTime (GLM3.2). In both models, the variables ‘stimulation session’ and ‘animal’ were assigned as the random part of the model, with the variable ‘stimulation session’ nested within the variable ‘animal’.

GLM3.1

$$actTim{e}_t={\beta }_0+{\beta }_{1}ROITU{S}_t+{\beta }_{2}rewar{d}_t+{\beta }_3\left(rewar{d}_t\ast ROITU{S}_t\right),$$

Where ROI TUS is the stimulation target and reward is the reward magnitude at trial t. To find out how stimulation targets might differ from each other without inflating type I error rate, ANOVA was followed by planned contrasts where the effect of ACC TUS on the relationship between reward magnitude and observed actTime was compared with the effects of BF, control POp and sham TUS.

GLM3.2

$$actTimebia{s}_t={\beta }_0+{\beta }_{1}ROITU{S}_t,$$

Where ROI TUS is the stimulation target and $actTimebia{s}_t$ is the absolute difference between observed and deterministic actTime at trial t:

$$\left|observedactTim{e}_t-deterministicactTim{e}_t\right|.$$

To estimate deterministic actTime, we first asked whether present and past contextual factors significantly contribute to the model. The Cox regression coefficients were significant for present (reward magnitude, dot speed, and ITI) and past (reward outcomes and past actTimes) contextual factors. However, compared to the first Cox model used to analyze the fMRI data, coefficients from the past trials were significant as far as 8 (past reward outcomes) and 9 (past actTime) trials back. Therefore, to increase the estimation accuracy, we used the Cox regression coefficients from all past 10 trials to estimate deterministic actTime from present and past contextual factors. ANOVA was followed by planned contrasts where the effect of BF TUS on actTime bias was compared with the effects of ACC, control POp and sham TUS.

Data and Code Availability

The data that support the findings of this study and the code to generate the results and the figures are available from the Lead Contact upon request.

Published: December 5, 2019

Supporting Citations

The following references appear in the Supplemental Information: Andrews and Heathcote, 2001, Balota et al., 2008, Cavanagh et al., 2016, Hunt et al., 2015, Kennerley et al., 2009, Lewandowsky and Farrell, 2011, Luce, 1991.