Frontal theta as a mechanism for cognitive control

Abstract

Recent advancements in cognitive neuroscience have afforded a description of neural responses in terms of latent algorithmic operations. However, the adoption of this approach to human scalp EEG has been more limited, despite the ability of this methodology to quantify canonical neuronal processes. Here we provide evidence that theta band activities over the mid-frontal cortex appear to reflect a common computation used for realizing the need for cognitive control. Moreover, by virtue of inherent properties of field oscillations, these theta band processes may be used to communicate this need and subsequently implement such control across disparate brain regions. Frontal theta is thus a compelling candidate mechanism by which emergent processes such as ‘cognitive control’ may be biophysically realized.

The prefrontal cortex allows us to transcend routines and habits in order to make better decisions. But how does it actually ‘do’ this? As cognitive neuroscientists, we need to aim to move beyond descriptive findings and psychological correlates for a better understanding of how the brain underlies the mind. A mechanistic perspective is ideal for addressing how latent neural features underlie emergent psychological constructs.

While the marriage of cognitive neuroscience and formal computational models has been fruitful, findings from human scalp electroencephalography (EEG) are rarely included in major reviews of this field [1–3]. This is a missed opportunity, as EEG is sensitive to the canonical computations that likely underlie emergent psychological constructs[4,5]. In this report we describe recent advancements in the endeavor to define the specific computational roles of neuronal population oscillations in frontal cortex as measured by human EEG. In particular, we focus on cortical theta-band oscillations as a candidate mechanism by which neurons could compute and communicate top down control across broad networks.

Theta reflects active cortical functioning

Primate theta band (~4–8 Hz) activities reflect a more discrete range of activities than the similarly named theta observed in rat hippocampus (~4–12 Hz). In primates, theta is broadly distributed across the brain [6] and appear to reflect active operations of the generative cortex, particularly during high-level cognitive processes such as memory encoding and retrieval, working memory retention, novelty detection, and realizing the need for top-down control [7–10]. While there is a wide array of complex cognitive operations reflected by theta, here we focus on a narrower subset of cognitive control processes characterized by a goal directed bias over habitual responses. We address these control processes in two sequential parts: 1) the realization of the need for control, and 2) ways by which that control may be instantiated. While this former area is becoming increasingly well defined, our understanding of the latter processes remains ripe with possibilities.

Frontal midline theta and the realization of the need for control

The realization of the need for control appears to be conveyed by frontal midline theta (FMθ) activities recorded from sensors overlying medial prefrontal cortex (mPFC). These FMθ activities have largely been quantified as Event-Related Potential (ERP) components that reflect mPFC-related control processes elicited by novel information, conflicting stimulus-response requirements, punishing feedback, and the realization of errors. These potentials are known by varied and sometimes overlapping initialisms; Figure 1 details the most prominent components (N2, FRN, CRN and ERN, see Fig. 1 legend for description of terminology). The scientific history and functional significance of these components have each been recently reviewed [11–13], and they certainly differ on an array of qualities. Here we focus on the overwhelming similarities: each of the eliciting events that evoke these responses share a need for increased cognitive control (novelty, conflict, punishment, and error), and these electrophysiological responses share a common spectral signature in the theta band [9,14–24]. This common theta-band characterization merges with a broader literature that has implicated FMθ power dynamics in cognitive effort[25], working memory [8], and even anxious temperament[26].

Figure 1. A variety of eliciting events is associated with a similar neuroelectrical signature on the scalp.

(A) Traditional event-related potential (ERP) components in the time-domain. N2: an ERP component elicited by novelty or stimulus/response conflict. Feedback Related Negativity (FRN): A similar N2-like component elicited by external feedback signaling that one’s actions were incorrect or yielded a loss. Correct-Related Negativity (CRN): a small, obligatory component evoked by motor responses even when these are correct according to the task, and enhanced by response conflict. Error Related Negativity (ERN): A massive ERP component evoked by motor commission errors. While these ERP components (i.e., peaks and troughs in the signal locked to particular external events and averaged across trials) are related to learning and adaptive control, they represent a small fraction of ongoing neural dynamics. (B) Time-frequency plots show richer spectral dynamics of event-related neuroelectrical activity which allow one to study power following particular events without requiring signals to be phased-locked. Here, significant increases in power to novelty, conflict, punishment and error are outlined in black, revealing a common theta-band feature. (C) Scalp topography of event-related theta activity. The distribution of theta power bursts is consistently maximal over the frontal midline. Data and statistical tests from [9].

While EEG certainly suffers from a lack of spatial specificity, there is compelling evidence from source estimation[11,14,27–30], EEG-informed fMRI[31,32], and invasive recordings in humans and monkeys[33–36] that these FMθ activities are generated by mid-cingulate cortex (MCC) and pre-Supplemental Motor Area (preSMA)(Fig 2a). An endogenously generated motor response or exogenously evoked percept instantiates an obligatory pattern of phase reset and power enhancement in mid-frontal sensors, largely in the theta band [9]. These theta dynamics are thought to act as temporal templates for organizing mid-frontal neuronal processes, which are then enhanced following events indicating a need for increased control [9]. Collectively these observations bolster the theory that FMθ reflects a common mechanism, a lingua franca, for implementing adaptive control in a variety of contexts involving uncertainty about actions and outcomes.

Figure 2. Theta as a biophysical mechanism for organizing local and distal neurocomputational functions.

(A) In humans, mid-frontal theta evoked by errors (here, the ERN) has been localized to MCC on the basis of dipole source modeling (red) and concurrent hemodynamic activity (blue). (B) Theta activity recorded from the rostral cingulate sulcus in rhesus macaques. Recordings were made in a region (shown in red) during performance of an anti-saccade task. Increased theta power on anti vs. pro saccade trials (blue > red traces) was associated with stronger spike-field coupling within the theta rhythm, demonstrating how MCC theta provides a temporal window for coincident neural activities that contribute to adaptive control. (C) Mid-frontal theta is thought to reflect the synchronization of goal-relevant information around critical decision points, such as action selection. In this example, theta activities co-ordinate inputs across cortical areas (arrows), particularly at the trough of the oscillation (grey bars). Action selection is likely to be executed when these sources of choice-relevant information (context, reward, memory, etc.) are successfully integrated (solid arrows). (D) Theta band phase consistency is thought to reflect the instantiation of transient functional networks (purple and green traces). For instance, inter-site theta band phase consistency following signals of the need for control have been observed between sources modeled in mid-cingulate cortex (MCC), lateral PFC (lPFC), motor areas, and sensory (i.e. extrastriate visual) cortex. Theta activity may also implement communications between MCC and the basal ganglia (BG). (a) reproduced from [31] with permission from the Society for Neuroscience; (b) reproduced from [35] with permission from the Proceedings of the National Academy of Sciences and [98] with permission from Cell press; (c) reproduced from [42] with permission from the authors.

Theta phase is a biologically plausible candidate for neuronal computation & communication

We propose that these theta-band similarities not only suggest that these phenomena are aspects of a common high-level process, but they also may indicate how the need for control is biophysically realized and communicated. Time-varying changes in the phase angle reflect population-wide oscillations of neuronal membrane potentials [37]. This synchronization can create temporal windows for segregating cortical populations [38], which can separate information intake and transfer processes [39,40]. Neuronal populations participating in a given frequency perturbation will be more (trough) or less (peak) likely to be excited as a function of the population oscillation, and thus will more likely to interact, exchange information, and modulate synaptic plasticity together [41].

Germane to the current topic, this type of spike-field coherence has been demonstrated in both rat [23] and m cingulate cortex [35], where increased theta power is associated with enhanced coupling between single neuron spikes and the phase of the population theta cycle (Fig 2b). It has previously been proposed that these mid-frontal theta phase-consistent activities could act to organize neural processes during decision points, such as where choice-relevant information is integrated to inform action selection [42], Fig 2c.

The phase-locked dynamics observed in FMΘ signals, and the proposed underlying oscillatory dynamics thereof, also suggest a possible mechanism by which signals of the need for control may entrain other brain networks. Rhythmic excitability has been proposed to instantiate transient functional networks between spatially distal sites [41], Fig 2d. The MCC is strongly interconnected to cortical and subcortical areas in a hub-like manner [43], suggesting that FMΘ signals could entrain disparate neural systems by this theta-band phase dynamic when cognitive control is needed. Indeed, a large-amplitude low-frequency temporal organization scheme may be ideal for organizing activities across large spatial distances [39,44]. Cross-cortical Information transmission could thus function in an emergent manner if phase-locked FMΘ naturally entrains activities in disparate neural systems.

Such theta band phase synchrony between mid-frontal and distal sites has been observed following a variety of FMθ signals of the need for control (Fig 3). In addition to eleven replications of this effect in humans [14–24], a similar pattern of theta-band phase synchrony has recently been observed in intracranial recordings from monkeys[36]. It remains an active question to determine the direction of information transfer during the instantiation of control, as cingulo-distal[19,36,45,46], distal-cingulate[47–49], and bi-directional[36] information transfers have all been described.

Figure 3. Theta band phase consistency between mid-frontal and lateral sites is transiently increased following events that indicate a need for control.

Eleven separate studies (A–K) have replicated the finding of theta-band phase synchrony between mid-frontal sites and varied cortical areas, including lateral prefrontal cortex (presumably for goal or attention reorientation), motor cortex (presumably to alter motor threshold), and sensory cortices (presumably to boost sensory gain). Legend: error feedback is punishment; there have been no studies of cue-locked error signals or feedback-locked conflict. Citations: A[14], B[15], C[16] D[17], E[18], F[19] G[20] H[21], I[22], J[23], K[24].

Potential roles of theta in the instantiation of control

It is becoming increasingly clear that these FMθ activities reflect uncertainty in varied circumstances (see Box 1). Given that the mPFC is sensitive to varied circumstances indicating a need for control [67], it should be expected that this system commonly reacts to novelty, conflict, punishment and error, each of which indicate a need for enhanced control processes in order to adaptively change behavior. It is thus important to consider whether this theta signal acts to communicate specific information to inform distal controllers, or if it functions as a generic ‘alarm’ signal without detailed information content per se.

Box 1. Surprise as a quantum of punctate uncertainty.

Major theories of neocortex suggest that it constantly learns from experience, and violations of learned expectations are expressed as prediction errors [50–52] which are used to enhance future predictability or to minimize free energy [53]. These prediction error signals thus serve as a teaching signal and an alarm of the need for network-wide adaptation[53,54]. Indeed, dynamic programming models of adaptive control require learning and acting to occur simultaneously [55]. The emergence of late ERP components is thought to represent transient expressions of this type of prediction error [56–58], encompassing traditional ERP-relevant elicitations of stimulus novelty, probability, entropy, surprise, mismatch or salience. These terms all reflect alterations in circumstantial mismatch processes, and they share a common feature in the realization of uncertainty.

Different theoretical reference frames for various FMΘ signals also share broadly common algorithmic quantification of information quality. In fact, each can be interpreted as a type of free energy or uncertainty to be reduced. The reinforcement learning theory of ERN and FRN originally proposed that these signals reflect a punishment prediction error [59], although increasing evidence has suggested these signals are actually unsigned prediction errors, implying that they reflect surprise regardless of whether the outcome is good or bad [32,60,61] (see Box 2). The conflict monitoring theory of the ERN and N2 suggest that these signals do not reflect prediction error per se, but rather response conflict in the form of Hopfield energy (the degree of co-activation of competing states) [29,62,63]. However, other authors have quantified conflict using entropy [64], or as a change in expectation at the decision level (i.e. from an initial prepotent response to a later controlled response) [65], all of which are directly translatable to a form of surprise. Thus, while each of these theoretical reference frames has explanatory advantages and drawbacks, they have much in common with each other, and even more in common with broader theories of the functional organization of neocortical processes.

In sum, prediction errors are a common neurocomputational currency that has specific representational content depending on the generative neural system [66], and they appear to be reflected by event-related EEG dynamics. Given that the FMθ surprise signals observed over mPFC are strongly influenced by the functional demands of the generative system, they appear to provide a succinct reflection of basic mPFC functions during adaptive control.

The precise and stereotyped nature of the FMθ response to endogenous and exogenous events suggests that the canonical phase-consistent templates (e.g. Fig 1a) would enable efficient information encoding, and thus may facilitate information transfer via synchronous inter-site phase relationships (e.g. Fig 2d). Low frequencies like theta have been found to act as a temporal template to carry lower power, higher information content signals such as gamma band activities via cross-frequency coupling [68–71]. In fact, gamma band correlations have been observed between cingulate and lateral frontal sites following events signaling the need for control [46]. While the theta phase duration may be too long to ideally facilitate direct Hebbian plasticity [69], it may underlie other aspects of information representation. As described in Fig 2c, theta may facilitate recurrent cycles of integration across multiple inputs (context, reward, memory, etc.) to inform controlled action selection, particularly within the cingulate hub[42]. For example, local theta spike-field coherence has been observed during long distance fronto-occipital theta synchrony thought to underlie working memory maintenance [72]. Other studies have shown how hippocampal-cingulate theta synchrony facilitates information transmission during controlled action selection [47,48], even eliciting phase procession of spike timing within the mPFC [49].

However, conflict does not preferentially modulate theta signals that are phase-locked to the conflict-eliciting stimulus, but rather appear to modulate the amplitude of induced theta oscillations [15,73]. Thus, synchronous phase relationships across frontal areas may not necessarily reflect specific information transfer related to the conflict-eliciting stimulus, but rather a more generic process such as gain adjustment [74,75], similar to that which has been posited for frontal cortical neuromodulators such norepinephrine[76]. Cingulate-influenced gain adjustment via induced local inhibition has been proposed to enhance effortful representation of a context or set shift between lateral prefrontal (lPFC) areas [74] and boost top-down influence of fronto-polar areas over lPFC [77], and it could also be utilized to sharpen neural precision for selective attention in sensory areas [78]. Indeed, distal control over local inhibition is a primary candidate for the induction of a synchronous phase relationship [40,75], suggesting that an array of information processing capabilities could be facilitated by basic structural relationships between mPFC and other brain areas. In sum, while evoked FMθ activities appear to be well structured to represent information, it is not known if this information is passed to other brain areas for adaptive control or if a simple alarm signal is used to entrain and override distal operations.

What to do with a surprise signal?

Here we describe some ways by which mPFC-generated surprise signals lead to task-specific adjustments in control (Fig 4). FMθ is sensitive to both unexpected uncertainty (volatility) and expected uncertainty (risk)[13], suggesting that it can serve as both a teaching signal and an alarm of the need for control. This observation suggests that the information content of the signal, at least as measured on the human scalp, is likely to be minimal. Yet even a simple signal of uncertainty can lead to a variety of adaptive adjustments[66,84]. This is particularly true when such surprise signals are generated from a neural system that acts as a hub for shifting attention and behavioral selection[43,54,67].

Figure 4. Algorithmic models of learning and decision making, and their potential relationships to theta band signals reflecting the need for control.

(A) Reinforcement (reward and punishment) learning can be modeled by a variety of similar algorithmic approaches. Shown here is a cartoon example of Q-Learning[99] during a probabilistic learning task[100]. The difference between expected and actual reward is calculated as a reward prediction error conveying whether events are better or worse than expected. These reward prediction errors are then used to adjust future expectations, scaled by a learning rate. (B) A common model of two-alternative forced choice is the Drift Diffusion Model (DDM), shown here[93]. Black lines indicate the accumulated evidence trace (drift rates) for one decision option over another across multiple example trials that grow towards one of two boundaries (decision thresholds), defining when a decision is made. (C) Punishment-induced FRN/FMθ power correlates with the prediction error (shown in (A))[60]. While many investigations have found stronger relationships between FRN/FMθ and worse-than-expected outcomes, more detailed investigations have revealed that even better-than-expected outcomes can also linearly relate to FRN/FMθ power, suggesting that much of this relationship is predicated on the need for change rather than the valence of the feedback per se. However, punishment may be associated with an overall larger response (i.e. higher intercept)[60,82] (D) Response conflict is not only greater during difficult perceptual-performance tasks (such as the Stroop, flanker or Simon task) but also as a function of uncertainty when choosing options with probabilistically different reinforcement rates[100]. This type of uncertainty can be quantified by estimating, for example, the Q-values in (A) as belief distributions with means (expected value) but also variance (estimation uncertainty). During dynamic foraging, the degree of theta response to high uncertainty can predict exploration. [60] (E) Stimulus-induced conflict not only signals a need for increased control (larger N2/FMθ), but this theta signal is related to a transiently increased decision threshold on a trial-by-trial level, effectively linking conflict-induced theta power to enhanced response caution (longer RT, more accurate at avoiding mistakes)[94].

Surprise can alter the learning rate to reduce volatility

While prediction errors can act as a basic learning signal, they can also help determine how much should be learned from the environment. This varied role of prediction errors is a consequence of being a common neural currency: signed reward prediction errors directly inform whether to reinforce or punish behavior [79] whereas unsigned prediction errors may indicate the degree of impact any given surprise should have on future predictions [80], see Box 2. This latter process is known as a learning rate (Fig 4a). The MCC and surrounding medial cortex are often specifically implicated in the adjustment of the effective learning rate during trial-and-error learning [85,86]. While it is abundantly clear that FMθ relates to the degree of surprise [32,60,61] (Fig 4c), it is unknown whether this signal is a reflection of volatility-influenced learning rate. If so, this may further specify a top-down (control-related) role of these signals instead of a commonly assumed bottom-up (midbrain dopamine learning-related) role[59].

Box 2. Signed vs. Unsigned Prediction Errors.

Differences between expectations and outcomes can come in many forms. If a system codes the degree of expectation violation and what to do about it (i.e. it was good or bad), then that system is proposed to code a reward prediction error (AKA a signed or Rescorla-Wagner prediction error)[79]. Some midbrain dopaminergic nuclei are proposed to signal such signed prediction errors, with firing rates scaling with the unexpectedness of a better outcome and pauses in baseline firing scaling with the unexpectedness of a worse outcome. If a system only codes the degree of expectation-outcome difference, that information quality is called simple surprise (AKA an unsigned or Pearce-Hall prediction error)[80]. Much of the rest of the brain appears to code simple surprise [50–53,56].

A signed prediction error is a special case of surprise and thus it requires a larger burden for empirical support. A signal that functions as a signed prediction error needs to conform to axiomatic criteria [81] and function in this manner in all cases (or else this signal would be unreliable and uninterpretable). While early studies of FMθ/FRN supported initial predictions [59] of a punishment or ‘negative reward’ prediction error, many of these studies were performed using tasks with a win-stay/lose-switch response requirement that confounds outcome valence and the need for behavioral adjustment. When tested with more complex tasks without clear win-stay/lose-switch requirements, FMθ/FRN amplitudes scale with unsigned surprise [32,60,61]. Given this violation of axiomatic criteria, it is clear that FMθ/FRN signals do not code for the more specific information quality of signed surprise.

Yet, the brain is a complex system and caveats and complexities are bound to challenge simple mechanistic hypotheses. Punishments appear to be associated with an overall larger response in MCC [60,82], which when combined with a surprise signal could be used to inform a downstream integration of negative prediction error. It has also been proposed that the phase-consistent and phase-varying aspects of FMθ may differentially contribute to signed vs. unsigned information qualities[83]. The absence of majority consensus in this field is likely to be overcome as empirical studies directly test these novel hypotheses.

Surprise leads to a shift in behavioral strategy

Surprise signals can indicate a need for higher-level cognitive control over action selection. In simple tasks, both FMθ and prediction error have been shown to predict subsequent behavioral switching [21,30]. Yet behavior is not always diagnostic of implemented control: during more complicated higher-level learning environments, it may be adaptive to weather temporary bad storms to stick with the best alternative. For example, FMθ does not predict switching during a probabilistic reversal learning task [87]. This suggests that FMθ reflects prediction error but does not predict overt policy adjustments. In other words, it indicates that something needs to be done but does not necessarily indicate what should be done. Given these difficulties in assessing latent features of control based on behavior, computational modeling has been utilized to reveal how FMθ predicts the propensity for enhanced instrumental control over pre-potent actions both between and within subjects [60,88], Fig. 4d.

Surprise indicates the need for performance adjustment

Surprise can also indicate the need for goal adjustment, attention re-alignment, or cautious behavioral restraint. While surprising events often initially elicit an orienting response characterized by motor inhibition[89,90], the features of subsequent deliberative performance adjustments have just begun to be understood[91]. One common example is observed following response errors, where the amplitude of the FMθ signal reliably predicts slower post-error response times[92]. Such slowing may not just reflect a simple orienting response; this phenomenon may also indicate a deliberative increase in response caution. The psychophysical outcome of this strategic adaptation appears as a shift in the speed/accuracy tradeoff with longer RTs for better accuracy. Such a shift is accounted for by an increase in the decision threshold as defined by latent models of forced choice decision making [93] (Fig 4b). FMθ has been shown to relate on a trial-by-trial level with such a conflict-induced adjustment of decision threshold, particularly mediated by the downstream subthalamic nucleus [94] (Fig 4e). Yet if FMθ functions as a non-specific alarm signal, it may also elicit different changes in other circumstances. For example, FMθ could relate to a honed adjustment of sensory evidence causing shorter RTs and better accuracy, which may be accounted for by an increase in the orthogonal latent quantity of drift rate (see Fig 4b).

Caveats for such a broad description

Any description of mPFC processes is bound to be complicated by the high base rate of activation in areas such as MCC across experimental demands [95]. It should be expected that some mPFC processes are not reflected by FMΘ, and that some FMΘ processes do not necessarily involve a phasic response to uncertainty. Moreover, other frequency bands have been shown to play a role in the implementation of control [19,22,90]. It remains an important goal to specify the role of frontal theta in relation to these other frequency bands. Here we advance the modest proposal that the class of ERP/FMθ signals commonly used to investigate action monitoring, cognitive control, and reinforcement learning (Fig 1) share a common feature in the realization of uncertainty and the communication of the subsequent need for enhanced control.

Summary

Even a simple surprise signal can be used to communicate many different things. If the mPFC responds to unsigned prediction errors using a theta-band process capable of inter-site entrainment, this would provide a plausible mechanism by which surprise could influence action selection, shift attention, cautiously adjust behavior, and enhance sensory precision. Most compellingly, such seemingly complex interactions may emerge simply by virtue of the connectivity and timing of biophysical processes facilitated by a common theta-band rhythm.

Box. Outstanding questions.

• Are frontal theta signals a common mechanism for invoking a punctate shift from prepotent (e.g. habitual, model-free, striatal) to deliberative (e.g. goal-directed, model-based, prefrontal) control over action selection?

• What is the directionality and information content of theta phase-synchronous relationships between brain areas during the need for control?

• To what degree are these same low-frequency frontal coupling phenomena present in other mammals during similar cognitive operations [23,36]?

• Do different spectral frequencies represent different types of information qualities during the need for control[22]? Is the specific modulated frequency also dependent on the temporal stage of increasingly evolving mismatch operations throughout the cortex[96]?

• Does the FMθ unsigned (Pearce-Hall) prediction error signal scale the use of a signed (Rescorla-Wagner) dopaminergic prediction error signal (i.e. does it reflect a dynamic learning rate)?

• Do mPFC and STN communicate using low-frequency oscillatory processes akin to those observed in cortex?

• Why is there a strong relationship between FMθ signals and dispositional anxiety[92,97]? Are these alarm signals inherently aversive to some degree (i.e. a negativity bias)?

• Are mPFC representations of negativity bias and sensitivity to surprise [60,82] combined anywhere to signal a signed negative prediction error, or is this conjunction simply task-dependent?

• Frontal theta is a candidate biophysical mechanism for cognitive control

• Frontal midline theta reflects a canonical computation of the need for control

• The implementation of control may emerge from inter-site theta phase synchrony

• The information content of frontal theta can be derived using computational models