Abstract task representations for inference and control

Abstract

Behavioral flexibility depends on our capacity to build and leverage abstract knowledge about tasks. Recently, two separate lines of research have implicated distinct brain networks in representing abstract task information: a fronto-parietal cortical network, and a network involving the medial temporal lobes (MTL), medial prefrontal, and orbitofrontal cortex (OMPFC). These observations have mostly been made in parallel, with little attempt to understand their relationship. Here, we hypothesize that abstract task representations in these networks differ primarily in format, not content. Namely, that the MTL-OMPFC network maintains task knowledge in a flexible cognitive map, while the fronto-parietal network formats this knowledge as productions that facilitate action selection. We discuss novel implications and predictions for behavioral flexibility arising from this hypothesis.

Behavioral flexibility and abstraction

Humans are capable of remarkable behavioral flexibility in new situations. For example, when we arrive in a foreign city, most of us can adapt almost immediately. We can navigate crossings when traffic comes from unfamiliar directions, or signals are different than what we know. Further, we can manage these adaptations on the fly, without extensive feedback or trial and error learning. Indeed, many tasks, like crossing the street, might not tolerate even one error.

This capacity for rapid behavioral adaptation in seemingly limitless settings remains unmatched by any artificial system. Artificial intelligences can now outperform humans on many skilled tasks after exhaustive training, but still struggle to generalize these skills to novel settings at human speeds [1,2]. It is thought that humans succeed where artificial intelligence fails because of our capacity to identify abstract relationships that can be leveraged to transfer or separate learning between settings [3–5], and to plan future actions in an abstract task-space [6]. Conversely, disruptions of behavioral and cognitive flexibility are hallmark symptoms of several neuropsychiatric illnesses like obsessive compulsive disorder, autism, and Parkinson’s disease, causing profound disruptions of daily functioning [7–9].

The basic and clinical scientific interest in behavioral flexibility, combined with recent advances in computational tools and increasingly explicit cognitive neuroscience theory, has brought forth a torrent of new research into the neural representations and circuits that support abstract task representations. Two streams of work have emerged in parallel from this surge, examining related questions but focusing on different networks of brain regions. One line of work has focused on explaining behaviors in tasks tapping learning and memory via representations of abstract task knowledge. This research has mostly implicated a highly interconnected set of regions that include the medial temporal lobes (MTL), orbitofrontal (OFC) and medial prefrontal cortex (MPFC) and closely connected parts of posterior medial parietal cortex – which we will refer to here as the MTL-OMPFC network for short [10–16]. The other line of research has focused on the learning and use of abstract task representations, usually during cognitive control, implicating a network that includes dorsolateral PFC (DLPFC) and lateral parietal cortex, along with the basal ganglia (i.e. the ‘fronto-parietal’ network [17–19]). However, to date, the field has mostly not considered why abstract task representations might be seen in distinct networks under different circumstances. Here, we focus directly on this question and review the behavioral and cognitive conditions under which abstract task representations are associated with these networks. Based on this literature, we propose a novel hypothesis for the functional role of abstract task representations in these different, interacting networks, focusing on a distinction in format rather than content.

What is an abstract task representation?

Abstract task representations have been cited with different nomenclature and definitions in a wide range of research domains where knowledge that permits processes of generalization and inference can explain behavioral and neural data. In this review, we use this term to mean neural codes that organize task conditions (i.e. a particular combination of contexts, stimuli and available actions) based on their similarity on a set of latent (i.e. not directly observable) features held in memory. As such, we do not mean representations of observable perceptual features or dimensions of a stimulus, though these might also involve a kind of abstraction (e.g. the perceptual similarity of different faces). Rather, we refer to a relationship among these features must be inferred from observations and associations held in memory. This definition includes both the unobservable states of a task, as well as conditional actions to be taken in those states, or even more abstract action plans.

For example, during the COVID-19 pandemic, many of us communicated with others almost entirely using one online platform. As a result, we repeatedly encountered task conditions that had highly similar stimuli in an almost unvarying context when we met with the same people in the same virtual environment. These common features often held whether it was a social or professional occasion. Yet, despite the close physical similarity of these encounters, we could reference information stored in memory (e.g. the task of a lab meeting or holiday party) – and infer how to behave with other people using the abstract task representation of social versus professional activity. Doing so, helped us enact appropriate behaviors in each situation. However, in other cases, there may have been observable signals that directly signaled the current context (e.g. logging onto Zoom and seeing a family member instead of a co-worker). In these situations, we could retrieve an appropriate set of behaviors without needing an abstract task representation – as there was an observable cue indicating a social scenario. The key advantage of abstract task representations is that they permit selection and generalization of goal-consistent behaviors and cognitive operations when prior direct experience and current sensory observations are insufficient to do so.

Neural systems representing abstract task structure

With this definition of abstract task representations in mind, we can review current evidence for these representations in both the MTL-OMPFC and fronto-parietal networks.

Abstract task representations in the medial temporal lobe and orbitofrontal cortex

Classic experiments in rodent models have provided evidence that the hippocampus and entorhinal cortex represent allocentric cognitive maps of physical space [20–22]. In the past few decades, rodent neurophysiology and human functional imaging have supported an expanded view of this cognitive map hypothesis, whereby the hippocampus represents associative relationships between objects [12,23–25], contextual dependencies that comprise the structure of a task [26,27], and the relations between items along conceptual dimensions retained in memory (e.g. learned social information about individuals; Figure 1a) [28–30]. Similarly, recent fMRI work has suggested that the entorhinal cortex and medial PFC maintain a map of abstract task information using a grid-like coding scheme [31,32], like that used in spatial navigation ([22]; though see [33]). While debate has remained about the interpretation of these studies [34], the broader findings support an expanded view of MTL function that connects the cognitive map hypothesis to a long-established role in episodic memory [35]. Specifically, the hippocampus is proposed to represent memories in an abstract relational space with multiple dimensions including physical space, time and associations (e.g. the names and places of specific subway stations) [11,36], while the entorhinal cortex encodes the latent organizational structure of this relational map (e.g. the overall layout of the subway system) [13,14].

Figure 1. Schematic of common experimental operational definitions of abstract task representations.

Circles represent task conditions; lines indicate relationships between conditions. Some relationships (solid black lines) are learned through direct observation, other relationships are inferred (red dashed lines). a. Relational mapping task where the relationships between pairs of task conditions along multiple dimensions or attributes is directly learned (e.g. A-B, B-C), which provide information to infer relationships that were not observed (e.g. A-C). b. Clustering of two separate triads of task conditions in a simplified associative inference task where direct associations between pairs of items (e.g. A-B, B-C) can be used to infer hidden relationships within a triad (e.g. A-C). c. Acquired equivalence task where shared associations between two conditions (e.g. A1-X and A2-X) link these conditions together so that new associations learned in one condition (e.g. A1-Y) can be transferred to the other condition by inference (e.g. A2-Y). d. A reversal learning task where visually identical task conditions in time are separated by a latent variable based on reward history (plusses and minuses).

The MTL interacts with a broader cortical network, and recent work has suggested that members of this network, such as the OFC, also represent a cognitive map of task-space [10]. Specifically, this hypothesis proposes that the OFC holds a relational map of latent task states that support inferences about upcoming conditions and the outcomes. This idea is supported by the observation that inactivation or lesioning of OFC in rodents compromises the ability to infer values based on higher-order relationships between task conditions [37,38]. These findings are complemented by human functional imaging studies that successfully decode information about the latent task states needed for task execution from multivoxel activity in the OFC [39–41]. Conceptually related to these observations, studies of humans with focal brain lesions, rodent electrophysiology, and fMRI have implicated the OFC and vmPFC in representing schema knowledge that facilitates learning, memory and decision-making [42–45]. OMPFC is thus thought to have an important role in representing higher-order structure about tasks and the environment.

The MTL-OMPFC network has not only been implicated in representing abstract task structure, but also leveraging that structure to support inferences about new relationships. Several fMRI studies have found evidence for the involvement of hippocampus and vmPFC in forming abstract concept representations in tasks where participants learn relationships between combinations of stimulus features and categories [46–48]. Functional imaging studies have also observed activation of the hippocampus and vmPFC in associative inference tasks where participants infer unseen relationships implied by direct associations [49,50] (Figure 1b). These same regions have been also been implicated in acquired equivalence tasks [51,52] (Figure 1c), suggesting a general involvement in learning abstract task knowledge from structured associations.

Lesion studies across species are mostly consistent with these observations from neuroimaging, and provide evidence of a causal role of these regions. In particular, damage to OFC, vmPFC and hippocampus in humans impairs the ability to infer latent relationships within associative inference tasks, while mostly sparing the ability to learn direct associations [53–55]. Entorhinal and medial PFC lesions – but not hippocampal lesions – impair acquired equivalence learning in rodents [56,57]. Thus, converging evidence across multiple species and paradigms support the hypothesis that MTL and OMPFC represent abstract task structure and use it to draw inferences about previously unobserved relationships.

Abstract task representations in fronto-parietal cortex

To test how the brain represents such novel inferred relationships when needed to perform a task, we adapted an acquired equivalence task for fMRI [58]. In this experiment, multiple contexts could be clustered together based on their shared category-value associations – indicating that they belonged to the same latent state. We trained participants on a subset of these contexts paired with a new set of category-value associations and then tested participants on these new associations in the held-out contexts without any feedback. Importantly, participants had to use what they had already learned about the abstract relationship between contexts that shared category-value associations to transfer the newly-learned category values to the held-out set, without any new feedback from which to learn. Using representational similarity analysis (RSA), we examined where multivoxel fMRI activity was more similar in conditions that belonged to the same cluster in this latent task space.

We had hypothesized that this representation of abstract task structure would be present in the OFC and hippocampus, given the association of these circuits with acquired equivalence and forming a cognitive map of task space [10,51,56,57,59]. However, we instead found the strongest evidence for this representation in a wide network that included mid-DLPFC and rostrolateral PFC (RLPFC), the inferior parietal lobule and the dorsal precuneus, substantially overlapping with fronto-parietal control networks that have been associated with higher-order hierarchical cognitive control tasks [60,61]. In contrast, we found limited evidence of an abstract task representation in OFC.

The involvement of the fronto-parietal cortex in representing abstract task representations is not entirely unexpected when considered in the context of the broader cognitive control literature. Several experiments have found evidence for the involvement of a similar fronto-parietal network in the representation of rules that can organize action selection. These studies include cognitive control experiments wherein the selection of a correct action depends on abstract, hierarchically organized conditional variables that may be directly instructed [62–64] or inferred from experience [61,65–68]. Likewise, these cortical regions have been implicated in representing rule and task information that generalizes across trial-types [69,70]. These observations have led to the proposal that this network maintains abstract task representations needed for controlling behavior [71–73].

Beyond these general associations of fronto-parietal systems with abstraction, we and others have further proposed that this network may by organized to implement hierarchical cognitive control [17,18,74,75]. Specifically, some evidence indicates that the frontal components of these networks are organized rostro-caudally, with subnetworks including RLPFC and mid-DLPFC supporting abstract task information and more concrete task information represented in premotor regions (i.e. context and sensory-motor information). These regions also modulate each other, and parietal cortex, in a roughly hierarchical manner, and control activation of task representations via cortico-striatal loops [64,76–78]. Thus, representations maintained in fronto-parietal networks are thought to provide a basis for controlling behavior and cognition based on internal abstract task contingencies, such that actions are not only guided by directly learned, or innate responses to external stimuli [73].

Several lines of evidence have also pointed to the involvement of fronto-parietal networks in conceptual reasoning [79,80]. Recent RSA studies have found that inferior parietal cortex, and lateral PFC encode abstract task knowledge about the relations between people and items in conceptual spaces that support novel inferences [81,82], and represent analogical relationships during reasoning tasks [83,84]. Lesions to left RLPFC also affect analogical reasoning [85], indicating a necessary role in these processes.

Thus, the MTL-OMPFC and fronto-parietal cortex have both been implicated in maintaining abstract task knowledge and making inferences based on analogical relationships. These parallel lines of observation raise several open questions: Why is this information maintained in multiple brain networks? Under what circumstances do these networks engage such representations? And to what extent, if any, do they differ in their functional contributions to cognition and behavior? The broader literature regarding these networks offers clues to answering these questions.

Formats for knowledge and action

We propose that the contrasting evidence for the involvement of these different brain networks in the representation of similar abstract task structures should be understood mostly in terms of their format rather than their content. It has been long recognized in cognitive science theory, inspired partly by computer science, that the content of a knowledge representation does not determine its format [86,87]. For example, on a computer, a block of text could be formatted in a way that is easy to search for a specific word (i.e. as a string), or in a more inconvenient way (e.g. as an image). In other words, the same information can be re-represented in different formats that are optimized for different uses. Applied to the present question, we hypothesize that both networks represent similar abstract task knowledge. However, the MTL-OMPFC network organizes this knowledge in a map-like format that allows an organism to evaluate its position in an abstract task space both presently and projected into the future or past. In contrast, fronto-parietal cortex organizes abstract task knowledge in terms of production rules, that relate states to contingent actions taken in that state (i.e., if/then contingencies) and so are used for actively guiding thought and behavior (Figure 2).

Figure 2. Schematic summary of hypothesized relationship between task knowledge and control processes with regions of the cortex.

Above are two gradients of the proposed functions of these networks in representing abstract task knowledge (blue-cyan gradient) versus productions (red-green gradient), organized by their degree of abstraction. These colors are overlaid over cortical territories of focus in this review according to suggestions about their respective functions in this hypothesis. Around the perimeter of this color map are written examples for each of these representations in a single novel, unpracticed task (making strawberry tofu). As these examples illustrate, the same concrete or abstract task content can be formatted as relational knowledge about task states and associated actions, or in conditional statements mapping states to actions. Mid-DLPFC, mid-dorsolateral prefrontal cortex; OFC, orbitofrontal cortex; RLPFC, rostrolateral prefrontal cortex; vmPFC, ventromedial prefrontal cortex.

Our hypothesis is rooted in the classic “knowledge-action dissociation”, which refers to the striking observation that people with frontal lobe damage can often articulate the rules and structure of a task and can have a strong desire to succeed, but can nonetheless have great difficulty actually enacting the very rules they can express and want to follow [88–91]. Rather than abolishing the capacity to understand and interpret complex abstract task rules, damage to fronto-parietal control networks particularly impairs the implementation of actions according to abstract rules and their strategic organization [62,92,93].

These observations have been taken as evidence that fronto-parietal networks may not be crucial for understanding abstract information needed to complete a task, but in conditionally selecting actions informed by this abstract task knowledge. Indeed, individuals with lateral frontal lobe lesions exhibit an intact understanding of conceptual semantic relationships [94], though they are less likely to use this semantic knowledge to strategically organize information in memory to aid recall [95]. And famously, in the Wisconsin Card Sorting Task, individuals with lateral frontal lobe damage perseverate using the previously rewarded rule following a switch. However, they do so while also expressing the requirements of the task and awareness that they are accumulating errors, a paradox which Brenda Milner called a “curious dissociation” [90].

In contrast, lesions to MTL, including the hippocampus and entorhinal cortex, do not generally affect performance on cognitive control paradigms that require following abstract rules or switching between them, such as the Wisconsin Card Sorting Task [96–98]. Damage to OMPFC likewise spares performance in basic rule learning or switching tasks in humans and non-human primates with focal lesions, whereas lateral PFC damage impairs performance [99,100]. While individuals with OMPFC damage show many social and decision-related impairments that could be described as failures of control [101], these impairments differ from those with lateral frontal damage in that they generally do not depend on following and switching between abstract, instructed rules.

However, tasks that require switching between option-values that vary according to a latent contextual variable may depend on the MTL-OMPFC network instead. In particular, damage to the OFC impairs reversal learning across species (reviewed here: [102]; Figure 1d). Unlike the WCST where rules are reinforced and switched, these tasks involve reinforcing a specific stimulus or spatial location before changing the reinforced option abruptly without warning. Classically, OFC damage causes worse performance after a reversal while leaving initial discrimination learning intact [103,104] (though see [105] for questions about the neuroanatomical basis of this finding). This deficit has been interpreted as a failure to separate reversal and initial learning of stimulus-value associations into separate latent task states – selectively slowing learning in the reversal phase [10]. Early non-human primate lesion work also found that MTL lesions impaired performance in spatial, but not object-based, reversal learning tasks [106]. A recent study observed that while healthy participants become faster in switching their behavior and learn to anticipate reversals, patients with MTL epilepsy do not show any of these behavioral improvements, though their performance on earlier reversals was comparable [107]. These findings suggest a role for the MTL in learn the higher-order structure of the task, even as their initial ability to switch responses was left intact.

Like the fronto-parietal network, multiple accounts have suggested that regions within the MTL-OMPFC network may be organized along a gradient of abstraction – with more specific information maintained in hippocampus and greater abstraction of knowledge within entorhinal cortex, OFC and vmPFC [108–110]. Other work on spatial memory has indicated an anterior-posterior gradient in abstraction within the hippocampus, from specific locations to contexts [111]. A recent fMRI study showed that this hippocampal gradient mirrors the anterior-posterior gradient in fronto-parietal cortex during a context-based hierarchical retrieval task [112]. Thus, both networks may maintain parallel representations of task knowledge at multiple levels of abstraction.

While the traditional division between procedural and declarative memory has been taken as evidence that the MTL is not involved in learning about actions, lesion evidence suggests otherwise. Studies in humans and non-human primates demonstrated the necessity of the hippocampus in acquiring arbitrary stimulus-action associations [93,113]. Experiments using serial reaction time tasks to study sequence learning have found that amnesics can acquire first-order sequential associations, but do not acquire higher-order sequential associations, and are less likely than controls to be explicitly aware of these associations after testing [114,115]. fMRI studies of this paradigm also show that the hippocampus is activated during encoding and retrieval of sequences [116]. A recent case study of an individual with bilateral MTL damage sheds light on the particular involvement of the MTL in learning about actions [117]. This individual was able to acquire new motor skills with sensorimotor perturbations (i.e. mirrors or visuomotor cursor rotations) over multiple sessions, but failed dramatically to adapt to learning in extinction (i.e. after the removal of these perturbations). Thus, rather than being uninvolved in the motor domain, the MTL could have a role in building and retrieving memories for complex stimulus-action associations and response strategies.

We suggest that this lesion and imaging evidence from across species indicates that the MTL-OMPFC network is particularly crucial for learning the latent structure of tasks and utilizing this knowledge for learning, planning and decision-making. In contrast, we suggest that the fronto-parietal cortex is needed for maintaining and flexibly switching between abstract production rules for efficient cognitive control. In the next section, we develop the implications of this separation of task representations for cognition and behavior.

Wherefore different formats for abstract task representations?

Why would the brain separate abstract representations of production rules and task knowledge? Maintaining representations of tasks in different formats could afford distinct behavioral advantages. In particular, efficient control in most tasks that have had even a moderate amount of practice would not be well-served by requiring that we reference the full relational structure of a task before each response. Conversely, shaping knowledge of the relational structure of a task based on control demands could pose a disadvantage to flexibly using this abstract task knowledge in the future. Having a cognitive map allows us to infer distances and plan new routes, but when actually travelling, we need to know where and when to take actions.

Behavioral studies of skill learning tasks provides a framework for understanding the advantages and disadvantages of task knowledge in different formats [87,118]. Specifically, they suggest that we might initially use an elaborative process that compares a novel problem with previously encountered analogous situations in order to infer what is the correct action to take. However, this process takes time, as it depends on a comparative search through declarative memory. But with greater experience, we quickly build abstract productions that speed up implementation, applying task rules that are more abstract and flexible compared to reasoning by way of analogy (Figure 3). With enough repetitions of the same problems, we may settle on highly efficient instance-based memorization of stimulus-action associations [87,119].

Figure 3. Schematic illustrating differences in representational format for production rules and a map of task knowledge in mock acquired equivalence task.

Distinct perceptual representations of stimuli in the environment (e.g. A1 and A2) are linked to common latent task states within a cognitive map of abstract task knowledge (e.g. A), where they are also retained with knowledge about appropriate responses (e.g. R1). The format of this knowledge makes it possible to read out information about the relations between these conditions in multiple ways (dashed lines). In parallel, this knowledge is represented in abstract production rules that specify the actions to commit in each latent task state. These production rules allow this action to be flexibly and efficiently executed by either cue without referencing a more extensive representation of the associations among task states. Both of these task representations also interact with each other (solid purple double-headed arrow), allowing information to flow between formats.

Acquisition of new tasks is often accompanied by an exponential improvement in accuracy and reaction times in the first few trials before reaching asymptotic levels of performance [119]. This improvement is indicative of a rapid process of learning and adjustment in the earliest experiences with a task. Recent evidence from our group and others suggests that this change on early trials is at least partly due to the acquisition of new productions, independent of knowledge of the task rules [120,121]. In particular, our group recently found that switching the production for updating information to working memory during a new task caused a negative transfer effect on early trials [120]. Participants were slower to respond on the new task compared to the start of the experiment, because they transferred a production from a previous task that was no longer efficient. These findings are noteworthy because this transfer effect was specific to the production rule, rather than the task rules or stimuli, indicating that this production was maintained in an abstract format that allows for its reuse between tasks that differ in their details but require common cognitive operations.

Imaging work has supported the notion that the fronto-parietal network is particularly engaged when converting declarative knowledge about a task into productions for controlling behavior. In particular, several studies have shown that the fronto-parietal network is more strongly engaged at the start of novel tasks, during task instructions, and when adding new rules to existing tasks [122–124]. This network thus seems to rapidly reformat task information from spoken and written language into control representations. It is possible that a similar process transforms abstract task knowledge stored in the MTL-OMPFC system into production rules that can facilitate rapid control of behavior.

We propose that that the MTL-OMPFC network maintains task knowledge in a cognitive map that can be used to make inferences about the relations between task condition and plan trajectories in this abstract state space. In contrast, fronto-parietal cortex, perhaps in interaction with subcortical structures in the basal ganglia, may enact abstract production rules that can generalize across different settings with similar control demands. Unlike representation of the relations of task elements in MTL-OMPFC, we suggest that these productions delineate a conditional and asymmetric relationship between abstract task variables and actions, both cognitive and motoric. In Box 1, we describe specific testable predictions for this hypothesis.

Box 1. Specific predictions.

Based on the hypothesis laid out here, we can draw some predictions about how specific task demands should impact the involvement of networks involved in representing abstract task structure, and behavioral differences that should accompany these task demands. Here we detail some of these predictions that could serve as tests of this hypothesis:

1. Abstract task knowledge and productions may be separated for greater behavioral efficiency. Namely, so that control of cognitive and behavioral operations does not depend on the slower process of referencing a more elaborate internal model [87]. We expect that the fronto-parietal network would be implicated more in tasks with higher demand for rapid, conditional selection of behaviors based on abstract rules. In contrast, the MTL-OMPFC system would likely be more engaged when this internal model is referenced for monitoring or updating abstract task knowledge. These scenarios may include inferring latent relationships or transferring knowledge between conditions that share latent associations.

2. Consolidation of relational task knowledge into abstract task information and control processes takes time and depends on mental exploration of the task space. Long reaction time trials and off-task periods are likely critical to this process as they afford opportunity to retrieve task conditions and recognize abstract similarities. We predict that exploration of this task space in the MTL-OMPFC network could explain long increases in reaction times in tasks that involve relatively simple inferences [58,130]. Disruption of these opportunities should thus prevent such representations from forming. However, once this abstract knowledge is transformed into a control process maintained in fronto-parietal cortex, such disruptions should have less impact on behavior. Yet other manipulations that interfere with implementing productions, like switching abstract task conditions, or recombining production rules in novel ways, should also result in reaction time costs.

3. Learning a conditional relationship between two cues and feedback can be described as reframing one cue as a ‘context’ and another as a ‘stimulus’ [66]. Action selection and learning is then framed with respect to the value of the stimulus, given the context. Doing so introduces directional asymmetry in cognitive operations so that the context is considered before the stimuli (e.g., [118]). Thus, task demands that reverse the direction of this operation, even with the same two cues, require interference to enact if this reverse productions has not been formed [118]. We predict that this reversal should place demands on retrieval of task knowledge from the MTL-OMPFC network prior to the formation of a new production, though this would be expected to quickly fall off as that production is put into place.

4. A hard action-knowledge separation implies that acquisition of task knowledge is insulated from an agent’s particular behavior. This case might be ideal for preventing implementational details from influencing learning about the structure of the environment (i.e. similar to the advantages posed by on- and off-policy learning). Alternatively, relational knowledge may be acquired by direct experience and supplemented by replay and simulation of counterfactual actions. These alternative predictions are similar to different computational theories of the MTL system that posit the centrality of learning from experienced associations [14] versus an action-less assimilation of experience [110].

A non-human primate electrophysiological experiment by Bernardi et al. [125] illustrates how representations with similar content may co-exist in PFC and the MTL during reversal learning, but with different representational roles. The authors used a clever cross-condition generalization test, where a classifier is trained on half of the task conditions and its generalization performance is tested on held-out conditions, providing a strong test that conditions are maintained in an abstract format within a neural state-space. This analysis revealed that both DLPFC and the hippocampus maintained an abstract representation of the latent task context (i.e. the reversal state). Notably, this representation started later in DLPFC and also fell off after stimulus onset, even while it was maintained in the hippocampus. The dynamics of these two regions suggest that DLPFC might receive such abstract task information from the hippocampus and change its representation to facilitate efficient control during response selection.

Several lines of work have provided evidence that abstract task representations in the MTL-OMPFC network may be particularly involved in making inferences and planning actions based on a representation of task space. Building on observations of grid-like representations of conceptual space [31,32], recent computational modeling has argued that the entorhinal cortex is involved in building generalizable knowledge that can aid learning and inference [14,110]. While differing in their computational details, these models suggest that information about task structure maintained in entorhinal cortex could act in concert with representations of specific memories in hippocampus to form inferences about the relationships between task conditions [126]. Multiple studies have implicated the hippocampus in generalization and inference [12,46,50,51], and modeling has suggested that generalizable representations are formed within the hippocampus itself [127]. Comparatively, physiological support for the involvement of the entorhinal cortex has been less forthcoming to date. However, a recent fMRI study has lent some support to this idea, showing that pattern activity in entorhinal cortex maintains a representation of latent task relations that could prove useful for updating different items that have correlated values in a reinforcement learning task [128].

Importantly, other theoretical work has emphasized fronto-parietal cortex in this process of learning and making inferences about task structure. It has been suggested that generalization of stimulus-action rules in cognitive control settings could rely on a similar process of re-describing tasks with a smaller set of representations that share stimulus-action-outcome associations, and amplify certain actions via control of cortico-striatal circuits [66]. A recent fMRI study supports the involvement of fronto-parietal cortex in this process, showing that activity in this network correlates with new information about the underlying causal relationships between cues and contexts provided by feedback [129].

Our hypothesis could help account for this apparent discrepancy. Namely, fronto-parietal cortex might update an abstract production that generalizes across contexts with shared control demands. The relations between these stimuli, contexts and outcomes may also be maintained in the MTL-OMPFC, but in a format that could enable more flexible comparisons between task stimuli and contexts [12,110]. vmPFC and RLPFC might work together to assess the appropriateness of current productions and select alternatives [68,72]. Some fMRI work supports the idea that abstract task knowledge in the MTL-OMFC system is optimized for flexibly forming novel inferences, demonstrating that the vmPFC, OFC and entorhinal cortex maintain representations of memorized task-relevant and irrelevant dimensions of stimuli during a social decision-making task [29]. Representations of such latent knowledge in this network could provide a basis for the fronto-parietal network to form abstract productions in new settings. Box 2 reviews the cortico-cortical connections and processes by which these networks might interact to allow abstract task knowledge to inform cognitive control.

Box 2. Transformation of knowledge to action.

A key implication of our hypothesis is that these networks must interact for knowledge about task conditions in the MTL-OMPFC network to influence the formation of production rules in fronto-parietal cortex. Figure I lays out a schematic view of the structural connections among the main cortical regions of the networks identified here based on anatomical studies of non-human primates. While somewhat segregated in their connections, these networks interact at several points. In particular, reciprocal connections within PFC, like those between mid-DLPFC and RLPFC with vmPFC and OFC, could allow representations of latent task conditions to influence the formation of appropriate productions [131–134]. The retrosplenial cortex, posterior cingulate, and to a lesser extent the parahippocamal cortex, also are convergent points of both networks [132,133,135–137] – suggesting a potential key role in integrating and transforming knowledge about the environment into control representations. The retrosplenial cortex in particular has been implicated in translating between allocentric and egocentric spatial reference frames for navigation [138], and could play a similar role in translating abstract task knowledge to productions. Our hypothesis would predict that disruption of activity within these regions could cause impairment in the transformation of abstract task knowledge into control representations, while leaving the capacity to learn and describe abstract task structure, or act on instructed rules, intact.

Notably, the MTL-OMPFC network described here has substantial overlap with the ‘default mode network’ – a set of regions with correlated activity that are commonly observed as more active at rest [60]. Mirroring anatomical connections, activity within a subcomponent of the fronto-parietal network (including RLPFC and mid-DLPFC) tends to be more correlated with the default mode network [139]. Interactions between these networks likely has an important role in schematic control, with highly abstract task knowledge flowing hierarchically from MTL-OMPFC into RLPFC and mid-DLPFC, and then parts of the fronto-parietal control network that are more closely connected to sensorimotor cortex [17,64]

The processes and mechanisms of the transformation from knowledge to action have not been well described, but likely depends on coordinated activity in fronto-parietal cortex and the MTL. Recent work [140] showed that hippocampal pattern activity separated representations of contexts that were similar in task demands, specifically the frequency of encountering different trial-types in a classification task. Notably, the degree of pattern separation in the hippocampus was predictive of reinstatement of pattern activity associated with the task demands in DLPFC, suggesting that separation of memories for task conditions in the hippocampus may facilitate retrieval of production rules in the fronto-parietal cortex. Other recent behavioral work has argued that control settings are encoded into memory for item-specific representations of task conditions [141,142].

Reactivation and reorganization of past experiences could also provide a mechanism for the creation of abstract task representations and rules. Recent magnetoencephalography investigations of replay have focused on the role of hippocampal-cortical circuits in consolidating task knowledge, particularly during rest periods between active task sessions. Notably, the ordering and content of these replay events appear to be actively directed towards reinforcing goal-directed behavior. These studies have found that hippocampal replay re-organizes the order of non-spatial task states from their observed presentation, in line with goal-relevant rules [143], and that unobserved alternative routes through non-spatial states have are replayed to aid updating of state-values in a reinforcement learning task [144]. Replay of such task states could help train the formation of rules in downstream cortical areas, facilitating the formation of abstract task representations [145].

Concluding remarks

Recent evidence indicates that the brain supports abstract task representations in multiple distinct networks that fulfill different roles depending on cognitive and behavioral demands. In particular, we have proposed that a network involving the MTL, medial PFC and OFC support the process of discovering task structure from experience. Its format is suitable for inferring latent task states and abstract relations from prior observations and feedback. In contrast, a network of lateral prefrontal and parietal cortical regions maintain abstract task representations as a set of state-action contingencies, or productions, in a format suitable for cognitive control and guided action selection. Broadly, we propose that differences in task knowledge will be difficult to distinguish between these networks. Rather, it will be more fruitful to test differences in the format and use of these representations (see Outstanding Questions).

Outstanding questions.

• What are the computational pressures and task demands that result in abstract task information being formatted differently by these two networks (e.g. demands on efficiency, updating, learning, control policies, etc.)?

• How would formatting abstract task knowledge in the form of a cognitive map or production rule influence the kinds of operations these representations can enact?

• How does this dissociation of formats for of abstract task knowledge in different neural systems relate to differences in learning and cognitive control across the lifespan and the development and deterioration of these neural systems?

• How do impairments in behavioral flexibility relate to challenges with learning about the abstract structure of tasks versus failure to utilize this knowledge for controlling behavior?

• What allows us to format task information for more efficient use? How do we balance the choice in this format between generalizability and a desire to avoid interference?

• Across many studies, there are major individual differences in the extent to which participants show evidence of inferring abstract task structure. Why do some participants successfully grasp task structure and others fail? In what cases is it a failure of inference about task structure versus generalizing the right productions? What are the cognitive processes that underlie learning and cognitive control based on abstract task knowledge that are common across different tasks?

• In contrast to the abstract task representations that have been the focus of this review, highly specific conjunctive representations of task stimuli and responses may have opposing computational advantages. Though it is not clear what neural systems represent them, there is evidence for these representations in EEG, and they appear to have a strong influence on task execution. One possibility is that such representations are encoded hippocampus, similar to other kinds of conjunctive representations of events. What computational pressures and neural mechanisms give rise to such highly specific conjunctive memories for task conditions versus forming more abstract and generalizable representations for cognitive control?

Figure I. Schematic representation of connections between brain regions of focus in this review, based on anatomical studies of non-human primates.

Light red and blue indicate regions and connections that are considered part of the fronto-parietal and MTL-OMPFC networks, respectively. Pink indicates connections and regions that cross between both networks. Prefrontal cortical (PFC) regions (central orbitofrontal cortex (cOFC), ventromedial PFC, medial OFC (mOFC), rostrolateral PFC and mid-dorsolateral PFC (mid-DLPFC)) are all strongly inter-connected with each other [131–133,146], allowing information to move between these networks. mid-DLPFC also projects to perirhinal cortex (PRC), providing a direct route to the closely interconnected MTL regions including the hippocampus (HPC), entorhinal cortex (ERC) and parahippocampal cortex (PHC) [131,147]. PHC shares similar connections to retrosplenial (RSC) and the posterior cingulate cortex (PCC), acting as a convergence point for connections from the MTL, PFC, inferior parietal lobule (IPL), and precunueus (PREC) [135–137,148,149].

Highlights.

• Recent research has identified two distinct cortical networks representing abstract task knowledge: a fronto-parietal network, and a network involving the medial temporal lobes (MTL) and medial prefrontal and orbitofrontal cortex (OMPFC).

• Here, we suggest that these networks differ in format but not content, with the MTL-OMPFC network maintaining abstract task knowledge in a map, while fronto-parietal cortex uses this knowledge to build rules for action selection.

• This format distinction would allow the MTL-OMPFC network to form novel inferences and plan trajectories in an abstract state space, and for the fronto-parietal network to rapidly apply abstract rules without referencing a more elaborate internal task model.

• This hypothesis makes novel predictions about the function of these networks in learning and promoting flexible behavior.

Glossary

Associative inference

A paradigm used to test the inference of unobserved relationships based on shared direct associations. Participants might learn that item A is directly associated with item B, and item B is directly associated with item C. Participants are then tested on these direct associations as well as the unobserved relationship between items A and C that can be inferred by their common association with B

Acquired equivalence

A paradigm where two or more cues are learned to be functionally equivalent based on their shared associations (e.g. cues A and B both share an association with cue X). Acquisition of this equivalence can be probed by updating the associations of one cue (e.g. pairing A and Y) and testing if this updating is transferred to other cues with shared associations via a generalization test (e.g. can you predict Y when observing B)

Cognitive control

the capacity to flexibly modify or condition actions or cognitive processes based on task demands and in line with internal goals

Cognitive map

an internal, organized description of information (often spatial, but also non-spatial) with geometric properties that can be used for planning and inference of shortcuts or new pathways. For example, a representation of the layout of your neighborhood that you could use to plan a detour when going to work

Latent state

a ‘state’ or variable describing the condition of the environment that has been inferred to exist by an agent or participant from statistical regularities in observations. Like an observable state in the reinforcement learning sense, such a representation could then provide the basis for conditionalizing learning and action selection, and relations between these states could be used for forward planning and credit assignment

On- and off-policy

Terms describing learning rules that differ in how they incorporate information about actions taken into updates. On-policy implies a rule that updates value estimates with the expectation of a return from the action taken according to the current behavioral policy (e.g. choose the best option most of the time). Off-policy learning rules updates value estimates according to an expectation that doesn’t reflect the current behavioral policy (e.g. an expectation of the best possible action instead of the action chosen)

Production rule

A conditional function describing a cognitive or motoric action to be taken in response to a state. For example, a set of conditional rules for actions to take on a detour route– like which way to turn at an intersection

Replay

A neurophysiological phenomenon where patterns of activity associated with previous experience are sequentially reactivated

Schemas

a structured body of knowledge that can facilitate learning, and organize and direct recollection of information in memory. For example, knowledge about doctor’s appointments (filling out forms, blood pressure readings, etc). This knowledge may be very broad and hierarchically organized, but is distinct from representation of particular instances with spatio-temporal specificity (i.e. episodic memories), and specific facts about the world (i.e. semantic knowledge)

Schematic control

The process of implementing cognitive control over thoughts or actions based on schematic knowledge about the environment or task