The Structure of Cognition: Attentional Episodes in Mind and Brain

Abstract

Cognition is organized in a structured series of attentional episodes, allowing complex problems to be addressed through solution of simpler subproblems. A “multiple-demand” (MD) system of frontal and parietal cortex is active in many different kinds of tasks, and using data from neuroimaging, electrophysiology, neuropsychology, and cognitive studies of intelligence, I propose a core role for MD regions in assembly of the attentional episode. Monkey and human data show dynamic neural coding of attended information across multiple MD regions, with rapid communication within and between regions. Neuropsychological and imaging data link MD function to fluid intelligence, explaining some but not all “executive” deficits after frontal lobe lesions. Cognitive studies link fluid intelligence to goal neglect, and the problem of dividing complex task requirements into focused parts. Like the innate releasing mechanism of ethology, I suggest that construction of the attentional episode provides a core organizational principle for complex, adaptive cognition.

Integrating findings from neuroimaging, electrophysiology, neuropsychology, and cognitive studies of intelligence, Duncan reviews the “multiple-demand” system of frontal and parietal cortex and its role in assembling complex cognition from a structured sequence of attentional episodes.

Main Text

Introduction

Since the information processing revolution of the 1950s, the spectacular diversity of human behavior has led to increasing fractionation in accounts of mind and brain, with dedicated analysis of problems from simple sensorimotor control to story grammars, human-computer interaction, and expertise in chess. Such fractionation suggests a need for organizational principles addressing how cognition in general is constructed and controlled (Newell, 1990). One widely accepted principle is modularity in mind and brain (Kanwisher, 2010), with dedicated cognitive/brain systems undertaking different kinds of information processing. In this paper, I consider an orthogonal organization—the construction of all complex cognition from a series of focused, momentarily assembled temporal fragments.

A useful point of reference is the ethological concept of the “innate releasing mechanism” or IRM (Lorenz, 1970; Tinbergen, 1951). The IRM is conceived as a neural process delivering a fixed fragment of behavior when released by a suitable triggering event—the moth flashing open its wings to reveal staring eye spots when touched by a predator, the toad orienting to the sight of a worm, or the human urged to protect an infant by the sight of large eyes and tall forehead. In much animal behavior, concatenation of such behavioral fragments, each controlled by its own IRM, produces complex, goal-directed sequences or programs of activity; for example, when two mating sticklebacks, each responding to the actions of the other, proceed by a series of stages into the nest where spawning takes place (Tinbergen, 1951).

In human cognition, too, thought and behavior unfold in a complex, structured sequence, with many component fragments assembled to achieve short- and long-term goals (Miller et al., 1960). By comparison with mating sticklebacks, however, human thought and behavior have essentially infinite flexibility and complexity. Unlike the IRM, fragments of human cognition must be momentarily constructed, shaped by the arbitrary requirements of current activity.

The sequential control of complex cognition has been most fully addressed in symbolic artificial intelligence, with systems such as the General Problem Solver (Newell et al., 1962), ACT (Anderson, 1983), and SOAR (Newell, 1990) dividing complex problems into a long, structured series of more solvable subproblems. In many ways, the sequential mental control programs produced by such systems resemble the serial processing of conventional computers. Artificial intelligence also makes it clear why it is that complex problems must be decomposed into simpler components. An example, from the work of Sacerdoti (1974), is shown in Figure 1 (for closely related arguments in different theoretical traditions see Botvinick et al., 2009; Snyder and Munakata, 2010). In this program, a robot inhabited a world of multiple rooms, some containing other objects. Given a start state and a goal state (Figure 1A), the robot was required to find a sequence of moves transforming one into the other. In Figure 1B are shown two solution paths through the total space of alternative problem states. In the first (Figure 1B, left), there is no chunking into subgoals such as “first reach the door of the current room.” Instead, all possible actions are considered simultaneously and, though the problem is eventually solved, this is achieved by a long and chaotic route. The essential difficulty is that, if all aspects of the problem are considered at once, the search space of possible alternative routes is simply too large and unconstrained, resulting in many suboptimal choices. In the second solution (Figure 1B, right), the architecture tends first to choose a relatively abstract subgoal and then work within that to elaborate a detailed solution. Now the path to the goal is orderly and direct, as one organizing subgoal after another comes into force and controls system function until it is achieved. In Figure 1B, as Sacerdoti (1974) notes, it is as if the program is given a series of “small problems to solve consecutively” (p. 129).

Figure 1.

Shaping Cognition by Division of Complex Problems into Simpler Subproblems

(A) Start and goal states for a problem in Sacerdoti (1974).

(B) Search paths through the space of possible problem states (nodes) without (left) and with (right) effective subgoaling. Without subgoaling, the route from start state (top) to goal state (bottom) is complex and inefficient, with exploration of many states not included on the final solution path. Subgoaling creates a series of small subproblems (indicated by breaks in search path), each of which can be solved with maximal efficiency. Adapted with permission from Sacerdoti (1974).

For any realistic behavior, on this analysis, there is a complex space of possible actions, states, and processing operations. Effective cognition requires a series of selections from this space, each defining a subproblem of relevant inputs, actions, and potential achievements. Often these will be organized hierarchically, so that each subgoal or task part is divided further into subgoals of its own (Miller et al., 1960; Sacerdoti, 1974). As selection is the defining characteristic of attention, cognition may be described as a series of attentional episodes, with each episode admitting into consideration only the contents of a momentary, focused subproblem.

A central role in attentional control is frequently given to the frontal lobes (Norman and Shallice, 1980; Desimone and Duncan, 1995; Miller and Cohen, 2001). The classic descriptions of Bianchi (1922), Penfield and Evans (1935), Luria (1966), and others document the global disorganization of behavior that can follow frontal lobe damage, often with intact behavioral fragments but no complex, goal-directed structure. In the words of Bianchi (1922), “The monkey which used to jump on to the window-ledge, to call out to his companions, after the operation jumps on to the ledge again, but does not call out. The sight of the window determines the reflex of the jump, but the purpose is now lacking, for it is no longer represented in the focal point of consciousness...Evidently there are lacking all those other images that are necessary for the determination of a series of movements coordinated towards one end” (Bianchi, 1922, p. 184). Indeed, classic descriptions of frontal lobe patients are strongly reminiscent of the two search paths of Figure 1B—chaotic behavior in complex problems but success if the task is externally structured into simple parts (Luria and Tsvetkova, 1964). In this paper, I review evidence for a specific, distributed frontoparietal system whose role, I propose, is to construct the sequential attentional episodes of complex cognition.

The Multiple-Demand System

One of the most robust results of human functional brain imaging, including positron emission tomography (PET) and fMRI, is a strong common core of brain activity resulting from cognitive challenges in many different domains—perception, response selection, language, many varieties of memory, problem solving, task novelty, and many more (Duncan and Owen, 2000; see also Cabeza and Nyberg, 2000; Cole and Schneider, 2007). Reflecting its ubiquity across many cognitive domains, I have called this the multiple-demand or MD pattern (Duncan, 2005, 2010b). Included in the MD pattern are cortex on the lateral frontal surface, along the middle frontal gyrus and extending posteriorly into premotor cortex; in the anterior insula and adjacent frontal operculum; in the dorsomedial frontal cortex, including presupplementary motor area (pre-SMA) and dorsal anterior cingulate; and within and surrounding the intraparietal sulcus. Accompanying activity is commonly seen in subcortical regions including basal ganglia, thalamus, and cerebellum. A similar pattern is seen in the “task positive” or “control” network commonly defined by analysis of temporal correlations in resting state data (e.g., Cole and Schneider, 2007; Seeley et al., 2007; Vincent et al., 2008), sometimes further divided into subnetworks including “frontoparietal control,” “dorsal attention,” and “cinguloopercular” (e.g., Dosenbach et al., 2007; Power et al., 2011).

The topography and functional properties of MD activity are illustrated in Figure 2, based on a recent study from Fedorenko et al. (2013). The study involved seven different tasks, covering demands in language, arithmetic, verbal and spatial working memory, and several varieties of response selection/inhibition. To examine brain activity associated with increased cognitive demand in each domain, each task had a harder and an easier version. Results of conventional random-effects analysis for each task (Figure 2A) show the usual MD activity pattern associated with increased difficulty in each separate task. At least in these visual tasks, extensive demand-related activity is also seen in higher visual cortex, though here I shall focus just on the frontoparietal MD regions. Average results across tasks show a “canonical” MD system (Figure 2B).

Figure 2.

Multiple-Demand System in the Human Brain

(A) Activity pattern for hard minus easy contrast in tasks tapping multiple cognitive domains (top to bottom: remembering word/nonword strings, arithmetic, spatial working memory, verbal working memory, and three versions of resisting response conflict).

(B) Mean hard minus easy activity pattern across all seven tasks. Generally bilateral activity has been captured by averaging across left and right hemispheres, and projecting the resulting mean image onto the right. Included in the full multiple-demand (MD) pattern are a posterior strip of the lateral frontal surface, from premotor cortex to the frontal operculum and anterior insula; an anterior-posterior strip extending along the inferior frontal sulcus; a dorsomedial strip extending from pre-SMA to the dorsal part of the anterior cingulate; and activity along the length of the intraparietal sulcus. At least in visual tasks, accompanying activity is generally seen in higher visual areas. Outside the cerebral cortex, activity is also seen in the medial cerebellum and elsewhere.

(C) Left hemisphere activity for two example participants, showing closely adjacent MD (blue; greater activity in memory for nonword strings versus sentences) and language (red; reverse contrast) regions. Adapted with permission from Fedorenko et al. (2013).

More telling are the results of single-subject analyses. Using one task as a localizer for MD voxels in a single subject’s brain, Fedorenko et al. (2013) went on to ask how these same voxels behaved in the other six tasks (see also Stiers et al., 2010; Wojciulik and Kanwisher, 1999). For all major regions shown in Figure 2B, results were similar; in individual subjects, voxels with strong response to the localizer also showed increased activity for other cognitive demands. In individual subjects, tellingly, MD regions were often immediately adjacent to regions with a very different functional profile, responding to increased difficulty neither in the localizer nor in other tasks. In left lateral frontal cortex, for example, a typical MD region often surrounded a quite different region, showing selective activity for language (Fedorenko et al., 2012; see Figure 2C). In a control region of the temporal lobe, equivalent analyses suggested no MD voxels in single subjects, i.e., no voxels with common response to multiple cognitive demands. Taken together, these data show tightly localized MD activity, varying in exact pattern from one person to another but with a highly consistent overall topography in frontal and parietal cortex.

To link imaging results to electrophysiology, a critical question is correspondence between human and animal systems. In the macaque, one fMRI study comparing pro- and antisaccades showed activity in lateral frontal, dorsomedial frontal, and parietal cortex, reminiscent of the human MD pattern (Ford et al., 2009). Somewhat similar patterns have also been obtained by a simple comparison of visual stimulation versus fixation (Stoewer et al., 2010) and by analysis of correlations in resting state data (Sallet et al., 2013; though see Mantini et al., 2013). In support of these functional data, anatomical studies confirm connections between lateral frontal, dorsomedial frontal, parietal, and insular regions (e.g., Cavada and Goldman-Rakic, 1989; Goldman-Rakic, 1988; Mufson and Mesulam, 1982).

Activity crossing multiple task demands suggests functions of importance in many different kinds of cognition. In the following sections, I suggest that the core function of the MD system is to control complex behavior in a series of attentional episodes. In multiple MD regions, I suggest that neurons have highly dynamic response properties, adapting to code the specific information and events within the current attentional focus (Duncan, 2001). With the transition between one episode and the next, neural coalitions for one kind of information processing dissolve and coalitions for the next episode form, producing a system in constant flux. In line with many other ideas of frontal lobe function (e.g., Norman and Shallice, 1980; Dehaene et al., 1998; Miller and Cohen, 2001), I suggest that focused MD coding of information relevant to a current attentional episode drives linked processing in multiple other brain regions, configuring widespread brain activity for solution of the selected problem. The universal importance of MD activity, I suggest, provides a core basis for the psychometric concept of fluid intelligence. I review recent findings linking fluid intelligence to the neuropsychology of control or executive disorders following frontal lobe damage and examine its cognitive basis in decomposing complex problems into simpler, more focused parts. Beyond psychometrics, I suggest that problem decomposition of this sort lies at the heart of abstract thought and rationality, providing infrastructure for the full complexity and diversity of human cognition (Duncan, 2010a; Zylberberg et al., 2011).

Physiological Properties of the MD System

The role of MD cortex in construction of attentional episodes is most clearly shown in single-unit data from the awake, behaving monkey. Many monkey studies have examined single-cell activity in the posterior part of the lateral frontal cortex, within the principal sulcus, and on the dorsolateral and ventrolateral surfaces above and below this sulcus. These studies show a picture of highly flexible neural properties, dynamically adapting to code the specific content of a current task episode (Duncan, 2001), with similar results increasingly reported also for regions of dorsomedial frontal cortex (e.g., Procyk et al., 2000), insula (e.g., Mizuhiki et al., 2007), and parietal cortex, including Brodmann’s area 7 on the inferior frontal convexity, perhaps extending into area LIP of the intraparietal sulcus (e.g., Freedman and Assad, 2006).

One striking property of prefrontal activity is breadth of information coding. Across attentional episodes, many different kinds of information may be critical to behavior, and in prefrontal cortex, correspondingly, cells code relevant stimuli, responses, rewards, rules, working memory contents, etc. (Duncan, 2001; Miller and Cohen, 2001). Indeed, one critical result is simply the high frequency of task-related activity found in this cortical region. Though neurons are randomly sampled, and later tested for activity related to the events of a particular, arbitrary task, high proportions of all cells show activity that is linked to task events. The result has been repeated for a very wide range of task types—same/different matching (Wallis et al., 2001), visual target detection (Kusunoki et al., 2010), dot counting (Nieder et al., 2002), and many more. When the animal monitors a series of images awaiting a specific target, for example, a recent study of ours found more than 50% of all prefrontal cells discriminating targets from nontargets (Kusunoki et al., 2010). If an animal classifies visual images as “cats” or “dogs,” between 20% and 40% of cells are found to behave as cat-dog categorizers (Freedman et al., 2001; Roy et al., 2010). In this large area of cortex, large proportions of cells code the specific information required in a current epoch of behavior. Similar data come from studies of rat frontal cortex, with many cells coding behavioral strategy, choice preference, etc., and the population showing rapid shifts of activity state when strategy or preference changes (e.g., Durstewitz et al., 2010; Karlsson et al., 2012).

Such large proportions of apparently “task-related” cells suggest flexibility, with neural properties shaped by current task context. Flexibility is a second critical property of attentional coding; as momentary task context changes, so too does required information. Correspondingly, information coding in prefrontal cells changes with momentary task relevance. In categorization tasks, for example, a cue to change between orthogonal categorizations of the same stimulus set alters prefrontal coding, with selective emphasis of the category boundary relevant on the current trial (Roy et al., 2010). Similar results follow a switch of attention between different visual features such as color and shape (Sakagami and Niki, 1994). In working-memory tasks, cells show sustained activity linked to stimulus identity when this information is required within the trial but then switch to coding of location when identity is no longer relevant and only location guides the final response (Rao et al., 1997).

A third critical property is flexible transition from one step of a mental program to the next. In the target-monitoring task (Kusunoki et al., 2009; Stokes et al., 2013), for example, a cue at trial onset indicates the target image for the current trial (Figure 3A). Activity can thus be examined during successive task stages: during initial cue presentation, during the delay between cue and choice stimuli, and during classification of each choice stimulus as target or nontarget. When the cue is presented, an initial burst of activity includes many cells coding cue identity (Kusunoki et al., 2009; Stokes et al., 2013). This initial code, however, is transient; across the cell population, a quite different pattern of activity is seen during the subsequent delay, with many cells again coding cue identity, but with cue preferences unrelated to those of the initial sensory response (Stokes et al., 2013; Figure 3B). In fact, activity patterns in cue and delay phases are approximately orthogonal: a cell’s activity during one phase is essentially unpredictive of behavior at another (Sigala et al., 2008; for similar evidence from rat frontal cortex, see Lapish et al., 2008). When choice stimuli appear, a first wave of activity discriminates one picture from another, irrespective of behavioral status on the current trial, but within 300 ms, the activity pattern is dominated by the behaviorally critical distinction between targets and nontargets (Stokes et al., 2013; Figure 3C). Again, the same cells that initially discriminate picture identity may, within a few hundred milliseconds, have switched to discriminating targets from nontargets. As the successive steps of the task unfold, activity in the same pool of neurons is repeatedly reorganized, at each stage coding behaviorally critical information. Other data confirm how, in different tasks, similar information can be coded by quite different activity patterns at different task stages (Hussar and Pasternak, 2012; Miller et al., 1996; Warden and Miller, 2010; though for contrary examples see Takeda and Funahashi, 2004; Saga et al., 2011).

Figure 3.

Dynamic Representation of Task-Critical Information in Monkey Prefrontal Cortex

(A) Three alternative cue pictures (cues 1 to 3) indicated three alternative target pictures (targets 1 to 3) for the current trial. Each trial began with a single cue to left or right of fixation, followed by a sequence of 0–3 nontargets and finally the cued target. The monkey was rewarded for a saccade to the location of the stimulus stream at target offset; saccades at other times aborted the trial without reward. Nontargets were a random sequence of neutral stimuli (associated with no cue and thus never serving as targets) and distractors (associated with other cues, and thus serving as targets on other trials). The cue thus established critical choice context for the current trial.

(B) Temporal specificity of cue coding. At each time from cue onset (test and train times), discriminative coding for any two cues (e.g., cue 1 versus cue 2) was measured by the pattern of cue preferences across the whole recorded cell population. The preference pattern was measured using one half of the data from each cell (train), then compared to patterns from the other half of the data (test), measured at the same (diagonal) or different (off-diagonal) times. Color scale shows pattern similarity. In the early period of cue processing, preference patterns were highly time specific; test patterns at one time point were similar to train patterns from nearby times but unrelated to train patterns from distant times. Late in cue processing and during the following delay period, preference patterns were more stable, indicated by approximately square region on top right of the pattern similarity plot.

(C) Coding of stimulus- and decision-related information following onset of each choice stimulus (target or nontarget). Red line shows mean coding strength for discrimination of two pictures, irrespective of status as target or nontarget. Blue line shows coding strength for discrimination of target versus nontarget, irrespective of picture identity. Again, coding strength is measured by similarity of preference patterns in train and test data, here for train and test patterns from identical times (cf. B, diagonal only). Adapted with permission from Stokes et al. (2013).

It is often proposed that, within and across brain regions, the components of a cognitive operation may be bound together by some form of neural synchrony or temporal coherence (Womelsdorf et al., 2010), and in several regions of frontal cortex, new neural coalitions—defined by new patterns of connectivity or coherence—are defined at transitions between task steps (Vaadia et al., 1995; Abeles et al., 1995; Lapish et al., 2008). Coherence can occur within (e.g., Buschman et al., 2012) and between (e.g., Buschman and Miller, 2007; Salazar et al., 2012) MD regions, with patterns of coherence sometimes coding the specific content of current cognition such as a remembered item or current decision rule (e.g., Buschman et al., 2012; Pipa and Munk, 2011; Salazar et al., 2012). Based on components of the local field potential (LFP), relative latency and Granger causality can be used to examine communication between pairs of MD regions, e.g., lateral and dorsomedial frontal (Rothé et al., 2011) or lateral frontal and parietal (Salazar et al., 2012). Causal influence probably occurs in both directions (e.g., Salazar et al., 2012), with the predominant direction likely varying with cognitive context (Rothé et al., 2011), timing within the trial, etc.

Context and timing are also critical in comparing the coding of task-relevant information between MD regions. Information preferentially encoded in one region of frontal cortex early in a trial, for example, may be more broadly represented shortly afterward (e.g., Kusunoki et al., 2010; Kaping et al., 2011), suggesting rapid distribution of information between frontal regions. While similar neural properties are often found in lateral and dorsomedial frontal cortex, some findings suggest especially strong dorsomedial activity when a new problem must be solved (e.g., Johnston et al., 2007), based on correct or error feedback (e.g., Rothé et al., 2011). Comparison of activity in lateral frontal and inferior parietal cortex usually suggests strongly overlapping properties, coding task-relevant information such as target location, stimulus category, or working memory contents (Chafee and Goldman-Rakic, 1998; Goodwin et al., 2012; Katsuki and Constantinidis, 2012; Merchant et al., 2011; Swaminathan and Freedman, 2012). Where similar information is represented in frontal and parietal neurons, mean differences in latency are again context specific (e.g., Buschman and Miller, 2007; Goodwin et al., 2012; Swaminathan and Freedman, 2012), usually with strong overlap of latency distributions from the two regions.

These single-cell data make sense of the MD pattern seen in fMRI. When cells adapt to code the specific information that is required in current behavior, the result should indeed be a pattern of widespread activity, irrespective of particular task content. In fMRI, information coding can be seen most directly using multivoxel pattern analysis (Haynes and Rees, 2006). An example (Woolgar et al., 2011a) is shown in Figure 4A. In this study, screen background color instructed which of two stimulus-response mapping rules should be used to link a visual stimulus (a square in one of four positions along the horizontal meridian) to a speeded keypress response (four keys, operated by middle and index fingers of the two hands). To test the hypothesis that coding would be sensitive to demand, stimulus positions were either easy or difficult to distinguish. Across multiple MD regions, detailed patterns of activity in individual subjects could be used to discriminate all relevant features of the task—rule, stimulus position, and response position—suggesting that coding of multiple task features was widespread across the MD system (Figure 4B; see also Woolgar et al., 2011b). In line with the demand hypothesis, furthermore, position coding was visible only in the more difficult condition (Figure 4B, middle row). Using both MVPA and fMRI adaptation, multiple studies converge to show widespread MD coding of relevant stimulus features, rules, intentions, and responses (e.g., Haynes et al., 2007; Li et al., 2007; Woolgar et al., 2011b; Thompson and Duncan, 2009), with specific focus on currently attended information (e.g., Hon et al., 2006; Li et al., 2009). MVPA studies have also begun exploring the context specificity of MD coding. Sometimes, for example, the pattern of activity associated with a given task rule shows cross-generalization from one context to another, suggesting a degree of common coding across somewhat different task states (Cole et al., 2011; Reverberi et al., 2012).

Figure 4.

Adaptive Coding of Task-Relevant Information in Human MD Regions

(A) Tasks from Woolgar et al. (2011a). Screen color indicated which of two stimulus-response mapping rules linked stimulus position to the required response. To manipulate perceptual demand, stimulus positions were widely or narrowly spaced.

(B) Coding of rule, position, and response across MD regions, measured with multivoxel pattern analysis. Ordinate shows discrimination of activity patterns (success in classifying patterns in test data from patterns in separate train data) for different task events (rule 1 versus rule 2; inner versus outer stimulus positions; inner versus outer response positions; coding of stimulus and response patterns as inner versus outer orthogonalizes rule, stimulus and response classification, and means that chance performance for each classifier is 50%). Asterisks above individual bars show comparison against chance; ^∗p < 0.05, ^∗∗p < 0.01. IFS, inferior frontal sulcus; AI/FO, anterior insula/frontal operculum; IPS, intraparietal sulcus; ACC, anterior cingulate cortex; pre-SMA, presupplementary motor area. Adapted with permission from Woolgar et al. (2011a).

Data from fMRI also support the prediction of strong MD activity when a new cognitive episode is created and in transitions from one episode to the next. Strong MD activity is seen, for example, at the boundary between perceived events, for example at the transition between one movement and the next in a piece of music (Sridharan et al., 2007, see Figure 5A; for a review, see Kurby and Zacks, 2008). In any complex behavior, goals (e.g., making breakfast) are generally achieved by means of a hierarchically structured series of subgoals (make coffee, tip in milk, etc.); a recent study shows extensive activity as each subgoal is completed, with increasing activity for subgoals progressively higher in the hierarchy (Farooqui et al., 2012; Figure 5B). MD activity increases when materials for short-term memory are formed into higher-level chunks, perhaps reflecting activity as the parts of a new chunk are bound into one whole (Bor et al., 2003). Strong MD activity is associated with the major change of cognitive context required in task switching (e.g., Dove et al., 2000; Wager et al., 2004). Electrophysiological data also link task transitions to peaks of activity and synchrony in frontal neurons (Fujii and Graybiel, 2003; Sakamoto et al., 2008).

Figure 5.

Brain Activity Linked to Episode Boundaries

(A) Activity at transitions between musical movements. Adapted with permission from Sridharan et al. (2007).

(B) Activity linked to goal-subgoal achievement. On each trial, participants monitored a sequence of letters (total sequence duration 52 s), searching first for the three letters of a cue word (here CAT), and finally for the letter X. Targets were to be detected in the correct order, so that the participant searched first for T1 (here C), then after T1 had been detected, searched for T2 (here A), etc. Detection of T3 thus completed the first subtask, while detection of T4 (the letter X) completed the whole task. Plots at the bottom show activity linked to target detection in MD regions (arbitrary units). Target-detection activity was widespread, often greater for T3 than for T1 or T2, and greatest for T4. For detection of two successive two-letter words, strong activity shifted from T3 to T2 (data not shown). L, left; R, right; APFC, anterior prefrontal cortex; other abbreviations are as in Figure 4. Adapted with permission from Farooqui et al. (2012).

Given differences in connections, cytoarchitecture, etc., it seems certain that the anatomically distinct parts of the MD system must have somewhat different physiological functions, and the current literature contains several important proposals (e.g., Hampshire et al., 2012). Along the middle frontal gyrus, for example, it is commonly suggested that there is some form of anterior-posterior gradient, with more anterior regions involved in higher-order or more abstract cognitive control (Badre and D’Esposito, 2009; Koechlin et al., 2003; though see Crittenden and Duncan, 2012; Reynolds et al., 2012). As mentioned earlier, resting state data suggest some division of the broad MD system into separate lateral frontoparietal and cingulo-opercular subnetworks, and it has been proposed that these focus respectively on more phasic and more tonic aspects of task control (Dosenbach et al., 2006, 2007). The limbic connections of the dorsomedial frontal cortex suggest a link to reward processing, and it has been suggested that this MD region is concerned in particular with evaluating the costs and benefits of alternative plans of action, and the effort needed for their implementation (e.g., Kolling et al., 2012; Shenhav et al., 2013). As the single-cell data make clear, however, direct comparisons of neural properties in different MD regions generally suggest only subtle differences, such as a difference of a few tens of milliseconds in mean latency for coding a particular type of task-relevant information. This makes it unsurprising that, at least as yet, the fMRI literature contains little consensus on clear, repeatable functional distinctions. Typically, I suggest, MD regions work together to manage the sequence of attentional episodes, with exchange of relevant information on a timescale far too short to be seen with fMRI. For this reason, fMRI findings may generally be dominated by coactivation, with dissociations at best matters of degree.

Creating the Attentional Episode: Configuring Brain-wide Processing Activity

Of course, preferential coding of attended information is seen in many regions of the brain. Throughout the visual system, for example, there is stronger response to attended than to ignored visual inputs (Bushnell et al., 1981; Moran and Desimone, 1985; Desimone and Duncan, 1995). A common proposal is that frontoparietal cortex provides control input to many other cortical and subcortical systems, biasing activity toward critical or task-relevant information (Norman and Shallice, 1980; Miller and Cohen, 2001; Dehaene et al., 1998; Desimone and Duncan, 1995). In line with such proposals, I suggest that MD activity binds together the components of an attentional episode, constructing the specific conjunction of processing events that current behavior requires. Thus, brain-wide activity is configured for solution of the current behavioral problem.

A potential mechanism is provided by a recent computational model (Rigotti et al., 2010). In this model, an example processing episode might be, “If the task context is X and the stimulus is Y, press button Z.” The model contains dedicated units for each component of an episode—e.g., the context X, the stimulus Y, and the response Z—along with a large pool of conjunction units, randomly connected to the dedicated units. Because of the random connections they send and receive, the conjunction units are useful for stabilizing activity in any arbitrary combination of dedicated units required in a current task epoch. The system has most power when each randomly connected unit is bound to approximately half of the dedicated units, reminiscent of widespread inputs from multiple brain regions into frontal cortex (Pandya and Yeterian, 1996) and rapid distribution of this information between frontal regions (Pucak et al., 1996). Under these circumstances, the randomly connected units show behavior closely resembling activity of prefrontal cells, with dynamic, selective coding of current task content and rapid changes in properties from one task step to the next. Like the randomly connected units of the model, neurons of lateral prefrontal cortex commonly code conjunctions of events (Rigotti et al., 2013), such as the combination of a particular task context and stimulus category (e.g., Kusunoki et al., 2009) or rule and response (Asaad et al., 2000; Tsujimoto et al., 2011).

The best-studied case of communication between frontal and posterior cortical regions concerns control of extrastriate activity by frontal eye fields (Moore and Armstrong, 2003; Armstrong et al., 2006; Gregoriou et al., 2009). For example, electrical microstimulation of the frontal eye field, targeted toward a local region of the visual field, biases V4 processing in favor of stimuli in that region (Armstrong et al., 2006). When attention is directed to a stimulus in the visual field, LFPs show increased coherence between frontal eye field and V4, with measures of Granger causality suggesting bidirectional causal influences (Gregoriou et al., 2009). Though other evidence is patchier, a variety of results show evidence for causal influences of MD regions on processing elsewhere. Cooling lateral prefrontal cortex, for example, impairs performance in a color-cued pro- and antisaccade task, at the same time decreasing preparatory activity but increasing visually driven activity in the superior colliculus (Koval et al., 2011). In patients with lateral frontal damage, impaired target detection in the contralateral hemifield is accompanied by reductions in visual event-related potentials recorded over occipital regions of the damaged hemisphere (Barceló et al., 2000; Voytek et al., 2010). Another potentially powerful approach is to combine recording of evoked potentials with transcranial magnetic stimulation (TMS); the widespread electrical activity evoked by a frontal TMS pulse, for example, changes with task context, suggesting an altered pattern of connectivity from frontal to posterior brain regions (Sakai, 2013). When a cue instructs subjects to store one of two objects (e.g., face and house) in working memory, fMRI shows increased activity in visual regions specialized for processing the selected object category (e.g., Higo et al., 2011). At the same time, the data suggest that functional connections are established between the selected visual region and frontal operculum/anterior insula, and with TMS over operculum/insula, the effects of cues on posterior visual activity are reduced (Higo et al., 2011). If subjects attend to color and ignore motion, then following error trials, activity in dorsomedial frontal cortex correlates with increased activity in color-selective, but decreased activity in motion-selective, visual regions (Danielmeier et al., 2011). Recent data suggest that, as compared to other major brain systems, MD regions are exceptional in the degree to which they change functional connectivity across changes in task (Cole et al., 2013). Preliminary though they are, these data suggest a rich pattern of causal influences between MD and other brain regions as attentional episodes are established and executed.

The MD System in Abstraction, Intelligence, and Control

A critical aspect of human cognition is the capacity for abstract, symbolic thought. In line with the evidence previously reviewed for prefrontal coding of abstract stimulus categories (Freedman et al., 2001; Roy et al., 2010), the frontal lobes have often been considered the seat of abstraction (Weigl, 1941; Goldstein and Scheerer, 1941). Indeed, there is much in common between the concepts of abstraction and attention. To define an abstraction, some common element must be isolated from multiple exemplars. In other words, this common element is isolated from all those accompanying and varying features that distinguish one exemplar from another. On this line of reasoning, it makes sense that a facility for selective attention to any arbitrary but isolated aspect of some cognitive representation is also a facility for abstraction. The facility is of course critical to rational thought, where conclusions (e.g., conclusions in a trigonometric proof) follow from focus on specific, selected aspects of a current situation (e.g., equality of two angles) and the exact implications of these selected facts for others (e.g., congruence of two triangles) (see e.g., Duncker, 1945; Simon, 1981).

In fMRI, the MD pattern implies cognitive functions of importance in many different kinds of activity. The existence of such functions is strongly reminiscent of a core finding in psychometrics, generally termed “positive manifold”: for any battery of cognitive tests, administered to a large and representative sample of the normal population, the entire matrix of between-task correlations will be positive, showing that, to some degree at least, people performing well on one test tend also to perform well on others (Spearman, 1904, 1927). For some kinds of measure, e.g., simple components of a highly familiar real-world skill (Duncan et al., 1993), correlations can be close to zero; the general principle of positive manifold, however, holds across an extremely wide variety of laboratory and real-world activities. A variety of explanations for positive manifold may be imagined (e.g., Thomson, 1951; van der Maas et al., 2006). Indeed, it is plausible that multiple factors contribute to positive correlations, including genes with widespread influences on brain function, mutual dependencies between lifetime experiences, etc. According to the present analysis, however, one major factor may be the role of MD regions in controlling all kinds of complex behavior through a structure of attentional episodes. This proposal of a common function in all kinds of behavior matches Spearman's own suggestion that some “general” or g factor contributes to success in all manner of activities (Spearman, 1904).

In the context of Spearman's theory, it is easy to determine which tests are best correlated with (i.e., are the best measures of) g (Spearman, 1904). Essentially, these are the tests that show the strongest pattern of positive correlations with other measures, i.e., the strongest overall ability to predict success in other laboratory and real-world activities. The best measures of g are generally tests of novel problem solving, such as series completions, analogies, and matrices (e.g., Raven et al., 1988; Institute for Personality and Ability Testing, 1973), commonly termed tests of “fluid intelligence.” On Spearman’s account, such tests should provide an especially strong clue to cognitive and brain functions underlying positive manifold and g.

A close link of fluid intelligence to MD function is strongly supported by functional imaging. A contrast of fluid intelligence tests with sensorimotor controls shows a strong and selective pattern of MD activity (Prabhakaran et al., 1997; Duncan et al., 2000; Bishop et al., 2008). The literature on brain lesions is more complex. For many years, indeed, it was accepted that frontal lobe lesions, though leading to a widespread impairment of “executive” or control functions and an accompanying broad disorganization of behavior (Luria, 1966), were essentially unrelated to psychometric “intelligence” (Hebb and Penfield, 1940; Teuber, 1972). The conclusion is paradoxical, since any functions of significance across tests of many different kinds, providing they show significant individual differences, should by definition contribute to positive manifold and g. Outside neuropsychology, indeed, broad control functions such as strategy optimization have sometimes been proposed as a plausible basis for g (Marshalek et al., 1983). In part, early conclusions from patient studies may have reflected failure to draw the important distinction between fluid and crystallized intelligence (Cattell, 1971). In contrast to novel problem solving, crystallized intelligence concerns previously acquired knowledge such as vocabulary and is often much less sensitive to brain damage (Cattell, 1971). Tests of fluid intelligence, such as the Culture Fair, show especially strong deficits after frontal lobe lesions (Duncan et al., 1995) but have often not been used in neuropsychological work. In part, too, early conclusions may follow from relatively crude comparisons, e.g., between “frontal” and “posterior” cortical lesions. Based on functional imaging results, fluid intelligence deficits should be linked not to frontal or parietal lobe lesions in general but to damage within the specific MD system. The prediction is confirmed in recent work, with fluid intelligence loss significantly predicted by volume of damage within but not outside the MD system (Woolgar et al., 2010; see also Gläscher et al., 2010).

In the normal population too, correlations of intelligence test performance with anatomical measures, including cortical thickness and local gray or white matter volume, most commonly implicate frontal and parietal lobes (Jung and Haier, 2007), though across studies, correlations have been reported for many different cortical and subcortical regions (Jung and Haier, 2007; Deary et al., 2010). To interpret such data, it is important to consider not just the correlation of behavior with anatomy in individual brain regions or voxels but also the correlation in anatomy between different regions (see Alexander-Bloch et al., 2013). It is hard, for example, to interpret a performance correlation with gray matter volume in any one voxel X, if volume in this voxel is also correlated with volumes in a different region Y, and questions of this sort require increased attention in future work. Additional relevant data come from analysis of connectivity, either “functional connectivity” (temporal correlations) in resting state fMRI or white matter connectivity measured with diffusion tensor imaging. Several results suggest that more efficient or long-range connectivity is associated with higher intelligence test performance (Li et al., 2009; van den Heuvel et al., 2009); MD regions are among the most globally connected regions of the brain (Cole et al., 2010b), and for lateral frontal cortex, the strength of both positive and negative temporal correlations covaries with fluid intelligence (Cole et al., 2012). Taken together, the evidence from functional and structural imaging suggests some link of g to MD function (Duncan, 1990; Kane and Engle, 2002) but with more to be done in integrating findings from different sources.

In neuropsychology, many different tests have been designed as measures of executive impairment after frontal lobe lesions. Often, impairment is interpreted with close reference to specific test content. In the Wisconsin Card Sorting Test (WCST), for example, the patient must learn to sort cards according to one stimulus feature (e.g., color), then later switch to another (e.g., shape); commonly, deficits in frontal patients (Milner, 1963) are ascribed to impaired switching of cognitive set. In verbal fluency, the patient must generate as many words as possible from a specified phonological or semantic category; often, the frontal impairment (Benton, 1968) is interpreted as a failure in spontaneous generation of new search strategies. In general, however, the convergent validity of such concepts has been disappointing. The critical result would be, for example, that multiple measures of set switching on the one hand, and of strategy generation on the other, converge to define two coherent, dissociable, and general kinds of executive deficit in frontal lobe patients. Evidence of this sort is largely lacking (see e.g., Robinson et al., 2012; for some suggestive evidence from studies of normal individual differences, see Miyake et al., 2000; Friedman and Miyake, 2004).

By the principle of positive manifold, of course, all tests, including any putative test of executive function, will be positively correlated with fluid intelligence. Given deficits in fluid intelligence following specific (MD) lesions within the frontal lobe, one may ask how far this deficit alone explains impairment in other neuropsychological tests. For some tests, indeed—including such traditional executive tests as WCST and verbal fluency—differences between patients and controls are largely or entirely removed after correction for the difference in fluid intelligence (Roca et al., 2010, 2012, 2013; for an additional, specific influence of left posterolateral frontal damage in verbal fluency, see Robinson et al., 2012). In such cases, it seems unlikely that impairments should be interpreted in terms of specific test content, since the deficit measured is actually much broader. Importantly, in contrast, other tests show deficits that are not explained by fluid intelligence. Two such cases are tests of social cognition/theory of mind (Stone et al., 1998) and tests of open-ended planning in a complex, relatively real-world task setting (Manly et al., 2002; see Shallice and Burgess, 1991). In both cases, functional imaging data would suggest dominant foci of frontal lobe activity well outside the MD system (Gilbert et al., 2006) and, indeed, the most relevant lesions may be much more anterior, around the frontal pole (Roca et al., 2010). Plausibly, removing the common effect of fluid intelligence, affecting results for tests of all kinds, may increase the clarity with which separate frontal deficits, associated with lesions outside the MD system, can then be identified. Meanwhile, it seems likely that, in many conventional tests, impairments largely or wholly reflect not specific executive functions, but the global importance of the MD system in managing any structured behavior as a series of attentional episodes.

Attentional Episodes in Goal Neglect

A link of attentional episodes to fluid intelligence is also seen in a striking form of performance failure I have called goal neglect (Duncan et al., 1996). Goal neglect is manifest in a mismatch of knowledge and behavior: the person understands and can describe task requirements, but these requirements seem not to control what is actually done. Goal neglect is well known in patients with major frontal lobe damage (e.g., Luria, 1966; Milner, 1963). For example, the patient may be asked to squeeze the hand when a light is seen; when the light is switched on, the patient says “I must squeeze!” yet in fact does nothing (Luria, 1966). Even in the normal population, however, a similar goal neglect can be seen when task requirements are complex. Again, the participant understands task rules, and in principle can obey them, but often a whole task is completed as if one or more rules did not exist (Duncan et al., 1996). Asked at the end of the experiment, neglecting participants report “not looking out for” some critical triggering conditions for the neglected rule, or seeing the relevant cues but “letting them go over my head.” Critically, neglect is strongly correlated with fluid intelligence—more strongly, for example, than several varieties of working memory (Duncan et al., 2012). The result suggests substantial overlap between critical cognitive limits in goal neglect and in standard fluid intelligence tests.

Task complexity is important in goal neglect and, critically, this is not the complexity of actual performance. Instead, the critical factor is complexity of initial task instructions. An experiment making this point (Duncan et al., 2008) is illustrated in Figure 6. On each trial (Figure 6A), two streams of symbols (asterisks, letters, and numbers) flashed up simultaneously on a computer screen, one to the left and one to the right of center. Rate in each stream was 400 ms/symbol. Streams were preceded by a verbal instruction to watch left or right, and on the attended side, there were two tasks to carry out—repeating each letter aloud and adding each pair of numbers. Near the end, a brief central arrow indicated the side to attend for the remainder of the trial. For the trial in Figure 6A, for example, a correct report would be “F, T, 5, B, L,” though on other trials, the arrow would call for a switch of sides. In tasks of this sort, it is the final side cue (here, an arrow) that is often neglected. Most commonly, neglect takes the form of completing all trials as though the final side cues did not exist, though participants can repeat at the end of the task what these cues required and, if alerted trial-by-trial to their errors, can immediately begin to perform correctly (Duncan et al., 1996).

Figure 6.

Goal Neglect and Instructional Complexity

(A) Stimulus sequence for a single trial in Duncan et al. (2008). Two streams of symbols (asterisks, letters and numbers; 400 ms/symbol) flashed up simultaneously on a computer screen, one to the left and one to the right of center.

(B) Mean goal neglect score for reduced- and full-instructions participants binned by Culture Fair IQ. A score of 0.5 indicates complete neglect of arrow cues; 0 indicates perfect performance. Adapted with permission from Duncan et al. (2008).

To examine the effects of complexity, the task can be simplified in various ways. One way is to instruct the participant, for a whole block of trials, that no numbers (letters) will appear, so that only the letter task (number task) need be borne in mind. This has no effect on neglect of arrow cues; neglect is insensitive to changing task complexity during actual performance. A different way is to fix complexity at the time of performance but now in the context of varied instructional complexity. One group is given pure letter or pure number instructions (Figure 6B, reduced instructions). The other (Figure 6B, full instructions) is given both letter and number instructions, and a single mixed practice trial, and then told that until further notice, one type of trial (letter or number) will not appear. Though all data are finally collected in pure letter or pure number trials, groups differ substantially in the rate of arrow neglect (Figure 6B). These data show a limit in use of complex task instructions to shape correct behavior.

Why should such a complexity limit occur? Neglect suggests that arrow cues do not trigger attention to the correct task requirement, reminiscent of the general argument that complex problems must be solved by appropriately focused attentional episodes. In the case of goal neglect, task instructions define a new body of knowledge, specifying correct behavior in this novel task domain, and in correspondence to classic problems in artificial intelligence (Figure 1), the selection of correct behavior is increasingly challenging as overall problem complexity increases. With low fluid intelligence, it may be hard to achieve attentional focus on the correct part of a complex task space, causing critical task rules to be ineffective in control of behavior.

A recent study by Bhandari and Duncan (2013) supports this proposal. In this study, task requirements explicitly encouraged division into two rather unrelated subtasks. For one subtask, for example, stimuli might involve decisions about images of motor vehicles (e.g., car or motorbike, driver or none), while for the other, there might be images of books. Independently, each subtask was simple (few requirements) or complex (more requirements). Instructions for both subtasks were given together before performance began, followed by trials of the two subtasks in random order. Thus requirements for both subtasks were to be entered into memory before performance began; in this case, however, the materials encouraged storage and use in distinct cognitive chunks. As before, there were many cases of gross performance failure (failure to use an instructed rule), analogous to previous cases of goal neglect and again, strongly correlated with fluid intelligence and not generally explained by explicit forgetting of task instructions. Again, too, there were strong effects of task complexity but now restricted by subtask structure. For a given task requirement, performance declined substantially with increasing number of other rules within the same subtask. Plausibly, these would be the rules most difficult to exclude from a current attentional episode. In contrast, performance was independent of complexity in the accompanying subtask.

There is a second important aspect to the Bhandari and Duncan (2013) results. In cases of gross neglect of some task rule, we found that, most commonly, some fixed pattern of behavior replaced the correct rule. This fixed but incorrect rule might take several forms: neglect of a critical task event, a rule reversal, a rule imported from another task, etc. Whatever it was, we found that the rate of using this fixed but incorrect rule increased rapidly over the first few practice trials and then stabilized; at the same time, usage of the correct rule, though weak even on the first trial, showed a rapid decline. The results suggest a critical difficulty in building the correct structure of attentional episodes over the first few practice trials, as knowledge from task instructions is first used to shape behavior.

The need to divide complex activities into focused parts is obvious in any complex activity and certainly in the problem-solving tasks used to measure fluid intelligence. In Figure 7A, for example, the problem may seem challenging if considered as an undifferentiated whole; problems of this level of complexity are frequently failed by people in the lower part of the fluid intelligence distribution. It is trivial once effectively divided into component subproblems, one focusing on color, one on shape, and one on size (Figure 7B). A second example is the embedded figures test, also strongly correlated with fluid intelligence (McKenna, 1984). Here, the problem (Figure 7C) is to find a target shape hidden in a camouflaging background. To find the target shape as a whole is challenging, and participants who perform poorly on this task may fail even after several minutes of search. The target is easily found by breaking the problem into subparts, e.g., a first search for the pointed tail, followed by search for an attached vertical edge, and so on. This reasoning suggests that the same limit in goal neglect—dividing a complex problem into a focused series of attentional episodes—may also be critical in standard fluid intelligence tests.

Figure 7.

Attentional Episodes in Complex Problem Solving

(A) Example matrix problem. The task is to decide which of the four response options (bottom) correctly completes the matrix (top). Complexity derives from concurrent variation in multiple stimulus features.

(B) Matrix problems with only a single varying feature in each matrix. Each problem is one component of the three-feature problem in (A) and appears trivial in isolation.

(C) Embedded figures problem. The task is to find the simple shape on top left hidden in the complex image below. A solution can be found by focus on successive parts of the target shape.

A variety of results links MD activity to task complexity, including complexity of new task instructions. In matrix problems, for example, MD activity is strongly modulated by the number of simultaneously varying stimulus attributes (Figure 7A versus Figure 7B; see Christoff et al., 2001). During new task instructions, there is phasic MD activity as new task rules are defined (Cole et al., 2010a; Dumontheil et al., 2011; Hartstra et al., 2011; Ruge and Wolfensteller, 2010), and as total task complexity increases with addition of successive rules, there is an accompanying increase in baseline MD activity (Dumontheil et al., 2011).

The Bigger Picture

All complex behavior must be organized in a structure of attentional episodes, dealing with one subproblem after another. The MD system, I have argued, plays a critical role in dividing complex problems into parts and hence in constructing component episodes.

Separation into focused parts, however, is only one part of the problem of complex behavior. A second challenge is to maintain parts in the correct structural relations, as when multiple subgoals are subordinate to an overarching supergoal (Miller et al., 1960), or when a line of reasoning is assembled from a structure of component arguments. Certainly, for example, task performance requires attentional episodes to be defined at different levels of abstraction, e.g., selecting a response rule and then applying this rule to a specific current stimulus (Badre and D’Esposito, 2009). Problems of hierarchical organization and structure have been extensively examined in symbolic artificial intelligence (e.g., Newell, 1990). The brain needs mechanisms not just for assembling attentional episodes but for maintaining their structural relations (Dehaene and Changeux, 1997).

Similarly, MD function must be put in the context of other systems contributing to behavioral organization. As noted earlier, both functional imaging and lesion data suggest somewhat different functions for MD and frontopolar regions (e.g., Koechlin et al., 2003; Roca et al., 2010). Parts of frontopolar cortex are incorporated within the well-known “default mode” network (e.g., Vincent et al., 2008; Power et al., 2011), whose activity increases with broad changes of cognitive perspective such as abandoning the present to recall a previous experience or seeing a problem from another’s point of view (Buckner and Carroll, 2007). Possibly, frontopolar activity is important in setting large-scale cognitive context, with the MD system constructing distinct, more local attentional episodes within this context (see also Sakai and Passingham, 2003; Cole et al., 2010a). A further topic not addressed here is the link between cortical and subcortical mechanisms of cognitive control, e.g., interacting control processes of frontal cortex and basal ganglia (Alexander et al., 1986).

The immense diversity of human behavior, I have argued, calls for principles that address how cognition in general is constructed and controlled. Consistent with activity during all kinds of tasks, I suggest that the MD system builds basic constituents for all complex cognition. In a series of attentional episodes, brain-wide processing activity is configured to satisfy the momentary, infinitely diverse requirements of complex thought and behavior.