A Unified attentional bottleneck in the human brain

Abstract

Human information processing is characterized by bottlenecks that constrain throughput. These bottlenecks limit both what we can perceive and what we can act on in multitask settings. Although perceptual and response limitations are often attributed to independent information processing bottlenecks, it has recently been suggested that a common attentional limitation may be responsible for both. To date, however, evidence supporting the existence of such a “unified” bottleneck has been mixed. Here, we tested the unified bottleneck hypothesis using time-resolved fMRI. Experiment 1 isolated brain regions involved in the response selection bottleneck that limits speeded dual-task performance. These same brain regions were not only engaged by a perceptual encoding task in Experiment 2, their activity also tracked delays to a speeded decision-making task caused by concurrent perceptual encoding (Experiment 3). We conclude that a unified attentional bottleneck, including the inferior frontal junction, superior medial frontal cortex, and bilateral insula, temporally limits operations as diverse as perceptual encoding and decision-making.

Although the human brain computes a rich representation of the sensory world, it does not have sufficient processing power to fully analyze all of the information it receives (1). Attentional mechanisms must therefore select important aspects of the environment for additional processing while filtering out less salient information (e.g., ref. 2). Although attention makes the vast amount of sensory information impinging on our senses manageable, it does so at a cost, for it results in bottlenecks that can both block awareness of, and disrupt decision-making for, behaviorally relevant events. These limitations are most evident in dual-task settings, as the concurrent performance of two or more tasks usually leads to impairment in at least one of the tasks.

Evidence supporting the view that attention to one event can block awareness of another comes from the attentional blink (AB) paradigm, in which participants report two—typically visual—targets (T1 and T2) presented in a rapid stream of distractors. The AB refers to the profound deficit in the explicit perception of T2 when that target follows T1 by ≈200–500 ms (3, 4). Although a number of hypotheses have been proposed to explain the AB (5), several lines of evidence suggest that deploying attention to consciously encode an initial target can constrain awareness of additional targets (6). Moreover, the duration of the attentional blink is modulated by the encoding load of T1 (7). Based on such findings, it has been proposed that the attentional demands of encoding information into working memory constitutes a bottleneck in information processing (henceforth, the encoding bottleneck; ref. 8).

Unlike the AB, where capacity limitations manifest as a failure of awareness, the psychological refractory period (PRP) paradigm exposes the serial nature of decisional processes. In it, participants perform two speeded sensorimotor tasks separated by a variable stimulus onset asynchrony (SOA). The PRP effect refers to the increase in the reaction time (RT) to the second stimulus as the SOA decreases. This effect is thought to result from an amodal attentional bottleneck responsible for such central operations as response selection (henceforth, the central or response selection bottleneck; refs. 9–12).

Encoding and response selection limitations have historically been treated as distinct (13, 14). However, recent behavioral work using hybrid AB/PRP designs suggests that they may be two manifestations of a common attentional bottleneck. Speeded response selection tasks can cause substantial interference with a subsequent encoding task (15–17). Similarly, simply encoding information for later recall can postpone subsequent speeded tasks, with the degree of postponement dependent on encoding load (17, 18). Based on these findings, a “unified” attentional bottleneck subsuming both the encoding and response selection bottlenecks has been proposed (8, 18). However, this unified bottleneck interpretation has been challenged. In particular, one alternative account attributes the interference observed in hybrid designs to task switching (19, 20): Because hybrid paradigms use different tasks, they require a task set switch before Task 2 can be performed, which can only take place after Task 1 processing is completed. According to this hypothesis, independent bottlenecks exist for visual encoding and response selection—interference is only observed because of the task switch requirement.

The concept of a unified attentional bottleneck for perception and decision-making could receive support from neurobiological investigations. Unfortunately, electrophysiological studies have provided only mixed evidence for such a unified bottleneck: Although some event-related potential (ERP) studies of the AB, PRP, and hybrid paradigms are more or less consistent with the unified bottleneck hypothesis (21, 22), others are not (23). Moreover, given the limited spatial resolution of ERP, this technique is mute on the extent to which encoding and decision-making bottlenecks arise from the same neural networks.

To date, no functional neuroimaging studies have specifically tested the unified bottleneck hypothesis. However, should one exist, an excellent case can be made that it would include the prefrontal cortex (PFC). A number of investigations have implicated posterior lateral PFC in both the PRP (22, 24–26) and the AB (27–29), which has led to the suggestion that this brain region may correspond to the neural substrates of a unified attentional bottleneck (30). In recent support of this hypothesis, the same brain region has been shown to play a general role in the control of attention (31, 32). Moreover, several studies point to a key role for PFC, perhaps in consortium with parietal regions, in general attentional capacity limitations (33), decision-making (34), and in both visual-spatial processing and action selection (35), leaving it well suited to act as a coordination center for behavior and executive control (36–38).

Here, we test the unified bottleneck hypothesis with fMRI, asking whether the same brain regions, particularly within PFC, act as a bottleneck for both perceptual encoding and response selection. Our approach is not simply to demonstrate overlapping activation between tasks, because such overlap does not necessarily imply that encoding and response selection interfere with each other. Instead, the experiments are designed to reveal the neural correlates of interference between these two processes by capitalizing on the temporal nature of these multi-tasking limitations; encoding delays response selection, and vice versa (15–17, 39). We therefore conducted three time-resolved fMRI experiments and a behavioral experiment to test the unified bottleneck hypothesis. Experiment 1 isolated the brain areas involved in the response selection bottleneck (23), whereas Experiment 2 showed that these same regions were also sensitive to pure manipulations of encoding load (18, 39). Critically, in Experiment 3, these regions were probed again, this time with a hybrid dual-task paradigm where encoding postpones subsequent response selection (34), and activity again matched the predictions of the unified bottleneck hypothesis. Finally, a behavioral experiment verified that our encoding manipulation not only affected concurrent response selection, but also the conscious encoding of additional stimuli (see Experiment 4 in SI Experimental Procedures and Fig. S1), as expected of an attentional bottleneck that limits both conscious perception and decision-making (18, 39).

Results

Experiment 1—Isolation of the Response Selection Bottleneck.

Experiment 1 used a PRP-like paradigm (Fig. 1A) to isolate regions that are sensitive to response selection demands and exhibit the temporal characteristics associated with the response selection bottleneck (25). The three trial types used were as follows: single-task (ST) auditory-vocal (AV) trials where participants were presented with one of three auditory stimuli, each requiring a different speeded vocal response; single-task visual-manual (VM) trials where participants were presented with one of three faces each requiring a different speeded manual response; and dual-task (DT) trials in which participants performed both the AV and VM tasks simultaneously (0 ms SOA).

Fig. 1.

Experimental task designs and behavioral results. (A) Experiment 1 task design, which included single-task auditory-vocal (AV) trials; single-task visual-manual (VM) trials; and Dual-Task (DT) trials. (B) Reaction time (RT) results for Experiment 1. (C) Task design for Experiments 2 and 3. For both experiments, Task 1 consisted of an array of one or four letters to be encoded for later recall at the end of the trial. In Experiment 3 only, the encoding array was followed at an SOA of 250 or 1,850 ms by Task 2, a sound requiring a speeded online response. (D) RT results for Experiment 3.

Behavioral results.

As shown in Fig. 1B, participants responded more slowly on DT trials than on ST trials, for both the AV, F(1, 11) = 37.0, P < 0.0001, and VM tasks, F(1, 11) = 16.1, P < 0.003. Accuracy results showed a similar pattern: AV task (single: 97.2%, dual: 90.4%), F(1, 11) = 9.1, P < 0.02, and the VM task (single: 96.6%, dual: 90.8%), F(1, 11) = 13.1, P < 0.01. Overall, the experimental design was effective in generating large dual-task costs (sum for both tasks: 684 ms).

fMRI results.

Regions constituting a central amodal response selection bottleneck should have the following characteristics. First, given that they should not be modality-specific, bottleneck regions should exhibit a blood-oxygen-level-dependent (BOLD) response in each ST condition (AV and VM). Second, because response selection for the two tasks must occur serially in the DT condition (9), the BOLD response should not only be larger in the DT condition than in the ST conditions, it should also be of longer duration. Finally, because the concurrent performance of two sensorimotor tasks typically leads to the postponement of only one of the tasks (9, 24, 25), bottleneck processing should begin at roughly the same time in all three conditions. To test these predictions, we used time-resolved fMRI, which takes advantage of temporal aspects of the hemodynamic response to infer the temporal dynamics of neural activity (24, 25). Although the sluggishness of the BOLD response precludes measuring the absolute timing of neural events, it does have the temporal resolution to examine the relative timing of such events (40). Moreover, one can infer particular changes in the temporal characteristics of brain activity from distinct alterations in the shape and timing of the hemodynamic response. Thus, shifts in the onset of brain activity correlate with shifts in the onset of the BOLD response (41), whereas changes in the intensity of brain activation lead to changes in the magnitude of the BOLD response without affecting its duration (42). Finally, changes in the duration of neural activity affect the duration of the BOLD response (together with its amplitude because the BOLD response is a sum of brain activity over time), without affecting its onset latency (43). Thus, we can estimate how response selection (Experiments 1 and 3) and encoding manipulations (Experiments 2 and 3) affect the temporal dynamics of brain activity by quantifying the peak amplitude, peak latency (a reliable measure of brain activity duration; ref. 39), and onset latency of curve-fitted BOLD response time courses (Experimental Procedures).

We first defined regions of interest (ROIs) for each participant by isolating voxels in the statistical parametric maps (SPMs) that were significantly activated in each of the two single tasks (conjunction of the open contrasts AV+, VM+; Experimental Procedures). Of the 12 conjointly activated foci (Table S1), five met the remaining response selection bottleneck inclusion criteria—namely they showed increased amplitude and delayed peak latency, but no delay in onset latency in the DT condition relative to either ST condition. These ROIs corresponded to anterior superior medial frontal cortex (aSMFC), left intraparietal sulcus (IPS), left inferior frontal junction (IFJ), and bilateral insula, although the peak latency difference between the DT and AV conditions for the right insula was marginal (P = 0.051). Curve fitted time courses for the prefrontal regions (aSMFC, IFJ, and insula) are plotted in Fig. 2 (for raw time courses, including for IPS, see Fig. S2).

Fig. 2.

BOLD response amplitudes and latencies for the response selection (Experiment 1) and encoding bottleneck (Experiment 3) tasks. (A) Typical participant’s SPM of the conjunction of the AV open contrast and VM open contrast showing aSMFC and IFJ ROIs. (B) Left and right hemisphere insula ROIs. (C–F) Curve-fitted BOLD time courses across frontal brain regions for the AV, VM, and dual-task trials in Experiments 1 (Left) and 3 (Right).

All of the regions identified in Experiment 1 have been associated with response selection in general (22, 24,25–26, 44, 45), or specifically with the response selection bottleneck (22, 24,25–26). The involvement of the left IPS in response selection is also consistent with its purported role in motor attention (46).

Experiment 2—Perceptual Encoding Manipulation.

If encoding and response selection depend on a unified bottleneck, then the response selection bottleneck regions identified in Experiment 1 should not only be activated during an encoding task, that activation should also be sensitive to the load of the encoding manipulation. Both of these hypotheses were tested in Experiment 2. On each trial, participants were presented with a visual array of one (low load) or four (high load) masked letters for later recall (Fig. 1C). To dissociate encoding from response selection related activity, the response probe was presented 14 s after the onset of the visual array. Two slice prescriptions—one frontal and one parietal—were used to cover the response selection bottleneck regions of Experiment 1. The former overlapped all prefrontal regions, whereas the latter covered the lone parietal region (Experimental Procedures). Finally, a TR of 200 ms was used to achieve the fine-grained temporal resolution required for this time-sensitive paradigm.

For all participants, the candidate bottleneck ROIs were first isolated as in Experiment 1 (Experimental Procedures) and then probed for latency and amplitude effects tied to the encoding load manipulation. All five response selection bottleneck regions were activated under both high and low encoding load and showed effects of encoding load on both peak amplitude and latency (Fig. 3, Fig. S3, and Table S2). Thus, as expected by the unified bottleneck account, our response selection bottleneck regions were also sensitive to perceptual encoding. This result is also inconsistent with task switching accounts that posit independent neural substrates for encoding and response selection.

Fig. 3.

Curve-fitted and raw BOLD response time courses for bottleneck regions identified in Experiment 1 as a function of encoding load in Experiment 2. (A) Left IFJ. (B) aSMFC. (C) Left insula. (D) Right insula.

Experiment 3—Isolation of the Unified Bottleneck.

The results of Experiments 1 and 2 are consistent with the existence of a unified bottleneck for response selection and perceptual encoding. These results, however, do not demonstrate that the same brain regions track the timing properties of both perceptual encoding and response selection operations in dual-task situations, as expected of a unified bottleneck. The goal of Experiment 3 was to fill in this final piece of the puzzle.

Experiment 3 was identical to Experiment 2 except that the visual array was followed, at a short (250 ms) or long (1,850 ms) SOA, by a sound requiring a speeded manual response (Fig. 1C). If encoding and response selection depend on a unified bottleneck, then at the short SOA, Task 1 encoding should postpone Task 2 response selection and RTs to Task 2 should be sensitive to encoding load. At the long SOA, this pattern of effects will not be expected because Task 1 encoding will be complete before Task 2 commences. To prevent Task 1 response selection or execution from contaminating Task 2 processing, the Task 1 response probe was presented 14 s after the onset of the visual array. Sensory interference between the tasks was minimized by using stimuli from different modalities and by using SOAs that exceeded the interval at which to-be-ignored visual stimuli interfere with the subsequent processing of task-relevant visual stimuli (47). Finally, to maintain the statistical power and temporal resolution (200 ms TR) of Experiment 2 while accommodating the doubling of the number of conditions caused by introducing the SOA manipulation, spatial coverage was limited to PFC.

Behavioral results.

As shown in Fig. 1D, encoding a visual array for later recall slowed RTs to Task 2 (the speeded auditory-manual task), as evidenced by the main effect of SOA, F(1, 11) = 38.4, P < 0.0001. RTs were also slower when four letters were encoded compared with one, F(1, 11) = 29.7, P < 0.0003. Most importantly, replicating earlier results (21, 39), SOA and encoding load interacted, F(1, 11) = 6.5, P < 0.03: The effect of encoding load on RTs to Task 2 was larger at the short (233 ms) than at the long (77 ms) SOA, demonstrating that encoding in one task interferes with response selection in another task. Finally, encoding accuracy was better at low load than at high load (97.2% vs. 91.3%, F(1, 11) = 8.3, P < 0.02), as was tone accuracy, albeit slightly (98.2% vs. 96.9%, F(1, 11) = 6.1, P < 0.04).

fMRI predictions.

If there are brain regions supporting a common attentional bottleneck for response selection and encoding, one or more of the ROIs identified in Experiments 1 and 2 will have the following properties. First, these brain regions will be sensitive to the encoding demands of Task 1, showing an increase in peak amplitude with increasing encoding demands. Second, echoing the behavioral results of this experiment and the imaging results of Experiment 2, the duration of activity will reflect the duration of serial processing in the central bottleneck. Specifically, unified bottleneck areas should exhibit a large effect of encoding load on peak latency when the Task 1 - Task 2 SOA is short—because Task 2 response selection is further delayed by the high Task 1 encoding load—but little effect when the SOA is long—because the timing of Task 2 response selection should be unaffected by Task 1 encoding load. Finally, no onset latency delays are predicted because Task 1 is expected to begin at roughly the same time in all load and SOA conditions (quantitative timing predictions of activations generated from the time course data of Experiment 2 and the behavioral results of Experiment 3 are presented in Fig. S4).

fMRI results.

Four of the candidate unified bottleneck regions from Experiments 1 and 2 were within the coverage of Experiment 3 (aSMFC, left IFJ, bilateral insula), of which at least three displayed a pattern of results consistent with a unified bottleneck responsible for both response selection and perceptual encoding in Experiment 3 (see below for right insula). As shown in Fig. 2 (and Table S3), aSMFC, left IFJ, and left insula were all sensitive to encoding demand (main effect of encoding load on peak amplitude) and showed no onset latency effects. Moreover, all three also exhibited peak latency delays that were modulated by Task 1 encoding demand and SOA, i.e., a Load × SOA interaction, as predicted by the unified bottleneck hypothesis (the fourth ROI, the right insula, also showed the pattern of results expected of a unified bottleneck region, including the key Load × SOA interaction, but only a marginal effect of encoding load on amplitude; P < 0.07). Interestingly, the size of the encoding load effect on peak latency at the short SOA (average of 823 ms across ROIs, see Table S4) was comparable to that observed in Experiment 2 (843 ms), and both were considerably larger than the effect of encoding load on Task 2 RT at the short SOA observed in Experiment 3 (233 ms). This interesting effect may reflect, at least partly, a nonlinear relationship between the cognitive effect of encoding load and the hemodynamic response to that load (see also ref. 48), or perhaps extended attention to encoding when this process is heavily taxed by a high load.

In a final analysis, SPMs of Experiment 3 (open contrast) did not reveal additional prefrontal regions beyond those highlighted in Experiment 1 (Fig. S5). This result suggests that, at least within the coverage of Experiment 3, there are no potential unified bottleneck areas that were omitted by confining our analyses to the ROIs defined in Experiment 1.

Discussion

Our findings point to a common unified neural bottleneck for perception and action. Experiment 1 isolated a prefrontal network with the characteristics expected of the response selection bottleneck. All regions were (i) activated by both tasks independently, (ii) more activated on DT trials than on ST trials, and (iii) showed a peak latency delay, but no onset latency delay, commensurate with delays observed on RTs. In Experiment 2, each of the regions identified in Experiment 1 was also engaged by an encoding task and showed sensitivity to encoding load. In Experiment 3, all of these ROIs also displayed characteristics of a unified bottleneck for response selection and encoding. They were (i) more activated when encoding demand was high, and (ii) showed peak latency delays, but no onset latency delays, with increasing encoding load at the short, but not at the long SOA. Finally, Experiment 4 (SI Experimental Procedures) confirmed that this encoding load manipulation also limits the conscious perception of additional events (Fig. S1). Taken as a whole, these results are consistent with these prefrontal regions being involved in both conscious encoding of visual events and in decision-making processes based on these events. At the same time, they are inconsistent with task-switching accounts of processing limitations in perception and decision-making.

The inclusion of the IFJ in a unified attentional bottleneck is consistent with prior reports that this brain region is centrally involved in the response selection bottleneck revealed by the PRP paradigm (24, 25) and in limitations of conscious perception in the AB paradigm (27, 29). The IFJ has also been implicated in the retrieval of abstract rules (49–51), competitive selection (52), and the conversion of a stimulus representation, either from memory or the environment, into a response (49). The present study extends this notion by demonstrating that left IFJ is also involved in the encoding of perceptual information into working memory (53). It is therefore possible that the IFJ may be more broadly described as being involved in not only retrieving items from memory (as part of a response selection task), but also in encoding items into memory (for later conscious report). Hence, this brain region may have a general role in access to working memory (54).

In addition to the IFJ, the aSMFC, bilateral insula, and possibly the left IPS were found to be involved in the unified bottleneck. This network is in good agreement with other studies of the PRP paradigm (22, 26, 44) and with past work examining perceptual and response selection (33, 44). More broadly, the inclusion of these brain regions in a unified attentional bottleneck is consistent with their purported role in cognitive control (55), interference resolution (56), task set implementation (57), and the guidance of behavior (58). It should be noted, however, that the neural validation of a unified bottleneck does not imply that this is the only attentional bottleneck in the human brain. Indeed, there is much evidence for the coexistence of neural structures devoted to specific processes or stages of information processing (e.g., refs. 26 and 56), perhaps each with their own attentional limitations (20).

In conclusion, the results from the present study provide neurobiological evidence in support of the existence of a unified attentional bottleneck responsible for capacity limitations in domains as diverse as the encoding of perceptual information and response selection. The idea of a common neural bottleneck for perception and decision-making is particularly appealing because it suggests a general mechanism that controls the flow of information—a neural central processing unit critical for flexible task implementations (57). This suggestion is consistent with the notion that an overlapping set of brain regions adaptively codes task-relevant information (59). What the present results point to is the severe capacity limit of this adaptive coding system in implementing more than one task set at a time, thereby impeding our ability to consciously perceive, and appropriately respond to, successive events in the world.

Experimental Procedures

Experiment 1.

Trial design.

On each trial, participants (12, five female; aged 21–33) performed an auditory-vocal (AV) sound discrimination task, a visual-manual (VM) face discrimination task, or both tasks simultaneously (dual-task, DT). A black fixation marker (0.1° of visual angle) on a gray background was continuously presented at screen center to facilitate fixation. RT and accuracy were measured for each task. For the VM task, one of three grayscale male faces was presented at fixation for 200 ms (Fig. 1). Each image subtended 6.4° of visual angle both vertically and horizontally (see ref. 29). Responses were made with the right index, middle, and ring fingers. For the AV task, one of three complex sounds (see ref. 24) was presented binaurally over headphones for 200 ms. Sounds were easily distinguishable and audible. Each sound was mapped to one of three verbal responses—“Koo,” “Tay,” and “Dah” (counterbalanced across participants). On DT trials, one auditory and one visual stimulus were presented simultaneously (0 ms SOA). Participants were instructed to place equal emphasis on both tasks and to respond as quickly and accurately as possible (similar to ref. 25).

A trial began with 12 s of fixation, with the last 2 s including an enlargement of the fixation point to alert participants that the stimuli were imminent. Stimuli were presented for 200 ms, initiating a 4-s response interval, followed by a 200-ms postresponse period before the onset of the next trial. Participants undertook nine fMRI runs (because of discomfort, one participant performed only seven runs) each consisting of six AV, six VM, and six DT trials. Before the imaging session, participants took part in a practice session (SI Experimental Procedures).

fMRI data acquisition.

The fMRI session was carried out in a 3 T Philips Intera Achieva scanner at the Vanderbilt University Institute of Imaging Science. The visual display was presented on a liquid crystal display panel and back-projected onto a screen at the front of the magnet. Participants lay supine in the scanner and viewed the display on a mirror positioned above them. A Commander XG MR compatible headset (Resonance Technology) was used to present auditory stimuli and record vocal responses. Manual responses were collected with five-key keypads (Rowland Institute of Science).

Anatomical 3D high-resolution T1-weighted images were acquired with conventional parameters. Functional (T2*) parameters were as follows: TR, 1,200 ms; TE, 35 ms; FA, 70°; FOV, 220 mm; 64 × 64 matrix with 20 slices (4.5 mm thick, 0.5-mm skip) acquired parallel to the anterior commissure (AC)–posterior commissure (PC) line. Stimulus presentation was synchronized with fMRI volume acquisition, with every second trial commencing on a scanner pulse and the intervening trials beginning 600 ms (1/2 TR) after the scanner pulse. This interleaved design doubled the effective temporal resolution to 600 ms (see below).

RT data analysis.

For each participant and condition, correct trials were screened for RT outliers (60) resulting in the removal of 2.8% of trials.

fMRI data analysis.

Image analysis was performed on outlier-screened correct trials with Brain Voyager QX 1.8 (Brain Innovation) and custom Matlab software (MathWorks). Data preprocessing included 3D motion correction, slice scan time correction, and high pass filtering (<3 cycles per run). Preprocessing also included voxel-wise removal of intensity spikes defined as raw MR values that were greater or equal to four SDs from the mean MR value for a given run. The time points that met this exclusion criterion (≈0.1% of all timepoints) were corrected by linearly interpolating between the timepoints before and after the intensity spike. Anatomical T1-weighted and functional data were coregistered and then transformed into standardized Talairach space (61).

ROIs were isolated from an SPM of the conjunction of the two single tasks (VM open contrast and AV open contrast; see ref. 25) at a threshold of q(FDR) < 0.05. The ROIs consisted of a 6 × 6 × 6 mm³ region centered on the peak voxel of each activated foci. Time courses were extracted, temporally aligned (to compensate for the 0.5 TR offset in half the trials; see above), and normalized to a percent signal change from the two volumes preceding trial onset. To reduce time course noise for statistical testing of onset latency, peak latency, and amplitude differences, a gamma function (SPM2, http://www.fil.ion.ucl.ac.uk/spm) with three free parameters (amplitude, delay, and onset) was first fitted to the data for each condition by participant. Because the delay and onset parameters are not independent, the peak time and onset time were estimated from the curves directly; onsets were defined as the time at which the curve had achieved 10% of its maximal amplitude, with the time of this maximum defined as the peak (31). The onset and peak times of the fitted functions were compared for each of two conditions (VM vs. DT and AV vs. DT) across participants by using a Student’s t test (random effects model). The same was done for peak amplitudes. For visualization purposes, trials in each condition were combined to generate event-related averages (ERAs), which were then averaged across participants (Fig. 2 C–F).

Experiment 2.

Ten participants (aged 21–31, 7 females) took part in this experiment, four of whom participated in Experiments 1 and 3. One of the 10 participants was removed because of excessive head motion (>5 mm) and concomitant poor data quality.

Trial design.

On each 28-s-long trial, participants encoded an array of four identical (low load) or four unique (high load) letters for later recall. The letters were drawn randomly without replacement from all consonants except S and Z. Task performance was assessed with a probe letter at the end of the trial. If the probe was from the encoding array (50% likelihood), participants made a right index finger response; if not, they made a right middle finger response.

To facilitate fixation, a black dot subtending 0.1° of visual angle was continuously present at screen center over a gray background. Each trial began with a 10-s prestimulus fixation period. Five hundred milliseconds before the end of the fixation period, the fixation marker doubled in size indicating that the stimuli were imminent. The encoding array was then presented for 250 ms. Letters were positioned at the corners of an invisible rectangle (2.8° by 2.6° of visual angle, height by width) centered on fixation. A mask consisting of a % symbol was presented for 100 ms at each letter position immediately following the encoding array. All characters subtended 0.6° of visual angle and were presented in black Geneva font. A poststimulus fixation period extended for 14 s after the onset of the encoding array. The probe letter was then presented for 2 s during which responses were collected. Each trial ended with a 2-s posttrial period before the next trial began. The 14-s interval between encoding array presentation and response probe presentation (and the subsequent 2 s + 10 s interval) permitted the isolation of Task 1 encoding-related activity from other cognitive components such as response selection and execution. There were four trial types consisting of each combination of encoding load (low/high) and probe (pres/abs). Each trial type was presented twice per run (8 trials per run). A brief practice session was conducted to familiarize participants with the task before scanning.

Participants that did not take part in Experiment 1 performed a localizer run to isolate ROIs associated with the response selection bottleneck. The localizer consisted of blocks of single-task AV and VM trials as in Experiment 1. Each block consisted of 10 trials lasting 2.4 s. Stimuli were drawn randomly without replacement from a set containing four of each stimulus (12 in total) for the task in question. The localizer run began and ended with a fixation period. Block order was pseudorandom and fixed—AV-VM-AV-VM-VM-AV-VM-AV, with blocks separated by a 16.8-s fixation period. The fixation marker doubled in size for the final second of each fixation period to alert participants that a single-task block was imminent. ROIs were defined as in Experiment 1.

fMRI data acquisition.

fMRI data was acquired as in Experiment 1 with the following exceptions. Acquisition parameters: three slices (8 mm thick, 0.5 mm skip); TR, 200 ms; TE, 35 ms; FA, 30°; FOV, 220 mm; 64 × 64 matrix with brain coverage alternated between frontal and parietal lobes across fMRI runs (6 runs per slice coverage). PFC slices were acquired perpendicular to the AC–PC line with the most posterior slice passing through the AC. The parietal slices were positioned parallel to the surface of the posterior parietal cortex (hence, axial oblique to the AC–PC line) to cover the superior-inferior extent of the IPS.

fMRI data analysis.

Data preprocessing was as in Experiment 1 but with the additional step of skull removal to conduct 3D motion correction with FSL 4.0 (for efficient motion correction that includes edge slices, an important feature given the small slice number). Time courses were extracted from the individual ROIs. The signal associated with each trial was then transformed to percent signal change versus a baseline of the 10 volumes preceding trial onset (corresponding to a 2.0-s baseline period, similar to Experiment 1). For statistical testing of onset latency, peak latency, and amplitude differences, gamma functions were fitted to the data (from 0 to 10 s to avoid the second, response-related peak) and key parameters derived from these curves as in Experiment 1. No peak was found within the data submitted to the curve-fitting procedure (e.g., the best-fitting function had an amplitude of 0 or a peak time >10 s) for eight (of 80) of the ROI × Subject conditions. When this occurred, the ROI for that participant was removed from further analyses. Onset times, peak times, and peak amplitudes of the fitted functions were compared across participants by using an ANOVA on encoding load (high vs. low). For visualization purposes, ERAs were generated as in Experiment 1 (Fig. 3).

Experiment 3.

Trial design.

Experiments 1 and 3 used the same participants. Trial design was identical to Experiment 2 except that a two-alternative sound discrimination task requiring a speeded response followed the encoding array at an SOA of either 250 or 1,850 ms. The sounds, which were complex and presented for 200 ms, required a left index or middle finger response and were different from those used in Experiment 1, although they came from the same original set (22). There were 16 trial types consisting of each combination of SOA (250/1,850 ms), sound (2), encoding load (low/high), and probe (pres/abs).

Participants performed between 9 and 12 eight-trial fMRI runs depending on available scanner time. Each trial type was presented once every two runs, but otherwise conditions were assigned to trials randomly. The paradigm allowed for the acquisition of 18–24 trials of each of the key conditions (2 SOAs × 2 encoding loads). Before the imaging session, one practice session was performed (SI Experimental Procedures).

Data acquisition and analysis.

The behavioral data were first screened for RT outliers, resulting in the removal of 3.0% of correct trials. fMRI acquisition and preprocessing was identical to the PFC slice prescription used in Experiment 2. Time courses were extracted from the individual ROIs defined in Experiment 1. Data analysis then progressed as in Experiment 2. Peak times for one ROI × Subject combination (of 60) were not found in the submitted data. The onset and peak times, as well as peak amplitudes, of the fitted functions were compared across participants using a 2 × 2 ANOVA (encoding load × SOA). For visualization purposes, ERAs were generated as in Experiment 1 (Fig. 2 C–F).