Early visual cortex reflects initiation and maintenance of task set

Abstract

The human brain is able to process information flexibly, depending on a person's task. The mechanisms underlying this ability to initiate and maintain a task set are not well understood, but they are important for understanding the flexibility of human behavior and developing therapies for disorders involving attention. Here we investigate the differential roles of early visual cortical areas in initiating and maintaining a task set.

Using functional Magnetic Resonance Imaging (fMRI), we characterized three different components of task set-related, but trial-independent activity in retinotopically mapped areas of early visual cortex, while human participants performed attention demanding visual or auditory tasks. These trial-independent effects reflected: (1) maintenance of attention over a long duration, (2) orienting to a cue, and (3) initiation of a task set. Participants performed tasks that differed in the modality of stimulus to be attended (auditory or visual) and in whether there was a simultaneous distractor (auditory only, visual only, or simultaneous auditory and visual). We found that patterns of trial-independent activity in early visual areas (V1, V2, V3, hV4) depend on attended modality, but not on stimuli. Further, different early visual areas play distinct roles in the initiation of a task set. In addition, activity associated with maintaining a task set tracks with a participant's behavior. These results show that trial-independent activity in early visual cortex reflects initiation and maintenance of a person's task set.

Introduction

The ability to process identical information differently depending on the information's relevance to a particular task is an integral component to many human behaviors. For example, your response to a ringing cell phone is different while attending a seminar vs. sitting at your desk. A task set is the configuration of cognitive processes that is actively maintained for subsequent task performance (Sakai, 2008) and is one component of cognitive control or executive function (Diamond, 2013). A classic and well-studied example in which task set influences stimulus processing is attention. Attention enhances accuracy and reaction time (Bashinski and Bacharach, 1980; Posner et al., 1980), improves acuity and contrast sensitivity (Carrasco et al., 2004; Herrmann et al., 2010; Montagna et al., 2009; Pestilli and Carrasco., 2005) and reduces interference from distracters (Shiu and Pashler, 1995). Humans can flexibly switch and maintain task sets with remarkable accuracy (Chiu and Yantis, 2009; Sakai, 2008). Thus, the nervous system transforms identical inputs into drastically different outputs as a function of the individual's task set.

Understanding how the brain initiates and maintains these task sets will allow the field to better understand what goes wrong in disorders in which task sets are disrupted (e.g., Attention Deficit Hyperactivity Disorder, Alzheimer's disease (Perry, 1999), and other disorders). Here we examine the role of early sensory cortex in switching and maintaining a task set. In these experiments, the participant's task set is modulated through instructions to attend to stimuli of different modalities (visual vs. auditory). We find different patterns among the different early visual areas, and also find that task set-related activity predicts performance.

It is well known that attention to a visual stimulus or a location in space can modulate trial-driven activity in early visual cortex (reviewed in Carrasco, 2011). A cue to anticipate a visual stimulus, even in the absence of the stimulus, modulates visual cortical activity (Kastner et al., 1999), and the level of that modulation predicts the level of modulation of trial-driven activity (Murray, 2008; Sylvester et al., 2009). Silver and colleagues (2007) found that this modulation was sustained throughout the time that a participant anticipated a near-threshold stimulus. Aspects other than spatial attention can also modulate activity in early visual cortex. For example, activity in V1 can be modulated by attention to an auditory stimulus (Swallow et al., 2012), or by task structure (Jack et al., 2006). Thus it is clear that the visual cortex plays roles in addition to processing visual stimuli. In contrast to previous experiments, here we examine the differential role of early visual cortical areas in configuring a task set and maintaining that task set over a period of time.

The mechanisms that underlie different aspects of a task set should follow different temporal patterns (Donaldson et al., 2001; Konishi et al., 2001; Petersen and Dubis, 2011). These distinct temporal patterns reflect the function of the activity being measured. Activity associated with processing information needed on individual trials (trial-driven activity) should be time-locked to the presentation of the trials, and should be transient, dying out after the information processing for that trial is complete. On the other hand, activity associated with components of a task that are not driven by a trial should show different temporal patterns. For example, activity associated with maintaining a task set may be sustained throughout the time a participant maintains a task set, despite the fact that individual trials occur only briefly and intermittently. Activity initiating a task set should be transient and time-locked to the beginning of the task. Activity that processes a cue should be transient and follow presentation of the cue. Functional Magnetic Resonance Imaging (fMRI) can be used to dissociate different timecourses of neural activity, and can separate activity associated with processing individual stimuli, maintaining a task set, initiating a task set, or responding to a cue. The role(s) that a brain area plays in cognition are reflected in the timecourses of activity observed there. Several experiments have used this approach of dissociating the function of neural activity based on its timecourse; most of these have focused on the role of frontal cortical areas in executive control (Braver et al., 2003; Chawla et al., 1999; Donaldson et al., 2001; Dosenbach et al., 2006; Velanova et al., 2003; Wenger et al., 2004). Here we use a similar approach, but we examine the role of early visual cortex in these distinct aspects of setting up and maintaining a participant's task set. No previous work has directly compared task-maintenance, cue-driven, and task-initiation related signals in retinotopically mapped early visual cortical areas.

Materials and methods

Participants

Twenty healthy right-handed participants took part in this study. Participants included 8 males and 12 females with a mean age of 26 years (range 19-32 years) who had normal hearing as measured using an Earscan 3 manual Audiometer (MicroAudiometrics Corp., Murphy, North Carolina, USA) and normal or corrected-to-normal vision (as measured with a Snellen eye chart). Participants were recruited through a campus wide advertisement. Recruitment procedures adhered to ethical standards as set and reviewed by the IRB at the University of Alabama at Birmingham. All participants provided a written consent prior to admission to the study. The study consisted of a total of 3 sessions: an initial behavioral measurement and two subsequent MRI sessions.

Task

During the fMRI experiment, participants performed attention demanding discrimination tasks in which they had to correctly discriminate between two successive auditory or visual stimuli. The four task conditions differed in attended modality (attend to Auditory vs. Visual) and whether or not auditory and visual stimuli were presented simultaneously. The tasks are described in Figure 1A. Participants indicated with a button press whether two successively presented stimuli were the same or different. During the Auditory Unimodal (AU) and Visual Unimodal (VU) conditions, either auditory or visual stimuli were presented alone. During the Auditory Bimodal (AB) and Visual Bimodal (VB), conditions, both the auditory and visual stimuli were presented simultaneously and the participant discriminated between the stimuli of only the cued modality. During these bimodal conditions, the unattended stimuli followed a random pattern so that the participant could not gain an advantage by paying attention to the irrelevant stimulus.

Figure 1.

A. The task. Diagram shows one trial of the “Visual Bimodal” task. The fixation symbol (here a blue circle and cross) indicates which task the participant should perform (here, Visual Bimodal). The stimulus (here, simultaneous visual and auditory), mask, and probe are presented for 500 ms each. Participants are given 2000 ms to respond, indicating whether the relevant stimulus and probe are the same or different. Here, because the relevant stimulus is the visual one, the correct response would be ‘different.’ B. Event-related design. Blue and gold arrow pairs indicate cue and stimulus presentation, respectively; some cues were not followed by a stimulus and were used to estimate “cue-driven activity”. In this experiment, “cued runs” were presented with an event-related design. Example BOLD responses to these trials are illustrated below. C. Mixed blocked/event-related design. Arrows represent individual trials presented during task blocks. The design is used to estimate “trial-driven activity” associated with individual trials (blue), “task-maintenance” activity associated with the block (red) and “task-initiation” activity associated with the start of a block (purple). In this experiment, “blocked runs” were presented with a mixed blocked/event-related design. Example BOLD responses modeled in our analyses are illustrated below. Note that in the general linear model used here, the task-initiation effect models the hemodynamic response function's shift to a sustained value, whose level is modeled as the task-maintenance effect and is assumed to be constant during the final 54 seconds of the task block (red bar). The task termination effect (treated as a regressor of no interest), models the hemodynamic response function's shift back to baseline.

Throughout the task, participants were instructed to keep central vision fixed on the location of the fixation mark in the middle of the screen. In order to monitor compliance with these instructions (and to confirm participants did not adopt a strategy of, e.g., closing eyes during presentation of irrelevant visual stimuli), participants' eye movements were monitored during the experiment using an Eyelink 1000 fMRI eye tracking system (SR Research Ontario, Canada). Eye position was calibrated at the beginning of each run, and monitored throughout.

Stimuli and Trials

The auditory and visual stimuli were chosen based on previous work, suggesting that cortical information processing (such as topographic mapping and lateral inhibition) is analogous between visual and auditory stimuli (Shamma, 2001; Visscher et al., 2007). Trials contained two successive stimuli, either identical or different. Auditory stimuli varied sinusoidally in time and tone; these stimuli are often referred to as “ripple sounds” (Shamma, 2001). Stimuli that were ‘different’ were modulated with different temporal frequencies, while the identical trials contained exactly the same temporal frequencies. Visual stimuli varied sinusoidally in luminance over space. The stimuli were gray-scale horizontal gratings often called Gabor patches and were presented centrally. Visual stimuli that were ‘different’ varied from each other in the width of the gratings, while the identical trials contained exactly the same grating width. The Gaussian window defining the contrast of the bars in the Gabor patch had a standard deviation of 2.71 degrees visual angle. Stimuli were the same as those used in a previous behavioral study (Visscher et al., 2007).

Four different cues were used to indicate the upcoming task. A small white central fixation cross remained on screen during all runs when no other stimuli or cues were present. The cues appeared at the location of the fixation cross and were small and of similar luminance, in order to minimize bottom-up sensory processing in response to the cue. A blue circle indicated the Visual Unimodal task while a blue circle with a cross within it indicated the Visual Bimodal task. A yellow cross indicated the Auditory Unimodal task, while a yellow circle with a cross within it indicated the Auditory Bimodal task.

The timeline of a trial is schematized in Figure 1. First, the cue was presented, indicating which task was to be performed. The cue was followed by two stimuli, each with a duration of 500 ms. The two stimuli were separated by a noise mask for 500 ms. For the auditory stimuli, the mask was white noise, filtered to include similar temporal frequencies to the range of auditory stimuli. For the visual stimuli, the mask was a white noise pattern filtered to include spatial frequencies similar to the range of frequencies of the visual stimuli. A question mark replaced the fixation mark during the two seconds during which the participant could make a response.

Threshold Estimation and Stimulus Parameters

In order to standardize the perceptual difficulty of the task across participants, each participant's just noticeable difference (JND) threshold for auditory and visual stimuli were measured prior to the scanning sessions. Thresholds were defined using the QUEST algorithm (Watson and Pelli, 1983) as the stimulus difference (in units of % difference between two stimulus values) at which participants could correctly perform a forced choice paradigm 70% of the time. For Auditory tasks, participants were asked to identify which of two sequentially presented 500 ms stimuli had a faster temporal frequency. For visual stimuli, participants were asked to identify which of two sequentially presented 500 ms stimuli had a finer (‘thinner’) spatial frequency. The threshold estimation procedure was repeated in the scanner at the beginning of the fMRI session in order to control for the different presentation hardware used in the scanner. For logistical reasons and in order to save time, this was done with the gradient noise of the scanner off. JND values did not differ significantly between in-scanner and out-of-scanner measurements (data not shown). Auditory stimuli were presented at sound levels that were well discriminable over the sound of the scanner (scanner sounds were muffled by over-ear headphones). Performance was similar for Auditory and Visual tasks, as indicated by Figure 2.

Figure 2. Behavioral results.

A. Accuracy for each task during blocked runs (left) and during cued runs (right). There was no statistically significant difference in accuracy between Visual and Auditory tasks, and no statistically significant difference between Unimodal and Bimodal tasks. B. Reaction time for each task condition during blocked runs (left), and during cued runs (right). Participants were faster on Visual trials compared to Auditory trials (T-test p < 0.05). There was no statistically significant difference between Unimodal and Bimodal conditions. Within-subject error bars are shown.

Only participants with a JND threshold lower than 33% difference were invited for the fMRI sessions. Six potential participants were excluded because they did not meet this requirement. This requirement was added so that the range of presented stimuli would both encompass a range of discriminable stimuli and fall within our audio and visual capabilities.

During the Auditory Unimodal, Visual Unimodal, Auditory Bimodal, and Visual Bimodal tasks performed during scanning, if a stimulus and probe were different on a given trial, they differed by 4 JNDs - making the stimuli readily discriminable. This was stable throughout all runs for an individual. This procedure is meant to equate difficulty of the task across attended modality (Auditory and Visual) and across participants.

Visual or auditory stimuli used on a trial were chosen from a set of 18 possible stimuli, differing from each other by ½ JND. The mean JND value for auditory stimuli was 15% difference. For a participant with that JND, the temporal modulation of auditory stimuli ranged between 8.8 and 28.8 Hz. The mean JND value for visual stimuli was 16% difference. For a participant with that JND, the spatial frequency of the visually presented Gabor stimuli ranged between 0.3 and 0.89 cycles per degree visual angle.

MRI data acquisition

Twenty participants performed the Visual Unimodal, Visual Bimodal, Auditory Unimodal and Auditory Bimodal tasks in a 3T Siemens Allegra fMRI scanner. Whole-brain BOLD-weighted images were obtained with a TR of 2 s, TE of 30 ms, and a voxel size 3.75mm × 3.75mm × 4mm. The visual stimuli were presented using a rear projection screen located outside of the magnet bore. The screen was visible through an angled mirror attached to the head coil that was placed above the participant's eyes. The auditory stimuli were delivered to the first 7 participants through MR safe Etymotics ER 30 earphones, with additional MR compatible ear protectors. However, due to participant discomfort with the in-ear devices, their use was substituted with auditory stimuli fed through the Siemens sound system via specialized headphones for the final 13 participants (there was no significant difference in performance between the two earphone models used).

Scanning time was divided into two sessions of about 2 hours each, performed on different days no more than 2 weeks apart. At the beginning of each session, an anatomical MPRAGE scan was taken of a participant's brain, producing an image with a voxel size of 1.0mm × 1.0mm × 1.1mm. Retinotopic scans were performed during the first session. During the remainder of the first session and all of the second session, participants performed alternating runs following an event-related design and a mixed blocked/event-related design, which are described below and in Figure 1.

Event-related design

We used an event-related design for 8 of the runs (Rosen et al., 1998) to measure activity in response to transient cues independently from activity in response to cues followed by a trial (Figure 1B). The cues, changes in color and shape of the fixation cross, were presented for 2 seconds (1 TR) prior to the stimulus. Twenty five percent of cues were not followed by the stimuli to be discriminated (cue only trials). Both the cue only and cue+ stimulus trials were presented in a randomized order. The trials were presented at intervals varying (in units of 1TR=2s) between 4 to 14 s between the onsets of successive trials. This jittering of presentation allows estimation of the shape of the hemodynamic response. The order of intertrial intervals was determined pseudo randomly. Between trials, a white fixation cross was presented in the center of the screen. Participants were instructed to look at the fixation mark in the center of the screen throughout each run. Runs were each 276 TRs (552 s) long. Runs presented under these conditions will be referred to as cued runs. The event-related design allowed for measuring activity associated with transient shifts in task set, as task could change from trial to trial. To measure sustained activity associated with maintaining a task set over a period of time, we used a separate design called the mixed blocked/event-related design.

Mixed blocked/event-related design

In an additional 8 runs, interleaved with the cued runs, we used a mixed blocked/event-related design to measure trial-driven activity, task-maintenance activity, and task-initiation activity (Dosenbach et al., 2006; Petersen and Dubis, 2011; Visscher et al., 2003). This design involves the presentation of identical trials jittered in time during a block, as shown in (Figure 1C). The details of jittering were the same as in the event-related design described in the previous paragraph and allowed estimation of the shape of the hemodynamic response function. In this design, however, the fixation mark cue was presented at the beginning of a task block and the cue remained on the screen throughout the block (unless it was covered by trial stimuli). Thus, participants knew they were to perform the same task throughout the 70-second duration block, and could maintain a task set for that period. This design allowed us to use a general linear model to estimate different components of fMRI activity (Donaldson, 2004). There were a total of 8 mixed blocked/event related design runs, each lasting 251 TRs (502 s). Each run contained a total of 40 trials presented in 5 blocks lasting 35 TR's each. There were 8 trials per task block, and between blocks there were intervening rest blocks of 12 TR's (24 seconds).

Preprocessing steps

The BOLD images were preprocessed using MATLAB scripts using SPM8 (Friston et al., 1994). The images were slice time corrected, realigned, and re-sliced. They were normalized to an MNI EPI template using rigid body translation and rotation and then smoothed using a 5 mm kernel. The Matlab toolbox Artrepair (Mazaika et al., 2005) was applied to minimize artifacts caused by movement. Artrepair replaced images in which more than 0.5 mm/TR movement occurred with an interpolated image made from temporally adjacent images. Runs were excluded from the analysis if more than 16 images total or 6 consecutive images had movement above the threshold of 0.5 mm. Runs were additionally excluded if the displacement ever reached more than 3 mm, that is, if the difference between any two time points was greater than 3 mm. Each participant included in the analysis had at least 4 cued and 4 blocked runs that met our strict criteria. On average 16.25 % of runs were excluded because they did not meet our strict criteria.

GLM analysis

BOLD images were analyzed using SPM8 based on the general linear model (GLM) (Friston et al., 1994). Data for cued runs and blocked runs were analyzed separately. For the blocked runs, as shown in Figure 1C, we modeled trial-driven effects, task-maintenance effects, and task-initiation effects. Trial-driven effects were modeled with 12 regressors in a finite impulse response (FIR) model, representing 24 seconds following stimulus presentation. Task-initiation effects were modeled using 7 FIR regressors representing the first 14 seconds after the start of the block. We also modeled task-termination effects time locked to the end of a block, as 12 FIR regressors after the end of a block. We did not examine these task-termination effects closely, treating them as regressors of no interest. Task-maintenance activity was modeled as a shift in baseline throughout the last 54 seconds of the block, using a single boxcar-shaped regressor that started 16 seconds from the beginning of the block (immediately following the task-initiation regressors) and ended at the end of a block. The regressor for task-maintenance activity was not convolved with any canonical hemodynamic response function because it represents a stable shift in baseline. The task-initiation and task-termination effects modeled the hemodynamic lag in the rise and fall of the signal.

Detailed methods for analysis and validation that these different timecourses can be independently estimated can be found elsewhere (Dosenbach et al., 2006; Visscher et al., 2003). Because the trials are jittered in time with respect to the task-initiation and task-maintenance effects, the GLM can isolate trial-driven from non-trial-driven effects, even though they are superimposed in time (Visscher et al., 2003).

The cued runs were modeled in a separate GLM that independently modeled cue-only trials and cue+stimulus trials, as shown in Figure 1B. Each trial type was modeled with 12 regressors in a finite impulse response model (Miezin et al., 2000; Ollinger et al., 2001).

The outputs of these GLMs are plotted here in units of beta values. For effects that are modeled with an FIR design (trial-driven effects, cue-driven effects, and task-initiation effects), in order to determine one number to compare in bar graphs and voxelwise maps, we use the magnitude at the time of the peak hemodynamic response. This peak timepoint was identified based on the mean across all 4 task conditions of the absolute value of the difference between that timepoint and the value at time 0. The peak timepoint and ranged between 6 to 8 seconds depending on the time course. For trial-driven activity in the blocked condition, this was at 8 seconds following the trial; for trial-driven activity in the cued condition, this was at 6 seconds following the trial; for cue-driven activity, this was at 6 seconds following the cue, and for task-initiation activity it was at 6 seconds following the cue.

Retinotopy and region of interest definition

The visual cortex was retinotopically mapped using standard methods (Warnking et al., 2002). During three BOLD scans, at the end of the first MRI session, counter phase flickering checkerboard stimuli were presented to participants as wedges rotating clockwise, counterclockwise, and contracting circles, all with frequencies of 0.042 Hz for 10 cycles each. Wedges and circles extended to a maximum of 8.13 degrees eccentricity. This allowed assessment of the portion of the visual field to which a given voxel was most responsive. The retinotopic maps were generated using the Freesurfer retinotopy processing stream (Dale et al., 1999).

Early visual cortical regions of interest (ROI) were constructed using Freesurfer. Regions of interest were defined using the vertical and horizontal meridians as boundaries between areas, and eccentricity cutoffs of 1.0 to 5.42 degrees. The lower end cut off of 1 degree eccentricity was used to avoid the foveal confluence of the visual cortex, because the fovea is difficult to map using retinotopy (Schira et al., 2009). The upper end cutoff was used because it corresponds to a location on the stimulus where the contrast between light and dark bars was less discriminable (2 times the standard deviation of the Gaussian defining the contrast of the bars). Thus these regions represent the parts of early visual cortex selective for processing the central visual stimuli used in a trial. Regions of interest were created for these near foveal representations of the early visual areas, including ventral and dorsal V1, V2, V3 and hV4. For the purposes of this study the ventral and dorsal aspects of each of these visual areas are combined.

Results

Behavioral results

Accuracy was similar for all four behavioral conditions and for both cued and blocked runs (Figure 2A). Figure 2A and all other bar plots use within-subject error bars, which allow visual assessment of differences for these within-subjects designs (Cousineau, 2005; Loftus and Masson, 1994). An analysis of variance (ANOVA) on accuracy data during the blocked runs with factors attended modality (Auditory vs. Visual) and number of stimuli (Unimodal vs. Bimodal) showed no effect of attended modality (F_1,19=0.010, p= 0.9207) or number of stimuli (F_1,19=0.101, p= 0.7542). The results were similar for the cued runs: no effect of attended modality (F_1,19=0.0136, p= 0.7159) or number of stimuli (F_1,19=0.048, p= 0.8294). The absence of a difference in accuracy between tasks types showed that the Auditory and Visual tasks were relatively similar in difficulty.

An ANOVA with factors of attended modality (Auditory vs. Visual) and number of stimuli (Unimodal vs. Bimodal) showed that there was a significant difference in reaction time between Auditory and Visual tasks for both the cued (F_1,19=14.059, p=0.0014) and blocked (F_1,19= 15.54, p= 0.0008) trials, but there was no significant difference due to number of stimuli (Unimodal vs. Bimodal) in either the cued (F_1,19=1.883, p=0.1859) or blocked runs (F_1,19=1.883, p= 0.1893). Only correct trials were considered for this analysis (Figure 2B). Longer reaction times for the Auditory tasks may reflect the fact that the auditory stimuli varied along a temporal dimension, which may have caused participants to require more time to perceive the ripple sounds' frequency.

Imaging results

Trial-driven activity in early visual cortex depends on stimulus, but not attention modality

Trial-driven activity in retinotopically mapped early visual cortex was strong in all conditions in which visual stimuli were presented, regardless of attended modality (Figure 3 B and C; Visual Unimodal, Visual Bimodal and Auditory Bimodal tasks show robust responses). Regions of interest were defined based on retinotopic mapping and included V1, V2, V3 and hV4 (see Methods). For the two tasks with identical stimuli (Auditory Bimodal and Visual Bimodal), attention to the visual stimulus did not result in significant increases in trial-driven activity in any of the early visual regions. However, the whole brain analysis of Figure 4 shows that in higher-order visually responsive regions (lateral occipital cortex and parietal lobe), trial-driven activity is stronger when the visual stimulus is attended.

Figure 3. Trial-driven activity in early visual cortex depends on presented stimuli, and depends little on attention modality.

A. Trial-driven hemodynamic responses in the V1 ROI, averaged across participants. Arrow indicates the rise from baseline to the peak after 8 seconds. B and C. Trial-driven activity in the visual cortical ROIs during the blocked runs (which peaked at 8 seconds) (B) and cued runs (which peaked at 6 seconds) (C). Trial-driven activity was lowest during the Auditory Unimodal task, and highest during tasks involving visual stimuli, regardless of whether the visual stimulus was attended. Within-participant error bars are shown.

Figure 4. Whole brain contrasts for trial-driven activity.

Top panels show trial-driven activity for the Auditory Bimodal task and the Visual Bimodal task. Colors reflect the T-score resulting from a T-test comparing responses at time 0 and 8 seconds following the start of a trial. Warmer colors reflect activations, while cooler colors reflect deactivations. Lower panel shows the contrast between Visual Bimodal and Auditory Bimodal tasks; warmer colors indicate Visual Bimodal > Auditory Bimodal, cooler colors indicate the opposite. Left hemisphere only is shown with a liberal, uncorrected threshold of p< 0.001 in order to best convey the extent of activation and to illustrate that most regions exhibited similar trial-driven responses to the two tasks. Note in the bottom panel that a lateral occipital region shows stronger responses in the visual condition than the auditory condition.

Trial-independent activity depends on attention modality, not on stimuli

If early visual cortical areas are involved in setting up and/or maintaining attention to a visual stimulus, independently of stimulus presentation, then trial-independent activity in the visual cortex should be modulated depending on task demands. These data examined three measurements of neural activity that depend on a participant's task set, but are not time-locked to trials of the task. Two measurements were derived from blocked runs, where trials are presented in blocks throughout which the participant performed the same task (task-initiation activity and task-maintenance activity, Figure 1C). We measured activity reflecting the initiation of the task set when each block of trials begins (Konishi et al., 2001). This is referred to as task-initiation activity. Also, we measured shifts in baseline activity that were maintained throughout each block of the task, independent of the presence of a trial. We refer to this sustained shift in baseline as task-maintenance activity (Figure 1C).

A third measurement of neural activity that depends on task, but is not time-locked to performance of a trial, is the activity driven by cues (Figure 1B). In the cued runs, attention to a relevant stimulus was directed by a cue that orients the participant to the task to be performed on that trial. Neural activity driven by the cue includes activity associated with switching to the relevant task and preparing for the stimuli. The participant knew throughout a cued run that a trial of any task could be presented at any time. Therefore, cue-driven responses are distinct from the task-initiation responses, which represent activity time-locked to a transition from a ‘rest only’ condition to performing a task. Cue-driven signals should not be influenced by changes in alertness as the participant moves from a rest block to a task block, because there is no rest block during those runs.

First we tested whether all three types of trial-independent activity are modulated by task. Using retinotopically-defined regions of interest, we extracted task-maintenance activity, cue-driven activity and task-initiation activity in V1, V2, V3 and hV4 for all tasks. A three factor ANOVA with factors of attended modality (Auditory vs. visual), number of stimuli (Unimodal vs. Bimodal) and activity type (cue-driven activity vs. task-maintenance activity vs. task-initiation activity) showed no interactions between attended modality and activity type (p > 0.05) for all four visual areas. However, each visual area showed a significant main effect of attended modality: V1 (F_1,19=34.0, p<0.0001), V2 (F_1,19=27.1, p=0.0001), V3 (F_1,19=19.381, p=0.0003) and hV4 (F_1,19=14.4, p=0.0011). Unsurprisingly, there was also a main effect of activity type in each visual area as well, reflecting the distinctions among these components of neural activity: V1 (F_1,19 = 19.1, p < 0.0001), V2 (F_1,19 = 17.9, p < 0.0001), V3 (F_1,19 = 11.1, p = 0.0002), V4 (F_1,19= 15.2, p < 0.0001).

In setting up this experiment we were particularly interested in whether early visual areas showed similar or different activity associated with maintaining a task set vs. switching task set. We followed up on this a priori question by directly comparing task-maintenance activity and cue-driven activity. Another benefit of this direct comparison is that these data were derived from different runs and analyzed in different models, and therefore there is no way that these modeled effects could interact. A repeated measures ANOVA with three factors: attended modality (Auditory vs. Visual), number of stimuli (Unimodal vs. Bimodal) and activity type (cue-driven activity vs. task-maintenance activity) was applied to each visual area. There was a significant effect of modality in all four visual areas tested (Visual > Auditory): V1 (F_1,19=22.83, p=0.0001), V2 (F_1,19=21.49, p=0.0002), V3 (F_1,19=17.15, p=0.0006) and hV4 (F_1,19=14.89, p=0.0011), and no interaction between modality and activity type (p > 0.05 for all visual areas). This indicates that trial-independent neural activity – both cue-driven activity and task-maintenance activity – are strongly modulated by task in each early visual cortical region, and this modulation is similar for each component of activity.

Task-maintenance activity and cue-driven activity are modulated by attention modality

Task maintenance activity (Figure 5A), cue-driven (Figure 5B) and task-initiation (Figure 6A) effects are stronger during visual (magenta and red) than auditory (cyan and blue) tasks. Figures 5C and 5D directly compare the levels of modulation of cue-driven and task-maintenance activity, and Figure 6B compares the same modulation for task-initiation activity. Note that the patterns in Figures 5C and 5D are almost identical, indicating that the attentional modulations of cue-driven and task-maintenance activity are analogous in all regions of interest. As noted in the previous section, ANOVA showed no interactions between attended modality and activity type in any region (p > 0.05), lending statistical support to this visual similarity.

Figure 5. Task-maintenance and cue-driven effects depend on attention modality, not stimulus.

A. Task-maintenance activity in retinotopically mapped visual ROIs depends on a participant's task set. Attention to visual stimuli results in a larger task-maintenance effect when compared to auditory stimuli (AU & AB are larger than VU & VB). B. Cue-driven activity in visual ROIs depends on a participant's task set. Attention to visual stimuli results in a larger cue-driven effect when compared to auditory stimuli (AU & AB are larger than VU & VB). The height of the bar represents the change in signal between the start of the cue to the peak, 6 seconds later. C and D. Modulation in task-maintenance activity (C) and cue-driven activity (D) as a function of the attended modality. Yellow bars indicate the difference in activity between the unimodal tasks (VU-AU) while the green bars indicate modulation between the bimodal tasks where stimuli are equated (VB-AB). The effect of attended modality (visual vs. auditory) is similar for task-maintenance effects and cue-driven effects in all visual ROI. Within-participant error bars are shown.

Figure 6. Early visual cortical ROIs show distinct patterns of task-initiation activity.

Conventions are as in Figure 5. A. Bars indicate the change in task-initiation signal between the start of a task block to 6 s later. ANOVA showed significant interaction of task (Auditory vs. Visual) by visual area. The effect of attended modality (visual vs. auditory) is stronger in earlier areas (V1 and V2) than in later areas (hV4). B. Modulation in task-initiation activity as a function of the attended modality. Yellow bars indicate the difference in activity between the unimodal tasks (VU-AU) while the green bars indicate modulation between the bimodal tasks where stimuli are equated (VB-AB). Within-participant error bars are shown.

Patterns of activity are different across early visual areas

Patterns of non-trial-driven activity differed across the four visual areas we examined. We performed ANOVA with factors attended modality (Visual vs. Auditory), number of stimuli (Unimodal vs. Bimodal), and visual areas (V1, V2, V3 vs. hV4) for task-maintenance activity, cue-driven activity and task-initiation activity. In each case, the interaction of modality by visual area was significant, indicating that non-trial-driven activity has different functions in the different early visual areas.

For task-maintenance activity, shown in Figure 5 A and C, an ANOVA with factors attended modality, number of stimuli and visual areas showed a significant interaction of modality by visual area (p=0.0086). Follow up ANOVA with factors of attended modality and number of stimuli showed that there were strong effects of modality (Visual > Auditory) in V1 (F_1,19=11.27, p=0.0033), V2 (F_1,19=10.97, p=0.0037), V3 (F_1,19=9.51, p=0.0061) and hV4 (F_1,19=8.89, p=0.0077).

For cue-driven activity, shown in Figures 5 B and D, ANOVA with factors attended modality, number of stimuli and visual areas showed a significant interaction of modality by visual area (p=0.023). Follow up ANOVA with factors of attended modality and number of stimuli showed that there were strong effects of modality (Visual > Auditory) in V1 (F_1,19=26.49, p=0.0001), V2 (F_1,19=21.97, p=0.0002), V3 (F_1,19=18.17, p=0.0004) and hV4 (F_1,19=13.52, p=0.0016).

For task-initiation activity, shown in Figure 6, ANOVA with factors attended modality, number of stimuli and visual areas showed a significant interaction of modality by visual area (p=0.010); the figure shows that earlier visual areas have a stronger effect of attended modality than do later visual areas. Follow up ANOVA with factors of attended modality and number of stimuli showed that there were strong effects of modality (Visual > Auditory) in V1 (F_1,19=18.61, p=0.0004), V2 (F_1,19=10.23, p=0.0047), and weaker effects of modality in V3 (F_1,19=7.02, p=0.016) and hV4 (F_1,19=7.70, p=0.012).

Task-maintenance activity in V1 predicts behavioral performance in the visual task

To examine the degree to which task performance depends on fMRI activity, we planned both a between-participant and a within-participant analysis, examining how changes in trial-independent activity relate to behavior.

We hypothesized that the ability to maintain attention to a task should influence task performance in the blocked condition. In principle, a person could switch tasks very well, but an inability to maintain that task set would result in overall poor performance in our blocked design. For this reason, we focused on sustained task-maintenance activity. We first examined the effect of sustained task maintenance activity on behavior by comparing higher performing to lower performing participants. We focused on activity in V1 because it is the first cortical area in the visual processing hierarchy and because, as can be seen clearly in Figure 5A, V1 shows strong task-maintenance activity during visual tasks. We also restrict our statistical tests to the Visual Bimodal vs. Auditory Bimodal conditions, because these are the conditions in which the stimuli were identical but the task was different. A person who did not maintain a task set well over time could be reminded of the appropriate attention modality in the Auditory Unimodal condition, for example, because the auditory stimuli were presented alone. Restricting this analysis to the Auditory Bimodal and Visual Bimodal conditions also decreases the number of factors in the ANOVA and removes any effect of stimulus, as the stimuli are the same in each condition.

While it is true that the stimuli were adjusted based on individual participants' perceptual thresholds measured prior to the experiment, that does not mean that task performance was the same for each participant. In fact, as for most experiments, we found that there was a range of performances. Performance on these types of match to sample tasks depend on a range of factors including the perceptual difficulty of discriminating two stimuli, but also the participant's ability to maintain the correct task set (e.g., remember whether to attend to vision vs. audition), and the participant's ability to remain alert throughout the task block. Because we standardized the perceptual difficulty of the task, performance differences are unlikely to arise because of variable perceptual difficulty, and are likely to depend on other factors (such as ability to maintain the correct task set or remain alert). We divided the participants into two groups: those who performed above 75 percent correct on both the Auditory Bimodal and Visual Bimodal tasks during the blocked runs, vs. those who performed less well on at least one of those tasks. These groups are labeled as Good and Poor performers. Participants who can effectively modulate their task set are expected to do well on both tasks (Good performers), and those who don't effectively modulate their task set may do more poorly on one or both tasks (Poor performers). This choice of threshold for behavior did not influence the effects observed; similar effects could be seen when a median split was used to discriminate good from poor performers.

We ran a two-way ANOVA on the task-maintenance effects in V1, examining the effect of the factors group (Good vs. Poor performers) and attended modality (Auditory Bimodal vs. Visual Bimodal; both conditions where the stimuli were identical). The results showed a significant interaction of the factors group and attended modality (F_1,18=4.84, p=0.041), indicating that good performers had a greater modulation of task-maintenance activity between the two tasks. Figure 7A illustrates this effect: Good performers modulate task-maintenance activity strongly between the two tasks, while Poor performers do not.

Figure 7. Task-maintenance activity influences behavior.

A. Task-maintenance activity is plotted as in Figure 5 for the V1 ROI. Data for “Good performers” and “Poor performers” (as defined by behavioral data) are plotted separately. We hypothesized that the ability to maintain attention to a task should influence performance in the blocked runs. Good performers exhibit strong modulation of task-maintenance activity between the tasks. On the other hand, poor performers show no modulation of task-maintenance activity between Auditory Bimodal and Visual Bimodal tasks. Within-participant error bars are shown. B. For a given participant, task-maintenance activity in V1 is stronger for groups of trials that were faster. Y-axis shows the difference in reaction time between the Visual Bimodal and Visual Unimodal conditions. X-axis shows the difference in task-maintenance activity for those conditions. The two variables are significantly correlated (p = 0.023). Participants who showed stronger task-maintenance activity in the Visual Bimodal task (larger values on X axis) were also faster on that task (smaller values on Y axis).

These across-subject effects are not due to a “difference in speed-accuracy tradeoff” between the groups. Analysis of response times for the two groups found no significant differences between them. For the Auditory Bimodal task, mean response time for Good performers was 1.03 s; mean response time for Poor performers was 1.16 s (T-test p = 0.11). For the Visual Bimodal task, mean response time for Good performers was 0.91s, mean response time for Poor performers was 0.96 s (T-test p = 0.27).

We also tested the behavioral significance of task-maintenance activity by relating task performance and task-maintenance activity on an individual participant level. To do this, we need an estimate in a given participant of task maintenance activity during trials for which they perform better vs. worse. We made use of the fact that the Visual Unimodal and Visual Bimodal tasks both required attention to the visual modality and they were similar in difficulty; they showed no differences in percent correct or reaction time, on average, across participants. On an individual participant basis, however, participants were faster on one task than the other. We chose to use reaction time rather than accuracy for this analysis because there was little variability in accuracy for a given participant. Our data showed no speed accuracy tradeoff, so they should not show different effects (Figure 2). We focused on the Visual Unimodal and Visual Bimodal conditions because we are observing changes in visual cortical activity related to the level of visual attention, and both these tasks require attention to the visual modality. Further, level of performance on each task is similar on average and may reflect variation in level of attention to the task. We hypothesized that during task blocks where a participant exerts more attention to vision, they will exhibit faster performance as well as relatively increased visual cortical activity.

For each participant, we calculated the difference in task-maintenance activity between the Visual Unimodal and Bimodal conditions, as well as the difference in reaction time between these tasks. We found a strong Pearson's correlation of these values across participants (Figure 7B, r = -0.50, p=0.024), meaning that those participants with faster reaction times on a given task also showed stronger V1 task-maintenance activity during that task.

Together, Figure 7 A and B show that variation in task-maintenance activity in V1 relates to performance, both between participants and within-participants.

Discussion

Our data show that cognitive control can influence task set-related and trial-driven neural activity differently. Non-trial-driven neural activity is strongly modulated by attentional modality, independent of the stimuli presented (Figures 5 and 6). Additionally, the level of modulation of task-maintenance activity predicts performance (Figure 7). These data add to previous work suggesting that early visual cortical areas contribute to a participant's task set (e.g., Carrasco, 2011; Luck et al., 1994; Silver et al., 2007). Here, in contrast to previous work, we dissociate different components of a participant's task set: task-initiation, task-maintenance, and responses to cues. We find neural activity in early visual areas reflecting each of these components. Early visual areas show different patterns of activity, demonstrating that these early visual areas differ in the degree to which they are involved in different components of a task set.

While is very unlikely that any one region or small set of brain regions would be able to initiate something as complicated as a full-blown task set, these data suggest that early visual cortical areas are involved in initiation and maintenance of a task set. The effects we see in V1, V2, V3 and V4h may reflect inputs from frontal and parietal areas involved in executive control, and/or may reflect recurrent feedback within an area or set of areas. Similarly, effects observed in frontal and parietal areas may reflect inputs from elsewhere. The brain is a distributed system, and as such, aspects of its function, including maintaining and initiating task sets, are likely to be performed by distributed brain networks. While neural activity in early visual areas certainly processes vision, that does not mean that this is the only function of activity there (Poldrack, 2006). We have known for some time that changes in task set (due to, for example, shifts in attention) can change the way that early visual areas respond to inputs. We hope that our contribution to the literature is to show that activity in early visual cortical areas helps maintain and initiate a task set. This rules out the possibility that higher-order areas exert attention to V1 only on a trial-by-trial basis, and leaves the possibilities that 1) higher-order areas maintain the connection to early visual cortex throughout task performance, or 2) at least some of the processes required for maintaining attention involve recurrent circuitry within early visual cortex, and higher order areas serve to initiate these long lasting processes.

Identifying function through timing of fMRI activity

Many experiments have classified the function of neural activity based on its timecourse. For example, in event-related fMRI designs, most experimenters examine neural activity that is time-locked to trials of a task, and therefore assumed to reflect processing of the information presented in that trial (Rosen et al., 1998). Other timecourses of neural activity that are not primarily driven by sensory stimuli are collectively classified as trial-independent activity. For example, cued designs (Ollinger et al., 2001) can examine activity driven by cues to adopt one task set or another. In a cued design, stimulus parameters are equated between different cue types to show that these cue-driven effects are driven, not by the stimulus properties themselves, but by the participant's preparation for the task. Using a mixed blocked/event-related design, we can separately extract task-maintenance activity, which is sustained throughout the task, and task-initiation activity, which is associated with developing and setting up a participant's task set (Dosenbach et al., 2006; Visscher et al., 2003). Thus, examining the timecourses of neural activity allows us to identify roles in different aspects of attention and task set.

Recent work using intrinsic optical imaging and electrode recordings have suggested that in visual cortex, trial-driven hemodynamic signals may be more tightly coupled to pyramidal cell spiking than non-trial-driven signals (Cardoso et al., 2012; Sirotin and Das, 2009). Thus, fMRI signals with different timing relationships to a task may reflect different psychological processes, and also may reflect different neural underpinnings. More work is needed to understand the degree to which trial-driven, cue-driven, task-maintenance and task-initiation signals may have distinct neural bases.

The experimental designs used here involve estimation of trial-driven effects that overlap in time with task-maintenance effects. Previous work has shown that these effects are separable using a GLM, because the trials are jittered in time and are modeled with a finite impulse response function (Dosenbach et al., 2006; Visscher et al., 2003). The data here reinforce this point. For example, in the Auditory Bimodal and Visual Bimodal conditions, trial-driven effects are almost identical in early visual areas (Figures 3 and 4). However, task-maintenance effects are distinct in these two cases, though the effects overlap in time (Figure 5). These data further support the use of the mixed blocked event-related design to robustly separate trial-driven signals from signals reflecting a participant's task set (Donaldson et al., 2001; Petersen and Dubis, 2011).

Activity in early visual cortex does more than process visual information

Standard models suggest that task relevant information is encoded within frontoparietal brain areas, which then interact with other brain areas such as early visual cortex that handle the more specific aspects of stimulus processing. However, the role of early visual cortex is not limited to processing stimuli. It is well documented that early visual cortical areas also show activity related to a participant's task set, but not the stimuli (e.g., Kastner et al., 1999; Silver et al., 2007; Somers et al., 1999). It has been unclear, however, how early visual cortical areas contribute to different components of producing an effective task set (e.g., task-maintenance, task-initiation, and orienting to a cue).

There are a range of types of contributions that visual cortical areas could make toward task performance. For example, higher order areas might send a single temporally constrained input at the time of a cue. Such a response might serve to set the system up to process visual inputs, without requiring a maintained change in neural activity within sensory cortex. Alternatively, higher order areas might set up sustained changes in activity in sensory cortex. A third alternative is that higher order areas send input on a trial-by-trial basis. Finally, some combination of these mechanisms may occur. Our experiments distinguish between these possibilities, showing that early visual cortical activity changes with task switching, and such activity is maintained throughout task performance. We also show that cue-driven signals in early visual areas, associated with task switching, show very similar patterns to sustained signals associated with maintaining task performance.

It is important to note that we do not argue that visual areas alone can initiate and maintain a task set. Certainly a network of frontal and parietal areas are necessary (but not sufficient) to accomplish that goal (Corbetta and Shulman, 2002; Ungerleider and Kastner, 2000). Activity within visual cortex is also involved in initiation and maintenance of task set. The functional contributions of activity in low-level areas such as V1 are sometimes ignored; any activity there is thought to represent stimulus processing. We hope that the current work will help to counteract this, making clear that activity within sensory cortex reflects both initiation and maintenance of a participant's task set.

Trial-driven activity in V1 to hV4 depends almost exclusively on the stimulus, not the task set

The stimuli used during the two bimodal tasks were identical. The only difference between the tasks was whether the participant's attention was directed to a visual or auditory stimulus. Lateral occipital regions, late in the visual hierarchy showed robustly more trial-driven signals when visual stimuli were attended (in the Visual Bimodal condition vs. the Auditory Bimodal condition, Figure 4). This is consistent with other data showing similar effects (Mozolic et al., 2008). Previous studies specifically examining attentional modulations of trial-driven activity in early visual cortex have shown mixed results and generally have manipulated attention within different parts of the visual field (Kastner et al., 1999; Moran and Desimone, 1985; Somers et al., 1999; Watanabe et al., 1998), rather than between auditory and visual stimuli, as we have done here.

Our early visual cortex data showed no significant difference in trial-driven activity between the two bimodal tasks used in our experiment. Instead, trial-driven activity depended on the stimulus presented, but not on the attention modality (it is small only for the Auditory Unimodal condition, Figure 3 B and C). This is important when considered in contrast to trial-independent effects on activity (Figures 5 and 6) whose response depended on task set, not stimuli. The results imply that information about the task set is carried in trial-independent signals in early visual cortex, not as much in trial-driven responses.

Task-maintenance activity depends on task set

Although the stimuli for the Auditory Bimodal and Visual Bimodal tasks were identical, participants' responses to those stimuli were not. To respond correctly to the stimuli, participants had to maintain the appropriate task set throughout the appropriate time period. Using the mixed blocked/event-related design we extracted fMRI activity associated with the maintenance of a task set. Our results show that task-maintenance activity in early visual areas depends on task demands rather than on the stimulus (Figure 5B). For example, if the participant's task was to attend to the auditory stimulus (Auditory Unimodal and Auditory Bimodal tasks), task-maintenance activity in visual cortex was similar (Figure 5A), regardless of whether the visual stimulus was presented. This suggests that there is activity within visual cortex (V1, V2, V3, hV4) that helps to maintain a participant's task set, regardless of the presence of visual stimulation.

Previous studies examining task-maintenance effects have concentrated on defining the higher cortical areas that maintain a persons' task set (Dennis et al., 2007; Dosenbach et al., 2006; Reynolds et al., 2004; Velanova et al., 2003; Wenger et al., 2004). For example, a prior study found task-maintenance and task-initiation signals in a group of frontal and parietal areas that are thought to be involved in cognitive control (Dosenbach, et al., 2006). The present paper shows that cortical areas even as early as V1 are involved in maintaining a participant's task set.

The level of task-maintenance activity in sensory cortex not only changed with task, but it also covaried with a participant's behavior. Figures 7A and 7B show that this effect occurs both across participants and within participants, reinforcing the point that task-maintenance activity underlies a participant's behavior.

Previous studies examining the behavioral consequences of neural activity just prior to stimulus presentation are consistent with this interpretation. Changes in pre-stimulus activity contribute to trial-to-trial variability in perception and performance in a multitude of recorded effects (e.g., Boly et al., 2007; Coste et al., 2011; Hesselmann et al., 2008; Sadaghiani et al., 2009). Non-trial-driven activity in early visual cortex also contributes to stimulus perception (Donner et al., 2008). These previous findings rely on stochastic fluctuations in activity. Here we show that a participant's task set can modulate activity in early visual cortex. Our data are consistent with the idea that the reason pre-stimulus activity in early sensory cortex predicts behavior is because early sensory cortex is involved in maintaining a participant's task set. When a participant's task set is not optimal, task performance suffers.

Cue-driven activity depends on task set

A separate set of runs allowed us to examine cue-driven activity, which reflects the switch of a participant's task set. The cue-driven effects we observed depended on the attentional modality that the cue implied, not the stimulus that would be presented (Figure 5B, Auditory Bimodal and Visual Bimodal conditions are different despite identical trial stimuli). This makes clear that preparatory activity in these early visual areas depends strongly on the participant's task or attentional set, and less on the anticipated stimuli.

Note that cue-driven effects for Auditory trials were frequently below baseline (less than “0”), indicating that the level of activity at baseline during the cued condition was somewhere between the level during Auditory and Visual cues. This implies that ‘baseline’ during the cued runs (“0” in Figure 5B) is distinct from the ‘baseline rest’ condition in the blocked runs (“0” in Figure 5A). This is an important distinction when comparing data from experiments with different designs. The data are consistent with the idea that, psychologically, the timepoints of ‘rest’ during the cued runs represent a time of alertness, during which the stimulus could be expected at any time, while the ‘rest’ condition in the blocked runs represents a more full rest where the participant knows that no stimuli will be presented.

Task-initiation activity

Initially, one might hypothesize that the neural signals in cue-driven activity and task-initiation activity would be analogous, because both timecourses are time-locked to cues indicating the participant's task. However, as noted, participants are likely to be much less alert during the resting block as opposed to the rest periods on cued runs. This means that the task initiation effects are time-locked to neural activity associated with switching from a resting, less alert, less aroused state to an alert state, as well as the factors shared with cue-driven signals: processing the cue and preparing for the particular task.

Figure 6 shows that there was strong task-initiation activity in all early visual cortical areas, consistent with a role for early visual cortex in initiating a task set. However, task-initiation activity is modulated by task in different ways across the visual areas (Figure 6, interaction of attended modality by visual area p < 0.05), suggesting that each of the early visual areas may be differentially involved in initiating task sets.

Interestingly, task initiation signals in the visual unimodal condition were generally greater than for the Visual Bimodal condition (Figure 6). This seems initially counterintuitive, as the Visual Bimodal task was more complex, requiring both preparation for a task-relevant visual stimulus as well as ignoring an auditory stimulus. The results show, therefore, that it is not simply the complexity of the task that drives task initiation activity. It is still an open question what is the precise function of task initiation activity in V1, but the data make clear that it depends strongly on attentional modality, and also on other factors.

Task-maintenance activity and cue-driven activity follow the same pattern of modulation

One purpose of using both of the designs described in Figure 1 was to identify whether the early visual cortical mechanisms of task switching and task preparation in response to a cue were distinct from mechanisms for maintaining a task set throughout many trials of a task. Previous research has shown that attentional cues produce transient activity in early visual cortex even in the absence of stimuli (e.g., Kastner et al., 1999). Here, we explicitly compared cue-driven effects to task maintenance effects. We are not concentrating on task-initiation effects because, as noted in the previous section, those signals likely include activity associated with changes in general level of alertness or arousal. We find that patterns of task-maintenance and cue-driven activity are similar in each of the early visual areas studied (see similarities in the direction of effect in Figures 5 C and 5 D). This suggests that cue-driven and task-maintenance activity may reflect the same process in early visual cortex. In both cases, the trial-independent but task set-related signal appears to represent a modification of ongoing brain activity to optimize task performance.

Importantly, it was not previously known that task-maintenance and cue-driven effects would follow the same patterns. In non-sensory areas, these two effects are often found independently. For example distinct parts of the prefrontal cortex are involved in task maintenance vs. task switching (Braver et al., 2003). However, we find that in early visual cortex, the effects of task-maintenance and cues are similar, suggesting that cue-driven signals and shifts in baseline throughout the task period (task-maintenance effects) play similar roles in visual processing at the level of early visual cortex.

Conclusions

The early visual cortex receives visual information from the external environment, and it is essential for processing visual input. Activity in this region is also essential for initiating and maintaining an individual's task set. Our study demonstrates that task-maintenance activity in visual cortex reflects the task to be performed and predicts behavior. Two components of neural activity that are task set-related but trial-independent – cue-driven and task-maintenance activity – serve similar roles in early visual cortical areas. This work demonstrates the importance of early visual cortex in flexibility of task performance.

Highlights.

• 3 measures of trial-independent but task set-related neural activity are compared

• Task set-related activity in early visual cortex reflects task and performance

• Cognitive control influences task set-related and trial-driven activity differently

• Early visual areas differentially reflect cognitive control