Functional specialization and dynamic resource allocation in visual cortex

Abstract

We studied the spatiotemporal characteristics of cortical activity in early visual areas and the fusiform gyri (FG) by means of magnetoencephalography (MEG). Subjects performed a visual classification task, in which letters and visually similar pseudoletters were presented in different surrounds and under different task demands. The stimuli appeared in a cued half of the visual field (VF). We observed prestimulus effects on amplitudes in V1 and Cuneus relating to VF and task demands, suggesting a combination of active anticipation and specialized routing of activity in visual processing. Amplitudes in the right FG between 150 and 350 ms after stimulus onset reflected task demands, while those in the left FG between 300 and 400 ms showed selectivity for graphemes. The contrasting stimulus‐evoked effects in the right and left FG show that the former area is sensitive to task demands irrespective of stimulus content, whereas the left FG is sensitive to stimulus content irrespectively of task demand. Hum Brain Mapp, 2010. © 2009 Wiley‐Liss, Inc.

INTRODUCTION

Visual information processing needs to be fast; how fast it can be depends on how efficiently the information can be routed to specialized processing areas. This article will consider how routing of stimulus information through the visual cortex is affected by semantic properties of the information [Ahissar and Hochstein,1997; Pylyshyn,1999; Schyns and Oliva,1999]. Semantic category determines in which hemisphere stimulus information is preferentially processed. The left fusiform gyrus (FG), for example, is involved in processing of words and letters [Callan et al.,2005; Cohen et al.,2000; Flowers et al.,2004; Garrett et al.,2000; James et al.,2005; Joseph et al.,2003; Pernet et al.,2005; Polk et al.,2002; Vickier et al.,2007], the right FG in detailed visual structure [Garoff et al.,2005; Koutstaal et al.,2001; Marsolek,1995; Marsolek et al.,1992; Tarr and Gauthier,2000]. One aim of this study is to compare the precise timing of activation in the left and right FG and elsewhere in the visual cortex for letters and nonletters. To this aim, we studied evoked cortical activity of letter and nonletter stimuli, using the high spatiotemporal resolution of magnetoencephalography (MEG).

The precise timing of visually evoked activity in left and right brain areas depends on their location in the visual field (VF). To identify retinotopic responses in primary visual cortex (V1) and use them to trace the evoked activity through visual cortex to FG, we presented the stimuli in the left and right lower VF. We chose the lower quadrants because they evoke the largest V1 response [Liu and Ioannides,2006; Portin et al.,1999; Tzelepi et al.,2001].

The second aim of this study was to determine to what extent functional specialization in FG and elsewhere in the visual cortex, and in particular its temporal properties, depend on the context of stimulus presentation. Visual processing is often codetermined by the surrounding context of the stimulus [Jordan and Thomas,2002] and the task at hand [Abu Bakar et al.,2009; Kastner and Ungerleider,2000; Stins and van Leeuwen,1993; van Leeuwen and van den Hof,1991]. We varied both types of context, surrounding stimulus information and task. We are interested in the question, when during stimulus processing these context factors start to have effect in designated processing areas.

With respect to the surrounding stimulus information, we considered two possible effects: flanker (in)congruency [Eriksen and Eriksen,1974] and crowding [Bouma,1970; Hagenaar and van der Heijden,1986; Miller,1991; Toet and Levi,1992]. Incongruency is a form of conflict; an area recently implicated in conflict resolution is the Cuneus [Wittfoth et al.,2006]. We therefore consider the left and right Cuneus as areas of interest in our study.

Recent behavioral experiments [Lachmann and van Leeuwen,2004; van Leeuwen and Lachmann,2004] showed that congruency of surrounding shapes sometimes differently affects letters and nonletters. Subjects generally preferred congruent surroundings over incongruent ones when they categorized letters and nonletters into arbitrary categories. But when the task required fine shape discrimination, negative congruence effects were obtained: subjects preferred incongruent surroundings and this effect occurred for letters only. This effect was taken as evidence of a special visual processing style reserved for reading. However, these congruency effects were obtained with foveal stimulus presentations characteristic of reading [Rayner et al.,1980]. In our present MEG study, using parafoveal presentation, we may additionally, or instead, expect crowding effects for these stimuli. In crowding conditions, effects of a surround are always detrimental and independent of the congruency between the surround and the target. In the current study, we compared isolated presentation with congruent and incongruent surrounds, allowing us to observe both congruency and crowding effects.

The second context factor in this study was task. van Leeuwen and Lachmann [2004] showed that the task determines whether letters and nonletters differ in congruency effects. In the current study, subjects memorized arbitrary categories of letters and nonletters (see Fig. 1). We manipulated the response categories such that correct categorization depended either on global shape processing or on detailed inspection of stimulus features. Detailed representations of visual stimuli often evoke stronger activity in the right FG [Garoff et al.,2005; Koutstaal et al.,2001; Marsolek,1995; Marsolek et al.,1992; Tarr and Gauthier,2000]. We contrasted activity in the left and right FG to investigate whether and how changing task demands affect the region of functional specialization for letters, the left FG, or (also) other parts of the visual system, like the right FG.

Figure 1.

Task, stimuli, and experimental procedure. (A) Two task conditions. The subjects learned to discriminate between the two categories on a row. Response category pairs in the Shape task had stimuli with similar shape, but in the Identity task shapes differed. (B) Rows of target stimuli and their possible surrounds. (C) The temporal order in a trial. Subjects memorized the categories and trained their categorization. We cued the presentation location at a random latency (600–1,200 ms) before stimulus onset. The cue stayed on screen, while subjects decided what category the target belonged to. After the response a new location cue initiated the next trial.

Taken together, the factors of visual field, semantic category, flank context, and task demand span a wide range of what can vary in visual perception of graphemes and grapheme‐like structures. The results therefore provide a summary perspective of how perceptual variation is accommodated in low‐ and high‐level visual areas.

MATERIALS AND METHODS

Subjects

Six male volunteers participated with no reported history of neurological deficits. All subjects were right handed, had a right dominant eye, as determined by Porta's test, and normal or corrected‐to‐normal vision. Their ages ranged from 25 to 54 years with a median of 30 (SD = 10.4). Four subjects were naive about the experiment's purpose; two others were authors (GP and AAI, showing similar main effects to the others). The ethics committee of RIKEN had approved all procedures; informed consent was obtained before the experiment.

Materials and Design

Figure 1 shows the stimuli and task conditions. The target stimuli consisted of the letters A, E, L, and C, and four visually similar pseudoletters. Targets could be embedded in a surround, subtending 6 degrees of visual angle. We created eight response category pairs that consisted of one letter and one pseudoletter. Half the time, the two stimuli within a category were of similar shape (Shape task) and half the time they were not (Identity task).

Using a within‐subjects design, we fully crossed the factors Task (Identity task, Shape task), Stimulus Type (letter, pseudoletter), Surround (isolated, congruent, incongruent), and VF (left, right). We blocked the presentation of the Identity and Shape task runs to encourage strategy effects. Subject did four consecutive Identity and four consecutive Shape runs in a counterbalanced order such that half of them did the Identity task runs first. All runs had identical instructions and subjects always performed the same categorization task. Within a Task block of four runs, the order of the response category pairs (Fig. 1) was pseudorandom so that the stimuli differed from those in the preceding run. Each response category pair occurred once as Category 1 (responded to with the left index finger) and once as Category 2 (respond with right index finger) in a pseudorandom and unique order for each subject. Within runs the trial order was determined by random draws from the stimulus pool (n = 144) without replacement.

Apparatus

Subjects were seated on a supportive chair inside a magnetically shielded room. An Omega whole head 151‐sensor system (CTF Systems, Vancouver, BC, Canada) recorded the MEG. Stimuli were projected onto a 17‐inch display with a refresh rate of 96 Hz with a high‐luminance LCD projector (NEC HIGHlite 8000Dsx+, NEC Viewtechnology, Tokyo, Japan) located outside the shielded room. Trial presentation was controlled by the Presentation software (Neurobehavioral Systems, Albany, CA). Two optical sensors on the armrests registered the responses. A photodiode measured stimulus onset times.

We monitored vertical eye movements (electrodes 1 cm above and below the left eye), horizontal eye movement (electrodes 1 cm lateral to the left and right outer canthus), and heart function (electrodes on left and right wrists, left ankle, and lead V2).

Procedure

We instructed subjects that the experiment was about category discrimination. Before each run subjects memorized a pair of categories, consisting of four target items each. They were labeled 1 and 2, and were presented simultaneously as in Figure 1, on the left and right side of the screen. Next, subjects practiced the task in a 24‐trial training session (one repetition of each stimulus in each VF). A trial started with a VF cue in the center of the screen, “<” or “>.” We instructed subjects to fixate on this cue during the entire trial. The stimulus appeared in the cued VF after a random interval (600–1,200 ms), centered at 8 degrees eccentricity from the center of the screen. Cue and stimulus remained on screen until the subject responded “Category 1” (by lifting the left index finger), or “Category 2” (right index finger). During training visual performance feedback was provided, but during recording runs (n = 8) no feedback was given. A recording run had 144 trials, i.e. 6 repetitions of each of the 24 unique trials (two letters and two pseudoletters, each occurring in three Surround conditions and 2 VFs), and lasted about 4 min. We revealed and discussed the experimental purpose after the experiment.

RT Analysis

We analyzed RTs of correct responses in the experimental runs that were between 250 ms and 5 SD above the mean of pooled data. We analyzed median RTs by calculating a repeated‐measures ANOVA of the factors Task (Identity task, Shape task), Stimulus Type (letter, pseudoletter), and Surround (isolated, congruent, incongruent).

MEG and MRI Coregistration

High‐resolution anatomical images of each subject's whole head were taken with a 1.5‐T Siemens MRI system (T1‐weighted with a voxel size of 1 × 1 × 1 mm³). We defined a coil‐based coordinate system using three coils, attached on the scalp above the nasion and on the left and the right preauricular points. With these coils we monitored head position during a run. We repeated a run if movement exceeded 3 mm. The subject's head shape was scanned using a 3D digitizer (Fastrak, Polhemus, Colchester, VT) and a 3D camera system (Vivid 700, Minolta, Japan). The digitized head shape was fitted on the MRI to get a transformation matrix between coil‐ and MRI‐based coordinate systems using Rapid Form (INUS, Korea) and in‐house software [Hironaga et al.,2002]. The coregistration accuracy was manually checked and kept within 1–2 mm.

MEG Signal Recording and Processing

The MEG signal was recorded with sampling rate of 1,250 Hz and hardware filters set for low‐pass at 400 Hz. Off‐line we removed environmental noise by forming the third gradient of the magnetic field. We then removed bad channels from the recording (less than three channels per subject had to be removed). We used ICA [Lee et al.,2003] to identify and remove components resulting from the power line, heart, blinks, and high‐frequency environmental noise. Within each run we then averaged trials of the same condition from 200 ms before to 500 ms after photodiode onset.

MFT Analysis

Magnetic Field Tomography (MFT) is a distributed source detection method, producing probabilistic estimates for the nonsilent primary current density vector J(r,t) at each time slice of the MEG signal [Ioannides et al.,1990]. The MFT algorithm is a nonlinear solution to the inverse problem with optimal stability and sensitivity for localized distributed sources [Taylor et al.,1999].

For each subject, four hemispherical source spaces were defined with good coverage of the left, right, superior, and posterior parts of the brain. For each of the source spaces sensitivity profiles (lead fields) were computed from a spherical head model for the conductivity of the head. The center of the sphere was chosen by a best fit to the local curvature of the inner surface of the skull below a set of 90 MEG channels. MFT estimated activity separately from the signal corresponding to the 90 channels selected for each of the four source spaces. The spatially overlapping estimates from the four source spaces were combined and stored in a 16 × 16 × 16 mm³ grid of 8 × 8 × 8 mm³ voxels covering the entire brain. The MFT algorithm estimates the vector field for the current source density in the entire brain, and it is sampled at regular grid points for storage purposes. We applied MFT with time steps of 0.8 ms to each of the 24 averages in the eight runs of each subject.

ROI Definition and Regional Activation Curve Analysis

To define ROIs we statistically compared the evoked current density vector for each voxel and time‐point with that during the prestimulus period. This comparison was done across a 3.2 ms window on all MFT solutions within subjects. The threshold for statistical significance used was P < 0.05 (Bonferroni‐corrected).

To assess commonality of the evoked activity across subjects, we transformed individual MRI scans to Talairach coordinates [Talairach and Tournoux,1988]. We defined ROIs as regions that were significantly activated in the first 200 ms after stimulus presentation in at least five out of six subjects, within a 3.2 ms time‐window. For each subject we adjusted the ROI centers to fall at the center of the individual pattern of significant deflection from baseline. The ROIs were spheres of 1.5 cm diameter, except in V1 where the diameter was 1 cm.

Since the MEG signal is not sensitive to current sources in the radial direction, the local current‐density vector at each grid point has no component along the local radial direction so that it is essentially confined to two dimensions. The variation of the local current density vector can therefore be conveniently quantified and displayed using circular statistics [Fisher,1993; Ioannides et al.,2005]. We used circular statistics to determine the main direction of the current‐density vector describing the instantaneous evoked response in each ROI of each subject at the latency corresponding to the earliest statistically significant increase of activity relative to prestimulus activity.

For each ROI we defined the predominant current direction of the first strong evoked peak in the current‐density modulus as positive. We call this the main direction. The main direction does not have an a priori significance, but it is plausible that it corresponds to the first clearly identifiable feed‐forward wave of brain activity. We projected the instantaneous current‐density vectors within ROIs along the main direction to create regional activation curves (RACs). Positive values of the RACs correspond to currents along the main evoked direction. The MFT solutions is a continuous vector field, so strong nearby sources may contribute to the RAC amplitudes, if they happen to have a similar direction. Nevertheless, it should be stressed that the MFT algorithm is computationally intensive precisely because by construction it allows for solutions with sharp discontinuities [Taylor et al.,1999].

We calculated RACs for each condition within each run. We then averaged RACs across runs, resulting in 48 RACs per subject, one for each condition (n = 8) and ROI (n = 6). For each time point independently, we did a repeated‐measures ANOVA [Pinheiro and Bates,2000] with fixed factors of Task (Identity, Shape), Stimulus Type (letter, pseudoletter), Surround (isolated, congruent, incongruent) and VF (left, right) and subjects as random factor. ANOVAs were calculated from 200 ms before to 500 ms after stimulus onset at every sample, with a step size of 0.8 ms. To avoid false positives across time we set the thresholds for statistical significance for each effect such that the false discovery rate [Benjamini and Hochberg,1995; Genovese et al.,2002] was 0.05.

In addition to the factorial analysis, posthoc comparisons were done to validate specific differences between conditions. For these comparisons we calculated individual differences using the median RAC value across a latency of interest. We used the median RAC value across time to assure robustness against outliers. For prestimulus effects we used the entire prestimulus period (−200 to 0 ms); for poststimulus effects we determined the latencies using the ANOVA results. When the differences between conditions were inconsistent across subjects, we calculated the 95% confidence interval of the observed differences using a nonparametric bootstrap method (n = 1,000). This way we determined whether the observed differences could be distinguished from zero across observers. The statistical analyses of RACs and RT were done in R [R‐Development‐Core‐Team,2004].

RESULTS

Behavioral Results

The overall percentage correct was 93.9% (SD = 0.04); no speed‐accuracy trade‐off was observed. RT showed a main effect for Task, F(1,5) = 9.48, P < 0.05, with faster RTs in the Shape (mean 569.9, SD 51.3 ms) than in the Identity task (677.9 ± 76.4). This showed that the Identity task was more demanding, as expected, because the Shape task was supported by the visual similarity of the letter and pseudoletter. The main effect of Stimulus Type, F(1,5) = 11.69, P < 0.05, showed faster RTs to letters (607.2 ± 74.0) than pseudoletters (640.6 ± 91.8). The effect of Surround, F(1,10) = 7.51, P < 0.05, indicated that responses to isolated targets were fastest (615.4 ± 81.7) and those to targets with a surrounds slowest (congruent, 624.4 ± 93.0; incongruent, 631.9 ± 81.0). The RT difference between congruent and incongruent surround was not significant in a posthoc test.

We observed two marginal interaction effects. One was between Task and Stimulus Type, F(1,5) = 5.76, P = 0.06, suggesting that the RT advantage for letters over pseudoletters was larger in the Identity task than in the Shape task. The other interaction was between Task and Surround, F(2,10) = 3.60, P = 0.07, suggesting that congruent surroundings, compared to incongruent ones, increased RT more in the Identity task than in the Shape task. This effect may be weak due to the lack of power in the RT analyses over small numbers of subjects (n = 6); notably, however, it is consistent with van Leeuwen and Lachmann [2004], who observed contrasting effects of congruence for the Identity and Shape tasks.

Commonly Activated Brain Areas

Figure 2A displays a representative average MEG signal from one recording run, showing the characteristic M70 visually evoked response [Tzelepi et al.,2001] around 70–80 ms. We used statistical parametric mapping (SPM) to compare the prestimulus values of the modulus of the current‐density vector obtained with MFT with those in the poststimulus period. The SPM analysis was performed within subjects for each VF location and done without baseline adjustment, thus preserving sustained prestimulus activity.

Figure 2.

Average signal and average ROI locations. (A) The average signal across 140 trials in the 151 sensors in a single run for one subject. (B) The average ROI locations (in green) and the spatial extent of the area where activity differs from baseline (in red), in at least five out of six subjects. The pattern of commonly activated areas was computed in Talairach coordinate space and back‐transformed onto an individual MRI. The actual ROI locations were individually adjusted based on the individual deflections from baseline. The displayed ROI locations are the average locations across subjects; see also Table I.

We selected brain areas that showed significant change in their poststimulus compared to prestimulus activity in at least five out of six subjects. We observed the following pattern of common significant deflection from prestimulus values in the first 200 ms after stimulus onset (Fig. 2B). Stimuli in the bottom right VF evoked highly common (all subjects) activation in the dorsal part of the left V1 at around 64 ms. These deflections lasted until 90 ms after stimulus onset and were followed by common (at least 5/6 subjects) activation of the left Cuneus between 90 and 150 ms. The left FG showed common activation between 130 and 155 ms, and again at 175 ms, more medially.

Stimuli in the bottom left VF produced highly common and long‐lasting activation in the dorsal part of the right V1 (Fig. 2B), from 64 to 170 ms, and recurring around 190 ms. The initial V1 activation was followed by highly common activity in an extensive region of the occipito‐temporal cortex. A region of common significant activation was observed around 70 ms in the right Cuneus, continuing until 170 ms after stimulus onset. The right FG first displayed common activation around 100 ms, persisting uninterrupted for over 100 ms with slight location shifts on the medial–lateral axis.

From 170 ms onward, common activation was observed in a rather wide medial area that included several regions close to the center of the head. Although separate sphere centers were used for each hemisphere's MFT computation (so in principle activity from this part of the head can produce a measurable MEG signal), accurate localization in this area cannot be assured with the head modeling used here. We therefore chose presently not to analyze the activity in this area further.

Based on the common significant deflections, we defined six ROIs: two in the dorsal part of the calcarine, corresponding to left and right V1, and one in the Cuneus and posterior FG of each hemisphere. Table I lists the average Talairach coordinates of the ROIs. The radius for V1 ROIs was 10 mm; for Cuneus and FG it was 15 mm. For each ROI, we determined its main current direction based on the evoked MFT vectors of all contralateral stimulus presentations in the following time‐ranges: V1, 40–80 ms; Cuneus, 80–120 ms; FG, 100–140 ms. Figure 2B depicts their average location and earliest pattern of common significant activity, superimposed on the MRI of an individual brain. 170:

Table I.

Mean (±standard deviation) of ROI Talairach coordinates across subjects and ROI radii (mm)

	X	Y	Z	Radius
Left V1	−8.0 (±2.53)	−84.17 (±5.0)	4.67 (±6.38)	10
Right V1	10.83 (±2.23)	−85.0 (±4.34)	5.17 (±5.88)	10
Left Cuneus	−25.83 (±3.76)	−76.5 (±3.67)	21.83 (±9.83)	15
Right Cuneus	30.17 (±5.53)	−73.33 (±5.72)	22.0 (±10.0)	15
Left FG	−32.17 (±5.27)	−63.5 (±7.66)	−7.17 (±5.78)	15
Right FG	35.67 (±6.15)	−59.5 (±2.35)	−5.17 (±3.19)	15

V1 Effects

We describe the V1 effects per hemisphere in chronological order. Figure 3 depicts the RACs for the significant main effects in the left and right V1. The left V1 showed a significant Task effect with slightly larger amplitudes in the Identity than in the Shape task before stimulus onset, at −63 ms. For posthoc comparison between the Task conditions, we calculated median RAC values of the entire prestimulus period for each subject. All subjects showed higher median RAC amplitudes in the Identity task, showing that this Task effect is a consistent effect that is sustained across the entire prestimulus period.

Figure 3.

V1 effects. (A) From left to right, the average RACs for the effects of VF, Surround, and Task in the left V1. White areas denote intervals of thresholded significant difference. (B) The right V1 main effects. (C) The right V1 interaction between Surround and VF at 66 ms after stimulus onset. The first evoked peak in the right V1 was selective for Surround only for contralateral stimuli. Error bars denote 95% confidence intervals around the mean; the depicted time‐point is that of the minimal P‐value for the interaction. [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.]

From 55 ms after stimulus onset left V1 showed a VF effect that recurred intermittently throughout the epoch. As dictated by retinotopy, contralateral stimuli evoked the largest amplitudes. The left V1 showed a Surround effect around 70 ms, with smaller peak values for stimuli without than with a surround.

The right V1 showed increased amplitudes with contralateral VF cueing at three intervals between −200 and 0 ms, and an evoked VF effect throughout most of the poststimulus period. Before stimulus onset, at −71 and −59 ms, a Task effect was observed; amplitudes were higher in the Identity task. In a posthoc comparison, the median prestimulus RAC value was higher in the Identity than the Shape task for 4/6 subjects. The amplitude difference between the two conditions could not be distinguished from zero using bootstrapped 95% confidence intervals. From 60 to 70 ms, the right V1 showed a Surround effect that was similar to that in the left V1. A significant interaction between Surround and VF also occurred around this interval, indicating that the Surround effect was restricted to contralateral presentations (Fig. 3C).

Cuneus Effects

Figure 4 shows the RACs for the Cuneus main effects. The left Cuneus showed a VF effect between 77 and 154 ms, and from 240 ms until the end of the epoch (Fig. 4A, left). It also showed a Surround effect between 177 and 195 ms, with larger amplitudes for stimuli with than that without surround (Fig. 4A, right).

Figure 4.

Cuneus effects. (A) The average RACs for the VF effects in left and right Cuneus and of Surround in the left Cuneus. White areas indicate intervals of significant difference. (B) The two left Cuneus interaction effects. The Surround and Stimulus Type interaction shows an anticipatory decrease in amplitude for letters with congruent surrounds and pseudoletters with incongruent surrounds; these stimuli are perceptually difficult [van Leeuwen and Lachmann,2004]. The right graph shows the evoked VF and Stimulus Type interaction at 44 ms poststimulus. Bars denote 95% confidence intervals around the mean; the depicted time‐points are those with minimal P‐values for the respective interactions. (C) Boxplots of the median prestimulus amplitudes in the left Cuneus for “difficult” and “easy” stimuli, separated for runs where the conditional probability for these stimuli varied more (left plot) or less (right plot). The anticipatory effects are most reliably observed for runs with low variability in the conditional probability for difficult stimuli. [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.]

The left Cuneus showed two significant interactions (see Fig. 4B). The first, of Stimulus Type and Surround, occurred between −200 and −198 ms, between −114 and −101 ms, and again just before stimulus onset. This effect had minimal P‐value at −113 ms, F(2,115) = 10.81, P < 0.0001. At all three latencies, letters with congruent surrounds had lower amplitudes than those with incongruent surrounds, while the reverse was true for pseudoletters. In a posthoc comparison, the median RAC values of the entire prestimulus period were lower in all six subjects for letters with congruent surrounds and pseudoletters with incongruent surrounds than for letters with incongruent surrounds and pseudoletters with congruent surrounds. This shows that this prestimulus interaction effect reflects a sustained effect consistently present across subjects.

Prestimulus amplitudes can only reflect the upcoming stimulus when that stimulus is anticipated with a greater‐than‐chance accuracy. We randomly drew our stimuli from the same pool without replacement. The effect of this drawing procedure, however, is that the probability of a given stimulus occurring depends on what stimuli have already occurred. That is, stimulus probability is conditional on the trial history within each run. When, for example, an isolated letter appears, the chance of another isolated letter occurring next is decreased, because few of them are left in the stimulus pool. Vice versa, when no isolated letter is shown for several trials, its probability of occurring on a next trial increases.

To test whether such subtle probability variations could lead to the observation of anticipation effects, we reasoned that the more often an anticipated stimulus fails to occur, the larger proportion of the anticipation effects will be randomly spread across all other stimuli, and thus will not show up in the averages. At the same time, the more often anticipation is violated within a run, the higher the variation in conditional probability of the anticipated stimuli in that run. Because each time an anticipated stimulus fails to occur, its conditional probability on the next trial is even higher. We therefore predicted that anticipation effects would be more reliably present in runs where the conditional probability of anticipated stimuli varies less.

We tested this hypothesis for the Stimulus Type–Surround interaction. The anticipation effects in the left Cuneus pertain to stimuli that are perceptually difficult, letters with a congruent surround, and pseudoletters with an incongruent surround [van Leeuwen and Lachmann,2004]. We split the runs of each subject into halves, one with high variability in the conditional probability for these difficult stimuli and the other with low variability. The average SD was 0.071 in high variability runs and 0.041 in low variability runs, based on a median split of the within‐run standard deviation of the conditional stimulus probability. In both high and low variability runs, the average probability for a difficult stimulus was 0.33, the expected value. Figure 4C shows the prestimulus amplitudes for high and low variability runs for difficult and easy stimuli (letters with incongruent surround and pseudoletters with congruent surround). In low variability runs, all subjects consistently showed lower prestimulus amplitudes for difficult stimuli. In the high variability runs, this difference was apparent in 5/6 subjects, but the bootstrapped 95% confidence interval overlapped with 0, meaning that no reliable difference was observed. This result, therefore, shows that the prestimulus differences were more reliably observed when conditional probabilities varied less. This is consistent with the notion that subjects' anticipation is indeed sensitive to subtle differences in conditional probabilities between stimuli.

The second interaction effect in the left Cuneus, of Stimulus Type and VF, occurred between 34 and 51 ms [F(1,115) = 12.56, P < 0.001, at the point of its minimal P‐value]. While greater negativity was observed for pseudoletters in the left VF, in the right VF letters evoked the largest negativity (Fig. 4B). This pattern was observed in 5/6 subjects for the median amplitudes across the effect latency. The bootstrapped 95% confidence interval did not overlap with 0, showing that the hemifield‐dependent differences between letters and pseudoletters were unlikely to have occurred by chance.

In a posthoc test we considered the possibility that these early evoked effects have anticipatory components. We averaged the currents for letters presented in the left VF and pseudoletters presented in the right VF across the entire prestimulus period to see whether they differed from those preceding the other stimuli (as in Fig. 4B, right panel). For 4/6 subjects this was the case, but bootstrapping showed that the prestimulus difference could not be reliably distinguished from 0. This suggests that prestimulus differences do not account for the interaction effect observed around 40 ms after stimulus onset.

In the right Cuneus, the VF effect (Fig. 4A, middle panel) started at −166 ms, recurring intermittently throughout the whole epoch. The prestimulus VF effect consisted of decreased amplitudes for contralateral cueing, whereas the evoked effect (after 70 ms) was mostly characterized by amplitude increases for contralateral stimuli.

Fusiform Gyrus Effects

The left FG showed a VF effect from 93 to 460 ms poststimulus (Fig. 5, top left panel). It also showed Stimulus Type effects between 319 and 386 ms, with letters evoking smaller amplitudes than pseudoletters (Fig. 5, top right panel). Posthoc analysis showed that this pattern was observed in all subjects for the median amplitudes between 300 and 400 ms after stimulus onset. This indicates that the difference between letters and pseudoletters is a sustained and consistent effect across this latency range.

Figure 5.

Fusiform gyrus effects. White areas indicate intervals of significant difference. The VF effects were similar in each FG. The right, but not left, FG showed a significant Surround effect. Left FG had lower amplitudes for letters than pseudoletters between 300 and 400 ms poststimulus. The right FG showed task‐specificity starting 150 ms after stimulus onset. [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.]

In the right FG, contralateral stimuli evoked significantly larger amplitudes from 87 to 175 ms and from 226 ms until the end of the epoch (Fig. 5, bottom left panel). A Surround effect was observed at 108 and 109 ms with isolated stimuli evoking the smallest amplitudes (Fig. 5, bottom middle panel). The right FG showed an evoked Task effect between 151 and 363 ms, with larger amplitudes in the Identity than the Shape task (Fig. 5, bottom right panel). This pattern was observed in all subjects for the median amplitudes between 150 and 400 ms after stimulus onset, showing that the evoked Task differences consistently hold across this latency range.

Results Overview

Figure 6 depicts the time course of the effects per ROI. Before stimulus onset we observed VF effects in right V1 (Fig. 3B, left panel) and the right Cuneus (Fig. 4A, middle panel). The left V1 showed larger prestimulus amplitudes with increased task demands (Fig. 3A,B, left panels). A similar effect was observed in the right V1, but posthoc testing showed that this effect did not hold across the entire prestimulus period. The left Cuneus showed prestimulus differences in anticipation of letters with a congruent surround and pseudoletters with an incongruent surround (Fig. 4B, left panel). This effect was observed in all subjects, across the entire prestimulus period, and depended on the within‐run variability of the conditional probability for these stimuli.

Figure 6.

The time sequence of effects in each ROI. Factors are color‐coded: Visual Field (green), Surround (red), Stimulus Type (purple), and Task (blue). Areas filled in the same color indicate main effects as a function of latency. Blobs in a different color correspond to interactions with the factor corresponding to that color. VF Effects were present in all ROIs. Right V1 and Cuneus showed prestimulus differences depending on cued VF. The Surround effect occurred around peak values in most ROIs, showing lower amplitudes for stimuli without surround. The left FG was the only ROI showing decreased amplitudes for letters relative to pseudoletters. Prestimulus Task effects were observed in bilateral V1, and evoked Task effects were restricted to the right FG.

Evoked VF effects were observed in all ROIs (left panels of Figs. 3A,B, 4A, and 5). Evoked Surround effects were observed in right V1 (Fig. 3A,B, middle panels), the left Cuneus (Fig. 4), and the right FG (Fig. 5). The right V1 only showed a Surround effect for contralateral presentation (Fig. 3C). Task effects were restricted to the right FG, while Stimulus Type effects were restricted to the left FG (Fig. 5). The poststimulus results show a clear spatiotemporal dissociation between task‐specific (right FG) and grapheme‐specific (left FG) processing, while the prestimulus effects suggest a role for anticipatory activity as a mechanism to allocate cortical resources.

DISCUSSION

Functional Dissociation of Left and Right FG and Elsewhere

We studied the routing of information in visual cortex depending on semantic properties, to left and right FG. Different stimuli and tasks selectively activated the left and right FG; the former being sensitive to the graphemic nature of the stimulus (letter or nonletter) irrespective of task; the latter sensitive to the task, irrespective of the graphemic nature of the stimulus. Stimulus‐evoked effects in the right FG become selective to task demands before those in the left FG become selective to letter versus nonletter differences.

Behaviorally, pseudoletters gave rise to longer RTs than letters. Cortically, increased amplitudes for pseudoletters were consistently observed in the left FG, between 300 and 400 ms after stimulus onset (Figs. 5 and 6). Our left FG ROI is in the posterior part of the visual word form area [VWFA; Cohen and Dehaene, 2004], a locus known to show letter specificity [Vickier et al.,2007]. We observed sustained decreases in amplitudes for letters. In line with previous PET and fMRI studies [Garrett et al.,2000; Gros et al.,2001; Pernet et al.,2005], this suggests that letters evoked less activity than pseudoletters, possibly resulting from repeated exposure to letters [Henson,2003].

Letter‐evoked responses have been typically reported at around 170 ms [Gros et al.,2002; Wong et al.,2005]. Around this latency, the grapheme effect in the left FG became apparent (although not statistically significant, see Fig. 5), so we cannot exclude that left FG plays a role in this early letter‐selectivity. It could be that at these latencies letter selectivity also involves more downstream areas. We observed surprisingly early letter‐specificity in the left Cuneus, dependent on VF, at around 40 ms. These results may reflect an early selectivity of the left hemisphere for letters. We know of no previous studies reporting such effects. In fact, the latency of the effects precedes that of the first common activation we observed in early visual areas, at around 65 ms. It therefore seems implausible that this effect is entirely stimulus driven. The prestimulus activity, however, did not consistently reflect the effect so that it cannot entirely be explained as an anticipation effect either. We obtained these effects using a data‐driven approach; future research should address the question of early letter‐selectivity in a hypothesis‐driven way so as to further validate these findings.

Because of the late timing and long duration of the effect, we suggest that left FG letter processing relies on integrated input over many cortical areas. The timing and direction of our effects is in line with that in previous EEG recordings [Wong et al.,2005]. We show that differences between letters and nonletters at these late latencies involve the left FG. Because the grapheme effect leads up to the N400 component, these processes may relate to a categorical distinction [Pylkkanen and Marantz,2003] in the left FG [Garoff et al.,2005; Koutstaal et al.,2001; Marsolek,1995; Marsolek et al.,1992]. This suggests that the grapheme effect reflects implicit categorical processes, because the letter–pseudoletter distinction was entirely incidental to the task.

The right FG showed higher amplitudes in the shape task between 150 and 400 ms after stimulus onset, starting at latencies of higher‐order object representation and, like the grapheme‐effect, ending around the N400 [Pylkkanen and Marantz,2003]. These dynamics suggest that task‐specific information is processed in the right FG before content‐specific processing becomes apparent in the left FG. The task‐dependency of the right FG is in line with its role in subordinate discrimination [Gauthier et al.,1999; Tarr and Gauthier,2000], with priming effects we observed previously [Liu et al.,2006] and with the observation that behavioral relevance increases right hemisphere activity [Corbetta and Shulman,2002].

These data show how shape processing, the detailed inspection of stimulus features, can be temporally and spatially separated from the functional specialization of letters. The spatial separation agrees with previous findings of the right FG contributing more to encoding specific stimulus features, and the left FG processing categorical stimulus properties [Garoff et al.,2005; Koutstaal et al.,2001; Marsolek,1995; Marsolek et al.,1992]. Our data reveal the time course of this asymmetry: in our task the right FG quickly reflected task demands irrespective of stimulus content, followed by implicit categorization in the left FG.

Surrounding Context

Stimuli with a surround yielded longer RTs and evoked larger responses in the left V1 and right V1 (for contralateral presentation only), the left Cuneus, and the right FG. These effects can either be ascribed to the greater size of stimuli with a surround or to crowding. The reason for these effects may differ per ROI. Response properties of V1 reflect stimulus size more than FG does so that surround effects in V1 may arise from stimulus size alone, whereas in higher‐level areas crowding may play an additional role. The behavioral effects, longer RTs for surrounded than isolated stimuli, suggest crowding [Bouma,1970; Hagenaar and van der Heijden,1986; Miller,1991; Toet and Levi,1992] as a more likely explanation. Crowding impedes the performance on a para‐foveal target independently of the congruency between the surrounding and surrounded stimulus. The behavioral predecessor study [van Leeuwen and Lachmann,2004] differentiated between surrounded stimuli, showing opposing congruence effects between foveal letters and pseudoletters. RTs to letters with congruent surrounds increased in the Identity task, which invokes detailed representations, while for pseudoletters incongruent surrounds increased RT. In our present study, however, we found only a trend in RT for more prominent negative congruence effects in the Identity task. Although this is consistent with the earlier observations, it fails to unambiguously replicate the negative congruence effect of van Leeuwen and Lachmann [2004]. This may be due to a lack of statistical power in the behavioral data; these effects tend to be small and only six subjects took part in the current experiment. On the other hand, the effect of isolated stimuli suggests that due to the present parafoveal presentation, crowding dominated in behavioral responses over congruence effects.

Cortical Anticipation

With the exception of the FG, all ROIs showed anticipation effects (Fig. 6), i.e. effects before stimulus onset. In line with previously reported effect of spatial attention in primary visual areas of the macaque [Luck et al.,1997; Motter,1993] and humans (Kastner et al.,1999; Silver et al.,2007], we observed cueing effects in prestimulus activity in the right V1 and Cuneus. More surprisingly, V1 showed increased amplitudes when the task required finer shape processing (Identity task). This prestimulus effect was restricted to V1 and therefore it is unlikely to have resulted from general arousal levels [Livingstone and Hubel,1981]. It could be argued that these effects result from the blocked presentation of the two task conditions, together with the use of an uncorrected baseline. This seems unlikely, however, because we counterbalanced the task order across subjects so that systematic effects of, e.g., fatigue on prestimulus baselines should have canceled out. Furthermore, the task effect in the left V1 was observed consistently in all subjects, across the entire prestimulus period. Together with a recent demonstration of task‐related enhancements in V1 [Jack et al.,2006], our data therefore point to the conclusion that V1 adapts its baseline activity to the anticipated task demands.

The anticipation of task demands and VF in V1 consisted of increased currents in the main evoked current direction. This may be understood as an increased sampling of the visual display that provides upstream areas with more information about the current display state and for making available a reliable structural representation of the stimulus at an early processing stage. This way V1 effectively decreases the load on later processing stages.

Perhaps the most remarkable, and admittedly unexpected, anticipation effects were observed in the left Cuneus for specific types of stimuli (Figs. 4 and 6). Previous RT work showed that letters with congruent surrounds and pseudoletters with incongruent surrounds induce perceptual conflict so that they take longer time to categorize [van Leeuwen and Lachmann,2004]. This ROI showed prestimulus selectivity for precisely these, and no other stimuli. These effects were present in all subjects and held across the entire prestimulus period. This suggests that this brain region shows anticipation of perceptual conflict‐inducing stimuli. This anticipation effect seems to emerge as a consequence of presenting stimuli randomly from a fixed pool without replacement. This presentation procedure results in subtle, trial‐to‐trial fluctuations in the conditional probability of stimulus types, which effectively enables correct anticipation. Our data suggest that the more regular the probability fluctuations are, the more pronounced the anticipation effect. This implies that stimulus anticipation effects should disappear when stimuli are drawn randomly with replacement. This is a testable hypothesis for future research that can shed light on the sensitivity of the Cuneus and other areas to the likelihood of future events.

CONCLUSIONS

In sum, the cortical anticipation effects suggest that early visual areas adapt their activity according to the anticipated stimulus location and task demands. The evoked effects showed clear task‐dependence in the right FG, which was independent of whether letters or nonletters are processed. In the left FG letters evoked smaller responses than nonletters and this effect appeared later than the task effects in the right FG.