Attentional modulation of perceptual grouping in human visual cortex: Functional MRI studies

Abstract

When presented with a complex visual scene, our visual system has to organize the discrete entities present into useful perceptual units. The current work investigated the neural substrates of perceptual grouping defined by Gestalt laws of proximity and similarity of shape, and whether the neural substrates underlying perceptual grouping are modulated by task relevance and spatial attention. In visual discrimination tasks, subjects identified the orientations of perceptual groups formed by proximity or similarity of local elements or alternatively identified colors of either dots around the grouped stimuli or the fixation cross. Using functional magnetic resonance imaging (fMRI), we identified that the calcarine cortex was involved in proximity grouping but not in the grouping process defined by similarity of shape. Moreover, we showed evidence that the neural correlates of proximity grouping in the calcarine cortex were weakened when the elements were of low task relevance and fell outside an attended area of field. The findings reveal the neural basis for basic grouping operations, as well as illustrating how attention and proximity grouping interact in human visual cortex. Hum Brain Mapp 2005. © 2005 Wiley‐Liss, Inc.

INTRODUCTION

Perceptual grouping refers to the process of organizing spatially distributed visual features into a perceptual whole. Since the time of Gestalt psychologists in the early twentieth century, it has been argued that our perception of an ordered world of objects is based on grouping processes sensitive to factors such as proximity and similarity of visual elements. For example, perceived columns or rows of local circles or squares in Figure 1 are formed based on proximity (the proximity target and nontarget) or on shape similarity (the similarity target and nontarget). A core idea of contemporary theories of visual perception is that perceptual organization depends on grouping operations at early stages of visual perception [Lamy and Tsal, 2001; Vecera and Behrmann, 2001], with these grouping processes operating even in the absence of focal attention [Humphreys, 1998; Mattingly et al., 1997; Ward et al., 1984]. Attention is then assumed to select for higher‐order analysis between perceptual objects formed at a preattentive stage by grouping operations [Duncan, 1984; Duncan and Humphreys, 1989; Kahneman and Henik, 1982; Marr, 1982; Treisman, 1986].

Figure 1.

Illustration of the stimulus arrays used in Experiment 1. Each stimulus array consists of a square lattice of gray circles or squares on a black background. Alternate circles and squares distributed evenly across the lattice in the uniform stimulus, but are grouped into rows or columns by either proximity or similarity of shape in the grouped stimuli. The target stimuli are the same as the nontargets except that they are smaller.

Evidence for effects of grouping operations in visual cortex has been shown in several single‐cell recording studies. The activity of cells within the striate cortex can be enhanced by grouping by collinearity [Kapadia et al., 1995] or common fate [Sugita, 1999] with stimuli both inside and outside the excitatory boundary of receptive fields. Whether these interactions form the basis for grouping effects in human perception can be questioned, because grouping effects in these single‐cell studies typically operate over much smaller areas than areas usually tested in human behavioral studies [Ben‐Av and Sagi, 1995; Han and Humphreys, 1999; Han et al., 1999a, b]. Recent event‐related brain potential (ERP) studies in humans have shown that relative to stimulus arrays in which local elements were distributed evenly in space (i.e., the uniform stimulus in Fig. 1), stimuli that group by proximity generated a positive neural activity occurring at about 100–120‐ms poststimulus (Pd100) over the medial occipital cortex [Han et al., 2001, 2002]. This implicates an early proximity‐grouping effect in visual cortex. Using motion‐defined local elements, Han et al. [2001] showed further that proximity grouping‐related positivity was evident even when low spatial frequencies were removed from both proximity and uniform stimuli, indicating that the proximity grouping‐related activity over the medial occipital cortex could not simply reflect the processing of low spatial frequency contents of the grouped stimuli. Although the ERP results suggest that grouping by proximity may have neural substrates in human visual cortex, it remains undefined whether the proximity grouping‐related activity arises from visual cortex because of the low spatial resolution of the ERP results.

Previous studies have shown that behavioral responses to the identification of perceptual groups are severely impaired when attention is engaged in a concurrent task [Ben‐Av et al., 1992; Mack et al., 1992], suggesting that perceptual grouping is affected by the allocation of attentional resources. Failures in behavioral responses to perceptual groups do not indicate directly that grouping processes are weakened under dual‐task conditions. Grouping may occur preattentively, but may not be remembered under attentionally taxing conditions [Moore and Egeth, 1997]. A more direct test of whether grouping is modulated by attention requires an evaluation of grouping‐related activity within the human visual cortex using brain‐imaging procedures.

The current work used brain‐imaging procedures to assess whether grouping affects the cortical processing in human visual cortex and whether grouping operation in visual cortex is modulated by attention. We measured brain activities from human subjects using functional magnetic resonance imaging (fMRI). To compare with the results of previous ERP studies [Han et al., 2001, 2002], Experiment 1 recorded event‐related fMRI signals associated with stimulus arrays consisting of local elements that were either spaced evenly (uniform stimulus) or grouped into columns or rows by proximity or similarity of shape (grouped stimuli; see Fig. 1). fMRI signals common to both uniform and grouped stimuli indexed initial sensory processing whereas neural responses specific to grouped stimuli also reflected perceptual organization of local elements into columns or rows. Grouping‐related neural activities were thus defined by stronger activations induced by grouped than by uniform stimuli. Experiment 2 recorded fMRI signals associated with high‐pass filtered images (i.e., local elements were defined by texture difference, see Fig. 2) to rule out the possible contribution of low spatial frequency to proximity grouping [Ginsburg, 1986]. Low spatial frequencies in such stimulus arrays are removed and proximity‐grouped stimuli cannot be distinguished from uniform stimuli by spatial frequency differences. Moreover, to examine if attention influences the grouping process defined by proximity in visual cortex, Experiment 2 employed a boxcar design so that task relevance of the grouping stimuli or the size of attentional window was modulated in different sessions of MRI scans. Subjects were asked to discriminate either the orientation of perceptual groups in targets (Task 1), the colors of dots around stimulus arrays (Task 2), or colors of the fixation cross (Task 3) in separate sessions. Participants may have to set a relatively large window of attention and identify a feature (i.e., orientation) related to the perceptual groups with either vertical or horizontal orientations in Task 1. In Task 2, a large attentional window again had to be set, to encompass the whole stimulus arrays, but the feature required for identification (i.e., color) was not relevant to the features of perceptual groups (i.e., orientation). In Task 3, a small attentional window was set to make most of the stimulus arrays fall outside the attentional window and the feature required for identification was of low task relevance. The contrast between these conditions provides a test of whether task relevance and the size of the attentional window modulate proximity‐grouping process of task‐irrelevant elements.

Figure 2.

Illustration of the stimuli used in Experiment 2. a: A nontarget grouped stimulus. b: A nontarget uniform stimulus. c: A proximity target stimulus used in Task 1. d: An uniform target stimulus used in Task 1. e: A nontarget stimulus used in Task 2, in which the peripheral dots are white. The dots were either red or green in target stimuli. f: A nontarget stimulus used in Task 3, in which the fixation is white. The fixation was either red or green in nontarget stimuli.

SUBJECTS AND METHODS

Subjects

Subjects were recruited from graduate and undergraduate students of Peking University. Sixteen subjects (8 males; 19–23 years of age) participated in Experiment 1. Thirteen subjects (8 males; 21–39 years of age) participated in Experiment 2. All subjects were right‐handed, had normal or corrected‐to‐normal vision, and were not color blind. All subjects were free of any history of neurologic or psychiatric problems. Informed consent was obtained according to the guidelines of Department of Psychology, Peking University.

Stimuli

The stimuli were presented through a LCD projector onto a rear‐projection screen located at the subject's feet. The screen was viewed with an angled mirror positioned on the head‐coil. A white fixation cross of 0.21 × 0.21 degrees (width and height) at a viewing distance of 270 cm was continuously visible in the center of the screen. In Experiment 1, each of the nontarget stimuli consisted of a square lattice of gray elements on a black background (either filled circles or squares) in an 8 × 8 array, as shown in Figure 1. The uniform stimulus consisted of alternate circles and squares distributed evenly across the lattice. This arrangement prevented the local elements from grouping into rows or columns. The proximity‐grouped stimuli consisted of alternate circles and squares arranged in arrays to form separate perceptual groups (i.e., rows or columns) by adjusting the distances between two adjacent rows or columns of local elements so that the distances between two near or remote rows (or columns) were 0.23 and 0.85 degrees, respectively. The similarity‐grouped stimuli were made by moving the circles and squares in the uniform stimulus to form rows or columns of elements with the same shape. The distance between two adjacent columns or rows was 0.47 degrees for the uniform and similarity‐grouped stimuli. Each local shape subtended an angle of 0.43 × 0.43 degrees and the global stimulus pattern subtended an angle of 6.8 × 6.8 degrees. Each display was presented for 200 ms with an interstimulus interval of either 3,400 or 4,200 ms. Target stimuli were the same as the nontargets except that they were 62% smaller.

In Experiment 2, the background consisted of uniform vertical line segments (white on a black screen) with a pseudorandom distribution. The local shapes were made by rotating the line segments in the areas (corresponding to those occupied by the local elements in stimulus arrays of Experiment 1) 45 degrees clockwise or anticlockwise (see Fig. 2a,b). Each line segment subtended a visual angle of 7.8′ × 1.1′. The size of local shapes and global arrays in nontarget stimuli were the same as those in Experiment 1. In the orientation discrimination task (Task 1), the grouped target stimuli were the same as the nontargets except that they were 62% smaller (Fig. 2c). The uniform target stimuli were either a vertical or a horizontal rectangle (4.0 × 2.0 degrees; Fig. 2d). In the peripheral dot‐discrimination tasks (Task 2), the same nontarget stimuli as those in Task 1 were presented; four dots (0.21 × 0.21 degrees) at the corners of an imaginary square appeared along with the stimuli. Each dot was 6.0 degrees from fixation. When the stimuli were nontargets, the dots were white. When the stimuli were targets, the dots were red or green (Fig. 2e). In the fixation discrimination tasks (Task 3), the same nontarget stimuli as those in Task 1 were presented but the fixation cross was either red or green in target displays (Fig. 2f). Each display was presented for 200 ms with the interstimulus intervals randomly varying between 400 and 600 ms.

Procedure

In Experiment 1, participants were asked to indicate the presence of row‐ or column‐grouped target stimuli (regardless of whether proximity or similarity cues produced grouping) by pressing one of two keys with either the left or the right index finger. Uniform stimuli required no response. Ten scans of 102 s were obtained from each subject.1 Each scan consisted of 25 trials alternating randomly between uniform, proximity‐grouped, and similarity‐grouped stimuli. On average, there were three targets in each scan. The first 30 slices (initial 6 s of acquisition) were discarded from the statistical analysis to obtain a steady baseline. The uniform stimulus, proximity‐grouped stimuli, and similarity‐grouped stimuli each were presented randomly on one‐third of the nontarget trials. Participants were instructed to maintain fixation on the central cross throughout the task while responding to targets as quickly and accurately as possible. Target orientations and the assignment of responding hand with the two types of targets (column vs. row) were counterbalanced across participants.

A boxcar design was employed in Experiment 2, in which 12 scans of 64 s were obtained from each subject. In each of four scans, subjects discriminated either the orientation of the uniform or the grouped stimuli (Task 1), the color of peripheral dots (Task 2), or the color of the fixation cross (Task 3) by pressing one of two keys with either the left or the right index finger. Each scan consisted of two epochs of 40 trials (30 s for each epoch), alternating between uniform and proximity‐grouping conditions. There were 10% target stimuli in each scan.

Functional MRI Image Acquisition and Analysis

Brain imaging was carried out using a 1.5‐T GE Signa MR scanner with a custom head coil. Fifteen axial slices of functional images that covered the whole cerebral cortex were acquired using echo‐planar imaging (64 × 64 × 15 matrix with 3.75 × 3.75 × 6‐mm spatial resolution, repetition time [TR] = 2,000 ms, echo time [TE] = 40 ms, filed of view [FOV] = 240 mm, and flip angle = 90 degrees). Anatomic images were obtained with a standard 3‐D T1‐weighted sequence (resulting in a 256 × 256 × 66 matrix with 0.938 × 0.938 × 2.0‐mm spatial resolution, TR = 585 ms, and TE = minimum). Subjects' heads were immobilized during the scanning sessions using pieces of foam.

SPM99 (Wellcome Department of Cognitive Neurology, UK) was used for data processing and analysis. After correction for differences in the timing of slice acquisition within a volume, the functional images were realigned to the first scan to correct for the head movement between scans using sinc interpolation. The anatomic image was coregistered with the mean functional image produced during the process of realignment. All images were normalized to a 2 × 2 × 2 mm³ Montreal Neurological Institute (MNI) template in Talairach space [Talairach and Tournoux, 1998] using bilinear interpolation. Functional images were spatially smoothed using a Gaussian filter with a full‐width at half‐maximum (FWHM) parameter set to 8 mm. The time series for each voxel were high‐pass filtered to 1/60 Hz and low‐pass filtered by a canonical hemodynamic response curve. The response to each trial, synchronized with the acquisition of the top slice, was modeled by a canonical hemodynamic response function (HRF) and its first temporal derivative [Friston et al., 1999]. In Experiment 1, the image data of 16 subjects were first estimated to establish a fixed‐effect model, using the event‐related analysis module of SPM99. In Experiment 2, the image data of 13 subjects were also first estimated to establish a fixed‐effect model, using a boxcar function. Random‐effect analyses were then conducted in both experiments across the group of subjects based on statistical parameter maps from each individual subject to allow population inference. Contrasts were used to compare the effect of grouping (grouped vs. uniform) for each task. Regions preferentially engaged in proximity grouping were defined as areas more activated by proximity‐grouped nontargets than by uniform nontargets. Regions preferentially engaged in similarity grouping were defined as areas more activated by similarity‐grouped nontargets than by uniform nontargets. Areas of significant activation were identified at the cluster level for P values exceeding 0.002 (uncorrected) and voxel numbers larger than 20 for the group analysis. Such a generous threshold was used because we had a prior hypothesis based on our previous ERP studies [Han et al., 2001, 2002; Han et al., in press] that suggested sources of the proximity grouping activity in the calcarine cortex. The SPM coordinates for standard brain from MNI were converted to Talairach coordinates using a nonlinear transform method (http://www.mrc-cbu.cam.ac.uk/Imaging/mnispace.html). SPM99 was also used in Experiment 2 to do region‐of‐interest (ROI) analysis to compare relative changes of fMRI signals in the visual cortex in association with the process of proximity grouping in different tasks. The activation cluster in the calcarine cortex found in Experiment 1 was used to define an ROI that was independent of tasks in Experiment 2. For each subject, the fMRI signals in the ROI associated with uniform and grouped stimuli in Tasks 1 and 2 were first obtained from the functional images of each scan. The percentage of increased fMRI signals in the ROI in the proximity grouping relative to the uniform conditions was calculated as follows: (fMRI signals associated with proximity‐grouped stimuli − fMRI signals associated with uniform stimuli)/fMRI signals associated with uniform stimuli. The mean percentage of increased fMRI signals in the ROI was then averaged across the four scans in each attention condition and then compared using one‐sample t test.

RESULTS

In Experiment 1, participants correctly responded to 96% of the targets, with responses being faster for proximity‐ than for similarity‐defined targets (781 vs. 853 ms; t[15] = 2.632, P < 0.05). Brain areas activated by proximity and similarity grouping are summarized in Table I and illustrated in Figure 3. Proximity‐grouping stimuli generated increased activations in the right calcarine area, bilateral inferior parietal cortices, and the right superior temporal cortex relative to the uniform stimulus, as illustrated in Figure 3a. Figure 3b showed the time course of blood oxygenation level‐dependent (BOLD) responses after proximity‐grouped and uniform stimulus onset (time zero on abscissa) in the calcarine area. Similarity grouping induced increased activations in the right middle occipital cortex and the left middle temporal cortex (Table I; Fig. 3c).

Table I.

Brain areas activated by proximity and similarity grouping in Experiment 1

Condition/region	Voxel no.	x	y	z	Z	P
Proximity
Calcarine area	24	14	−88	17	2.85	0.002
Right inferior parietal cortex	572	55	−26	32	3.57	0.0005
Left inferior parietal cortex	72	−64	−35	26	3.03	0.001
Right superior temporal cortex	86	50	4	0	2.95	0.002
Similarity
Right middle occipital cortex	30	24	−82	22	3.46	0.0005
Left middle temporal cortex	817	−53	−56	8	4.17	0.0005

Voxel no., number of voxels in a cluster.

Figure 3.

Brain activations associated with proximity and similarity grouping in Experiment 1. a: Brain areas showing stronger activations for the proximity‐grouped nontargets than the uniform nontarget in Experiment 1. These include the right calcarine cortex, bilateral inferior parietal cortices, and right superior temporal cortex. b: The time course of BOLD responses after proximity‐grouped and uniform stimulus onset (time zero on abscissa) in calcarine area. c: Brain areas showing stronger activations for similarly‐grouped than for uniform stimuli. These include the right middle occipital cortex and left middle temporal cortex.

In Experiment 2, participants correctly responded to 94.6, 95.4, and 99.4% of the targets in Tasks 1, 2, and 3, respectively, with reaction times being 957, 858, and 833 ms, respectively. A repeated measures analysis of variance (ANOVA) confirmed that response speeds of Task 3 were faster than those of Task 2, which in turn were faster than those of Task 1 (F[2,24] = 19.346; P < 0.001). Responses did not differ between grouped and uniform targets in Task 1 (F < 1). There was no significant difference in response accuracy between the tasks (P > 0.05).

In Task 1, the contrast between proximity‐grouped stimuli and uniform stimuli showed stronger activations in the calcarine area (Fig. 4a) and the left superior parietal sulcus (Fig. 4b; Table II). Figure 4c shows the time course of the fMRI signal change in the calcarine cortex as a function of grouping of local elements in the stimulus arrays. The x‐axis of the time course is for the whole session, which contained 4 scans. Each local segment of the time course contains one 30‐s epoch in each scan corresponding to the proximity‐grouped or uniform stimuli. The mean image values obtained from the average of the four scans were used as a baseline. The time courses were averaged from raw fMRI signals across the 13 subjects.

Figure 4.

Brain areas showing stronger activations for proximity‐grouped than for uniform stimuli in Task 1 of Experiment 2. a: Grouping‐related activation in the calcarine cortex. b: Grouping‐related activation in the superior parietal sulcus. c: The time courses of the signal change in the calcarine cortex.

Table II.

Brain areas activated by proximity grouping in Experiment 2

Condition/region	Voxel no.	x	y	z	Z	P
Discrimination of orientation
Calcarine area	119	4	−78	1	3.27	0.001
Left parietal cortex	135	−28	−62	49	4.22	0.0005
Discrimination of dot color
Calcarine area	430	2	−87	12	3.52	0.0005

Voxel no., number of voxels in a cluster.

In Task 2, the contrast between proximity‐grouped stimuli and uniform stimuli showed stronger activation only in the calcarine area (see Fig. 5a; Table II). Figure 4c shows the time course of the fMRI signal change in the calcarine cortex as a function of proximity grouping of local elements in the stimulus arrays. No activation was observed in the same contrast in Task 3.

Figure 5.

Brain areas showing stronger activations for proximity‐grouped than for uniform nontargets in Task 2 of Experiment 2. a: Grouping‐related activation in the calcarine cortex. b: The time courses of the signal change in the calcarine cortex.

To compare further the strength of proximity‐grouping effect in the calcarine cortex between Tasks 1 and 2, fMRI signal increases in proximity grouping relative to the uniform conditions were calculated in a task‐independent ROI defined by the activation cluster in the calcarine cortex found in Experiment 1 (centered at 14, × −88, 17). Pairwise t tests confirmed increased fMRI signals in the ROI in the proximity grouping compared to uniform conditions in Task 1 (0.31%; t[12] = 3.86, P < 0.002) and Task 2 (0.33%; t[12] = 3.70, P < 0.003) but not in Task 3. In addition, the fMRI signal in the ROI was larger in Tasks 1 and 2 than in Task 3 (t[12] =3.87 and 3.42, respectively; both P < 0.005), indicating that the proximity‐grouping effect in the calcarine cortex was stronger in Tasks 1 and 2 than in Task 3. The fMRI signals in the ROI did not differ significantly between Task 1 and Task 2 (P > 0.8), suggesting the proximity‐grouping effect in the ROI was not different between the two conditions of high and low task relevance.

DISCUSSION

The current work used fMRI to identify the neural correlates underlying perceptual grouping defined by proximity and similarity of shape in human brain. We found hemodynamic effects associated with proximity and similarity grouping by contrasting responses to displays in which local elements were either grouped together or evenly spaced. The hemodynamic proximity‐grouping effect revealed by event‐related fMRI was situated in the calcarine cortex in the right hemisphere in Experiment 1 (Fig. 3a), when local elements were defined by contrast differences. The locus of the calcarine activation is consistent with our previous studies that found that the early proximity‐grouping effect in ERPs (e.g., the positive activity peaking at about 100 ms after sensory stimulation [Pd100]) was over the medial occipital cortex [Han et al., 2001, 2002]. This early proximity grouping‐related Pd100 has been localized to the right calcarine cortex using dipole modeling based on a realistic head model in our recent ERP studies [Han et al., in press]. The combined fMRI and ERP results therefore indicate that human calcarine cortex (particularly in the right hemisphere) is involved in the early grouping process defined by proximity.

Similar proximity grouping‐related activation was also observed in Experiment 2 (Figs. 4 and 5) where local elements were defined by texture differences and a block design was adopted. As the stimulus arrays in Experiment 2 filtered out low spatial frequencies in both grouped and uniform stimuli, it is unlikely that the calcarine activation associated with proximity grouping reflected per se the processing of low spatial frequency information in the proximity‐grouped stimuli. Although the current data did not separate whether the calcarine activation was in the striate or extrastriate cortex, the fMRI results provide novel neuroimaging evidence that the grouping processes defined by proximity may occur in human visual cortex. Our fMRI results converge with findings from animal studies, where responses of single neurons in the primary visual cortex of monkeys have been shown to be modulated by grouping with stimuli both inside and outside the receptive field [Kapadia et al., 1995; Sugita, 1999]. Taken together, our findings suggest that the human calcarine cortex plays an important role in the larger‐scale organization of discrete visual elements (here based on proximity grouping).

Other recent neuroimaging studies have shown that global shapes generated through grouping by collinearity or through binocular disparity elicit activations in the striate/extrastriate cortex and the lateral occipital complex (LOC) [Altmann et al., 2003]. Perception of an object formed by grouping by motion is also linked to activation in the LOC [Ferber et al., 2003]. As the studies have required participants to respond to the global shapes, it is unclear whether any associated neural changes reflect grouping operations per se or the response to identification of the global shapes. In our study, neural activity was measured to nontargets to which no behavioral responses were required, making it more likely that the effects were linked to the grouping processes. In addition, we discriminated between the effects with proximity‐ and with similarity‐defined groups. If the calcarine activity reflected identification of the orientations of perceptual groups, we would expect similar activations in the calcarine cortex for both types of grouped stimuli. Because the calcarine activity was evident for proximity grouping but not for similarity grouping even though subjects carried out the same discrimination task in the two stimulus conditions, it may be proposed that the calcarine activity reflects neural activity specific to the grouping processes defined by proximity. One may notice that the calcarine activation in association with proximity grouping was slightly lower in Task 1 of Experiment 2 than in Task 2 of Experiment 2 and in Experiment 1. It seems that the grouping process in the visual cortex may be slightly different depending upon the presence of low SFs in a stimulus display. In addition, the size of an attentional window, which was slightly larger in Task 2 than in Task 1 of Experiment 2, might also modulate the location of the grouping operation in the visual cortex. The underlying neural mechanisms could not be disclosed by the current fMRI data, and need to be studied in future work.

Proximity grouping was also reflected in activations over bilateral inferior parietal cortices in Experiment 1 (Fig. 3a). The fMRI results suggest that there are possibly two stages of grouping operations based on proximity. An initial grouping operation takes place in the calcarine cortex to link the local elements into larger chunks. At a later stage, the inferior parietal cortices are engaged in processing of orientations of the perceptual groups formed at the earlier stage. This proposal is in line with our previous ERP results that showed proximity‐grouping effects at two subsequent stages, one over the medial occipital cortex and the other over the parietal area [Han et al., 2001, 2002]. The latter process might be weakened when subjects were asked to identify a feature (e.g., color) independent of the orientations of perceptual groups, as shown in fMRI results of Task 2 in Experiment 2.

The proximity‐grouping condition also generated reliable activation in the right superior temporal region in Experiment 1 (Fig. 3a). This may reflect the perception of the grouped rows or columns, which were composed of local shapes, so that the grouped wholes may be represented hierarchically in terms of relations between the local parts. Both prior imaging studies [Han et al., 2004] and neuropsychological studies [Lamb et al., 1990] have indicated that the superior temporal lobe may mediate attention to the global forms of hierarchical shapes.

Grouping by shape similarity generated activation in the right middle occipital cortex and the left temporal cortex in Experiment 1 (Fig. 3c). Unlike proximity grouping, local shape processing was needed for similarity grouping to take place. These areas, additional to those associated with proximity grouping, may be particularly concerned with shape processing, and previous studies using single‐cell recording have indicated that medial and inferior temporal cortex is engaged in processing object shape [Logothetis and Sheinberg, 1996; Tanaka, 1993]. Whatever the case, the differences in the neural correlates of proximity and similarity grouping indicate that grouping operations defined by different Gestalt laws have distinct neural mechanisms with unique spatial characteristics. These neural differences may underpin previous behavioral data showing faster responses to proximity‐grouped stimuli than to similarity‐grouped stimuli [Han et al., 1999a, b].

Contrasts between the results of different tasks in Experiment 2 provide evidence for attentional modulations of proximity‐grouping operations in visual cortex. The proximity grouping‐related neural activity occurred within the calcarine cortex in Task 1 where participants had to judge the orientation of spatial groups. Increased proximity grouping‐related neural activity was also observed in the calcarine cortex in Task 2 when subjects identified colors of peripheral dots that set a large attentional window to encompass the whole stimulus arrays. The features of the perceptual groups (i.e., their orientation) were of high task relevance for orientation discrimination but of low task relevance when the task required discrimination of the colors of peripheral dots. The fMRI results therefore did not demonstrate differences in the hemodynamic responses associated with proximity grouping as a failure of the relevance of grouping to the task.

Nevertheless, contrary to the hypothesis that perceptual grouping is indifferent to the effects of attention [Humphreys, 1998; Moore and Egeth, 1997], we have provided neuroimaging evidence that proximity grouping is modulated by whether stimuli fall within an attended spatial region. In Tasks 1 and 2, the tasks required participants to adopt a relatively large attentional window (note that the whole stimulus arrays fell within the attentional window when subjects identified the colors of peripheral dots). In contrast, Task 3 (identify the colors of the fixation cross) emphasized a small attentional window, in which case local elements forming perceptual groups may fall outside the attended area, and made the grouping displays be of low task relevance. Under this circumstance, the proximity grouping‐related neural activity in the calcarine cortex was eliminated. The results suggest that combined task relevance and the size of attention spotlight influence the proximity‐grouping operations in visual cortex. Such modulations of the grouping process cannot be explained simply by the difference in task difficulty. Previous studies have shown that perceptual grouping is impaired when attention is engaged in a difficult concurrent task [Ben‐Av et al., 1992; Mack et al., 1992]. If task difficulty contributed to the proximity‐grouping‐related calcarine activity in the current study, one would expect impaired grouping operations (weak calcarine activation) in Tasks 1 and 2 (difficult tasks indexed by long reaction times) rather than in Task 3 (an easy task indexed by short reaction times). Our fMRI data showed a reverse pattern. The attentional modulations of proximity grouping‐related calcarine activity observed here provide possible neural substrates for the findings of previous behavioral studies, where grouping effects seem to be reduced when attention is withdrawn [Ben‐Av et al., 1992; Mack et al., 1992]. When attention is applied to a focal stimulus simultaneously presented with the grouped stimuli, early grouping operations in visual cortex are weakened and thus behavioral responses to the perceptual groups are impaired.

The fMRI results reported here are consistent with our recent ERP findings [Han et al., in press], where the proximity‐grouping‐related Pd100 was of larger amplitude when the grouped stimuli were of high task relevance and fell inside the attended area than when the grouped stimuli were of low task relevance and fell outside the attended area. The ERP results showed further that the Pd100 was enlarged when orientation discrimination was required explicitly compared to that when participants had to discriminate the colors of peripheral dots. The current fMRI results only showed a similar trend to the ERP results. It is possible that part of the long‐latency hemodynamic responses (relative to ERPs) in the calcarine cortex resulted from the feedback modulation from higher‐level brain structures, and was thus not sensitive enough to disclose the difference of the early proximity grouping‐related neural activities in the calcarine cortex between conditions of high and low task relevance.

In conclusion, the current work localized neural activities in relation to grouping by proximity in the calcarine cortex as well as in other higher‐level brain structures (inferior parietal and temporal cortices). Similarity grouping is mediated by other brain regions, which are likely “downstream” of the calcarine cortex. In addition, proximity grouping‐related activity in the brain was modulated by whether the grouped‐properties were task relevant and by whether grouped elements fell in an attended area of field. Even at early stages of processing, visual proximity‐grouping is sensitive to the deployment of visual attention.