Setting Boundaries: Brain Dynamics of Modal and Amodal Illusory Shape Completion in Humans

Abstract

Normal visual perception requires differentiating foreground from background objects. Differences in physical attributes sometimes determine this relationship. Often such differences must instead be inferred, as when two objects or their parts have the same luminance. Modal completion refers to such perceptual “filling-in” of object borders that are accompanied by concurrent brightness enhancement, in turn termed illusory contours (ICs). Amodal completion is filling-in without concurrent brightness enhancement. Presently there are controversies regarding whether both completion processes use a common neural mechanism and whether perceptual filling-in is a bottom-up, feedforward process initiating at the lowest levels of the cortical visual pathway or commences at higher-tier regions. We previously examined modal completion (Murray et al., 2002) and provided evidence that the earliest modal IC sensitivity occurs within higher-tier object recognition areas of the lateral occipital complex (LOC). We further proposed that previous observations of IC sensitivity in lower-tier regions likely reflect feedback modulation from the LOC. The present study tested these proposals, examining the commonality between modal and amodal completion mechanisms with high-density electrical mapping, spatiotemporal topographic analyses, and the local autoregressive average distributed linear inverse source estimation. A common initial mechanism for both types of completion processes (140 msec) that manifested as a modulation in response strength within higher-tier visual areas, including the LOC and parietal structures, is demonstrated, whereas differential mechanisms were evident only at a subsequent time period (240 msec), with amodal completion relying on continued strong responses in these structures.

Introduction

Normal visual perception requires discriminating foreground from background objects. This is sometimes determined from physical differences (e.g., luminance, color, etc.). Frequently, this must be inferred, as when two objects or their parts have the same luminance. Consequently, object borders are perceptually “filled-in” and hence termed “illusory” (see Fig. 1a). Sometimes such object borders are perceived above inducing stimuli and concurrently with a perceived brightness enhancement of the completed shape to produce illusory contours (modal completion), whereas in other cases they are perceived as lying below them and not producing a concurrent brightness enhancement (amodal completion) (Michotte et al., 1964). The basis for the modal-amodal nomenclature derives from whether local contrast changes are perceived, and some limit the use of the term “llusory contour” specifically to the case of modal completion. Nonetheless, in both cases, forms and their borders are perceived, with psychophysical evidence indicating equivalent use of perceived borders to complete a discrimination task (Gold et al., 2000). More fundamentally, it remains controversial whether both instances of perceptual filling-in share common spatiotemporal neural mechanisms and whether perceptual completion, in general, is a bottom-up, feedforward process.

Figure 1.

a, Examples of stimulus configurations forming and not forming modal or amodal illusory shapes. b, Group-averaged (n = 9) VEPs from each stimulus condition at a left and right posterior electrode. c, Results of the spatiotemporal topographic pattern analysis. d, Global field power waveforms from each stimulus condition, as well as the corresponding bar graphs showing results of area measures (SEM shown). See Results for full details.

Using animal intracranial microelectrode recordings, Peterhans and von der Heydt (1989) and Lee and Nguyen (2001) found that lower-tier areas V2/V1 show distinct responses to borders defined modally versus amodally. Neurons responded to modally completed borders with magnitudes resembling those of luminance-defined borders; however, responses from these same neurons were severely diminished by “closed notch” stimuli that would produce amodal completion (Peterhans and von der Heydt, 1989). Collectively, these data suggest that lower-tier regions are sensitive to modal but not to amodal completion. Others, using human behavioral measures, similarly conclude that distinct mechanisms operate for each type of completion process (Sambin, 1987; Corballis et al., 1999).

In contrast, others observed similar responses within macaque V2/V1 to both modal and amodal shapes (Sugita, 1999; Bakin et al., 2000; Zhou et al., 2000), supporting a common bottom-up and low-level mechanism. Evaluation of response latencies in these areas as well as V4, however, reveals that V4 showed the earliest selective response to illusory boundaries (Zhou et al., 2000). Thus, although lower-tier regions may show similar responses to modal and amodal completion, this may reflect feedback modulation (Lee and Nguyen, 2001; Murray et al., 2002; Nieder, 2002).

Such does not preclude the possibility of common mechanisms for both completion types within higher-tier regions. Our previous research examined the mechanisms of modal completion (Murray et al., 2002; Pegna et al., 2002). We observed early visual evoked potential (VEP) modulation (∼90 msec) to the presence versus absence of modally completed shapes that lagged visual cortical response onset by ∼40 msec. Likewise, source estimations and functional magnetic resonance imaging (fMRI) localized this effect to the lateral occipital complex (LOC) bilaterally (Murray et al., 2002). We proposed that the earliest illusory contour sensitivity occurs within higher-tier visual areas, with previous observations in V2/V1 likely reflecting LOC feedback modulation. Here, we further tested this model by comparing modal and amodal completion mechanisms with high-density VEP recordings, topographic analyses, and local autoregressive average (LAURA) source estimations.

Materials and Methods

Subjects. Nine (four female), right-handed neurologically normal, paid volunteers, age 21-33 years (mean ± SD = 24.0 ± 4.3) participated. All had normal or corrected-to-normal vision and provided written consent to the procedures as approved by the Institutional Review Board of the Nathan S. Kline Institute.

Stimuli and procedure. Kanisza-type (Kanisza, 1979) stimuli were presented (114 cm distance) while subjects fixated a central cross. These stimuli were constructed from four “Pacman” inducers oriented to either form or not form an illusory contour (IC and NC, respectively). Square and circular shapes were used (maximal width = 6.5°; ratio between physically present border and that induced = 40%). Inducers were circular, subtended 2.6° of visual angle in diameter, and appeared gray on a black background. Stimuli producing amodal completion were identical to those producing modal completion, except for the inclusion of a gray outline (1.5 mm thick) encircling the inducers. For circular IC shapes, inducers appeared along the vertical and horizontal meridians at 3.25° eccentricity. For square IC shapes, inducers appeared along 45° diagonals from central fixation at 4.6° eccentricity. Thus there were four stimulus conditions: modal IC (MIC), modal NC (MNC), amodal IC (AIC), and amodal NC (ANC) (see Fig. 1a).

Stimuli appeared for 200 msec (randomized 1-2 sec interstimulus interval; 200 msec steps; blank screen containing only the fixation point). Subjects indicated the presence or absence of illusory shapes, regardless of whether they were modally or amodally completed. Subjects had no difficulty with this task, performing at near ceiling levels. Thus behavioral data will not be discussed in further detail here. Each subject completed at least eight 192-trial blocks (mean ± SD = 10.9 ± 2.4). All stimulus conditions were equally probable and randomized within a block of trials. Subjects were encouraged to take breaks between blocks to maintain high concentration and prevent fatigue.

EEG acquisition and analyses. Continuous 128-channel EEG was acquired through Neuroscan Synamps (impedances <5 kΩ), referenced to the nose, bandpass filtered at 0.05-100 Hz, and digitized at 500 Hz. Peristimulus epochs of continuous EEG (-100 to 500 msec) were averaged from each subject separately for each condition to compute the VEP. Baseline was defined as the 100 msec prestimulus period. Trials with blinks or eye movements were rejected off-line, using horizontal and vertical electro-oculography. An artifact criterion of ±60 μV was applied at all other electrodes. The average number of accepted sweeps per condition was 397 ± 109 (range, 259-679). Data from artifact electrodes from each subject and condition were interpolated (Perrin et al., 1987) according to their digitized electrode positions (Polhemus Fastrak). After this procedure and before group-averaging, each subject's data were 40 Hz low-pass filtered, down-sampled to a common 111-channel montage, and recalculated against the average reference.

The initial 350 msec of the VEPs from each condition were submitted to two independent analyses of the electric field at the scalp. The methods applied here have been described in detail previously (Murray et al., 2004). The first was a topographic pattern (i.e., map) analysis. Maps were compared over time within and between conditions, because topographic changes indicate differences in the active generators of the brain. This method is independent of the reference electrode and insensitive to pure amplitude modulations across conditions (topographies of normalized maps are compared). A modified cross-validation criterion determined the number of maps that explained the whole group-averaged data set (Pascual-Marqui et al., 1995). The pattern of maps observed in the group-averaged data was statistically tested by comparing each of these maps with the moment-by-moment scalp topography of individual subjects' VEPs from each condition. Each time point was labeled according to the map with which it best correlated. This revealed whether a given experimental condition is described more often by one map versus another, and therefore whether different generator configurations better accounted for particular experimental conditions.

The second analysis used the instantaneous global field power (GFP) for each subject and stimulus condition to identify changes in electric field strength. GFP is equivalent to the spatial SD of the scalp electric field (Lehmann and Skrandies, 1980). The observation of a GFP modulation does not exclude the possibility of a contemporaneous change in the electric field topography or topographic modulations that nonetheless yield statistically indistinguishable GFP values; however, observation of a GFP modulation without simultaneous topographic changes is explained most parsimoniously as amplitude modulation of statistically indistinguishable generators across experimental conditions. The analysis of a global waveform measure of the VEP was motivated in part by the desire to minimize observer bias that can follow from analyses restricted to specific selected electrodes. GFP area measures were calculated (vs the 0 μV baseline) and submitted to a two-way repeated measures ANOVA, using within subjects factors of modal versus amodal inducers and illusory shape presence versus absence.

Finally, we estimated the sources in the brain underlying the VEPs from each condition, using the LAURA distributed linear inverse solution (Grave de Peralta Menendez et al., 2001, 2004) [for a comparison of inverse solution methods, see Michel et al. (2004)]. LAURA selects the source configuration that better mimics the biophysical behavior of electric vector fields (i.e., activity at one point depends on the activity at neighboring points according to electromagnetic laws). The solution space was calculated on a realistic head model that included 4024 nodes, selected from a 6 × 6 × 6 mm grid distributed equally within the gray matter of the Montreal Neurological Institute's average brain. The results of the GFP and topographic pattern analyses defined time periods showing effects of contour completion and type of inducer with stable scalp topographies for which intracranial sources were estimated. We emphasize that these estimations provide visualization, rather than a statistical analysis, of the likely underlying sources.

Results

As in our previous study (Murray et al., 2002), visual inspection of the group-averaged VEPs revealed a difference between stimulus configurations forming and not forming illusory shapes over the period encompassing the peak of the N1 component at lateral posterior scalp sites bilaterally (Fig. 1b). This evidence of illusory shape sensitivity was observed for both modal and amodal inducer types and was statistically tested using area measurements over the 140-238 msec period from a set of three lateral posterior electrodes bilaterally. These values were submitted to a 2 (shape presence vs absence) × 2 (inducer type) × 2 hemiscalp × 3 electrode ANOVA. Of the main effects, only that of shape presence versus absence was significant (F_(1,8) = 25.104; p < 0.001), with larger responses for conditions forming illusory shapes. There was also a significant interaction between inducer type and hemiscalp (F_(1,8) = 9.873; p = 0.014), with a greater right versus left asymmetry for modal than amodal inducer types. Lastly, there was a significant three-way interaction between shape presence versus absence, inducer type, and hemiscalp (F_(1,8) = 10.833; p = 0.011). This was caused by a larger left versus right hemiscalp difference for modal completion and a larger right versus left hemiscalp difference for amodal completion, although responses to all stimulus conditions were of larger amplitude over the right hemiscalp. No other interactions reached our p < 0.05 significance criterion.

These effects observed at a local scale (specific electrodes) were likewise statistically examined at a global scale to identify effects caused by changes in the underlying brain sources and modulations in response strength. Two analyses of the electric field at the scalp identified the likely neurophysiological basis of modal and amodal completion. The first, the spatiotemporal topographic pattern analysis, provided no indication of distinct maps, and by extension generators, for any stimulus condition. Rather, VEPs from all conditions showed the same sequence of scalp topographies. Five different scalp topographies accounted for the collective 350 msec poststimulus periods of all conditions (Fig. 1c). Different maps were observed over the 0-68, 70-138, 140-238, 240-298, and 300-350 msec periods in each condition and corresponded well with previously described components observed at the waveform level (Doniger et al., 2001; Murray et al., 2002). These time periods of stable scalp topography were then used to define time windows for the GFP analysis, with the rationale that periods of stable scalp topography are an objective means for defining VEP components (Michel et al., 2004).

Similar to the observations at specific electrodes, a repeated measures ANOVA using GFP area over the 140-238 msec period yielded a significant main effect of shape presence versus absence (F_(1,8) = 29.800; p < 0.001). Neither the main effect of inducer type nor the interaction between factors reached our significance criterion. Over the ensuing 240-298 msec period, there was a main effect of shape presence versus absence (F_(1,8) = 7.160; p = 0.028) and a significant interaction between factors of shape presence and inducer type (F_(1,8) = 14.305; p = 0.005), indicative of the larger GFP modulation over this period for amodal versus modal completion. In contrast, the main effect of inducer type was not significant. Over the 300-350 msec period, neither main effect nor their interaction reached our significance criterion. In addition to GFP area, we likewise tested GFP peak latency over the 140-238 msec period. Neither main effect nor their interaction reached our significance criterion. Rather, mean peak latency was 184.0 ± 3.9, 183.1 ± 3.9, 186.0 ± 4.0, and 183.8 ± 4.2 msec for the MIC, MNC, AIC, and ANC conditions, respectively.

To this point, analyses at global and local levels revealed sensitivity to the presence versus absence of illusory shapes during the 140-238 msec period that was independent of the inducer type used. There was no evidence that this modulation followed from a change in the scalp topography. Rather, a common stable scalp topography was observed over this period across all conditions. In contrast, there was a robust GFP modulation, suggestive of a change in the response magnitude of the same underlying network of active brain areas. This is in solid agreement with our previous research comparing illusory and luminance defined contours (Pegna et al., 2002). Over the later 240-298 msec period, a different stable scalp topography (vs that over the 140-238 msec period) was observed for all stimulus conditions indicative of the activity of a distinct brain network; however, there was no topographic modulation between stimulus conditions, providing no evidence of distinct brain networks responsive to modal or amodal completion processes. Rather, there was again a GFP modulation with the presence versus absence of illusory shapes. Additionally, responses to amodal completion were of significantly larger amplitude than those to modal completion or either control condition (Fig. 1). In other words, over this later period there was brain discrimination of amodal and modal completion in terms of the strength of responses, rather than in terms of particular brain areas that were active, with amodal completion yielding stronger activity.

Source estimations were therefore conducted over this 140-238 msec period, as well as over the subsequent 240-298 msec period. We first averaged the VEP for each subject and each experimental condition over each time period. LAURA source estimations were then performed and subsequently averaged across subjects (Fig. 2). In agreement with the implications of the above analyses, identical sources were observed across all conditions that varied in their strength. For the 140-238 msec period, sources were observed within lateral occipital and posterior parietal cortices for all conditions and were stronger in response to conditions inducing an illusory shape. For the 240-298 msec period, sources were again observed for all conditions within lateral occipital and posterior parietal cortices. Additional sources were observed within more anterior regions of the left parietal cortex and left occipitotemporal junction. The most readily apparent modulations over this later period were those within the posterior parietal cortex and lateral occipital cortices, where responses were strongest for amodal completion. Neither period demonstrated strong sources within regions of the calcarine sulcus (i.e., the locus of areas V1/V2).

Figure 2.

Group-averaged (n = 9) LAURA source estimation for each stimulus condition over the 140-238 and 240-298 msec periods.

Discussion

The present study provides evidence that modal and amodal completion processes share a common initial spatiotemporal neural mechanism. Both modal and amodal illusory contour completion processes manifested as a strength modulation of statistically indistinguishable generators without evidence of either topographic or latency shifts across conditions. These effects were first evident over the 140-238 msec period. LAURA source estimations of the active generators during this initial period of shape completion were localized to higher-tier visual areas bilaterally, including areas within posterior parietal and lateral occipital cortices. In what follows, we discuss the implications of these findings on our understanding of perceptual completion processes.

This study constitutes the first electrophysiological demonstration in humans of common mechanisms for modal and amodal completion processes. Evidence from humans had been limited thus far to psychophysical or hemodynamic measures, which are ill equipped to address temporal dynamics of these processes, or to the related process of texture segregation (Caputo et al., 1999) (using VEPs). The high temporal resolution of VEPs is particularly important for determining the commonality of brain mechanisms. In addition, the current analysis procedure provided a statistical means for identifying and differentiating changes in response strength versus scalp topography (generators). These analyses indicate that the earliest detectable sensitivity to illusory shapes, regardless of the inducer type, is attributable to amplitude modulation of statistically indistinguishable scalp topographies. That is, the initial illusory shape processes do not specifically recruit activity in a new area or network, but rather modulate the responses of areas responding to inducer arrays that do not produce illusory perceptions [for corroborating fMRI evidence, see Mendola et al. (1999)]. Psychophysical research similarly supports a common boundary completion mechanism, with discrimination of inducer rotation on the “thin/fat” task improved by the presence of modal and amodal illusory shapes (Ringach and Shapley, 1996; Gold et al., 2000). Although performance differed at short exposure durations (Ringach and Shapley, 1996), suggestive of differential processing at some stage before response execution, this performance difference is not forcibly the result of distinct boundary completion mechanisms. Here, this may be borne out during the 240-298 msec period, which demonstrated an interaction between inducer type and illusory shape presence in GFP amplitude but not scalp topography across conditions. One possibility is that amodal completion may rely more heavily on horizontal and recursive volleys of activity. In support of this, the source analysis of this time period shows stronger responses in posterior parietal and lateral occipital cortices to amodal completion than all other stimulus conditions, including modal completion. This posterior parietal locus is consistent with visuospatial grouping and depth placement functions (Foxe et al., 2003; Ritzl et al., 2003), and the timing of this modulation is consistent with our previous observations suggesting that this time period may reflect more effortful object recognition processes (Doniger et al., 2001; Murray et al., 2002).

Greater GFP amplitude in response to amodal completion relative to all other conditions also speaks to the possibility of selective involvement of V1/V2 in modal completion during this later period. If this were the case, one strong prediction would be a topographic shift across conditions, because these areas would be active for one condition but not others. The present data do not appear to support this prediction. A second prediction would be that modal completion would yield the strongest response magnitude (GFP) over this period, in light of previous results from nonhuman primates (Lee and Nguyen, 2001). This was not the case. Rather, the stimulus configuration leading to amodal completion demonstrated statistically stronger response magnitude.

Given the previous intracranial evidence of completion-related modulation in areas V1/V2, one might ask whether VEP measures are simply insensitive to modulations in these areas or whether activity in such areas might be overshadowed by stronger responses elsewhere (e.g., the LOC and parieto-occipital areas). Despite the evidence from nonhuman primates, however, several recent fMRI investigations of these processes also did not observe completion-specific modulations within these areas, although robust V1/V2 responses versus a baseline condition were obtained (Mendola et al., 1999; Kruggel et al., 2001; Murray et al., 2002; Ritzl et al., 2003). Likewise, our previous work on modal completion did not show completion-specific activity in lowertier areas, whereas clear and robust modulation of early visual processing was obtained as a function of low-level stimulus features and at latencies preceding those sensitive to perceptual completion (Murray et al., 2002). A recent magnetoencephalography study (Halgren et al., 2003) replicated these results, and in this study the authors found some later and presumably feedback modulation in the region of V1/V2, albeit short lived (i.e., <30 msec). These collective results thus favor a feedback interpretation of effects observed in lower-tier areas of nonhuman primates (De Weerd et al., 1996; Lamme and Spekreijse, 2000; Ramdsen et al., 2001; Halgren et al., 2003). Additional experimentation combining anatomic tracing and lesioning-cooling would be better able to resolve the role of areas V1/V2 in perceptual completion. More germane, however, is the fact that the initial sensitivity to modal and amodal completion appears to share a common neurophysiological mechanism in humans, with differential processing occurring only at later latencies.

In summary, noninvasive electrical neuroimaging in humans provide an interpretational framework for animal studies. Our results run counter to predictions based on intracranial results from nonhuman primates (Peterhans and von der Heydt, 1989; Lee and Nguyen, 2001), showing neurons in lower-tier regions responding to modally but not to amodally completed shapes. The strong prediction in the extension to humans would be for topographic-generator differences across stimulus conditions, with modal completion selectively recruiting lower-tier areas (Corballis et al., 1999). A series of analyses revealed that this was not the case. Rather, a common initial mechanism for both types of completion processes is demonstrated (140-238 msec), manifesting as a modulation in response strength within higher-tier LOC and parietal structures. Differential mechanisms were only evident later (240-298 msec), with amodal completion relying on continued strong responses in these structures. The present results thus provide further support both for a model of illusory contour sensitivity that initiates within higher-tier visual areas, as well as for the interpretation of effects seen in lower-tier areas V1/V2 as reflecting feedback modulation.