Disambiguating the roles of area V1 and the lateral occipital complex (LOC) in contour integration

Abstract

Contour integration, the linking of collinear but disconnected visual elements across space, is an essential facet of object and scene perception. Here, we set out to arbitrate between two previously advanced mechanisms of contour integration: serial facilitative interactions between collinear cells in the primary visual cortex (V1) versus pooling of inputs in higher-order visual areas. To this end, we used high-density electrophysiological recordings to assess the spatio-temporal dynamics of brain activity in response to Gabor contours embedded in Gabor noise (so-called “pathfinder displays”) versus control stimuli. Special care was taken to elicit and detect early activity stemming from the primary visual cortex, as indexed by the C1 component of the visual evoked potential. Arguing against a purely early V1 account, there was no evidence for contour-related modulations within the C1 timeframe (50-100 msecs). Rather, the earliest effects were observed within the timeframe of the N1 component (160-200 msecs) and inverse source analysis pointed to principle generators in the lateral occipital complex (LOC) within the ventral visual stream. Source anlaysis also suggested that it was only during this relatively late processing period that contextual effects emerged in hierarchically early visual regions (i.e. V1/V2), consistent with a more distributed process involving recurrent feedback/feedforward interactions between LOC and early visual sensory regions. The distribution of effects uncovered here is consistent with pooling of information in higher order cortical areas as the initial step in contour integration, and that this pooling occurs relatively late in processing rather than during the initial sensory-processing period.

Introduction

Contour integration, the ability to link collinear but disconnected visual information across space, is considered an essential element of object and scene perception. Despite significant progress in our understanding of the physiology of the visual cortex, the precise neural mechanisms of contour integration remain unclear. Two alternative mechanisms have been suggested. The first relies on lateral horizontal connections within the primary visual cortex, which connect cells with similar orientation tuning characteristics across the visual field. These so-called collinear cells, which are sensitive to neighboring regions of visual space, are posited to communicate via long-range horizontal connections resulting in local integration of contour elements (Bauer and Dicke, 1997; Bauer and Heinze, 2002; Gray, 1999; Grossberg and Williamson, 2001; Hess et al., 2003; Kapadia et al., 1995; Kovacs, 1996; Stettler et al., 2002). The second possible mechanism instead involves integration across different levels of the visual hierarchy, with inputs from primary visual cortex converging on cells with larger receptive fields in higher-order regions of visual cortex such as the lateral occipital complex (Hubel and Wiesel, 1962; Spillmann and Werner, 1996). These two mechanisms make a set of clearly dissociable predictions, not only in terms of the cortical regions involved, but also in terms of the timing of contour integration. If the initial integration step occurs locally in V1, one would predict correspondingly early differential neural response modulations originating from neuronal ensembles in the primary visual cortex (Gray, 1999; Hess et al., 2003; Kapadia et al., 1995; Kovacs, 1996; Stettler et al., 2002). Alternatively, if contour integration initially depends on the pooling of inputs in higher order visual areas, the initial contour-specific response should be detected somewhat later in processing and in higher-order extrastriate visual areas (Foxe et al., 2005; Murray et al., 2002).

Arguments in support of contour computation in the primary visual cortex come from anatomical and physiological studies in animal models suggesting that V1 is capable of integrating information over a larger extent of the visual field than initially considered. Although classical receptive fields extend only a fraction of a degree in the fovea (Hubel and Wiesel, 1962), they can be modulated by local horizontal connections responding to visual elements outside the classical receptive field (Allman et al., 1985; Angelucci et al., 2002; Blakemore and Tobin, 1972; Gilbert et al., 1996; Gilbert et al., 1990; Levitt and Lund, 1997; Nelson and Frost, 1978; Rockland and Lund, 1983; Walker et al., 1999). Neurons of a given orientation selectivity are preferentially connected to other neurons (across orientation columns) with a similar orientation preference and adjacent receptive fields (Hirsch and Gilbert, 1991; Schwarz and Bolz, 1991; Stettler et al., 2002; Ts'o et al., 1986; Weliky et al., 1995). Support for the role of early visual cortex has also been derived from behavioral studies indicating that the physical constraints of contour integration are consistent with the physical organization of hypercolumn within the primary visual cortex. One of the classical psychophysical paradigms explores so-called “association fields” of visual cortex, using “pathfinder displays.” Here, Gabor patches, designed to be well-matched to the tuning properties of cells in V1, are arranged to form continuous contours of about ten-element length while their relative orientation and distance is varied (see Figure 1) (Field et al., 1993). The term “association field” describes the tendency of neighboring elements to be grouped together within certain constrains (i.e., ≤ 6 carrier wavelengths (λ) spatial separation and ≤30° orientation difference) (Field et al., 1993; Hess et al., 2003). Together with anatomical considerations (i.e., preferential connectivity between hypercolumns with similar tuning characteristics) (Das and Gilbert, 1995; Gilbert and Wiesel, 1989; Malach et al., 1993; Ts'o et al., 1986), these behavioral tests of “association fields” have been taken to suggest that V1 plays a major role in contour processing (Field et al., 1993; Kovacs, 1996).

Figure 1. Exemplar Pathfinder Stimulus Arrays.

Arrays of ten-by-ten randomly oriented Gabor elements (4 cycles per degree) were constructed with inter-element separation randomly jittered between 1 and 1.5 degrees of visual angle. In the case of contour-present arrangements, 10 of the Gabor elements were oriented within ± 30 degrees relative to the neighboring elements with no phase shift. An example is shown in the left hand panel where this arrangement of elements results in the perception of a snaking contour, which can be seen in the lower right corner of the panel. In the right hand panel, orientations of all elements are fully random and no countour is seen.

The neural underpinnings of contour integration have also been investigated using so-called ‘illusory contour’ (IC) stimuli, a stimulus class specifically constructed to induce perceptions of contours spanning across gaps where the discrete luminance boundaries of real contours are absent. Although earlier intracranial studies in animal models and human neuroimaging studies localized IC processing to V1 and V2 (Bakin et al., 2000; Ffytche and Zeki, 1996; Larsson et al., 1999; Lee and Nguyen, 2001; Peterhans and von der Heydt, 1989; Ramsden et al., 2001; Seghier et al., 2000; von der Heydt and Peterhans, 1989), these studies did not assess the timing of these effects, leaving open the possibility that modulations in V1 and V2 represented feed-back from higher-order visual regions.

In contrast, more recent neuroimaging and event-related potential (ERP) studies have emphasized the contributions of generators within higher-order ventral visual stream regions – the so-called Lateral Occipital Complex (LOC) (Altschuler et al., 2011; Altschuler et al., 2012; Fiebelkorn et al., 2010; Foxe et al., 2005; Halgren et al., 2003; Kruggel et al., 2001; Mendola et al., 1999; Murray et al., 2004; Murray et al., 2006; Murray et al., 2002; Ritzl et al., 2003; Shpaner et al., 2009). In these studies, the timing of initial IC processing was explicitly interrogated, and it was shown that initial modulations actually occurred relatively late in processing, and were first observed in the LOC rather than in early visual regions. As such, these studies suggested that previously observed V1/V2 modulations to IC stimuli likely reflected feedback processes.

Here we used high-density electrophysiology to arbitrate between the thesis that long-range horizontal connections in V1 provide the principle initial substrate of contour integration, and the competing notion that this integration initially relies on pooling of neuronal inputs in higher-level visual cortex (i.e., the LOC). For comparability with the preceding literature, stimuli were modeled on those used in association field studies (Field et al., 1993). We reasoned that if initial contour-related modulations were observed during the earliest phases of sensory processing and were localized to primary visual cortex, this would support the intrinsic horizontal connectivity model. Alternatively, if initial contour-related modulations were found to be localized in higher-order regions of visual cortex such as the LOC, and were observed during later sensory-perceptual processing phases, this pattern of results would support a model wherein pooling of inputs in visual regions with larger receptive fields represents the initial stage of contour integration.

Materials and Methods

Participants

Twenty neurologically normal volunteers participated. Four participants were dropped from the study because they did not cooperate with the task (based on the assessment of the EEG technician and confirmed by very low behavioral accuracy as indicated through calculation of d′ values), an additional participant was dropped because he could not maintain fixation. The remaining fifteen participants (6 female) were between 22 and 40 years of age (mean age = 28±5). All participants had normal or corrected-to-normal vision, and were right-handed as assessed with the Edinburgh Handedness Inventory (Oldfield, 1971). Participants provided written informed consent, and the Institutional Review Boards of the Nathan Kline Institute for Psychiatric Research and the City College of New York approved all procedures. All procedures conformed to the principles of the Helsinki declaration. Participants received a modest monetary compensation ($10 per hour).

Design and Task

Gabor patches of 0.25 carrier wavelength, λ, (i.e., 4 cpd) were used to generate forty different contours embedded in randomly oriented Gabor elements (an algorithm from Field and Hess (Field et al., 1993) was employed using Matlab v. 15). Contours consisted of ten Gabor elements, oriented within ± 30 degrees relative to the neighboring elements (a random value, generated from a uniform non-Gaussian distribution, determined the orientation), with no phase shift, presented at 100 percent Michelson contrast. Arrays of ten by ten elements were constructed. Inter-element separation was jittered between 4-6 λ (i.e., 1-1.5 degrees visual angle). Orientation of the elements was not jittered relative to the path. Contours did not close to form full geometric shapes. Forty randomly oriented displays of Gabor patches were also constructed by randomly rotating the orientation of the contour elements. This resulted in identical displays for the contour-present and contour-absent conditions, except for the orientation of the ten aligned contour elements (Figure 1). Stimulus arrays subtended 10×10 degrees of visual angle (with each Gabor occupying 1 degree of visual angle), and were presented either to the upper left or the lower right quadrant of the screen. Off-center presentation was necessary to evoke the retinotopically generated C1 component of the visual evoked potential. Only displays with contours relatively close to the central fixation cross were selected by visual inspection, as peripheral contours proved to be too difficult to detect, with participants performing at chance during pilot sessions. Specifically, when automatically generated ten-element contours were limited to the most peripheral (farthest away from fixation) quadrant of the 10×10 stimulus array, they were rotated into central space.

Stimuli were presented on the screen for 200 msec, after which participants had unlimited time to make a button push response indicating presence or absence of the contour with the index and middle fingers of their right hand. The finger assignment was counterbalanced across participants. The response was followed by a 100-500 msec variable inter-trial interval after which the next trial began. Participants maintained fixation on a centrally presented cross. Each of the 80 possible displays (40 contours and 40 no-contours) was presented 12 times in random order for a total of 480 trials per condition. To assess behavioral performance, d′ measures were calculated based on the formula appropriate for a yes-no paradigm: d′ = z(Hits) – z (FalseAlarms) (where z is the inverse of the normal distribution function) (Macmillan and Creelman, 2005). The d′ measures were submitted to a two-tailed paired Student's t-test with the factor of side of presentation. Response bias was estimated using the c metric, c=−1/2 [z(Hits) + z (FalseAlarms)] (Macmillan and Creelman, 2005).

Electrophysiological Data Acquisition and ERP Derivation

Continuous EEG was acquired through a Biosemi ActiveTwo system from 168 scalp electrodes, digitized at 512 Hz, and referenced to the CMS-DRL ground (which functions as a feedback loop driving the average potential of the subject, i.e. the Common Mode voltage, as close as possible to the AC reference voltage of the AD box, i.e. the amplifier zero). EEG processing and analyses were performed using the Cartool software by Denis Brunet (http://brainmapping.unige.ch/cartool.htm). For each participant, the continuous EEG was segmented into 700 msec epochs that were timelocked to the stimulus events (from −200 msec before stimulus onset to 500 msec after stimulus onset). Individual subject averages were then generated for each of the two stimulus condition. Only correct responses were included in the average. An artifact rejection criterion of ±75 μV was used at all scalp sites to reject trials with excessive EMG, horizontal or vertical eye movements, and other noise transients. The average number of accepted sweeps per condition was 301±82 with a range of 127 to 439 sweeps. Data from electrodes containing major noise transients were interpolated (Perrin et al., 1987). Prior to group-averaging, ERP data were baseline corrected from −50 to +20ms and re-referenced to the average reference. A 45Hz low-pass filter was applied for visualization of the waveforms.

Earliest ERP Component Analysis Strategy

The key question for the current study is the delineation of the timecourse and localization of contour integration. To this end, special care was taken to achieve enough power to detect the earliest cortical differences. C1 is the earliest visual ERP component, believed to be a marker of V1 activity due to its highly retinotopic distribution, early timing (peaking between 65-90 msec) and repeated source localization to the primary visual cortex (Di Russo et al., 2002; Foxe et al., 2008; Gomez Gonzalez et al., 1994; Jeffreys and Axford, 1972). V1 has been shown to have highly variable anatomy across subjects, a feature likewise observed in subject-by-subject analysis of the C1 (Foxe and Simpson, 2002; Jeffreys and Axford, 1972; Molholm et al., 2002; Rademacher et al., 1993). That is, individual differences in V1 folding patterns influence the orientation of the C1 generators, leading to highly variable C1 scalp distributions and even in some cases an inability to detect the electrical activity with scalp recordings. At the same time, subjects with a robust C1 will reliably exhibit “typical” C1 topography (e.g. upper field stimuli will produce a contralateral negative potential, while lower field stimuli will produce a contralateral positive potential) (Kelly et al., 2008). In this study, we used retinotopically constructed presentations designed to elicit positive and negative going C1s. All participants exhibited a robust C1 component and were therefore included in subsequent analyses.

We first undertook the analysis of the C1 component to determine whether earliest cortical contour integration effects can be established in V1. Due to the high degree of inter-subject variability in the topography of the C1 component (as described above), C1 was defined on an individual basis as the earliest deflection peaking before 100 msec post-stimulus onset (Kelly et al., 2008). For each participant, electrode sites showing the peak positive and negative C1 deflections respectively (corresponding to lower and upper stimulus presentations) were identified automatically using Cartool software and used in the subsequent analysis. Mean amplitude over time (68-100 msec) was submitted to a 2×2×2 MANOVA with the factors of stimulus condition (“contour” vs. “no contour”), stimulus location, and hemisphere. For this and subsequent statistical analyses, the precise temporal window was defined based on the results from the topographic pattern analysis (described below). This is an objective approach to defining periods of stable topography centered on the major visual components of interest.

Statistical Analyses of Later Components

Analyses were focused on the N1 and the Ncl (“closure negativity”) components, which have been repeatedly implicated in contour integration. Electrodes for all tests were picked based on the location of the peak amplitude of the response of interest (N1 or Ncl) in the group average data. The following time-periods were used: 165-193 msec for the N1 and 315-350 msec for the Ncl, based on the identification of stable topographies (see below) within the timeframes of interest. Mean amplitude measures over time were submitted to a 2×2×2 MANOVA with the factors of stimulus condition, stimulus location, and hemisphere.

Detection of the Onset of Contour Integration Processes

To calculate when differential processing of the contour-present stimulus onset, we employed point-wise paired t-tests between the ERP responses to the two stimulus conditions (“contour” vs. “no contour”). For each electrode, the first time point where the t test exceeded the 0.05 α criterion for at least 10 consecutive data points (19.5 msec at a 512 Hz digitization rate) was labeled as the onset of the effects (see Guthrie and Buchwald, 1991; Wylie et al., 2003 for rationale).

Topographic Analyses

Three steps were taken in analyzing the topographies of the responses. In the first, the presence of differences in topography between the conditions was assessed; in the second, periods of stable topography were identified; and in the third, the intracranial sources of the topographies were modeled.

To statistically identify periods of topographic modulation, the topographic analysis of variance (TANOVA) procedure was used. This procedure is complementary to the analysis described in the preceding paragraph in that it identifies the onset of contour-related topographic modulation. ERP differences that do not coincide with differences in topography reflect modulation of identical intracranial generators. TANOVA computes global dissimilarity (Lehmann and Skrandies, 1980) between conditions for each time point of each subject's data. Global dissimilarity is an index of configuration differences between two electric fields, independent of their strength. This parameter equals the square root of the mean of the squared differences between the potentials measured at each electrode (vs. the average reference), each of which is first scaled to unitary strength by dividing by the instantaneous standard deviation. Dissimilarity can range from 0 to 2, where 0 indicates topographic homogeneity and 2 indicates topographic inversion. A Monte Carlo MANOVA is then applied (Manly, 1997) to test for statistical differences in the dissimilarity between conditions. This is a nonparametric bootstrapping procedure, wherein each subject's data from each time point is permutated such that they could “belong” to either stimulus condition. The dissimilarity is then calculated for each of 8000 such permutations for each time point and is used to generate a distribution of values against which the observed data are compared. From this, we determined the probability of obtaining a dissimilarity value from the permutations that exceeded the actual measured value. Since electric field changes are indicative of changes in the underlying generator configuration (Lehmann, 1987); this test provides a statistical means of determining if and when the brain network activated by the two conditions differs.

To further characterize electric field potentials for the different types of stimuli, periods of stable topography were estimated using topographic pattern analysis (Michel et al., 2001). A modified cross-validation criterion determined the number of maps that explained the whole group-averaged data set (Pascual-Marqui et al., 1995). This method is reference independent and insensitive to pure amplitude modulation across conditions and over time. Different topographic maps reflect differences in the active generators of the brain. The maps obtained as a result of this procedure were used to define the stable topographic time periods for ERP statistical analyses and for estimating the sources of the ERP modulation for each stimulus type (see below). Since we did not detect any topographic differences in the contour and no-contour responses in the C1 and N1 timeframes, contour and no-contour conditions were combined for C1 and N1 localizations.

We next estimated the underlying neural generators of the responses of interest with LAURA distributed linear inverse solution. LAURA employs a realistic head model with 4024 nodes, arranged in a 6X6X6 mm grid within the gray matter of the Montreal Neurological Institute's average brain (Grave de Peralta et al., 2001; Grave de Peralta et al., 2004). The procedure selects the best source configuration based on the biophysical behavior of electric vector fields according to electromagnetic laws. Individual source models for each subject were obtained and averaged to produce the grand average model. Differences in the inverse solution space were also obtained for the Ncl time-period by subtracting solution for the no-contour condition from the contour condition. Strength modulations of over 20 percent are reported.

Results

Behavioral Results

The contour detection task proved to be challenging for participants, likely due to the lateralized presentations. Accuracy of contour detection for the left and right visual field presentations was 62±17 and 72±15 percent respectively (mean±SD). Accuracy of no-contour detection was 88±8 and 81±12 percent respectively. d′ for the left and right visual field presentations was 1.66±0.45 and 1.63±0.39 respectively. Most participants were biased to respond as if no contour was present, c= 0.48±0.42 and 0.14±0.43 for left and right visual field presentations, respectively. Note that only correct trials were used for the ERP analyses. The t-test with the factor of side of presentation was not significant (t=0.8); neither stimulus location resulted in an advantage in behavioral performance.

Earliest ERP Component Originating in the Primary Visual Cortex

This study was designed to uncover the earliest possible ERP modulations resulting from contour integration processes. The C1 component inverted polarity in a retinotopically predictable fashion for opposite sides of presentation as anticipated, and there was a clear inversion across the electrode sites distributed over the head (Figure 2). Consistent with the literature, left-lateralized presentations resulted in a negative deflecting C1 over the medial occipital sites and a positive deflection over the right lateral occipital sites (Di Russo et al., 2002; Kelly et al., 2008). Right-lateralized presentations resulted in a positivity over the left medial occipital sites and a negativity over the right lateral occipital sites. Visual inspection of the waveforms revealed no differences between contour and no-contour conditions for either side of presentation. The MANOVA over the 68-100 msec, as well as the Running t-tests (Figure 4), revealed a complete absence of significant differences as a function of condition (F_1,14=0.606, p=0.449). As expected, there was an interaction between hemisphere and side of presentation (F_1,14=141.865, p<0.001) consistent with the C1 inversion. The main effect of hemisphere also reached significance with more positive-going waveforms observed over the right scalp (F_1,14=4.821, p=0.045).

Figure 2. The C1 response.

The scalp topography of the group-averaged C1 response (from 68-100 msec, averaged across CON and NOCON conditions) is presented separately for lower-right presentations (top of left side of panel) and upper-left presentations (bottom of left side of panel). This illustrates clear C1 polarity inversion as a function of stimulus location, consistent with the retinotopic organization of V1/V2. To the right of the scalp topographies, waveforms for CON (red trace) and NOCON (blue trace) conditions illustrate the similarity of the CON and NOCON responses in the C1 timeframe for a given presentation location, and C1 inversion across the lower-right versus upper-left presentations.

Figure 4. Statistical Cluster Plots of Contour Effects.

The statistical cluster plots depict significant differences between CON and NOCON conditions from pair-wise running t-tests (threshold criterion of p<0.05 for 19.5 consecutive msec), for the lower right (upper panel) and upper left (lower panel) presentations. Electrodes are arranged vertically according to the head montage on the left. This illustrates that contour effects onset in the timeframe of the N1 (∼170 ms), and persisted into the Ncl timeframe (∼200-350 ms). Gray shading reflects periods of topographic divergence between the CON and NOCON responses according to the TANOVA analysis.

LAURA solutions for the C1 were consistent with generators within the primary visual cortex. The upper-left stimuli led to generator sources in the lower bank of the calcarine fissure within the lingual gyrus (x=-17, y=-87, z=1), while the lower-right stimuli had generators in the middle occipital gyrus (x=29, y=-81, z=4), on the upper bank of the calcarine fissure.

C1 Power Analysis

Clearly, a central aspect of the current approach involves ruling out modulations during the early sensory processing period represented by the C1 component, and this necessitates some embrace of the null-hypothesis. Thus, we conducted follow-up power analyses to assess the magnitude of modulations that can reasonably be ruled out. Adopting an industry standard of 0.8 power, modulations of 7% or greater could be ruled out for left-sided presentations and of 9% or greater for right-sided presentations.

Later ERP Components: N1 and Ncl

ERP responses to the contour and the no-contour conditions are displayed in Figure 3, together with topographic plots for the N1 time-period (165-193 msec). The omnibus MANOVA revealed a main effect of contour (F_1,14=9.71, p=0.008), with a more negative going response to the contour than the no-contour condition. There was a three-way interaction between contour, hemisphere and side of presentation (F_1,14=10.552, p=0.006). The contour-related effect was lateralized primarily to the contralateral scalp relative to the side of presentation, as evident from the topographical plots (see insets of Figure 3, panels A and B). The MANOVA also resulted in a main effect of hemisphere (F_1,14=15.814, p=0.001), with more negative N1 amplitudes over the right hemiscalp; and an interaction between hemisphere and side of presentation (F_1,14=14.787, p=0.002), with more negative N1 amplitude on the contralateral side of presentation. The running t-tests (Figure 4) showed that, like previous ERP and MEG results using IC stimuli (Foxe et al., 2005; Halgren et al., 2003; Kruggel et al., 2001; Mendola et al., 1999; Murray et al., 2004; Murray et al., 2006; Murray et al., 2002; Ritzl et al., 2003), contour related modulation onset about 10 msec prior to the peak of the N1 component, at 150 msec for the right side of presentation and at 159 msec for the left (Figures 3 and 4). The TANOVA analysis revealed topographic divergence between the conditions shortly after the onset of statistical differences followed at 190 msec for the left presentation and 195 msec for the right (gray shaded areas on Figure 4). Inverse solutions in this timeframe were consistent with intracranial generators in the LOC, and were stronger in the contralateral hemifield to the side of presentation (Figure 5, panel A). The solution maxima were found within the LOC in the inferior occipital gyrus (x=29, y=-81, z=-4) for the left side of presentation and in the middle temporal gyrus (x=−47, y=-58, z=0) for the right side of presentation. In addition to the strong primary bilateral sources within the LOC regions, separable sources were also observed during this timeframe within hierarchically earlier bilateral visual regions, mainly in the region of the cuneus/lingual gyri: in the left hemisphere (x=−17, y=-87, z=1) for both stimulus locations and in the right (x=17, y=-87, z=1) for the right side of presentation only (Figure 5, panel C).

Figure 3. Contour integration effects.

Waveforms illustrate the contour integration effect in the N1 timeframe, with the CON response (red trace) more negative-going than the NOCON response (blue trace). For bottom right stimulus presentations (A and C, representing left and right scalp regions respectively), the differential response is largest over left occipital scalp (see A: waveforms and voltage map of the difference wave (from 165-193 msec)). For top left stimulus presentations (B and D, representing left and right scalp regions respectively) the differential response is focused over right occipital scalp (see D: waveforms and voltage map of the difference wave (from 165-193 msec)).

Figure 5. Source Estimations of Contour-Related Effects.

Group-averaged LAURA distributed linear source estimations were calculated for the ERP waveforms over the N1 (165-193 msec) time period (panel A and C) and the Ncl (315-350 msec) time period (panel B). No topographic divergence was observed in the N1 time-period, with both conditions showing similar intracranial generators. Thus, only a single source estimation is presented for each stimulus location. Panel A depicts strong contributions from inferior occipital gyrus (left side presentation) and middle temporal gyrus (right side presentation) in this time period. The center panel displays the ventral view (R= right side, L=left side). Panel B depicts topographies for contour-present (CON) and contour-absent (NOCON) conditions separately because there was a significant contour-related topographic divergence during the Ncl time-period. Distributed source activations were observed over the occipital and temporal regions (see Table I for specific details). Panel C depicts source estimations derived from the subtraction waves between the NOCON and CON conditions. The strong contributions from LOC to differential processing are evident in this timeframe, and there is also evidence of potential feedback modulations in hierarchically earlier visual sensory regions.

The waveforms showed a second period of significant divergence in the timeperiod of the Ncl (Doniger et al., 2000; Doniger et al., 2001; Foxe et al., 2005; Sehatpour et al., 2010; Sehatpour et al., 2006; Sehatpour et al., 2008), with responses in the contour condition more negative-going than in the no-contour condition (F_1,14=9.652, p=0.008). There was an interaction between hemisphere and contour (F_1,14=5.522, p=0.034), with larger contour effects on the left. There was also an interaction between side of stimulation and hemisphere (F_1,14=19.792, p=0.001) due to the fact that modulations were primarily seen over contralateral hemisphere. Finally, there was a three-way interaction (F_1,14=13.153, p=0.003) due to stimulation side influencing the hemisphere where contour effects were seen. The TANOVA analysis revealed topographic differences between contour and no-contour conditions in this time-frame (Figure 4, gray shading). Therefore, inverse solutions were obtained separately for each condition and resulted in bilateral sources in the temporal and occipital lobes, within Brodmann areas 37, 22, 21 and 19 (see Table I and Figure 5, panel B). LAURA analysis revealed that the major sources of topographic divergence between the conditions (obtained by source modeling the subtraction of the responses to the no-contour and contour conditions) were in the left frontal and the left post-central gyri for the right presentations, whereas for left-side presentations they were in the middle temporal regions bilaterally, left postcentral gyrus, and right precuneus and right middle frontal gyrus (Table I).

Table I. Ncl Source Estimations.

condition	x	Y	z	Anatomical structure	BA
con left	41	-70	-4	Right Inferior Occipital Gyrus	BA 19
con left	-59	-28	3	Left Superior Temporal Gyrus	BA 22
nocon left	47	-58	-5	Right Inferior Temporal Gyrus	BA 19
con right	-53	-51	9	Left Superior Temporal Gyrus	BA 22
con right	-47	-63	10	Left Middle Temporal Gyrus	BA 37
nocon right	47	-58	0	Right Inferior Temporal Gyrus	BA 19
nocon right	-53	-51	9	Left Superior Temporal Gyrus	BA 22
nocon right	-59	-29	-1	Left Middle Temporal Gyrus	BA 21
right-diff	-47	18	28	Left Middle Frontal Gyrus	BA 9
right-diff	-65	-21	30	Left Postcentral Gyrus	BA 2
left-diff	-47	-69	10	Left Middle Temporal Gyrus	BA 39
left-diff	-41	-37	59	Left Postcentral Gyrus	BA 2
left-diff	17	-73	38	Right Precuneus	BA 7
left-diff	47	8	45	Right Middle Frontal Gyrus	BA 6

Talairach coordinates for the focal points of the inverse solutions for the Ncl timeframe (‘diff’ indicates at least 20% strength modulation between contour (con) and no-contourn (nocon) conditions). BA = Broadman's area.

Discussion

This investigation set out to arbitrate between competing theories of contour processing. High-density recordings of ERPs were made to well-matched visual stimuli that either did or did not contain a contour, and the spatio-temporal properties of contour processing assayed. To arbitrate between theories, it was important that the methods were sensitive to detecting the earliest visuo-cortical modulations possible. Careful steps were therefore taken to ensure maximal sensitivity to modulations of the C1 response, and data were only considered for trials in which the contours were correctly detected. Despite these specific efforts to evoke larger early activations of V1/V2 through the use of retinotopic mapping strategies, we found no evidence whatsoever of modulations in the C1 timeframe. Since the C1 is considered to reflect initial activation of the early visual cortex, the data argue against a model of contour processing in which long-range horizontal connections in V1 subserve the initial integration of contour elements. In fact, earliest ERP differences were detected in the N1 time period beginning considerably later at 170 msec. These differences were primarily localized to higher-tier ventral stream visual areas, primarily the LOC. No topographic differences were identified over this time period, pointing to a modulation of the strength of activity within identical intracranial generators. The initial divergence was followed by a further topographic difference, onsetting around 190 msec. This topographic difference extended into the later Ncl timeframe and may reflect higher order decision-making processes, involving prefrontal, LOC and subcortical interactions (e.g., Murray et al., 2002, Sehatpour et al., 2008). Taken together, the current results support a model whereby initial contour integration occurs via pooling of neuronal responses in the LOC and suggest that previous contour processing effects observed in early visual areas are likely due to feedback inputs from these higher-tier regions. Indeed, source analyses during the N1 processing timeframe did point to additional contextual effects in lower-tier regions (i.e. V1/V2) after 170 ms, suggesting a distributed process involving recurrent feedback/feedforward interactions between LOC and early visual sensory regions.

In order to embrace the lack of effects in early visual areas, we needed to ascertain that our metrics have sensitivity to detect such effects. Prior studies reported modulations of the C1 component ranging from 20 to 50 percent in response to different levels of luminance contrast (e.g., Butler et al, 2007) as well as in the context of attentional amplification (Kelly et al., 2008) and multisensory integration (Molholm et al., 2002). Analogous amplitude modulations can be found in the animal literature, where early attentional (McAdams and Reid, 2005) and contrast-related (Pooresmaeili et al., 2010) effects have been reported in the spiking rates of V1 cells. Power analysis of our data indicated that we were powered to detect very modest modulations of the C1 component in the range of 7 to 9 percent at the 0.8 level. The high level of power achieved in our dataset gives us confidence that activity stemming from the primary visual cortex was indeed adequately sampled. Nevertheless, it remains possible that the present methods were insensitive to even smaller modulations of the C1 component, although examination of the response waveforms shows no hint of modulation whatsoever.

It is also important to point out that the contour stimuli used in this study, while readily detected by the participants, were not as perceptually salient as some varieties of this class of stimuli. For example, contour paths did not form complete circles or other “closed” forms, and these Gestalt properties tend to make pathfinder displays “pop-out” perceptually (Kovacs and Julesz, 1993; Kuai and Yu, 2006). It might be suggested that presentation of more salient contours of this variety could engage V1 during the feedforward volley. Clearly, however, the physical features that the contours are constructed from do not vary between open and closed pathfinder displays. As such, the salience of the stimulus is dependent on the very closure processes that are being tested. Until these processes have been engaged and the figure has been closed, the fact that the display is “open” or “closed” cannot be accessed by the system. This dissociation must necessarily be made at a later perceptual processing level, rather than during initial sensory processing. An alternate account might be that through exposures to repeated versions of the more salient “closed” displays, the entire system is recalibrated in a top-down manner such that V1 is now engaged in initial sensory processing of these stimuli. This seems highly unlikely to us for the following reasons. First, if anything, the slightly more difficult perceptual task of identifying the less salient “open” displays would surely be the more likely condition to result in engagement of earlier processes, given that the task of detecting the contours is more attentionally demanding. Second, the notion that V1 has two entirely different modes of operation during the initial sensory processing period for two varieties of a single class of stimuli, both constructed from precisely the same basic elements, does not have parsimony on its side.

Nonetheless, a limitation of the present study is that only one variety of the pathfinder displays was assessed. A number of other parameters can be varied to influence the perceptual saliency of these displays, such as the use of straight line versions (Beaudot and Mullen, 2001) and more closely spaced elements (Li et al., 2006). While the same arguments as forwarded above should also apply to these varieties, the use of more closely spaced inducing elements may present a special case. Since the premise of the local integration model relies on collinear cells that are sensitive to neighboring regions of visual space, and these are posited to communicate via long-range horizontal connections, the closer together the elements of the display, the shorter the distance across which this communication must propagate. It remains possible that at inter-element distances below a certain threshold, yet to be determined, a local integration model might apply. Nonetheless, the study by Li and colleagues (2006) is instructive in this regard. They recorded responses in V1 neurons to precisely these sorts of closely spaced linear pathfinder arrays in macaque monkeys performing a contour detection task. They manipulated the perceived salience of the arrays by varying the distance between the collinear elements or by varying the raw number of elements in the linear array. The animals were presented with bilateral arrays and performed the task by saccading to the side containing the contour. While the main finding of the paper was that the responses of orientation selective V1 neurons were highly sensitive to the contextual influence of the illusory pathfinder displays, and that this sensitivity was tightly correlated with behavioral measures of perceptual saliency, it is in the relative timing of these context-driven response modulations that we find strong support for our position. In both animals, the observed contextual modulation of V1 firing, regardless of the level of saliency, was seen to onset considerably later than the initial onset of responsivity in a given neuron. That is, while the onset of responses in these cells was seen between 50-60 msec (see their Figure 7), the contextual effects were found to begin considerably later at 90-100 msec in one animal and 150-160 msec in the second animal. When one compares the timing of contextual effects between highly salient and less salient displays, there is no evident difference in the lateness of these contextual effects. While Li and colleagues, who only recorded responses in V1, generally preferred to interpret their results as support for the horizontal propogation model, they do allow that the data could also be consistent with the involvement of “interaction between feedback connections from higher-order visual areas to V1 and long-range horizontal connections intrinsic to V1.” Regardless, it is clear from their data that increasing perceptual saliency of the pathfinder displays did not induce earlier modulations of V1 firing during the initial feedforward afferent volley.

Bauer and Heize (2002) recorded single cell spiking activity in supragranular layers of the primary visual cortex of monkeys in response to pathfinder displays. They compared cellular response to the display where the classic Response Field (cRF) was part of a path to the sum of responses where the cRF was not part of a path and where the path was present but the Gabor in the cRF was actually missing (i.e., there was a gap in the path). A non-linearity between the comparison conditions (the response to the contour was not a linear sum of its parts) was taken to signal integration. They found initial inhibition of the transient response to the continuous path followed by facilitation around 150 to 200 msec post-stimulus onset. These authors chose to interpret these results in terms of local facilitation via long-range horizontal connections. Once again, however, given the very late timing of these results, feedback facilitation would also be entirely consistent in our view.

To date, there exist less than a handful of human ERP studies that can speak to association field theory (Khoe et al., 2004; Mathes and Fahle, 2007; Mathes et al., 2006; Tanskanen et al., 2008). To recap, the association field theory argues that contour integration is based upon interactions between neighboring receptive fields in V1 that lie upon a smooth curve. Mathes and colleagues (2007; 2006) used contours embedded in noise to study the physiology of contour integration. They consistently found a relatively late (>200 msec) ERP negativity for contours as compared to noise. Earlier effects, in the timeframe of the visual N1 (∼170 msec peak) were somewhat inconsistent. Although the authors attributed these inconsistencies to different task demands, it is more likely that they lacked the statistical power to observe early effects due to the low sweep counts and participant numbers used in these studies. Data in these studies were not tested for any earlier ERP differences than the N1. Our data are consistent with these results since we also observed ERP differences in the N1 and Ncl timeframes; moreover, earlier differences were not detected even with a design specifically targeting early components. At the same time, Tanskanen et al. (2008) analyzed the early timeframes of magnetoencephalographic recordings in response to pathfinder displays. In accord with the results of the present study, they did not find significant differences between collinear and randomly oriented displays during the earliest timeframe of the evoked response localized to the primary visual cortex. In apparent contrast, Khoe and colleagues (2004), observed early differences in ERP measures using a collinear facilitation paradigm, but as we will argue below, their stimuli may have tapped a different neural mechanism.

Role of Extrastriate Areas in Contour Completion

As mentioned in the introduction, ERP research on illusory contour processing reveals initial response modulations during the N1 timeframe, modulations that are localized to the LOC (Foxe et al., 2005; Halgren et al., 2003; Knebel and Murray, 2012; Kruggel et al., 2001; Mendola et al., 1999; Murray et al., 2004; Murray et al., 2006; Murray et al., 2002; Ritzl et al., 2003; Shpaner et al., 2009). These modulations are very similar to the effects observed in object categorization studies (e.g., Bentin et al., 1996; Itier et al., 2006; Rousselet et al., 2005; Rousselet et al., 2007; Thorpe et al., 1996) and presumably reflect initial automatic and implicit processing of objects. The results of this study highlight the role of LOC as an essential node in general pattern recognition processes. Consistent with a previous illusory contour study (Murray et al., 2002), LOC in the present dataset responded to relatively peripheral contours. The initial modulation in LOC is followed by a protracted Ncl effect, 50-100 msec later (Doniger et al., 2002; Doniger et al., 2000; Doniger et al., 2001; Sehatpour et al., 2010; Sehatpour et al., 2006; Sehatpour et al., 2008). This effect has been correlated with the identification of objects in perceptual closure studies and intracranial recordings in humans have revealed sources across a network of areas that include the LOC, the hippocampus and prefrontal regions (Table 1 and Figure 5) (Sehatpour et al., 2008). The direction of the ERP effects in the present study, together with the results of inverse solutions in this time-frame, likely reflects a similar mechanism.

Studies Challenging the Involvement of Local Horizontal Connections in Contour Completion

There are a number of studies qualifying the involvement of long-range horizontal connections in contour completion. Using a binocular rivalry paradigm, Alais et al. (2006) measured inter- and intra- hemispheric association strength between two Gabor patches. Subjects viewed two Gabor patches and two random noise patches in corresponding locations dichoptically. They were asked to respond to the presence of the Gabor patches with two buttons, each corresponding to a certain Gabor patch. Gabor patch coherence was assessed as a function of the probability of seeing two Gabor patches (alternatively, subjects might see a Gabor patch and a noise stimulus or two noise stimuli, but responses were only made to the presence of the Gabor patches). The relative timecourses of the two response streams tracking the simultaneous perception of the Gabors were correlated as a measure of coherence. V1 does not have local horizontal connections spanning across the hemispheres, and ipsilateral perception in V1 is limited by a small area of nasotemporal overlap (Fendrich and Gazzaniga, 1989; Murray et al., 2001; Sugishita et al., 1994; Sugishita et al., 1993; Wessinger et al., 1996). If local horizontal connections were at play, one would expect much lower coherence across the hemispheres than within the hemispheres. The relatively high degree of coherence between stimuli presented to different cerebral hemispheres (albeit of lower strength and smaller spatial extent than the intra-hemispheric effects) was taken as evidence for the involvement of feedback from higher-level areas during contour perception.

Samonds et al. (2006) tested how well synchronous activity in layers 2/3 of cat V1 supported the predictions of association field theory. They compared firing rates and synchronous activity in response to gratings and ring stimuli. They reasoned that if lateral horizontal connections were facilitative, gratings would induce higher firing rates than rings due to their higher degree of alignment (supporting the orientation rule of the association field theory). However, when synchronous activity was contrasted with firing rates, it was found to be a better predictor of the orientation rule; and, conversely, the firing rates were nearly the same between stimulus conditions. The authors also speculated that the observed synchrony was not due to direct excitatory connections, since there were no consistent lag times between pairs of cells tested. They concluded that such synchronous activity must be integrated in higher order visual areas.

A number of developmental considerations also argue against exclusive involvement of lateral horizontal connections in contour integration. While basic spatial acuity matures relatively early in infancy, contour detection abilities are much more protracted in humans and monkeys alike (Kiorpes and Bassin, 2003; Kovacs, 2000; Kovacs et al., 1999). At the same time, the anatomy of the lateral horizontal connections has a relatively early maturational timecourse (Burkhalter et al., 1993; Callaway, 1998; Coogan and Van Essen, 1996), while the network of intercortical projections has a much slower maturation profile (Barone et al., 1995; Batardiere et al., 2002; Burkhalter et al., 1993; Coogan and Van Essen, 1996). It appears that the ability to detect contours is more temporally consistent with the development of recurrent intercortical projections than it is with lateral horizontal connections (Kiorpes and Bassin, 2003).

Putative Function of the Long-range Horizontal Connections

What then is the functional significance of the long-range horizontal connections? Another behavioral paradigm often used to study contour integration is the so-called lateral masking paradigm (Polat and Sagi, 1993), where threshold detection of the central low-contrast Gabor patch is facilitated by flanking high-contrast Gabor patches of similar orientation and close proximity (maximal within 2-3λ separation). The assumption is that lower thresholds in these studies signify higher degrees of lateral interactions. Using the lateral interaction paradigm, Khoe and colleagues (2004) investigated neurophysiological markers of collinear facilitation. Early C1 effects were observed when collinear flankers were compared to orthogonal flankers. The topography of the effects was consistent with V1 sources, suggesting a putative role for long-range horizontal connections in this task. Based on our results, it appears that low-contrast threshold stimuli could be processed in a different manner than high-contrast suprathreshold contours (Hess et al., 1998; Hess et al., 2003). In fact, Polat and colleagues (1998) found suppression, not facilitation, of neural response in the striate cortex when high contrast stimuli were presented with two flanking stimuli in anesthetized cats. In Khoe et al. (2004), while the flankers had a relatively high contrast of 0.5 (Michelson contrast), recruiting primarily the parvocellular system, the pedestal contrasts were fixed at 0.1, putatively relying mostly on the magnocellular system. The interaction between the two systems could be quite different under such conditions than under conditions of high contrast throughout.

Angelucci and colleagues (2002) showed that horizontal connections in the primate V1 are isotropic and attributed contour completion to anisotropic feedback connections. These authors implicated local horizontal connections in short-range collinear facilitation; i.e. enhancement of the classical receptive field response to a low-contrast stimulus by co-oriented and coaxial high-contrast stimuli, a phenomenon possibly underlying upstream perceptual grouping of contour elements (Khoe et al., 2004; Khoe et al., 2006; Polat and Sagi, 1993; Polat and Sagi, 1994). Taken together with the results of this study, these reports suggest that contour integration mechanisms may depend on low-level visual features as well as on the spatial extent of contours.

Summary

In this study of pathfinder displays in humans, we assessed the timecourse of early cortical mechanisms of contour integration in order to arbitrate between two possible mechanisms of contour integration, one relying on local horizontal connections in V1 and the other relying on pooling of neuronal responses in higher-order cortical areas. Our data support the thesis that contour integration initially relies more heavily on pooling of information in higher order visual cortices (i.e., the LOC), at least under the high contrast conditions used here.