Retinotopy of the face aftereffect

Abstract

Physiological results for the size of face-specific units in inferotemporal cortex (IT) support an extraordinarily large range of possible sizes — from 2.5° to 30° or more. We use behavioral test of face-specific aftereffects to measure the face analysis regions and find a coarse retinotopy consistent with receptive fields of intermediate size (10° to 12° at 3 ° eccentricity). In the first experiment, observers were adapted to a single face at 3° from fixation. A test (a morph of the face and its anti-face) was then presented at different locations around fixation and subjects classified it as face or anti-face. The face aftereffect (FAE) was not constant at all test locations -- it dropped to half its maximum value for tests 5° from the adapting location. Simultaneous adaptation to both a face and its anti-face, placed at opposite locations across fixation, produced two separate regions of opposite aftereffects. However, with 4 stimuli, faces alternating with anti-faces equally spaced around fixation, the FAE was greatly reduced at all locations, implying a fairly coarse localization of the aftereffect. In the second experiment, observers adapted to a face and its anti-face presented either simultaneously or in alternation. Results showed that the simultaneous presentation of a face and its anti-face leads to stronger FAEs than sequential presentation, suggesting that face processing has a dynamic nature and its region of analysis is sharpened when there is more than one face in the scene. In the final experiment, a face and two anti-face flankers with different spatial offsets were presented during adaptation and the FAE was measured at the face location. Results showed that FAE at the face location was inhibited more as the distance of anti-face flankers to the face stimulus was reduced. This confirms the spatial extent of face analysis regions in a test with a fixed number of stimuli where only distance varied.

1. Introduction

The surface of human retina is approximately 1100 mm2 (Bron et al, 1997). An ordinary object, like a face viewed at 1 m, spans about 10 degrees and covers about 11 mm2 on the retina, 1% of its area. In every day vision, this relatively small image can land anywhere on the retina engaging widely diverse neural populations on the retina and early retinotopic brain areas. Proper visual function requires objects to be recognized across all these possible locations, a property called translation invariance or tolerance (for review see: Shepard & Cooper 1982; Walsh & Kulikowski 1997). Translation invariance could be supported by many independent local analyses as is the case for early visual features like orientation, color, motion, and spatial frequency (Chalupa & Werner, 2003). However, it is less plausible that the extensive computations required for object recognition could be duplicated over many locations. Nevertheless, the human visual system is able to tolerate large degrees of retinal translation -at least in priming preparations (Biederman & Cooper 1991)- and the alternative is that the large receptive fields in object-analysis areas of the brain provide the neural substrate for translational invariance.

Inferior temporal (IT) cortex is the major brain area responsible for object recognition (Tanaka, 1996; Logothetis & Sheinberg, 1996) and face recognition (Afraz et al, 2006). Although there is no exact equal for monkey IT in the human brain (see Orban 2004), human cortical areas LO, STS and FFA show selective responses to faces and express similarities with monkey IT (see Kanwisher & Yovel for review). Early electrophysiological recordings from IT cortex reported very large receptive fields —even as wide as 30 degrees- for IT neurons (Gross et al, 1972). Large receptive fields for IT cells are also reported in later studies (Desimone et al, 1984; Tovee et al, 1994; Missal et al, 1999). Moreover, the selectivity of IT neurons to highly complex stimuli such as faces is largely independent of the stimulus location within their large receptive fields (Ito et al, 1995; Logothetis et al, 1995; Schwartz et al 1983; Tovee et al, 1994). On the other hand, some electrophysiology studies have reported much smaller receptive fields for IT neurons (Op de Beeck & Vogels, 2000) even as small as 2.5 degrees in diameter (Dicarlo & Maunsell, 2003). Even for IT cells with large receptive fields, absolute firing levels may vary with retinal location (Schwartz et al 1983), a response modulation that can carry information about object location within the receptive field boundary. There is also recent fMRI results showing retinotopy in human face selective brain areas (Rajimehr et al submitted). Also, in another recent fMRI study, Hemond et al (2007) found strong contralateral preference in FFA and other face related areas. The large discrepancy between different studies possibly results from the wide range of experimental preparations and procedures they used. Overall, regarding the variation in experimental details, species and results in all these studies, the mechanisms underlying translation invariance in high level human cortical areas remain an open question.

Perceptual aftereffects have long been used to evaluate the analysis area of different types of processing. Distortion in the perceived curvature of simple geometrical shapes following presentation of a flashed line was found to transfer over large distances (Suzuki & Cavanagh, 1998). Figural aftereffects in perception of faces were reported first in 1999 by Webster and MacLin (Webster & MacLin 1999; see also Leopold, 2001; Webster et al, 2004; Blanz & Vetter, 1999). Several studies have shown that the perceptual distortions in FAE are not just the result of adaptation of low level visual areas and can only be systematically explained in a high dimensional norm-based face space (Leopold et al, 2001; Leopold et al, 2005; Rhodes & Jeffery 2006). Face aftereffects are tolerant to a wide range of rotation and, significantly for our purposes, some amount of translation (Rhodes et al, 2003; Leopold et al, 2001).

The face aftereffect depends on conscious perception of the adapting face stimulus (Moradi et al, 2005). In contrast, low level visual adaptations to orientation or motion can occur even without awareness of the stimuli (Blake & Fox, 1974; He et al 1996; He & MacLeod, 2001; Rajimehr, 2004; Rajimehr et al, 2004; Rajimehr et al 2003; Vul & MacLeod, 2006). These contrasting results suggest that the FAE is a high-level aftereffect that probably results from adaptation of face selective neurons in IT cortex (Leopold et al, 2001; Leopold et al, 2006;). This makes the FAE a good candidate to measure the translation tolerance of face representations in the brain.

Leopold et al, (Leopold et al, 2001) found FAE strength remains almost the same even if the test stimulus is presented up to 6° away from the adapting stimulus location on the retina. However they used large stimuli 11.25° (both adapting and test stimuli) and the retinal displacement of the object was always within the object boundary. This does not undermine their original claim that low level aftereffects such as orientation or spatial frequency aftereffects are unlikely to explain the observed face aftereffect but it does not offer a very strong test of translation invariance. A recent paper by Melcher (2005) reports that adaptation to a foveally presented face stimulus can be completely transferred to a location 10° in the periphery after a saccadic eye movement if that location corresponds to the same location on the screen as the adapting stimulus. This spatiotopic effect may rely on remapping processes triggered by the saccade (Colby et al, 1995; Melcher & Morrone, 2003; Burr, 2004; Melcher, 2005; Burr & Morrone, 2005). Although these studies didn’t focus on translation tolerance of FAE, they suggest a very large degree of translation invariance for FAE which supports the “non-retinotopic object representation” view.

If the FAE is invariant to translation, we expect not only an aftereffect undiminished by distance but also a single aftereffect at all locations following simultaneous adaptation to two different faces. However, a simple demonstration (Figure 1) shows that this does not occur. Instead, simultaneous presentation of two adapting stimuli leads to two aftereffects in opposite directions at the two adapted locations. The same observation is also reported for face aftereffects in the original Webster & MacLin paper (1999).

Figure 1.

Face aftereffect in opposite directions: Move your eyes up and down on the three red points on the top row for one minute. Then look down and fixate on the red dot on the bottom row. Do the adjacent test faces look the same? Or does one briefly look more like Bush and the other like Clinton? Most observers report that the test faces look different. Existence of two different aftereffects challenges the concept of a translation invariant aftereffect.

Is it possible for the face aftereffect to be translation invariant as some articles suggest (Leopold et al, 2001; Melcher, 2005) even though adapting to two faces produces two different, local aftereffects? One possibility is that the region of analysis for face identity is dynamic and although it may be very large when only one face is presented (as in Leopold et al, 2001; Melcher, 2005) it may shrink to more local regions if more than one face is present. The goal of this study is to evaluate the spatial extent of the face aftereffect and how this varies with the number of adapting faces present in the display.

The face aftereffect can be measured by the shift in the psychometric function in discrimination of test stimuli with different levels of morphing between the face stimulus and its corresponding anti-face (see methods; also see Leopold et al, 2001). We evaluate the FAE at various spatial offsets between adapt and test. The area around the adapting stimulus location in which a significant FAE is observed will be referred to as “aftereffect zone”. A wide aftereffect zone will be interpreted as support for a large region of translation invariance for cortical neurons responsible for FAE.

2. Experiment 1

2.1. Introduction

In the first experiment, we map the aftereffect zone following adaptation to a single face and following simultaneous adaptation to a face and its anti-face at different spatial separations. The translation invariance predicts a uniform global aftereffect zone following adaptation to a single face and “no aftereffect” following simultaneous adaptation to a face and its anti-face at any spatial separation (assuming that the face and its anti-face are appropriately matched). Any retinotopy in facial analysis predicts a local aftereffect centered on the adapting stimulus location. In this case, aftereffects in the opposite directions should be observed near face and anti-face locations following simultaneous adaptation to the face and anti-face. If the retinotopy is coarse, the FAE will cancel by spacing denser than the size of the face analysis region.

2.2. Methods

2.2.1. Psychophysics

Subjects were trained to identify two individual color faces (a face and its anti-face). Experimental sessions started after subjects reached 85% performance level on the face identification task. Please note the initial training task included the whole range of morphing values to familiarize subjects with the main task. Subjects were given feedback for their correct and incorrect key presses at this stage. They could never reach 100% performance because there were very difficult (near average face) stimuli in the set as well as faces with strong identity strength (far from the average).

8 subjects including one of the authors participated in Experiment 1. 216 adaptation trials were collected from each of the subjects. Trials from the different conditions of the experiment were randomly ordered in each experiment. Experiments were conducted in a dim lit room with the subject’s head resting on a chin and forehead rest 57cm away from the screen. Stimulus presentation procedures were controlled by a PC processor using MATLAB psychtoolbox (version 2.54) and displayed on a 60Hz 17inch monitor. Face stimuli used for the test phase in all experiments were spanning 9 levels of morphing (including the average face and four levels in each direction) from 20% face to 20% anti-face (-20%) identity along one identity axis from Max Plank face set ([Blanz & Vetter, 1999]) . Adaptor stimuli were the anti-face (-50%) and the 50% face of the same axis. We used “50% face” stimulus as the face adaptor to make its adaptation strength comparable with the most extreme available anti-face which was at -50% morphing level.

Each trial started with appearance of a small red fixation point in the middle of the screen. After 1 second the adapting stimulus/stimuli was/were displayed for 5 seconds (see figure 2). The size of face stimuli was ∼2° of visual angle in diameter and they were presented at 3° of visual angle eccentricity from the fixation point. We used a fixed eccentricity for the adapting and test faces to avoid the dramatic decline in recognition that occurs further in the periphery. Pilot experiments suggested that presentations at 3° supported reasonably good face recognition and also gave enough space (a span of 6°) to test various inter-stimulus distances and probe aftereffect translation. To avoid local adaptation, adapting stimuli were slightly moved around their presentation location during the adaptation. This was a back and forth smooth motion on a very short path around the display circle spanning a 11.25° sector of the imaginary circle on which the stimuli were presented (the display circle). This span on the display circle is equal to 0.6 visual degree (1.2° for Experiment 3). The movement speed was 0.6 visual angle/sec (1.2 visual angle/sec for Experiment 3). The midpoint of this 11.25° sector was counted as the adaptation location for the corresponding stimulus. This location was randomly selected before each trial and the adapting stimulus could appear at any location on the display circle. Instead of face stimuli, blank oval stimulus/stimuli of the same size, color, number and spatial arrangement as adapting stimuli for each experiment was/were presented in the non-adapted trials of adaptation phase. The color of the blank surface used in the non-adapted condition was chosen from a point on the averaged face that makes the whole blank surface have same photometrically measured luminance as the averaged face stimulus (31.8 cd/m²). The size of the oval blank stimulus was equal to the area of the averaged face. Following the adaptation phase, and a delay of 100ms the test stimulus was presented for 500ms. The fixation point was on during the whole trial. The test stimulus was randomly selected from 9 different morphing levels. The location of the test stimulus on the display circle was randomly selected for each trial. The location of the test stimulus “relative” to the adapting stimulus was saved after each trial and used for further analysis. Please note that all distances and locations reported in the results are relative to the adapting stimulus location. Subjects had to make choices to discriminate the face and anti-face by pressing one of the two keyboard buttons.

Figure 2.

Experiment one.

a) An adaptation trial. Stimuli were presented at 3° eccentricity. Adapting stimuli could be 1) a single face; 2) one face and its related anti-face (this condition is shown here); 3) two faces and two anti-faces evenly spaced; 4) oval blank surfaces with the average size and color of face stimuli. On each trial, adapting stimuli were moved slowly back and forth around their initial presentation point during five seconds of adaptation to avoid local afterimages. Following a 100ms delay, a 500ms test stimulus with various possible morphing values was presented at a random location around the display circle. Subjects discriminate it as being face or anti-face in a 2AFC task.

b) and c) Sample psychometric functions from the adaptation condition shown in “a”. The abscissa shows different morphing values of the test stimulus. Zero represents the averaged face and positive and negative values correspond to face and anti-face directions respectively. The ordinate indicates the proportion of face choices. Red and blue colors correspond to adapted and non-adapted conditions, respectively. Plot “b” shows the results obtained from the “face” location and “c” illustrates results obtained from the anti-face location. As seen here “b” and “c” show significant (Logistic regression, p<0.05) shifts in the PSE (shown by green arrows) in opposite directions for different parts of the visual field corresponding to the adapting face and anti-face locations respectively.

2.2.2. Data analysis

To provide aftereffect maps of Experiment 1, data from all non-adapted trials from all around the display circle were pooled to make a baseline psychometric curve. Then to provide the aftereffect strength value at each point on the aftereffect maps (see figure 3), adapted trials were pooled from a ±15° sector on the display circle on the sides of the adaptation location. Please note this is 15 degrees of the display circle (1/24 of the circumference) where ±15° on the display circle covers ∼1.5 ° of visual angle. This determines the resolution of our mapping. For example if we had presented our test stimuli on 6 fixed evenly spaced points around the fixation, our resolution would be 360/6=60° of the display circle equal to ∼3° of visual field. However, instead of using a fixed number of test points, we presented test stimuli at random locations and then pooled the data across sectors as small as possible that still yielded enough data points within each sector to perform a reasonable curve fitting (goodness of fit more than 0.9). In other words, we smoothed the data with a sliding window of a given resolution. We found that the resolution (∼1.5 ° of visual angle) provided by this procedure was enough for the purpose of our experiment. Data points obtained from the smoothing window (the ±15° sector of the display circle) for each adaptation location were fitted and compared with the non-adapted baseline curve to calculate the PSE shift value at that location on the display circle.

Figure 3.

FAE maps. Each point on the map shows the FAE strength at a location on the display circle relative to the adapting stimulus/stimuli location/s. The amount of FAE strength (shift of the psychometric function) is indicated with the color with red and blue shades corresponding to rightward and leftward PSE shift. Short green sectors on the inner side of the map circle indicate significance of the shift at eat location on the map (logistic regression, p<0.05).

Top, middle and bottom maps correspond to the three adaptation conditions and the locations of the adapting stimuli are shown near each map.

To calculate the amount of shift, data were fit using a logistic function formula:

$$P\left(x\right)=\frac{1}{1+e^{-(\alpha +\beta x+\lambda I)}}$$

Where x is the morphing percent and P(x) is the probability of face response. I is a binary variable, set to either 1 or 0 to indicate the presence or absence of adaptation (with out adaptation, I=0; with adaptation I=1). α, β and λ are free parameters that were fit using the maximum likelihood fitting procedure (Meeker & Escobar, 1995). Based on the above formula λ/β was used to determine the shift of the morph value at the PSE that is caused by adaptation.

Logistic regression analysis (using above formula) determined the significance of PSE shifts at p < 0.05 unless mentioned otherwise. The 0.05 alpha level for significance test used in figure 3 (Experiment 1), aftereffect maps is not corrected for the effect of multiple comparisons.

2.3. Results

Participants had to discriminate a test stimulus presented at a random location around the fixation point as being face or anti-face. Test stimuli were randomly selected from 9 morphing levels between the face and its anti-face. Before test presentation, subjects were adapted to one of the following stimuli for 5 seconds: 1) A single face; 2) a single face and its anti-face on opposite sides of the fixation (see Figure 2a); 3) two faces and two anti-faces evenly spaced and alternating; and 4) one to four ellipses evenly spaced with the average size and color of face stimuli (non-adapt baseline). To calculate the strength of FAE at each given location on the display circle, data from a sector of ±15° around that given location were pooled (see Methods for more detail) and a psychometric function was plotted as the proportion of face responses against the degree of stimulus morphing (see Figures 2b and c). The psychometric functions for various conditions were fitted with a logistic curve and the FAE strength was measured as the shift in the 50% criterion value (PSE) for every condition relative to the non-adapted condition (see Methods).

The strength of FAE (shift in the PSE relative to the non-adapted condition) was measured for every location on the display circle for the three adapted conditions of the experiment (one, two and four adapting stimuli) to provide an aftereffect map for each condition. Figure 3 shows these aftereffect maps for the three adapted conditions. The color of each point on the circular map corresponds to the FAE strength at each location on the display circle relative to the adapting stimulus/stimuli location/s. Darker colors show smaller shifts in PSE and black indicates no effect. Yellow and blue shades correspond to rightward and leftward shift in the psychometric function respectively. Green marks on the internal side of each sector on the map, indicate significant shift in the PSE at p < 0.05 (logistic regression analysis).

The aftereffect map obtained from adaptation to a single face (top row in figure 3) reveals a wide aftereffect zone with large translation tolerance and a significant effect observed even on the other side of the display circle opposite to the adapting stimulus (6° of visual angle away from the adaptation stimulus). However, the amount of the aftereffect is substantially reduced at that distance. The PSE shift at the location of the adapting stimulus is 17.5% (this numbers means that adaptation to a face at this location leads to a shift in the apparent identity of a subsequently presented averaged face equal to 17.5% of the distance between the face and anti-face stimulus). The aftereffect strength drops to 1/3 of this value (5.9%) on the opposite side of the display circle. Logistic regression shows a significant difference between these two values (Wald=3.9, Exp(B)=3.2 and p<0.05). Also, there is a highly significant negative correlation between the aftereffect strength and distance from the adaptor stimulus (r=-0.79, N=360, p<0.001, regression line intercept=19.03 and regression line slope=-1.89) (also see Discussion). This indicates a crude retinotopy which will be discussed later (see Discussion). Simultaneous presentation of the face and the anti-face leads to independent and significant aftereffects with opposite direction on the opposite sides of the display circle (the middle map in figure 3). The amount of aftereffect in face location is 9% which is non-significantly (logistic regression, Wald=2.2, Exp(B)=2.28 and p=0.14) smaller than single face aftereffect strength at the same location. This value is -11.5% at the anti-face location. As with the two-faced demonstration in Figure 1, independent aftereffects are also seen here and not the global cancellation of the aftereffect that would be predicted by global translation invariance.

Adaptation to two evenly spaced face/anti-face pairs (bottom map in figure 3) results in a substantial decrease in the aftereffect strength (to 2.8% at face locations and 4.1% at anti-face locations). Almost no significant effect was observed at any location in this condition. Contrasting data averaged from the two face locations in this condition with data from the face location in the single face adaptation condition reveals a significant difference (logistic regression, Wald=7.25, Exp(B)=4.2 and p<0.01). This condition clearly shows that the independent local aftereffects observed in condition two (middle map) can be cancelled by denser spacing of faces and anti-faces. The spacing that produces this cancellation is a reflection of the spatial resolution of neural structures responsible for the FAE.

The blank control stimulus was chosen to have same luminance, color and size as the adapting face stimuli. However, other factors (like spatial frequency content, color shadings and local orientation energy) differ between the blank oval control and the adapting face and these factors could affect the baseline (non-adapted) values at different offsets between the control and the test. To avoid this problem, we pooled all non-adapted trials to use as a baseline value for all spacings. We also tested whether there was any effect of the spacing between the control oval and the test face on PSE values. We binned non-adapted trials into six groups based on the distance of the test stimulus from the blank surface and found no significant effect of distance from the control to the adaptor (logistic regression, Wald= 0.1, Exp(B)=0.97 and p=0.69).

As seen in the aftereffect map of a single adapting face (fig. 3. top map) the amount of aftereffect decreases as a function of distance from the adapting stimulus. We also tested whether the observed aftereffect map following adaptation to both the face and anti-face (fig. 3 middle map) is the linear sum of the face (fig. 3, top) and anti-face aftereffect maps, measured in isolation. To estimate the adaptation map of a single adapting anti-face we inverted the aftereffect map of the single adapting face and shifted it 180 degrees, then we summed this and the original map to make the “predicted map” of adaptation to two stimuli based on a simple linear sum. There was a high significant correlation between the linear sum and the combined face and anti-face adaptation (r=0.86, N=360, p<0.0001). However, the slope of the regression line, 0.58, is significantly lower than 1.0 (0.58 < 1, t(358)=23.1, p<0.0001) that would be required for true additivity. The face and anti-face adaptation appear to interact in a way that reduces the effect of each. Part of the deviation from additivity might be the result of our assumption that adaptation strength is equal (and opposite) for the face and the anti-face. However, the data do show relatively equal strengths of FAE to face and anti-face — in their corresponding loci (9% and -11.5% respectively). To investigate this further, we took the absolute adaptation values for the face and anti-face, averaged them and plotted them in a 4.2 visual degree span from the adapting stimulus location to the point with equal distance from the two stimuli (90 and 270 degrees on the display circle, minimum adaptation) for both observed and predicted linear sum data (figure 4). The plot shows a wider tuning of the adaptation function for the linear sum data (half width at half maximum of ∼2.4°) than for the observed data (∼1.9°). These informal analyses suggest a narrowing of the “aftereffect zone” when both face and anti-face are presented simultaneously compared to when they are presented alone. To investigate this possibility in a direct way, a second experiment was designed.

Figure 4.

The comparison between the observed and expected adaptation value for two adapting faces. The solid line shows the absolute adaptation value after simultaneous adaptation to the face and the anti-face as a function of distance from the adaptor. The dashed line shows the expected data for this comparison based on data obtained from adaptation with a single face (see text for more details).

3. Experiment 2

3.1. Introduction

One of the problems with translation invariant object representations is the possibility of “object clutter” in natural scenes when there is more than one object in the visual field (Zoccolan et al, 2005). One way to solve this problem is to dynamically resize the grain of representation when there is more than one object in the scene. Two studies report that simultaneous presentation of more than one object in the receptive field of an IT neuron leads to shrinkage of the cell’s receptive field (Moran & Desimone, 1985; Chelazzi et al, 1998;). This effect has been called “biased competition” (Desimone, 1998) (also see discussion section). Based on these findings, we might expect that the “aftereffect zone” induced by adaptation to a single face shrinks when there is more than one face present in the adaptation phase. The informal analysis mentioned in the results of Experiment 1 suggests this dynamical narrowing of the “aftereffect zone” but we designed Experiment 2 to address this question in a more direct way.

The second experiment investigates the dynamic properties of the “aftereffect zone” by comparing simultaneous adaptation to a face and its anti-face versus sequential adaptation. If face analysis zones become smaller when two faces are present simultaneously, then the simultaneous condition should produce stronger local aftereffects than the consecutive presentation (where larger analysis areas and therefore larger aftereffect zones will lead to greater cancellation).

3.2. Methods

2.2.1. Psychophysics

Face aftereffect strength was compared between simultaneous versus consecutive adaptation. The aftereffect was measured only at the adapting “face” location in three conditions: 1) Simultaneous condition: the adapting face and anti-face were presented for 10 seconds intermittently at 0.5Hz (1s on, 1s off). However, in this condition the stimuli were turned on and off simultaneously (both on or both off) ; 2) Consecutive condition: the face and anti-face were presented each for 10 seconds intermittently but non-simultaneously (face on/anti-face off face off/anti-face on); 3) the baseline condition with intermittent presentation of blank oval stimuli (see figure 5.a left panel for simultaneous and right panel for non-simultaneous conditions). In both simultaneous and consecutive conditions, each stimulus was “on” for the total time of 5 seconds (half of the 10 seconds of intermittent presentation). In the consecutive condition, the adaptation ended with the anti-face presentation on half of the trials and with the face presentation on the other half.

Figure 5.

Simultaneous vs. consecutive adaptation. a) Simultaneous and consecutive adaptation paradigms are shown on the left and right sides respectively. Each stimulus was presented for 5 total seconds at 0.5Hz alternation. b) Proportion of face responses plotted against morphing value in a typical subject (right plot shows this for data averaged over all subjects). Blue, red and green colors correspond to non-adapted, consecutive and simultaneous adapted conditions respectively. The rightward shift of the psychometric function corresponds to FAE strength and is strongest for the simultaneous adaptation condition.

Four subjects — including one of the authors — participated in Experiment 2. Each subject completed 324 adaptation trials for Experiment 2. Every other parameter and procedural detail was identical to Experiment 1.

2.2.2. Data analysis

Data obtained from the three conditions (simultaneous, consecutive and non-adapted conditions) were fit with a logistic function and the shift from the non-adapted condition was measured based on the following formula:

$$P\left(x\right)=\frac{1}{1+e^{-(\alpha +\beta x+{\lambda }_1{I}_1+{\lambda }_2{I}_2)}}$$

Where x is the morphing percent and P(x) is the probability of a face response. I₁ and I₂ are binary variables that indicate simultaneous and consecutive conditions respectively. In other words, I₁=1 and I₂=0 indicate simultaneous condition, I₁=0 and I₂=1 indicate consecutive condition and when both I₁ and I₂ are equal to zero, that indicates the non-adapted condition in the fitting function. All other statistical details are same as Experiment 1.

3.3. Results

Figure 5.b shows the results of Experiment 2 for a typical subject and the averaged data of all subjects. Results from other 3 subjects are provided in the supplementary material (supplementary figure 1.) and show the same pattern. The results show a significant rightward shift in the psychometric function for both the consecutive (6.1% shift for the typical subject of figure 5, 6.5% for averaged data) and simultaneous (11.9% shift for the typical subject of figure 5, 13.4% for averaged data) conditions. The shift was significantly larger for the simultaneous condition (logistic regression, Wald=23.5, Exp(B)=3.76 and p<0.01 for the averaged data, also p < 0.01 all subjects). For consecutive condition, there was no significant difference between results of those trials that ended with the anti-face and other trials that ended with the face stimulus presentation (logistic regression, Wald=1.07, Exp(B)=1.29 and p=0.3 for the averaged data, also p > 0.1 all subjects) and both had a significantly smaller shift than the simultaneous condition (logistic regression, p < 0.05 all subjects both conditions). This indicates that the last viewed adapting stimulus did not determine the aftereffect size.

4. Experiment 3

4.1. Introduction

The final experiment measures the pure effect of inter-stimulus distance on the FAE. As mentioned above, retinotopic model of FAE predicts decrease of FAE strength as face and anti-face space more closely. In the first experiment, spacing and number of stimuli varied together so one cause of the reduced FAE for closer spacing may be the decrease in attention to each stimulus (because there are more of them). In this experiment, the number of stimuli is kept constant at three: one face and two anti-faces with spatial offsets are presented in the adaptation phase and the FAE is always measured at the adapting face location.

4.2. Methods

2.2.1. Psychophysics

To measure the pure effect of inter-stimulus spacing on FAE, we present three stimuli in all conditions: one face and two anti-faces. All stimuli were located around the display circle, this time with 6.2° radius and slightly larger faces (∼2.35° in diameter). During adaptation, anti-faces were located symmetrically on the two sides of the face stimulus with three possible face/anti-face distances: 2°, 4.6° or 5.9°. There was also a baseline adaptation condition with non-face ovals. Test stimuli with various morphing levels were always presented at the adapting face location. All other parameters and procedural effects were the same as Experiment 1.

Four subjects — including one of the authors — participated in this experiment. Each subject completed 324 adaptation trials.

2.2.2. Data analysis

Data obtained from the four conditions (the three face/anti-face distance conditions and the non-adapted conditions) were fit with a logistic function and the shift from the non-adapted condition was measured based on the following formula:

$$P\left(x\right)=\frac{1}{1+e^{-(\alpha +\beta x+{\lambda }_1{I}_1+{\lambda }_2{I}_2+{\lambda }_3{I}_3)}}$$

Where x is the morphing percent and P(x) is the probability of face response. I₁ , I₂ and I₃ are binary variables that indicate the three distance conditions respectively (see methods section of Experiments 1 and 2). Just like other experiments, to provide the baseline PSE of non-adapted condition I₁ , I₂ and I₃ are all set to zero for non-adapted trials in the fitting function. All other statistical details are same as previous experiments.

4.3. Results

Figure 6 shows the results for a typical subject (results from other subjects are similar and provided in supplementary figure 2.). The inter-stimulus distance had a significant effect on the aftereffect (logistic regression, Wald=117.6, Exp(B)=0.5 and p<0.001 for the averaged data, also p < 0.01 for all subjects) reducing it as the anti-faces got closer to the face stimulus (see Figure 6).

Figure 6.

Effect of inter-stimulus distance during adaptation on the FAE. FAE is measured at the adapting face location with anti-face flankers at different separations from the face in the adaptation phase. Blue shows the non-adapted baseline condition and green, brown and red indicate the three spatial separations. The aftereffect at the face location (rightward PSE shift) is much smaller when the flankers are close to the adapting face. Left and right plots show the results for a typical subject and averaged data for all subjects respectively.

5. Discussion

5.1. Experiment 1

5.1.1. Size of face translation area

Adaptation to a single face in Experiment 1 demonstrated a broad spatial extent for the face aftereffect, dropping below half its maximum by 6° from the adapting location (for our stimuli and tested locations). To estimate the spatial limits of FAE translation, we plotted the averaged FAE strength (PSE shift) as a function of linear distance from the single face adaptor in Experiment 1 (condition one) averaged across all subjects and adapting locations. Figure 7 depicts this function; mirroring the data to the left and right of the adapting location and fitting the data with a Gaussian function.

Figure 7.

Estimation of spatial extent of FAE. PSE shift values, after adaptation to a single face in experiment one are plotted as a function of distance from the adapting stimulus location. Data are mirrored on left and right and fitted with a Gaussian curve. The abscissa shows the distance from the adapting stimulus and the ordinate indicates percent of shift in the psychometric function after adaptation. Half width at half height of the fitted function is 5.4°.

The Gaussian fitted function of figure 7 estimates that the FAE induced at 3° eccentricity covers a spatial extent of about 10.8° diameter (full width at half height). This number indicates a crude retinotopy with spatial resolution of about 10° at this eccentricity for brain structures underlying face representation. As mentioned in the introduction, electrophysiological estimation of the diameter of IT cells’ receptive fields have resulted in a very wide range of reports from 30° (Gross et al 1972) to 2.5° (Dicarlo & Maunsell 2003). Our results indicate that -at least at the behavioral level- the analysis area for faces does not extent over the whole visual field and is limited to about 10-11° at 3° eccentricity. This finding argues against translation invariant theories of object representation and suggests separate and relatively independent representations for faces across the visual field. Although we are not directly measuring the size of IT receptive fields here, this behavioral measurement is consistent with the median size of IT receptive fields in most of reports (Gross et al, 1972; Desimone et al, 1984; Tovee et al, 1994; Missal et al, 1999; Ito et al 1995; Logothetis et al 1995; Schwartz et al 1983; Op de Beeck & Vogels 2000). In a recent study, Rousselet et al (2002 & 2004) have shown that humans can make an animal-nonanimal decision just as fast for two as for one stimulus. Their behavioral and electrophysiology results suggest independent processing of the two stimuli in the ventral stream. Notably, the distance between the two bilaterally presented stimuli (in the Rousselet et al, 2002 study) was 7.2° (bilateral presentation at 3.6° eccentricity). Another set of human electrophysiology studies (Jacques & Rossion, 2004 & 2006) showed strong attenuation of face selective ERP responses when two faces are presented simultaneously, suggesting common resources for representation of the two stimuli in the ventral stream. Interestingly, the distance between the two stimuli was smaller in these studies (3.1°) (also one of the two stimuli was presented foveally which makes a direct comparison difficult). These findings are consistent with our observation that FAE of the face and anti-face are independent for spacings beyond 6 degrees (for eccentricities of around 3 degress) but interact at smaller spacings.

The face aftereffect probably results from adaptation at multiple levels of processing in the visual system. The highly significant negative correlation of the FAE strength with test-adaptor distance (see Results of exp. 1) clearly indicates the retinotopy of this aftereffect. Nevertheless, it is still possible to imagine a high level, non-retinotopic uniform component of FAE that is added to the retinotopic component arisings from lower brain areas. Data from our Figure 3 argue against this however. Specifically, the top map of Figure 3 (please note green significance marker on the inner side of the circle) shows several test locations with non-significant FAEs on the opposite side from the adapting face location. If there is a non-retinotopic FAE, its strength lies below the significance level of our experiment.

One could also claim that following adaptation to two opposite stimuli, face and anti-face, any high-level non-retinotopic aftereffects would cancel leaving only local, low-level components of adaptation that would support opposing aftereffects. However, the opposing face and anti-face adaptation start to cancel at much larger spacings than the typical spread seen for low level aftereffects such as those of color, contrast or spatial frequency (Williams et al, 1982; Ejima & Takahashi, 1984; Ejima & Takahashi, 1985).

Overall, our data suggest some degree of retinotopy in the analyses underlying the face aftereffect. The levels of processing responsible for the face aftereffect may certainly include face specific analyses but may also include mid-level shape processing areas.

5.1.2. Hemifield effects

The stimulus location for adapting and test stimuli was selected randomly, so in some cases the adapt and test stimuli were presented to different visual hemifields, but in other cases both stimuli were presented within one hemifield. It is possible to imagine less transfer of the FAE when the adapt and test locations are in separate hemifields as neurons in many object-processing cortical areas respond mostly to the contralateral field and little to the ipsilateral field (see DiCarlo & Maunsell, 2003; Hemond et al, 2007). For example, Kovacs et al (2005) have shown that the FAE is smaller for test stimuli presented to the opposite hemifield from adaptation compared to that for test stimuli in the same hemifield as the adaptation. However, they could not determine whether the loss in FAE was due solely to the separate test and adapt hemifields or whether there was also an effect of the distance between adapting and test stimuli.

In our study, we pooled the data for bilateral and unilateral presentations, and larger adapt-test spacings would naturally include more trials where the adapt and test stimuli fell in different hemifields. If the FAE does not transfer across the vertical meridian, this effect would appear as an effect of distance in our results. To investigate possible hemifield effects and also to measure the within-hemifield effect of adapt-test spacing, we reanalyzed the results of Experiment 1 separately for the unilateral, within-hemifield trials and the bilateral, across-hemifield trials. Results from both bilateral and unilateral trials demonstrated significant negative correlation between FAE strength and adapt-test spacing (see figure 8, r=-0.58, N=338 and r=-0.56, N=336 for unilateral and bilateral conditions respectively. p<0.001 for both). The regressions had similar slopes and intercepts in both cases (intercept=18.48 and 18.16 and slope= -1.63 and -1.75 for unilateral and bilateral conditions respectively.). For very small and very big (near 180 degree on the display circle) inter-stimulus separations, there are very few bilateral and unilateral trials, respectively, and these sparse and noisy extremes (46 out of 720 data points) were excluded from the correlation analysis. These results indicate that FAE strength depends on the distance from the adapting stimulus to the same degree whether or not the test and adapt stimuli were in the same hemifield. We find no evidence of a hemifield effect here.

Figure 8.

Comparison of the effect of distance on FAE strength in unilateral and bilateral presentations. In both case, increasing the distance between the adapt and test stimuli leads to a similar decrease in FAE.

5.1.3. Eye fixation

Eye fixation was not monitored during the experiments but all subjects were experienced psychophysical observers and had strict instructions to maintain their fixation during the trial. Any tendency to leave the fixation and foveate the stimulus (either adapting or test) would reduce retinotopic effects as the eye movements will reduce the effective distance between adapt and test, making the aftereffect appear to extend across large distances with no loss. In contrast, our data show a significant drop off in FAE with distance.

5.1.4. Possible interaction with size

It has been shown that face-distortion aftereffects do transfer to test stimuli that have a different size than the adapting stimuli (Zhao & Chubb, 2001; Yamashita et al, 2005; Anderson & Wilson, 2005). However, the transfer does drop off as the relative size difference increases, showing a crude retinotopy in terms of size that is similar to the crude retinotopy we find here for distance. However, the effect of size and distance may be interdependent. For example, Leopold et al (2001) found no loss in FAE with a 6° retinal displacement between adapt and test. In contrast, in our Experiment 1, there was little or no FAE with a 6° adapt-test shift. The major difference between the two studies is the size of stimuli. The size of the faces used in Leopold et al’s study was about 6 times larger than ours (11.25° compared to 2°) so that the 6° retinal displacement remained within the boundaries of the adapting face. It will be important to address the interaction of size and distance effects and also the effect of retinal displacements within and outside object boundaries in future studies.

5.2. Experiment 2

5.2.1. Sharpening of receptive fields

Experiment 2 provides evidence in support of a sharpening of the face analysis region when there are two faces in the scene simultaneously. The results show that the adaptation is stronger at the adapting face location when the anti-face is presented simultaneous with the face. This result indicates that the anti-face aftereffect spreads more widely when there is no competing stimulus present at the time, and therefore cancels the face aftereffect at the face location more effectively. Electrophysiology studies have not shown sharpening of IT receptive fields in the presence of more than one object in the scene (Sato, 1989; Miller et al, 1993; Rolls & Tovee, 1995; Sheinberg & Logothetis, 2001; Rolls et al, 2003; Zoccolan et al, 2005;). However, the results of our Experiment 2 can be explained in the context of “biased competition” model (Desimone, 1998) of interactions among neurons representing visual objects: Attention to a stimulus within the receptive field of extrastriate and IT neurons leads to shrinkage of the receptive field around the attended object (Moran & Desimone, 1985; Chelazzi et al, 1998;). Although there was no explicit instruction about attention in our experiments, it is possible that simultaneous presentation of face and anti-face have triggered competitive mechanisms to select and individuate each stimulus which might lead to partial shrinkage of receptive fields and less interaction between the stimuli in the simultaneous presentation condition. The results of Experiment 1 (and Experiment 3) show that this sharpening has its limits — with two or more faces present, the spatial extent of each face analysis region may be smaller than when only one is present at a time, but it is still substantial.

5.2.2. Shape contrast effect

Although results of Experiment 2 are consistent with “receptive field sharpening” when two faces are present simultaneously, that is not the only possible explanation for these results. An alternative model could be based on a shape contrast effect (see Robbins et al, 2007). Contrasting shapes of the face and the anti-face stimulus in the simultaneous presentation condition might increase their apparent identity strength and consequently increase their power as adapting stimulus. Both face and anti-face aftereffects would increase but the aftereffect is measured only at the adapting location for the face stimulus. The anti-face aftereffect at that location, at some from the anti-face’s own adapting site, must be less than its maximum and so any increase due to shape contrast would be proportionally smaller as well, leaving a net increase in the FAE at the face location. In other words, our result could be attributed to shrinking or shifting of selectivity in face space as well as in retinal space. Further experiments would be required to determine whether multiple stimuli lead to changes in face selectivity (shape contrast) or spatial selectivity (biased competition).

5.3. Experiment 3

Experiment 1 showed that the FAE at the adapting location was less when two faces were present during adaptation (a face and an anti-face, separated by 180 when only a single was present for adaptation. The FAE was lost completely when four adapting faces were presented (two face and two anti-face stimuli). Could this drop off be accounted for by the reduction in attention available for each adapting face, as their number increased? Experiment 3 was designed to measure the pure effect of adapt-test distance on FAE when the number of stimuli was kept constant and only the inter-stimulus distance varied.

Results showed large rightward shift of the psychometric function (a significant FAE to the face stimulus) when anti-face flankers were at 5.9° (of visual field) distance from the adapting face. The FAE decreased drastically with closer spacing of stimuli, showing the pure effect of flanker distance on the aftereffect. This rules out possible effect of dividing attention over a larger number of stimuli in Experiment 1 and again demonstrates the cancellation of face and anti-face aftereffects over distances less than 6°. At the closest interstimulus spacing in our data set, 2° of visual angle separation between the face and anti-face flankers, the aftereffect at the face stimulus location was strongly attenuated but still significant. This residual effect may reflect a small, local component of the FAE mediated by lower level brain areas.

5.4. Conclusion

Overall, the three experiments provide clear evidence for retinotopy in face aftereffects with an analysis region of about 10 to 12 degrees width for stimulus at 3 degrees eccentricity. Clearly, the strategy for face recognition -- and by extension, object recognition — is not to use very large receptive fields to reduce the required number of highly specialized units. The intermediate size suggested by our results implies that recognition for any given stimulus will have to be learned independently at several locations in order to achieve full translation invariance. Indeed, some behavioral results do indicate that recognition of complex stimuli when initially trained at one location does not transfer over large distances (Nazir & O’Regan, 1990; Dill & Fahle, 1997; Dill & Fahle, 1998; Dill & Edelman, 2001). Nevertheless, the size we find will limit the amount of position-specific learning required and also offer the capability to analyze multiple objects if they are spaced sufficiently far apart to fall in separate receptive fields. This compromise may offer the optimal strategy for object recognition with its trade-off of position-specific learning and localization of recognition.