Skip to main content
PMC home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2011 Aug 3;31(31):11305–11312. doi: 10.1523/JNEUROSCI.1935-11.2011

Mirror-Image Sensitivity and Invariance in Object and Scene Processing Pathways

Daniel D Dilks 1,✉, Joshua B Julian 1, Jonas Kubilius 2, Elizabeth S Spelke 3, Nancy Kanwisher 1
PMCID: PMC3160766  NIHMSID: NIHMS315873  PMID: 21813690

Abstract

Electrophysiological and behavioral studies in many species have demonstrated mirror-image confusion for objects, perhaps because many objects are vertically symmetric (e.g., a cup is the same cup when seen in left or right profile). In contrast, the navigability of a scene changes when it is mirror reversed, and behavioral studies reveal high sensitivity to this change. Thus, we predicted that representations in object-selective cortex will be unaffected by mirror reversals, whereas representations in scene-selective cortex will be sensitive to such reversals. To test this hypothesis, we ran an event-related functional magnetic resonance imaging adaptation experiment in human adults. Consistent with our prediction, we found tolerance to mirror reversals in one object-selective region, the posterior fusiform sulcus, and a strong sensitivity to these reversals in two scene-selective regions, the transverse occipital sulcus and the retrosplenial complex. However, a more posterior object-selective region, the lateral occipital sulcus, showed sensitivity to mirror reversals, suggesting that the sense information that distinguishes mirror images is represented at earlier stages in the object-processing hierarchy. Moreover, one scene-selective region (the parahippocampal place area or PPA) was tolerant to mirror reversals. This last finding challenges the hypothesis that the PPA is involved in navigation and reorientation and suggests instead that scenes, like objects, are processed by distinct pathways guiding recognition and action.

Introduction

Discriminating the image of an object from its reflection about the vertical axis is difficult for many species, including octopuses, fishes, rats, monkeys, and human children and adults (Corballis and Beale, 1976; Bornstein, 1982). Electrophysiological data support this mirror-image confusion (Logothetis et al., 1995; Rollenhagen and Olson, 2000; Freiwald and Tsao, 2010). For example, Rollenhagen and Olson (2000) found that neuronal responses from inferotemporal cortex in the macaque monkey were more similar between members of a left–right mirror-image pair of objects than between an up–down mirror-image pair. Similarly, human functional magnetic resonance imaging (fMRI) studies have demonstrated that pictures of objects and their mirror reversals are encoded as the same object within some ventral visual regions (Eger et al., 2004; Vuilleumier et al., 2005; Dehaene et al., 2010a) (but see Kim et al., 2009), and neuropsychological studies have described impairments in discriminating between mirror images of objects (Riddoch and Humphreys, 1988; Turnbull and McCarthy, 1996; Warrington and Davidoff, 2000; McCloskey, 2004). At some level of processing, therefore, the object recognition system treats mirror images as equivalent. Thus, generalization may be adaptive, because the left–right orientation of an object is generally irrelevant to the object's identity (Corballis and Beale, 1976; Bornstein, 1982; Walsh, 1996).

Unlike the case for objects, however, the navigability of a scene is completely different when mirror reversed. Indeed, behavioral evidence shows clearly that the “sense” of a scene (which distinguishes the spatial layout of a scene from its reflection about the vertical axis) is discriminated and used in navigation by pigeons, rats, human infants, and adults (for review, see Cheng and Newcombe, 2005; Spelke et al., 2010). Thus, we predicted that representations in object-selective cortex will be tolerant (i.e., at least partially “invariant”) to mirror reversals, but representations in the scene-selective cortex will not.

To test our predictions, we used an event-related fMRI adaptation paradigm (Grill-Spector and Malach, 2001). Participants viewed trials consisting of two successively presented images, either both objects or both scenes. Each pair of images consisted of one of the following: (1) the same image presented twice; (2) two completely different images; or (3) an image followed by the mirror-reversed version of the same stimulus. If object representations in object-selective cortex are tolerant to reflection, then mirror-reversed images of objects will be treated as the same image, and the neural activity in object-selective cortex will show adaptation across mirror-image changes. On the other hand, if scene representations are sensitive to sense information, then the mirror-reversed images of scenes will be treated as different images, producing no adaptation across mirror-image changes in scene-selective cortex. We examined the encoding of mirror-image information in two object-selective regions [the lateral occipital sulcus (LO) and the posterior fusiform sulcus (pFs)] and three scene-selective regions [the temporal occipital sulcus (TOS), the parahippocampal place area (PPA), and the retrosplenical complex (RSC)].

Materials and Methods

Participants.

Twenty-eight healthy individuals (ages 18–31; 17 females) were recruited for the experiment. All participants gave informed consent. All had normal or corrected-to-normal vision and were naive to the purpose of the experiment. Two participants were excluded for excessive motion, and one participant was excluded as a result of nonsignificant localizer results in all regions of interest.

Design.

We used a region of interest (ROI) approach, where we localized category-selective regions (localizer scans) and then used an independent set of data to investigate the response of these regions to pairs of objects or scenes that were identical, different, or mirror reversed (experimental scans).

For the localizer, we used a standard method described previously (Epstein and Kanwisher, 1998) to identify ROIs. Specifically, participants viewed 4 runs during which a total of 16 s blocks each (16 stimuli per block) of faces, objects, scenes, or scrambled objects were presented, interleaved with 16 s of fixation. Each image was presented for 400 ms, followed by a 600 ms interstimulus interval (ISI). Each run contained 21 such blocks totaling 5 min and 36 s. Participants performed a one-back task, responding every time the same image was presented twice in a row.

For the experimental scans, participants completed 8 runs with 136 trials (40 of these were fixation trials, used as a baseline condition) per run. On each nonfixation trial, an image of either a scene or an object was presented for 300 ms, followed by an ISI of 400 ms and then by another image of the same stimulus category presented for 300 ms [following the method of Kourtzi and Kanwisher (2001) and many subsequent papers]. After presentation of the second image, there was a jittered interval of ∼3 s (ranging from 1 to 6 s) before the next trial began. Each pair of images consisted of one of the following: (1) the same image presented twice (Same condition); (2) two completely different images (Different condition); or (3) an image followed by the mirror-reversed version of that same image (Mirror condition) (Fig. 1). Scene and object trials were intermixed within a run. Trial sequence was generated using the FreeSurfer optseq2 function, optimized for the most accurate estimations of hemodynamic response (Burock et al., 1998; Dale et al., 1999). There were 32 different images of objects and 32 images of scenes. All stimuli were grayscale and were 7° × 8° in size. Subjects were instructed to remain fixating on a white fixation dot that remained on the screen for the duration of the stimuli. Each image was presented at the central fixation and then moved either up or down. Participants performed an orthogonal task (not related to whether an image was a scene or object or whether it was mirror reversed), responding via button box whether each image in a pair was moving in the Same or Different direction. The motion task was also particularly chosen to eliminate any early retinotopic confounds.

Figure 1.

Figure 1.

Open in a new tab

Example stimuli from each of the categories by condition (i.e., Same, Mirror, Different).

fMRI scanning.

Scanning was done on a 3T Siemens Trio scanner at the Athinoula A. Martinos Imaging Center at the McGovern Institute for Brain Research at the Massachusetts Institute of Technology (Cambridge, MA). Functional images were acquired using the standard 12 channel head matrix coil and a gradient echo single-shot echo planar imaging sequence [16 slices, repetition time (TR) = 2 s for the localizer scans and TR = 1 s for the experimental scans, echo time (TE) = 30 ms, voxel size = 3.1 × 3.1 × 4.0 mm, and 0.4 mm interslice gap]. For all scans, slices were oriented approximately between perpendicular and parallel to the calcarine sulcus, covering the occipital lobe as well as the posterior portions of the temporal lobe. High-resolution anatomical images were also acquired for each participant for reconstruction of the cortical surface.

Data analysis.

fMRI data analysis was conducted using FreeSurfer Functional Analysis Stream (FS-FAST; http://surfer.nmr.mgh.harvard.edu/), and ROI analysis was conducted using FS-FAST ROI toolbox (fROI; http://froi.sourceforge.net/). Before statistical analysis, images were motion corrected (Cox and Jesmanowicz, 1999). Additionally, localizer data, but not experimental data, were spatially smoothed (6 mm kernel), detrended, and fit using a gamma function (delta = 2.25 and tau = 1.25).

After preprocessing, scene-selective regions TOS, PPA, and RSC were bilaterally defined in each participant (using data from the independent localizer scans) as the regions that responded more strongly to scenes than objects (p < 10−4), as described previously (Epstein and Kanwisher, 1998) (Fig. 2B). TOS was identified in at least one hemisphere in 23 of the 25 participants, PPA in 24 of the 25 participants, and RSC in 22 of the 25 participants (Table 1). Object-selective regions LO and pFs were bilaterally defined in each participant (using data from the localizer scans) as the regions that responded more strongly to intact objects than to scrambled objects (p < 10−4), as described previously (Grill-Spector et al., 1998) (Fig. 2A). LO and pFs were each identified in at least one hemisphere in all 25 participants (Table 1). The size and location of each ROI are also summarized in Table 1. As a control region, we also anatomically defined a bilateral foveal cortex (FC) ROI—the region of cortex responding to foveal stimulation (Dougherty et al., 2003). Specifically, the FC ROIs were drawn at the posterior end of the calcarine sulcus (the occipital pole) with a surface area ∼200 mm2 for each participant following the method used in many prior papers (Baker et al., 2005, Baker et al., 2008; Dilks et al., 2009). The FC ROIs were defined on the reconstructed cortical surface.

Figure 2.

Figure 2.

Open in a new tab

A. Object-selective cortical regions from an example participant. Using independent data, LO and pFs were localized as regions that responded more strongly to objects than scrambled objects (p < 10−4). B, Scene-selective cortical regions from an example participant. Using independent data, TOS, PPA, and RSC were localized as regions that responded more strongly to scenes than objects (p < 10−4). The mean Talairach coordinates for all ROIs are in Table 1.

Table 1.

Number of participants that showed each ROI by hemisphere, as well as the average size (number of voxels) and mean Talairach coordinates (determined from the center of gravity) for each ROI by hemisphere

ROI Number of participants
 Average size (number of voxels)
 Mean Talairach coordinates (x, y, z)
LH RH LH RH LH RH
LO 24 21 75 73 −43 −76 0 43 −73 0
pFs 23 24 64 31 −34 −44 −13 33 −45 −12
TOS 21 23 54 38 −33 −77 21 34 −74 21
PPA 20 24 30 41 −24 −43 −6 24 −43 −7
RSC 22 22 52 43 −19 −56 12 18 −55 5
Open in a new tab

LH, Left hemisphere; RH, right hemisphere.

For each ROI of each participant, the mean time courses (percentage signal change relative to a baseline fixation) for the experimental conditions were extracted across voxels. To determine the time point to use as the peak value in further analyses, the time courses for either the two object-selective regions or the three scene-selective regions across conditions and participants were averaged together, and the peak response was identified (i.e., 5 s after the trial onset for the object-selective regions and 4 s after the trial onset for the scene-selective regions). Next, for each participant, the peak responses for each object- and scene-selective region were then extracted for each condition (Different, Mirror, Same), and repeated-measures ANOVAs and paired t tests between conditions were performed. A 2 (hemisphere: left, right) × 3 (condition: Different, Mirror, Same) repeated-measures ANOVA for each ROI was conducted. We found no hemisphere × condition interaction in any of the ROIs (all F values <2, all p values >0.20). Thus, both hemispheres were collapsed for further analyses.

Results

Object-selective cortex

As predicted, we found that one object-selective region, pFs, showed tolerance to mirror-reversed images of objects. A three-level (condition: Different, Mirror, Same) repeated-measures ANOVA revealed a significant main effect of condition (F(2,48) = 13.07, p < 0.001), with a significantly greater response to the different condition than to either the Mirror or Same conditions (main effect contrasts, all p values <0.05) (Fig. 3A). To examine these effects more specifically, paired t tests revealed a significantly greater response to the Different compared to the Same conditions (t(24) = 5.93, p < 0.00), a significantly greater response to the Different compared to Mirror conditions (t(24) = 3.48, p < 0.01), but no significant difference between the Mirror and Same conditions (t(24) = 0.70, p = 0.50). These findings demonstrate not only the expected fMRI adaptation effect (i.e., different > same), but also the predicted adaptation across mirror images, revealing that pFs treats mirror-reversed images of objects as the same image.

Figure 3.

Figure 3.

Open in a new tab

Hemodynamic time courses (percentage signal change) of two object-selective regions of cortex, pFs (A) and LO (B) to (1) two completely different images of objects (red line labeled “different”), (2) the same image of an object presented twice (blue line labeled “same”), and (3) an object followed by the mirror-reversed version of the same object (green line labeled “mirror”). Note tolerance to mirror-image reversals in pFs, yet sensitivity to mirror-image reversals in LO.

However, the other object-selective region, LO, was found to be sensitive to mirror-reversed images of objects. A three-level (Condition: Different, Mirror, Same) repeated-measures ANOVA revealed a significant main effect of condition (F(2,48) = 9.31, p < 0.01), with a significantly greater response to both the Different and Mirror conditions compared to the Same condition (main effect contrasts, both p values <0.05) (Fig. 3B). Furthermore, paired t tests revealed a significantly greater response to the Different compared to Same conditions (t(24) = 4.11, p < 0.001), no significant difference between the Different compared to Mirror conditions (t(24) = 1.53, p = 0.14), but a significantly greater difference between the Mirror and Same conditions (t(24) = 2.54, p < 0.05). This finding again demonstrates the expected fMRI adaptation effect, but not the adaptation across mirror images, revealing that LO, unlike pFs, treats mirror-reversed images of objects as two different images (Table 2).

Table 2.

Summary of the results from the three-level (condition: Different, Mirror, Same) repeated measures ANOVA conducted on each ROI

ROI p value of the main effect Results of the contrasts Mirror sensitivity to preferred category?
LO 0.001 D, M > S Yes
pFs 0.0001 D > M, S No
TOS 0.006 D, M > S Yes
RSC 0.004 D, M > S Yes
PPA 0.0001 D > M, S No
Open in a new tab

For example, the ANOVA conducted on LO revealed a significant main effect of condition (F(2,48) = 9.31, p = 0.001) with a significantly greater response to both the Different (D) and Mirror (M) conditions compared to the Same (S) condition (main effect contrasts, both p values <0.05). This finding demonstrates the expected fMRI adaptation effect (D > S), but not the adaptation across mirror images (M > S), revealing that LO treats mirror-reversed images of objects as two different images. In other words, LO is sensitive to the left–right orientation of objects. Note that the “comma” in “D, M > S” indicates no significant difference between those conditions (i.e., D and M in this case).

The above analyses suggest that the two object-selective regions encode sense information of objects differently, so we directly tested this suggestion by comparing the differences in response between the two ROIs. Specifically, for each ROI the difference between the peak responses for two different images of objects and the same images was compared to the difference between the peak responses for two mirror reversals and the same images (Fig. 4). A 2 (ROI: pFs, LO) × 2 (difference score: Different-Same, Mirror-Same) repeated-measures ANOVA revealed a significant interaction (F(1,24) = 4.14, p = 0.05; partial η2 = 0.15, where η2 > 0.14 constitutes a “large” effect size when df = 1) (Cohen, 1988) with a significantly greater difference between the different and same conditions than between the Mirror and Same conditions for pFs, relative to LO. Thus, sense information of objects is represented differently by the two object-selective regions: pFs is tolerant to left–right orientation of objects, while the more posterior LO encodes such information. This finding accords with evidence that anatomical position—from posterior to anterior—is associated with decreasing sensitivity to low- and mid-level feature characteristics such as size, position, illumination, and pose (Ito et al., 1995; Grill-Spector et al., 1998; Grill-Spector et al., 1999; Lerner et al., 2001; Sawamura et al., 2005; Rust and Dicarlo, 2010).

Figure 4.

Figure 4.

Open in a new tab

For each object-selective ROI, the difference between the peak responses for two different images of objects and the same images (labeled “Different-Same”) was compared to the difference between the peak responses for two mirror reversals and the same images (labeled “Mirror-Same”). A 2 (ROI: pFs, LO) × 2 (difference score: Different-Same, Mirror-Same) repeated-measures ANOVA revealed a significant interaction (F(1,24) = 4.14, p = 0.05) with a significantly greater difference between the Different and Same conditions than between the Mirror and Same conditions for pFs, relative to LO. Thus, sense information of objects is represented differently by the two object-selective regions: pFs is tolerant to left–right orientation of objects, while the more posterior LO encodes such information.

Given that our stimuli included scenes as well as objects, we also were able to investigate how pFs and LO responded to sense information of scenes (the nonpreferred category). For pFs, we did not even find fMRI adaptation for different versus same scenes (t(24) = 1.38, p = 0.18); thus, the question of sensitivity to sense information in scenes for pFs is moot. By contrast, for LO, a three-level (condition: Different, Mirror, Same) repeated-measures ANOVA revealed a significant main effect of condition (F(2,48) = 4.49, p < 0.05), with a significantly greater response to the Different and Mirror conditions compared to the same condition (main effect contrasts, both p values <0.05). This finding demonstrates not only fMRI adaptation for Different versus Same scenes, but also reveals that LO is sensitive to sense information of scenes as well as objects. This result is not surprising given that many of the scene stimuli contained “objects” (e.g., boats, benches, umbrellas) whose positions and orientations both were reversed in the mirror image.

Might it be the case that the sensitivity to left–right orientation of objects in LO is simply due to a feedforward effect from earlier visual areas, rather than characteristic of mirror sensitivity to objects in particular? While we do not think this could be the case (because participants were asked to fixate, and thus the stimuli were moving across the fovea), we directly addressed this question by comparing the peak response to the three conditions in FC (anatomically defined; see Materials and Methods), and, not surprisingly, found that FC did not even show fMRI adaptation for different versus same objects (t(24) = 1.46, p = 0.16), confirming that the sensitivity of LO to mirror images of objects is not simply due to adaptation in early visual cortex.

Scene-selective cortex

As predicted, two scene-selective regions, TOS and RSC, were found to be sensitive to mirror-reversed images of scenes. For TOS, a three-level (condition: Different, Mirror, Same) repeated-measures ANOVA revealed a significant main effect of condition (F(2,44) = 7.17, p < 0.01), with a significantly greater response to the Different and Mirror conditions than the Same condition (main effect contrasts, both p values <0.05) (Fig. 5A). Additionally, paired t tests revealed a significantly greater response to the Different compared to Same conditions (t(22) = 3.12, p < 0.01), no significant difference between the Different compared to Mirror conditions (t(22) = 0.32, p = 0.75), but a significantly greater response between the Mirror and Same conditions (t(22) = 2.74, p < 0.05). Similarly, for RSC a three-level (condition: Different, Mirror, Same) repeated-measures ANOVA revealed a significant main effect of condition (F(2,42) = 6.17, p < 0.01), with a significantly greater response to the Different and Mirror conditions than the Same condition (main effect contrasts, both p values <0.05) (Fig. 5B). Paired t tests revealed a significantly greater response to the Different compared to Same conditions (t(21) = 3.10, p < 0.01), no significant difference between the Different compared to Mirror conditions (t(21) = 0.56, p = 0.58), but a significantly greater response between the Mirror and Same conditions (t(21) = 2.65, p < 0.05).Together, these results demonstrate the expected fMRI adaptation effect in both TOS and RSC, but no adaptation across mirror images, revealing that these two scene-selective regions treat mirror-reversed images of scenes as two different images.

Figure 5.

Figure 5.

Open in a new tab

Hemodynamic time courses (percentage signal change) of three scene-selective regions of cortex—TOS (A), RSC (B), and PPA (C) to (1) two completely different images of scenes (red line labeled “different”), (2) the same image of a scene presented twice (blue line labeled “same”), and (3) a scene followed by the mirror-reversed version of the same scene (green line labeled “mirror”). Note sensitivity to mirror-image reversals in both TOS and RSC, but tolerance to mirror-image reversals in PPA.

In contrast, the other scene-selective region, PPA, showed tolerance to mirror-reversed images of scenes. A three-level (condition: Different, Mirror, Same) repeated-measures ANOVA revealed a significant main effect of condition (F(2,46) = 26.29, p < 0.001), with a significantly greater response to the Different condition compared to either the Mirror or Same conditions (main effect contrasts, both p values <0.05) (Fig. 5C). Furthermore, paired t tests revealed a significantly greater response to the Different compared to Same conditions (t(23) = 6.26, p < 0.001), a significantly greater difference between the Different compared to Mirror conditions (t(23) = 6.23, p < 0.001), but no significant difference between the Mirror and Same conditions (t(23) = 1.74, p = 0.10). Twenty-one of the 24 participants showed this effect (Fig. 6). This finding demonstrates the expected fMRI adaptation effect, as well as the mirror-image adaptation effect, revealing that PPA, unlike TOS and RSC, treats mirror-reversed images of scenes similar to identical image pairs, indicating a tolerant response to mirror image reversals (Table 2).

Figure 6.

Figure 6.

Open in a new tab

The peak amplitude response (percentage signal change) of the PPA for the Different minus Same conditions (labeled “Different-Same”) and the Mirror minus Same conditions (labeled “Mirror-Same”) for each participant. Twenty-one of the 24 participants show a greater difference for the Different-Same conditions than Mirror-Same conditions, indicating not only the typical fMRI adaptation effect (Different > Same), but also the tolerance to mirror-image scenes in nearly 90% of the participants.

Despite the tolerance to mirror-reversed images of scenes for PPA at the peak response (i.e., 4 s), note that in Figure 5C there appears to be a trend toward sensitivity to mirror reversals at the next time point (i.e., 5 s): the Mirror condition lies between the Different and Same conditions. A paired t test indeed revealed that the response in the Mirror condition is significantly lower than in the Different condition (t(24) = 4.91, p < 0.001) and significantly greater than the Same condition (t(24) = 3.53, p < 0.001). This analysis shows that PPA does not show total invariance across mirror image reversals, but it nonetheless shows substantial tolerance to such reversals (i.e., significant adaptation across mirror image reversals compared to completely new scenes).

Do the three scene-selective regions encode sense information of scenes differently? To address this question, we compared the difference between the peak responses for two different images and mirror reversals to the difference between the peak responses for two mirror reversals and the same images (Fig. 7). A 3 (ROI: TOS, RSC, PPA) × 2 (difference score: Different-Same, Mirror-Same) repeated-measures ANOVA revealed a significant interaction (F(2,40) = 6.83, p < 0.01; partial η2 = 0.26—where η2 > 0.25 constitutes a “large” effect size when df > 1) (Cohen, 1988; Levine and Hullett, 2002), with a significantly greater difference between the Different and Same conditions than between the Mirror and Same conditions for PPA, relative to TOS or RSC. This result suggests that sense information is represented differently by the scene-selective regions: PPA is tolerant to changes in the directional orientation of scenes, while TOS and RSC encode these changes.

Figure 7.

Figure 7.

Open in a new tab

For each scene-selective ROI, the difference between the peak responses for two different images of scenes and the same images (labeled “Different-Same”) was compared to the difference between the peak responses for two mirror reversals and the same images (labeled “Mirror-Same”). A 3 (ROI: TOS, RSC, PPA) × 2 (difference score: Different-Same, Mirror-Same) repeated-measures ANOVA revealed a significant interaction (F(2,40) = 6.83, p < 0.01), with a significantly greater difference between the Different and Same conditions than between the Mirror and Same conditions for PPA, relative to TOS or RSC. This result suggests that sense information is represented differently by the scene-selective regions: PPA is tolerant to changes in the directional orientation of scenes, while TOS and RSC encode these changes.

Given that our stimuli included objects as well as scenes, we were also able to investigate how TOS, PPA, and RSC might respond to mirror reversals of objects (the nonpreferred category). We found that none of the scene-selective regions revealed fMRI adaptation for different versus same objects (t(24) = 0.96, p = 0.35 for TOS; t(24) = 1.90, p = 0.08 for PPA; RSC showed the complete opposite effect—a significantly greater response to the Same condition than the Different condition); thus, the question of sensitivity to sense information in objects for scene-selective regions is moot.

Might it be the case that the sensitivity to left–right orientation of scenes in TOS or RSC is simply inherited from earlier visual areas, rather than characteristic of mirror sensitivity to scenes? Again, while we do not think this could be the case (because participants were asked to fixate, and thus the stimuli were moving across the fovea), we compared the peak response to the three conditions in FC and found that FC did not even show fMRI adaptation for Different versus Same scenes (t(23) = 0.49, p = 0.63), thus confirming that neither TOS nor RSC sensitivity to mirror images of scenes is due to adaptation in early visual cortex.

Discussion

The current study asked whether representations in object-selective cortex and in scene-selective cortex are tolerant to mirror reversals. The results suggest that both mirror-image sensitive and mirror-image tolerant representations exist in both the object and scene processing pathways. Specifically, using a fMRI adaptation paradigm we found tolerance to mirror reversals in one object-selective region (pFs), but not in another more posterior region (LO), suggesting a hierarchy of object processing where left–right information is represented at earlier (more posterior) stages in the hierarchy and invariance is then computed at later (more anterior) stages. Surprisingly, we also found that the PPA, a scene-selective region, was tolerant to mirror reversals, suggesting that the sense information that is critical for navigation is not encoded in the PPA. By contrast, we found that two other scene-selective regions (TOS and RSC) were sensitive to mirror reversals; the computations necessary for navigation and reorientation could be extracted in part by one or both of these regions.

Our finding of both sensitivity and tolerance to mirror-image reversals in the object-processing system might shed some light on a fundamental problem for neuroscience and computational neuroscience: How are invariant representations generated, given large changes in size, position, illumination, etc. in the sensory input? A recent study has addressed this question for face processing (Freiwald and Tsao, 2010), suggesting that mirror-invariant representations might constitute a crucial intermediate step toward full invariance. Perhaps something similar occurs in the object-processing pathway, with LO constituting the early stage with no viewpoint invariance and pFs a later stage where mirror invariance has been accomplished. An important hypothesis to test in the future is whether the object-processing pathway contains a stage (perhaps pFs) at which invariance to mirror reversals is found but full invariance has not yet been obtained.

We found sensitivity to mirror images of scenes in the scene-selective cortical regions TOS and RSC, as expected given that the chirality of a scene is crucial for navigation. By analogy, letter recognition crucially requires sense information (e.g., to distinguish between the letters b and d), and indeed a recent fMRI study reported that the visual word form area, a region in the human ventral stream, exhibited sensitivity to the left–right orientation of single letters (Dehaene et al., 2010a,b; Pegado et al., 2011). However, surprisingly, the PPA was tolerant to mirror-image reversals, challenging hypotheses that this region is involved in navigation (Ghaem et al., 1997; Ino et al., 2002; Janzen and van Turennout, 2004; Rosenbaum et al., 2004; Rauchs et al., 2008) and reorientation (Epstein and Kanwisher, 1998; Cheng and Newcombe, 2005; Spelke et al., 2010), functions that require this sense information. One alternative hypothesis is that the PPA is involved in the recognition of scene categories (e.g., “bedroom,” “playground,” “desert”), a function where sense information is not critical. In particular, at some level of scene processing a pathway devoted to scene category recognition may exist distinct from the pathway primarily involved in encoding information necessary for navigation. If that hypothesis is correct, then the PPA may contribute to the former pathway, and TOS, RSC or both may contribute to the latter pathway.

While current data do not directly address our “two streams for scene processing” hypothesis, several neuroimaging studies highlighting the role of RSC in human navigation support this hypothesis (O'Craven and Kanwisher, 2000; Cooper et al., 2001; Maguire, 2001; Cain et al., 2006; Iaria et al., 2007). Moreover, patients with RSC damage have been reported to recognize salient landmarks but not use these landmarks to orient themselves or to navigate through a larger environment; Takahashi et al. (1997) suggested that these patients had lost a “sense of direction.” This finding contrasts with patients who have PPA damage and have deficits in simple identification of scenes or landmarks (Aguirre and D'Esposito, 1999; Mendez and Cherrier, 2003). Note, however, that these studies suggest a somewhat different division of labor between RSC and PPA—as guiding navigation by “heading direction” (RSC) versus landmarks (PPA) than the “two streams for scene processing” hypothesis that we propose—as guiding navigation (RSC) versus scene categorization (PPA).

While we found that scene representations in PPA were tolerant to reflections about the vertical axis (a transformation of 180°), the further question of whether these representations are also tolerant to less severe transformations (e.g., 45° or 90° rotations) or are only tolerant to mirror-reversals is an interesting one. Indeed, two neuroimaging studies found that the PPA was sensitive to viewpoint changes (well under 180° rotations) (Epstein et al., 2003; Park and Chun, 2009). However, these same researchers also found viewpoint invariance within the PPA as subjects become familiar with the scenes over the course of an experimental session (Epstein et al., 2005; Epstein and Higgins, 2007), or when the viewpoint change across panoramic scenes was not continuous (Park and Chun, 2009).

In conclusion, we have shown that within both the object and scene processing pathways, some regions show sensitivity, whereas others show tolerance to mirror image reversals of the stimulus. We speculate that two phenomena are at play here. First, whereas early stages of processing (e.g., TOS and LO) generally show less tolerance to image changes (Ito et al., 1995; Grill-Spector et al., 1998; Grill-Spector et al., 1999; Lerner et al., 2001; Sawamura et al., 2005; Rust and Dicarlo, 2010), later stages (e.g., pFs and PPA) representing the abstract identity of the object or scene require tolerance to image changes. But second, even at apparently higher levels of processing, the computations underlying visually guided action for both objects (e.g., grasping) and scenes (e.g., navigation) require sensitivity, not tolerance, to viewing conditions, as reported for objects in the dorsal visual pathway (Milner and Goodale, 1998) and as found here for RSC. On this interpretation, scenes, like objects, are processed along two distinct pathways, one for recognition, and the other for action.

Footnotes

This work was supported by National Institutes of Health Grant EY013455 (N,K,). J.K. is currently a Research Assistant of the Research Foundation-Flanders (FWO-Vlaanderen) and a member of a research group supported by a Methusalem Grant from the Flemish Government (METH/08/02). We thank the Athinoula A. Martinos Imaging Center at the McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA.

References

  1. Aguirre GK, D'Esposito M. Topographical disorientation: a synthesis and taxonomy. Brain. 1999;122:1613–1628. doi: 10.1093/brain/122.9.1613. [DOI] [PubMed] [Google Scholar]
  2. Baker CI, Peli E, Knouf N, Kanwisher NG. Reorganization of visual processing in macular degeneration. J Neurosci. 2005;25:614–618. doi: 10.1523/JNEUROSCI.3476-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Baker CI, Dilks DD, Peli E, Kanwisher N. Reorganization of visual processing in macular degeneration: replication and clues about the role of foveal loss. Vision Res. 2008;48:1910–1919. doi: 10.1016/j.visres.2008.05.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bornstein MH. Perceptual anisotropies in infancy: ontogenetic origins and implications of inequalities in spatial vision. Adv Child Dev Behav. 1982;16:77–123. doi: 10.1016/s0065-2407(08)60068-3. [DOI] [PubMed] [Google Scholar]
  5. Burock MA, Buckner RL, Woldorff MG, Rosen BR, Dale AM. Randomized event-related experimental designs allow for extremely rapid presentation rates using functional MRI. Neuroreport. 1998;9:3735–3739. doi: 10.1097/00001756-199811160-00030. [DOI] [PubMed] [Google Scholar]
  6. Cain DP, Boon F, Corcoran ME. Thalamic and hippocampal mechanisms in spatial navigation: a dissociation between brain mechanisms for learning how versus learning where to navigate. Behav Brain Res. 2006;170:241–256. doi: 10.1016/j.bbr.2006.02.023. [DOI] [PubMed] [Google Scholar]
  7. Cheng K, Newcombe NS. Is there a geometric module for spatial orientation? Squaring theory and evidence. Psychon Bull Rev. 2005;12:1–23. doi: 10.3758/bf03196346. [DOI] [PubMed] [Google Scholar]
  8. Cohen J. 2nd Ed. Hillsdale, NJ: L. Erlbaum Associates; 1988. Statistical power analysis for the behavioral sciences. [Google Scholar]
  9. Cooper BG, Manka TF, Mizumori SJ. Finding your way in the dark: the retrosplenial cortex contributes to spatial memory and navigation without visual cues. Behav Neurosci. 2001;115:1012–1028. doi: 10.1037//0735-7044.115.5.1012. [DOI] [PubMed] [Google Scholar]
  10. Corballis MC, Beale IL. The psychology of left and right. Hillsdale, NJ: Lawrence Erlbaum Associates; 1976. [Google Scholar]
  11. Cox RW, Jesmanowicz A. Real-time 3D image registration for functional MRI. Magn Reson Med. 1999;42:1014–1018. doi: 10.1002/(sici)1522-2594(199912)42:6<1014::aid-mrm4>3.0.co;2-f. [DOI] [PubMed] [Google Scholar]
  12. Dale AM, Greve DN, Burock MA. Optimal stimulus sequences for event-related fMRI. Paper presented at the Fifth International Conference of Functional Mapping of the Human Brain; June 11–16; Duesseldorf, Germany. 1999. [Google Scholar]
  13. Dehaene S, Nakamura K, Jobert A, Kuroki C, Ogawa S, Cohen L. Why do children make mirror errors in reading? Neural correlates of mirror invariance in the visual word form area. Neuroimage. 2010a;49:1837–1848. doi: 10.1016/j.neuroimage.2009.09.024. [DOI] [PubMed] [Google Scholar]
  14. Dehaene S, Pegado F, Braga LW, Ventura P, Nunes Filho G, Jobert A, Dehaene-Lambertz G, Kolinsky R, Morais J, Cohen L. How learning to read changes the cortical networks for vision and language. Science. 2010b;330:1359–1364. doi: 10.1126/science.1194140. [DOI] [PubMed] [Google Scholar]
  15. Dilks DD, Baker CI, Peli E, Kanwisher N. Reorganization of visual processing in macular degeneration is not specific to the “preferred retinal locus.”. J Neurosci. 2009;29:2768–2773. doi: 10.1523/JNEUROSCI.5258-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Dougherty RF, Koch VM, Brewer AA, Fischer B, Modersitzki J, Wandell BA. Visual field representations and locations of visual areas V1/2/3 in human visual cortex. J Vis. 2003;3:586–598. doi: 10.1167/3.10.1. [DOI] [PubMed] [Google Scholar]
  17. Eger E, Schyns PG, Kleinschmidt A. Scale invariant adaptation in fusiform face-responsive regions. Neuroimage. 2004;22:232–242. doi: 10.1016/j.neuroimage.2003.12.028. [DOI] [PubMed] [Google Scholar]
  18. Epstein R, Kanwisher N. A cortical representation of the local visual environment. Nature. 1998;392:598–601. doi: 10.1038/33402. [DOI] [PubMed] [Google Scholar]
  19. Epstein R, Graham KS, Downing PE. Viewpoint-specific scene representations in human parahippocampal cortex. Neuron. 2003;37:865–876. doi: 10.1016/s0896-6273(03)00117-x. [DOI] [PubMed] [Google Scholar]
  20. Epstein RA, Higgins JS. Differential parahippocampal and retrosplenial involvement in three types of visual scene recognition. Cereb Cortex. 2007;17:1680–1693. doi: 10.1093/cercor/bhl079. [DOI] [PubMed] [Google Scholar]
  21. Epstein RA, Higgins JS, Thompson-Schill SL. Learning places from views: variation in scene processing as a function of experience and navigational ability. J Cogn Neurosci. 2005;17:73–83. doi: 10.1162/0898929052879987. [DOI] [PubMed] [Google Scholar]
  22. Freiwald WA, Tsao DY. Functional compartmentalization and viewpoint generalization within the macaque face-processing system. Science. 2010;330:845–851. doi: 10.1126/science.1194908. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Ghaem O, Mellet E, Crivello F, Tzourio N, Mazoyer B, Berthoz A, Denis M. Mental navigation along memorized routes activates the hippocampus, precuneus, and insula. Neuroreport. 1997;8:739–744. doi: 10.1097/00001756-199702100-00032. [DOI] [PubMed] [Google Scholar]
  24. Grill-Spector K, Malach R. fMR-adaptation: a tool for studying the functional properties of human cortical neurons. Acta Psychol (Amst) 2001;107:293–321. doi: 10.1016/s0001-6918(01)00019-1. [DOI] [PubMed] [Google Scholar]
  25. Grill-Spector K, Kushnir T, Hendler T, Edelman S, Itzchak Y, Malach R. A sequence of object-processing stages revealed by fMRI in the human occipital lobe. Hum Brain Mapp. 1998;6:316–328. doi: 10.1002/(SICI)1097-0193(1998)6:4&#x0003c;316::AID-HBM9&#x0003e;3.0.CO;2-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Grill-Spector K, Kushnir T, Edelman S, Avidan G, Itzchak Y, Malach R. Differential processing of objects under various viewing conditions in the human lateral occipital complex. Neuron. 1999;24:187–203. doi: 10.1016/s0896-6273(00)80832-6. [DOI] [PubMed] [Google Scholar]
  27. Iaria G, Chen JK, Guariglia C, Ptito A, Petrides M. Retrosplenial and hippocampal brain regions in human navigation: complementary functional contributions to the formation and use of cognitive maps. Eur J Neurosci. 2007;25:890–899. doi: 10.1111/j.1460-9568.2007.05371.x. [DOI] [PubMed] [Google Scholar]
  28. Ino T, Inoue Y, Kage M, Hirose S, Kimura T, Fukuyama H. Mental navigation in humans is processed in the anterior bank of the parieto-occipital sulcus. Neurosci Lett. 2002;322:182–186. doi: 10.1016/s0304-3940(02)00019-8. [DOI] [PubMed] [Google Scholar]
  29. Ito M, Tamura H, Fujita I, Tanaka K. Size and position invariance of neuronal responses in monkey inferotemporal cortex. J Neurophysiol. 1995;73:218–226. doi: 10.1152/jn.1995.73.1.218. [DOI] [PubMed] [Google Scholar]
  30. Janzen G, van Turennout M. Selective neural representation of objects relevant for navigation. Nat Neurosci. 2004;7:673–677. doi: 10.1038/nn1257. [DOI] [PubMed] [Google Scholar]
  31. Kim JG, Biederman I, Lescroart MD, Hayworth KJ. Adaptation to objects in the lateral occipital complex (LOC): shape or semantics? Vision Res. 2009;49:2297–2305. doi: 10.1016/j.visres.2009.06.020. [DOI] [PubMed] [Google Scholar]
  32. Kourtzi Z, Kanwisher N. Representation of perceived object shape by the human lateral occipital complex. Science. 2001;293:1506–1509. doi: 10.1126/science.1061133. [DOI] [PubMed] [Google Scholar]
  33. Lerner Y, Hendler T, Ben-Bashat D, Harel M, Malach R. A hierarchical axis of object processing stages in the human visual cortex. Cereb Cortex. 2001;11:287–297. doi: 10.1093/cercor/11.4.287. [DOI] [PubMed] [Google Scholar]
  34. Levine TR, Hullett CR. Eta-squared, partial eta-squared, and misreporting of effect size in communication research. Hum Commun Res. 2002;28:612–625. [Google Scholar]
  35. Logothetis NK, Pauls J, Poggio T. Shape representation in the inferior temporal cortex of monkeys. Curr Biol. 1995;5:552–563. doi: 10.1016/s0960-9822(95)00108-4. [DOI] [PubMed] [Google Scholar]
  36. Maguire EA. The retrosplenial contribution to human navigation: a review of lesion and neuroimaging findings. Scand J Psychol. 2001;42:225–238. doi: 10.1111/1467-9450.00233. [DOI] [PubMed] [Google Scholar]
  37. McCloskey M. Spatial Representations and mutliple-visual-systems hypothesis: evidence from a developmental deficit in visual location and orientation processing. Cortex. 2004;40:677–694. doi: 10.1016/s0010-9452(08)70164-3. [DOI] [PubMed] [Google Scholar]
  38. Mendez MF, Cherrier MM. Agnosia for scenes in topographagnosia. Neuropsychologia. 2003;41:1387–1395. doi: 10.1016/s0028-3932(03)00041-1. [DOI] [PubMed] [Google Scholar]
  39. Milner AD, Goodale MA. The visual brain in action. Psyche. 1998;4 [Google Scholar]
  40. O'Craven KM, Kanwisher N. Mental imagery of faces and places activates corresponding stiimulus-specific brain regions. J Cogn Neurosci. 2000;12:1013–1023. doi: 10.1162/08989290051137549. [DOI] [PubMed] [Google Scholar]
  41. Park S, Chun MM. Different roles of the parahippocampal place area (PPA) and retrosplenial cortex (RSC) in panoramic scene perception. Neuroimage. 2009;47:1747–1756. doi: 10.1016/j.neuroimage.2009.04.058. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Pegado F, Nakamura K, Cohen L, Dehaene S. Breaking the symmetry: mirror discrimination for single letters but not for pictures in the Visual Word Form Area. Neuroimage. 2011;55:742–749. doi: 10.1016/j.neuroimage.2010.11.043. [DOI] [PubMed] [Google Scholar]
  43. Rauchs G, Orban P, Balteau E, Schmidt C, Degueldre C, Luxen A, Maquet P, Peigneux P. Partially segregated neural networks for spatial and contextual memory in virtual navigation. Hippocampus. 2008;18:503–518. doi: 10.1002/hipo.20411. [DOI] [PubMed] [Google Scholar]
  44. Riddoch MJ, Humphreys GW. Description of a left/right coding deficit in a case of constructional apraxia. Cogn Neuropsychol. 1988;5:289–314. [Google Scholar]
  45. Rollenhagen JE, Olson CR. Mirror-image confusion in single neurons of the macaque inferotemporal cortex. Science. 2000;287:1506–1508. doi: 10.1126/science.287.5457.1506. [DOI] [PubMed] [Google Scholar]
  46. Rosenbaum RS, Ziegler M, Winocur G, Grady CL, Moscovitch M. “I have often walked down this street before”: fMRI studies on the hippocampus and other structures during mental navigation of an old environment. Hippocampus. 2004;14:826–835. doi: 10.1002/hipo.10218. [DOI] [PubMed] [Google Scholar]
  47. Rust NC, Dicarlo JJ. Selectivity and tolerance (“invariance”) both increase as visual information propagates from cortical area V4 to IT. J Neurosci. 2010;30:12978–12995. doi: 10.1523/JNEUROSCI.0179-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Sawamura H, Georgieva S, Vogels R, Vanduffel W, Orban GA. Using functional magnetic resonance imaging to assess adaptation and size invariance of shape processing by humans and monkeys. J Neurosci. 2005;25:4294–4306. doi: 10.1523/JNEUROSCI.0377-05.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Spelke E, Lee SA, Izard V. Beyond core knowledge: natural geometry. Cogn Sci. 2010;34:863–884. doi: 10.1111/j.1551-6709.2010.01110.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Takahashi N, Kawamura M, Shiota J, Kasahata N, Hirayama K. Pure topographic disorientation due to right retrosplenial lesion. Neurology. 1997;49:464–469. doi: 10.1212/wnl.49.2.464. [DOI] [PubMed] [Google Scholar]
  51. Turnbull OH, McCarthy RA. Failure to discriminate between mirror-image objects: a case of viewpoint-independent object recognition. Neurocase. 1996;2:63–72. [Google Scholar]
  52. Vuilleumier P, Schwartz S, Duhoux S, Dolan RJ, Driver J. Selective attention modulates neural substrates of repetition priming and “implicit” visual memory: suppressions and enhancements revealed by FMRI. J Cogn Neurosci. 2005;17:1245–1260. doi: 10.1162/0898929055002409. [DOI] [PubMed] [Google Scholar]
  53. Walsh V. Reflections on mirror images. Neuropsychology. Curr Biol. 1996;6:1079–1081. doi: 10.1016/s0960-9822(02)70670-2. [DOI] [PubMed] [Google Scholar]
  54. Warrington EK, Davidoff J. Failure at object identification improves mirror image matching. Neuropsychologia. 2000;38:1229–1234. doi: 10.1016/s0028-3932(00)00040-3. [DOI] [PubMed] [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES