Interactions of Memory and Perception in Amnesia: The Figure–Ground Perspective

Abstract

The medial temporal lobes (MTLs) have been thought to function exclusively in service of declarative memory. Recent research shows that damage to the perirhinal cortex (PRC) of the MTL impairs the discrimination of objects sharing many similar parts/features, leading to the hypothesis that the PRC contributes to the perception when the feature configurations, rather than the individual features, are required to solve the task. It remains uncertain, however, whether the previous research demands a slight extension of PRC function to include working memory or a more dramatic extension to include perception. We present 2 experiments assessing the implicit effects of familiar configuration on figure assignment, an early and fundamental perceptual outcome. Unlike controls, PRC-damaged individuals failed to perceive the regions portraying familiar configurations, as figure more often, than the regions comprising the same parts rearranged into novel configurations. They were also impaired in identifying the familiar objects. In a third experiment, PRC-damaged individuals performed poorly when asked to choose a familiar object from pairs of familiar and novel objects comprising the same parts. Our results demonstrate that the PRC is involved in both implicit and explicit perceptual discriminations of novel and familiar configurations. These results reveal that complex object representations in the PRC subserve both perception and memory.

Introduction

Amnesia results from the bilateral damage to the medial temporal lobe (MTL), a set of heavily interconnected structures, including the hippocampus and surrounding entorhinal, perirhinal, and parahippocampal cortices (Scoville and Milner 1957). An influential view is that the MTL constitutes a system responsible for long-term declarative memory—our conscious memory for facts, objects, and events (e.g., Eichenbaum and Cohen 2001; Squire and Wixted 2011). On this view, the primary function of the MTL, including all of its substructures, is for declarative memory and not for other aspects of cognition, such as perception (Suzuki 2009; Clark et al. 2011; Kim et al. 2011).

Recent reports have challenged the traditional view that the mechanisms and the representations underlying memory and perception are anatomically segregated, suggesting instead that the MTL—in particular the perirhinal cortex (PRC)—is not only important for declarative memory but also essential for certain forms of perception (e.g., Buckley et al. 2001; Lee, Bussey, et al. 2005; Devlin and Price 2007; O'Neil et al. 2009; McTighe et al. 2010; Barense et al. 2011). In support of this view, lesion studies in humans, monkeys, and rats demonstrate that the PRC of the MTL fundamentally supports memory and perceptual tasks when successful performance requires the processing of feature configurations rather than single features (Bussey et al. 2002; Barense et al. 2005, 2007; Bartko et al. 2007b; Barense, Rogers, et al. 2010; but see Clark et al. 2011). These findings support the view that the PRC may best be understood as an extension of the representational hierarchy within the ventral visual stream, in that complex conjunctions of stimulus features are represented in the anterior inferotemporal cortical regions (including PRC) and individual features comprising these conjunctions are represented in more posterior regions (e.g., V4, TEO) (Desimone and Ungerleider 1989; Riesenhuber and Poggio 1999; Bussey and Saksida 2002; Bussey et al. 2002; Barense et al. 2005; McTighe et al. 2010). Under this alternative view, common representations and computational mechanisms in the MTL are used for both the mnemonic and the perceptual functions (Bussey and Saksida 2007; Murray et al. 2007; Baxter 2009; Graham et al. 2010).

One main issue has clouded the interpretation of experiments showing the visual discrimination deficits following PRC damage: because the tasks used to date require a series of eye movements or overt exploration and involve the working memory, it is impossible to separate the deficits of perception from those of the working memory. For example, one frequently used task requires an “oddity judgment” task, in which participants must scan multiple simultaneously presented images to determine the odd one out (e.g., Buckley et al. 2001; Lee, Buckley, et al. 2005; Bartko et al. 2007a; Barense, Rogers, et al. 2010). Working memory for previously scanned locations and object–location pairings is crucial to this task. A more recent experiment used an object decision task, in which participants determined whether pictured large 3D objects were structurally possible or impossible (Lee and Rudebeck 2010). Inasmuch as the object decision task requires when comparing the 3D structure of different corners of an object, it also requires the eye movements and working memory for form and location. Thus, it is unclear whether the extant evidence supports the transformative view that the PRC can play a role in perception or the less controversial view that the PRC plays a role in working memory (Ranganath and Blumenfeld 2005; Hannula et al. 2006; Nichols et al. 2006; Olson et al. 2006; Warren et al. 2010).

To determine whether PRC plays a role in the perception of feature configurations, we must employ a task that indexes configural processing but has no working memory burden. One such task is figure assignment. Figure assignment is fundamental to object perception. It entails determining which of 2 regions that share a border in the visual field is shaped by that border (the figure) and which appears to simply continue behind the figure (the ground). Figure assignment occurs very early in the course of perceptual processing (Lamme et al. 1992; Zipser et al. 1996; Caputo and Casco 1999; Zhou et al. 2000), yet stored object representations exert an influence. The figure is more likely to be perceived when one side of a border depicts the parts of a well-known object in their typical “intact” configuration rather than in a novel “scrambled” configuration, in which the parts are rearranged (Fig. 1a) (e.g., Peterson et al. 1991, 1998; Gibson and Peterson 1994; Peterson and Gibson 1994a, 1994b). Furthermore, when the figure–ground displays are inverted, participants are less likely to perceive the figure on the side where the intact typical configuration lies, presumably because the stored object representations specify an object's typical orientation (i.e., its more “familiar” orientation) and the access to these representations by inverted configurations is too slow to exert an influence on the figure–ground perception (e.g., Jolicoeur 1988; Perrett et al. 1998). The orientation dependency of these effects of typical, or familiar, configuration indicated that the stored representations of the typical/familiar configurable structure of well-known objects must be accessed quickly if they are to exert an influence on the figure–ground perception.

Figure 1.

(a) Test of effects of familiar configuration on figure assignment (Experiments 1 and 2). The critical regions of the sample stimuli in the top row depict the intact configuration of familiar objects; from left to right: a guitar, a standing woman, and a table lamp. The critical regions of the stimuli in the bottom row depict versions of these objects with their parts (delimited by successive minima of curvature) spatially rearranged. For display purposes, the critical regions here are always shown in black and on the left (this was fully counterbalanced in the experiment). For each stimulus, participants indicated whether they saw the black or the white region as figure. (b,c) The Gestalt configural cue of convexity (Experiments 1 and 2). Sample (b) 2-region stimuli and (c) 8-region stimuli from the test of convexity effects on figure assignment. The region(s) with convex parts is(are) black in (b) and white in (c). There is typically a bias to see convex regions as a figure that is larger in the 8- than in the 2-region displays. For each stimulus, participants indicated whether they saw the black or the white region as figure. (d) Sample trial from the familiarity discrimination task (Experiment 3). Participants viewed pairs of intact and part-rearranged objects in black and indicated which was more familiar.

It is important to note that a declarative knowledge regarding the well-known objects depicted in the figure–ground displays is neither necessary nor sufficient for these effects of familiar configuration on figure assignment. Effects of familiar configuration on figure assignment were not restored for inverted displays or for part-rearranged scrambled configurations when the subjects were informed of how they were created from the intact upright configurations, demonstrating that explicit object knowledge is not sufficient for the familiar configuration effects on the figure assignment (Peterson et al. 1991; Peterson and Gibson 1994a). In addition, a patient with intact object recognition abilities and only mild simultanagnosia following damage to Brodmann Areas 18 and 19 failed to show effects of familiar configuration on figure assignment, even though he was able to explicitly identify the well-known objects portrayed by the figures he perceived (Peterson et al. 2000). Evidence that the declarative object memory is not necessary for effects of familiar configuration on figure assignment was obtained in tests of a visual agnosic, who—despite her severely impaired explicit object identification abilities—showed normal effects of familiar configuration on the figure assignment, even though she could not identify the well-known objects portrayed by the figures she perceived (Peterson et al. 2000). Together with other results, which were obtained using briefly exposed displays and/or priming paradigms with non–brain-damaged participants, these results show that stored representations of the part configurations of well-known objects are accessed quickly and automatically without awareness (Peterson and Lampignano 2003; Peterson and Enns 2005; Peterson and Skow 2008; Trujillo et al. 2010; for discussions of how fast access would be instantiated at a neural systems level, see Lamme and Roelfsema 2000; Bullier 2001). When fast access is slowed by brain damage or by inverting the displays, effects of a familiar configuration on the figure assignment are not observed. Because explicit object recognition is neither necessary nor sufficient for the effects of a familiar configuration on the figure assignment, this task provides a strong test of the hypothesis that the PRC plays a role in implicit perception as well as in memory.

Here, we ask whether PRC damage weakens “effects of familiar configuration on figure assignment.” Such a finding would provide a strong support for the hypothesis that the PRC plays a role in perception when access to the configurations, rather than to the individual parts or features, is key. To investigate these issues, the performance of 2 groups of amnesic patients, one with selective bilateral hippocampal damage (HC cases) and another with damage to the MTL regions, including PRC (MTL cases), was assessed on 2 tests of figure assignment. In the first figure–ground task, we assessed the effects of familiar configuration on figure assignment. Participants reported their first impression of which 2 contiguous regions in bipartite black and white displays appeared to be the figure (Fig. 1a). In each display, one region is a “critical” region (shown in black in Fig. 1a). Half of the critical regions portrayed intact configurations of mono-oriented well-known/familiar objects (top row of Fig. 1a); the other half portrayed novel configurations created by spatially rearranging the parts of the familiar objects (bottom row of Fig. 1a). Participants also performed a control test assessing contributions of convexity, a generic Gestalt figural cue, to figure assignment (Fig. 1b,c; Peterson and Salvagio 2008). In a second experiment, inverted displays were used to remove the effects of familiar configuration. In Experiment 3, the MTL cases were asked to discriminate between well-known/familiar and novel object configurations comprising the same parts (Fig. 1d). We found that the MTL cases, but not the HC cases, failed to show standard effects of familiar configuration on the figure assignment. All groups, however, showed normal contributions of convexity to the figure assignment. Across all 3 experiments, the performance of MTL patients was based on the familiarity of “individual parts” rather than on the familiarity of the “configuration of parts.” The pattern of results supports the view that the PRC of the MTL is dynamically involved in the perceptual discrimination of familiar and novel configurations, a role that is essential for unimpaired effects of a familiar configuration on the figure–ground perception.

Materials and Methods

Participants

Based on the structural and volumetric analyses of the critical regions within the MTL, the 4 patients were categorized as follows: 1) individuals with bilateral MTL damage that included PRC (MTL cases: n = 2, mean age = 69.5 years) and 2) control patients with selective bilateral hippocampal damage (HC cases: n = 2, mean age = 50.5 years). Details of each case's etiology, demographics, and performance on an extensive neuropsychological battery are provided in the Supplementary Table 1. These individuals have been described in previous reports, and for consistency, the same labels are used here as those used previously (HC2, HC3, MTL2, and MTL 3 described in Barense et al. 2005, 2007; Lee, Bussey, et al. 2005; Graham et al. 2006). MTL2, MTL3, and HC2 experienced viral encephalitis, HC3 suffered carbon monoxide poisoning. To briefly summarize their neuropsychological performance, both the groups of patients had severe deficits in episodic memory. For instance, both patient groups performed poorly on the immediate recall, delayed recall, and recognition subtests of the Logical Memory (WMS-III, Story 1 and 2) and on delayed recall of the Rey Complex Figure. Their visuoperceptual performance was within the normal control range as measured by the traditional neuropsychological tests, such as the Benton Face Test, Rey Complex Figure copy, and Visual Object Space Perception battery. We emphasize, however, that these perceptual tasks are not sufficiently taxing to reveal perceptual deficits of the type previously observed in these patients (Barense et al. 2005, 2007; Lee, Buckley, et al. 2005; Lee, Bussey, et al. 2005).

In addition, 30 age-matched (all P > 0.1) healthy volunteers, recruited from the Adult Volunteer Pool at the University of Toronto, served as the control participants for Experiments 1 and 2. These controls were split into a younger and an older group, to match the HC cases and MTL cases, respectively. There were no significant differences in performance between the 2 control groups on any of the experimental tasks described below (all P > 0.25), and thus for simplicity, they are treated as a single group. Seventeen controls were tested with upright displays in Experiment 1 (see below for details) and 13 were tested with inverted displays in Experiment 2 (Experiment 1: mean age = 64.3 years, range 52–76, 10 females; Experiment 2: mean age = 64.6 years, range 55–78, 4 females). Ten control participants age-matched (Ps > 0.2) to the MTL cases were recruited for Experiment 3 (mean age = 67.3 years, range 59–77, 5 females). All participants gave informed consent before taking part in the study. This work received ethical approval from the Ethics Review Office at the University of Toronto and the Oxfordshire Research Ethics Committee.

Volumetric Assessment of Patient Lesions

The structural magnetic resonance imaging (MRI) scans of patients HC2, HC3, and MTL3 were analyzed in comparison to matched female neurologically healthy control participants (Table 1, Supplementary Fig. 1). Due to claustrophobia, it was not possible to obtain a research-quality structural MRI scan for the patient MTL2 that was suitable for volumetric analyses. Nonetheless, qualitative visual ratings of a previous clinical MRI scan (described in Supplementary Material) revealed significant damage to the PRC, hippocampus, temporopolar cortex, amygdala, medial bank of the collateral sulcus, and the medial bank of the occipitotemporal sulcus, but not the lateral temporal cortex (Supplementary Table 2; Barense et al. 2005, 2007; Lee, Bussey, et al. 2005). The volumetric data for Patients HC3 and MTL3 and 11 matched female control participants (mean age 55.27 years, standard deviation [SD] = 10.80) were taken from a previous study (Lee and Rudebeck 2010). The structural scan of Patient HC2 (256 × 122 × 256 in size, voxel dimensions 0.86 × 1.80 × 0.86 mm) was acquired on a 1.5T GE Signa scanner at the MRI Department, Addenbrooke's Hospital, Cambridge, UK at age 39 years and compared to the same female control data (no significant difference in age between Patient HC2 and controls, t₁₀ = 1.44, P = 0.18). Regions of interest (ROIs) were manually traced on coronal slices in each hemisphere using the MRIcron software (Rorden and Brett 2000) and previously published methods (Lee and Rudebeck 2010). The hippocampus and amygdala were defined using the Mayo Clinic method (Watson et al. 1997), whereas the temporopolar cortex, entorhinal cortex, and PRC were identified using the Insausti protocol (Insausti et al. 1998). The parahippocampal cortex was measured from the slice following the posterior boundary of the PRC and the fusiform gyrus was measured from the slice coinciding with the anterior boundary of the PRC. The posterior boundaries of both the parahippocampal cortex and the fusiform gyrus coincided with the posterior boundary of the hippocampus. A measure for lateral temporal cortex was obtained by measuring the gray matter of the entire temporal cortex from the tip of the temporopolar cortex to the posterior end of the hippocampus and subtracting the volumes for temporopolar cortex, entorhinal cortex, PRC, parahippocampal cortex, and the fusiform gyrus. The fusiform gyrus and lateral temporal volumes were subdivided into 2 by measuring separately the slices anterior and posterior to the midpoint. All measured volumes were corrected for intracranial volume, which was determined by drawing around the brain tissue in all coronal slices including the gray and white matter, ventricular space, and excluding the brainstem below the level of the pons. Repeatability was assessed by remeasuring all the ROIs in 9 of the cases at least 6 weeks after the first measurement (all 3 patients and 6 controls) and calculating intraclass correlation coefficients. Good repeatability was found in all areas (all r > 0.9; Supplementary Table 3).

Table 1.

Individual patient's Z scores for each measured brain region in the left and right hemispheres

	Temporopolar cortex	Amygdala	Entorhinal cortex	Perirhinal cortex	Hippocampus	Parahippocampal cortex	Anterior fusiform gyrus	Posterior fusiform gyrus	Anterior lateral temporal cortex	Posterior lateral temporal cortex
Left
HC2	0.83	0.24	1.01	0.04	−2.48	1.58	0.03	1.82	−0.34	1.89
HC3	1.06	1.86.	1.44	0.18	−4.78	−0.74	−0.57	0.39	−0.43	0.49
MTL3	−0.17	−3.23	−4.72	−2.19	−3.46	−3.59	−1.08	−1.36	−0.58	−0.45
Right
HC2	3.63	0.41	0.24	0.30	−2.30	1.95	1.15	0.12	−0.09	1.33
HC3	0.43	0.94	0.31	−0.90	−3.92	−0.73	−0.09	0.78	−0.33	−0.53
MTL3	−7.01	−9.94	−4.63	−3.21	−6.66	−2.84	−3.31	−1.87	−5.27	−1.41

Note: Bold indicates a significantly reduced volume compared with the healthy control group (Z < −1.96).

Calculated Z scores for every measured brain region for each patient compared with the healthy controls revealed that patients HC2 and HC3 had significant bilateral HC damage (Z score < −1.96), with no significant damage beyond this structure (Table 1; Supplementary Fig. 1). As is common in amnesic patients with large MTL lesions, patient MTL3 has additional damage to the PRC bilaterally, as well as the entorhinal cortex, amygdala, and parahippocampal cortex bilaterally, and the temporopolar cortex, anterior fusiform gyrus, and anterior lateral temporal cortex in the right hemisphere. Importantly, although there was significant damage to the PRC and HC bilaterally, there was not significant atrophy to the posterior fusiform gyrus or posterior lateral temporal cortex in either hemisphere, suggesting intact posterior visual regions or lateral temporal areas. Moreover, 2 of the patients (HC3 and MTL3) have undergone functional neuroimaging, which revealed a normal parahippocampal place area (PPA), fusiform face area (FFA), and lateral occipital complex (LOC) (Lee and Rudebeck 2010). Thus, it is unlikely that the cortical regions more typically associated with the visual processing are damaged in these patients. Their profile of performance is consistent with the 2 convergent lines of research that allow more selective localization of the PRC: 1) animal studies that have demonstrated object discrimination deficits after selective PRC damage (Buckley et al. 2001; Bussey et al. 2002, 2003; Bartko et al. 2007a) and 2) functional neuroimaging studies revealing PRC activity in healthy participants during object discrimination tasks (Devlin and Price 2007; Lee et al. 2008; O'Neil et al. 2009; Barense, Henson, et al. 2010).

Stimuli and Behavioral Procedure

Participants were administered the test of effects of familiar configuration on figure assignment (the object memory effects on figure assignment [OMEFA] test, Peterson et al. 1998, 2000) and a test of their ability to use Gestalt configural cues on figure assignment (Peterson and Salvagio 2008). The 2 MTL cases were each given the test of the effects of familiar configuration on figure assignment 3 times, twice upright (Experiment 1) and once inverted (Experiment 2). These 3 test administrations were distributed over the course of approximately 12 months and were given in the following order: upright first test, inverted, and upright second test. The 2 HC cases were each given the test of the effects of familiar configuration on figure assignment once upright (Experiment 1). Before each administration of this test in Experiments 1 and 2, participants first completed the Gestalt configural cues test of convexity. This convexity test was administered first to introduce participants to figure–ground judgments. After viewing convexity displays, participants typically make figure judgments for displays like those in Figure 1a, immediately and confidently (e.g., Peterson et al. 2000). Given our directional hypotheses, all t-tests were one-tailed. Details of the tests are provided below.

Experiments 1 and 2: Gestalt Configural Cues (Convexity)

We assessed whether participants showed normal effects of convexity on figure assignment and normal “convexity concatenation effects” (i.e., a bias to see convex regions as a figure more than the concave regions, which increases with larger numbers of alternating convex and concave regions) (Peterson and Salvagio 2008). The displays were composed of either 2 or 8 alternating regions with convex or concave parts enclosed within a rectangular frame that cut the leftmost and rightmost regions in half (Fig. 1b,c). The leftmost region was convex in half of the displays and concave in the other half. Convex and concave regions were equal in area; hence, no known figural cue other than the convexity distinguished between them. Convex regions were black in half of the displays and white in the rest; concave regions were filled with the contrasting achromatic shade. The 2-region displays ranged from 3 to 5.4 cm in width and from 6.7 to 8.9 cm in height. The 8-region displays ranged from 14.8 to 21 cm in width and from 6.7 to 8.9 cm in height. These displays were superimposed on a 25.2 × 19 cm medium gray background so that the black and white regions contrasted equally with the background. All displays were presented on paper.

Before beginning the test, participants were instructed on the nature of figure–ground perception through a short demo. They were told that, for each display, either the black or the white regions would appear to be the figure in that they would appear to stand out as having a definite shape and seem closer than the other regions. By contrast, the regions of the other achromatic color would appear to form a background that seemed to continue behind the figures. For each display, participants were asked to report whether the black or the white regions appeared to stand out as figures. They were told that there are no correct or incorrect responses; that different people see the displays differently, and that we wanted to know their first impression of the displays. Participants had no trouble understanding these instructions. After this demo, they completed 8 practice trials followed by the actual test. The stimuli were shown one at a time and remained present until the response. All participants responded quickly and confidently. The test consisted of 28 unique 2-region displays and 28 unique 8-region displays, presented in a blocked order. Half the control participants viewed the 2-region displays first. Across the 3 testing sessions for the MTL patients, MTL2 received the 2-region display block first, twice and MTL3 received the 8-region display block first, twice. In their single testing session, the HC patients received the 2-region display blocks first.

Experiments 1 and 2: Test of Effects of Familiar Configuration on the Figure Assignment

A set of 48 displays was used to assess the effects of familiar configuration on the figure assignment (stimuli used in Peterson et al. 1998, 2000). Each stimulus comprised adjacent black and white regions of approximately equal area that shared a central border and were enclosed within a virtual rectangular frame ranging from 1.8 to 6.7 cm in width and from 6.1 to 7 cm in height. This was superimposed on a 25.2 × 19 cm medium gray background (Fig. 1a). The displays were presented on paper. They were presented upright for Experiment 1 and upside down for the inverted version of the test in Experiment 2.

For half the stimuli (Experimental Stimuli, n = 24), one of the 2 regions portrayed an “intact configuration” of a portion of a familiar object that was identified correctly by at least 65% of the pilot observers, when it was seen as a figure (Peterson et al. 2000). This critical region was assumed to provide a good match to a representation of a known object in the memory. For the remaining stimuli (Control Stimuli, n = 24), neither region of the display depicted a familiar object. One of the regions, however, was created by rearranging the component parts of the familiar object configurations depicted by the critical regions of the Experimental Stimuli. This was done by breaking the central border of the familiar object into parts delimited by 2 successive concave cusps. These parts were spatially rearranged (maintaining polarity) until the resulting part-rearranged region no longer depicted a familiar configuration (<22% agreement between the pilot observers on what object this region portrayed, Peterson et al. 2000). We assumed that these “part-rearranged critical regions” did not provide a good match to a representation of a known object in the memory. For both the Experimental and Control Stimuli, the region adjacent to the critical region did not depict a familiar object when seen as figure (<22% of pilot observers agreed on a single interpretation for any of these regions, Peterson et al. 2000).

Part-rearranged and intact familiar configurations occurred equally often in black and in white and on the left and right sides of the central border. These critical regions were equated on variables known to be relevant to figure–ground assignment (e.g., area and convexity). The crucial difference is that the critical regions (intact familiar configurations) in the Experimental Stimuli depicted real world objects that observers had encountered previously outside the laboratory and for which they had preexisting object representations, whereas the critical regions (part-rearranged novel configurations) in the Control Stimuli did not. Thus, this test provides a good assessment of contributions from familiar object configuration on figure assignment: Familiar configuration effects on figure assignment are implicated if the observers are more likely to see the intact familiar configurations as figure than the part-rearranged novel configuration (e.g., Peterson 1994).

The test of effects of familiar configuration on figure assignment was administered immediately following the Gestalt configural cues test. As with the Gestalt configural cues test, participants viewed the stimuli one at a time and reported whether the black or white region appeared to be the figure. Participants were asked to identify any familiar objects they saw, after they reported which region appeared to be the figure. The stimuli remained present until the response. The test began with 2 familiarization trials (one trial clearly portrayed a half silhouette of a cat and the other trial did not portray a familiar object). Participants had no trouble transferring the instructions they had followed on the Gestalt configural cues test to this test. Their responses were immediate and confident.

Experiment 3: Familiarity Discrimination

On each of the 21 trials, participants viewed a vertical arrangement of the intact configuration of a well-known (familiar) object and the corresponding part-rearranged version from the stimulus set used in Experiments 1 and 2 (Fig. 1d). In order to ensure that these critical regions were perceived as a figure, they were shown in black and placed at the far left of a rectangular frame (15.5 cm wide × 6 cm high), leaving considerable white space to the right. The remaining background on the paper was also white. Top/bottom location of the familiar configuration was balanced. Participants were instructed to indicate which of the 2 images was more familiar; they were not asked to name any of the stimuli. They responded quickly and confidently.

Results and Discussion

Experiment 1

We expected that control participants (both those without brain damage and those with HC damage) would show the standard effect and would perceive the critical regions portraying intact configurations of the parts of well-known/familiar objects as the figure, substantially more often than matched critical regions composed of the same parts arranged in novel configurations (Fig. 1a). If the PRC plays a role in discriminating between familiar and novel configurations, this “familiar minus novel configuration difference” should be reduced for individuals in the MTL group.

The percentages of critical regions seen as figures are displayed in Figure 2a (individual data for control participants are shown in Supplementary Table 4). Control participants without brain damage showed the standard effect of familiar configuration: they perceived regions portraying portions of familiar objects as a figure substantially and significantly more often than regions portraying novel configurations created by rearranging the parts of familiar objects (t₁₆ = 12.7, P < 0.001). The familiar minus novel configuration difference score is shown in Figure 2b. This difference score was significantly greater than zero for the control participants without brain damage. The control participants with damage limited to the HC demonstrated the same pattern. Crawford's Modified t-tests, a statistical method to compare a single case with a control sample (Crawford et al. 2009), showed that figure responses to both intact familiar configurations and part-rearranged novel configurations did not differ in the age-matched and HC control participants (ts₁₆ < 1, P > 0.2). The familiar minus novel configuration difference scores in the HC patients were not significantly different from the mean difference score of the non–brain-damaged controls (both ts₁₆ < −0.3, Ps > 0.21). In contrast, the MTL cases showed a different pattern of performance: As seen in Figure 2b, across all testing sessions, the familiar minus novel configuration difference score was significantly reduced for all administrations in the MTL cases compared with the non–brain-damaged controls (all ts₁₆ > 1.9, Ps < 0.05). The difference score was reduced because the MTL cases made “fewer” figure responses than the controls to the intact familiar configurations (all ts₁₆ between 1.6 and 5.7; Ps between 0.07 and 0.001) and because they showed a trend to make significantly “more” figure responses than the controls to the critical region, in the part-rearranged novel displays (all ts₁₆ = −1.5, Ps < 0.07, with the exception of MTL2 second: t₁₆ = −0.2, P = 0.41). Thus, it is important to note that it was the “combination” of decreased figure responses to the intact familiar configurations and increased figure responses to part-rearranged novel displays that reduced the familiar minus novel configuration difference scores in the MTL cases.

Figure 2.

Experiment 1: upright displays. (a) Percentage of figure reports made to the critical regions of the displays containing intact familiar configurations of objects and displays containing part-rearranged novel versions of these objects. (b) The familiar minus novel configuration difference score gives a measure of the effect of familiar configuration on figure assignment. **P < 0.01, *P < 0.05, +P < 0.07 for the comparison of MTL patients relative to controls. Error bars represent standard error of mean.

The finding that MTL patients, but not HC patients, failed to show effects of familiar configuration on the figure assignment indicates that the extrahippocampal MTL structures (such as the PRC) are critical in modulating the perceptual effects that depend upon the implicit discrimination of familiar versus novel configurations. As such, these structures can play a role in perception as well as in memory. A clue regarding the role of the PRC is provided by the finding that compared with control subjects, PRC-damaged individuals both were “less” likely to report critical regions portraying familiar configurations as figures and showed a trend to be more likely to report critical regions portraying familiar parts in unfamiliar (scrambled) configurations as figures. Based on the well-established evidence that the PRC plays a role in the novelty/familiarity detection (e.g., Xiang and Brown 1998; Henson et al. 2003; Kohler et al. 2005; Albasser et al. 2010; Burke et al. 2010; McTighe et al. 2010), these results led to the hypothesis that upon detecting a familiar configuration, an intact PRC facilitates familiarity responses at lower levels where the parts are represented (Fig. 3a). As a consequence, familiarity is signaled at both high and low levels of the visual hierarchy, and these familiarity signals serve as figural cues at multiple levels. In contrast, upon detecting a novel configuration, an intact PRC inhibits any familiarity responses that exist at lower levels where the object's parts are represented (Fig. 3b). Consequently, familiarity is not signaled at either low or high levels, and no effects of familiarity on figure assignment are observed, even though the parts of part-rearranged novel configurations are themselves familiar. When the PRC is damaged, however, its novelty/familiarity detection function is eliminated, and the figure assignment is based on the familiarity of the individual parts alone, which are neither facilitated when the configuration is familiar nor inhibited when the configuration is novel. The result is that figure responses are reduced for intact familiar configurations (due to the loss of facilitation at the part level; Fig. 3c) and elevated for part-rearranged novel configurations (due to the loss of inhibition at the part level; Fig. 3d). Consistent with this dynamic interaction hypothesis, it has been shown that familiarity is represented at both high levels of the visual stream where configurations are represented, and at lower levels where parts are represented (Baker et al. 2002). Moreover, feed-forward and feed-backward signals between PRC and area TE have been shown to support the representation of learned associations between stimulus patterns, with PRC representing the associations between elements and area TE representing the individual elements themselves (Higuchi and Miyashita 1996; Liu and Richmond 2000; Naya et al. 2001, 2003; Takeuchi et al. 2011). We test the dynamic interaction hypothesis in Experiments 2 and 3.

Figure 3.

Schematic illustration of proposed role for the PRC in coordinating high- and low-level responses to familiar and novel configurations. Individual letters represent parts of the objects and the combination of 4 letters represents the complete object configuration. Lower-case fs represent the familiarity response to object parts. (a,b) In the intact brain, the PRC detects whether a configuration represents a familiar or novel object. (a) When familiarity is detected, the PRC facilitates familiarity responses at these lower levels (facilitation shown by arrows and underlined bold fs). As a result, both high-level and low-level familiarity responses affect figure assignment when intact familiar configurations are present. Note that with an intact PRC, it is not possible to separate effects of familiar configurations from effects of familiar parts. (b) When novelty is detected, an intact PRC inhibits familiarity responses at lower levels of the visual system where object parts are represented (inhibition indicated by arrows and crossed out fs). As a result, neither high-level nor low-level familiarity responses affect figure assignment in the part-rearranged novel configurations. (c) By contrast, when the PRC is damaged, the intact configuration is not detected as familiar, and the familiarity of the individual parts is no longer enhanced. Without enhancement of the low-level part familiarity response from a high-level configuration response, subjects perceive the intact familiar configurations as figure significantly less than controls. (d) Similarly, when the PRC is damaged, the part-rearranged configuration is not detected as novel, and familiarity responses at lower-levels are no longer inhibited. As a result, effects of familiar parts are unmasked, and PRC-damaged individuals are likely to see the part-rearranged configurations as figure significantly more than controls.

Identification Accuracy

During the test assessing effects of familiar configuration on figure assignment, participants were asked to identify any well-known/familiar objects portrayed by the figures. Control participants accurately identified 84.6% of the intact familiar objects they saw as figures (Table 2). The MTL patients were severely impaired (identifications ranged from 28.6% to 62.5%; all ts₁₆ > 2.8, Ps < 0.005). By contrast, the HC cases performed normally (identifications were 78.0% and 90.9%; ts₁₆ < 0.8, Ps > 0.21). These data suggest that the PRC is critical for explicit object identification under impoverished conditions such as those in the present experiment, where portions of (rather than whole) familiar objects were presented with very few details. The familiar objects portrayed in the bipartite figure–ground displays here are drawn from a limited set in that they must have a vertical axis of elongation and they must be identifiable on the basis of the small number of parts depicted (3–6). Notably, the MTL patients performed normally on less demanding tests of object identification (Supplementary Table 1). This suggests that outputs from PRC are critical to object identification under conditions when only a small number of parts are shown in silhouette form and there is no single feature that clearly identifies the object. These results are consistent with the view that the representations in the PRC play a role in both declarative memory and perception.

Table 2.

Percentage of accurate identification of intact familiar configurations of objects seen as figures by controls and each patient (SDs in parentheses)

	Controls	HC2	HC3	MTL2 first	MTL2 second	MTL3 first	MTL3 second
Upright displays	84.6 (7.5)	78.0	90.9	40.9	31.7	28.6	62.5
Inverted displays	30.5 (15.8)	—	—	12.5	—	10.6	—

Note: Individual data for control participants are shown in Supplementary Tables 4 and 5.

The different test administrations for the MTL cases are shown separately. There were 2 test administrations of the upright test of effects of familiar configuration on figure assignment and one administration of the same test with inverted displays. The HC cases were not tested with inverted displays.

Experiment 2

To test the hypothesis that for MTL patients figure assignment was influenced by part familiarity rather than configuration familiarity, as predicted by the dynamic interaction hypothesis, we showed both the MTL cases and a separate set of controls (n = 13) the same test but with the stimuli turned upside down. This orientation manipulation reduces familiarity but does not alter the effectiveness of other figural cues such as protrusion and convexity (Peterson and Gibson 1994a). Effects of familiar configuration on the figure assignment are typically reduced substantially by the orientation inversion (e.g., Peterson et al. 1991; Gibson and Peterson 1994; Peterson and Gibson 1994a). On the assumption that part familiarity as well as configuration familiarity have been established on the basis of previous experience with upright objects (Baker et al. 2002), we hypothesized that orientation inversion should remove both part and configuration familiarity effects. Therefore, if the MTL patients' tendency to see the part-rearranged critical regions as figure in Experiment 1 was due to the effects of part familiarity on the figure assignment, then they should be less likely to perceive the part-rearranged critical regions as figures when they are inverted. In contrast, if part familiarity did not contribute to the control participants' figure reports for the part-rearranged critical regions, then they should be no less likely to report seeing the part-rearranged critical regions as figures when they are inverted. Finally, for both the MTL patients and the control participants, figure reports for the critical regions that portray familiar configurations should be reduced because inversion diminishes the familiarity of both parts and configurations.

Consistent with the previous evidence, the control subjects who viewed inverted displays in Experiment 2 reported perceiving the familiar configurations as a figure less often than the control subjects who viewed upright displays in Experiment 1 (t₂₈ = 8.81, P < 0.001; Fig. 4a). Notably, control subjects' figure responses to part-rearranged novel configurations were unaffected by the orientation change (t₂₈ = 0.33, P = 0.74), indicating that in the intact brain figure responses to part-rearranged novel configurations are not mediated by part familiarity. (Individual data for control participants are shown in the Supplementary Table 5.) Consistent with the hypothesis that part familiarity mediated figure assignment in the MTL patients in Experiment 1, inverting the displays (and thus reducing the familiarity of the parts themselves as well as the familiar configuration) reduced their figure responses to the part-rearranged novel critical regions as well to the critical regions portraying intact familiar configurations. As shown in Figure 4a, both MTL cases performed within the normal range for both the inverted familiar configurations and the inverted part-rearranged configurations; as a consequence, their familiar minus novel configuration difference scores shown in Figure 4b fell within the normal range (all ts₁₂ < 0.9, Ps > 0.20). By contrast, for upright displays in Experiment 1, MTL patients had perceived the intact familiar configuration as figure less often, and tended to perceive the part-rearranged novel configuration as figure more often, than the non–brain-damaged controls. The fact that the differences between the MTL patients and controls observed with upright displays in Experiment 1 were not observed with inverted displays in Experiment 2, supports the hypothesis that the differences observed in Experiment 1 were due to the part familiarity responses in lower levels of the visual hierarchy. In the intact brain, these lower levels of familiarity are functionally suppressed when higher levels detect novel configurations, and are supplemented when higher levels detect familiar configurations (Fig. 3).

Figure 4.

Experiment 2: inverted displays. (a) Percentage of figure reports made to the critical regions of the inverted displays containing intact familiar configurations of objects and inverted displays containing part-rearranged novel versions of these objects. For comparison, we also include the figure reports for the upright displays in Experiment 1 (first administration for the MTL cases). The arrows indicate that inverting the displays reduced the figure responses in the MTL cases, but not in the control participants. (b) The familiar minus novel configuration difference score gives a measure of the effect of object familiarity on figure assignment. **P < 0.01, *P < 0.05, +P < 0.07 for the comparison of MTL patients relative to the controls in the upright displays. No differences were found for inverted displays. Error bars represent standard error of mean.

Identification Accuracy

Not surprisingly, identification accuracy for the objects seen as figures was greatly reduced for the inverted displays in control participants, with 30.5% correctly identified for inverted compared with the 84.6% for upright (t₂₈ = 12.43, P < 0.001; see Table 2). Identification accuracy was also reduced in the MTL patients (an average of 11.6% [∼2 items] for the inverted compared with 40.9% [∼8.75 items] for upright). Identification of familiar configurations was no longer significantly lower in MTL-damaged individuals than in controls (ts₁₂ < 1.2, Ps > 0.14).

Experiment 1 and 2 Control Task

As a control task, participants in Experiments 1 and 2 completed a test examining whether their use of a more generic cue for figure assignment was intact (Fig. 1b,c). Table 3 shows responses to the generic configural cue of convexity. In all cases and across all test administrations, the MTL and HC patients did not perform significantly differently from controls (all ts < 1.3, Ps > 0.1) and none of the patients' scores fell more than 2 SDs below the control mean. Thus, figure assignment based on the generic cue of convexity is not impaired in these cases: all participants reported perceiving the regions with convex parts as the figures, significantly more often than expected, on the basis of chance. Convexity is known to operate locally (Stevens and Brooks 1988), and its effectiveness is unchanged by the order of convex parts along a border (Peterson and Salvagio 2008). Thus, the MTL patients are not impaired in the figure–ground perception per se but in effects attributable to stored representations of the configuration of well-known objects in particular. In addition, all subjects were more likely to perceive regions with convex parts as figures in the 8-region displays than in the 2-region displays. These “convexity concatenation effects” require global processing (Peterson and Salvagio 2008; Goldreich and Peterson Forthcoming ), and thus, the finding that the MTL patients showed normal convexity concatenation effects is consistent with the hypothesis that they are not impaired in all types of global processing, but specifically in processing the spatial configuration of the object parts.

Table 3.

Percentage of figure responses to the convex regions for controls and each patient (SDs in parentheses)

	Controls (n = 30)	HC2	HC3	MTL2	MTL3
Convexity (2 region displays)	77.5 (22.9)	100	64.3	83.3 (14.9)	69.0 (27.7)
Convexity (8 region displays)	94 (11.7)	100	92.9	94.0 (4.1)	100 (0)

Note: Individual data for control participants are shown in Supplementary Table 6.

The percentages for the MTL cases reflect the average score across their 3 test administrations.

Experiment 3

In a final experiment, we tested PRC-damaged participants' ability to discriminate explicitly between novel and familiar configurations composed of familiar parts by asking them to choose which was more familiar, a familiar configuration or its part-rearranged novel counterpart. We hypothesized that if the novelty/familiarity discrimination function of the PRC was impaired by brain damage, an explicit task that relies on this function should be impaired as well as the implicit task assessed in Experiment 1. On each trial, participants viewed a vertical arrangement of the 2 critical regions created from the same parts: the intact familiar configuration and its part-rearranged novel counterpart. Here, we were not assessing figure–ground perception; we were assessing explicit familiarity judgments. The stimuli used in Experiment 3 were designed so that the critical regions would be perceived as figures. The critical regions were always shown in black on the left side of an elongated rectangle enclosed by a thin black frame. The complement was always white and was 3 times the area of the critical region. Thus, a small area favored seeing the critical regions as figures. The backdrop on which the elongated rectangle lay was also white; thus, a contrast with the backdrop favored perceiving the black critical regions as figures. Subjects perceived the critical regions as figure and responded “top” or “bottom” to indicate which was the familiar object. If MTL patients base their responses on the familiarity of the parts, rather than on the familiarity of the configuration, they should find it difficult to discriminate between the part-rearranged novel configurations and the intact familiar configurations.

Control participants identified the intact familiar configuration as more familiar than the part-rearranged configuration on the majority of trials (mean of 87.1%; Fig. 5). Individual data for control participants are shown in Supplementary Table 7. Consistent with the hypothesis that the novelty/familiarity discrimination function of the PRC was impaired, MTL3 was unable to discriminate between the familiar and part-rearranged configurations and performed at chance (52%, which was significantly worse than the controls: t₉ = 3.8, P < 0.01). MTL2 identified the intact familiar configuration on 76% of the trials, which was not significantly different from the control performance (t₉ = 1.2, P = 0.13), but was at the low end of the control range (only one control performed below MTL2). Thus, at least for explicit judgments of familiarity, MTL2 seems to have some minimal access to familiar configurations.

Figure 5.

Experiment 3: familiarity discrimination. Percentage of trials on which the intact familiar configuration was identified as more familiar than the part-rearranged novel configuration. **P < 0.01 for the comparison of MTL patients relative to controls. Error bars represent standard error of mean.

General Discussion

Using a classic perceptual task where memories of familiar configurations affect the first perceived figure–ground organization, we found that the amnesic patients whose damage included the PRC, but not those with hippocampal damage that spared the PRC, failed to show standard effects of familiar configuration on figure assignment. Notably, this was not only because they reported seeing the critical region portraying a familiar object configuration as a figure less often than controls but also because they reported seeing a matched novel configuration in which the same familiar parts had been spatially rearranged as figure more often than controls. This pattern, evident in Experiment 1, suggested that the familiarity of the individual parts served as a figural cue in patients with PRC damage. Interestingly, part familiarity did not act as a figural cue in control participants, suggesting that the PRC damage unmasked effects of familiar parts that were not evident in the performance of the control subjects (Fig. 3). We tested this part familiarity hypothesis in Experiment 2 with inverted displays, a manipulation that reduced the familiarity of both the individual parts and the object configurations. Consistent with the part familiarity hypothesis, the PRC-damaged individuals now reported seeing both the critical regions portraying familiar configurations and those portraying part-rearranged novel configurations as a figure, less often than they did when viewing upright displays, whereas control participants figure reports were reduced only for the critical regions portraying familiar configurations. Critically, for inverted displays, the behavior of PRC-damaged individuals was indistinguishable from that of controls, supporting the view that part familiarity was responsible for the differences observed with upright displays. Taken together, the results of Experiments 1 and 2 indicate that the novelty/familiarity discrimination in the PRC modulates lower-level part familiarity responses. Therefore, the present results provide strong support for the hypothesis that the PRC plays a role in perception when configurations of parts are important for the performance. Moreover, these experiments reveal that the intact PRC is dynamically involved in the interactions between high- and low-level representations of familiarity, as others have proposed (Higuchi and Miyashita 1996; Naya et al. 2001, 2003; Takeuchi et al. 2011). In this case, the dynamic interactions reduce low-level part familiarity responses when novelty is detected at higher levels, and enhance them when familiarity is detected at higher levels.

Our results imply that familiarity is detected at multiple levels before figure assignment occurs. With an intact PRC, the familiarity signal detected at the highest level—where objects are represented globally and at low resolution—dominates the figure assignment. When the PRC is damaged, familiarity detected at lower levels of the system dominates the figure assignment. Thus, in addition to revealing that the PRC plays a role in the dynamically unfolding response to a familiar versus a novel configuration, the present results provide the first empirical evidence that figure assignment is a system-wide response. Other experiments show that figure–ground perception results from the inhibitory competition between potential objects that might be perceived on the opposite sides of a border (Peterson and Lampignano 2003; Peterson and Enns 2005; Peterson and Skow 2008). Such results, and the dynamic interaction model we have proposed for the PRC, are consistent with the physiological evidence and computational models of a figure assignment that involve the feedback from the higher to lower levels in the visual hierarchy (e.g., Kienker et al. 1986; Vecera and O'Reilly 2000; Jehee et al. 2007). The view, that the figure assignment represents a system-wide response, is far from traditional conceptions that it occurs at low levels and is impenetrable by the high-level processes such as memory (cf., Pylyshyn 1999).

In the present experiment, the particular instances of familiar objects portrayed in the figure–ground test have not been encountered before. As such, each new instance of an object category (e.g., a guitar and a table lamp) is categorized correctly at a basic level because it is similar in structure to previously seen category members, a process that likely requires conceptual processing (e.g., Tyler et al. 2004; Patterson et al. 2006). Is it possible that the absence of familiar configuration effects on figure assignment in the MTL patients is due to impaired declarative semantic memory for the well-known objects portrayed by our stimuli? It has been proposed that the PRC is critical for processing stimulus meaning and associations (Murray and Bussey 1999; Liu and Richmond 2000; Holdstock et al. 2009; Taylor et al. 2009; Barense, Rogers, et al. 2010; Wang et al. 2010; Barense et al. 2011). Consistent with this, the MTL patients in the present study showed deficits in conscious object recognition/semantic memory: They were impaired at explicit object identification of the upright familiar objects in Experiment 1 and at discriminating between familiar and novel configurations composed of the same parts in Experiment 3. Might their observed perceptual deficits stem entirely from this semantic memory impairment? Such an explanation would preserve the traditional view of the MTL as a declarative memory system (Squire and Wixted 2011). Although this traditional interpretation remains logically possible, we think it is unlikely for a number of reasons. First, substantial previous evidence (reviewed in the Introduction) indicates that explicit declarative object knowledge is neither necessary nor sufficient for effects of familiar configuration on figure assignment. Second, the MTL patients in the present study demonstrated effects of familiar parts on figure assignment (as evidenced by elevated figure responses to upright but not inverted part-rearranged configurations), but they did not show effects of familiar configurations (as evidenced by reduced figure responses to the upright intact familiar configurations). If the observed results stemmed from a primary deficit in semantic memory, one would expect semantic memory for object parts and object configurations to be affected equally, yet they were not. In contrast, the differential impact on the familiarity of parts versus the familiarity of their configurations is perfectly consistent with the idea that the PRC is an extension of the representational hierarchy within the ventral visual stream, where parts, and part familiarity, are represented at lower levels than the configurations and their familiarity (Baker et al. 2002). Under this alternative view, the PRC participates in “both” perception and memory when complex configural representations are required, but not when lower-level representations suffice (Bussey et al. 2002; Cowell et al. 2010). This alternative view parsimoniously explains the results of the present study.

Moreover, evidence from other experiments speaks to this issue. First, patients with PRC damage are more impaired on discrimination tasks involving novel unfamiliar objects (e.g., greebles, blobs, and barcodes) than they are on discrimination tasks involving familiar everyday objects (Barense et al. 2005; Barense et al. 2007; MacKay and James 2009). If the PRC's contribution to perception were merely via its role in explicit semantic memory, one would have expected the reverse pattern. Second, neuroimaging investigations in healthy controls have revealed above-baseline PRC activity for discriminations involving completely novel objects with which participants had no past experience, indicating that the human PRC plays a role in detecting novel displays (Barense, Henson,et al. 2010; Barense et al. 2011). Such observations are not consistent with the view that MTL damage results only in declarative object memory deficits but are consistent with the idea that the PRC plays a role in both perception and memory tasks that entail detection of familiar versus novel complex configurations.

Before closing, we want to emphasize that we certainly do not wish to argue that the PRC is not involved in memory—decades of data have demonstrated its essential role in old–new object recognition. How, then, can we reconcile the PRC's well-established role in memory with its more controversial role in object perception? Our view is that there is no clear neuroanatomical dividing line between memory and perception in the MTL: These 2 processes share common neural representations and computational mechanisms. The complex configural representations of objects processed and housed in the PRC are critical to both memory and complex object perception, and thus, PRC damage impairs performance in both cognitive domains. Although both memory and perceptual tasks may require common representations in PRC, what may differ across the 2 domains is the pattern of functional connectivity between the PRC and other brain regions based on the differing task demands. For example, a recent functional MRI study found comparable PRC activity on a forced-choice memory and a perceptual oddball task, but distinct patterns of functional connectivity between the PRC and other regions across the 2 tasks (O'Neil et al. 2011).

In summary, here for first time, we use a quintessentially perceptual task with no working memory component to assess human MTL contributions to object perception. Across 3 experiments, we provide evidence that the MTL (in particular the PRC) is critical for the representation of familiar object configurations. These findings suggest that brain structures traditionally considered to be high level, such as the PRC, can play an important role in the perceptual processes that were traditionally considered quite low level, such as figure–ground segmentation. More generally, these findings complement a growing body of evidence and suggest that the traditional approach of drawing sharp distinctions between perceptual and mnemonic functions may not be a useful approach to understanding the functional organization of the human brain.

Supplementary Material

Supplementary material can be found at: http://www.cercor.oxfordjournals.org/

Funding

Discovery Grant from the Natural Sciences and Engineering Research Council of Canada to M.D.B. and by a grant from the National Science Foundation (BCS-0960529) to M.A.P.