Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Jun 19.
Published in final edited form as: Neuron. 2013 Jun 19;78(6):1127–1137. doi: 10.1016/j.neuron.2013.04.018

Detecting changes in scenes: The hippocampus is critical for strength-based perception

Mariam Aly 1, Charan Ranganath 1,2, Andrew P Yonelinas 1,3
PMCID: PMC3694276  NIHMSID: NIHMS470241  PMID: 23791201

Summary

Recent findings have ignited a controversy over whether the hippocampus is critical for visual perception as well as memory. Some studies have shown that hippocampal damage impairs perception of scenes, but others found no evidence for hippocampal involvement. These studies measured perception as a unitary phenomenon, but recent findings indicate that perceptual discriminations can be based on two kinds of information: states of perceiving local differences, or global strength of relational match. In the current study, we separated state- and strength-based perception using a novel change detection paradigm with scenes. Patients with selective hippocampal damage exhibited significant reductions in strength-based perception but showed spared state-based responses. In a follow-up neuroimaging study, hippocampal activation linearly tracked confidence in strength-based perception, and was not differentially associated with state-based responses. The hippocampus therefore plays a selective role in perception, contributing high-resolution strength information possibly through its role in the representation of relational information.

The medial temporal lobe (MTL), including the hippocampus and parahippocampal gyrus, has long been known to be critical for long-term memory (Scoville and Milner, 1957). Patients with MTL damage have profound impairments on measures of long-term memory, while performing normally on neuropsychological tests of perception, skill learning, and other cognitive functions (Eichenbaum and Cohen, 2001). Such observations motivated the proposal that the MTL is a specialized memory system that is necessary for long-term declarative/episodic memory formation but is not required for normal perception, working memory, implicit memory, or skill learning (Baddeley and Warrington, 1970; Graf and Schacter, 1984; Squire and Zola-Morgan, 1991; Suzuki, 2009).

Recent research has challenged this view by demonstrating that selective hippocampal damage can impair high-level scene perception (Graham et al., 2010; Lee et al., 2005a,b; Lee et al., 2012; Warren et al., 2012) and that hippocampal activation in healthy adults is increased during the performance of challenging scene discrimination tasks (Barense et al., 2010; Lee and Rudebeck, 2010; Lee et al., 2010; Mundy et al., 2012). These findings have led to the proposal that the hippocampus is important for the representation of complex conjunctive (Graham et al., 2010; Lee et al., 2012; Saksida and Bussey, 2010) or relational (Cohen and Eichenbaum, 1993; Olsen et al., 2012) information, in the service of both visual perception and memory.

Reports of hippocampal involvement in perception are highly controversial because other similar studies have reported that hippocampal damage does not consistently impair visual perception (Hartley et al., 2007; Kim et al., 2011; Knutson et al., 2012; Shrager et al., 2006). The reason for these discrepancies is currently unknown (Baxter, 2009; Jeneson and Squire, 2012; Kim et al., 2011; Lee et al., 2012), but it has been suggested that a failure to show hippocampal involvement may occur if individuals rely on individual features to discriminate between stimuli (Baxter, 2009; Lee et al., 2012), thus bypassing the relational (Cohen and Eichenbaum, 1993) or complex conjunctive (Lee et al., 2012; Saksida and Bussey, 2010) processing demands that are critical for hippocampal involvement. Perhaps the most critical factor, however, is that all prior studies have included only a single-point measure of perception (e.g. percentage of correct visual discriminations). Such an approach is insufficient to fully characterize perceptual discrimination if performance can be based on different kinds of information (Aly and Yonelinas, 2012; Rensink, 2004).

Indeed, recent work has shown that visual perceptual decisions are supported by access to two qualitatively different kinds of information, each associated with different functional characteristics and subjective experiences (Aly and Yonelinas, 2012). For example, Aly and Yonelinas (2012) examined change detection with visual scene stimuli and collected response confidence judgments to perform a receiver operating characteristic (ROC) analysis (Green and Swets, 1966; Macmillan and Creelman, 2005). Analysis of the ROCs revealed that perceptual judgments reflected the combined and independent contributions of two kinds of perception: a discrete state in which individuals became consciously aware of specific details that differentiated two similar images, and assessments of the strength of relational match between pairs of images. State- and strength-based perception were functionally independent; state-based perception played a larger role when specific, local details differentiated pairs of images, while strength-based perception played a larger role when images differed in relational/configural information. These functional differences were accompanied by different subjective experiences; subjective reports of state-based perception were associated with access to local, specific details, whereas subjective reports of strength-based perception were associated with a general feeling of overall match/mismatch in the absence of identifying any specific detailed differences.

Thus, overall perceptual discrimination can be based on state-based access to local details, or assessments of the strength of relational match; but the role of the hippocampus in these different types of perception has never been examined. Recent theories have proposed that the hippocampus supports the formation of relational or complex conjunctive representations (Cohen and Eichenbaum, 1993; Lee et al., 2012; Norman and O’Reilly, 2003; Olsen et al., 2012; Saksida and Bussey, 2010); thus to the extent that strength-based perception reflects the relational or conjunctive match of two stimuli, the hippocampus should be critical for strength-based perceptual judgments. In addition, it has been argued that the hippocampus is not necessary for forming representations of single items (Diana et al., 2007; Eichenbaum et al., 1994; Lee et al., 2012); thus to the extent that state-based responses reflect the identification of individual objects that differ across scenes, the hippocampus should not be involved in state-based perceptual responses.

To determine the role of the hippocampus in perception, we conducted patient and neuroimaging studies of complex scene perception. We used scenes because previous work has suggested that patients with selective hippocampal damage or more extensive MTL damage show scene perception impairments, whereas face and object perception do not seem to be impaired in patients with selective hippocampal damage (Lee et al., 2005a,b). Given these findings, and the role of the hippocampus and parahippocampal cortex in spatial processing (Epstein and Kanwisher, 1998; Lee et al., 2008; O’Keefe and Nadel, 1978), we considered scenes to be the optimal stimulus to assess the contribution of the hippocampus and other MTL regions to state- and strength-based perception.

In the patient study, we tested 3 patients with bilateral hippocampal damage and 2 patients with more extensive unilateral MTL damage that included the hippocampus (Tables 1–2, Figure 1) on a perceptual discrimination task we used previously (Aly and Yonelinas, 2012). Individuals were presented with pairs of scenes that were either identical or differed, in that the scenes were slightly contracted or expanded relative to one another (Figure 2A). The manipulation was a ‘pinching’ or ‘spherizing’, which keeps the size of the scenes the same, but contracts (‘pinches’) or expands (‘spherizes’) each scene with the largest changes at the center and gradually decreasing changes towards the periphery. These changes alter the configural or relational information within the scenes (i.e. the relative distance or position between different components) without adding or removing any objects. Individuals can make perceptual judgments on these stimuli with either strength-based assessments of relational match, or state-based detection and identification of changes (Aly and Yonelinas, 2012). The identified changes that serve as the basis for state-based responses may be relatively local differences, such as the orientation or size of specific features or objects that are changed when the scene is expanded or contracted.

Table 1.

Patient characteristics and neuropsychological test scores.

Patient Damage Age Education WMS-R z-score (Ver/Vis/Gen/Att/Del) Doors and People %ile COWA z-score Letter/Category WAIS-R IQ
1 Bilateral hippocampus 30 16 −1.3/0.3/−0.87/0.2/−2.1 1 −2.0/−1.1 111
2 Bilateral hippocampus 60 12 −1.8/−0.3/−1.5/0.1/−2.2 n/a −1.1/−0.3 112
3 Presumed hippocampus 52 16 −2.5/2.0/−1.4/−0.2/−1.1 0.5 −0.2/0.2 96
4 R MTL 40 18 0.9/−0.9/0.1/1.2/−0.1 10 −0.4/0.7 106
5 L MTL 64 12 n/a n/a 1.7/−1.1 n/a
Open in a new tab

Table 2.

MTL gray matter volume measures. Corrected gray matter volume in the MTL for Patient 1, Patient 2, and five age-matched controls for each patient. Data shown are gray matter volume in each region divided by the total gray matter volume. Patient 1’s left and right hippocampal gray matter volumes were substantially reduced compared to controls (6 SD and 7.5 SD below normal, for the left and right hippocampus, respectively). No other MTL region for this patient showed significant volume reduction. Patient 2’s left and right hippocampal volumes were also reduced compared to controls (1.6 SD and 4 SD below normal, for the left and right hippocampus, respectively); the right hippocampus showed greater volume reduction than the left. No other MTL region showed significant volume reduction.

Hippocampus Parahippocampal Cortex Perirhinal Cortex Entorhinal Cortex
Left Right Left Right Left Right Left Right
Patient 1 0.0041 0.0035 0.0034 0.0042 0.0051 0.0036 0.0037 0.0024
Controls (n=5) 0.0053 0.0050 0.0041 0.0038 0.0037 0.0040 0.0016 0.0020
SD 0.0002 0.0002 0.0003 0.0003 0.0006 0.0005 0.0002 0.0003
Patient 2 0.0040 0.0027 0.0037 0.0037 0.0033 0.0048 0.0016 0.0018
Controls (n=5) 0.0046 0.0044 0.0038 0.0039 0.0027 0.0024 0.0012 0.0014
SD 0.0004 0.0004 0.0007 0.0010 0.0006 0.0009 0.0002 0.0007
Open in a new tab

Figure 1.

Figure 1

Open in a new tab

MRI scans for patients with selective hippocampal damage and a healthy control.

Figure 2.

Figure 2

Open in a new tab

Hippocampal damage impairs perceptual judgments based on strength-based relational match, but not those based on discrete states of identifying local differences. (A) Scenes were presented simultaneously for 1.5 seconds, then replaced by a 1–6 scale for a self-paced confidence judgment. The manipulations consisted of contracting or expanding the scenes, keeping the size of the images the same. This changes the configural/relational information within the scenes without adding or removing any objects. In the example shown here, the windows near the center of the building are closer together in the scene on the left than the scene on the right. (B) Aggregate ROCs suggest an impairment in strength-based perception (reduced curvilinearity) in the patients compared to controls, whereas state-based perception (upper x-intercept) is unaffected. (C) Parameter estimates confirm that patients are selectively impaired in strength-based perception; state-based perception is intact. Data for individual patients are overlaid on the patient average; filled shapes are hippocampal patients, open shapes are patients with left (circle) or right (diamond) MTL damage. State- and strength-based perception are on different scales (probability and d′, respectively). Error bars depict +/− 1 SEM.

On each trial, participants made same/different judgments using a 6-point confidence scale (sure/maybe/guess ‘different’ or ‘same’). Importantly, demands on long-term memory were minimized by presenting the scenes simultaneously, so memory across delays or learning over trials was not necessary in the task. The neuroimaging study used a similar paradigm (Figure S1A). Confidence data were used to plot ROCs to assess overall performance and separate the contributions of state- and strength-based perception.

Experiment 1: Patient Study

Results

Confidence-based ROCs were plotted for each individual, and aggregate ROCs are shown in Figure 2B (see Supplemental Table S1 for response times). Overall accuracy was computed for each participant by quantifying the area under the ROC curve (Macmillan and Creelman, 2005). Overall accuracy was significantly lower for patients [M=.67, SD=.08] than for controls [M=.75, SD=.06; t(13)=2.36, p=.02]. In order to characterize the nature of this impairment, we assessed the contribution of state- and strength-based perception to performance by examining the ROC parameters (Aly and Yonelinas, 2012; Yonelinas, 1994).

The degree of curvilinearity in the ROC provides an estimate of perception based on assessments of continuously-graded strength information, while the upper x-intercept reflects the probability of discrete, state-based perception (Aly and Yonelinas, 2012). The more the ROC curves away from the chance diagonal, the greater the estimate of strength-based perception; the further left the upper x-intercept is shifted, the greater the probability of state-based perception.

Visual examination of the aggregate ROCs (Figure 2B) shows that the patients’ ROCs are less curved than the controls’, whereas the upper x-intercepts of the groups are identical. This suggests that MTL damage selectively impairs the ability to make strength-based perceptual judgments, and judgments based on discrete states of perceiving specific differences are preserved. These observations from the aggregate ROCs were confirmed in the average parameter estimates (Figure 2C). A 2 (group: patient or control) x 2 (perception: state or strength) mixed-model analysis of variance revealed a significant main effect of group [F(1,13)=6.27, p=.026], a significant main effect of perception [F(1,13)=12.72, p=.003], and a significant group by perception interaction [F(1,13)=8.53, p=.012]. The interaction arose because strength-based perception was reduced by more than 50% in the patients compared to controls [.33 and .76, respectively], leading to a statistically significant impairment, [t(13)=3.24, p=.003]. In contrast, there was no difference in estimates of state-based perception between patients and controls, t<1.

These results held for patients with selective hippocampal damage as well as patients with more extensive MTL damage (filled and open shapes in Figure 2C, respectively). Both the hippocampal patient group [M=.45, SD=.08] and the larger MTL lesion group [M=.14, SD=.20] had reduced estimates of strength-based perception compared to controls [M=.76, SD=.26]. This reduction was statistically significant for both the hippocampal lesion patients [t(11)=1.98, p=.04] and the MTL lesion patients [t(10)=3.16, p=.005]. The MTL group tended to perform more poorly than the hippocampal lesion patients. The impairment in strength-based perception for patients with selective hippocampal lesions suggests that the hippocampus itself plays a necessary role in graded perceptual responses.

A critical aspect of the current data is that patients and controls did not differ in performance at very conservative or very lax response criteria (left- and right-most ends of the ROCs). Thus, if only binary same/different judgments were collected, the results could have varied from no significant impairment [p=.23 at the leftmost point on the ROC] to a statistically significant impairment [p=.02 at the ROC midpoint]. An examination of performance across a range of confidence, and the different kinds of perception that underlie that performance, is therefore necessary to reveal and characterize the nature of the perceptual impairment. Importantly, even without interpreting the data in terms of state- and strength-based perception, this multi-point approach to characterizing performance shows that MTL patients exhibit a selective deficit in just one type of perceptual judgment; lower-confidence, but not high-confidence, responses are less accurate in the patients. We include additional analyses of the ROCs in Supplemental Information.

It is worth noting that, although one of the MTL patients had a 0 estimate of state-based perception, one control also had an estimate of 0. Likewise, 3 controls performed similarly to the lowest-performing hippocampal patient. Thus, there was no indication that the patients exhibited lower state-based estimates than controls; patients’ performance on state-based perception was within the control range. The fact that one patient and one control had state-based estimates of 0 might suggest that the lack of a patient deficit in state-based responding could be related to floor effects. However, both the patient and control groups produced average estimates of state-based perception that were significantly above 0 (p<.02 for both groups), and in both groups state-based perception for individual participants reached as high or higher than 40% of ‘different’ trials. Moreover, using the same paradigm, experimental manipulations have lead to significant reductions in state-based perception below the levels observed here (Aly and Yonelinas, 2012), indicating that state-based estimates were not constrained by floor effects. Finally, patients were numerically higher than controls on estimates of state-based perception, so removing the lowest-performing controls would only bring controls’ performance closer to that of the patients.

One possible concern is that the patients might have used the confidence scale in the perception test a different way than the controls did because the patients’ memory impairments have led them to be less certain about their cognitive abilities. An examination of Figure 2B, however, shows that the ROC points varied across a comparable range in the patients and controls, suggesting that the response criteria did not different substantially across the groups. More importantly, an advantage of ROC analysis is that differences in response criteria affect only the criteria parameters and not estimates of state- and strength-based perception. So even if there were differences in response criteria between groups, it would not be expected to affect estimates of perceptual sensitivity. Finally, group differences in response criteria could not explain the results of the fMRI experiment (Experiment 2), in which no patients were examined.

Because the hippocampus was the only structure that was damaged in all the patients, these data suggest that the hippocampus itself plays a necessary and selective role in scene perception based on the strength of relational match. Conversely, perceptual judgments based on discrete states of identifying specific, local differences in scenes do not seem to depend on the hippocampus or the MTL.

We next conducted an fMRI study with healthy individuals to provide a second test of the hypothesis that the hippocampus is involved in strength-based, but not state-based, perception. The paradigm was similar to that used in Experiment 1 (Figure S1A), with two differences. First, we used sequential rather than simultaneous stimulus presentations, to reduce excessive eye movements that may impact the BOLD response (Kimmig et al., 2001). Second, the confidence scale was changed in order to enable us to directly assess the role of the hippocampus in state- versus strength-based perception. Rather than the highest-confidence ‘different’ responses being ‘sure different’, as in the patient study, the highest-confidence ‘different’ responses in the fMRI study were reserved for trials in which individuals experienced a state of being consciously aware of specific details that were different between the images and could, if asked, report those differences. In our previous work (Aly and Yonelinas, 2012), when individuals made these ‘perceive different’ responses, they were highly accurate at reporting the specific details that had changed. In this way, we could directly examine how the hippocampus is modulated by varying levels of strength-based perception (response confidence ‘1’ to ‘5’), as compared to the discrete state of identifying specific details (perceive different, ‘6’ responses).

Experiment 2: fMRI Study

Results

Behavioral data are shown in Figure S1B–C. The pattern of results was consistent with previous studies (Aly and Yonelinas, 2012) and the patient study, with performance based on a combination of strength- and state-based perception.

To identify MTL subregions that contributed to change detection judgments on scenes in this task, we first tested for regions that showed greater activity for correct ‘different’ trials (i.e. 4, 5, and 6 responses on ‘different’ trials) than correct ‘same’ trials (i.e. 1, 2, and 3 responses on ‘same’ trials). This contrast revealed activation in 3 MTL regions: left posterior hippocampus (Figure 3A), and bilateral parahippocampal cortex (PHc; Figure S2A). Given that these MTL regions showed activity patterns consistent with successful change detection (Figure 3B; Figure S2B), we next sought to break down whether this effect was indicative of a role in state-based perception, strength-based perception, or neither. That is, greater activation on correct ‘different’ than correct ‘same’ trials might emerge if (1) a region shows elevated activity for ‘6’ judgments, in which individuals have access to specific details, and is not modulated by the strength of overall match (‘1’ to ‘5’ responses); (2) a region monotonically tracks the strength of overall match (an increase in activation from ‘1’ to ‘5’), but is not disproportionately active for judgments based on access to specific details (‘6’s); or (3) a region does not show response characteristics of either state-based or strength-based perception, but responds with greater activation for all ‘different’ judgments (‘4’–‘6’) than all ‘same’ judgments (‘1’–‘3’). Given that patients with hippocampal damage were impaired at making judgments based on continuously-graded strength information, we predicted that the hippocampus would show pattern (2), indicative of supporting strength-based perceptual judgments. The patient data do not specifically implicate the PHc as necessary for strength-based perception, so we did not have strong predictions about which pattern the PHc would show.

Figure 3.

Figure 3

Open in a new tab

The hippocampus continuously tracks the strength of perception. The left posterior hippocampus (A) was more active on correct ‘different’ than correct ‘same’ scene trials (B). Activation tracked the strength of evidence, with increasing activation with increased confidence in difference (C). MNI coordinates for peak voxel : −15, −34, 3. Error bars depict +/− 1 SEM. Also see Figure S2.

The regions of interest (ROIs) for this analysis were the 3 clusters of activation in the MTL from the correct ‘different’ greater than correct ‘same’ contrast (i.e. left posterior hippocampus and bilateral PHc). For these ROIs, we extracted parameter estimates indexing activation associated with each confidence bin versus the baseline null trials, and then tested whether the average slope of this line, across participants, was significantly greater than zero (Figure 3C; Figure S2C). As predicted, the hippocampus showed increasing activation with greater confidence, suggesting an involvement in perceptual judgments based on continuously-graded strength information [linear trend: t(17)=3.15, p=.006]. There was no evidence that this region was particularly sensitive to perceptual judgments based on identifying specific, local differences, as no significant difference in activation was observed between ‘5’ and ‘6’ responses, t(17)<1. The left and right PHc also showed increasing activation with increased confidence [linear trends: t(17)=3.46, p=.003 and t(17)=3.39, p=.003, respectively]. These fMRI data therefore converge with the patient data, suggesting that the hippocampus provides strength-based signals in support of scene perception, and in addition show that the PHc may also play a role in strength-based perceptual judgments.

To ensure that the statistically significant linear trend for the hippocampus was the most adequate fit of the data, we examined whether the alternative possibility, that of state-based effects in the hippocampus, could fit the data as well. A state-based effect would manifest as a ‘hockey stick’ function, i.e. a flat line from confidence responses 1 to 5 and then a disproportionate increase to response 6. We used log-likelihood to estimate the best-fitting flat line (no slope), linear trend, hockey stick function, and combination of linear trend + hockey stick. We then compared the goodness of fit of these functions to the data using hierarchical likelihood ratio tests.

Compared to the best-fitting flat line (no slope), the linear trend provided a significantly better fit to the data (χ2= 4.55, p=.03). In contrast, compared to the flat line, the hockey stick function did not provide a better fit (χ2= 2.16, p=.14). Comparison of the Akaike information criterion values for the linear trend and hockey stick function provided moderate evidence in favor of the linear trend (AIC difference = 2.39; Burnham and Anderson, 2002), and adding a hockey stick function to the linear trend did not significantly improve the fit of the latter, as might be expected if this region showed state-based effects (χ2=0.14, p=.71). These data therefore provide evidence that activation in the left posterior hippocampus is well-described as linearly tracking strength-based perception.

Because the MTL ROIs were identified by contrasting correct ‘different’ and ‘same’ trials, it is important to determine whether the linear trend across confidence is a result of collapsing across ‘same’ and ‘different’ trials when extracting parameter estimates for each ROI (i.e. since ‘different’ trials contribute a declining number of trials to each confidence bin as confidence decreases). We therefore extracted parameter estimates for the ROIs for only the ‘different’ trials, restricting the analysis to response bins for which there were enough trials to reliably extract parameter estimates (i.e. confidence responses 4, 5, and 6). A role in strength-based perception would be supported by increased activity from ‘4’ to ‘5’ responses, with no additional increase for state-based judgments based on access to specific details (i.e. ‘6’s). In contrast, a role in state-based perception would be evident by higher activation for ‘6’ responses than ‘5’ and ‘4’ responses, which should not differ from one another.

For all 3 ROIs, activation for the ‘6’ responses was not different from the ‘5’ responses [t(17)=.03, p=.97, t(17)=1.07, p=.30, and t(17)=1.31, p=.21 for the hippocampus and left and right PHc, respectively], but activation for the ‘5’ responses was significantly greater than for the ‘4’ responses [t(17)=2.19, p=.04, t(17)=2.19, p=.04, and t(17)=2.14, p=.05 for the hippocampus, and left and right PHc, respectively]. Thus, even when the analysis was restricted to ‘different’ trials, MTL activity was more consistent with a graded strength signal than a state-related change.

Our next analysis turned to the question of how hippocampal output might influence scene perception. Previous work has described a network of occipito-temporal areas that contribute to scene perception, including the lingual gyri (Aguirre et al., 1998; Menon et al., 2000) and the lateral occipital complex (LOC; Malach et al., 1995; Park et al., 2011). We therefore conducted a psychophysiological interaction (PPI) analysis in order to determine whether the hippocampus contributes to detection of scene changes through functional interactions with occipito-temporal visual areas. The seed region for the PPI analysis was the left posterior hippocampus ROI from the preceding analyses, and ROIs for the left and right lingual gyrus and the left and right LOC were selected by identifying voxels in these regions that showed greater activation for scenes than for faces.

For both the left and right lingual gyrus and LOC, functional connectivity with the posterior hippocampus increased with increasing perceptual decision confidence [left lingual gyrus, t(17)=1.60, p=.06; right lingual gyrus, t(17)=2.05, p=.03; left LOC, t(17)=1.74, p=.05; right LOC, t(17)=1.89, p=.04]. These results are similar to findings that the posterior hippocampus exerts top-down modulation of visual cortical areas in a task that involves constructing and maintaining scene representations over a brief amount of time (Chadwick et al., 2012). The current findings suggest that the hippocampus forms a network with visual scene processing regions in the service of assessing the strength of perceptual match/mismatch.

General Discussion

The current study yielded converging patient and neuroimaging evidence in support of a role for the hippocampus in visual scene perception (Lee et al., 2012). Furthermore, the results implicate the hippocampus specifically in strength-based perceptual discriminations, but not in state-based perception. Patients with hippocampal damage, including those with focal hippocampal lesions, were selectively impaired at making perceptual judgments based on continuously-graded strength information, and hippocampal activity varied in a graded manner with perceptual decision confidence. Our findings potentially reconcile the controversy about MTL involvement in perception by suggesting that the hippocampus may be specifically necessary for one kind of perceptual judgment – perception based on the strength of relational match. Indeed, our data demonstrate that if only binary same/different judgments were collected, the presence or absence of a deficit in patients would depend on the response criteria used by participants.

Relation to previous work on perception and the MTL

According to some theories, the hippocampus is necessary for complex spatial perceptual decisions in which conjunctions of features, rather than individual features, are diagnostic for task performance (Lee et al., 2012; Saksida and Bussey, 2010). Other theoretical frameworks stress the role of the hippocampus in spatial processing in general (Burgess et al., 2002; O’Keefe and Nadel, 1978), a role that could be extended to perceptual judgments on scenes. It has also been argued that the hippocampus is necessary in perceptual tasks that require binding of information (Warren et al., 2012). These ideas have been challenged by studies failing to find scene perception impairments in patients with hippocampal damage (Hartley et al., 2007; Kim et al., 2011; Shrager et al., 2006). The current study suggests that the distinction between state- and strength- based perception can help to reconcile the conflict in the literature.

In previous studies (Aly and Yonelinas, 2012), we found strong evidence that strength-based perception is affected by manipulations of global featural relationships, whereas state-based perception is disproportionately driven by detection of relatively local, item-level differences. For example, when the only difference between a pair of scenes was a specific feature (e.g. a window in one scene that is absent in the other), perceptual decisions were based primarily on state-based perception. In contrast, when the featural relations within the scenes differed from one another, performance relied more heavily on strength-based perception. Moreover, individuals reported identifying specific, local details when responses were state-based, and generalized feelings of overall difference/sameness when responses were strength-based. In the current fMRI study, the hippocampus and parahippocampal cortex were sensitive to strength-based perception, but, importantly, we also found that other regions of the brain were sensitive to state-based perception. For example, the posterior parietal cortex exhibited state-based, but not strength-based, effects (Aly, Ranganath, & Yonelinas, in prep).

Viewed in the context of our previous studies, the present results suggest significant constraints on when and how the hippocampus would be expected to contribute to perception. We propose that the hippocampus is involved in perceptual discriminations that require a representation of relational or conjunctive information. Not only did the hippocampus track the perceived ‘strength’ of perceptual change, the more basic finding of hippocampal adaptation (greater activation for ‘different’ than ‘same’ trials) suggests the hippocampus forms precise representations of visual scenes. The differences we introduced were subtle – on a given trial, the paired scenes are essentially identical with very small distortions. Thus, finding hippocampal adaptation for such small visual differences provides further evidence that the hippocampus represents precise relational information (Bakker et al., 2008; Lacy et al., 2010).

Because state-based perception plays a larger role in performance when perceptual manipulations involve discrete features (e.g. adding or removing specific objects; Aly and Yonelinas, 2012), we expect that hippocampal lesions would be associated with a reduced impairment for detecting discrete featural changes compared to relational changes. This argument is consistent with results implicating the hippocampus in relational long-term memory. For example, hippocampal lesions impair eye movements to relational changes in scenes (Ryan et al., 2000), and patients with hippocampal lesions fail to form an extended relational and/or spatial representation of scenes beyond the boundaries of the studied image (Mullally et al., 2012). Thus, the hippocampus may play a general role in relational processing (Cohen and Eichenbaum, 1993) in both perception and memory.

Finally, this work is consistent with the proposal that the hippocampus is critical for perceptual discriminations that involve spatial feature ambiguity; that is, discriminations that require the representation of complex conjunctions of spatial features (Graham et al., 2010; Lee et al., 2012; Saksida and Bussey, 2010). Further work will be necessary to determine whether the role of the hippocampus in strength-based perceptual judgments is specific to discriminations of spatial relationships in scenes or if it also extends to complex, feature ambiguous object discriminations.

Challenges to the proposed role of the MTL in perception

It has been argued that deficits on perceptual tasks in patients with hippocampal/MTL damage are a result of impairments in long-term memory and not perception (Kim et al., 2011; Knutson et al., 2012; Suzuki, 2009; see Graham et al., 2010 and Lee et al., 2012). That is, if healthy controls benefit from long-term memory on a perceptual task, impairments for patients may be the result of the patients’ failure to benefit to a similar extent. This can occur if some components of the stimuli are repeated across trials, so that controls can benefit from long-term memory representations of those stimuli and improve over the course of the task (Kim et al., 2011). Additionally, in tasks with multiple scenes to be compared, long-term memory may allow one to hold on to a representation of one item while examining others (Knutson et al., 2012; Lee et al., 2005a).

These arguments are difficult to reconcile with the current data. The stimuli were trial-unique, so long-term memory for particular stimulus components would not be beneficial. Furthermore, if a long-term memory deficit was the driving force for impairment on the perceptual task, it is not clear why one kind of perceptual judgment would be affected (i.e. strength-based perception) but not the other (i.e. state-based perception). The selective impairment in only one aspect of perception argues against a more general deficit in long-term memory leading to impaired performance. In order to account for the current data with a post-hoc memory explanation, it would be necessary to argue that state-based perception truly depends on perceptual mechanisms, while strength-based perception depends on long-term memory. The more parsimonious explanation is that only some types of perception are impaired following hippocampal and/or MTL damage.

Previous studies investigating the role of the MTL in perception have been criticized for using patients with extensive lesions that encroach on the ventral visual stream (Suzuki, 2006). We agree that it is important to rule out that perceptual impairments are a result of damage to these visual areas as opposed to the MTL. In the current study, 2 patients had verified selective hippocampal damage, whereas a third was unlikely to have damage outside of the hippocampus given his etiology (Gadian et al., 2000; Hopkins et al., 1995; Kono et al., 1983; Rempel-Clower et al., 1996; Smith et al., 1984); and these patients showed deficits in strength-based perception. In addition, the neuroimaging results obtained from young, healthy participants, converged in revealing a role of the hippocampus in strength-based perceptual judgments.

State and strength information in perception and long-term memory

The finding of hippocampal involvement in strength-based perceptual judgments in the current task is seemingly at odds with a number of studies of long-term memory, which generally suggest that the hippocampus supports memory decisions based on discrete states (Eichenbaum et al., 2007). That is, previous studies have shown that recollection generally is state-based in the sense that recollection occurs for some items and fails entirely for others (e.g. Harlow and Donaldson, 2013; Parks and Yonelinas, 2009), whereas familiarity usually is manifest as graded and strength-based. In typical recognition memory studies, many patients with hippocampal damage show severe recollection impairments and intact familiarity (Yonelinas et al., 2002; Yonelinas et al., 2010). Neuroimaging studies have also reliably shown that hippocampal activity during encoding (Ranganath et al., 2003) and retrieval (Montaldi et al., 2006; Yonelinas et al., 2005) is tightly linked to state-based recollection, and is generally not related to strength-based familiarity.

There are, however, some situations in which recollection shows strength-based, rather than state-based, response characteristics in long-term memory. For example, when materials have a high degree of feature overlap or complexity (Elfman et al., 2008; Parks et al., 2011), recollection becomes more graded or strength-based, like the strength-based signals seen in the current perception experiments (also see Harlow and Donaldson, 2013).

Importantly, computational modeling work indicates that manipulations that affect the dynamics of recollection have parallel effects on hippocampal output. For instance, in models of typical recognition memory tests, hippocampal output is threshold-like (i.e. state-based), such that some studied items elicit a large hippocampal response and the rest elicit small responses. Under conditions of high feature overlap, however, hippocampal output becomes more continuous or strength-based (Elfman et al., 2008, Norman and O’Reilly, 2003).

The graded long-term memory signal from the hippocampus may be related to the role of CA1 as a “comparator” that compares current sensory input to recent experience, and outputs a graded mismatch signal (Duncan et al., 2012; Vinogradova, 2001; Yassa and Stark, 2011). The hippocampus may play a similar role in perception, tracking the strength of relational match/mismatch.

These findings suggest that the hippocampus does not generally produce a state-based signal in long-term memory, but may produce state- or strength-based signals depending on the nature of the materials and demands of the task. In the current perception study, we found a linearly graded signal from the hippocampus, which may be a result of complex, feature-ambiguous materials and/or a graded comparison process. The critical point is that it is necessary to assess state- and strength-based memory and perception to elucidate the role of the hippocampus in these cognitive domains. Further studies examining the conditions in which the hippocampus produces state-based or strength-based output will be important.

Conclusions

The current neuroimaging and patient findings converge to indicate that the hippocampus is involved in, and is necessary for, perceptual judgments of scenes, and this role is specific to perceptual judgments based on continuously-graded strength signals. Scene perception based on discrete states of identifying specific differences does not seem to depend on the hippocampus. The findings highlight the surprising reach of the hippocampus, affording precision in both memory and perception.

Experimental Procedures

Experiment 1

Participants

Mean age of the patients was 49.2 years (SD=14.1) and mean education was 14.8 years (SD=2.7). Mean age of the controls was 47.7 years (SD=15.6) and mean education was 15.2 years (SD=2.0). Patients and controls were not significantly different in age or education (ts<1). Each patient had 1–3 controls that were closely matched to the patient’s age and education.

Patients

Patient characteristics and neuropsychological scores are shown in Table 1. Patient 1 had selective hippocampal damage following a traumatic brain injury due to a car accident. Clinical scans appeared normal with the exception of volume reductions in the hippocampus. Table 2 provides estimates of gray matter volume for MTL structures for this patient and age-matched controls. The left and right hippocampus were significantly reduced in volume for the patient compared to controls; no other MTL structure showed significant volume reduction (Figure 1).

Patient 2 had limbic encephalitis, and MRI scans suggested damage to the hippocampus bilaterally, with no damage apparent in the surrounding parahippocampal gyrus (Figure 1). Table 2 provides estimates of gray matter volume for this patient and 5 age-matched controls. The left and right hippocampus were reduced in volume, and no other MTL structure showed significant volume reduction.

Patient 3 suffered a mild hypoxic episode as a result of a cardiac arrest, and has presumed selective hippocampal damage (Gadian et al., 2000; Hopkins et al., 1995; Kono et al., 1983; Rempel-Clower et al., 1996; Smith et al.,1984). This patient has a defibrillator and is thus unable to undergo structural MRI scanning to confirm the extent and selectivity of the damage.

Patient 4 had viral encephalitis and as a result has extensive volume loss and encephalomalacia in the right temporal lobe, right hippocampus/MTL, and right orbitofrontal cortex. He was assessed by a neurologist at the University of California, Davis Medical Center. The extent of damage was determined from the patient’s MRI scan.

Patient 5 had a craniotomy in the left temporal region to remove an astrocytoma and an arachnoid cyst. The surgery was a standard left anterior temporal lobe resection, in which approximately 4 cm of the anterior temporal lobe, including the anterior third of the hippocampus and the amygdala, were removed. The rest of the brain appeared normal on a clinical MRI scan. These assessments were made by neurologists at the Veteran’s Affairs clinic in Martinez, CA.

Controls

None of the controls (n=10) had any history of neurological or psychological disorders and all performed normally on neuropsychological tests.

Patients and controls were not included in the study if they had a history of drug use or evidence of gross visual problems despite corrective glasses.

Estimation of gray matter volume

To determine total gray matter volume for each individual, each individual’s high-resolution structural scan was segmented into gray matter, white matter, and CSF. The spm_read_vols function (SPM8; http://www.fil.ion.ucl.ac.uk/spm/software/spm8) was then used to determine the number of voxels in the segmented gray matter.

In order to determine gray matter volume in each MTL subregion, each subregion was manually traced on individual native-space MPRAGE images. The subregions were delineated using criteria outlined by Insausti et al. (1998), Duvernoy and Bourgouin (1998), and Zeineh et al. (2001). The number of voxels in the masks for each subregion was determined using the spm_read_vols function.

Stimuli, Design, and Procedure

The stimuli, design, and procedure were identical to a study conducted with young adults (Aly and Yonelinas, 2012; Expt 2C). 160 colored photographs of scenes served as experimental stimuli; an additional 4 were used for practice. For each stimulus, 2 altered versions were created in Adobe Photoshop. The first was expanded outward slightly using the ‘spherize’ option and the second was contracted inward slightly using the ‘pinch’ option.

The experiment consisted of 1 block of 160 trials. 80 trials were ‘same’ trials in which identical stimuli were presented (i.e. 2 of the same ‘pinched’ or ‘spherized’ stimulus, with these trial types occurring equally often). 80 trials were ‘different’ trials in which the 2 altered versions of the stimulus were presented (i.e. ‘pinched’ followed by ‘spherized’ or vice versa, with these trial types occurring equally often). Pinched and spherized stimuli occurred equally frequently as the left and right stimuli. 2 stimulus lists were created so that each stimulus was tested on both ‘same’ and ‘different’ trials across participants. Same and different trials were presented in a random order.

On each trial, participants viewed a ‘get ready’ screen for 1.5 s, followed by two (same or different) scenes presented to the left and right of a fixation cross for 1.5 s (Figure 2A). The scenes were then replaced with a 1–6 confidence scale for a self-paced judgment: 1=sure different, 2=maybe different, 3=guess different, 4=guess same, 5=maybe same, 6=sure same. The numbers and verbal descriptions were presented until the participant made a response.

Before the experiment, participants viewed 3 pairs of sample scenes, which had been altered using the same distortions used for the experimental stimuli. Participants looked through the images to observe the types of changes to expect in the experiment. They also completed a 4-trial practice block.

Data Analysis

Performance was assessed by plotting confidence-based ROCs (Green and Swets, 1966; Macmillan and Creelman, 2005). For each participant, ROCs were fit using maximum likelihood estimation to obtain parameter estimates of state- and strength–based perception (Aly and Yonelinas, 2012; Yonelinas, 1994). One-tailed t-tests were used to compare parameter estimates of state- and strength-based perception for patients and controls because it was predicted that any difference would be in the direction of an impairment for the patients.

Experiment 2

Participants

18 healthy individuals (9 male) participated. Mean age was 27 years (SD=4.4) and mean education was 17.2 years (SD=2.3).

Behavioral Paradigm

Stimuli, Design, and Procedure

The stimuli and behavioral paradigm were modified from a previous study (Aly and Yonelinas, 2012). Stimuli were grayscale versions of the scenes used in Experiment 1 (and 80 additional scenes modified in the same way) and grayscale faces. For consistency with Expt 1, which incorporated only scene stimuli, we focused fMRI analyses on scene discrimination trials.

Stimuli were projected on a screen viewed on a mirror attached to the head coil. Each trial consisted of a 1s presentation of the first image, then a dynamic 50ms noise mask, then the corresponding ‘same’ or ‘different’ image for 1 s (Figure S1A). This was followed by a fixation screen for 1.95s. The scale was shown on the screen while the second image was presented, and then removed.

Individuals responded with a confidence judgment either while the second image was on the screen or during the fixation period following. Judgments were made with a modified version of the confidence scale used with the patients, and based on a scale used previously (Aly and Yonelinas, 2012). Participants were told to respond with a ‘6’ judgment only if they experienced a mental state in which they were able to provide specific, qualitative details about how the two images differed. If they thought the images were different but were not able to provide such details, they were told to respond with a ‘5’ (maybe different). A ‘4’ indicated ‘guess different’, ‘3’ was ‘guess same’, ‘2’ was ‘maybe same’, and ‘1’ was ‘sure same’. Participants made confidence responses with two button boxes. All participants used the left hand for ‘same’ responses (1–3) and the right hand for ‘different’ responses (4–6).

The experiment was divided into 8 runs of 90 trials each. Each run consisted of 30 face trials (half different), 30 scene trials (half different), and 30 null trials. Null trials were 2s presentations of the fixation cross. The duration of null events ranged from 2–10s (M=3s, SD=1.5s). Each run began with 10s of fixation to allow time for signal normalization and ended with 12s of fixation to allow time for the response to the final trial to be collected.

Order of trial types was optimized using optseq2 (Dale, 1999; http://surfer.nmr.mgh.harvard.edu/optseq/). 8 trial sequences were assigned to each of the 8 runs to form 8 different orders, so that each sequence was used in each run across participants. Each of these 8 orders was run in 2 counterbalancing conditions, allowing each item to be tested as both ‘same’ and ‘different’ for different participants.

Before the experiment, participants looked at practice images (as in the patient study), and did a short practice phase while in the scanner (not scanned).

fMRI Acquisition and Preprocessing

Participants were scanned at the UC Davis MRI Facility for Integrative Neuroscience. fMRI data were collected on a 3T Siemens Skyra scanner with a 32-channel head coil. Functional images were obtained with a gradient-echo EPI sequence (TR=2000 ms, TE=25 ms, flip angle=90 degrees, FoV=205 mm, voxel size=3.2 mm isotropic). Each functional volume consisted of 34 slices oriented parallel to the AC-PC line, and acquired in an interleaved sequence. Co-planar high-resolution (1.0mm isotropic) T1-weighted structural images were acquired for each participant using an MPRAGE sequence.

All pre-processing and data analysis were conducted using Statistical Parametric Mapping software (SPM8; http://www.fil.ion.ucl.ac.uk/spm/software/spm8). Pre-processing included, in order, slice-timing correction, motion correction, co-registration of the structural image to the mean EPI, and segmentation of the structural image. All of the participants’ segmented gray- and white-matter images were then imported into the DARTEL toolbox (Ashburner, 2007) to create an average gray- and white-matter template. The template and individual-participant flow fields were used to normalize each participant’s EPIs and structural image to MNI space. The EPIs were also resampled to 1.5 mm isotropic voxel dimensions and smoothed with a 6 mm FWHM Gaussian kernel. The structural images were then averaged together for displaying the functional data.

fMRI Data Analysis

Event-related BOLD responses were analyzed using a general linear model (GLM). Activity related to each trial was modeled with a stick function, representing the onset of the first image, convolved with the canonical hemodynamic response function. Serial correlations in the time series were accounted for using the autoregressive model [AR(1)]. A high-pass filter of 128 s was used. There was a covariate of interest for each confidence bin (i.e. 1–6) for each of the 4 trial types (i.e. scene/face different/same) for each of the 8 runs. Covariates of no interest were the 6 motion covariates for each run estimated during the realignment step of preprocessing. Contrast coefficients were weighted to account for different numbers of trial types in each run. Contrast images from first-level analyses were then entered into second-level analyses. 3DClustSim (Cox, 1996; http://afni.nimh.nih.gov/pub/dist/doc/program_help/3dClustSim.html) was used to determine the cluster correction for p<.05 across the whole brain [p<.001 and k=86 voxels].

We opted to define the hippocampal ROI functionally, rather than structurally, because it would give us more power to detect hippocampal involvement. Previous studies have consistently found posterior, but not anterior, hippocampal involvement in scene perception (e.g. Lee and Rudebeck, 2010; Lee et al., 2010; Mundy et al., 2012), which is consistent with work in the rodent implicating the dorsal (septal) hippocampus in spatial context memory (Fanselow and Dong, 2010; Moser and Moser, 1999). Accordingly, averaging over the entire anterior-posterior extent of the hippocampus could have reduced the power to detect hippocampal involvement.

To save the functional clusters as ROIs, we used MarsBaR (Brett, M., Anton, J-L., Valabregue, R., & Poline, J-P., International Conference on Functional Mapping of the Human Brain, 2002). Parameter estimates were extracted and averaged for the voxels within the cluster for each confidence bin. Responses were collapsed across ‘same’ and ‘different’ trials because an insufficient number of misses prevented an examination of only ‘different’ trials for confidence responses 1–3.

For the analysis with ‘different’ trials only, we restricted the analysis to response bins ‘4’, ‘5’, and ‘6’ because there were adequate trial numbers in those bins for every participant (i.e. every participant met the criterion of at least 10 trials in each response bin; average number of trials were 20, 32, and 42 for ‘4’, ‘5’, and ‘6’ response bins, respectively). Because only 1 participant met the criterion of more than 10 responses in each bin for the ‘1’, ‘2’, and ‘3’ responses on ‘different’ trials, we could not reliably extract parameter estimates for those responses (average number of trials were 6, 9, and 10 for ‘1’, ‘2’, and ‘3’ responses, respectively). See Supplemental Figure S1.

The seed for the PPI analysis was the left posterior hippocampus cluster from the preceding ROI analysis. We extracted the time course for each run separately, using MarsBaR. The psychological factor was a linear contrast; each trial was weighted based on the participant’s response: ‘6’ responses were weighted 0, ‘5’=+2, ‘4’=+1, ‘3’=0, ‘2’=−1, and ‘1’=−2. These weights were chosen based on the assumption that regions involved in graded strength-based perception should monotonically track confidence; that is, the greater the evidence for difference, the greater the activation in that region should be. ‘6’ responses were weighted 0 so that a linear trend could not be driven by increased activation on trials in which individuals had access to specific details. As with the ROI analysis, both ‘same’ and ‘different’ trials were included because of an insufficient number of misses.

The PPI term was obtained by multiplying the time course for each run and the psychological factor for that run. A GLM was then run with 9 regressors for each run: the PPI term, the time course, the psychological factor, and the 6 motion regressors. The contrast of interest was a ‘1’ weight for the PPI term and a ‘0’ for all other covariates.

Supplementary Material

01

Highlights.

  • Controversy centers around whether the hippocampus is involved in perception

  • Assessed the role of the hippocampus using a novel scene change-detection paradigm

  • Patient and fMRI data reveal a role in strength- but not state-based perception

  • Hippocampus supports perception through assessments of graded relational match

Acknowledgments

This research was supported by grants MH59352 and MH083734. We would like to thank Maureen Ritchey, Iain Harlow, Luke Jenkins, and the UC Davis Memory Group for helpful advice.

Footnotes

Author Contributions

M.A., C.R., and A.P.Y. conceived and designed the experiments. M.A. performed the experiments and analyzed the data. M.A., C.R., and A.P.Y. wrote the paper.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Aguirre GK, Zarahn E, D’Esposito M. An area within human ventral cortex sensitive to “building” stimuli: Evidence and implications. Neuron. 1998;21:373–383. doi: 10.1016/s0896-6273(00)80546-2. [DOI] [PubMed] [Google Scholar]
  2. Aly M, Yonelinas AP. Bridging consciousness and cognition in memory and perception: Evidence for both state and strength processes. PLoS ONE. 2012;7:e30231, 1–16. doi: 10.1371/journal.pone.0030231. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Ashburner J. A fast diffeomorphic image registration algorithm. NeuroImage. 2007;38:95–113. doi: 10.1016/j.neuroimage.2007.07.007. [DOI] [PubMed] [Google Scholar]
  4. Baddeley AD, Warrington EK. Amnesia and the distinction between long- and short-term memory. J Verb Learn Verb Beh. 1970;9:176–189. [Google Scholar]
  5. Bakker A, Kirwan CB, Miller M, Stark CEL. Pattern separation in the human hippocampal CA3 and dentate gyrus. Science. 2008;319:1640–1642. doi: 10.1126/science.1152882. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Barense MD, Henson RN, Lee ACH, Graham KS. Medial temporal lobe activity during complex discrimination of faces, objects, and scenes: Effects of viewpoint. Hippocampus. 2010;20:389–401. doi: 10.1002/hipo.20641. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Baxter MG. Involvement of medial temporal lobe structures in memory and perception. Neuron. 2009;61:667–677. doi: 10.1016/j.neuron.2009.02.007. [DOI] [PubMed] [Google Scholar]
  8. Burgess N, Maguire EA, O’Keefe J. The human hippocampus and spatial and episodic memory. Neuron. 2002;35:625–641. doi: 10.1016/s0896-6273(02)00830-9. [DOI] [PubMed] [Google Scholar]
  9. Burnham KP, Anderson DR. Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach. 2. Springer; New York: 2002. [Google Scholar]
  10. Chadwick MJ, Mullally SL, Maguire EA. The hippocampus extrapolates beyond the view in scenes: An fMRI study of boundary extension. Cortex. 2012 doi: 10.1016/j.cortex.2012.11.010. http://dx.doi.org/10.1016/j.cortex.2012.11.010. [DOI] [PMC free article] [PubMed]
  11. Cohen NJ, Eichenbaum H. Memory, Amnesia, and the Hippocampal System. Cambridge, MA: MIT Press; 1993. [Google Scholar]
  12. Cox RW. AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages. Comp Bio Res. 1996;29:162–173. doi: 10.1006/cbmr.1996.0014. [DOI] [PubMed] [Google Scholar]
  13. Dale AM. Optimal experimental design for event-related fMRI. Hum Brain Map. 1999;8:109–114. doi: 10.1002/(SICI)1097-0193(1999)8:2/3&#x0003c;109::AID-HBM7&#x0003e;3.0.CO;2-W. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Diana RA, Yonelinas AP, Ranganath C. Imaging recollection and familiarity in the medial temporal lobe: A three-component model. Tren Cog Sci. 2007;11:379–386. doi: 10.1016/j.tics.2007.08.001. [DOI] [PubMed] [Google Scholar]
  15. Duncan K, Ketz N, Inati SJ, Davachi L. Evidence for CA1 as a match/mismatch detector: A high-resolution fMRI study of the human hippocampus. Hippocampus. 2012;22:389–398. doi: 10.1002/hipo.20933. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Duvernoy HM, Bourgouin P. The human hippocampus: Functional anatomy, vascularization and serial sections with MRI. 2. New York: Springer; 1998. [Google Scholar]
  17. Eichenbaum H, Cohen NJ. From conditioning to conscious recollection: Memory systems of the brain. New York, NY: Oxford University Press; 2001. [Google Scholar]
  18. Eichenbaum H, Otto T, Cohen NJ. Two functional components of the hippocampal memory system. Beh Brain Sci. 1994;17:449–517. [Google Scholar]
  19. Eichenbaum H, Yonelinas AP, Ranganath C. The medial temporal lobe and recognition memory. Annu Rev Neurosci. 2007;30:123–152. doi: 10.1146/annurev.neuro.30.051606.094328. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Elfman KW, Parks CM, Yonelinas AP. Testing a neurocomputational model of recollection, familiarity, and source recognition. J Exp Psych: Learn Mem Cog. 2008;34:752–768. doi: 10.1037/0278-7393.34.4.752. [DOI] [PubMed] [Google Scholar]
  21. Epstein R, Kanwisher N. A cortical representation of the local visual environment. Nature. 1998;392:598–601. doi: 10.1038/33402. [DOI] [PubMed] [Google Scholar]
  22. Fanselow MS, Dong HW. Are the dorsal and ventral hippocampus functionally distinct structures? Neuron. 2010;65:7–19. doi: 10.1016/j.neuron.2009.11.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Gadian DG, et al. Developmental amnesia associated with early hypoxic-ischaemic injury. Brain. 2000;123:499–507. doi: 10.1093/brain/123.3.499. [DOI] [PubMed] [Google Scholar]
  24. Graf P, Schacter DL. Implicit and explicit memory for new associations in normal and amnesic subjects. J Exp Psych: Learn, Mem, Cog. 1984;11:501–518. doi: 10.1037//0278-7393.11.3.501. [DOI] [PubMed] [Google Scholar]
  25. Graham KS, Barense MD, Lee ACH. Going beyond LTM in the MTL: A synthesis of neurospsychological and neuroimaging finds on the role of the medial temporal lobe in memory and perception. Neuropsychologia. 2010;48:831–853. doi: 10.1016/j.neuropsychologia.2010.01.001. [DOI] [PubMed] [Google Scholar]
  26. Green DM, Swets JA. Signal detection theory and psychophysics. New York: John Wiley and Sons; 1966. [Google Scholar]
  27. Hartley T, et al. The hippocampus is required for short-term topographical memory in humans. Hippocampus. 2007;17:34–48. doi: 10.1002/hipo.20240. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Harlow IM, Donaldson DI. Source accuracy data reveal the thresholded nature of human episodic memory. Psychon Bull Rev. 2013;20:318–325. doi: 10.3758/s13423-012-0340-9. [DOI] [PubMed] [Google Scholar]
  29. Hopkins RO, Kesner RP, Goldstein M. Item and order recognition memory in subjects with hypoxic brain injury. Brain Cogn. 1995;27:180–201. doi: 10.1006/brcg.1995.1016. [DOI] [PubMed] [Google Scholar]
  30. Insausti R, Juottonen K, Soininen H, Insausti AM, Partanen K, Vainio P, Laakso MP, Pitkanen A. MR volumetric analysis of the human entorhinal, perirhinal, and temporopolar cortices. Am J Neuroradiol. 1998;19:659–671. [PMC free article] [PubMed] [Google Scholar]
  31. Jeneson A, Squire LR. Working memory, long-term memory, and medial temporal lobe function. Learn & Mem. 2012;19:15–25. doi: 10.1101/lm.024018.111. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Kim S, et al. Memory, visual discrimination performance, and the human hippocampus. J Neuro. 2011;31:2624–2629. doi: 10.1523/JNEUROSCI.5954-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Kimmig H, et al. Relationship between saccadic eye movements and cortical activity as measured by fMRI: Quantitative and qualitative aspects. Exp Brain Res. 2001;141:184–194. doi: 10.1007/s002210100844. [DOI] [PubMed] [Google Scholar]
  34. Knutson AR, Hopkins RO, Squire LR. Visual discrimination performance, memory, and medial temporal lobe function. Proc Nat Acad Sci USA. 2012;109:13106–13111. doi: 10.1073/pnas.1208876109. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Kono E, Kono R, Shida K. Computerized tomographies of 34 patients at the chronic stage of acute carbon monoxide poisoning. Archiv fur Psychiatrie und Nervenkrankheiten. 1983;233:271–278. doi: 10.1007/BF00345797. [DOI] [PubMed] [Google Scholar]
  36. Lacy JW, Yassa MA, Stark SM, Muftuler LT, Stark CEL. Distinct pattern separation related transfer functions in human CA3/dentate and CA1 revealed using high-resolution fMRI and variable mnemonic similarity. Learning & Memory. 2010;18:15–18. doi: 10.1101/lm.1971111. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Lee ACH, et al. Specialization in the medial temporal lobe for processing of objects and scenes. Hippocampus. 2005a;15:782–797. doi: 10.1002/hipo.20101. [DOI] [PubMed] [Google Scholar]
  38. Lee ACH, et al. Perceptual deficits in amnesia: Challenging the medial temporal lobe ‘mnemonic’ view. Neuropsychologia. 2005b;43:1–11. doi: 10.1016/j.neuropsychologia.2004.07.017. [DOI] [PubMed] [Google Scholar]
  39. Lee ACH, Rudebeck SR. Investigating the interaction between spatial perception and working memory in the human medial temporal lobe. J Cog Neuro. 2010;22:2823–2835. doi: 10.1162/jocn.2009.21396. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Lee ACH, Scahill VL, Graham KS. Activating the medial temporal lobe during oddity judgment for faces and scenes. Cer Cort. 2008;18:683–696. doi: 10.1093/cercor/bhm104. [DOI] [PubMed] [Google Scholar]
  41. Lee ACH, Yeung LK, Barense MD. The hippocampus and visual perception. Front Hum Neuro 6, Article. 2012;91:1–17. doi: 10.3389/fnhum.2012.00091. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Macmillan NA, Creelman CD. Detection Theory: A user’s guide. 2 New York: Cambridge University Press; 2005. [Google Scholar]
  43. Malach R, et al. Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proc Nat Acad Sci USA. 1995;92:8135–8138. doi: 10.1073/pnas.92.18.8135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Menon V, White CD, Eliez S, Glover GH, Reiss AL. Analysis of a distributed neural system involved in spatial information, novelty, and memory processing. Hum Brain Map. 2000;11:117–129. doi: 10.1002/1097-0193(200010)11:2&#x0003c;117::AID-HBM50&#x0003e;3.0.CO;2-M. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Montaldi D, Spencer TJ, Roberts N, Mayes AR. The neural system that mediates familiarity memory. Hippocampus. 2006;16:504–520. doi: 10.1002/hipo.20178. [DOI] [PubMed] [Google Scholar]
  46. Moser MB, Moser EI. Functional differentiation in the hippocampus. Hippocampus. 1999;8:608–619. doi: 10.1002/(SICI)1098-1063(1998)8:6<608::AID-HIPO3>3.0.CO;2-7. [DOI] [PubMed] [Google Scholar]
  47. Mullally S, Intraub H, Maguire EA. Attenuated boundary extension produces a paradoxical memory advantage in amnesic patients. Current Biology. 2012;22:261–268. doi: 10.1016/j.cub.2012.01.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Mundy ME, Downing PE, Graham KS. Extrastriate cortex and medial temporal lobe regions respond differentially to visual feature overlap within preferred stimulus category. Neuropsychologia. 2012;50:3053–3061. doi: 10.1016/j.neuropsychologia.2012.07.006. [DOI] [PubMed] [Google Scholar]
  49. Norman KA, O’Reilly RC. Modeling hippocampal and neocortical contributions to recognition memory: A complementary-learning-systems approach. Psych Rev. 2003;110:611–646. doi: 10.1037/0033-295X.110.4.611. [DOI] [PubMed] [Google Scholar]
  50. O’Keefe J, Nadel L. The hippocampus as a cognitive map. Oxford University Press; 1978. [Google Scholar]
  51. Olsen RK, Moses SN, Riggs L, Ryan JD. The hippocampus supports multiple cognitive processes through relational binding and comparison. Front Hum Neuro 6, Article. 2012;146:1–13. doi: 10.3389/fnhum.2012.00146. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Park S, Brady TF, Greene MR, Oliva A. Disentangling scene content from spatial boundary: Complementary roles for the parahippocampal place area and lateral occipital complex in representing real-world scenes. J Neuro. 2011;31:1333–1340. doi: 10.1523/JNEUROSCI.3885-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Parks CM, Murray LJ, Elfman K, Yonelinas AP. Variations in recollection: The effects of complexity on source recognition. J Exp Psych: Learn, Mem, & Cog. 2011;37:861–873. doi: 10.1037/a0022798. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Parks CM, Yonelinas AP. Evidence for a memory threshold in second-choice recognition memory responses. Proc Nat Acad Sci USA. 2009;106:11515–11519. doi: 10.1073/pnas.0905505106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Rempel-Clower NL, Zola SM, Squire LR, Amaral DG. Three cases of enduring memory impairment after bilateral damage limited to the hippocampal formation. J Neuro. 1996;16:5233–5255. doi: 10.1523/JNEUROSCI.16-16-05233.1996. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Ranganath C, et al. Dissociable correlates of recollection and familiarity within the medial temporal lobes. Neuropsychologia. 2003;42:2–13. doi: 10.1016/j.neuropsychologia.2003.07.006. [DOI] [PubMed] [Google Scholar]
  57. Rensink RA. Visual sensing without seeing. Psych Sci. 2004;15:27–32. doi: 10.1111/j.0963-7214.2004.01501005.x. [DOI] [PubMed] [Google Scholar]
  58. Ryan JD, Althoff RR, Whitlow S, Cohen NJ. Amnesia is a deficit in relational memory. Psych Sci. 2000;11:454–461. doi: 10.1111/1467-9280.00288. [DOI] [PubMed] [Google Scholar]
  59. Saksida LM, Bussey TJ. The representational-hierarchical view of amnesia: Translation from animal to human. Neuropsychologia. 2010;48:2370–2384. doi: 10.1016/j.neuropsychologia.2010.02.026. [DOI] [PubMed] [Google Scholar]
  60. Scoville WB, Milner B. Loss of recent memory after bilateral hippocampal lesions. J Neurol Neurosur Neuropsych. 1957;20:11–21. doi: 10.1136/jnnp.20.1.11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Shrager Y, Gold JJ, Hopkins RO, Squire LR. Intact visual perception in memory-impaired patients with medial temporal lobe lesions. J Neuro. 2006;26:2235–2240. doi: 10.1523/JNEUROSCI.4792-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Smith ML, Auer RN, Siesjo BK. The density and distribution of ischemic brain injury in the rat following 2–10 min of forebrain ischemia. Acta Neuropathologica. 1984;64:319–332. doi: 10.1007/BF00690397. [DOI] [PubMed] [Google Scholar]
  63. Squire LR, Zola-Morgan S. The medial temporal lobe memory system. Science. 1991;253:1380–1386. doi: 10.1126/science.1896849. [DOI] [PubMed] [Google Scholar]
  64. Suzuki WA. Perception and the medial temporal lobe: Evaluating the current evidence. Neuron. 2009;61:657–666. doi: 10.1016/j.neuron.2009.02.008. [DOI] [PubMed] [Google Scholar]
  65. Vinogradova OS. Hippocampus as comparator: Role of the two input and two output systems of the hippocampus in selection and registration of information. Hippocampus. 2001;11:578–598. doi: 10.1002/hipo.1073. [DOI] [PubMed] [Google Scholar]
  66. Warren DE, Duff MC, Jensen U, Tranel D, Cohen NJ. Hiding in plain view: Lesions of the medial temporal lobe impair online representation. Hippocampus. 2012;22:1577–1588. doi: 10.1002/hipo.21000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Yassa MA, Stark CE. Pattern separation in the hippocampus. Trends Neuro. 2011;34:515–525. doi: 10.1016/j.tins.2011.06.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. Yonelinas AP. Receiver-operating characteristics in recognition memory: Evidence for a dual-process model. J Exp Psych: Learn Mem Cogn. 1994;20:1341–54. doi: 10.1037//0278-7393.20.6.1341. [DOI] [PubMed] [Google Scholar]
  69. Yonelinas AP, Aly M, Wang WC, Koen JD. Recollection and familiarity: Examining controversial assumptions and new directions. Hippocampus. 2010;20:1178–1194. doi: 10.1002/hipo.20864. [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Yonelinas AP, et al. Effects of extensive temporal lobe damage or mild hypoxia on recollection and familiarity. Nat Neuro. 2002;11:1236–1241. doi: 10.1038/nn961. [DOI] [PubMed] [Google Scholar]
  71. Yonelinas AP, Otten LJ, Shaw KN, Rugg MD. Separating the brain regions involved in recollection and familiarity in recognition memory. J Neurosci. 2005;25:3002–3008. doi: 10.1523/JNEUROSCI.5295-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Zeineh MM, Engel SA, Thompson PM, Bookheimer SY. Unfolding the human hippocampus with high resolution structural and functional MRI. Anat Rec. 2001;265:111–120. doi: 10.1002/ar.1061. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

01
NIHMS470241-supplement-01.docx (7.4MB, docx)

RESOURCES