Match–Mismatch Processes Underlie Human Hippocampal Responses to Associative Novelty

Abstract

The hippocampus has long been proposed to play a critical role in novelty detection through its ability to act as a comparator between past and present experience. A recent study provided evidence for this hypothesis by characterizing hippocampal responses to sequence novelty, a type of associative novelty where familiar items appear in a new temporal order. Here, we ask whether a hippocampal match–mismatch (i.e., comparator) mechanism operates selectively to identify the violation of predictions within the temporal domain or instead also underlies the processing of associative novelty in other domains (e.g., spatial). We used functional magnetic resonance imaging and a repetition paradigm in which subjects viewed sequences of objects presented in distinct locations on the screen and performed an incidental target detection task. The left hippocampus exhibited a pattern of activity consistent with that of an associative match–mismatch detector, with novelty signals generated only in conditions where one contextual component was novel and the other repeated. In contrast, right hippocampal activation signaled the presence of objects in familiar locations. Our results suggest that hippocampal match–mismatch computations constitute a general mechanism underpinning the processing of associative novelty. These findings support a model in which hippocampal mismatch signals rely critically on the recall of previous experience, a process that only occurs when novel sensory inputs overlap significantly with stored representations. More generally, the current study also offers insights into how the hippocampus automatically represents the spatiotemporal context of our experiences, a function that may relate to its role in episodic memory.

Introduction

We have all had the experience of walking into a familiar place and immediately noticing that something has changed, for example, that the furniture in our office has been rearranged. Empirical evidence also bears out the ecological salience of associative novelty, where familiar items appear in novel configurations, with changes in the spatial arrangement (Peeke and Veno, 1973) or temporal sequence of stimuli (Honey et al., 1998) sufficient to elicit an orienting response. Surprisingly, however, little is understood about how the brain represents the presence of associative novelty in our environment.

Although previous studies in both animals and humans suggest that the hippocampus plays a critical role in processing associative novelty [for review, see Brown and Aggleton (2001) and Kumaran and Maguire (2007)], the underlying neural computations are not well understood. In a recent study, we addressed this question by investigating hippocampal responses to sequence novelty, a form of (temporal) associative novelty where familiar items appear in new temporal order (Kumaran and Maguire, 2006). We found that hippocampal activation to sequence novelty only occurred under conditions of associative mismatch, where the first half of the object sequence was the same as previously experienced and the second half novel (i.e., A-B-D-C, where the initial presentation was denoted as A-B-C-D). As such, no hippocampal activation was observed in response to the mere presence of associative novelty per se, when the entire sequence was rearranged (i.e., C-A-D-B). These results, therefore, provide empirical support for the longstanding proposal that the hippocampus functions as an associative match–mismatch detector (or comparator), generating mismatch signals when predictions derived from previous experience are violated by current sensory inputs (Gray, 1982; Vinogradova, 2001).

An important question, however, concerns whether hippocampal match–mismatch computations play a general role in the processing of associative novelty or instead operate selectively to detect the violation of sequential predictions within the temporal domain. Our previous study (Kumaran and Maguire, 2006) raises the possibility that the generation of hippocampal novelty signals in response to associate mismatch (i.e., D-C) may be contingent on the occurrence of an associative match component (i.e., A-B) at an earlier point in time that triggers the recall of predictions concerning what will appear next. Indeed, the role of the hippocampus in novelty detection has been closely linked to its capacity to store and recall sequences (Lisman and Grace, 2005). In contrast, other evidence suggests that the hippocampus plays an important role in the detection of associative novelty within not only the temporal but also other (e.g., spatial) domains (Honey et al., 1998; Wan et al., 1999; Ryan et al., 2000; Fyhn et al., 2002; Lee et al., 2005). An alternative hypothesis, therefore, is that hippocampal match–mismatch processes are engaged whenever novel input patterns consist of both associative match and mismatch components, regardless of whether these are present at the same time or at discrete points in time. If so, this would suggest that match–mismatch computations constitute a general mechanism engaged during the processing of associative novelty.

The primary goal of the current study, therefore, was to determine whether hippocampal match–mismatch computations function solely within the temporal domain or instead also underlie the processing of associative novelty within other domains. We addressed this question by asking whether hippocampal mismatch signals occur when associative match and mismatch components are present at the same time and in separate domains (i.e., spatial and temporal). If so, this would demonstrate that hippocampal match–mismatch computations are not limited to detecting prediction violations in the temporal domain but instead function whenever novel input patterns are composed of associative match and mismatch components. Given that the generation of novelty signals necessitates the use of previous experience, we also hoped to provide insights into how the hippocampus automatically creates representations of the spatial and temporal context of our experiences.

Materials and Methods

Participants.

Eighteen healthy, right-handed native English speakers, who were currently undertaking or had recently completed a university degree, participated in this experiment (age range, 19–31 years; average age, 24.3 years; 10 female). All subjects were free from neurological and psychiatric disease and gave informed written consent in accordance with the local research ethics committee.

Stimuli.

A total of 724 color pictures of objects were used in this study. Pictures were obtained from the Hemera Photo-Objects image collection were devoid of background context and included pictures of household objects, animals, and cars but no faces. Objects were placed on a white background. Objects were arranged into 140 quartets for use in the four main experimental conditions, 40 quartets for use as target trials, and one quartet for use in the baseline condition (see below). There was an approximately equal balance of animate and inanimate objects in the four main experimental conditions. In addition, the allocation of stimuli was randomized between subjects. Examples of objects used in the experiment are shown in Figure 1.

Figure 1.

Experimental design. A, Color pictures of objects were presented in groups of four constituting an object quartet. Objects were presented consecutively for 1 s each in a distinct location on the screen on an invisible 3 × 2 grid (FIRST), followed by a 3 s period during which a fixation cross was displayed. B, After this 3 s period, the same object quartet was represented (second presentation) in one of four possible ways, where the sequence or object location could be either novel or repeated (a 2 × 2 factorial design). The main conditions therefore were as follows: S_RL_R, S_NL_R, S_RL_N, and S_NL_N (see Materials and Methods). Importantly, each object quartet was only presented twice in total during the experiment (i.e., first and second presentations). Note that when object quartets were presented in novel object locations during the second presentation, the same locations in the invisible 3 × 2 grid were used as in the first presentation, although objects themselves occupied different positions. Subjects performed an incidental target detection task throughout the experiment.

Experimental design and procedures.

The four objects in each quartet were presented consecutively for 1 s each on a black screen [first presentation (FIRST)] with no interstimulus gap. Each object within a quartet was presented in a different location on an invisible 3 × 2 grid. After the display of a central fixation cross (3 s duration), the same object quartet was re-presented in one of four possible ways (second presentation). Thus, there were four main experimental conditions constituting a factorial design with factors: sequence and object location. The main conditions (35 trials of each) were as follows: (1) repeated sequence and repeated object locations (S_RL_R); (2) novel sequence and repeated object locations (S_NL_R); (3) repeated sequence and novel object locations (S_RL_N); and (4) novel sequence and novel object locations (S_NL_N).

Importantly, each object quartet was only presented twice in total (i.e., first and second presentations). Note that when object quartets were presented in novel object locations during the second presentation, the same four locations in the invisible 3 × 2 grid were used as in the first presentation, although the objects themselves occupied different positions. Also, when object quartets were presented in a novel sequence during the second presentation, the permitted orders were: B-D-A-C and C-A-D-B (with the first presentation of the object quartet denoted by A-B-C-D). These orders were selected to ensure all objects occupied a different ordinal position in the second presentation, with no forward (or backward) sequential associations preserved between first and second presentations. There was also a baseline condition (BASE) (30 trials) in which one object quartet was repeatedly presented with objects always in the same sequence and locations. Subjects were pre-exposed to this quartet in a prescan training session, rendering objects, sequence, and object locations entirely familiar in this condition. The order in which each experimental condition was presented was randomized for each subject with the constraint that a given condition did not occur more than two times consecutively.

Task.

Before scanning, subjects were informed that they would be participating in a speed target detection task. Our choice of task reflected the desire to minimize awareness of our experimental manipulations while ensuring that subjects attended to both the objects themselves and the locations in which they were presented. Subjects were instructed to respond with a key press if either the object itself or the presentation location repeated in immediate succession (1-back task). To encourage the processing of object quartets as individual “episodes,” subjects were told that they should only look out for two-in-a-row object and location repeats within (and not between) object quartets. Thus, although subjects were aware before scanning that objects would be presented in groups of four, they were not informed of our experimental manipulations. Subjects were warned that we would be monitoring their reaction times and thus encouraged to stay alert throughout to respond to a repeat (i.e., target) as fast as possible, without compromising accuracy. Over the course of the entire experiment, there were 40 trials in which targets (i.e., object or location repeats) were present, requiring subjects to respond with a key press. These trials were excluded from the main analyses. Target trials were equally distributed between all experimental conditions (including the baseline condition), such that the probability of a target occurring was constant across first and second presentations of object quartets.

Scanning consisted of two sessions of 21 min each. There was a short interval between sessions during which subjects remained inside the scanner. Each session was subdivided into three runs with a 1 min break between each during which the scanner continued running. After scanning, subjects participated in a debriefing session in which they were asked in general about how they performed the task and any strategies they might have used.

Functional magnetic resonance imaging design.

The temporal pattern of stimulus presentation was designed to maximize statistical efficiency while preserving psychological validity, in line with established procedures. The trial onset asynchrony (TOA) was fixed at 7 s (i.e., 4 s during which the four objects were presented followed by 3 s fixation cross). Trial onsets were therefore temporally jittered with respect to scan onsets because the TOA is not a simple integer multiple of the TR (time for acquisition of one scanning volume, 4.05 s). Importantly, the hemodynamic response to events that occur a few seconds apart is explicitly modeled (via a hemodynamic response function), and therefore can be estimated separately for each event type by implementing the general linear model as is standard when using statistical parametric mapping software (SPM2; www.fil.ion.ucl.ac.uk/spm/) (Friston et al., 1998).

Imaging parameters and acquisition.

T2*- weighted echo planar (EPI) images with blood oxygen level-dependent contrast were acquired (358 volumes/session; two scanning sessions) on a 1.5 tesla Siemens (Erlangen, Germany) Sonata magnetic resonance imaging (MRI) scanner using a specialized sequence to minimize signal dropout in the medial temporal lobe (Deichmann et al., 2003). We used the following scanning parameters to achieve whole-brain coverage: 45 oblique axial slices angled at 30° in the anteroposterior axis [TR, 4.05 s; 2 mm thickness; 1 mm gap; echo time (TE), 30 ms; in-plane resolution, 3 × 3 mm; field of view, 192 mm; 64 × 64 matrix. A preparation pulse (duration, 1 ms; amplitude, +1 mT/m*ms) was used in the slice selection direction to compensate for through-plane susceptibility gradients predominant in the hippocampus (Weiskopf et al., 2005). A high-resolution (1 × 1 × 1 mm) T1-weighted structural MRI scan was acquired for each subject after functional scanning. This was coregistered to the functional EPIs and averaged across subjects to aid localization.

Functional MRI data processing (SPM2).

Images were analyzed in a standard manner using the statistical parametric mapping software SPM2. After the first six “dummy volumes” were discarded to permit T1 relaxation, images were spatially realigned to the first volume of the first session, followed by spatial normalization to a standard EPI template, resulting in a functional voxel size of 3 × 3 × 3 mm. Normalized images were smoothed using a Gaussian kernel with full-width at half-maximum of 8 mm. After preprocessing, the functional MRI (fMRI) data were analyzed in SPM2 using the general linear model (GLM) using established procedures. As a first step, vectors corresponding to the onset times for each of the condition types were created. Our main interest was in the 4 s period during which the object quartet was presented. Therefore, these vectors were then coded as boxcar functions (of 4 s duration) constituting regressors of interest in the GLM (i.e., the first level design matrix). These regressors were then convolved with the canonical hemodynamic response function (HRF). One regressor of no interest was also included coding for target trials. In addition, subject-specific movement parameters (derived from realignment) were included as regressors of no interest. A high-pass filter with a cutoff of 128 s was used to remove low-frequency drifts. Temporal autocorrelation was modeled using an AR(1) process.

Model estimation proceeded in two stages. In the first stage, condition-specific experimental effects (parameter estimates or regression coefficients pertaining to the height of the canonical HRF) were obtained via the GLM in a voxel-wise manner for each subject. In the second (random-effects) stage, subject-specific linear contrasts of these parameter estimates, collapsed across the three sessions, were entered into a series of one-sample t tests [as is standard when using SPM and a factorial design (Frackowiak et al., 2004)], each constituting a group-level statistical parametric map.

Given our clear a priori anatomical hypotheses, we report activations within the hippocampus at a threshold of p < 0.001 (uncorrected for multiple comparisons). We also report when hippocampal activations survive, small volume correction (SVC) at a threshold of p < 0.05 corrected for multiple comparisons. We applied reduced spherical search volumes, of 10 mm diameter, centered on coordinates [in Montreal Neurological Institute (MNI) space] obtained from our previous study (Kumaran and Maguire, 2006) (left hippocampus, −27, −18, −18) and a previous study exploring hippocampal responses to associative novelty (right hippocampus, 24, −24, −3) (Kohler et al., 2005). The SVC procedure, as implemented in SPM2 using the family wise error correction (p < 0.05), allows results to be corrected for multiple nonindependent comparisons with a defined region of interest. Activations in other areas that have been associated previously with the process of novelty detection (e.g., prefrontal and parietal cortex) are reported for completeness at a level of p < 0.001 uncorrected for multiple comparisons. Activation in other brain regions were only considered significant if they survive whole-brain correction for multiple comparisons at p < 0.05, in line with established procedures (Frackowiak et al., 2004). All activations are displayed on sections of the average structural image of all the participants. Reported voxels conform to MNI coordinate space. The right side of the brain is displayed on the right side.

Results

Behavioral data

In common with numerous previous studies exploring how the brain automatically processes novelty regardless of its behavioral relevance (Stern et al., 1996; Yamaguchi et al., 2004; Bunzeck and Duzel, 2006), subjects performed an incidental task during scanning that was unrelated to our experimental manipulations. As such, we aimed to emphasize the task-irrelevant processing of novel information, in keeping with the notion that the automatic generation of novelty signals is an essential property of an efficient novelty/familiarity discrimination system. Subjects were required to respond with a key press whenever either the object or the location was repeated (see Materials and Methods). Trials in which repeats occurred were subsequently excluded from the main neuroimaging analyses.

Subjects detected consecutive repeats of objects with 89.0 ± 10.5% accuracy and consecutive repeats of locations with 87.0 ± 11.0% accuracy. Reaction time for the detection of object repeats was 610 ± 72 ms and for location repeats was 606 ± 72 ms. There was no significant difference between the two types of targets, either in terms of detection accuracy (t17 = 0.69; p = 0.50) or reaction times (t17 = 0.44; p = 0.66). Moreover, there was no significant difference in reaction time for detection of repeats between the different conditions, including the baseline condition (F_(4,68) = 0.74; p = 0.57). The behavioral data therefore suggest that subjects attended equally to both the objects themselves and the locations in which they were presented, and that attention was well matched between conditions. Subjects were debriefed after scanning to assess how they performed the task. As expected, none of the subjects reported naming or explicitly trying to memorize or retrieve the configurations of stimuli during the experiment. Instead, subjects remained focused on the target detection task, in line with our instructions.

Neuroimaging data

We first identified the overall pattern of activity elicited by viewing a new quartet of objects presented sequentially in distinct spatial locations on the screen. This was achieved by comparing the first presentation of the object quartet with the baseline condition in which subjects viewed familiar objects always presented in the same sequence and spatial locations (FIRST > BASE). There was significantly greater activation in this contrast within the ventral visual stream, consistent with a response to stimulus novelty (Tulving et al., 1994; Schacter and Wagner, 1999; Kirchhoff et al., 2000; Ranganath and Rainer, 2003; Nyberg, 2005), extending into areas such as the parahippocampal gyrus and hippocampus (supplemental Fig. S1, available at www.jneurosci.org as supplemental material).

The primary focus of the study, however, was to determine whether hippocampal match–mismatch computations represent a general mechanism underlying the processing of associative novelty. We therefore asked whether hippocampal mismatch signals are generated when novel input patterns consist of associative match and mismatch components present simultaneously in separate (i.e., spatial and temporal) domains. Hence, we directed our neuroimaging analyses to evaluating differences in brain activity between the four experimental conditions representing the possible spatial and temporal configurations in which objects could appear during the second presentation.

Pattern of responses in the left hippocampus is consistent with a match–mismatch detector

We first considered the simple effect of sequence novelty in conditions in which object locations were repeated (i.e., S_NL_R > S_RL_R). Significantly greater activation was observed in the body of the left hippocampus in this comparison (Fig. 2A). In contrast, when we examined the effect of sequence novelty in the context of novel object locations (i.e., S_NL_N > S_RL_N), no significant activation was observed within the hippocampus, nor when we considered the main effect of sequence novelty (i.e., S_NL_N + S_NL_R > S_RL_R + S_RL_N). This was the case even at liberal statistical thresholds (i.e., p < 0.01, uncorrected for multiple comparisons). Thus, significant hippocampal activation was observed only in response to novel sequences in the context of repeated object locations but not when object locations were novel. This is further suggested by the presence of significant activation in the left hippocampus when the S_NL_R condition was contrasted against the condition entailing the maximal level of associative novelty (S_NL_R > S_NL_N) (Fig. 2B). Note that right hippocampal activation observed in this contrast likely reflects the response of this region to the presence of objects in familiar locations (see below). Importantly, these results cannot be attributed to differences in the spatial sequence of eye movements between the experimental conditions, because it is likely that the change in spatial sequence was similar for all conditions in which associative novelty was present (i.e., S_NL_R, S_RL_N, S_NL_N) (Fig. 1).

Figure 2.

Increased left hippocampal responses occur only when one contextual component is repeated and the other novel. A, Simple effect of sequence novelty when object locations are repeated (S_NL_R > S_RL_R). A coronal section at the level of the peak activation in the left hippocampus (x, y, z = −24, −21, −9; z = 3.45) is displayed at the top left. The color bar indicates the t statistic associated with each voxel and the z-score equivalent. Threshold is set at p < 0.005 (uncorrected for display purposes). R, Right side of brain. Activation in the hippocampus is significant at p < 0.001 (uncorrected for multiple comparisons) and also survives SVC at threshold p < 0.05 (corrected) (see Materials and Methods). B, Coronal section showing significant left hippocampal activation (x, y, z = −24, −24, −6; z = 3.46) in contrast with S_NL_R > S_NL_N. Significant activation in the right hippocampus (x, y, z = 33, −21, −9; z = 3.72) in this comparison likely reflects a response to object location familiarity. Hippocampal activations are significant at p < 0.001 (uncorrected for multiple comparisons) and also survive SVC at threshold p < 0.05 (corrected). C, Simple effect of object location novelty when sequence is repeated (S_RL_N > S_RL_R). A coronal section at the level of the peak activation in the left hippocampus (x, y, z = −39, −21, −15; z = 3.43) is displayed at the bottom left. Activation in the right thalamus (x, y, z = 12, −21, −12; z = 3.49) is also visible but not significant at predefined statistical threshold (see Materials and Methods). Threshold is set at p < 0.005 (uncorrected for display purposes). Activation in the hippocampus is significant at p < 0.001 (uncorrected for multiple comparisons).

A similar pattern of findings was observed when we examined the effect of object location novelty. Significant activation of left hippocampal body was associated with object location novelty only in the context of repeated sequences (i.e., S_RL_N > S_RL_R) (Fig. 2C) and not in the context of novel sequences (i.e., S_NL_N > S_NL_R). Similarly, no hippocampal activation was seen when we considered the main effect of object location novelty (i.e., S_NL_N + S_RL_N > S_RL_R + S_NL_R). Also note that no hippocampal activation was observed in the condition when both sequence and object location were novel, in the aforementioned comparisons (i.e., S_NL_N > S_RL_N or S_NL_N > S_NL_R), or indeed when we examined the comparison S_NL_N > S_RL_R, even at liberal statistical thresholds.

Left hippocampal novelty responses to sequence or object location novelty were therefore only observed in conditions where the other contextual component was repeated (i.e., S_NL_R and S_RL_N). This is reflected by the presence of a significant interaction (i.e., S_NL_R − S_RL_R > S_NL_N − S_RL_N) between the two factors in the left hippocampus (Fig. 3). The pattern of activation observed in the left hippocampus, therefore, demonstrates that the hippocampus plays a broader role as a match–mismatch detector during the processing of associative novelty, rather than being limited to detecting prediction violations within the temporal domain.

Figure 3.

Interaction effect in the left hippocampus. A, Interaction of sequence and object location (i.e., S_NL_R + S_RL_N > S_RL_R + S_NL_N). A sagittal section showing activation in the left hippocampus (x, y, z = −24, −21, −9; z = 3.72) is shown. Activations in the left intraparietal sulcus (visible on sagittal section) (x, y, z = −27, −42, 60; z = 4.18) and right thalamus (not visible) (x, y, z = 12, −21, −12; z = 4.25) is also present but not significant at predefined statistical threshold (see Materials and Methods). Threshold is set at p < 0.005 (uncorrected for display purposes). R, Right side of brain. Activation in the hippocampus is significant at p < 0.001 (uncorrected for multiple comparisons) and also survives SVC at threshold p < 0.05 (corrected). B, Condition-specific parameter estimates (β) in arbitrary units at peak voxel in the left hippocampus.

Activity in the right hippocampus reflects the presence of objects in familiar locations

Whereas the left hippocampus exhibited a pattern of responses consistent with that of a match–mismatch detector, the right hippocampus responded more generally to the presence of objects in familiar locations. As such, there was a significant main effect of object location familiarity in the right hippocampal body (S_RL_R + S_NL_R > S_RL_N + S_NL_N) (Fig. 4) and no significant interaction (i.e., S_NL_R − S_RL_R > S_NL_N − S_RL_N), even at liberal statistical thresholds. This conclusion receives additional support from the direct comparison of both conditions consisting of novel and one repeated component (S_NL_R > S_RL_N) (supplemental Fig. S2, available at www.jneurosci.org as supplemental material), which revealed significant activation in the right hippocampus. As expected, no significant activation was observed in the left hippocampus in this contrast, consistent with its proposed role in match–mismatch detection. Our findings therefore provide evidence that the right hippocampus automatically and, after a single trial, signals the presence of objects that appear in familiar locations in the environment.

Figure 4.

Main effect of repeated object locations in the right hippocampus. A, Main effect of repeated object locations (S_RL_R + S_RL_R > S_NL_N + S_RL_R). The top left panel shows a coronal section at the level of the peak activation in the right hippocampus (x, y, z = 27, −21, −6; z = 3.48). A sagittal section showing right hippocampal activation is also displayed at the bottom left. Threshold is set at p < 0.005 (uncorrected for display purposes). R, Right side of brain. Activation in the hippocampus is significant at p < 0.001 (uncorrected for multiple comparisons) and also survives SVC at threshold p < 0.05 (corrected) (see Materials and Methods). B, Condition-specific parameter estimates (β) in arbitrary units at the peak voxel in the right hippocampal body. Error bars represent SE.

Discussion

In this fMRI study, we set out to establish whether hippocampal match–mismatch computations play a general role in the processing of associative novelty or act selectively to detect the violation of sequential predictions within the temporal domain. We observed increased activity in the left hippocampus under conditions when one spatial or temporal contextual component was repeated and the other novel, and not when both components were novel. In contrast, activity in the right hippocampus reflected the presence of objects in familiar locations, regardless of object sequence. Our results support the notion that hippocampal match–mismatch computations constitute a fundamental mechanism underlying the processing of associative novelty. Critically, these findings suggest that hippocampal mismatch signals occur whenever novel input patterns contain both a novel (i.e., mismatch) and familiar (i.e., match) associative component, regardless of whether these components are present within the same or different domains or occur simultaneously or separated in time. Together, our findings support a model in which hippocampal mismatch signals to associative novelty depend on the recall of information pertaining to previous experience, a process that only occurs when novel sensory inputs overlap significantly with stored representations.

The pattern of left hippocampal activation observed in this study is consistent with the operation of an associative match–mismatch detector. As such, our findings argue strongly against the operation of a familiarity mechanism within the hippocampus during the processing of associative novelty. Given that familiarity-based mechanisms have been proposed to underpin the coding of stimulus novelty in the perirhinal cortex (Brown and Bashir, 2002; Henson et al., 2003; Norman and O'Reilly, 2003), a similar mechanism might conceivably have accounted for hippocampal responses to associative novelty. In essence, familiarity mechanisms compute the global match between current input patterns and stored representations, outputting a scalar familiarity signal inversely proportional to the degree of similarity (Gillund and Shiffrin, 1984; Hintzman, 2001). Therefore, the operation of a familiarity mechanism would predict that the condition with the greatest change from the first presentation (S_NL_N) would be associated with the highest level of hippocampal activation. In contrast, we observed differential activation in the left hippocampus only in the S_NL_R and S_RL_N conditions in a region closely situated to that implicated in match–mismatch detection on the basis of previous work (Kumaran and Maguire, 2006). Our findings therefore provide additional support for the operation of match–mismatch computations in the hippocampus, rather than a familiarity mechanism, during the processing of associative novelty.

The present results therefore demonstrate that hippocampal match–mismatch processes operate when match and mismatch components are present in separate domains (i.e., spatial and temporal), rather than solely within the same (i.e., temporal) domain. Critically, this demonstrates that hippocampal mismatch signals are not engaged solely by the violation of sequential predictions, when a period of associative familiarity (i.e., match) is followed later by associative novelty (i.e., mismatch). Instead, these findings suggest that hippocampal mismatch signals are generated whenever novel sensory inputs consist of both associative match and mismatch components, regardless of whether they occur at the same time (as in this experiment) or consecutively [as in the previous study (Kumaran and Maguire, 2006)]. As such, the hippocampus would appear to play a broader role as a match–mismatch detector during the processing of associative novelty, rather than selectively detecting the violation of sequential predictions within the temporal domain.

Why might hippocampal mismatch signals to associative novelty depend on the presence of an associative match component? A fundamental property of match–mismatch (or comparator) models of novelty detection (Sokolov, 1963; Gray, 1982; Levy, 1989; Hasselmo and Schnell, 1994; Knight, 1996; Lisman and Otmakhova, 2001; Vinogradova, 2001) is that they rely on not only a comparator mechanism for detecting matches versus mismatches but also a mechanism that allows entire stored patterns to be recalled in response to a partial input cue (i.e., pattern completion). Importantly, therefore, the generation of mismatch signals can be seen to be contingent on the process of hippocampal recall. As such, an associative match component may constitute a partial input cue necessary for triggering the recall of entire stored patterns pertaining to previous experience, to which current inputs are compared. Considered from a related computational perspective (Norman and O'Reilly, 2003), novel input patterns that contain an associative match component engender a high degree of overlap with stored representations, a factor proposed to be crucial in triggering recall processes because of the pattern separated nature of hippocampal representations (for review, see Kumaran and Maguire, 2007). Together, our findings suggest that a model in which the similarity of novel sensory inputs to stored patterns is a critical determinant of whether hippocampal recall occurs can account for the generation of hippocampal mismatch signals to associative novelty.

The current findings may also help explain why hippocampal responses to associative novelty have been observed previously in some experiments but not others. Several studies found increased hippocampal activation in response to both spatial (i.e., object location) and nonspatial (i.e., object–object) associative novelty (Duzel et al., 2003; Pihlajamaki et al., 2004; Kohler et al., 2005). A key aspect of those experiments is that associative novelty (i.e., mismatch, e.g., object in a new location) was always presented in conjunction with associative match (i.e., match, e.g., repeated object–object pairings). Given the findings reported here, it would seem likely that the observed hippocampal activations reflect the operation of associative match–mismatch processes, although these previous studies were not designed with the aim of testing this hypothesis. In contrast, hippocampal activation was absent under conditions when novel associative information did not contain an associative match (i.e., familiar) component (Kirwan and Stark, 2004; Strange et al., 2005). For example, no hippocampal responses were observed in a recent study where subjects viewed spatial arrays consisting of new pictures not previously encountered (Strange et al., 2005). Previous data, therefore, support the hypothesis that the hippocampus operates as an associative match–mismatch detector, generating mismatch signals only when novel input patterns consist of both associative match and mismatch components.

Whereas the left hippocampus exhibited a pattern of activation consistent with that of a match–mismatch detector, activity in the right hippocampus signaled the presence of objects in familiar locations. Although not the primary focus of this study, it is interesting to consider how this lateralized pattern of hippocampal activation relates to previously proposed functions of the left and right hippocampus. Our findings accord well with data suggesting that the right hippocampus may play a relatively greater role in spatial processing (O'Keefe and Nadel, 1978; Burgess et al., 2002). For example, right, but not left, temporal lobectomy patients exhibit deficits when asked to recall the spatial locations of objects in an array, the magnitude of which correlates with the amount of hippocampal damage present (Smith and Milner, 1981, 1989; Nunn et al., 1999). However, it is important to note that the right hippocampus does not process spatial information exclusively, given that it also appears to play a distinct role in memory for nonverbal/visuospatial information (e.g., faces) (Pigott and Milner, 1993; Golby et al., 2001; Coleshill et al., 2004). In contrast, the left hippocampus has been suggested to play a role in mediating the context-dependent (e.g., temporal) aspects of autobiographical memory (O'Keefe and Nadel, 1978; Maguire, 2001; Burgess et al., 2002), in addition to subserving memory for verbal information (Lee et al., 2002). For instance, a previous study found that left, but not right, unilateral temporal lobectomy patients were impaired when tested about the temporal order in which they received objects during exploration in a virtual environment (Spiers et al., 2001). The present findings, therefore, are consistent with the idea that the left hippocampus may play a role in representing the temporal (i.e., sequential) aspects of our episodic experiences. In the future, it will be important to establish a theoretical framework to clarify more precisely how the relative functions of the left and right hippocampus might differ.

Our findings also provide insights into the nature of representations supported by the hippocampus. In this respect, our repetition paradigm has conceptual similarities with the fMR adaptation (fMRA) approach (Grill-Spector and Malach, 2001; Grill-Spector et al., 2006), which has been used to characterize the reductions in neural activity that typically result from the repetition of individual stimuli or stimulus attributes. Although generally involving single stimuli (cf. associative information) and shorter timescales, fMRA has been widely used to infer the nature of neural representations supported by different areas (Grill-Spector and Malach, 2001; Grill-Spector et al., 2006). In our paradigm, hippocampal mismatch signals evoked during the second presentation necessarily rely on accessing information pertaining to previous experience (i.e., first presentation). It would therefore seem likely that the left hippocampus creates representations of the spatial and temporal context in which objects appear during the first presentation, which form the basis for the generation of mismatch signals subsequently (i.e., second presentation). Nevertheless, it could perhaps be argued that other brain regions (e.g., prefrontal cortex) support representations concerning the first presentation, with the hippocampus acting solely as a match–mismatch detector. However, recent evidence demonstrating that the hippocampus plays an important role in certain types of short-term memory argues against this possibility (Ranganath and Blumenfeld, 2005; Hannula et al., 2006; Olson et al., 2006). For example, patients with bilateral hippocampal damage and amnesia are impaired at remembering the locations of trial-unique objects even when the delay between study and test is of the order of a few seconds (Olson et al., 2006), implying that other brain regions are unable to sustain such representations even over a brief period. In our paradigm, the effective delay between the start of the first and second presentations was ∼7 s, with several intervening objects presented in this time window. Our results, therefore, support theoretical proposals that the hippocampus is critical for episodic memory through its unique ability to rapidly bind together the disparate elements of episodes together with the spatiotemporal context in which they occur (Marr, 1971; Cohen and Eichenbaum, 1993; McClelland et al., 1995; Eichenbaum, 2004).

The detection of associative novelty within our environment is critical to adaptive behavior. The current study addressed a fundamental question that has received little attention to date, namely the neural mechanisms by which the hippocampus signals the presence of associative novelty. Our results provide support for the hypothesis that the hippocampus acts as an associative match–mismatch detector and critically suggest that this function represents a general mechanism engaged during the processing of associative novelty. Based on these findings, we propose a model in which hippocampal mismatch signals depend on the previous recall of information pertaining to previous experience, a process that only occurs when novel sensory inputs overlap significantly with stored representations. In the future, it will be of interest to vary the similarity of novel inputs to stored representations in a more finely graded manner, with the aim of predicting the amplitude of hippocampal novelty signals using a formal model.