Skip to main content
PMC home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2010 Jul 21;30(29):9890–9897. doi: 10.1523/JNEUROSCI.0826-10.2010

Object Unitization and Associative Memory Formation Are Supported by Distinct Brain Regions

Bernhard P Staresina 1,✉, Lila Davachi 2,3
PMCID: PMC2927970  NIHMSID: NIHMS223310  PMID: 20660271

Abstract

While it has been established that the medial temporal lobe (MTL) is critical for successful memory formation, the precise contribution of one of the key MTL subregions, the perirhinal cortex (PrC), has been the subject of intense focus and debate. Although this region has been implicated in nonassociative item encoding, recent neuroimaging data have revealed that it also contributes to the binding of specific item-related associations (e.g., an object–color association). Based on neuropsychological evidence that associative memory for unitized word pairs does not require the hippocampus, it has been proposed that PrC contributes to associative memory formation by means of unitization. However, the role of PrC in unitization processes remains unclear. Here, we used fMRI to assess the involvement of PrC in object unitization as well as in successful episodic encoding of item-related details. Our results show that while PrC activation linearly tracks the amount of item-related information successfully encoded, it is not modulated by object unitization demands. Instead, the present data reveal that unitization of object fragments may be accomplished in visual/ventral temporal processing stages before PrC. Indeed, we observed a gradual transition from unitization to successful memory formation across the ventral visual pathway. This suggests that PrC may specifically serve to encode item-related event details, whereas their perceptual integration is established along preceding processing stages.

Introduction

A major challenge in the study of human memory concerns the specific contributions of medial temporal lobe (MTL) subregions to successful memory formation (Squire et al., 2004; Henson, 2005; Davachi, 2006; Diana et al., 2007; Konkel and Cohen, 2009). One view holds that the perirhinal cortex (PrC) supports simple item encoding promoting subsequent familiarity, whereas the hippocampus supports associative memory formation, which gives rise to later recollection (Aggleton and Brown, 1999; Eichenbaum et al., 2007; Mayes et al., 2007). However, the assertion that PrC only supports nonassociative item recognition/familiarity was recently questioned by data showing episodic associative encoding effects in PrC (Staresina and Davachi, 2006, 2008). Interestingly, one possible mechanism that may reconcile these seemingly discrepant findings has emerged from neuropsychological findings: specifically, associative memory performance has been demonstrated to be relatively spared in patients with selective hippocampal damage if the to-be-remembered information is processed in a unitized manner (Giovanello et al., 2006; Quamme et al., 2007) [the term “unitization” denotes the process of representing separate items/stimulus components as a single unit (LaBerge and Samuels, 1974; Graf and Schacter, 1989)]. Although the aforementioned data remain agnostic as to (1) what brain region, if not the hippocampus, effectively encodes the unitized components, and (2) where the separate components, before their mnemonic encoding, are actually fused or unitized into an integrated representation, one intriguing hypothesis is that associative memory through unitization is supported by PrC (Giovanello et al., 2006; Diana et al., 2007, 2008). A recent fMRI study by Haskins et al. (2008) further strengthened the link between PrC and unitization. In their experiment, two separate nouns (e.g., “slope” and “bread”) had to be processed either as a novel compound representation (a “slopebread”; compound trials) or as individual items separated by a sentence frame (sentence trials). Not only did PrC activation predict associative memory exclusively for compound trials, but it was also enhanced during overall task performance (regardless of subsequent memory) for compound relative to sentence trials, suggesting an active role in the process of unitization.

However, in that paradigm, PrC activation may in fact reflect the amount and/or conceptual novelty of item-related information being processed, rather than unitization per se. That is, “slopebread” is a conceptually novel target item that is being processed in the compound but not the sentence condition. Thus, it is not clear whether PrC engagement represents the active fusion of the separate event components as opposed to the processing/encoding of novel object information. In the current study, to systematically increase the demands on unitization without including additional item/object information, we used real object images instead of word pairs. Objects were either left intact, separated into two fragments, or separated into four fragments, and subjects were instructed to unitize the object fragments into a single representation for further processing. A subsequent memory test assessed object recognition as well as memory for an associated color, allowing us to probe differential and overlapping effects of unitization and episodic encoding of item-related event details.

Materials and Methods

Subjects.

Eighteen (9 female) right-handed native English speakers with normal or corrected-to-normal vision participated in the experiment (mean age: 21 years, range: 18–28). Informed consent was obtained in a manner approved by the institutional review board at New York University, and subjects were paid for their participation. One subject was excluded from all analyses due to poor memory performance (recognition memory d′ < 2 SDs below the sample mean).

Stimuli.

The stimuli consisted of 450 grayscale object images, 300 of which served as study items and 150 of which served as lures during a recognition memory test. The critical manipulation in this experiment was the way in which the target object was visually presented during encoding. For zero-fragments (F0) trials, the object images were left visually intact/integrated. For two-fragments (F2) and four-fragments (F4) trials, the object images were split into two or four parts, respectively. F2 objects were split along the horizontal axis if the object's height exceeded its width and along the vertical axis if its width exceeded its height, and the resulting parts were shifted up–down or left–right, respectively. The four parts in F4 objects were randomly rotated clockwise for 1–3 iterations, so that no part aligned vertically or horizontally with its originally neighboring part in either direction (if the original image consists the image-part matrix [11, 12; 21, 22], the F4 versions were either [21, 11; 22, 12], [22, 21; 12, 11], or [12, 22; 11, 21]).

The 300 study items were divided into three sets of 100 items per fragmentation level that were evenly assigned to the colors blue, green, red, and brown (25 trials per color). The stimulus material was counterbalanced so that across subjects, every object was shown with every color in each fragmentation level and was used both as a study item and as a lure for the subsequent recognition memory test.

Procedure.

For each scanned 4 s encoding trial, subjects were presented with an object and a surrounding color square (Fig. 1). Subjects were instructed to imagine the given object in the associated color and to rate the plausibility of that particular object/color combination as plausible or implausible. Responses were given via a magnet-compatible button box placed under the subject's left hand. Importantly, subjects were instructed to press a third button in case they could not perform the task on a given trial, be it because they did not recognize the object or because they could not imagine the object in the given color. Those trials, as well as trials for which no response was given, were excluded from all further analyses.

Figure 1.

Figure 1.

Open in a new tab

Experimental design. During the scanned encoding session, subjects were asked to rate the plausibility of the given object/color combination. Critically, object images were either left intact (0-Fragmentation), split into two parts (2-Fragmentation), or split into four parts (4-Fragmentation), systematically increasing the demand on perceptual unitization. Encoding trials were intermixed with an active baseline condition (“arrows task,” not shown). During the surprise postscan retrieval test, previously seen (old) objects were intermixed with objects not previously seen (new), and subjects had to indicate whether the object was old (including confidence ratings) or new. For “old” responses, subjects were asked to indicate the color with which the object had been presented during encoding (response options: blue, green, red, brown, and “don't know”).

Following the encoding session, subjects were given an unscanned and self-paced surprise recognition memory test (Fig. 1), consisting of all 300 previously presented objects presented in grayscale as well as 150 novel objects (lures). First, the intact/integrated image of an object was presented on the screen and subjects were instructed to indicate whether the object was old (presented during the encoding session) or new (not presented during the encoding session). For objects endorsed as old, subjects provided confidence ratings by rating the object as “unsure old” (low confidence) or as “sure old” (high confidence). Regardless of the confidence level, objects endorsed as old were followed by a prompt to indicate the color with which the object was associated during encoding. Response options were “blue,” “green,” “red,” “brown,” and “?.” Importantly, color response options were represented by their verbal labels, not by actual color frames as used during encoding, biasing responses toward color recollection rather than color familiarity (further discussed below). Question mark responses were further included to avoid familiarity-based guesses. This testing protocol was used to sort the scanned encoding trials not only based on fragmentation level, but also based on the amount of item-related event details successfully encoded for subsequent memory performance. The average duration of the test portion was 26 min (SD = 7 min).

MRI scanning and data analysis.

Scanning was performed on a 3 T Siemens Allegra MRI system using a whole-head coil. Functional data were acquired using a gradient-echo, echo-planar pulse sequence (TR = 2000 ms, TE = 30 ms, 35 slices oriented perpendicular to the hippocampal axis, 3 × 3 × 3 mm voxel size, 0.6 mm interslice gap, 400 volume acquisitions per run). High-resolution T1-weighted (MP-RAGE) images were collected for anatomical visualization. A vacuum pillow minimized head motion. Visual stimuli were projected onto a screen that was viewed through a mirror, and responses were collected with a magnet-compatible button box.

The scanned encoding portion of the experiment was divided into two runs. Encoding trials were intermixed with an active, sensorimotor baseline task [“arrows task” (Stark and Squire, 2001)]. Arrows that randomly pointed to the left or to the right for 1 s were repeatedly presented for the length of a baseline trial (2–12 s), and subjects had to press the left middle finger key if the arrow pointed to the left and the left index finger key if it pointed to the right. The sequence of encoding trials of each fragmentation level (F0, F2, and F4) and of variable duration baseline trials was pseudorandom and optimized for rapid event-related fMRI (Dale, 1999).

Data were analyzed using SPM5 (Wellcome Department of Cognitive Neurology, London, UK). During preprocessing, images were corrected for differences in slice acquisition timing, followed by motion correction across both runs. Structural images were coregistered to the functional images and spatially normalized to a T1 template provided in SPM5. The resulting normalization parameters were applied to the functional images, followed by spatial smoothing with a 6 mm full-width, half-maximum isotropic Gaussian kernel. Statistical analyses were performed using the general linear model (GLM) implemented in SPM5. For the main analysis, encoding trials were separated by (1) fragmentation level (F0, F2, F4) and (2) subsequent memory—old objects subsequently misclassified as new [misses (M)], objects correctly recognized as old, without remembering the associated color [object only (OO)], and objects subsequently recognized as old, including memory for the correct associated color [object and color (OC)]. Thus, this paradigm resulted in a 3 × 3 factorial design, consisting of the factors fragmentation level (F0, F2, F4) and subsequent memory (M, OO, OC). For subsidiary analyses, OO and OC trials were further separated based on high- vs low-confidence object recognition (see below).

For each of the nine conditions of interest, neural activity was modeled via delta functions at stimulus onset and convolved with a canonical hemodynamic response function together with its first-order temporal derivative. The resulting vectors were entered as regressors into a fixed-effects GLM, together with nuisance regressors modeling session means and scanner drift. Parameter estimates (β weights) for each condition of interest were derived for each subject and carried forward to a second-level group analysis. Here, individual subjects' β weights for the nine conditions of interest were entered into a repeated-measures ANOVA. As a first step, to identify—in a stringent and unbiased manner—MTL regions sensitive to any of our experimental manipulations, we used a voxelwise omnibus F test. The resulting regions were then further examined for main effects and interactions of the factors fragmentation level and subsequent memory. Next, to reveal regions sensitive to fragmentation level and subsequent memory at the whole-brain level, we used targeted ANOVAs in conjunction with masking procedures. All fMRI results were thresholded at a minimum of five contiguous voxels whose test statistic exceeds an uncorrected p value of .001. Voxel coordinates are reported in Montreal Neurological Institute (MNI) space. ANOVA results were Greenhouse–Geisser corrected for nonsphericity where appropriate (indicated via adjusted degrees of freedom).

Results

Behavioral results

Because increasing object unitization demands should increase processing time, we expected a linear increase in response times (RTs) for plausibility ratings as a function of fragmentation level. The average RTs (in seconds ± SEM) for F0, F2, and F4 trials were 1.69 (0.10), 1.81 (0.12), and 2.00 (0.11), respectively. To test whether RTs systematically varied with fragmentation level and/or subsequent memory performance, we conducted a repeated-measures ANOVA with the factors fragmentation level (F0, F2, F4) and subsequent memory (M, OO, OC). While the main effect of fragmentation level on RTs was highly significant (F(2,32) = 74.06, p < 0.001), there was no effect of subsequent memory nor a subsequent memory × fragmentation level interaction (both F < 1.59, p > 0.22). For the factor fragmentation level, pairwise t tests showed a significant stepwise increase in RTs with increasing fragmentation levels (F0 vs F2, F2 vs F4; both t(16) > 4.66, p < 0.001).

With regard to subsequent memory performance, we expected object recognition to be superior for F0 trials, as recognition probes were always shown intact/integrated and thus perceptually matched the study items presented during F0 trials, but not those presented during F2 or F4 trials. However, color memory should not be affected by fragmentation level, as subjects were instructed to apply the color to the object after successful unitization of the fragments. Overall, 79% (SEM = 3%) of old objects were correctly identified as old (hits), while 89% (SEM = 2%) of the lures were correctly identified as new (correct rejections) [d′ = 2.07 (SEM = 0.10)]. As expected, object recognition memory (hits vs misses) varied as a function of fragmentation level (F(2,32) = 6.03, p < 0.01). The proportion of hits was higher for F0 trials than for both F2 and F4 trials (both t(16) > 2.98, p < 0.01), but did not differ between F2 and F4 trials (t(16) = 0.74, p = 0.47). Importantly, there were no significant increases in color memory (following object recognition) from F0 trials to F2 trials or from F2 trials to F4 trials (both t(16) < 1.42, p > 0.16).

FMRI results

Our first approach was to identify, via an unbiased omnibus F test (using the nine conditions of interest described above), MTL regions that are sensitive to any effect (or combination of effects) in our paradigm. Shown in Figure 2a, the only MTL regions emerging from this analysis were located in bilateral PrC. Given the anatomical variability of the collateral sulcus across individuals (Insausti et al., 1998), we confirmed that the resulting clusters lie within PrC across subjects (supplemental Figs. S1, S2, available at www.jneurosci.org as supplemental material). Both peaks survived small volume correction (Worsley et al., 1996) using 15-mm-radius spheres (both pcorrected < 0.005). We then extracted the subject-specific β weights of the voxels within the resulting clusters and applied a repeated-measures ANOVA with the factors fragmentation level (F0, F2, F4), subsequent memory (M, OO, OC), and hemisphere (left, right). This analysis revealed a main effect of subsequent memory (F(1.21,19.33) = 18.60, p < 0.001), but no effect of fragmentation level (F(2,32) = 0.60, p = 0.55) or interaction between fragmentation level and subsequent memory (F(2.04,32.58) = 0.56, p = 0.58). As the factor hemisphere did not interact with subsequent memory, fragmentation level, or their interaction (all F < 1.73, p > 0.19), data from left and right PrC were pooled for further analyses. Paired two-tailed t tests showed that the main effect of subsequent memory was due to a significant stepwise increase of PrC activation from M to OO (t(16) = 3.90, p < 0.005) and from OO to OC (t(16) = 3.66, p < 0.005), linearly tracking the amount of item-related information successfully encoded (Fig. 2b). However, no difference in PrC activation was seen between F0 and F2 trials (t(16) = 0.47, p = 0.65), between F0 and F4 trials (t(16) = 0.60, p = 0.56), or between F2 and F4 trials (t(16) = 1.16, p = 0.26). Critically, this result strongly suggests that the fragmentation level and, thus, the demand on unitization did not influence activation of PrC. However, replicating our previous findings (Staresina and Davachi, 2006, 2008) and extending them to using real object images, PrC activation was systematically modulated by the amount of item-related information successfully encoded.

Figure 2.

Figure 2.

Open in a new tab

PrC encoding activation. a, Bilateral perirhinal clusters (shown on the mean anatomical scan across subjects) were identified via an unbiased omnibus F test. b, Effects of fragmentation level and subsequent memory in the resulting clusters (collapsed across left and right PrC). While PrC activation was not sensitive to the demands on perceptual unitization, it was systematically modulated by the amount of item-related information successfully encoded. *p < 0.05 (paired, two-tailed t test); n.s., not significant.

To ensure that the absence of unitization effects in PrC was not due to low statistical power resulting from a reduced signal-to-noise ratio in this region (Ojemann et al., 1997; Olman et al., 2009), we dropped the statistical threshold for the effect of fragmentation level to p < 0.1 (uncorrected). However, no significant clusters emerged in PrC even at this reduced threshold.

Next, we examined whether other brain regions were sensitive to the level of object fragmentation. That is, where else in the brain does object unitization occur, and are there regions that are sensitive to both unitization and successful encoding (i.e., show overlapping effects)? To isolate the effects of fragmentation level, subsequent memory, and their respective overlap, we applied a three-step masking procedure. First, within the same whole-brain repeated-measures ANOVA reported above, we computed the main effect of fragmentation level (p < 0.001 uncorrected, minimum of five contiguous voxels) and masked out voxels that also showed an effect of subsequent memory at the liberal threshold of p < 0.1 (uncorrected). It should be mentioned that when exclusively masking a target contrast with another contrast, using a liberal threshold for the contrast used as an exclusive mask renders the assessment of regions selectively sensitive to the target contrast more conservative. Second, the inverse procedure was used to reveal regions sensitive to successful episodic encoding but not to unitization. Note that we also excluded regions from the analysis showing negative subsequent memory effects, i.e., greater activation for objects later forgotten compared to objects later remembered. Third, regions showing overlapping effects of fragmentation level and subsequent memory were identified by inclusively masking the main effect of subsequent memory with the main effect of fragmentation level. Figure 3 depicts the results of this three-step analysis. While the effects of fragmentation level were significant in posterior, visual cortical regions (Fig. 3a, bottom), more anterior regions, including fusiform gyrus, were sensitive to both fragmentation level and subsequent memory (Fig. 3a, middle). Finally, consistent with previous fMRI studies on episodic memory formation (Wagner et al., 1998; Kirchhoff et al., 2000; Staresina and Davachi, 2006), even more anterior regions, including bilateral inferior temporal/anterior fusiform gyrus, left PrC, and bilateral inferior frontal gyrus, were selectively sensitive to subsequent memory, but not to fragmentation level (Fig. 3a, top) [note that the right PrC cluster identified in the omnibus F test above also emerges in this analysis, albeit at the slightly reduced threshold of p < 0.003]. For illustrative purposes, we visualize this transition along ventral temporal cortex from (1) regions only showing effects of object fragmentation to (2) regions only showing subsequent memory effects by liberally thresholding the respective statistical maps at p < 0.05 (uncorrected) (Fig. 3c). Although this gradient is reminiscent of processing hierarchies across ventral temporal cortex for increasingly complex visual stimuli (Desimone et al., 1984; Ungerleider and Haxby, 1994; Tanaka, 1996), our data reveal a gradient not within the “cognitive domain” of perceptual unitization, but across the cognitive domains of perceptual unitization and episodic memory formation.

Figure 3.

Figure 3.

Open in a new tab

Cortical stages of unitization and memory formation. a, Effects of successful memory formation (top) and of fragmentation level (bottom) and their overlap (middle) revealed via whole-brain ANOVAs and masking procedures (see text for details). b, Pattern of activation (represented via β weights) as a function of fragmentation level and subsequent memory in the respective ventral temporal maxima clusters (peak voxel including 3-mm-radius sphere). c, To illustrate the posterior/anterior transition from unitization to memory encoding, the individual effects in a were thresholded at p < 0.05 (red, selective effects of fragmentation level; green, overlapping effects of fragmentation level and subsequent memory; blue, selective effects of subsequent memory). *p < 0.05 (paired, two-tailed t test); n.s., not significant.

Finally, a formal whole-brain interaction analysis revealed only one cluster located in the left insula (peak MNI coordinates −45, 3, 9). However, this interaction was due to a crossover in the direction of subsequent memory effects for F2 and F4 trials (increase from M to OO to OC for F4 trials (F(2,32) = 6.99, p < 0.005), with a corresponding decrease for F2 and F0 trials), but there was no systematically increasing subsequent memory effect across increasing fragmentation levels. Importantly, these results confirm that we had power to reveal regions involved in object unitization in our paradigm. However, our data suggest that object unitization is accomplished in processing stages before PrC. This suggests that the object input to PrC has already been integrated and implicates a role of PrC in the encoding of these representations into memory. Of course, it is possible that the fragmentation manipulation used herein engenders additional processing demands. For instance, presenting two or four perceptual fragments is likely to increase demands on visuospatial attention. Attention can, in turn, modulate activation in posterior visual cortical regions (Pessoa et al., 2003; Buffalo et al., 2010). However, even with the added demands on attention and increased response times, PrC did not show increased activation with increasing levels of fragmentation.

Finally, as mentioned in the Introduction, PrC has been strongly implicated in supporting familiarity-based recognition (Diana et al., 2007; Eichenbaum et al., 2007). Given that we assessed object familiarity (high- and low-confidence object recognition) and color recollection (indicating the color label out of four choices) in two independent steps, our paradigm allowed us to test whether PrC activation correlates only with later familiarity-based object recognition, or whether successfully binding the associated color recruits additional PrC engagement. We thus further separated OO trials and OC trials by the levels of object recognition confidence before color assessment, resulting in “object only” trials in which the object was recognized with low confidence (OO_lc), “object only” trials in which the object was recognized with high confidence (OO_hc), and “object and color” trials in which the object was recognized with high confidence (OC_hc) [there were not enough cases in which low-confidence object recognition was followed by correct color recollection (OC_lc) to be included in the analysis]. If PrC engagement were restricted to supporting subsequent familiarity-based recognition, we would expect enhanced activation for OO_hc and OC_hc trials (both reflecting high-confidence object recognition) relative to OO_lc trials, but no additional increase for OC_hc relative to OO_hc trials (because the level of object confidence, and hence, familiarity is the same). Alternatively, PrC may not be directly modulated by the levels of subsequent familiarity, but only by the amount of item-related event details encoded. In that case, we would expect no difference between OO_lc and OO_hc trials (both reflecting memory for the object without memory for the associated color), but enhanced activation for OC_hc trials (where object and an additional object detail were encoded). After confirming a significant main effect of subsequent memory (OO_lc, OO_hc, OC_hc) (F(1.90,18.32) = 6.02, p < 0.05), paired two-tailed t tests revealed a significant increase from OO_lc to OO_hc (t(16) = 2.17, p < 0.05) as well as from OO_hc to OC_hc (t(16) = 2.32, p < 0.05). These results are consistent with the notion that PrC activation correlates with subsequent levels of familiarity, but also indicate that additional binding of item-related details (an associated color) yields an additional increase in PrC engagement (Fig. 4). It should be emphasized that in the test part of our current paradigm, we (1) used four different colors, (2) deliberately chose verbal labels of the possible colors instead of presenting object and color in the same perceptual format as during encoding, and (3) allowed for “?” responses instead of having subjects guess the correct color. While familiarity-based recognition is thought to drive performance in forced-choice tests, open test formats (e.g., source tests allowing for “?” responses) are thought to bias responses toward a recollection-based memory process rather than toward a bottom-up familiarity signal (Mandler, 1980; Jacoby, 1991; Holdstock et al., 2002; Norman and O'Reilly, 2003). Thus, we would argue that our experimental assessment of associative color memory is unlikely to rely solely on familiarity-based recognition.

Figure 4.

Figure 4.

Open in a new tab

PrC encoding beyond familiarity-based recognition. OO trials were split into those with high-confidence object recognition (“sure old”) and those with low-confidence object recognition (“unsure old”). Both object familiarity and recollection of an item-related detail (OC) modulate PrC encoding activation. *p < 0.05 (paired, two-tailed t test).

Discussion

This study was designed to further elucidate the relationship between unitization and memory formation. In particular, we assessed whether regions of PrC associated with episodic encoding of item-related event details would also be modulated by the demands to fuse perceptual stimulus components. The demands on unitization were modulated by presenting object images in varying degrees of fragmentation, while the amount of item-related memory encoding was assessed by probing subsequent object recognition along with associative color memory (Fig. 1). Consistent with prior work, we found that the extent of successful item-related encoding systematically modulated PrC engagement (supporting object recognition as well as memory for an associated color) (Figs. 2, 4). However, this region remained unaffected by increasing demands on unitization in our paradigm. Instead, our data suggest that object unitization is accomplished in prior processing stages along the ventral visual pathway (Fig. 3), revealing a posterior–anterior gradient with posterior visual cortical regions showing modulation by object fragmentation and anterior regions, including perirhinal cortex, showing modulation by successful memory formation and regions in between showing both patterns.

PrC and unitization

As mentioned in the Introduction, it has been challenging to reconcile recent findings of PrC contributions to associative memory formation (Staresina and Davachi, 2006, 2008) with the established role of this region in supporting nonassociative or familiarity-based recognition (Davachi et al., 2003; Ranganath et al., 2004; Uncapher and Rugg, 2005; Kensinger and Schacter, 2006; Dougal et al., 2007). Given that associative encoding effects in PrC were found to be restricted to binding item-related elements [such as a color in which an object had to be imagined (Staresina and Davachi, 2008)], one intriguing possibility that has been offered is that PrC supports the associative unitization of episodic details (Diana et al., 2008), where unitization denotes the representation of disparate elements (items or stimulus components) as a single/integrated unit (LaBerge and Samuels, 1974; Graf and Schacter, 1989). One appeal of this proposal is that it bridges associative encoding effects in PrC with this region's putative role in supporting familiarity rather than recollection (Diana et al., 2007; Eichenbaum et al., 2007), as unitization has been shown to enhance subsequent measures of familiarity during associative recognition (Yonelinas et al., 1999; Quamme et al., 2007). Indeed, recent neuropsychological data have shown that associative memory, typically linked to hippocampal functioning (Mayes et al., 2007), is relatively preserved after hippocampal damage if the associations are processed in a unitized manner (Giovanello et al., 2006; Quamme et al., 2007). Showing that PrC activation was enhanced during unitization of two unrelated words (relative to separate processing), a recent fMRI study by Haskins et al. (2008) further corroborated the idea that PrC supports associative memory by means of unitization.

Upon closer examination, however, it is not clear whether (1) PrC mediates the process of unitization, or (2) whether PrC supports episodic encoding of representations resulting from unitization. That is, in the study by Haskins et al. (2008), subjects were instructed to create a new representation (e.g., “steamtoken”) from two individual representations (“steam” and “token”) in the unitization condition, which results in a conceptually novel object representation being processed during that trial. Thus, enhanced involvement of PrC in processing and encoding of word pairs in the unitized condition may reflect an increase in the amount of objects or item-related details being encoded. Similarly, the neuropsychological findings mentioned above (showing relatively spared associative memory for unitized representations after hippocampal damage) do not show that unitization is accomplished by PrC, but indicate that the hippocampus is not needed for (1) the process of unitization and/or (2) encoding of unitized representations. However, the creation of unitized representation may well be accomplished in areas other than PrC.

By holding the number and novelty of objects being processed constant while controlling (1) the demands on unitization [three levels of fragmentation (F0, F2, F4)] and (2) the amount of item-related details successfully encoded [forgetting, recognition of the object, additional memory for an associated color (M, OO, OC)] (Fig. 1), our data are consistent with the notion that PrC contributes to episodic encoding of item-related event details (Fig. 2) but do not show that activation is modulated by unitization demands. Importantly, however, fragmentation level and the ensuing demand on unitization directly modulated engagement of regions in preceding processing stages along the ventral visual pathway (Fig. 3). These results suggest that unitization may occur in processing stages before PrC and that PrC primarily supports episodic encoding of items (objects) and item-related event details (an associated object color).

However, one potentially critical difference between our current paradigm and the study by Haskins et al. (2008) is that we operationalized unitization as establishing a perceptually intact object representation as opposed to new semantic concepts (the concept of a “steamtoken” in the example above). If not for unitization per se, PrC may play a significant role in mediating conceptual representations, consistent with findings of conceptual repetition suppression (reduced response to repeated processing of semantic information) (O'Kane et al., 2005) or cross-modal integration (sensitivity to overlapping features) (Taylor et al., 2006) in this region. Although one advantage of the present paradigm is that it controls for this factor given that the fused representation does not lead to an additional concept being created, future studies are needed to clarify whether PrC specifically contributes to semantic unitization/generation rather than to perceptual unitization.

PrC and familiarity

Previous work has consistently shown that PrC activation correlates with later familiarity-based recognition [for review, see Diana et al. (2007) and Eichenbaum et al. (2007)]. However, its contribution to later familiarity may only be a particular subcomponent of this region's role in memory formation. In fact, one could consider that, from a mechanistic perspective, it is unlikely that a single brain region is primarily concerned with a high-level phenomenological state such as the introspective level of item familiarity. Instead, the factor likely modulating PrC engagement is the amount of item-related event details successfully encoded. As familiarity is thought to be largely supported by the mnemonic availability of item details (Mandler, 1980), the finding that PrC supports later familiarity is entirely consistent with this view. Importantly, though, familiarity may be a byproduct of item-related encoding in PrC, not its ultimate contribution to memory formation. Accordingly, the present data, along with previous work (Staresina and Davachi, 2006, 2008; Awipi and Davachi, 2008) show that PrC also supports memory for item-related details that are later recollected (beyond item familiarity) (Fig. 4).

Is it possible that additional memory for the associated color merely enhanced item familiarity, leaving familiarity as the critical dimension for PrC engagement? Indeed, in an elegant recent study Diana et al. (2008) showed, by manipulating the task instructions, that encoding the same event detail (color) as an item-related feature impacts familiarity parameters for subsequent color memory more than does encoding it as contextual information (derived from ROC curves). However, that does not rule out the possibility that binding the color as an item-related detail/feature also contributes to recollection. In fact, in the study by Diana and et al. (2008), the parameter estimates of color recollection were consistently above zero both when color was made an item feature and when it served to indicate a context (and in some cases even greater when made an item feature), suggesting that item–color associations may well elicit recollection (see also Haskins et al., 2008). In sum, we suggest that the contribution of PrC to memory formation might be best characterized as supporting subsequent memory for item-related event details, which may encompass, but is not restricted to, enhanced feelings of familiarity.

PrC and non-mnemonic aspects of cognition

Finally, our current results also bear on the debated role of PrC in cognitive processes beyond memory [see Baxter (2009) and Suzuki (2009) for recent discussion]. Briefly, it has been proposed that the role of PrC is not restricted to episodic memory, but may also support perception and visual discrimination by solving “feature ambiguity” (Bussey and Saksida, 2005; Murray et al., 2007). Object fragmentation resulted, as expressed in RTs, in increased perceptual difficulty in our current paradigm, but PrC activation was not sensitive to this factor. However, given that featural overlap across trials did not necessarily increase with object fragmentation, this is not a well suited paradigm to test the proposed contribution of PrC to perception. Nevertheless, resolving feature ambiguity may be a viable account for color binding effects observed in PrC. That is, feature ambiguity across trials in our paradigm may increase because we use four repeating colors throughout the study. Although methodologically challenging in our current paradigm, it would be interesting to assess whether PrC shows similar mnemonic color-binding effects when using trial-unique colors. Likewise, PrC may contribute to establishing sufficiently separate complex/conjunctive “gestalts,” which would render the item representation more resistant to interference (Cowell et al., 2006). Thus, the more elaborate the object representation/gestalt is, the less net overlap with potentially interfering objects/features. In sum, although “item-related episodic encoding” may be the most parsimonious account of PrC contributions, the underlying mechanisms remain a challenge for future investigation.

Footnotes

This work was supported by National Institute of Mental Health Grant R01MH074692 to L.D.

References

  1. Aggleton JP, Brown MW. Episodic memory, amnesia, and the hippocampal-anterior thalamic axis. Behav Brain Sci. 1999;22:425–444. discussion 444–489. [PubMed] [Google Scholar]
  2. Awipi T, Davachi L. Content-specific source encoding in the human medial temporal lobe. J Exp Psychol Learn Mem Cogn. 2008;34:769–779. doi: 10.1037/0278-7393.34.4.769. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Baxter MG. Involvement of medial temporal to be structures in memory and perception. Neuron. 2009;61:667–677. doi: 10.1016/j.neuron.2009.02.007. [DOI] [PubMed] [Google Scholar]
  4. Buffalo EA, Fries P, Landman R, Liang H, Desimone R. A backward progression of attentional effects in the ventral stream. Proc Natl Acad Sci U S A. 2010;107:361–365. doi: 10.1073/pnas.0907658106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bussey TJ, Saksida LM. Object memory and perception in the medial temporal lobe: an alternative approach. Curr Opin Neurobiol. 2005;15:730–737. doi: 10.1016/j.conb.2005.10.014. [DOI] [PubMed] [Google Scholar]
  6. Cowell RA, Bussey TJ, Saksida LM. Why does brain damage impair memory? A connectionist model of object recognition memory in perirhinal cortex. J Neurosci. 2006;26:12186–12197. doi: 10.1523/JNEUROSCI.2818-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Dale AM. Optimal experimental design for event-related fMRI. Hum Brain Mapp. 1999;8:109–114. doi: 10.1002/(SICI)1097-0193(1999)8:2/3&#x0003c;109::AID-HBM7&#x0003e;3.0.CO;2-W. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Davachi L. Item, context and relational episodic encoding in humans. Curr Opin Neurobiol. 2006;16:693–700. doi: 10.1016/j.conb.2006.10.012. [DOI] [PubMed] [Google Scholar]
  9. Davachi L, Mitchell JP, Wagner AD. Multiple routes to memory: distinct medial temporal lobe processes build item and source memories. Proc Natl Acad Sci U S A. 2003;100:2157–2162. doi: 10.1073/pnas.0337195100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Desimone R, Albright TD, Gross CG, Bruce C. Stimulus-selective properties of inferior temporal neurons in the macaque. J Neurosci. 1984;4:2051–2062. doi: 10.1523/JNEUROSCI.04-08-02051.1984. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Diana RA, Yonelinas AP, Ranganath C. Imaging recollection and familiarity in the medial temporal lobe: a three-component model. Trends Cogn Sci. 2007;11:379–386. doi: 10.1016/j.tics.2007.08.001. [DOI] [PubMed] [Google Scholar]
  12. Diana RA, Yonelinas AP, Ranganath C. The effects of unitization on familiarity-based source memory: testing a behavioral prediction derived from neuroimaging data. J Exp Psychol Learn Mem Cogn. 2008;34:730–740. doi: 10.1037/0278-7393.34.4.730. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Dougal S, Phelps EA, Davachi L. The role of medial temporal lobe in item recognition and source recollection of emotional stimuli. Cogn Affect Behav Neurosci. 2007;7:233–242. doi: 10.3758/cabn.7.3.233. [DOI] [PubMed] [Google Scholar]
  14. Eichenbaum H, Yonelinas AP, Ranganath C. The medial temporal lobe and recognition memory. Annu Rev Neurosci. 2007;30:123–152. doi: 10.1146/annurev.neuro.30.051606.094328. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Giovanello KS, Keane MM, Verfaellie M. The contribution of familiarity to associative memory in amnesia. Neuropsychologia. 2006;44:1859–1865. doi: 10.1016/j.neuropsychologia.2006.03.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Graf P, Schacter D. Unitization and grouping mediate dissociations in memory for new associations. J Exp Psychol Learn Mem Cogn. 1989;15:930–940. doi: 10.1037//0278-7393.15.1.3. [DOI] [PubMed] [Google Scholar]
  17. Haskins AL, Yonelinas AP, Quamme JR, Ranganath C. Perirhinal cortex supports encoding and familiarity-based recognition of novel associations. Neuron. 2008;59:554–560. doi: 10.1016/j.neuron.2008.07.035. [DOI] [PubMed] [Google Scholar]
  18. Henson R. A mini-review of fMRI studies of human medial temporal lobe activity associated with recognition memory. Q J Exp Psychol B. 2005;58:340–360. doi: 10.1080/02724990444000113. [DOI] [PubMed] [Google Scholar]
  19. Holdstock JS, Mayes AR, Roberts N, Cezayirli E, Isaac CL, O'Reilly RC, Norman KA. Under what conditions is recognition spared relative to recall after selective hippocampal damage in humans? Hippocampus. 2002;12:341–351. doi: 10.1002/hipo.10011. [DOI] [PubMed] [Google Scholar]
  20. Insausti R, Juottonen K, Soininen H, Insausti AM, Partanen K, Vainio P, Laakso MP, Pitkänen A. MR volumetric analysis of the human entorhinal, perirhinal, and temporopolar cortices. AJNR Am J Neuroradiol. 1998;19:659–671. [PMC free article] [PubMed] [Google Scholar]
  21. Jacoby LL. A process dissociation framework: separating automatic from intentional uses of memory. J Mem Lang. 1991;30:513–541. [Google Scholar]
  22. Kensinger EA, Schacter DL. Amygdala activity is associated with the successful encoding of item, but not source, information for positive and negative stimuli. J Neurosci. 2006;26:2564–2570. doi: 10.1523/JNEUROSCI.5241-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Kirchhoff BA, Wagner AD, Maril A, Stern CE. Prefrontal-temporal circuitry for episodic encoding and subsequent memory. J Neurosci. 2000;20:6173–6180. doi: 10.1523/JNEUROSCI.20-16-06173.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Konkel A, Cohen NJ. Relational memory and the hippocampus: representations and methods. Front Neurosci. 2009;3:166–174. doi: 10.3389/neuro.01.023.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. LaBerge D, Samuels S. Toward a theory of automatic information processing in reading. Cogn Psychol. 1974;6:293–323. [Google Scholar]
  26. Mandler G. Recognizing: the judgment of previous occurrence. Psychol Rev. 1980;87:252–271. [Google Scholar]
  27. Mayes A, Montaldi D, Migo E. Associative memory and the medial temporal lobes. Trends Cogn Sci. 2007;11:126–135. doi: 10.1016/j.tics.2006.12.003. [DOI] [PubMed] [Google Scholar]
  28. Murray EA, Bussey TJ, Saksida LM. Visual perception and memory: a new view of medial temporal lobe function in primates and rodents. Annu Rev Neurosci. 2007;30:99–122. doi: 10.1146/annurev.neuro.29.051605.113046. [DOI] [PubMed] [Google Scholar]
  29. Norman KA, O'Reilly RC. Modeling hippocampal and neocortical contributions to recognition memory: a complementary-learning-systems approach. Psychol Rev. 2003;110:611–646. doi: 10.1037/0033-295X.110.4.611. [DOI] [PubMed] [Google Scholar]
  30. Ojemann JG, Akbudak E, Snyder AZ, McKinstry RC, Raichle ME, Conturo TE. Anatomic localization and quantitative analysis of gradient refocused echo-planar fMRI susceptibility artifacts. Neuroimage. 1997;6:156–167. doi: 10.1006/nimg.1997.0289. [DOI] [PubMed] [Google Scholar]
  31. O'Kane G, Insler RZ, Wagner AD. Conceptual and perceptual novelty effects in human medial temporal cortex. Hippocampus. 2005;15:326–332. doi: 10.1002/hipo.20053. [DOI] [PubMed] [Google Scholar]
  32. Olman CA, Davachi L, Inati S. Distortion and signal loss in medial temporal lobe. PLoS One. 2009;4:e8160. doi: 10.1371/journal.pone.0008160. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Pessoa L, Kastner S, Ungerleider LG. Neuroimaging studies of attention: from modulation of sensory processing to top-down control. J Neurosci. 2003;23:3990–3998. doi: 10.1523/JNEUROSCI.23-10-03990.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Quamme JR, Yonelinas AP, Norman KA. Effect of unitization on associative recognition in amnesia. Hippocampus. 2007;17:192–200. doi: 10.1002/hipo.20257. [DOI] [PubMed] [Google Scholar]
  35. Ranganath C, Yonelinas AP, Cohen MX, Dy CJ, Tom SM, D'Esposito M. Dissociable correlates of recollection and familiarity within the medial temporal lobes. Neuropsychologia. 2004;42:2–13. doi: 10.1016/j.neuropsychologia.2003.07.006. [DOI] [PubMed] [Google Scholar]
  36. Squire LR, Stark CE, Clark RE. The medial temporal lobe. Annu Rev Neurosci. 2004;27:279–306. doi: 10.1146/annurev.neuro.27.070203.144130. [DOI] [PubMed] [Google Scholar]
  37. Staresina BP, Davachi L. Differential encoding mechanisms for subsequent associative recognition and free recall. J Neurosci. 2006;26:9162–9172. doi: 10.1523/JNEUROSCI.2877-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Staresina BP, Davachi L. Selective and shared contributions of the hippocampus and perirhinal cortex to episodic item and associative encoding. J Cogn Neurosci. 2008;20:1478–1489. doi: 10.1162/jocn.2008.20104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Stark CE, Squire LR. When zero is not zero: the problem of ambiguous baseline conditions in fMRI. Proc Natl Acad Sci U S A. 2001;98:12760–12766. doi: 10.1073/pnas.221462998. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Suzuki WA. Perception and the medial temporal lobe: evaluating the current evidence. Neuron. 2009;61:657–666. doi: 10.1016/j.neuron.2009.02.008. [DOI] [PubMed] [Google Scholar]
  41. Tanaka K. Inferotemporal cortex and object vision. Annu Rev Neurosci. 1996;19:109–139. doi: 10.1146/annurev.ne.19.030196.000545. [DOI] [PubMed] [Google Scholar]
  42. Taylor KI, Moss HE, Stamatakis EA, Tyler LK. Binding crossmodal object features in perirhinal cortex. Proc Natl Acad Sci U S A. 2006;103:8239–8244. doi: 10.1073/pnas.0509704103. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Uncapher MR, Rugg MD. Encoding and the durability of episodic memory: a functional magnetic resonance imaging study. J Neurosci. 2005;25:7260–7267. doi: 10.1523/JNEUROSCI.1641-05.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Ungerleider LG, Haxby JV. ‘What’ and ‘where’ in the human brain. Curr Opin Neurobiol. 1994;4:157–165. doi: 10.1016/0959-4388(94)90066-3. [DOI] [PubMed] [Google Scholar]
  45. Wagner AD, Schacter DL, Rotte M, Koutstaal W, Maril A, Dale AM, Rosen BR, Buckner RL. Building memories: remembering and forgetting of verbal experiences as predicted by brain activity. Science. 1998;281:1188–1191. doi: 10.1126/science.281.5380.1188. [DOI] [PubMed] [Google Scholar]
  46. Worsley KJ, Marrett S, Neelin P, Vandal AC, Friston KJ, Evans AC. A unified statistical approach for determining significant signals in images of cerebral activation. Hum Brain Mapp. 1996;4:58–73. doi: 10.1002/(SICI)1097-0193(1996)4:1<58::AID-HBM4>3.0.CO;2-O. [DOI] [PubMed] [Google Scholar]
  47. Yonelinas AP, Kroll NE, Dobbins IG, Soltani M. Recognition memory for faces: when familiarity supports associative recognition judgments. Psychon Bull Rev. 1999;6:654–661. doi: 10.3758/bf03212975. [DOI] [PubMed] [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES