Recollection versus strength as the primary determinant of hippocampal engagement at retrieval

Abstract

We examined whether hippocampal activity in recognition relates to the strength of the memory or to recollective experience, a subject of considerable current debate. Participants studied word pairs and then made two successive recognition decisions on each item: first on the uncued target and then on the target presented with the studied cue word. We compared recollection and familiarity patterns of activation in fMRI for these decisions. Critically, our analyses attempted in two ways to equate perceived memory strength while varying the associative information available. First, activity for targets judged familiar before cueing was contrasted with activity for the same items in the second decision as a function of whether the targets converted to recollection or remained familiar when the context cues were provided. We found increased hippocampal activity following cueing only with recollective conversion. Second, we investigated whether hippocampal activity was modulated by the rated familiarity strength of cued items or whether it increased uniquely in recollection. Hippocampal activation was not modulated parametrically by familiarity strength and recollected items were associated with greater activity relative to highly familiar items. Together, our results support the notion that it is recollection of context, rather than memory strength, that underlies hippocampal engagement at retrieval.

The precise role of the hippocampus in recognition memory is contested. According to the dual-process view of recognition, the hippocampus is crucial to recollection, which is a process that supports retrieval of contextual information, but not to familiarity, which is a process that leads to a feeling that an item was previously encountered without retrieval of contextual details (1, 2). Unitary accounts of recognition propose that there is a single process and that hippocampal involvement is related to memory strength, irrespective of whether or not recognition is accompanied by retrieval of context (3). Although both views can account for the majority of functional neuroimaging results at encoding and retrieval (4), there are fundamental differences in their respective emphases on key memory concepts including stored memory representations, memory processes, and associated conscious experience. By the recollection view, the critical variable underlying hippocampal engagement is retrieval of contextual information. The strength view emphasizes memory representation, such that retrieval of strongly encoded memories is likely to engage the hippocampus, irrespective of recovery of context. It is difficult to adjudicate between these proposals, as recollection is typically associated with strong memories and strong memories often include contextual information.

Ideally, one would compare hippocampal activation related to subjective familiarity and recollection while minimizing differences in strength. We attempted to do this in two ways. First, we kept the target (and presumably the memory strength) constant and provided cues that could potentially transform the memory experience from familiarity-based to recollection-based recognition across two successive decisions. We modeled this after the everyday experience in which one sees a target individual in an unfamiliar setting [e.g., the butcher on the bus (5)] and is unable to recollect anything about previous encounters despite a strong conviction that the person is highly familiar. With minimal appropriate cues (e.g., from conversation or context), one can now readily access a full-blown recollection of the individual. Although there have been numerous experiments attempting to elucidate the effects of various independent variables (such as experimental conditions, aging, or brain lesions) on familiarity and recollection (for reviews, see refs. 2, 6, and 7), none has directly examined the “conversion” of the memory experience with cueing and its possible hippocampal correlate.

Second, we compared hippocampal activation for items of the same perceived strength before cueing but different associative properties (i.e., cohorts of items judged highly familiar with and without recollection), a strategy used by others. As a more powerful statistical control, we also used masking to exclude brain regions that were modulated by degree of familiarity. This was an attempt to minimize, as much as possible in a study comparing different items, the influence of variations in strength as a contributing factor to increased hippocampal activation typically observed with the recollection of context. Any remaining activation in the hippocampus would therefore have to be attributed to the effect of recollection.

In the present experiment (see Fig. 1 for schematic), participants studied pairs of words (A–B) and were scanned while performing a recognition task for the first member of the pairs (A). There were two types of trials at test: (i) uncued trials that included studied (A) and new (C) words, and (ii) cued trials that included intact pairs (A–B) and new–old pairs (C–B). For each trial type (uncued and cued), participants made memory decisions for the target items using a variant of the Remember-Know procedure. This procedure was initially described by Tulving (8) and adapted by Gardiner (9) to capture the subjective experience of familiarity and recollection and has been used in many subsequent studies to show dissociations based on a variety of factors including hippocampal damage (for reviews, see refs. 6 and 7). The version used here, identical to one used by Yonelinas and colleagues (10), includes a subjective recollection response (R), as well as a four-point confidence rating scale for nonrecollected items (1 = sure new, 2 = unsure new, 3 = unsure familiar, 4 = sure familiar). It has been established that carefully instructing participants regarding the familiarity/recollection judgment is critical to ensure that the decision is not simply based on confidence; otherwise it is possible that they may simply treat R as “even more sure familiar” (11). Here, we stressed that recollected items and highly familiar items (rated “4” on the confidence scale) should differ in terms of the presence or absence of retrieved contextual information, but not in terms of confidence or strength. Participants were required to give justifications for their responses during a practice session to ensure that they were applying this operational definition appropriately. Critically, these variables (cueing condition and response type) afford means of testing specific differential predictions of the strength and recollection views with respect to hippocampal engagement (see Table 1).

Fig. 1.

Examples of test trials.

Table 1.

Predicted hippocampal activation according to the strength and recollection views

Analyses	Contrasts	Strength predictions	Recollection predictions
1. Same items (cueing effect)
Remain familiar	cFF > uFF	= or +	=
Conversion to recollection	cFR > uFR	=	+
Initial familiarity	uFR > uFF	+	=
2. Same subjective confidence
Familiarity strength	1→4	+	=
Recollection	cR > cF4	+	+
Recollection excluding familiarity strength	cR > cF4 masked (1→4)	=	+

u indicates uncued recognition trials and c indicates cued recognition trials. FF refers to items that are familiar on cued and uncued trials; FR are items that are familiar before cueing and recollected after cueing. 1 to 4 refers to the familiarity ratings given to cued trials and R indicates recollected. + or = describes predicted hippocampal activation patterns resulting from the specified contrasts for each theory.

In the first set of analyses, we compared brain activation for uncued target items judged as familiar (rated 3 or 4) on the first recognition decision (uFF and uFR) to activity in the subsequent cued trial as a function of whether they remained familiar (cFF) or “converted” to recollection (cFR). Specific examples of these item types are presented in Fig. 1. Although the overall memory strength of those familiar targets might increase when appropriate context is provided (i.e., all c > all u conditions), the strength view makes no differential prediction for items that remain familiar versus those that give rise to a recollective experience as the nominal cues are identical (i.e., the original studied pairs). In a similar vein, if strength of the encoded representation predicts hippocampal activation at retrieval (4, 12), we might expect greater activity on initial uncued recognition trials for familiar items that subsequently convert (uFR) in comparison to those that remain familiar (uFF). In contrast, should conversion represent engagement of different memory processes resulting in different conscious experiences, the recollection view predicts increased hippocampal activation following conversion but not when items remain familiar, and no difference between the two types of uncued trials, which are both judged to be familiar.

In the second set of analyses, we compared brain activation for cued targets judged as highly familiar (cF4) to that of cued items declared recollected (cR) while excluding brain regions positively modulated by familiarity strength (4 > 3 > 2 > 1). Whilst both the recollection and strength view predict increased hippocampal activation for recollected items relative to highly familiar ones, their predictions differ once regions that track familiarity strength are excluded. The recollection view posits that hippocampal engagement would be unaffected by this statistical manipulation, as there is no relationship between familiarity strength and hippocampal signal (10, 13, 14). In contrast, the strength view predicts that hippocampal engagement would be substantially attenuated, because memory confidence is the major determinant of activation there (3, 4, 12).

Results

Behavior.

The proportions of response types per item types are presented in Table 2. In keeping with previous findings (15), cueing enhanced recognition memory accuracy by increasing the hit rate (uncued targets M = 0.83, SD = 0.11; cued targets M = 0.91 SD = 0.09; t = 3.12, P < 0.01) and decreasing the false alarm rate (uncued lures: M = 0.36, SD = 0.15; cued lures M = 0.30, SD = 0.17; t = 3.83, P < 0.005). Furthermore, cueing increased the proportion of recollected answers (uncued M = 0.19, SD = 0.16; cued M = 0.55, SD = 0.20; t = 15.64, P < 0.001) and decreased the proportion of familiar responses (combined unsure and sure familiar: uncued M = 0.64, SD = 0.17; cued M = 0.38, SD = 0.18; t = 9.25, P < 0.001). As anticipated, cueing strongly enhanced recollection.

Table 2.

Proportion of response type per item type

Item type	Sure new (1)	Unsure new (2)	Unsure familiar (3)	Sure familiar (4)	Recollected (R)
Uncued target	0.03 (0.03)	0.13 (0.06)	0.25 (0.12)	0.38 (0.15)	0.19 (0.16)
Uncued lure	0.28 (0.15)	0.36 (0.07)	0.27 (0.11)	0.09 (0.12)	0.01 (0.01)
Cued target	0.02 (0.02)	0.05 (0.04)	0.12 (0.09)	0.26 (0.14)	0.55 (0.20)
Cued lure	0.37 (0.20)	0.32 (0.07)	0.21 (0.12)	0.08 (0.08)	0.01 (0.02)

Total target items = 120; total lures = 60; n = 13

With cueing, an average of 34.31 (range 17–51) targets converted from familiarity to recollection and 34.54 (range 11–72) target items remained familiar with cueing. There was also an average of 30.15 (range 15–47) lures that remained correctly rejected. Only cued items were used for the confidence analyses. There was an average of 58.00 (range 33–95) recollected cued targets and 32.45 (range 13–68) sure familiar items (F4). For the parametric analyses of familiarity, there was an average of 27.91 (range 5–85) items (including targets and lures) across responses 1 to 4.

fMRI Results.

To investigate the effect of cueing, we used fours trial types in planned contrasts of the fMRI data (uFR: uncued familiar items subsequently recollected; cFR: converted cued items; uFF: uncued familiar items subsequently familiar; cFF: cued familiar items which remained familiar). The contrasts specified the conversion from familiarity to recollection (cFR>uFR), the effect of cueing on items that remain familiar (cFF>uFF) and the baseline familiarity of items that remain familiar or convert to recollection (uFR>uFF). We also included lures that were consistently rejected with and without cues (uNN: uncued new items; cNN: cued new items) in our hippocampal ROI analyses. These were included to provide a baseline indexing novelty or low familiarity (uNN) and to control for familiarity or recollection related to the cues rather than to the target items (cNN). Of note, all cues were previously studied and may have engaged both familiarity and recollection processes but participants were instructed to make decisions only on the target item.

To isolate recollection from familiarity across cohorts of items, we contrasted recollected cued targets (cR) and sure familiar cued targets (cF4) while excluding brain regions that were modulated by familiarity strength. Specifically, we first conducted a positive linear* parametric analysis using all items receiving ratings from 1 to 4 (without recollection responses). We then used results of this analysis as an exclusive mask in our cR>cF4 contrast [see Vilberg and Rugg for the use of a similar masking method (16, 17)]. In this analysis, activation in the hippocampus would be attributable to retrieval of contextual information, because the contribution of increasing strength has been removed. We present two sets of ROI data; one that includes the above item types (cR and cF4), and one that includes all ratings of the cued items (ratings from 1 to 4 and recollection).

Whole brain results are presented in the SI Text and in supplementary tables (Tables S1, S2, and S3), which is published on the PNAS website. Results pertaining to the hippocampus are presented in Table 3 and Fig. 2. From the cueing/conversion analyses, we found increased hippocampal activation only for items that became recollected with cueing (cFR>uFR) and not for items that remained familiar (cFF = uFF). In addition, there was no evidence that familiar items that subsequently became recollected were different in their initial hippocampal activation before cue presentation in comparison to those that remained familiar (uFR = uFF). Results from the ROI analysis also showed that the hippocampal involvement was not due to memory for the cue itself. Peak activation was greatest for the recollected cued items (cFR) relative to both the cued familiar items (cFF; t = 3.54, P < 0.005) and the cued correctly rejected lures (cNN, t = 3.62, P < 0.005). Furthermore, a reverse pattern was observed in this peak voxel following cueing of the other types of items; activation decreased for items that remained familiar (uFF vs. cFF; t = 3.11, P < 0.01) and numerically decreased for items that remained correctly rejected although this trend did not reach statistical significance (uNN vs. cNN; t = 1.31, P = 0.21).

Table 3.

Hippocampal activation as a function of contrasts (SVC, P < 0.05)

Analyses	Left or right hippocampus	MNI coordinates			Z-value
		x	y	z
1. Same items (cueing effect)
Remain familiar (cFF > uFF)	n.s.
Conversion to recollection (cFR > uFR)	L	−32	−12	−12	2.01
Initial familiarity (uFR > uFF)	n.s.
2. Same subjective confidence
Familiarity strength (1→4)	n.s.
Recollection excluding familiarity strength     (cR > cF4 masked 1→4)	L	−20	−12	−16	3.63
	L	−32	−24	−16	2.53
	R	24	−8	−24	2.99

Fig. 2.

Pattern of hippocampal activation. (A) Lack of hippocampal engagement for items that remain familiar after cueing or for initial presentation (uncued) of cohorts of familiar items based on recognition decision following cueing (R vs. F). (B) Left hippocampal activation for items that become recollected following cueing. (C) Percent signal change in the peak hippocampal voxel from the contrast described in B as a function of item types [i.e., items that remain familiar (uFF, cFF), convert to recollection (uFR, cFR) or are judged as novel on uncued and cued trials (uNN, cNN)]. (D) Absence of hippocampal activity modulation with familiarity ratings on cued trials. (E) Bilateral hippocampal activity for recollected versus highly familiar cued items with exclusive masking for regions showing linear familiarity function (i.e., ratings 1 to 4). (F) Percent signal change in the peak hippocampal voxel from the contrast described in e as a function of recognition responses.

A similar pattern of results was found for the analyses based on confidence ratings. We found increased hippocampal activation for recollected targets (cR) relative to highly familiar targets (cF4) despite the exclusion of regions engaged in familiarity. Of note, familiarity strength did not modulate hippocampal activation (either positively or negatively) in parametric analyses and, as shown in the ROI analysis, the hippocampal activation did not differ across familiarity ratings (all paired t tests: t < 1). This lack of modulation suggests that the decreased signal for nonrecollected cued items (cFF and cNN) relative to uncued items (uFF and uNN) noted above is unlikely to represent a “novelty” signal (18).

Discussion and Conclusion

Together, our results are consistent with the recollection view. We showed that only recollection engages the hippocampus, even when minimizing differences in memory strength by testing memory for the same nominal items (and arguably the same representations) or by contrasting recollected items to highly familiar items of the same subjective strength while excluding regions that track strength. This pattern indicates that the crucial variable differentiating the degree of hippocampal activation is not related to memory strength per se, but to the interaction between stored information and cueing at retrieval [ecphory (19)], an interaction that is reflected in subjective experience at test.

Previous fMRI studies that tested these positions have typically compared different pools of items that could therefore have differed on both strength and experiential dimensions irrespective of attempts at experimental control [see Wais (4) for similar criticism]. In our comparison of recollected and highly familiar items, we attempted to gain greater control over possible strength confounds by using a statistical mask (i.e., excluding regions parametrically modulated by strength). Note that we, like others (10, 13, 14), failed to find a linear relationship between the familiarity-based confidence ratings and hippocampal activity. In a review of previous data, Squire and colleagues (3) have noted that this function is simply not linear but characterized by a sharp increase at the recollection end, which may reflect some ‘tipping point’ in the strength/activation relationship. This observation cannot, in our view, serve to validate the strength account unless it can be demonstrated in some fashion other than with recollection as an endpoint. This was not seen in our own confidence rating data (excluding recollection), as hippocampal activity for the high familiar items was not greater than that for any of the weaker ratings. Thus, the same results are obtained if we used simple contrast results (e.g., 4 > 1 or 4 > 2) or a quadratic function instead of a linear function as our exclusive mask. Furthermore, the null effects for linear increases in hippocampal activation by confidence/strength here and in previous studies are not a reflection of incapacity; we have reported such linear patterns in other encoding and retrieval studies (20, 21).

Our analysis of the conversion from familiarity to recollection with cueing provides a stringent technique for disentangling these components of recognition. We compared activation for the very same items, and thus the same representations, but before and after cueing, which could give rise to different experiential effects. As shown by the comparison between the initial activation of items that are later recollected or remain familiar, there is no demonstrable relation between the initial strength of the item, as determined by the participant's rating, and the changes effected by the cues. The effect is also not related to memory for the cues themselves or the cue-target pairings. This is supported by the fact that we found greater hippocampal activation with cueing only for the converted items despite the fact that all cues were previously studied.

Proponents of the strength view place their emphasis on the encoded trace, leaving the question of ecphoric strength (i.e., combination of retrieval cues and activated information) somewhat ambiguous in their accounts. For example, Wais (4) has written that hippocampal activation at retrieval may “contribute to strong memories that were associated with hippocampal activation at encoding,” but the nature of that contribution is not specified. In a subsequent publication from the same group, presented as a critical test of the strength view, initial hippocampal activity at encoding was examined as a function of subsequent memory, the assumption being that the strength of initial encoding is a determinant of subsequent memory (12). Thus, it appears that this view is more closely aligned with a “read out” of representational strength at encoding. Such a view would certainly have difficulty accounting for the set of findings we present in this study, which shows that hippocampal activation is determined by recollection at retrieval and not by item strength either before or after the presentation of retrieval cues.

Taken as a whole, the current data are some of the most compelling to be offered in this debate. They not only argue in favor of the recollection hypothesis of hippocampal function, but also emphasize that the neural basis of memory, and its experiential correlate, cannot be understood fully without considering the interaction between encoding and retrieval. These findings confirm an insight that Endel Tulving and others (19) reached decades before the advent of functional neuroimaging, and provide a neurobiological foundation for it.

Methods

Participants.

Thirteen participants (six women) were recruited from the University of Toronto community and received $60 compensation. Their mean age was 26.8 (range of 20–35). All were right-handed fluent English speakers, had normal or corrected vision, and had no history of neurological disease or any contraindications for MRI. This project was approved by the research ethics board of University Health Network.

Material and Procedure.

We selected 444 words (252 seven-letter words and 192 six-letter words) from the MRC psycholinguistic database. We created 252 word pairs by combining one seven-letter word to an unrelated six-letter word; 60 of these were alternate pairings (i.e., one word had two possible associates). These pairs were equated in terms of Kucera-Francis frequencies (first word: M = 25.54, SD = 0.58; second word: M = 42.41 SD = 3.69) and concreteness ratings (first word: M = 467.48 SD = 7.15, second word: M = 482.58, SD = 21.20).

The experiment was programmed using E-prime software (Psychology Software Tools Inc). Practice and study phases were conducted outside the scanner. The study phase consisted of two 11.2-min blocks, each containing 90 critical pairs (A–B) and six buffer pairs (three at the beginning and three at the end of each block to control for primacy and recency effects). Each pair was presented for 6s and was followed by a 1s fixation cross. Participants were instructed to generate a sentence mentally that incorporated both words and to indicate, for each pair, the difficulty level of this task on a 5-point scale (“1”- very easy, “5” - very difficult).

The test phase began approximately 30 min after the end of the study phase. Items were presented using MR-compatible goggles (Avotec Inc.) and recognition responses were made using two four-button handheld response pads (fORP).

Participants were scanned while performing a recognition task for the first member of the pairs (A). This task was divided into six 9.3-min runs. Participants were presented with an uncued test item, which was either a studied (A) or new word (C), and indicated whether the item was new (rated 1 = sure new or 2 = unsure new), familiar (rated 3 = unsure familiar or 4 = sure familiar), or recollected. Detailed instructions were provided to explain each response. Crucially, participants were instructed that they should not distinguish between “sure familiar” responses and “recollected” responses based on response confidence, as confidence should be equivalent for both types, but rather based on their ability to retrieve contextual information (i.e., subjective experience). A practice run in which participants justified their responses was conducted to ensure they understood and applied this distinction. A cued trial followed immediately each uncued trial, in which intact pairs (A–B) or new-old pairs (C–B) were presented; the recognition decision was again to be based only on the first item (A or C), which was the same item as the one presented in the previous uncued trial. The recognition task included 120 uncued target items (A), 120 cued lure items (A–B), 60 uncued lure items (C), and 60 cued new items (C–B). The test phase also included 30 uncued low level baseline items (+++++++) and 30 cued low level baseline items (+++++++ - ++++++) for which participants were instructed to give a “3” answer.

Each test item was presented for 5.5 s followed by a 500 ms fixation cross randomly jittered by 2 s (mean of 8 s per trial). Thus, each block contained 35 uncued and 35 cued items (20 uncued and 20 cued targets, 10 uncued and 10 cued lures, and five cued and five uncued low level baseline items). Item presentation was pseudorandomised with the restriction of a maximum of three subsequent items of the same type. New responses (1 and 2) were entered using the left hand, and familiar (3 and 4) and recollected answers were entered using the right hand.

Data Acquisition.

Anatomical and functional data were acquired on a 3-T Signa MR System (GE Medical Systems). The anatomical scans were taken first (T1-weighted sequence, 120 slices, 220 mm FOV, 256 × 256 matrix, resulting in a voxel size of 0.78125 × 0.78125 × 1.0). Functional data were acquired in an interleaved order (TR = 2 s; 28 slices for six participants and 35 slices for seven participants, 440 mm FOV, 64 × 64 matrix, resulting in a voxel size of 3.75 × 3.75 × 5.0). These were taken in an oblique orientation, with each slice being perpendicular to the long axis of the hippocampus. For each of the six runs, 285 frames were acquired. The first three of these were dropped for signal equilibrium.

Data Processing and Statistical Analyses.

Functional and structural scans were coregistered using a Matlab script. Slice timing, realignment, normalization to the Montreal Neurological Institute EPI template, and smoothing (FWHM 8 mm) were performed using SPM5 (Statistical Parametric Mapping 5; Welcome Department of Imaging Neuroscience). SPM5 was also used for contrast analyses. Each stimulus event was modeled by SPM5's canonical hrf (applied at item presentation onset). First level analyses of individual participants' data were conducted using a fixed-effects model. Second level analyses were conducted on the contrast images obtained in the previous step using a random-effect model.

Whole brain results are presented in the SI Text and Tables S1, S2, and S3. All maxima reported were significant at a level of P < 0.005 (uncorrected) and survived an extent threshold of five contiguously activated voxels (4 × 4 × 4 mm³). For the familiarity exclusive mask, we used a conservative threshold of P < 0.05.

To investigate hippocampal activation, we used small volume correction with an ROI mask from MARINA (Bertram Walter Bender Institute of Neuroimaging, University of Giessen) with a threshold of P < 0.05 and five contiguous voxels (4 × 4 × 4 mm³). We reported only local maxima identified to fall within the hippocampus proper based on cytoarchitectonic probabilities using the SPM5 Anatomy Toolbox v.1.5 (Institute of Medicine, Research Center Jülich).

Using SPM5 Marsbar Toolbox ROI analyses were conducted on the peak hippocampal voxels (spherical ROIs radius = 3 mm) from the conversion (cFR>uFR) contrast and for the recollection contrast (cR>cF4 masked with familiarity). For the conversion/cueing analyses, percent signal change was extracted for six item types (uFF, cFF, uFR, cFR, uNN, cNN). For the confidence analyses, percent signal change was extracted for cued recollected (cR), sure familiar (cF4) and sure correct rejection (cN1) as well as for all cued items rated sure new (1), unsure new (2), unsure familiar (3), sure familiar (4), and recollected (R).

Partial datasets were acquired for two participants due to technical problems (three and four runs were collected). Furthermore, two runs were excluded for one participant due to excessive head motion. All participants were included in the conversion analyses and eleven participants were included in the recollection versus high familiarity analyses. Two participants (with incomplete datasets) were excluded because they had few sure familiar judgments to cued items (<10).