Recall versus familiarity when recall fails for words and scenes: The differential roles of the hippocampus, perirhinal cortex, and category-specific cortical regions

Abstract

This fMRI study examined recall and familiarity for words and scenes using the novel recognition without cued recall (RWCR) paradigm. Subjects performed a cued recall task in which half of the test cues resembled studied items (and thus were familiar) and half did not. Subjects also judged the familiarity of the cue itself. RWCR is the finding that, among cues for which recall fails, subjects generally rate cues that resemble studied items as more familiar than cues that do not. For words, left and right hippocampal activity increased when recall succeeded relative to when it failed. When recall failed, right hippocampal activity was decreased for familiar relative to unfamiliar cues. In contrast, right Prc activity increased for familiar cues for which recall failed relative to both familiar cues for which recall succeeded and to unfamiliar cues. For scenes, left hippocampal activity increased when recall succeeded relative to when it failed but did not differentiate familiar from unfamiliar cues when recall failed. In contrast, right Prc activity increased for familiar relative to unfamiliar cues when recall failed. Category-specific cortical regions showed effects unique to their respective stimulus types: The visual word form area (VWFA) showed effects for recall vs. familiarity specific to words, and the parahippocampal place area (PPA) showed effects for recall vs. familiarity specific to scenes. In both cases, these effects were such that there was increased activity occurring during recall relative to when recall failed, and decreased activity occurring for familiar relative to unfamiliar cues when recall failed.

1. Introduction

Human recognition memory is the ability to realize that some or all of a present experience echoes an experience in the past, as when one recognizes that one knows a person, that a landmark has been seen before, or that a song has been heard before. A remarkable feat of human recognition is that it can occur even when recall of the prior experience fails (Cleary, 2004; Cleary et al., 2012; Ryals and Cleary, 2012). One theoretical framework for explaining this ability is the dual-process framework, according to which at least two processes contribute to human recognition memory: Recollection and familiarity (Diana et al., 2006, 2010; Mandler, 2008; Yonelinas, 2002). Recollection-based recognition depends on retrieval of specifics about a prior experience while familiarity-based recognition occurs when a current situation elicits a mere sense of prior experience without specifics. Thus, when recollection fails, it is still possible to recognize a situation as familiar. Previous literature has suggested that recollection and familiarity may be supported by different medial temporal lobe (MTL) structures. One view is that the hippocampus is involved in recollection whereas surrounding regions (e.g., perirhinal cortex) are involved in familiarity (see Yonelinas, 2002). Mechanistically, certain MTL structures may give rise to full recall resulting from pattern completion using existing memory representations, whereas other regions may allow recognition when there is a partial match (e.g., Graham et al., 2010; Saksida and Bussey, 2010). Given the proposed role of the hippocampus in relational encoding and retrieval (e.g., Norman and O’Reilly, 2003) and the proposed role of the perirhinal cortex (Prc) in item-based conjunctive featural processing (e.g., Staresina and Davachi, 2008, 2010), it is possible that the hippocampus is more involved in full cued recall and the Prc is involved in partial matches of cues to item memory representations which may support familiarity-based recognition.

A useful paradigm for investigating these roles of MTL structures is the recognition without cued recall (RWCR) method (Cleary, 2004; Cleary et al., 2012; Ryals and Cleary, 2012). This method allows for an examination of both cued recall itself and familiarity from feature resemblance that occurs when recall fails.¹ Unlike many recognition paradigms, the RWCR task relies on resemblance of cues to earlier presented items. Specifically, this task allows for the objective separation of test trials into those accompanied by successful cued recall, those that are familiar when recall fails, and those that are novel and unfamiliar. Among the test cues, half resemble studied items and half do not. For each cue presented, subjects attempt to recall a studied item that resembles it. Even when they fail to do so, they rate the familiarity of the cue itself. RWCR is the finding that, among cues for which recall fails, individuals indeed still show resemblance recognition: They give higher ratings to cues that resemble studied items than to cues that do not resemble studied items.

Our first goal was to use this novel RWCR task to seek converging evidence that recall and familiarity when recall fails recruit different MTL regions as has been suggested by some previous neuropsychological and neuroimaging work (see Eichenbaum et al. (2007), for a review). Our second goal was to examine the role of the Prc in full vs. partial matches of cues to memory representations and how this role differs from that of the hippocampus; whereas the hippocampus may be critically involved in full target recall from similar cues (via pattern completion, for instance), the Prc may be involved in the type of partial matching that contributes to recognition when recall fails. Our third goal was to compare two different categories of visual stimuli using the RWCR task: scenes (Cleary et al., 2009, 2012) and words (Cleary, 2004). This comparison allows us to determine how recall and familiarity when recall fails are affected by stimulus type, which is important for two reasons. First, some evidence suggests that not all forms of dual-process theory provide a good fit across both word and scene stimuli (Onyper et al., 2010), and it is not yet clear whether the hippocampal and Prc involvement in recall and familiarity, respectively, are domain-specific or domain-general with regard to modality and type of information. Second, the comparison allows us to examine how activity in high level category-specific cortical regions specialized for processing words and scenes may relate to recall and familiarity for words and scenes. Most research on the neural substrates of recall and familiarity has focused on the medial temporal lobe (MTL) (see Eichenbaum et al. (2007), for a review), with some studies focusing on the frontal lobes (e.g., Kirwan et al., 2008). By comparing memory for words and scenes, we will additionally examine whether word-specific (i.e., the visual word form area) and scene-specific (i.e., parahippocampal place area) cortical processing areas are selectively involved in recall and/or familiarity for words and scenes.

1.1. Recollection and familiarity in the medial temporal lobe

Most research on the neural substrates of recollection and familiarity has focused on the medial temporal lobe (MTL), which can be subdivided into the hippocampal formation and its surrounding structures. Collectively, these surrounding structures are known as the parahippocampal region (or MTL cortex), and include the parahippocampal cortex (posterior) and the entorhinal cortex and Prc (anterior) (Eichenbaum et al., 2007).

Two main sources of evidence converge on the idea that whereas recall involves the hippocampus, familiarity involves anterior parts of the MTL cortex, particularly the Prc. One line of support comes from a double dissociation in clinical populations. On the one hand, amnesics who have sustained hippocampal damage but whose anterior MTL cortices are intact are deficient on tasks thought to require recollection but unimpaired on tasks thought to tap familiarity-based recognition (e.g., Holdstock et al., 2005; Mayes et al., 2002; Turriziani et al., 2008; Vann et al., 2009). On the other hand, after surgical lesioning of the Prc to reduce seizures, a patient demonstrated impairment in familiarity-based recognition on three different tasks while demonstrating unimpaired recollection-based recognition on these same tasks (Bowles et al., 2007).

A second line of support comes from neuroimaging studies, which have generally implicated the hippocampus and para-hippocampal cortex in recollection, while the perirhinal and possibly entorhinal cortices are involved in familiarity (e.g., Cohn et al., 2009; see Diana et al., 2007 and Eichenbaum et al., 2007, for reviews).

Almost all previous neuroimaging studies of recollection and familiarity have used one of three different paradigms, all of which have generated controversy: the remember–know (R/K) method (13/40 contrasts reviewed in Diana et al., 2007), source memory judgments (19/40 contrasts reviewed in Diana et al., 2007), or recognition confidence judgments (9/40 in Diana et al., 2007). Importantly, none of these studies have assessed recollection by objectively indexing item recall itself (as is done in free recall or cued recall tasks). Recall studies have historically been difficult due to the limitations of the MR scanning environment and response acquisition systems.

In source memory paradigms subjects both indicate whether they recognize a stimulus and answer a question about the context in which the stimulus was encoded (e.g., whether the word was presented on a red or blue background). Familiarity is defined as correct recognition without source retrieval, and recollection is defined as correct source retrieval and recognition together. This results in a strict criterion for recollection, as it is possible for a subject to successfully recall an item but merely fail to recall the source. Source memory tasks also recruit many regions beyond the medial temporal lobe (e.g., Kirwan et al., 2008).

The R–K task indexes recollection and familiarity based on participants’ self-reports of the underlying bases of their recognition judgments: remember judgments indicate recollection-based responding and know judgments indicate familiarity-based responding. Some suggest that the R–K task does not successfully separate the two processes because recollection actually contaminates “know” reports (Wais et al., 2008; Johnsonet al, 2009).

In recognition confidence studies subjects indicate their recognition response using a 4 to 6 point Likert scale. There is a fundamental disagreement as to how these confidence judgments should be interpreted. Some argue, based on a specific instantiation of dual-process theory in which recollection results from a threshold process whereas familiarity is continuous (e.g., Diana et al., 2007), that high confidence recognition ratings are reserved for instances of recollection and lower confidence recognition ratings indicate varying degrees of familiarity. Others argue that recollection is continuous, rather than a threshold process (e.g., Mickes et al., 2009) and that any neural differences found between high and low confidence reflect only memory strength, not separate recollection and familiarity processes.

The RWCR task that we use avoids the limitations of the above three tasks. First, we use an item-based cued recall measure to objectively and sensitively identify trials on which there was recall of the study item itself and on which recall of the study item failed. This avoids requiring memory of extra-item associations (as in source memory tasks), or relying on subjective measures of recall (as in R/K), or relying on controversial assumptions regarding high versus low recognition confidence ratings. Second, we objectively manipulate familiarity through cue resemblance to studied items, as cues that resemble studied items should be more familiar on average than cues that do not resemble studied items, even when recall fails. This again avoids reliance on subjective measures such as Know responses or assumptions about low recognition confidence ratings.

1.2. Visual category effects in memory: Scenes vs. words

We examined recall and familiarity in two different visual categories, words and scenes, for three primary reasons. First, we sought to examine the generality of our hypothesis that recall would involve the hippocampus whereas familiarity would involve the Prc. Second, Onyper et al. (2010) have shown that some of the inconsistencies in the dual process literature may be due to the fact that scenes and words show different patterns with regard to whether dual-process or single-process models are a better fit for the data. Thus, it is important to examine whether the neural correlates of recall and familiarity differ depending on whether word or scene stimuli are used. Third, examining memory for words and scenes allowed us to determine what aspects of memory are common across categories (which we will refer to as non material-specific processing), and which differentiate between these two categories.

Previous functional imaging studies have typically found that the hippocampus exhibits non material-specific patterns of activation, with similar recruitment across visual categories. Different studies have compared faces and scenes (Preston et al., 2010) words, objects and scenes (Duarte et al., 2011), and objects, toys, abstract objects, faces and scenes (Diana et al., 2008).²

Similarly, previous functional imaging studies that have directly compared words and scenes have all found similar degrees of Prc recruitment for both categories (Diana et al., 2008; Duarte et al., 2011; Preston et al., 2010). This does not necessarily indicate that the Prc is completely non material-specific; some research indicates that the Prc has a special role in processing complex three dimensional objects (animals, tools and other artifacts) and conjunctions of object features (Barense et al., 2005; Lee et al., 2005). This is thought to be due to functional circuitry in which the Prc is at the apex of the visual processing hierarchy in the temporal lobe (Graham et al., 2010; Murray et al., 2007). Functional imaging studies provide mixed evidence as to whether objects lead to greater Prc recruitment than non-object categories; Diana et al. (2008) found similar recruitment for objects and non-objects, whereas Duarte et al. (2011) report greater activity for objects. For the purposes of the present study, we can remain agnostic as to whether objects in fact play a special role in the Prc since we do not use objects as stimuli.

In contrast to the hippocampus and Prc, regions of the temporal lobe visual processing stream do show category specificity. Most relevant for the current study are regions selectively activated for words and for scenes, which have been termed the visual word form area (VWFA) and para-hippocampal place area (PPA), respectively. The VWFA is most commonly localized to the fusiform gyrus. It is sensitive to the visual components of words and is active when viewing whole words and pseudowords (letter strings incorporating commonly experienced letter combinations; Cohen et al., 2002; Glezer et al., 2009; however see Price and Devlin, 2003 for an alternative view). Lesions to this region lead to pure alexia, sometimes called letter-by-letter reading, in which subjects are severely impaired at reading due to their inability to visually combine letters into representations of complete words (Epelbaum et al., 2008; Gaillard et al., 2006). Mei et al. (2010) found that encoding related recruitment of the VWFA predicted later memory success for words.

The PPA is a functionally-defined region that overlaps with the parahippocampal cortex (PHC) often extending posterior to the PHC, and it is typically medial and anterior to the VWFA. Research by Diana et al., 2008 found that functionally-defined PPA (and anatomically-defined PHC) is recruited to a greater degree for scenes than for other visual categories. Research by Preston et al., 2010 offers similar evidence examining only anatomically-defined PHC. Furthermore, this region is sensitive to both configural aspects of scenes as well as landmarks (such as buildings) that typically are present within scenes (Epstein, 2008). There is controversy over whether the PPA is involved in memory. Overall the PPA is active whenever scenes are perceived and attended (Epstein, 2008). PPA activity is sometimes (Epstein et al., 2007) but not always modulated by the familiarity of the scene (Epstein et al., 2007). Turk-Browne et al. (2006) found that repetition related decreases in PPA activity to repeated scenes (repetition suppression, or priming) were related to behavioral measures of priming only when the scenes were later explicitly remembered. It should be noted that some researchers argue that the PPA is sensitive to memory for contextual information more broadly, and that scenes are just one common type of contextual processing (Bar et al., 2008).

1.3. Overview of the present study

Our study emphasizes how recognition differs when a person succeeds at recall versus when a person fails to recall. We adapted the RWCR task first used with words by Cleary (2004) and Ryals and Cleary (2012), and extended to scenes by Cleary et al. (2009, 2012), using both a word and a scene version. In the word version, participants first studied a set of words (e.g., curious) and were then presented at test with pseudo-word cues, half of which resembled studied items orthographically and phonologically (e.g., cuniaus). For each test cue, participants first judged its familiarity and then tried to recall a resembling word from study. In the scene version, participants studied easily namable color scenes along with their names (e.g., bathroom, kitchen, gym) and were then tested with a list of new color scenes, half of which resembled studied scenes in their configuration of elements and half of which did not. For each test scene, participants first rated its familiarity and then attempted to recall the name of a similar scene from the study list.

Our first set of predictions involves the hippocampus and Prc. In accordance with prior hippocampal research on recall, we predict that the hippocampus will be recruited for recall of words and scenes. In accordance with prior research on familiarity, we predict that the Prc, but not the hippocampus, will be modulated within the familiarity (RWCR) contrast, but not for the recollection contrast. It is unclear whether familiar items will result in less activity than novel items (a priming effect), or more activity. Though prior studies suggest that the Prc generally shows decreased activity in response to familiarity brought on by repetition (see Eichenbaum et al., 2007), we use novel test stimuli that resemble studied items but that are themselves new (i.e., non-repetitions). There is no clear precedent upon which to predict directionally specific effects in Prc for these types of novel test stimuli. Furthermore, studies using other methods have suggested a possible increase in the Prc (particularly the right Prc) with increased feelings of familiarity (e.g., Devinsky et al., 2010).

Our second set of predictions involved the visual category specific regions. Based on previous studies of cortical reinstatement and its posited role in recollection (e.g., Johnson and Rugg, 2007) as well as on theories regarding how the hippocampus facilitates pattern completion via cortical interactions to allow recollection (Norman and O’Reilly, 2003), we predicted that for scene recall, there would be scene specific recruitment of the scene region (PPA), while for word recall, there would be word specific recruitment of the word region (VWFA). For familiarity that occurs when recall fails, we anticipated two possibilities regarding how visual category specific regions might be modulated. The first possibility is that visual category specific regions will increase in activity for familiar cues. This is plausible given the findings of Johnson et al. (2009), which showed that cortical reinstatement occurred during both remember and know judgments. The second possibility is that we will instead find a decrease in activity for familiar relative to unfamiliar test items. This prediction is based in part on prior findings in the literature showing that repeated or familiarized stimuli often lead to decreased cortical activity associated with priming (e.g., Buckner et al., 1998, 1995), and in part on the association between familiarity and perceptual fluency, which is thought to be a product of priming (Huber et al., 2008; Johnston et al., 1991).

2. Results

2.1. Behavioral results: Words

Behavioral data were analyzed to examine both recall itself and cue familiarity when recall failed (as measured by the RWCR effect). Verbal test responses were coded by hand to ensure that they were correctly binned in the recalled or unrecalled categories. Trials were labeled as correctly recalled if the participant verbally identified the name of the corresponding study word (e.g., invest for the cue imwest). Trials were labeled as unrecalled if the participant verbally responded “don’t know” for the test trial. On average across all scans, participants correctly recalled 75.25% of studied words during the test phase (M=33.85 words, SD=3.05), and on average, participants failed to recall 24.75% of the studied words during the test phase (M=11.15, SD=4.27). Because the nonword test cues were created to correspond to real English words in the stimulus pool, regardless of their study-status in the experiment, participants sometimes correctly identified the unstudied target word corresponding to a nonword cue. However, on average, participants only identified 8.5% of the unstudied words in response to their cues (M=3.85 words, SD=3.57). The tendency to correctly guess was highly variable across subjects, with six subjects having either zero or one correct guess overall in this category. Therefore, we did not analyze the data in this category. Finally, participants failed to recall an average of 91.4% of the nonstudied words during the test phase (M=41.15 words, SD=8.98).

A shown in Fig. 2, when cued recall failed, recognition ratings given to test cues that resembled studied words were significantly higher than ratings given to test cues that did not resemble studied words, [t(10)=5.31, SE=.30, p<.001, d=3.72]. This replicates the RWCR effect found by Cleary (2004) and Ryals and Cleary (2012). The recognition ratings for each subject are located in the lower left portion of Fig. 2; a visual inspection indicates that all but one showed the RWCR effect. Among test cues that resembled studied items, recognition ratings were significantly higher when recall succeeded than when it failed, [t(10)=7.33, SE=.64, p<.001, d=7.50], and when recall succeeded than when the cues did not correspond to studied items, [t(10)= 15.10, SE=.44, p<.001, d=14.84].

Fig. 2.

Top row: The behavioral RWCR effect for words and scenes. In the absence of cued recall, recognition ratings are higher for cues that resembled studied items than those that did not. Bottom left: Mean recognition ratings for pseudoword cues by participant. Bottom right: Mean recognition ratings for scene cues by participant. (Note:** denotes a significant effect at p<.01).

2.2. Behavioral results: Scenes

Behavioral data from the scene scans were also analyzed to examine recall and the RWCR effect. Verbal test responses were coded by hand to ensure they were binned in the correct category (recalled or unrecalled). Items were labeled as recalled if the participant identified the name of the configurally similar study scene (e.g., library). Items were labeled as unrecalled if the participant responded “don’t know”. Occasionally, a verbal response did not match the name of the studied scene exactly but correctly identified the corresponding studied scene (e.g., train tracks for the studied scene train station); these items were labeled as recalled. On average, participants recalled 25% of studied scenes during the test phase (M=7.50 scenes, SD=5.20), and they failed to recall 75% of studied scenes during the test phase (M=22.50, SD=6.73), It is important to note that whereas in the word condition correct guessing of the target word from the non-word cue was a possibility, this likelihood was substantially reduced in the scene condition. On average, participants correctly guessed the identity of only 0.13% of unstudied scenes (M=.04 scenes, SD=0.87), whereas they failed to guess the identity of 99.87% of the unstudied scenes as expected (M=29.96, SD=.87).

Scene recognition ratings are also displayed in Fig. 2. When cued recall failed, mean recognition ratings given to test scene cues that configurally resembled studied scenes were significantly higher than ratings given to novel test scene cues, [t(12)=4.88, SE=.17, p<.001, d=2.82). This demonstrates a scene RWCR effect similar to that found by Cleary et al. (2009). Individual subject ratings are displayed in the lower right portion of Fig. 2; 10 out of 13 subjects showed the RWCR effect with scenes. Among test scenes that resembled studied scenes, recognition ratings were significantly higher when recall succeeded than when it failed, [t(12)=11.71, SE=.31, p<.001, d=7.92], and when recall succeeded than when the cues did not resemble studied scenes, [t(12)=15.50, SE=.29, p<.001, d=9.80].

2.3. fMRI results

2.3.1. Key memory contrasts of interest

Based on prior studies of RWCR (Cleary, 2004; Cleary et al., 2009, 2012; Ryals and Cleary, 2012), we created two key memory contrasts of interest for our fMRI data. Our first contrast, Cued Recall, compared activity correlated with successful cued recall to activity correlated with recall failure. Our second contrast, RWCR, compared familiar cues for which recall failed (i.e., those that resembled studied items but did not elicit cued recall) to novel cues (i.e., those that did not resemble studied items). We did not design this study to have sufficient statistical power to examine encoding related activity, and consistent with that decision our exploratory analyses did not reveal any significant effects at encoding.

2.3.2. Hippocampal patterns of activation for word stimuli

As illustrated in the top panel of Fig. 4, the Cued Recall contrast indicated that successful cued recall of words recruited the hippocampus (studied recalled>studied unrecalled). Among cues that resembled studied words, there was significantly higher activation in the right hippocampus for cues that elicited successful recall of studied words than for cues for which recall failed, [t(10)=4.42, SE=.05, p=.001]. The same pattern also reached significance in the left hippocampus [t(10)=3.01, SE=.07 p=.01]. Interestingly, when recall failed, a significant difference was found in the right hippocampus for the RWCR contrast that was opposite that found for the aforementioned recall contrast [t(10)= −2.50, SE=.06 p=.03]. As displayed in the bottom panel of Fig. 4, among cues for which recall failed, those resembling studied items elicited a significant decrease in hippocampal activity compared to cues not resembling studied items. Although it did not reach statistical significance in the left hippocampus, a similar pattern emerged there. As we propose later in the discussion, this may reflect the sensitivity of the hippocampus to novelty.

Fig. 4.

Top panel: Parameter estimates for word blocks for Recall (studied recalled>studied unrecalled) in right hippocampus, left hippocampus, right Prc, and left Prc at retrieval. Bottom panel: Parameter estimates for word blocks for Familiarity (studied unrecalled>unstudied unrecalled) in right hippocampus, left hippocampus, right Prc, and left Prc at retrieval. (Note:** denotes a significant effect at p<.01 * denotes a significant effect between p=.01 and p=.05,, ǂ denotes a trend between p=.05 and p=.15

2.3.3. Perirhinal cortex (Prc) patterns of activation for word stimuli

As shown in the bottom panel of Fig. 4, RWCR revealed significant recruitment of the right Prc (familiar cues of studied unrecalled words>unfamiliar cues of unstudied words). Activity for familiar cues for which recall failed was significantly higher than activity for unfamiliar cues (cues that did not resemble studied words), [t(10)=2.23, SE=.11, p=.04]. A significant pattern was found in the right Prc for our cued-recall contrast (displayed in the top panel of Fig. 4) however, it is critical to note that this activity was in a direction opposite that in the Prc for our RWCR contrast. This pattern suggests decreased Prc activity for cues eliciting successful recall compared to cues for which recall failed [t(10)=2.20, SE=.10, p=.05]. A comparison of both the top and bottom of Fig. 4 suggests that familiar cues for which recall failed led to greater Prc activity than either familiar cues for which recall succeeded or for unfamiliar cues. In the left hemisphere, a marginally significant RWCR Prc effect emerged whereby activity for familiar cues for which recall failed was higher than activity for unfamiliar cues (cues that did not resemble studied words), [t(10)=2.01, SE=.10, p=.07].

In order to examine the generality of processing across the Prc, we performed a fixed effects follow-up ROI analysis in which we included ROIs along the entire anterior-posterior axis of the right Prc. As illustrated in Fig. 6, the middle slices showed a significant effect of familiarity and posterior slices a trend towards an effect.

Fig. 6.

Familiarity effects for words and scenes across Perirhinal cortex ROIs. Six anatomical ROIs were hand-drawn along the anterior-posterior axis of the right Prc. These ROIs are illustrated in the upper left image with each ROI depicted in a different color. In the other two images, each ROI is color coded based on the p value of the RWCR contrast (studied-unrecalled vs unstudied-unrecalled). Red: p>.1. Orange: a trend between p=.05 and p=.1. Yellow: p<.05. Only the right Prc was examined for this analysis because effects were strongest in our R Prc ROIs, and familiarity effects have been reported to be right lateralized in several other previous studies (e.g., Devinsky et al., 2010).

2.3.4. Patterns of hippocampal activation for scene stimuli

Next we examined cued recall for scenes, as illustrated in the top panel of Fig. 5 (studied recalled>studied unrecalled). This contrast revealed significantly higher activity in the left hippocampus for configurally similar scenes (i.e., those that resembled studied scenes in configuration) whose studied counterparts were recalled than for those whose studied counterparts were unrecalled, [t(12)=2.40, SE=.10, p=.03], Importantly, no significant differences were found in the hippocampal ROIs for RWCR. Additionally, no effects were found for either the recall or RWCR contrasts in the right hippocampus.

Fig. 5.

Top panel: Parameter estimates for scene blocks for Recall (studied recalled>studied unrecalled) in right hippocampus, left hippocampus, right Prc, and left Prc at retrieval. Bottom panel: Parameter estimates for scene blocks for Familiarity (studied unrecalled>unstudied unrecalled) in right hippocampus, left hippocampus, right Prc, and left Prc at retrieval. (Note: * denotes a significant effect between p=.01 and p=.05).

2.3.5. Patterns of perirhinal cortex (Prc) activation for scene stimuli

As displayed in the bottom panel of Fig. 5, RWCR for scenes (studied unrecalled>unstudied unrecalled) recruited right perirhinal cortex, such that activity in this region was significantly higher for familiar than unfamiliar items [t(12)=2.29, SE=.07, p=.04]; this effect did not reach significance in the left Prc. Importantly, no significant differences were found in Prc ROIs for our recall contrast. In order to examine the generality of processing across the Prc, we performed an exploratory fixed effects follow-up ROI analysis in which we included ROI slices along the entire anterior-posterior axis of the right Prc. As illustrated in Fig. 6, the middle and posterior slices showed a significant effect of RWCR.

2.3.6. Cross-region interaction analyses

We examined the effects of familiarity and recollection across the Prc and hippocampus. For recollection, we performed 2 × 2 × 2 ANOVAs separately for word and scene stimuli with within subject factors of Recall Status (studied recalled,studied unrecalled), MTL Region (Prc, hippocampus) and Laterality (right and left hemisphere), using ROI and condition specific beta values (which can be transformed into percent signal change values) as our dependent measure. We report only interactions with Recall Status, since direct comparisons of activity in different neural regions can result from irrelevant factors such as regional differences in the hemodynamic response function. For word recall, we found a significant interaction between Recall Status and MTL Region, F(1,9)=16.14, p<.01. This interaction was not significant for scenes (F<1). For words and scenes alike, there were no significant interactions between Laterality and Recall Status, and there were no three way interactions.

For familiarity we performed similar ANOVAs in which our variable (Study Status) included the studied unrecalled and unstudied unrecalled conditions from the RWCR contrast. We found a significant interaction between Study Status and MTL Region for words, F(1,9)=12.14, p<.01, and a trend toward an interaction for scenes F(1,12)=3.45, p=.08. Again, there were no significant interactions between Laterality and Study Status, and there were no three way interactions.

2.3.7. Category-specific cortical activation for word stimuli

The Cued Recall and RWCR contrasts were examined within the three category-specific cortical ROIs: The fusiform face area (FFA), the parahippocampal place area (PPA) and the visual word form area (VWFA). As described in Methods, these ROIs were defined for individual subjects based on the localizer scan. A plot of the beta values from the random effects GLMs for each of these ROIs can be found in Fig. 7. For the Cued Recall contrast, there was greater activation for cues corresponding to studied words that were recalled than for cues corresponding to studied words that were unrecalled in both the left VWFA, [t(10)=3.75, SE=.07, p=.003], and the right VWFA, [t(10)= 3.66, SE=.08, p=.004]. There were no significant differences or trends in right or left FFA or PPA for this contrast. For the RWCR contrast (our familiarity contrast), there was a significant effect in the left VWFA such that familiar cues for which recall failed were characterized by decreased activation relative to unfamiliar cues (familiar cues of studied unrecalled words<unfamiliar cues of unstudied words), [t(10)=−2.93, SE=.06, p=.01]. Note that the direction of this category-specific cortical familiarity effect is opposite to that found in the Prc. A trend in the same direction emerged in the right VWFA for our recall contrast (p=.12). No significant differences or trends emerged in left or right FFA or PPA for this contrast.

Fig. 7.

Recall and familiarity parameter estimates within category-specific regions for word blocks 1–3. Top panel: Word Recall (studied recalled>studied unrecalled), bottom panel: Word familiarity (studied unrecalled>unstudied unrecalled). Category-specific regions: left and right fusiform face area (FFA), left and right visual word form area (VWFA), and left and right parahippocampal place area (PPA). (Note: ³³ denotes a significant effect at p<.01 * denotes a significant effect between p=.01 to p=.05,, ǂ denotes a trend between p=.05 and p=.15).

2.3.8. Category-specific cortical activation for scene stimuli

The same category-specific cortical ROIs were examined for the scene blocks as for the word blocks. A plot of the beta values from the random effects general linear models for each of the scene-specific ROI comparisons can be found in Fig. 8. For the Cued Recall contrast, we found significantly higher activation in the left PPA when cued recall of studied scenes succeeded than when it failed (studied recalled>studied unrecalled), [t(10)=2.42, SE=.14, p=.04]. No significant differences or trends emerged for this contrast in the right PPA, or in the right or left FFA or VWFA.

Fig. 8.

Recall and familiarity parameter estimates within category-specific regions for scene blocks 4–5. Top panel: Scene Recall (studied recalled>studied unrecalled), bottom panel: Scene Familiarity for left and right fusiform face area (FFA), left and right visual word form area (VWFA), and left and right parahippocampal place area (PPA). (Note: ³ denotes a significant effect between p=.01 to p=.05, ǂ denotes a trend between p=.05 and p=.15)

For the RWCR contrast, there was a trend toward lower activity for familiar cues for which recall failed than for unfamiliar cues (those that did not resemble studied scenes) in the left PPA, (p=.12) Similar to the pattern found in the VWFA for words described above, this decrease is the opposite of what was shown for recollection, and suggests a possible priming effect that can be detected when successful recall is removed from the pool of data under consideration. Comparisons for right and left FFA and VWFA once again failed to reveal any significant differences or trends.

2.3.9. Exploratory whole-brain analysis

In order to identify other neural regions potentially associated with Cued Recall or RWCR we performed an exploratory random effects whole-brain analysis with a statistical threshold of p<.05 uncorrected. Full tables of active clusters derived from this analysis are available in the Supplementary Materials.

3. Discussion

3.1. Overview of the present findings

Taken together, our patterns of results provide convergent evidence, using a novel task, for the separation of recollection and familiarity within the medial temporal lobe. Additionally, our results support the theory that the hippocampus and the Prc support their respective memory processes in a relatively non material-specific fashion. For the word condition, we found a pattern in the MTL whereby hippocampal activity was increased when recall succeeded compared to when it failed for familiar cues (i.e., cues resembling studied items), and was decreased for familiar relative to unfamiliar cues when recall failed. In contrast, Prc activity was increased for familiar cues for which recall failed compared to both familiar cues for which recall succeeded and unfamiliar cues. For the scene condition, we found a similar pattern in which left hippocampal activity was increased when recall succeeded compared to when it failed and right Prc activity was increased for familiar cues for which recall failed relative to for unfamiliar cues.

We found very different patterns of results in visual category specific regions. Specifically, we found sensitivity that was specific to the material type being studied and which differed depending on whether recollection or familiarity was examined: For word recall, the VWFA was recruited, with greater activity for recalled than unrecalled words, whereas for word familiarity when recall failed, greater VWFA activity occurred for novel cues than familiar cues. These effects were specific to the VWFA and did not occur in scene (PPA) and face (FFA) specific regions. Complementarily, we found that scene specific regions showed the same patterns of recollection and familiarity modulation, but only for scenes, not for words. For scene recall, the PPA was recruited, with greater activity for recalled than unrecalled scenes; for scene familiarity when recall failed, there was a trend toward greater activity for novel scenes than for familiar scenes. These scene recall and familiarity effects were specific to the PPA and did not occur in word (VWFA) and face (FFA) specific regions.

3.2. The perirhinal cortex (Prc) in familiarity

Insofar as we showed that Prc activity increased for familiarity and not recall, our results support prior work suggesting a role of the Prc in familiarity (e.g., Bowles et al., 2007; Diana et al., 2007; Holdstock et al., 2005; Mayes et al., 2002; Vann et al., 2009). Our results extend these previous findings in several ways. First, we used a novel, objective task for separating familiarity and recall. Second, our Prc involvement was for familiarity brought on by resemblance to a prior situation in memory rather than by an exact repetition of the stimulus. Finally, our results also suggest that the Prc is involved in familiarity regardless of stimulus type.

However, whereas prior neuroimaging studies that have used exact repetitions of studied items have generally found decreased Prc activity (i.e., repetition priming) with increased familiarity (see Eichenbaum et al., 2007 for review), we found increased Prc activity with increased familiarity. The fact that our test stimuli were all novel (i.e., were not repeats of earlier items but instead resembled studied items) may have played a role in this. Also, we only found increased Prc activity for familiar cues when recall failed (not when it succeeded). In these respects, our results are consistent with case reports and stimulation studies indicating that increased activity in the Prc region, particularly the right side (where we found activity), may be responsible for hyperfamiliarity and inappropriate feelings of déjà vu (e.g., Bartolomei et al., 2004; Bowles et al., 2007; Devinsky et al., 2010). It may be that our cue resemblance paradigm taps a similar Prc mechanism of familiarity, and that this familiarity either only occurs when recall fails, or the Prc specifically signals a retrieval attempt that is brought on by familiarity when recall fails. Familiarity may signal the presence of a target in memory, which may in turn, prompt a further retrieval attempt when the first attempt did not succeed.

Our results are consistent with the suggestion that the Prc is involved in memory for the information present within an item itself (Staresina and Davachi, 2010), as the familiar features present in the test cues that corresponded to studied items were features of the studied items themselves (i.e., the graphemic characteristics of a studied word or the configural characteristics of a studied picture). Staresina and Davachi (2010) argue that the Prc may help to create “complex/conjunctive gestalts” that integrate item features to help strengthen individual item representations. Our results are compatible with this idea in that we are showing that a test cue that resembles the “gestalt” of a studied item, or has feature overlap with a studied item, leads to both increased familiarity with the cue and increased activation of the Prc when recall fails.

It is possible that the role of the Prc is to signal the summed feature-match of the test item to memory representations. Global matching models (e.g., Clark and Gronlund, 1996; Gillund and Shiffrin, 1984; Hintzman, 1988; Norman and O’Reilly, 2003) specify that the familiarity signal is the direct result of a feature-matching process whereby the features in the test item are matched, on a feature-by-feature basis, with the features that have been stored in memory for the study list. A greater degree of summed feature-overlap across items in memory leads to a more intense familiarity signal whereas less feature-overlap leads to a less intense familiarity signal. Our study items and their test cues differed but were designed to share particular features. Previously published studies support the global matching approach to familiarity brought on by these types of cues when recall fails. As the degree of feature-overlap between a novel test cue and the features stored in memory increases, so do familiarity ratings given when recall fails. This feature-matching pattern holds for the graphemic cues (Ryals and Cleary, 2012) and the configurally similar scene cues (Cleary et al., 2012)³ used in the present study, suggesting that feature-matching forms the basis of the increased familiarity for these cues when recall fails. It is possible, therefore, that Prc activity varies with the degree of feature-overlap between the test cue and the memory representations. Future research should examine how increasing or decreasing the degree of feature-overlap from study to test affects levels of Prc involvement as well as how this Prc activity theoretically coalesces with existing global matching theories.

3.3. The hippocampus in recollection

The fact that hippocampal activation increased in cases of successful cued recall (i.e., recollection) but not in cases of increased familiarity during recall failure is consistent with many previous neuroimaging studies on dual-process theory (e.g., Cohn et al., 2009; see Eichenbaum et al., 2007 for a review). Because these prior studies’ interpretation remains controversial due to disagreements regarding the methods of separating familiarity from recollection, our use of a novel paradigm is important.

One source of controversy has centered on the remember–know task (R–K), with some researchers suggesting that it does not successfully separate recollection and familiarity because recollection actually contaminates “know” reports (Wais et al., 2008; Johnson et al., 2009). Another source of controversy is that some argue that recollection is continuous, rather than thresholded (e.g., Mickes et al., 2009) and that apparent dissociations in literature involving confidence ratings, remember–know judgments, and source judgments simply reflect differences between strong and weak memories (Wais, 2008). Additionally, Wais (2008) and Wixted and Squire (2011) propose that the hippocampus may be more active for strong memories while leaving the Prc to show activity for weak memories. Though a similar argument might be made about the present results (i.e., recognition that occurs in the presence of recall represents strong memories while recognition that occurs in the absence of recall represents weak memories), two novel aspects of our results warrant consideration.

First, unlike prior studies, we have an objective index of when item recall succeeded vs. failed, which avoids some of the issues surrounding both the R–K paradigm and source memory tasks. Second, although our method is agnostic regarding whether recollection is thresholded or continuous, our differential pattern of activity between the hippocampus and the Prc is inconsistent with a strong versus weak memory explanation of recognition with and without recall, respectively. To explain the present findings in terms of a single, continuous memory process that varies only in strength, one would have to assume not only that only stronger memories evoke increased activation in the hippocampus, but that it is only the weaker memories (which also happen to be unaccompanied by objective item recall success), that evoke increased activation in the Prc, and not stronger memories (for which objective item recall succeeded). If recognition with and without cued recall both result from the same continuous memory process, where recognition with recall constitutes strong memory and recognition without recall constitutes weak memory, then Prc activity should increase not just in cases of recall failure; it should increase both when objective item recall succeeds and when it fails. To explain our pattern in terms of strong memories selectively increasing hippocampal activity and weak memories selectively increasing Prc activity, one would have to assume that there is something fundamentally different about strong and weak memories that require two different neural mechanisms for each type of memory. This assumption implies two different types of memory, rather than one continuous process, which converges with other evidence suggesting functional heterogeneity in the MTL in both human and rodent models with respect to recollection and familiarity (e.g., Bowles et al., 2010; Cohn et al., 2010; Martin et al., 2011; Sauvage et al., 2008; Winters et al., 2004; Yonelinas et al., 2007).

That said, other theories of MTL function that are agnostic to the controversy surrounding single vs. dual-process theories, such as theories proposing a perceptual feature overlap view of the MTL (Barense et al., 2005; Graham et al., 2010; O’Neil et al., 2009; Saksida and Bussey, 2010), may be compatible with our findings. For instance, the Emergent Memory Account (EMA) (Graham et al. 2010) proposes that, rather than solely being a storehouse for episodic memory information, the MTL may involve processing of “complex conjunctive object and spatial representations critical for both accurate perception, and as a result, accurate memory” (p. 832). An additional view that is agnostic to the controversy surrounding single vs. dual-process theories is that the hippocampus and Prc differ in either the type of memory representations themselves (e.g., full vs. partial feature matching to the cue) or in how they react to partial feature matches between the test cue and the memory representations. For example, if the partial feature-match between the test cue and a memory representation is sufficient to lead to pattern completion (e.g., Norman and O’Reilly, 2003), this pattern completion may manifest as full target recall and be subserved by the hippocampus. In contrast, if the match of features between the test cue and the memory representation is only partial or ambiguous, this may fail to lead to pattern completion but this partial feature overlap in the absence of pattern completion may be signaled by the Prc, and may function to spur a more comprehensive search in memory for a corresponding target. Consistent with this account are the findings that (1) increased hippocampal activity was observed for successful full target recall but not for cues familiar from resemblance during recall failure, and (2) increased Prc activity was observed only for instances in which cues were familiar from resemblance to studied items but full target recall failed.

Our finding of hippocampal recruitment (i.e., increased activation) during cued recall is also consistent with recent theories of the computational roles of different subregions of the hippocampal formation. According to these theories (e.g., Becker, 2005; Leutgeb et al., 2007; Norman and O’Reilly, 2003; Norman, 2010; Rolls, 2010) one of the computational mechanisms of the hippocampus is pattern completion. At test, recurrent connections are used to take the partial information available in the cortex and use it to reconstruct the full pattern of information from encoding. Thus, in this task, when recall succeeded, the cue stimuli were able to lead to retrieval of the related study stimulus, leading to increased hippocampal activation.

It is worth noting that recall is difficult to examine using fMRI because of the issue of how to collect recall responses during scanning. Our use of an MRI-compatible microphone and filtering system for collecting voice responses presents a viable means of collecting recall responses with minimal head movement. Our unique item-based cued recall task along with this type of voice-response system present a means of objectively examining the neural correlates of recognition when recall succeeds versus fails.

3.4. Novelty processing in the hippocampus

Another pattern of activity within the hippocampus that is worth noting is the decreased hippocampal activation for familiar nonword cues for which recall failed relative to unfamiliar (i.e., novel) nonword cues. This pattern is the opposite of that found for successful recall (where hippocampal activation was increased when recall succeeded compared to when it failed among familiar cues). One possible reason for this pattern is that the hippocampus is recruited for novelty-related processing for the unfamiliar nonword cues (i.e., those cues that did not resemble studied words). Recent research has demonstrated that the hippocampus is sensitive to novelty (e.g., Bowles et al., 2010; Kumaran and Maguire, 2009; Poppenk et al., 2008).

3.5. Material specific memory effects

Although we did not find category specific memory effects in the Prc and hippocampus, we did find such effects in visual category specific regions. For word memory, we found effects only in the VWFA; for scene memory we found effects only in the PPA. For words, we found opposite patterns of activity in the VWFA associated with successful word recall and word-form familiarity when recall failed, respectively. For scenes, we found opposite patterns of activity in the PPA for successful scene recall and scene familiarity when recall failed, respectively.

For recollection (i.e., successful cued recall), we found greater activity in the category specific cortical regions for cues that resembled study items and elicited successful recall than for cues that resembled studied items but did not elicit recall. As discussed in more detail above, correct recall also recruited the hippocampus. This raises the question of what the relationship is between both regions that leads to this parallel recruitment. One possibility is that information about initial differences in cue related activity in category specific regions projects to the hippocampus in a bottom-up fashion; this information then passes through the structures of the MTL and results in the retrieval of the target item. Increased activity in category specific regions thus would directly lead to increased recall accuracy. This possibility is consistent with Norman and O’Reilly’s (2003) model of the hippocampus, which specifies that the information present in an effective retrieval cue first reactivates a representation in the hippocampus, which then triggers cortical reinstatement through a pattern completion process. Another possibility is that top-down attentional processes increase activity to the cue in category specific regions to enable processing of details of the test stimulus that are important for memory recall; activity would therefore be higher for correct recall because of the association between these attentional resources and retrieval success. Current fMRI techniques do not have sufficient temporal resolution to discriminate between these possibilities; however, future research using functional connectivity measures may shed light on whether the visual category specific regions and MTL interact directly, or their interaction is mediated by other regions, such as the prefrontal cortex.

For familiarity, we found modulation of category specific cortical regions such that there was greater activity for novel cues than for familiar cues (those that resembled studied items) for which recall failed. This difference was significant for words in the left VWFA while there was a trend for this difference with scenes in the left PPA.

Though our results for recollection in category specific cortical areas are consistent with the idea that recollection is associated with cortical perceptual reinstatement during retrieval (e.g., Norman and O’Reilly, 2003), they are inconsistent with the idea that familiarity is associated with the same type of reinstatement. Although some research using the R–K paradigm has suggested that cortical reinstatement may occur in familiarity as well as in recollection (Johnson et al., 2009), our finding that familiarity and recollection led to opposite patterns of activity in category specific cortical areas suggests that familiarity and recollection in our study were driven by different mechanisms within perceptual cortex.

The direction of difference for familiarity (greater activity for novel than familiar stimuli) is consistent with a large literature in perceptual priming, which typically finds reduced levels of activity for repeated identical and similar stimuli (Buckner et al., 1995, 1998). Our results suggest that there may be two opposing neural effects at work across recognition test trials: An increase in activation in category specific cortical areas when recollection succeeds and a decrease in activation in these areas when recollection fails but the test stimuli are familiar. Our paradigm presents a means of separating these opposing neural effects by objectively identifying trials for which recall succeeded and trials for which recall failed. It is possible that in standard recognition tasks, these two effects cancel each other out.

4.Experimental procedures

4.1. Participants

Thirteen participants were recruited from Colorado State University (Fort Collins, CO).⁴ All participants were healthy, right-handed adults (7 males, 6 females) with an mean age of 21.7 years (range: 19–28 ). Participants were English speakers and were screened for a history of neurological and psychiatric disorders, use of psychoactive substances, and contra-indications to MRI (i.e., metallic implants). Each individual participated in one scanning session, completing the procedure described below. Behavioral and scanning data from two participants were excluded for the word portion of the study (blocks 1–3 ) due to a high number of scanning artifacts coupled with a low number of trials in the studied unrecalled category. One additional subject was excluded from the category-specific region analyses due to technical problems during the localizer scan.

4.2. Word stimuli

Word stimuli were 90 emotionally neutral words (Arousal M=5.08, SD=0.66; Valence M=5.16, SD=0.41) 4–11 letters in length (M=6.33, SD=.61) (Bradley and Lang, 1999). Following from Ryals and Cleary (2012), orthographically and phonologically similar nonword test cues were created by substituting 1–3 letters per word (M=1.70, SD=0.53) while preserving the first and last letters of each word. For example, the nonword “cuniaus” was the test cue for the word “curious”. Mean length of nonword test cues was equivalent to the mean length of real word counterparts (test cues: M=6.48 letters, SD=1.55; real word counterparts: M=6.33 letters, SD=1.55; [t(178)=.61, SE=.24 p=.54]). In an attempt to capitalize on orthographic and phonological similiarity, as well as to preserve pronounceability of nonword test cues, letters were either substituted, or in some cases added to nonword test cues. For example, if a studied word was “python”, an additional letter was added to the nonword test cue “pivthen” to preserve resemblance and pronounceability. One letter was added to 13 out of the total 90 stimuli (14%).

4.3. Scene stimuli

Scene stimuli were screenshots of 90 nameable 500 ×375 pixel color scenes that were created for the virtual reality study by Cleary et al. (2012) using the 2^™ software (Electronic Arts Inc., Redwood City, CA).⁵ Each study scene had a corresponding configurally similar scene (see Fig. 1 for examples). All scenes were built on a grid on which walls, floors, ceilings, architectural features, landscape terrains and elements (e.g., chairs, plants, light fixtures) could be placed. The grid allowed for a precise match between primary and configurally similar element positions. We achieved configural preservation from study to test by maintaining the angles of walls, the placement of objects on the grid, and the vantage point in each screenshot. As seen in Fig. 1, a participant might study the scene “train station.” At test, the unnamed scene “canal” contains a waterway at the same angle relative to other objects in the scene and as the train tracks displayed in the train station. As another example, a participant might study the scene “library.” At test, the unnamed scene “art gallery” contains walls displaying canvases in the precise locations of the bookshelves displayed in the earlier-viewed library.

Fig. 1.

Top row: Illustration of the RWCR paradigm from word blocks. Bottom row: Illustration of the scene RWCR paradigm from scene blocks.

4.4. Voice response acquisition

Confidence ratings and identification responses were gathered using a dual-channel FOMRI II^™ fiber-optic microphone (Optoacoustics Ltd., Israel) with a frequency response of 50–4000 Hz, noise reduction capabilities in the range of 15–40 dB, and a sensitivity of 50 mV/PA±10% at 1 kHz. The microphone body was mounted on the scanner head coil and the sensor rested on the end of a flexible gooseneck extension positioned approximately 2 cm from the mouth which enabled them to move only their lips while their head and jaw remained immobilized. Participants were pre-trained to move only their lips while speaking prior to being scanned so as to avoid excessive head movement. Verbal responses were captured online in stereo WAV format using Audacity v1.3 audio software (Sourceforge.net) on an Apple laptop. Verbal responses were isolated offline using the spectrum feature in Audacity combined with a low-pass filter and a noise reduction tool where necessary to increase the signal to noise ratio.

4.5. Procedure

4.5.1. Word RWCR

The word RWCR task was similar in structure to the procedure used by Cleary (2004), and it comprised the first three scans in the experiment, with one study block and one test block per scan. The procedure is illustrated at the top of Fig. 1. Each study block consisted of 15 encoding trials followed by 30 test trials. At encoding, a 9 s warning screen was followed by the sequential presentation of the 15 study words, each presented in lowercase white 30 pt Courier font against a black background for 6 s apiece. The encoding phase utilized a rapid event-related design with variable inter-trial jitter. Each word was separated by a jittered inter-trial interval created from a randomly sampled geometric distribution using Matlab (The MathWorks Inc, Natick, MA). Jitter values were 1500 ms, 3000 ms, 4500 ms, and 6000 s, (i.e., 50% of the trials 1500 ms inter-trial interval, 25% of the trials had 3000 ms inter-trial interval, 12.5% of the trials had 4500 ms inter-trial interval, and 12.5% of the trials had 6000 ms inter-trial interval).

After the encoding phase, a 3 s warning screen preceded the beginning of the cued-recall test phase. For the test phase, 30 nonword cues were presented on the screen for 9 s each. Because trial onsets were separated by 9 s, at which point slow event related designs without jitter do not have substantially lower power than rapid event related designs with jitter (Dale, 1999), we chose utlized a slow event related design without inter-trial jitter. Half (15) of the randomly-ordered nonword cues graphemically resembled studied words, (e.g., cuniaus for the studied word curious), and half (15) did not resemble studied words. Cues were presented in the middle of the screen in lowercase white 30 pt Courier font, and a 0–10 Likert scale appeared below each word. Participants first provided a numerical rating using the 0–10 Likert scale based on the likelihood that the nonword test cue resembled a studied word. For example, a rating of 0 indicated they were completely sure the cue did NOT resemble a study word and a rating of 10 indicated they were absolutely sure the cue DID resemble a studied item. Participants were encouraged to use the entire scale of ratings and not just the endpoints.

Participants had been instructed that after verbally giving a rating, they should try to identify a word from the immediately-preceding study list that resembled the cue by speaking the word aloud, and if they could not, they were to verbally respond “don’t know”. For example, if the nonword cue was cuniaus, the participant might have responded, “9…curious” or “2…don’t know.” Each test trial was followed immediately by the beginning of the next 9 s test trial. Study word and test cue presentation order was randomized within each block. Prior to beginning the study, participants had numerous practice trials in which they learned to provide a rating and then a recall response within the 9 s response window.

4.5.2. Scene RWCR

The scene RWCR task was similar to the method used in Cleary et al. (2009), and it comprised the last two scans of the experiment, with one study block and one test block per scan. The overall design and procedure was the same as that used in the word RWCR task, and is illustrated at the bottom of Fig. 1. As with the word RWCR task, after a 9 s warning, each study scene was presented sequentially on the screen for 6 s each. Interstimulus intervals were once again jittered using the same method described above. A 3 s warning screen preceded the beginning of the cued-recall test phase. For the test phase, 30 new scenes were presented on the screen for 9 s each. Half (15) of the randomly-ordered test scenes were configurally similar to studied scenes, and the other half (15) were completely novel. A 0–10 Likert scale appeared below each test scene and participants indicated their confidence that the test scene resembled a studied scene.

As with the word RWCR task described above, participants were instructed to try to verbally identify a configurally similar scene from the immediately-preceding study list after giving a rating by speaking the name of that scene aloud; if they could not, they either verbally responded “don’t know” or said nothing. For example, for the test scene “canal,” the participant may have responded, “9…train station” or “2…don’t know.” Each test trial was followed immediately by the beginning of the next 9 s test trial. Once again, study scene and test scene presentation order was randomized within each block, and participants practiced the task prior to beginning the scanning session.

4.5.3. Localizer tasks

For our three localizer tasks, participants viewed stimulus trains of words, faces, and scenes. A total of 135 items (45 in each category) were presented across a total of 15 alternating blocks, five blocks of each category. Words were similar to the words presented in word blocks 1–3 of the study, scenes were similar to those used in scene blocks 4–5 of the study, and face stimuli were taken from a previous localizer task used in Seger et al. (2010). Participants performed a simple 1-back working memory task whereby they responded “yes” if the same item was repeated twice in a row. Within each block, one out of eight items was chosen pseudorandomly for repetition; therefore, each block contained nine items (one duplicate). Each stimulus was presented for 1900 ms each with a 100 ms interstimulus interval. No feedback was given during the task. After the scanning session was completed, verbal “yes” responses were verified offline to ensure all participants performed the task as instructed.

4.6. MRI image acquisition

Images were obtained on a research-dedicated 3.0 T whole-body MRI scanner (GE Healthcare, Milwaukee, WI) at the Brain Imaging Center at the University of Colorado Denver (Aurora, CO). The scanner was equipped with an 8-channel, high-resolution phased array head coil using GE’s Array Spatial Sensitivity Encoding Technique (ASSET) software. Anatomical images were collected using a T1-weighted SPGR sequence (minimum TR; TE, 3.95 ms; TI, 950 ms; FA, 10°; FOV, 220-mm; 256 ×256 coronal matrix; 166 1.2-mm slices). Functional images were reconstructed from 26 axial oblique slices obtained using a T2-weighted EPI-Gradient-Recalled Echo sequence (TR, 1500 ms; TE, 30 ms; FA, 64°; FOV, 220-mm; 64 ×64 matrix; 4.0-mm slices; no inter-slice gap), in order to measure BOLD signal change. Additionally, the first five volumes, recorded before longitudinal magnetization reached a steady state, were discarded. Visual stimuli were presented using a magnet-compatible projector that projects visual images onto a mirror attached to the RF head coil. A computer running E-Prime 2.0 experiment software (Psychology Software Tools Inc., Pittsburg, PA) was used to control stimulus presentation. Earplugs and headphones were provided to protect the participants’ hearing. Head movement was minimized using small foam pads placed on each side of the head inside the RF head coil.

4.6.1. Image preprocessing

Image analysis was performed using Brain Voyager QX 1.0 (Brain Innovation, Maastricht, The Netherlands). Functional data was first subjected to preprocessing, consisting of (1) three dimensional motion correction using trilinear interpolation, (2) slice scan time correction using cubic spline interpolation, (3) temporal data filtering with a high-pass filter of 3 cycles in the time course and (4) linear trend removal. For eleven of the participants, motion parameters across all five RWCR scans and the localizer task did not exceed 2.50 mm in X, Y, or Z axis and did not exceed 2°of rotation in any of the three directions (yaw, pitch, roll). The remaining two participants only modestly exceeded this amount, with a maximum net motion in one direction at the end of the localizer scan of 4.19 mm for one subject and 3.93 mm for the second subject. Each subject’s high-resolution anatomical image was normalized to the Tailarach and Talairachand Tornoux, 1988 brain template. The normalization process consisted of two steps: an initial rigid body translation into the AC-PC plane, followed by an elastic deformation into the standard space performed on 12 individual sub-volumes. The resulting set of transformations was applied to the subject’s functional image volumes to form volume time course representations to be used in subsequent statistical analyses. Finally, the volume time course representations were spatially smoothed using a Gaussian kernel, full-width at half maximum (FWHM) of 6.0 mm.

4.6.2. Independently-defined ROI analyses

Our primary analysis approach focused on regions of interest (ROIs). This is appropriate for studies such as this that are based on strong a priori predictions and avoids the problems with multiple comparisons and Type 1 errors that whole brain analyses are subject to (Poldrack, 2007). Furthermore, the use of a priori defined ROIs avoids problems with circular analysis that could result from using functionally based ROIs from a whole brain analyses (Kriegeskorte et al., 2009, 2010). As stated in 2.3.9 above, results of an exploratory whole-brain analysis conducted at a lenient statistical threshold can be found in the Supplementary Materials for both words and scenes in both our recollection and familiarity contrasts.

Statistical tests within ROIs were performed using the Brain Voyager ROI General Linear Model tool, which implements an ordinary least-squares regression. All contrasts were calculated as within-subject random effects (RFX) analyses controlling for between subject variability. These analyses were across all the voxels within the defined ROIs; time courses were z-normalized separately for each run. The RFX GLM procedure in Brain Voyager uses a two level process in which beta values are calculated at the first level, then serve as the inputs to the second level analysis. Because these beta value estimates are unbiased even in the case of serial correlations, this approach avoids the need to correct for serial correlations. Subject-specific ROIs were used for the visual category specific analyses as described in 4.6.4 below. Each trial was modeled as a 3 TR epoch encompassing the first 4.5 s of each trial, which was sufficient to encompass stimulus perception, familiarity rating, and recall response. The baseline condition was not explicitly modeled in order to avoid potential colinearity among the regressors. Contrasts were subjected to an alpha value of p<.05.

4.6.3. Hippocampus and perirhinal cortex ROI definitions

ROIs were defined independently for hippocampus and Prc based on a priori predictions. ROIs were drawn manually on an averaged anatomical image formed from the normalized high-resolution anatomical scans of individual subjects and were defined separately for the Words and Scenes conditions. For the hippocampus, previous research suggested that recollection-related activity is preferentially located in middle (body) or posterior (tail) regions (Daselaar et al., 2006; Montaldi et al., 2006; Park and Rugg, 2010; Ranganath et al., 2004; Seger et al., 2011; Slotnick, 2010). Right and left hemisphere ROIs were manually drawn; the resulting ROIs were 3 ×3 × 3 voxel cubes centered at the following Tailarach coordinates for Words (right: 35, −26, −6; left: −35, −26, −6) and Scenes (right: x=31, y=20, z=−16; left x=−32, y=−20, z=−15). Fig. 3 presents these ROIs displayed on a normalized anatomical scan from one individual. These ROIs are located in the body of the hippocampus, near to regions associated with recollection in several published studies including those using the R–K procedure (Seger et al., 2011; Slotnick, 2010; Park and Rugg, 2010) and those using confidence judgments (Daselaar et al., 2006).

Fig. 3.

(a) Coronal view of hippocampal ROIs for words and scenes. (b) Coronal and sagittal views of Prc ROIs for words and scenes.

The MTL literature is notably mixed with regard to the location of the perihinal cortex, and previous research has found a large degree of between subject variability in both the anatomical location of this structure (Blaizot et al., 2010; Insausti et al., 1998), and functional activation within this structure (Ford et al., 2010; Staresina and Davachi, 2008, 2010). We performed a comprehensive review of studies reporting familiarity related perirhinal cortex activity in group analyses and found a wide range of reported locations, ranging from as far anterior as y=10 to as far posterior as y=−28 in Tailarach coordinates.. As suggested in previous research in humans (e.g., Insausti et al., 1998; Pruessner et al., 2002) the Prc runs the length of the MTL axis, although its width is notably different from anterior to posterior. The entorhinal cortex (Erc) falls medial to the the Prc along much of the axis, with a portion of Prc located lateral to the Erc and anterior to the parahippocampal cortex (Phc). As noted in Pruessner et al. (2002), the precise division between Prc and Erc in particular depends crucially on the length of the collateral sulcus. This may be one reason for the large degree of intrasubject variability in this region. In creating our independent ROIs, we aimed to select a central and unambiguously Prc region within the dorsal-ventral and lateral-medial extent of this structure. Along the anterior-posterior MTL axis, we chose to locate our ROIs in a relatively posterior, yet still unambiguously perirhinal, region of the Prc for several regions. First, this region clearly showed familiarity related activity in previous studies (e.g., Ford et al., 2010; Taylor et al., 2006; Yassa and Stark, 2008). Furthermore, one condition of our study involved memory for scenes, and posterior regions of the Prc have been shown to more strongly represent scene information than more anterior regions (Liang et al., 2012). Right and left hemisphere ROIs were manually drawn on the parahippocampal gyrus bordering on the collateral sulcus, consistent with the neuoranatomical description of Prc given in Insausti et al. (1998). As shown in Fig. 3, the resulting Prc ROIs were 3×3×3 voxel cubes centered at the following Tailarach coordinates (right words: x=19, y=−25, z=−14; left words x=−20, y=−25, z=−16; right scenes: x=18, y=−25, z=−17; left scenes: x=−20, y=−25, z=−17). Given the cytoarchitectonic asymmetry between hemispheres in this MTL region (Insausti et al., 1998) the ROIs for the left hemisphere are in a slightly different location than those for the right; however they reflect the same general location. The coordinates of the resulting ROIs were confirmed as likely falling within Brodmann areas 35 and 36 (cytoarchitectonic regions associated with perirhinal cortex) using the Talairach daemon (Lancaster et al., 2000). Furthermore, these ROIs were within the boundary coordinates for Prc outlined in the Brede database (Nielsen, 2009).

4.6.4. Functional localizer ROI definitions

Visual category-specific ROIs were functionally defined for each individual subject based on the activity in the localizer task. For word stimuli, we identified the visual word form area (VWFA) on the basis of the Word>Face+Scenes contrast from the localizer scan. For faces, we identified the fusiform face area (FFA) on the basis of the Face>Words+Scenes contrast from the localizer scan. For scenes, we identified the parahippocampal place areas (PPA) on the basis of the Scenes>Word+Faces contrast from the localizer task. All ROIs were defined by taking the peak activated voxel and selecting the surrounding cortex as cubes between 4 ×4 × 4 and 5 ×5 × 5 voxels.

4.6.5. Limitations and future directions

The present study relies on the use of independently-defined ROIs at the group level based on Talairach space. A promising future direction may involve using high resolution imaging to segment the MTL in native space in order to provide fine-grained support for our differential patterns in hippocampal, perirhinal, and category-specific regions. Signals in the MTL are notoriously difficult to localize and separate. In addition to high-resolution imaging, future research should incorporate EPI distortion correction (e.g., Chung et al., 2011). Additional concerns to address in future studies include improved spatial normalization to rule out poor MTL registration as well as the use of relatively large voxel sizes coupled with spatial smoothing which may raise concerns about blurring of activity from one MTL subregion to another. Future experiments using the RWCR paradigm might also incorporate reaction time as a measurement to determine the time-course of processing associated with recalled, unrecalled familiar and unfamiliar items. Finally, some may also be concerned with the number of participants in the present study coupled with a relatively low number of trials in some analyses. Future neuroimaging research using the RWCR paradigm should attempt to increase both sample size and trial numbers in replication of the current findings.

5. Conclusion

Our results provide support for non material-specific processes underlying familiarity and recall in the Prc and hippocampus, respectively, in conjunction with visual category-specific processes which are differently recruited for recall and familiarity in perceptual category-specific cortical areas. Our results also suggest that the different neural signatures of recall and familiarity can be found with both word and scene stimuli, which is important in light of recent findings that not all instantiations of dual-process theory can accommodate both words and scenes (Onyper et al., 2010). These data are consistent with an emerging view of interactions between medial temporal lobe subregions and cerebral cortex in which patterns of activity in perceptual cortex project to the dentate-hippocampal loop, which then uses this information at recall to perform pattern completion processes to reconstruct the full memory that was present at encoding. Our results further elucidate the types of neural processing that can still take place even when retrieval fails, and that may underlie associated feelings of familiarity. Broadly, familiarity may result from the integration of neural signals at many levels of analysis, including both fluency within category-specific perceptual processing regions, and non material-specific processing of items in the Prc.

Appendix A. Supporting information

Supplementary data associated with this article can be found in the online version at http://dx.doi.org/10.1016/j.brainres.2012.10.068.